1st Place Solutions for RxR-Habitat Vision-and-Language Navigation Competition (CVPR 2022)
Dong An, Zun Wang, Yangguang Li, Yi Wang, Yicong Hong, Yan Huang, Liang Wang, Jing Shao
This report presents the methods of the winning entry of the RxR-Habitat
Competition in CVPR 2022. The competition addresses the problem of
Vision-and-Language Navigation in Continuous Environments (VLN-CE), which
requires an agent to follow step-by-step natural language instructions to reach
a target. We present a modular plan-and-control approach for the task. Our
model consists of three modules: the candidate waypoints predictor (CWP), the
history enhanced planner and the tryout controller. In each decision loop, CWP
first predicts a set of candidate waypoints based on depth observations from
multiple views. It can reduce the complexity of the action space and facilitate
planning. Then, a history-enhanced planner is adopted to select one of the
candidate waypoints as the subgoal. The planner additionally encodes historical
memory to track the navigation progress, which is especially effective for
long-horizon navigation. Finally, we propose a non-parametric heuristic
controller named tryout to execute low-level actions to reach the planned
subgoal. It is based on the trial-and-error mechanism which can help the agent
to avoid obstacles and escape from getting stuck. All three modules work
hierarchically until the agent stops. We further take several recent advances
of Vision-and-Language Navigation (VLN) to improve the performance such as
pretraining based on large-scale synthetic in-domain dataset, environment-level
data augmentation and snapshot model ensemble. Our model won the RxR-Habitat
Competition 2022, with 48% and 90% relative improvements over existing methods
on NDTW and SR metrics respectively.