Vision-Language Navigation with Random Environmental Mixup
Open accessAbstract
Vision-language Navigation (VLN) tasks require an agent to navigate step-by-step while perceiving the visual observations and comprehending a natural language instruction. Large data bias, which is caused by the disparity ratio between the small data scale and large navigation space, makes the VLN task challenging. Previous works have proposed various data augmentation methods to reduce data bias. However, these works do not explicitly reduce the data bias across different house scenes. Therefore, the agent would overfit to the seen scenes and achieve poor navigation performance in the unseen scenes. To tackle this problem, we propose the Random Environmental Mixup (REM) method, which generates cross-connected house scenes as augmented data via mixuping environment. Specifically, we first select key viewpoints according to the room connection graph for each scene. Then, we cross-connect the key views of different scenes to construct augmented scenes. Finally, we generate augmented instruction-path pairs in the cross-connected scenes. The experimental results on benchmark datasets demonstrate that our augmentation data via REM help the agent reduce its performance gap between the seen and unseen environment and improve the overall performance, making our model the best existing approach on the standard VLN benchmark.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
VIMA, built on Ravens, is a multimodal manipulation benchmark for evaluating robot performance across vision, language, and physics constraints.
“Meanwhile, VIMA [291], built on Ravens, is a multimodal manipulation benchmark designed for evaluating robot performance across multi-modal inputs (vision, language, and physics constraints).”
From Multimodal fusion and vision–language models: A survey for robot vision (2026) · cites this paper as [291]
Unlike VLMbench and GemBench, VIMA integrates multi-modal task execution and precise instruction following.
“Unlike VLMbench [290] and Gem-Bench [289], which primarily evaluate vision-language models (VLMs) in robotic control, VIMA [291] further integrates multi-modal task execution and precise instruction following, making it distinct in its benchmarking approach.”
From Multimodal fusion and vision–language models: A survey for robot vision (2026) · cites this paper as [291]
VIMA includes 17 tasks that test a robot's ability to understand complex task instructions and execute them.
“VIMA [291] includes 17 tasks that test a robot's ability to understand complex task instructions and execute them accordingly.”
From Multimodal fusion and vision–language models: A survey for robot vision (2026) · cites this paper as [291]
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 56,035 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
Wu, Y., & Vapnik, V. N. (1999) · Technometrics · doi:10.2307/1271368 · Cited by 27,236 · see citing works
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533 · see citing works
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625 · see citing works
Character-level Convolutional Networks for Text ClassificationPreprint
Zhang, X., Zhao, J., & LeCun, Y. (2015) · Neural Information Processing Systems · doi:10.48550/arxiv.1509.01626 · Cited by 7,552 · see citing works
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132 · see citing works
mixup: Beyond Empirical Risk Minimization
Zhang, H., Cisse, M., Dauphin, Y. N., & Lopez-Paz, D. (2017) · arXiv · doi:10.48550/arxiv.1710.09412 · Cited by 5,831 · see citing works
Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering
Anderson, P., He, X., Buehler, C., Teney, D., Johnson, M., Gould, S. J., & Zhang, L. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00636 · Cited by 5,117 · see citing works
A faster algorithm for betweenness centrality*
Brandes, U. (2001) · Journal of Mathematical Sociology · doi:10.1080/0022250x.2001.9990249 · Cited by 4,977 · see citing works
VQA: Visual Question Answering
Antol, S., Agrawal, A., Lu, J., Mitchell, M., Batra, D., Zitnick, C. L., & Parikh, D. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.279 · Cited by 4,487 · see citing works
Multi-column deep neural networks for image classification
Ciresan, D. C., Meier, U., & Schmidhuber, J. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6248110 · Cited by 3,760 · see citing works
End to End Learning for Self-Driving Cars
Bojarski, M., Del Testa, D., Dworakowski, D., Firner, B., Flepp, B., Goyal, P., Jackel, L. D., Monfort, M., Muller, U., Zhang, J., et al. (2016) · arXiv · doi:10.48550/arxiv.1604.07316 · Cited by 3,198 · see citing works
Domain randomization for transferring deep neural networks from simulation to the real world
Tobin, J., Fong, R., Ray, A., Schneider, J., Zaremba, W., & Abbeel, P. (2017) · 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8202133 · Cited by 3,124 · cited 1× · see citing works
“Data augmentation has also been investigated in RL, including domain randomization [52, 42, 44], cutout [13] and random convolution [32].”
MethodBest practices for convolutional neural networks applied to visual document analysis
Simard, P., Steinkraus, D., & Platt, J. C. (2005) · Proceedings of the International Conference on Document Analysis and Recognition · doi:10.1109/icdar.2003.1227801 · Cited by 2,805 · see citing works
LXMERT: Learning Cross-Modality Encoder Representations from Transformers
Tan, H., & Bansal, M. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1514 · Cited by 2,319 · see citing works
EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks
Wei, J., & Zou, K. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1670 · Cited by 2,242 · see citing works
ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks
Lu, J., Batra, D., Parikh, D., & Lee, S. (2019) · arXiv · doi:10.48550/arxiv.1908.02265 · Cited by 2,109 · see citing works
(2013) · Cited by 2,056 · see citing works
Generative adversarial imitation learning
Ho, J., & Ermon, S. (2016) · arXiv · doi:10.48550/arxiv.1606.03476 · Cited by 1,942 · see citing works
Matterport3D: Learning from RGB-D Data in Indoor Environments
Chang, A., Dai, A., Funkhouser, T., Halber, M., Niebner, M., Savva, M., Song, S., Zeng, A., & Zhang, Y. (2017) · 2017 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2017.00081 · Cited by 1,858 · see citing works
Hierarchical Question-Image Co-Attention for Visual Question AnsweringPreprint
Lu, J., Yang, J., Batra, D., & Parikh, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.00061 · Cited by 1,303 · see citing works
Habitat: A Platform for Embodied AI Research
Savva, M., Kadian, A., Maksymets, O., Zhao, Y., Wijmans, E., Jain, B., Straub, J., Liu, J., Koltun, V., Malik, J., et al. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00943 · Cited by 1,237 · see citing works
Anderson, P., Wu, Q., Teney, D., Bruce, J., Johnson, M., Sunderhauf, N., Reid, I., Gould, S., & van den Hengel, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00387 · Cited by 1,187 · see citing works
Reinforcement Learning with Unsupervised Auxiliary TasksPreprint
Jaderberg, M., Mnih, V., Czarnecki, W. M., Schaul, T., Leibo, J. Z., Silver, D., & Kavukcuoglu, K. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.05397 · Cited by 826 · see citing works
Gibson Env: Real-World Perception for Embodied Agents
Xia, F., Zamir, A. R., He, Z., Sax, A., Malik, J., & Savarese, S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00945 · Cited by 763 · see citing works
Sandfort, V., Yan, K., Pickhardt, P. J., & Summers, R. M. (2019) · Scientific Reports · doi:10.1038/s41598-019-52737-x · Cited by 684 · see citing works
On evaluation of embodied navigation agents
(2018) · doi:10.48550/arxiv.1807.06757 · Cited by 622 · see citing works
CAD2RL: Real Single-Image Flight Without a Single Real Image
Sadeghi, F., & Levine, S. (2017) · Robotics: Science and Systems XIII · doi:10.15607/rss.2017.xiii.034 · Cited by 618 · see citing works
Contextual Augmentation: Data Augmentation by Words with Paradigmatic Relations
Kobayashi, S. (2018) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n18-2072 · Cited by 617 · see citing works
AI2-THOR: An Interactive 3D Environment for Visual AI
Kolve, E., Mottaghi, R., Han, W., VanderBilt, E., Weihs, L., Herrasti, A., Deitke, M., Ehsani, K., Gordon, D., Zhu, Y., et al. (2017) · arXiv · doi:10.48550/arxiv.1712.05474 · Cited by 578 · see citing works
Wang, X., Huang, Q., Celikyilmaz, A., Gao, J., Shen, D., Wang, Y.-F., Wang, W. Y., & Zhang, L. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00679 · Cited by 522 · see citing works
Cognitive Mapping and Planning for Visual Navigation
Gupta, S., Davidson, J., Levine, S., Sukthankar, R., & Malik, J. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.769 · Cited by 521 · see citing works
Asymmetric Actor Critic for Image-Based Robot Learning
Pinto, L., Andrychowicz, M., Welinder, P., Zaremba, W., & Abbeel, P. (2018) · Robotics: Science and Systems · doi:10.15607/rss.2018.xiv.008 · Cited by 369 · see citing works
Hierarchical Question-Image Co-Attention for Visual Question Answering
Lu, J. (2024) · TIB Data Manager · doi:10.57702/uhdmfu1r · Cited by 357 · see citing works
TOUCHDOWN: Natural Language Navigation and Spatial Reasoning in Visual Street Environments
Chen, H., Suhr, A., Misra, D. K., Snavely, N., & Artzi, Y. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01282 · Cited by 348 · see citing works
Quantifying Generalization in Reinforcement LearningPreprint
Cobbe, K., Klimov, O., Hesse, C., Kim, T., & Schulman, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.02341 · Cited by 348 · see citing works
GAN Augmentation: Augmenting Training Data using Generative Adversarial NetworksPreprint
Bowles, C. T., Chen, L., Guerrero, R., Bentley, P., Gunn, R. N., Hammers, A., Dickie, D. A., Hernández, M. D. C. V., Wardlaw, J. M., & Rueckert, D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.10863 · Cited by 347 · see citing works
Learning to Navigate Unseen Environments: Back Translation with Environmental Dropout
Tan, H., Yu, L., & Bansal, M. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1268 · Cited by 293 · see citing works
Speaker-Follower Models for Vision-and-Language Navigation
Fried, D., Hu, R., Cirik, V., Rohrbach, A., Andreas, J., Morency, L.-P., Berg-Kirkpatrick, T., Saenko, K., Klein, D., & Darrell, T. (2018) · arXiv · doi:10.48550/arxiv.1806.02724 · Cited by 291 · see citing works
Cognitive Mapping and Planning for Visual Navigation
Gupta, S., Tolani, V., Davidson, J., Levine, S., Sukthankar, R., & Malik, J. (2019) · International Journal of Computer Vision · doi:10.1007/s11263-019-01236-7 · Cited by 275 · see citing works
Building Generalizable Agents with a Realistic and Rich 3D Environment
Wu, Y., Wu, Y., Gkioxari, G., & Tian, Y. (2018) · CaltechAUTHORS (California Institute of Technology) · doi:10.48550/arxiv.1801.02209 · Cited by 272 · see citing works
Towards Learning a Generic Agent for Vision-and-Language Navigation via Pre-Training
Hao, W., Li, C., Li, X., Carin, L., & Gao, J. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01315 · Cited by 264 · see citing works
Chapelle, O., Weston, J., Bottou, L., & Vapnik, V. N. (2000) · Cited by 244 · see citing works
Vision-language navigation with self-supervised auxiliary reasoning tasks
Zhu, F., Zhu, Y., Chang, X., & Liang, X. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01003 · Cited by 223 · see citing works
Self-citationImproving Vision-and-Language Navigation with Image-Text Pairs from the Web
Majumdar, A., Shrivastava, A., Lee, S., Anderson, P., Parikh, D., & Batra, D. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58539-6_16 · Cited by 201 · see citing works
Self-Monitoring Navigation Agent via Auxiliary Progress Estimation
Ma, C.-Y., Lu, J., Wu, Z., AlRegib, G., Kira, Z., Socher, R., & Xiong, C. (2019) · arXiv · doi:10.48550/arxiv.1901.03035 · Cited by 200 · see citing works
A dataset for developing and benchmarking active vision
Ammirato, P., Poirson, P., Park, E., Kosecka, J., & Berg, A. C. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989164 · Cited by 187 · see citing works
Wang, X. E., Xiong, W., Wang, H., & Wang, W. Y. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01270-0_3 · Cited by 187 · see citing works
The Regretful Agent: Heuristic-Aided Navigation Through Progress Estimation
Ma, C., Wu, Z., AlRegib, G., Xiong, C., & Kira, Z. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00689 · Cited by 166 · see citing works
Stay on the Path: Instruction Fidelity in Vision-and-Language Navigation
Jain, V., Magalhaes, G., Ku, A., Vaswani, A., Ie, E., & Baldridge, J. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1181 · Cited by 165 · see citing works
Tactical Rewind: Self-Correction via Backtracking in Vision-And-Language Navigation
Ke, L., Li, X., Bisk, Y., Holtzman, A., Gan, Z., Liu, J., Gao, J., Choi, Y., & Srinivasa, S. S. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00690 · Cited by 148 · see citing works
APAC: Augmented PAttern Classification with Neural NetworksPreprint
Sato, I., Nishimura, H., & Yokoi, K. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.03229 · Cited by 108 · see citing works
Robust Navigation with Language Pretraining and Stochastic Sampling
Li, X., Li, C., Xia, Q., Bisk, Y., Çelikyılmaz, A., Gao, J., Smith, N. A., & Choi, Y. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1159 · Cited by 97 · see citing works
Network Randomization: A Simple Technique for Generalization in Deep Reinforcement LearningPreprint
Lee, K., Lee, K., Shin, J., & Lee, H. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.05396 · Cited by 97 · see citing works
Language and Visual Entity Relationship Graph for Agent NavigationPreprint
Hong, Y., Rodríguez-Opazo, C., Qi, Y., Wu, Q., & Gould, S. J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.09304 · Cited by 97 · see citing works
Transferable Representation Learning in Vision-and-Language Navigation
Huang, H., Jain, V., Mehta, H., Ku, A., Magalhaes, G., Baldridge, J., & Ie, E. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00750 · Cited by 82 · see citing works
BabyWalk: Going Farther in Vision-and-Language Navigation by Taking Baby Steps
Zhu, W., Hu, H., Chen, J., Deng, Z., Jain, V., Ie, E., & Sha, F. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.229 · Cited by 60 · see citing works
Sub-Instruction Aware Vision-and-Language Navigation
Hong, Y., Rodríguez, C., Wu, Q., & Gould, S. J. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.271 · Cited by 60 · see citing works
General Evaluation for Instruction Conditioned Navigation using Dynamic Time WarpingPreprint
Ilharco, G., Jain, V., Ku, A., Ie, E., & Baldridge, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.05446 · Cited by 56 · see citing works
Evolving Graphical Planner: Contextual Global Planning for Vision-and-Language NavigationPreprint
Deng, Z., Narasimhan, K., & Russakovsky, O. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.05655 · Cited by 55 · see citing works
A Recurrent Vision-and-Language BERT for NavigationPreprint
Hong, Y., Wu, Q., Qi, Y., Rodríguez-Opazo, C., & Gould, S. J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.13922 · Cited by 38 · see citing works
Perceive, Transform, and Act: Multi-Modal Attention Networks for Vision-and-Language NavigationPreprint
Landi, F., Baraldi, L., Cornia, M., Corsini, M., & Cucchiara, R. (2019) · arXiv (Cornell University) · Cited by 27 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805
Flagged first, then by citations
6 of the 71 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directions
Gu, J., Stefani, E., Wu, Q., Thomason, J., & Wang, X. (2022) · Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2022.acl-long.524 · Cited by 216 · see citing works
ETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environments
An, D., Wang, H., Wang, W., Wang, Z., Huang, Y., He, K., & Wang, L. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3386695 · Cited by 103 · see citing works
Envedit: Environment Editing for Vision-and-Language Navigation
Li, J., Tan, H., & Bansal, M. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01497 · Cited by 87 · see citing works
Discuss Before Moving: Visual Language Navigation via Multi-expert Discussions
Long, Y., Li, X., Cai, W., & Dong, H. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10611565 · Cited by 73 · see citing works
Bird’s-Eye-View Scene Graph for Vision-Language Navigation
Liu, R., Wang, X., Wang, W., & Yang, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01007 · Cited by 69 · see citing works
Vision-language navigation: a survey and taxonomy
Wu, W., Chang, T., Li, X., Yin, Q., & Hu, Y. (2023) · Neural Computing and Applications · doi:10.1007/s00521-023-09217-1 · Cited by 65 · see citing works
Scaling Data Generation in Vision-and-Language Navigation
Wang, Z., Li, J., Hong, Y., Wang, Y., Wu, Q., Bansal, M., Gould, S., Tan, H., & Qiao, Y. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01103 · Cited by 64 · see citing works
NavCoT: Boosting LLM-Based Vision-and-Language Navigation via Learning Disentangled Reasoning
Lin, B., Nie, Y., Wei, Z., Chen, J., Ma, S., Han, J., Xu, H., Chang, X., & Liang, X. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3554559 · Cited by 64 · see citing works
Self-citationTarget-Driven Structured Transformer Planner for Vision-Language Navigation
Zhao, Y., Chen, J., Gao, C., Wang, W., Yang, L., Ren, H., Xia, H., & Liu, S. (2022) · Proceedings of the 30th ACM International Conference on Multimedia · doi:10.1145/3503161.3548281 · Cited by 54 · see citing works
HOP+: History-Enhanced and Order-Aware Pre-Training for Vision-and-Language Navigation
Qiao, Y., Qi, Y., Hong, Y., Yu, Z., Wang, P., & Wu, Q. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3234243 · Cited by 52 · see citing works
ADAPT: Vision-Language Navigation with Modality-Aligned Action Prompts
Lin, B., Zhu, Y., Chen, Z., Liang, X., Liu, J., & Liang, X. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01496 · Cited by 52 · see citing works
Self-citationVision-and-Language Navigation via Causal Learning
Wang, L., He, Z., Dang, R., Shen, M., Liu, C., & Chen, Q. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01248 · Cited by 51 · see citing works
Self-citationReinforced Structured State-Evolution for Vision-Language Navigation
Chen, J., Gao, C., Meng, E., Zhang, Q., & Liu, S. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01501 · Cited by 45 · see citing works
Adaptive Zone-aware Hierarchical Planner for Vision-Language Navigation
Gao, C., Peng, X., Yan, M., Wang, H., Yang, L., Ren, H., Li, H., & Liu, S. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01432 · Cited by 43 · see citing works
Volumetric Environment Representation for Vision-Language Navigation
Liu, R., Wang, W., & Yang, Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01544 · Cited by 43 · see citing works
Multimodal fusion and vision–language models: A survey for robot vision
Han, X., Chen, S., Fu, Z., Feng, Z., Fan, L., An, D., Wang, C., Guo, L., Meng, W., Zhang, X., et al. (2026) · Information Fusion · doi:10.1016/j.inffus.2025.103652 · Cited by 41 · see citing works
Learning Navigational Visual Representations with Semantic Map Supervision
Hong, Y., Zhou, Y., Zhang, R., Dernoncourt, F., Bui, T., Gould, S., & Tan, H. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00284 · Cited by 42 · see citing works
Kamath, A., Anderson, P., Wang, S., Koh, J. Y., Ku, A., Waters, A., Yang, Y., Baldridge, J., & Parekh, Z. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01041 · Cited by 39 · see citing works
Wang, L., He, Z., Dang, R., Chen, H., Liu, C., & Chen, Q. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3233554 · Cited by 36 · see citing works
Self-citationLearning from Unlabeled 3D Environments for Vision-and-Language Navigation
Chen, S., Guhur, P.-L., Tapaswi, M., Schmid, C., & Laptev, I. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-19842-7_37 · Cited by 35 · see citing works
Multimodal Transformer with Variable-Length Memory for Vision-and-Language Navigation
Lin, C., Jiang, Y., Cai, J., Qu, L., Haffari, G., & Yuan, Z. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20059-5_22 · Cited by 35 · see citing works
Embodied navigation with multi-modal information: A survey from tasks to methodology
Wu, Y., Zhang, P., Gu, M., Zheng, J., & Bai, X. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102532 · Cited by 34 · see citing works
Visual-Language Navigation Pretraining via Prompt-based Environmental Self-exploration
Liang, X., Zhu, F., Lingling, L., Xu, H., & Liang, X. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.332 · Cited by 27 · see citing works
Self-citationSelf-Supervised 3-D Semantic Representation Learning for Vision-and-Language Navigation
Tan, S., Sima, K., Wang, D., Ge, M., Guo, D., & Liu, H. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3395633 · Cited by 24 · see citing works
A Dual Semantic-Aware Recurrent Global-Adaptive Network for Vision-and-Language Navigation
Wang, L., He, Z., Tang, J., Dang, R., Wang, N., Liu, C., & Chen, Q. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/164 · Cited by 23 · see citing works
Self-citationAugmented Commonsense Knowledge for Remote Object Grounding
Mohammadi, B., Hong, Y., Qi, Y., Wu, Q., Pan, S., & Shi, Q. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i5.28223 · Cited by 21 · see citing works
Correctable Landmark Discovery via Large Models for Vision-Language Navigation
Lin, B., Nie, Y., Wei, Z., Zhu, Y., Xu, H., Ma, S., Liu, J., & Liang, X. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3407759 · Cited by 21 · see citing works
Self-citationVLN-PETL: Parameter-Efficient Transfer Learning for Vision-and-Language Navigation
Qiao, Y., Zheng, Y. H., & Wu, Q. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01416 · Cited by 20 · see citing works
Boosting Efficient Reinforcement Learning for Vision-and-Language Navigation With Open-Sourced LLM
Wang, J., Wang, T., Cai, W., Xu, L., & Sun, C. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3511402 · Cited by 19 · see citing works
Zhao, C. K., Qi, Y., & Wu, Q. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3611736 · Cited by 18 · see citing works
Grounded Entity-Landmark Adaptive Pre-training for Vision-and-Language Navigation
Cui, Y., Xie, L., Zhang, Y., Zhang, M., Yan, Y., & Yin, E. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01106 · Cited by 19 · see citing works
PASTS: Progress-aware spatio-temporal transformer speaker for vision-and-language navigation
Wang, L., Liu, C., He, Z., Li, S., Yan, Q., Chen, H., & Chen, Q. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.107487 · Cited by 16 · see citing works
Self-citationVision-and-Language Navigation via Latent Semantic Alignment Learning
Wu, S., Fu, X., Wu, F. H., & Zha, Z.-J. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2024.3358112 · Cited by 16 · see citing works
Enhancing Vision and Language Navigation With Prompt-Based Scene Knowledge
Zhan, Z., Qin, J., Zhuo, W., & Tan, G. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3401451 · Cited by 16 · see citing works
VLN-Video: Utilizing Driving Videos for Outdoor Vision-and-Language Navigation
Li, J., Padmakumar, A., Sukhatme, G. S., & Bansal, M. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29813 · Cited by 15 · see citing works
FOAM: A Follower-aware Speaker Model For Vision-and-Language Navigation
Dou, Z.-Y., & Peng, N. (2022) · Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2022.naacl-main.322 · Cited by 16 · see citing works
Diagram Perception Networks for Textbook Question Answering via Joint Optimization
Ma, J., Liu, J., Chai, Q., Wang, P., & Tao, J. (2023) · International Journal of Computer Vision · doi:10.1007/s11263-023-01954-z · Cited by 13 · see citing works
DAP: Domain-Aware Prompt Learning for Vision-and-Language Navigation
Liu, T., Hu, Y., Wu, W., Wang, Y.-K., Xu, K., & Yin, Q. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10446504 · Cited by 12 · see citing works
Wang, S., Zhou, D., Xie, L., Xu, C., Yan, Y., & Yin, E. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.107320 · Cited by 11 · see citing works
Adaptive Cross-Modal Experts Network with Uncertainty-Driven Fusion for Vision–Language Navigation
Wu, J., Wu, C., Shen, X., & Wang, L. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112735 · Cited by 11 · see citing works
Bridge Gap in Pixel and Feature Level for Cross-Modality Person Re-Identification
Ling, Y., Zhong, Z., Luo, Z., Li, S., & Sebe, N. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3338813 · Cited by 10 · see citing works
Source-Free Elastic Model Adaptation for Vision-and-Language Navigation
Tan, M., Chen, P., Zhi, H., Mai, J., Rosman, B., Ji, D., & Zeng, R. (2025) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2025.3535356 · Cited by 10 · see citing works
Learning Disentanglement with Decoupled Labels for Vision-Language Navigation
Cheng, W., Dong, X., Khan, S. H., & Shen, J. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20059-5_18 · Cited by 10 · see citing works
Wen, S., Gong, S., Zhang, Z., Yu, F. R., & Wang, Z. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112610 · Cited by 9 · see citing works
DICCR: Double-gated intervention and confounder causal reasoning for vision-language navigation
Zhou, D., Deng, J., Pang, Z., & Li, W. (2024) · Neural Networks · doi:10.1016/j.neunet.2024.107078 · Cited by 8 · see citing works
Wang, J., Wang, T., Xu, L., He, Z., & Sun, C. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3398300 · Cited by 8 · see citing works
Navigating Beyond Instructions: Vision-and-Language Navigation in Obstructed Environments
Hong, H., Wang, S., Huang, Z. H., Wu, Q., & Liu, J. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681640 · Cited by 7 · see citing works
Recursive bidirectional cross-modal reasoning network for vision-and-language navigation
Wu, J., Wu, C., Shen, X., Wu, F., & Wang, L. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.126442 · Cited by 7 · see citing works
MossVLN: Memory-Observation Synergistic System for Continuous Vision-Language Navigation
Yu, T., Wu, Y., Cui, Q., Huang, Q., & Yu, J. (2025) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2025.3586105 · Cited by 7 · see citing works
3D Gaussian Map with Open-Set Semantic Grouping for Vision-Language Navigation
Gao, J., Liu, R., & Wang, W. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00864 · Cited by 7 · see citing works
Controllable Navigation Instruction Generation with Chain of Thought Prompting
Kong, X., Chen, J., Wang, W., Su, H., Hu, X., Yang, Y., & Liu, S. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73397-0_3 · Cited by 6 · see citing works
PRET: Planning with Directed Fidelity Trajectory for Vision and Language Navigation
Lu, R., Meng, J., & Zheng, W. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72848-8_5 · Cited by 6 · see citing works
Visual and Textual Commonsense-Enhanced Layout Learning for Vision-and-Language Navigation
Gao, F., Shi, L., Tang, J., Wang, J., Li, S., Ma, S., & Yu, J. (2025) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2025.3604018 · Cited by 5 · see citing works
Multiple Visual Features in Topological Map for Vision-and-Language Navigation
Liu, R., Kong, P., & Zhang, W. (2024) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros58592.2024.10803061 · Cited by 5 · see citing works
PANDA: Prompt-Based Context- and Indoor-Aware Pretraining for Vision and Language Navigation
Liu, T., Hu, Y., Wu, W., Wang, Y.-K., Xu, K., & Yin, Q. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-53305-1_15 · Cited by 5 · see citing works
CAVEN: An Embodied Conversational Agent for Efficient Audio-Visual Navigation in Noisy Environments
Liu, X., Paul, S., Chatterjee, M., & Cherian, A. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i4.28167 · Cited by 6 · see citing works
“Instruction-at-start is a well-explored research area (Anderson et al. 2018; Hong et al. 2021; Ke et al. 2019; Liu et al. 2021; Majumdar et al. 2020; Ma et al. 2019a,b; Zhu et al. 2020; Chen et al. 2021c; Pashevich et al. 2021; Guhur et al. 2021) in which the agent is given a language instruction…”
BackgroundZhang, J., & Ma, K. (2024) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros58592.2024.10801689 · Cited by 4 · see citing works
Lightweight multimodal Cycle-Attention Transformer towards cancer diagnosis
Liu, S., Ma, X., Deng, S., Suo, Y., Zhang, J., & Ng, W. W. Y. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.124616 · Cited by 4 · see citing works
Qin, W., Misu, T., & Wijaya, D. T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.14267 · Cited by 4 · see citing works
A comprehensive review of recent advancements in vision-and-language navigation
Khan, J. T., Aafaq, N., Ali, Q., & Mohsin, M. (2026) · Discover Computing · doi:10.1007/s10791-026-09977-z · Cited by 3 · see citing works
monoVLN: Bridging the Observation Gap between Monocular and Panoramic Vision and Language Navigation
Lu, R., Zhou, Y., Cheng, H., Meng, J., & Zheng, W.-S. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00884 · Cited by 3 · see citing works
Vision Language Navigation with Knowledge-driven Environmental Dreamer
Zhu, F., Lee, V. C. S., Chang, X., & Liang, X. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/204 · Cited by 3 · see citing works
Self-citationVisual-and-Language Navigation: A Survey and TaxonomyPreprint
Wu, W., Chang, T., & Li, X. (2021) · arXiv (Cornell University) · Cited by 3 · see citing works
Improved VLN-BERT with Reinforcing Endpoint Alignment for Vision-and-Language Navigation
Jin, C., Yang, B., & Liu, R. (2024) · Communications in computer and information science · doi:10.1007/978-981-97-6125-8_10 · Cited by 2 · see citing works
Enhancing Non-Coal Object Recognition Using Deep Learning on Conveyor Belts
Li, Y., Hou, Y., Xie, Y., Sheng, M., Wang, M., & Ma, T. (2024) · doi:10.1145/3686490.3686500 · Cited by 2 · see citing works
Kong, P., Liu, R., Xie, Z., & Pang, Z. (2025) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra55743.2025.11127961 · Cited by 2 · see citing works
Evaluating Explanation Methods for Vision-and-Language Navigation
Chen, G., Yang, L., Chen, G., & Pan, J. (2023) · Frontiers in artificial intelligence and applications · doi:10.3233/faia230295 · Cited by 2 · see citing works
Ambiguity Resolution in Vision-and-Language Navigation with Large Language Models
Wei, S., Wang, C., & Qi, J. (2024) · IEEE International Conference on Information Technology, Big Data and Artificial Intelligence (ICIBA) · doi:10.1109/iciba62489.2024.10868019 · Cited by 1 · see citing works
Action-Aware Visual-Textual Alignment for Long-Instruction Vision-and-Language Navigation
Huang, B., Zheng, Y., Lan, C., Sui, D., Zhao, X., Zhang, X., Xiao, M., & Yu, D. (2025) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3748656 · Cited by 1 · see citing works
History-Guided Prompt Generation for Vision-and-Language Navigation
Guo, W. Q., Wang, Z., Hu, Y., & Gao, J. (2025) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2025.3613147 · Cited by 1 · see citing works
Observation-Graph Interaction and Key-Detail Guidance for Vision and Language Navigation
Xie, Y., Ou, B., Ma, F., & Liu, Y. (2025) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros60139.2025.11247072 · Cited by 1 · see citing works
Bridging the Visual Semantic Gap in VLN via Semantically Richer Instructions
Stanke, J. O., Earle, B., & Soto, Á. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19836-6_4 · Cited by 1 · see citing works
Yuan, Z., Tang, P., Sang, X., Zhang, F., & Zhang, Z. (2024) · The Visual Computer · doi:10.1007/s00371-024-03469-1 · Cited by 1 · see citing works
Model Adaptation for Time Constrained Embodied Control
Song, J., Yoo, M., & Woo, H. (2024) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr52733.2024.01561
Language-Conditioned Waypoint Predictor for Continuous Vision-and-Language Navigation
Wang, Z., Qi, Y., An, D. U., Xu, Y., Li, H., & Zhang, Z. (2025) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme59968.2025.11209085
SDA-LLM: Spatial DisAmbiguation via Multi-turn Vision-Language Dialogues for Robot Navigation
Chen, K.-L., Wei, T.-T., Lee, M.-L., Yeh, L.-T., Kao, E., Tseng, Y., & Chen, J. (2025) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros60139.2025.11246115
HAVEN: From Human Guidance to Assistant by Evolution Network in Vision-and-Language Navigation
Lai, P., Xie, Z., & Yang, H. (2025) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11227307
Manifold-Aware Historical Topology Mapping for Foundation Model-Based Robotic Navigation
Li, H., Zhan, Z., Yang, Y., Meng, F., Chen, Y., Zhao, C., & Wang, Q. (2026) · Computing and Informatics · doi:10.31577/cai_2026_2_488
DCSEN: Dual Channel Semantic Enhancement for Vision-and-Language Navigation
Li, H., Liu, L., Dong, X., Zhu, L., Chang, X., & Zhang, H. (2026) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2026.3714818
Self-citationFarouq, A. A., Hosoda, Y., & Lee, J.-H. (2026) · International Conference on Ubiquitous Robots (UR) · doi:10.1109/ur69537.2026.11626792
Yan, Y., Li, P., & Yin, J. (2026) · Knowledge-Based Systems · doi:10.1016/j.knosys.2026.117088
Trajectory-Diversity-Driven Robust Vision-And-Language Navigation
Li, J., Wan, C., Dong, S., Ding, C., Wang, Q., Ma, Z., & Gong, Y. (2026) · doi:10.1109/cvprf71909.2026.00877
SCTNav: Scene-Constrained Monocular Navigation With Ternary Spatio-Temporal Waypoint Prediction
Guo, W., Wang, J., Zheng, Y., Zhang, X., Zhang, X., & Yu, D. (2026) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2026.3747589
- Authors
- Liu, C., Zhu, F., Chang, X., Liang, X., Ge, Z., & Shen, Y.-D.
- Affiliations
- Chinese Academy of Sciences (CN)
- Monash University (AU)
- RMIT University (AU)
- Sun Yat-sen University (CN)
- Venue
- 2021 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2021-10
- Pages
- 1624-1634
- OpenAlex
- W3165915253
- Cited by
- 80
- References
- 71
- Citations by year
- 2026: 5 · 2025: 16 · 2024: 29 · 2023: 17 · 2022: 11 · 2021: 2
- FWCI
- 4.82
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by-nc-sa)
- arXiv (Cornell University)
- Funders
- National Key Research and Development Program of China (2014CB340301)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
- Keywords
- vision-and-language navigation, data augmentation, data bias, generalization to unseen environments, mixup
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
