Transferable Representation Learning in Vision-and-Language Navigation
Open accessAbstract
Vision-and-Language Navigation (VLN) tasks such as Room-to-Room (R2R) require machine agents to interpret natural language instructions and learn to act in visually realistic environments to achieve navigation goals. The overall task requires competence in several perception problems: successful agents combine spatio-temporal, vision and language understanding to produce appropriate action sequences. Our approach adapts pre-trained vision and language representations to relevant in-domain tasks making them more effective for VLN. Specifically, the representations are adapted to solve both a cross-modal sequence alignment and sequence coherence task. In the sequence alignment task, the model determines whether an instruction corresponds to a sequence of visual frames. In the sequence coherence task, the model determines whether the perceptual sequences are predictive sequentially in the instruction-conditioned latent space. By transferring the domain-adapted representations, we improve competitive agents in R2R as measured by the success rate weighted by path length (SPL) metric.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
NaVid uses a video-driven VLM to navigate from RGB video and human instructions.
“Similarly, NaVid [114] leverages a videodriven Vision-Language Model (VLM), allowing robots to navigate using RGB video and human-issued instructions.”
From Multimodal fusion and vision–language models: A survey for robot vision (2026) · cites this paper as [114]
NaVid achieved SOTA on R2R and R2R-RxR and showed Sim2Real transferability.
“It has achieved state-of-the-art (SOTA) performance on R2R and R2R-RxR tasks while exhibiting remarkable Sim2Real transferability.”
From Multimodal fusion and vision–language models: A survey for robot vision (2026) · cites this paper as [114]
InstructNav, NaVid, BEVBert, and CVDN demonstrate transformative potential of multimodal fusion in embodied navigation.
“These methods, such as InstructNav [9], NaVid [114], BEVBert [116], and CVDN [124], demonstrate the transformative potential of multimodal, fusion in embodied navigation.”
From Multimodal fusion and vision–language models: A survey for robot vision (2026) · cites this paper as [114]
Integrating visual input, language processing, and spatial reasoning lets robots follow natural language instructions, navigate to visual goals, or perform goal-directed tasks with greater flexibility and adaptability.
“By integrating visual input, language processing, and spatial reasoning, robots can now follow natural language instructions, navigate toward visual goals, or perform goal-directed navigation tasks with greater flexibility and adaptability”
From Multimodal fusion and vision–language models: A survey for robot vision (2026) · cites this paper as [114]
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · see citing works
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · see citing works
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071 · see citing works
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684 · see citing works
Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839 · see citing works
Bidirectional recurrent neural networks
Schuster, M., & Paliwal, K. K. (1997) · IEEE Transactions on Signal Processing · doi:10.1109/78.650093 · Cited by 10,172 · see citing works
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248 · see citing works
Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Machine Learning · doi:10.1007/bf00992696 · Cited by 7,936 · see citing works
Show and tell: A neural image caption generator
Vinyals, O., Toshev, A., Bengio, S., & Erhan, D. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298935 · Cited by 6,318 · see citing works
Representation Learning with Contrastive Predictive Coding
Oord, A. V. D., Li, Y., & Vinyals, O. (2018) · arXiv · doi:10.48550/arxiv.1807.03748 · Cited by 6,242 · see citing works
Deep visual-semantic alignments for generating image descriptions
Karpathy, A., & Fei-Fei, L. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298932 · Cited by 4,902 · see citing works
VQA: Visual Question Answering
Antol, S., Agrawal, A., Lu, J., Mitchell, M., Batra, D., Zitnick, C. L., & Parikh, D. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.279 · Cited by 4,487 · see citing works
Stacked Attention Networks for Image Question Answering
Yang, Z., He, X., Gao, J., Deng, L., & Smola, A. J. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.10 · Cited by 2,036 · see citing works
Matterport3D: Learning from RGB-D Data in Indoor Environments
Chang, A., Dai, A., Funkhouser, T., Halber, M., Niebner, M., Savva, M., Song, S., Zeng, A., & Zhang, Y. (2017) · 2017 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2017.00081 · Cited by 1,858 · see citing works
Long-Term Recurrent Convolutional Networks for Visual Recognition and Description
Donahue, J., Hendricks, L. A., Rohrbach, M., Venugopalan, S., Guadarrama, S., Saenko, K., & Darrell, T. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2599174 · Cited by 1,547 · see citing works
Target-driven visual navigation in indoor scenes using deep reinforcement learning
Zhu, Y., Mottaghi, R., Kolve, E., Lim, J. J., Gupta, A., Fei-Fei, L., & Farhadi, A. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989381 · Cited by 1,533 · see citing works
From captions to visual concepts and back
Fang, H., Gupta, S., Iandola, F., Srivastava, R. K., Deng, L., Dollár, P., Gao, J., He, X., Mitchell, M., Platt, J. C., et al. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298754 · Cited by 1,305 · see citing works
Generation and Comprehension of Unambiguous Object Descriptions
Mao, J., Huang, J., Toshev, A., Camburu, O., Yuille, A., & Murphy, K. P. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.9 · Cited by 1,211 · see citing works
Anderson, P., Wu, Q., Teney, D., Bruce, J., Johnson, M., Sunderhauf, N., Reid, I., Gould, S., & van den Hengel, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00387 · Cited by 1,187 · see citing works
Prediction, cognition and the brain
Bubić, A. (2010) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2010.00025 · Cited by 658 · see citing works
On evaluation of embodied navigation agents
(2018) · doi:10.48550/arxiv.1807.06757 · Cited by 622 · see citing works
Video Paragraph Captioning Using Hierarchical Recurrent Neural Networks
Yu, H., Wang, J., Huang, Z., Yang, Y., & Xu, W. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.496 · Cited by 568 · see citing works
Wang, X., Huang, Q., Celikyilmaz, A., Gao, J., Shen, D., Wang, Y.-F., Wang, W. Y., & Zhang, L. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00679 · Cited by 522 · see citing works
Segmentation from Natural Language Expressions
Hu, R., Rohrbach, M., & Darrell, T. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46448-0_7 · Cited by 467 · see citing works
Das, A., Kottur, S., Gupta, K., Singh, A., Yadav, D., Moura, J. M. F., Parikh, D., & Batra, D. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.121 · Cited by 445 · see citing works
Das, A., Kottur, S., Gupta, K., Singh, A., Jain, D., Moura, J. M. F., Parikh, D., & Batra, D. (2017) · Cited by 432 · see citing works
A framework for behavioural cloning
Bain, M., & Sammut, C. (2000) · doi:10.1093/oso/9780198538677.003.0006 · Cited by 370 · see citing works
A Survey of Available Corpora For Building Data-Driven Dialogue Systems: The Journal Version
Serban, I. V., Lowe, R., Henderson, P., Charlin, L., & Pineau, J. (2018) · Dialogue & Discourse · doi:10.5087/dad.2018.101 · Cited by 342 · see citing works
Walk the talk: connecting language, knowledge, and action in route instructions
MacMahon, M., Stankiewicz, B. J., & Kuipers, B. J. (2006) · Cited by 310 · see citing works
Learning to Navigate Unseen Environments: Back Translation with Environmental Dropout
Tan, H., Yu, L., & Bansal, M. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1268 · Cited by 293 · see citing works
Speaker-Follower Models for Vision-and-Language Navigation
Fried, D., Hu, R., Cirik, V., Rohrbach, A., Andreas, J., Morency, L.-P., Berg-Kirkpatrick, T., Saenko, K., Klein, D., & Darrell, T. (2018) · arXiv · doi:10.48550/arxiv.1806.02724 · Cited by 291 · see citing works
Video Captioning via Hierarchical Reinforcement Learning
Wang, X. E., Chen, W., Wu, J., Wang, Y.-F., & Wang, W. Y. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00443 · Cited by 269 · see citing works
Learning to Navigate in Cities Without a MapPreprint
Mirowski, P., Grimes, M. K., Malinowski, M., Hermann, K. M., Anderson, K., Teplyashin, D., Simonyan, K., Kavukcuoglu, K., Zisserman, A., & Hadsell, R. T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.00168 · Cited by 248 · see citing works
Visual Representations for Semantic Target Driven Navigation
Mousavian, A., Toshev, A., Fišer, M., Košecká, J., Wahid, A., & Davidson, J. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8793493 · Cited by 227 · see citing works
Self-Monitoring Navigation Agent via Auxiliary Progress Estimation
Ma, C.-Y., Lu, J., Wu, Z., AlRegib, G., Kira, Z., Socher, R., & Xiong, C. (2019) · arXiv · doi:10.48550/arxiv.1901.03035 · Cited by 200 · see citing works
Wang, X. E., Xiong, W., Wang, H., & Wang, W. Y. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01270-0_3 · Cited by 187 · see citing works
The Regretful Agent: Heuristic-Aided Navigation Through Progress Estimation
Ma, C., Wu, Z., AlRegib, G., Xiong, C., & Kira, Z. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00689 · Cited by 166 · see citing works
Talk the Walk: Navigating New York City through Grounded DialoguePreprint
de Vries, H., Shuster, K., Batra, D., Parikh, D., Weston, J., & Kiela, D. (2018) · arXiv (Cornell University) · doi:10.4230/lipics.cosit.2022.24 · Cited by 112 · see citing works
Learning models for following natural language directions in unknown environments
Hemachandra, S. M., Duvallet, F., Howard, T. M., Roy, N., Stentz, A., & Walter, M. R. (2015) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2015.7139984 · Cited by 75 · see citing works
End-to-End Retrieval in Continuous SpacePreprint
Gillick, D., Presta, A., & Tomar, G. S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.08008 · Cited by 65 · see citing works
Learning Transferable Policies for Monocular Reactive MAV Control
Daftry, S., Bagnell, J. A., & Hebert, M. (2017) · Springer proceedings in advanced robotics · doi:10.1007/978-3-319-50115-4_1 · Cited by 56 · see citing works
Optimal Perceived Timing: Integrating Sensory Information with Dynamically Updated Expectations
Luca, M. D., & Rhodes, D. (2016) · Scientific Reports · doi:10.1038/srep28563 · Cited by 47 · see citing works
Grounded Language Learning: Where Robotics and NLP Meet
Matuszek, C. (2018) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/810 · Cited by 35 · see citing works
Grounded Language Learning: Where Robotics and NLP Meet
Matuszek, C. (2021) · Maryland Shared Open Access Repository (USMAI Consortium) · doi:10.13016/m2fiye-fv1d · Cited by 32 · see citing works
Learning Transferable Policies for Monocular Reactive MAV ControlPreprint
Daftry, S., Bagnell, J. A., & Hebert, M. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1608.00627 · Cited by 27 · see citing works
Multi-modal Discriminative Model for Vision-and-Language Navigation
Huang, H., Jain, V., Mehta, H., Baldridge, J., & Ie, E. (2019) · doi:10.18653/v1/w19-1605 · Cited by 22 · see citing works
Self-citationLearning to Follow Directions in Street View
Hermann, K. M., Malinowski, M., Mirowski, P., Banki-Horvath, A., Anderson, K., & Hadsell, R. T. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6849 · Cited by 6 · see citing works
Flagged first, then by citations
5 of the 53 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Grounding
Ku, A., Anderson, P., Patel, R., Ie, E., & Baldridge, J. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.356 · Cited by 274 · see citing works
Self-citationVision-language navigation with self-supervised auxiliary reasoning tasks
Zhu, F., Zhu, Y., Chang, X., & Liang, X. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01003 · Cited by 223 · see citing works
How Much Can CLIP Benefit Vision-and-Language Tasks?Preprint
Shen, S., Li, L. H., Tan, H. X., Bansal, M., Rohrbach, A., Chang, K.-W., Yao, Z., & Keutzer, K. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.06383 · Cited by 272 · see citing works
Airbert: In-domain Pretraining for Vision-and-Language Navigation
Guhur, P.-L., Tapaswi, M., Chen, S., Laptev, I., & Schmid, C. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00166 · Cited by 139 · see citing works
Episodic Transformer for Vision-and-Language Navigation
Pashevich, A., Schmid, C., & Sun, C. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01564 · Cited by 135 · see citing works
SOON: Scenario Oriented Object Navigation with Graph-based Exploration
Zhu, F., Liang, X., Zhu, Y., Yu, Q., Chang, X., & Liang, X. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01250 · Cited by 136 · see citing works
Structured Scene Memory for Vision-Language Navigation
Wang, H., Wang, W., Liang, W., Xiong, C., & Shen, J. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00835 · Cited by 123 · see citing works
Multimodal research in vision and language: A review of current and emerging trends
Uppal, S., Bhagat, S., Hazarika, D., Majumder, N., Poria, S., Zimmermann, R., & Zadeh, A. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.07.009 · Cited by 112 · see citing works
Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directions
Gu, J., Stefani, E., Wu, Q., Thomason, J., & Wang, X. (2022) · Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2022.acl-long.524 · Cited by 216 · see citing works
Video Moment Localization via Deep Cross-Modal Hashing
Hu, Y., Liu, M., Su, X., Gao, Z., & Nie, L. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3073867 · Cited by 100 · see citing works
A survey on visual navigation for artificial agents with deep reinforcement learning
Zeng, F., Wang, C., & Ge, S. S. (2020) · IEEE Access · doi:10.1109/access.2020.3011438 · Cited by 98 · see citing works
Envedit: Environment Editing for Vision-and-Language Navigation
Li, J., Tan, H., & Bansal, M. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01497 · Cited by 87 · see citing works
Room-and-Object Aware Knowledge Reasoning for Remote Embodied Referring Expression
Gao, C., Chen, J., Liu, S., Wang, L., Zhang, Q., & Wu, Q. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00308 · Cited by 82 · see citing works
HOP: History-and-Order Aware Pretraining for Vision-and-Language Navigation
Qiao, Y., Qi, Y., Hong, Y., Yu, Z., Wang, P., & Wu, Q. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01498 · Cited by 82 · see citing works
Vision-Language Navigation with Random Environmental Mixup
Liu, C., Zhu, F., Chang, X., Liang, X., Ge, Z., & Shen, Y.-D. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00167 · Cited by 80 · see citing works
Counterfactual Vision-and-Language Navigation via Adversarial Path Sampler
Fu, T.-J., Wang, X. E., Peterson, M., Grafton, S. T., Eckstein, M. P., & Wang, W. Y. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58539-6_5 · Cited by 79 · see citing works
Object-and-action aware model for visual language navigation
Qi, Y., Pan, Z., Zhang, S., van den Hengel, A., & Wu, Q. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58607-2_18 · Cited by 100 · see citing works
Active visual information gathering for vision-language navigation
Wang, H., Wang, W., Shu, T., Liang, W., & Shen, J. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58542-6_19 · Cited by 74 · see citing works
Language-Guided Navigation via Cross-Modal Grounding and Alternate Adversarial Learning
Zhang, W., Ma, C., Wu, Q., & Yang, X. (2020) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2020.3039522 · Cited by 73 · see citing works
Bird’s-Eye-View Scene Graph for Vision-Language Navigation
Liu, R., Wang, X., Wang, W., & Yang, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01007 · Cited by 69 · see citing works
Vision-language navigation: a survey and taxonomy
Wu, W., Chang, T., Li, X., Yin, Q., & Hu, Y. (2023) · Neural Computing and Applications · doi:10.1007/s00521-023-09217-1 · Cited by 65 · see citing works
Navgpt-2: Unleashing navigational reasoning capability for large vision-language models
Zhou, G., Hong, Y., Wang, Z., Wang, X. E., & Wu, Q. (2025) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-72667-5_15 · Cited by 63 · see citing works
BabyWalk: Going Farther in Vision-and-Language Navigation by Taking Baby Steps
Zhu, W., Hu, H., Chen, J., Deng, Z., Jain, V., Ie, E., & Sha, F. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.229 · Cited by 60 · see citing works
Self-citationSub-Instruction Aware Vision-and-Language Navigation
Hong, Y., Rodríguez, C., Wu, Q., & Gould, S. J. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.271 · Cited by 60 · see citing works
Wang, H., Liang, W. G., Shen, J., Gool, L. V., & Wang, W. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01503 · Cited by 57 · see citing works
Environment-Agnostic Multitask Learning for Natural Language Grounded Navigation
Wang, X. E., Jain, V., Ie, E., Wang, W. Y., Kozareva, Z., & Ravi, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58586-0_25 · Cited by 54 · see citing works
Self-citationADAPT: Vision-Language Navigation with Modality-Aligned Action Prompts
Lin, B., Zhu, Y., Chen, Z., Liang, X., Liu, J., & Liang, X. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01496 · Cited by 52 · see citing works
HOP+: History-Enhanced and Order-Aware Pre-Training for Vision-and-Language Navigation
Qiao, Y., Qi, Y., Hong, Y., Yu, Z., Wang, P., & Wu, Q. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3234243 · Cited by 52 · see citing works
Dreamwalker: Mental Planning for Continuous Vision-Language Navigation
Wang, H., Liang, W. G., Gool, L. V., & Wang, W. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00998 · Cited by 50 · see citing works
Target-Driven Structured Transformer Planner for Vision-Language Navigation
Zhao, Y., Chen, J., Gao, C., Wang, W., Yang, L., Ren, H., Xia, H., & Liu, S. (2022) · Proceedings of the 30th ACM International Conference on Multimedia · doi:10.1145/3503161.3548281 · Cited by 54 · see citing works
Diagnosing the Environment Bias in Vision-and-Language Navigation
Zhang, Y., Tan, H., & Bansal, M. (2020) · Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/124 · Cited by 51 · see citing works
Visual language navigation: a survey and open challenges
Park, S., & Kim, Y. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-022-10174-9 · Cited by 45 · see citing works
Multimodal fusion and vision–language models: A survey for robot vision
Han, X., Chen, S., Fu, Z., Feng, Z., Fan, L., An, D., Wang, C., Guo, L., Meng, W., Zhang, X., et al. (2026) · Information Fusion · doi:10.1016/j.inffus.2025.103652 · Cited by 41 · see citing works
“Sparsh [166] uses a universal tactile representation method pre-trained on a large tactile dataset, enhancing slip detection, grasp stability, and planning accuracy in dynamic environments.”
On the Evaluation of Vision-and-Language Navigation Instructions
Zhao, M., Anderson, P., Jain, V., Wang, S., Ku, A., Baldridge, J., & Ie, E. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-main.111 · Cited by 45 · see citing works
Self-citationLANA: A Language-Capable Navigator for Instruction Following and Generation
Wang, X., Wang, W., Shao, J., & Yang, Y. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01826 · Cited by 37 · see citing works
Diagnosing Vision-and-Language Navigation: What Really Matters
Zhu, W., Qi, Y., Narayana, P., Sone, K., Basu, S., Wang, X., Wu, Q., Eckstein, M. P., & Wang, W. Y. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.438 · Cited by 35 · see citing works
Embodied navigation with multi-modal information: A survey from tasks to methodology
Wu, Y., Zhang, P., Gu, M., Zheng, J., & Bai, X. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102532 · Cited by 34 · see citing works
Learning Vision-and-Language Navigation from YouTube Videos
Lin, K., Chen, P., Huang, D.-W., Li, T. H., Tan, M., & Gan, C. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00764 · Cited by 30 · see citing works
Improving Cross-Modal Alignment in Vision Language Navigation via Syntactic Information
Li, J., Tan, H. X., & Bansal, M. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.82 · Cited by 26 · see citing works
Visual-Language Navigation Pretraining via Prompt-based Environmental Self-exploration
Liang, X., Zhu, F., Lingling, L., Xu, H., & Liang, X. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.332 · Cited by 27 · see citing works
Correctable Landmark Discovery via Large Models for Vision-Language Navigation
Lin, B., Nie, Y., Wei, Z., Zhu, Y., Xu, H., Ma, S., Liu, J., & Liang, X. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3407759 · Cited by 21 · see citing works
Active Perception for Visual-Language Navigation
Wang, H., Wang, W., Liang, W., Hoi, S. C. H., Shen, J., & Gool, L. V. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-022-01721-6 · Cited by 20 · see citing works
Grounded Entity-Landmark Adaptive Pre-training for Vision-and-Language Navigation
Cui, Y., Xie, L., Zhang, Y., Zhang, M., Yan, Y., & Yin, E. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01106 · Cited by 19 · see citing works
Qiao, Y., Lyu, W., Wang, H. Z., Wang, Z., Li, Z., Zhang, Y., Tan, M., & Wu, Q. (2025) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra55743.2025.11127584 · Cited by 16 · see citing works
Counterfactual Vision-and-Language Navigation via Adversarial Path SamplingPreprint
Fu, T.-J., Wang, X. E., Peterson, M., Grafton, S. T., Eckstein, M. P., & Wang, W. Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.07308 · Cited by 16 · see citing works
PASTS: Progress-aware spatio-temporal transformer speaker for vision-and-language navigation
Wang, L., Liu, C., He, Z., Li, S., Yan, Q., Chen, H., & Chen, Q. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.107487 · Cited by 16 · see citing works
Deep Learning for Embodied Vision Navigation: A SurveyPreprint
Zhu, F., Zhu, Y., Lee, V. C., Liang, X., & Chang, X. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.04097 · Cited by 19 · see citing works
From Seeing to Moving: A Survey on Learning for Visual Indoor Navigation (VIN)Preprint
Ye, X., & Yang, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.11310 · Cited by 14 · see citing works
Kuo, C.-W., Ma, C., Hoffman, J., & Kira, Z. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00116 · Cited by 11 · see citing works
Object-aware navigation for remote embodied visual referring expression
Zhan, Z., Lin, L., & Tan, G. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.10.026 · Cited by 9 · see citing works
Learning to Follow and Generate Instructions for Language-Capable Navigation
Wang, X., Wang, W., Shao, J., & Yang, Y. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3341828 · Cited by 9 · see citing works
On the Evaluation of Vision-and-Language Navigation InstructionsPreprint
Zhao, M., Anderson, P., Jain, V., Wang, S., Ku, A., Baldridge, J., & Ie, E. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.10504 · Cited by 9 · see citing works
Self-citationA Priority Map for Vision-and-Language Navigation with Trajectory Plans and Feature-Location Cues
Armitage, J., Impett, L., & Sennrich, R. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00115 · Cited by 8 · see citing works
Wang, J., Wang, T., Xu, L., He, Z., & Sun, C. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3398300 · Cited by 8 · see citing works
Mind the Error! Detection and Localization of Instruction Errors in Vision-and-Language Navigation
Taioli, F., Rosa, S., Castellini, A., Natale, L., Bue, A. D., Farinelli, A., Cristani, M., & Wang, Y. (2024) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros58592.2024.10801822 · Cited by 7 · see citing works
Depth-Guided AdaIN and Shift Attention Network for Vision-And-Language Navigation
Sun, Q., Zhuang, Y., Chen, Z., Fu, Y., & Xue, X. (2021) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme51207.2021.9428422 · Cited by 7 · see citing works
Generative Language-Grounded Policy in Vision-and-Language Navigation with Bayes' RulePreprint
Kurita, S., & Cho, K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.07783 · Cited by 7 · see citing works
Modular Graph Attention Network for Complex Visual Relational Reasoning
Zheng, Y., Wen, Z., Tan, M., Zeng, R., Chen, Q., Wang, Y., & Wu, Q. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-69544-6_9 · Cited by 6 · see citing works
VALAN: Vision and Language Agent NavigationPreprint
Lansing, L., Jain, V., Mehta, H., Huang, H., & Ie, E. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.03241 · Cited by 6 · see citing works
Self-citationHierarchical Spatial Proximity Reasoning for Vision-and-Language Navigation
Xu, M., & Xie, Z. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3477129 · Cited by 5 · see citing works
Survey on Multimodal Transformers for RobotsPreprint
Miyazawa, K., & Nagai, T. (2023) · doi:10.36227/techrxiv.21993317 · Cited by 5 · see citing works
Multimodal Pretrained Knowledge for Real-world Object Navigation
Yuan, H. L., Huang, Y., Yu, N., Zhang, D., Du, Z., Liu, Z., & Zhang, K. (2025) · Machine Intelligence Research · doi:10.1007/s11633-024-1537-x · Cited by 4 · see citing works
CLEAR: Improving Vision-Language Navigation with Cross-Lingual, Environment-Agnostic Representations
Li, J., Tan, H., & Bansal, M. (2022) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2022.findings-naacl.48 · Cited by 5 · see citing works
Visual-and-Language Navigation: A Survey and TaxonomyPreprint
Wu, W., Chang, T., & Li, X. (2021) · arXiv (Cornell University) · Cited by 3 · see citing works
A comprehensive review of recent advancements in vision-and-language navigation
Khan, J. T., Aafaq, N., Ali, Q., & Mohsin, M. (2026) · Discover Computing · doi:10.1007/s10791-026-09977-z · Cited by 3 · see citing works
Evaluating Explanation Methods for Vision-and-Language Navigation
Chen, G., Yang, L., Chen, G., & Pan, J. (2023) · Frontiers in artificial intelligence and applications · doi:10.3233/faia230295 · Cited by 2 · see citing works
Local Slot Attention for Vision and Language Navigation
Zhuang, Y., Sun, Q., Fu, Y., Chen, L., & Xue, X. (2022) · International Conference on Multimedia Retrieval · doi:10.1145/3512527.3531366 · Cited by 2 · see citing works
Survey on the Research Progress and Development Trend of Vision-and-Language Navigation
Niu, K., & Wang, P. (2022) · Journal of Computer-Aided Design & Computer Graphics · doi:10.3724/sp.j.1089.2022.19249 · Cited by 2 · see citing works
Rethinking the Spatial Route Prior in Vision-and-Language NavigationPreprint
Zhou, X., Liu, W., & Mu, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.05728 · Cited by 2 · see citing works
Tree-Structured Trajectory Encoding for Vision-and-Language Navigation
Zhou, X., & Mu, Y. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i3.25494 · Cited by 1 · see citing works
Towards Coarse-grained Visual Language Navigation Task Planning Enhanced by Event Knowledge Graph
Zhao, K., Song, Y., Zhao, H., Liu, H., Li, T., & Li, Z. (2024) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3627673.3679711 · Cited by 1 · see citing works
Closed Loop Interactive Embodied Reasoning for Robot Manipulation
Nazarczuk, M., Behrens, J. K., Štěpánová, K., Hoffmann, M., & Mikolajczyk, K. (2025) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra55743.2025.11127480 · Cited by 1 · see citing works
Weakly-supervised VLM-guided Partial Contrastive Learning for Visual Language Navigation
Wang, R., Yu, T., Wu, J., Liu, Y., McAuley, J., & Yao, L. (2025) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros60139.2025.11247068 · Cited by 3 · see citing works
Fine-Grained Alignment Supervision Matters in Vision-and-Language Navigation
He, K., Huang, Y., Jing, Y., Wu, Q., & Wang, L. (2026) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2026.3658949 · Cited by 1 · see citing works
Yuan, Z., Tang, P., Sang, X., Zhang, F., & Zhang, Z. (2024) · The Visual Computer · doi:10.1007/s00371-024-03469-1 · Cited by 1 · see citing works
Efficient Hashing Based Video Moment Localization
Liu, M., Hu, Y., Guan, W., & Nie, L. (2025) · Springer eBooks · doi:10.1007/978-3-031-87588-5_7 · Cited by 1 · see citing works
YFormer: A New Transformer Architecture for Video-Query Based Video Moment Retrieval
Huo, S., Zhou, Y., & Wang, H. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-18913-5_49 · Cited by 1 · see citing works
Survey on Multimodal Transformers for RobotsPreprint
Miyazawa, K., & Nagai, T. (2023) · doi:10.36227/techrxiv.21993317.v1 · Cited by 1 · see citing works
Grounding Implicit Goal Description for Robot Indoor Navigation via Recursive Belief Update
Chen, R., Zhao, J., & Zhang, L. (2020) · doi:10.1109/iccr51572.2020.9344198
Cluster-based Curriculum Learning for Vision-and-Language Navigation
Wang, T., Wu, Z., Liu, Z., & Wang, D. (2022) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn55064.2022.9892100
Robots that can see : Learning visually guided behavior
Pashevich, A. (2021) · theses.fr (ABES) · doi:10.70675/03640d1fz838cz4bcbz8665z5f769976ea7b
Trajectory-Diversity-Driven Robust Vision-And-Language Navigation
Li, J., Wan, C., Dong, S., Ding, C., Wang, Q., Ma, Z., & Gong, Y. (2026) · doi:10.1109/cvprf71909.2026.00877
- Authors
- Huang, H., Jain, V., Mehta, H., Ku, A., Magalhaes, G., Baldridge, J., & Ie, E.
- Affiliations
- Google (United States) (US)
- Venue
- 2019 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2019-10
- Pages
- 7403-7412
- OpenAlex
- W2981799368
- Cited by
- 82
- References
- 53
- Citations by year
- 2026: 3 · 2025: 6 · 2024: 8 · 2023: 15 · 2022: 16 · 2021: 18 · 2020: 14 · 2019: 2
- FWCI
- 4.07
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by-nc-sa)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
- Keywords
- vision-and-language navigation, domain adaptation, natural language instructions
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
