VLN↻BERT: A Recurrent Vision-and-Language BERT for Navigation
Abstract
Accuracy of many visiolinguistic tasks has benefited significantly from the application of vision-and-language (V&L) BERT. However, its application for the task of vision-and-language navigation (VLN) remains limited. One reason for this is the difficulty adapting the BERT architecture to the partially observable Markov decision process present in VLN, requiring history-dependent attention and decision making. In this paper we propose a recurrent BERT model that is time-aware for use in VLN. Specifically, we equip the BERT model with a recurrent function that maintains cross-modal state information for the agent. Through extensive experiments on R2R and REVERIE we demonstrate that our model can replace more complex encoder-decoder models to achieve state-of-the-art results. Moreover, our approach can be generalised to other transformer-based architectures, supports pre-training, and is capable of solving navigation and referring expression tasks simultaneously.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
VLMbench is suited for evaluating vision-language models such as CLIP and GPT-4V, but its 8-task set may limit comprehensiveness for benchmarking generalizable policies.
“While VLMbench [290] is well-suited for evaluating visionlanguage models (VLMs) such as CLIP and GPT-4V, its task set is relatively small (only 8 tasks), potentially limiting its comprehensiveness in benchmarking generalizable policies.”
From Multimodal fusion and vision–language models: A survey for robot vision (2026) · cites this paper as [290]
Unlike VLMbench and GemBench, VIMA integrates multi-modal task execution and precise instruction following.
“Unlike VLMbench [290] and Gem-Bench [289], which primarily evaluate vision-language models (VLMs) in robotic control, VIMA [291] further integrates multi-modal task execution and precise instruction following, making it distinct in its benchmarking approach.”
From Multimodal fusion and vision–language models: A survey for robot vision (2026) · cites this paper as [290]
FSN passes messages through a small number of bottleneck latents, requiring the model to purify the most necessary information from each modality for crossmodal sharing.
“FSN passes on the messages through a small number of bottleneck latents, thus requiring the model to purify the most necessary information from each modality for crossmodal sharing.”
From Multimodal learning with transformers: A survey (2023) · cites this paper as [175]
FSN's fusion-bottleneck strategy improves fusion performance and reduces computational cost.
“This strategy uses the fusion bottleneck as a bridge, and not only improves fusion performance, but also reduces computational cost.”
From Multimodal learning with transformers: A survey (2023) · cites this paper as [175]
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · see citing works
Jakubův, J., Chvalovský, K., Goertzel, Z., et al. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,648 · see citing works
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812 · see citing works
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 56,035 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
Layer NormalizationPreprint
Ba, J., Kiros, J., & Hinton, G. E. (2016) · arXiv.org · doi:10.48550/arxiv.1607.06450 · Cited by 13,745 · see citing works
Decoupled Weight Decay Regularization
Loshchilov, I., & Hutter, F. (2017) · arXiv · doi:10.48550/arxiv.1711.05101 · Cited by 12,522 · see citing works
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132 · see citing works
Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering
Anderson, P., He, X., Buehler, C., Teney, D., Johnson, M., Gould, S. J., & Zhang, L. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00636 · Cited by 5,117 · see citing works
Self-citationVisual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations
Krishna, R., Zhu, Y., Groth, O., Johnson, J., Hata, K., Kravitz, J., Chen, S., Kalantidis, Y., Li, L.-J., Shamma, D. A., et al. (2017) · International Journal of Computer Vision · doi:10.1007/s11263-016-0981-7 · Cited by 5,107 · see citing works
Places: A 10 Million Image Database for Scene Recognition
Zhou, B., Lapedriza, A., Khosla, A., Oliva, A., & Torralba, A. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2723009 · Cited by 4,042 · see citing works
LXMERT: Learning Cross-Modality Encoder Representations from Transformers
Tan, H., & Bansal, M. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1514 · Cited by 2,319 · see citing works
ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks
Lu, J., Batra, D., Parikh, D., & Lee, S. (2019) · arXiv · doi:10.48550/arxiv.1908.02265 · Cited by 2,109 · see citing works
UNITER: UNiversal Image-TExt Representation Learning
Chen, Y.-C., Li, L., Yu, L., El Kholy, A., Ahmed, F., Gan, Z., Cheng, Y., & Liu, J. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58577-8_7 · Cited by 2,055 · see citing works
Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks
Li, X., Yin, X. P., Li, C., Zhang, P., Hu, X., Zhang, L., Wang, L., Hu, H., Li, D., Wei, F., et al. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58577-8_8 · Cited by 1,736 · see citing works
VideoBERT: A Joint Model for Video and Language Representation Learning
Sun, C., Myers, A., Vondrick, C., Murphy, K. P., & Schmid, C. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00756 · Cited by 1,212 · see citing works
VisualBERT: A Simple and Performant Baseline for Vision and Language
Li, L. H., Yatskar, M., Yin, D., Hsieh, C.-J., & Chang, K.-W. (2019) · arXiv · doi:10.48550/arxiv.1908.03557 · Cited by 1,198 · see citing works
Anderson, P., Wu, Q., Teney, D., Bruce, J., Johnson, M., Sunderhauf, N., Reid, I., Gould, S., & van den Hengel, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00387 · Cited by 1,187 · see citing works
Self-citationVL-BERT: Pre-training of Generic Visual-Linguistic Representations
Su, W., Zhu, X., Cao, Y., Li, B., Lu, L., Wei, F., & Dai, J. (2019) · arXiv · doi:10.48550/arxiv.1908.08530 · Cited by 950 · see citing works
MAttNet: Modular Attention Network for Referring Expression Comprehension
Yu, L., Lin, Z., Shen, X., Yang, J., Lu, X., Bansal, M., & Berg, T. L. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00142 · Cited by 857 · see citing works
Unicoder-VL: A Universal Encoder for Vision and Language by Cross-Modal Pre-Training
Li, G., Duan, N., Fang, Y., Gong, M., & Jiang, D. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6795 · Cited by 795 · see citing works
On evaluation of embodied navigation agents
(2018) · doi:10.48550/arxiv.1807.06757 · Cited by 622 · see citing works
Anticipating Human Activities Using Object Affordances for Reactive Robotic Response
Koppula, H. S., & Saxena, A. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2430335 · Cited by 596 · see citing works
Wang, X., Huang, Q., Celikyilmaz, A., Gao, J., Shen, D., Wang, Y.-F., Wang, W. Y., & Zhang, L. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00679 · Cited by 522 · see citing works
Das, A., Kottur, S., Gupta, K., Singh, A., Yadav, D., Moura, J. M. F., Parikh, D., & Batra, D. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.121 · Cited by 445 · see citing works
Das, A., Kottur, S., Gupta, K., Singh, A., Jain, D., Moura, J. M. F., Parikh, D., & Batra, D. (2017) · Cited by 432 · see citing works
12-in-1: Multi-Task Vision and Language Representation Learning
Lu, J., Goswami, V., Rohrbach, M., Parikh, D., & Lee, S. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01045 · Cited by 410 · see citing works
HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training
Li, L., Chen, Y.-C., Cheng, Y., Gan, Z., Yu, L., & Liu, J. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.161 · Cited by 408 · see citing works
TOUCHDOWN: Natural Language Navigation and Spatial Reasoning in Visual Street Environments
Chen, H., Suhr, A., Misra, D. K., Snavely, N., & Artzi, Y. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01282 · Cited by 348 · see citing works
REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environments
Qi, Y., Wu, Q., Anderson, P., Wang, X., Wang, W. Y., Shen, C., & van den Hengel, A. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01000 · Cited by 331 · see citing works
Self-citationLearning to Navigate Unseen Environments: Back Translation with Environmental Dropout
Tan, H., Yu, L., & Bansal, M. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1268 · Cited by 293 · see citing works
Speaker-Follower Models for Vision-and-Language Navigation
Fried, D., Hu, R., Cirik, V., Rohrbach, A., Andreas, J., Morency, L.-P., Berg-Kirkpatrick, T., Saenko, K., Klein, D., & Darrell, T. (2018) · arXiv · doi:10.48550/arxiv.1806.02724 · Cited by 291 · see citing works
VL-BERT: Pre-training of Generic Visual-Linguistic Representations
Su, W., Zhu, X., Cao, Y., Li, B., Lu, L., Wei, F., & Dai, J. (2020) · International Conference on Learning Representations · Cited by 276 · see citing works
Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Grounding
Ku, A., Anderson, P., Patel, R., Ie, E., & Baldridge, J. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.356 · Cited by 274 · see citing works
Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environments
Krantz, J., Wijmans, E., Majumdar, A., Batra, D., & Lee, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58604-1_7 · Cited by 266 · see citing works
Towards Learning a Generic Agent for Vision-and-Language Navigation via Pre-Training
Hao, W., Li, C., Li, X., Carin, L., & Gao, J. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01315 · Cited by 264 · see citing works
Vision-language navigation with self-supervised auxiliary reasoning tasks
Zhu, F., Zhu, Y., Chang, X., & Liang, X. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01003 · Cited by 223 · see citing works
Improving Vision-and-Language Navigation with Image-Text Pairs from the Web
Majumdar, A., Shrivastava, A., Lee, S., Anderson, P., Parikh, D., & Batra, D. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58539-6_16 · Cited by 201 · see citing works
Self-Monitoring Navigation Agent via Auxiliary Progress Estimation
Ma, C.-Y., Lu, J., Wu, Z., AlRegib, G., Kira, Z., Socher, R., & Xiong, C. (2019) · arXiv · doi:10.48550/arxiv.1901.03035 · Cited by 200 · see citing works
Furnari, A., & Farinella, G. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00635 · Cited by 182 · cited 1× · see citing works
“However, we believe that it has a huge potential in addressing other tasks which require sequential interactions/decisions, such as language and visual dialog [5, 8, 21, 25, 45], dialog navigation [9, 40, 41] and action anticipation for reactive robot response [1, 13, 20].”
BackgroundTaskmaster-1: Toward a Realistic and Diverse Dialog Dataset
Byrne, B., Krishnamoorthi, K., Sankar, C., Neelakantan, A., Goodrich, B., Duckworth, D., Yavuz, S., Dubey, A., Kim, K.-Y., & Cedilnik, A. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1459 · Cited by 174 · see citing works
Thomason, J., Murray, M., Cakmak, M., & Zettlemoyer, L. (2019) · arXiv · doi:10.48550/arxiv.1907.04957 · Cited by 173 · see citing works
Visual Question Generation as Dual Task of Visual Question Answering
Li, Y., Duan, N., Zhou, B., Chu, X., Ouyang, W., Wang, X., & Zhou, M. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00640 · Cited by 171 · see citing works
Tactical Rewind: Self-Correction via Backtracking in Vision-And-Language Navigation
Ke, L., Li, X., Bisk, Y., Holtzman, A., Gan, Z., Liu, J., Gao, J., Choi, Y., & Srinivasa, S. S. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00690 · Cited by 148 · see citing works
Nguyen, K., & Daumé III, H. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1063 · Cited by 115 · see citing works
Talk the Walk: Navigating New York City through Grounded DialoguePreprint
de Vries, H., Shuster, K., Batra, D., Parikh, D., Weston, J., & Kiela, D. (2018) · arXiv (Cornell University) · doi:10.4230/lipics.cosit.2022.24 · Cited by 112 · see citing works
BERT Representations for Video Question Answering
Yang, Z., García, N., Chu, C., Otani, M., Nakashima, Y., & Takemura, H. (2020) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv45572.2020.9093596 · Cited by 108 · see citing works
Nguyen, K., Dey, D., Brockett, C., & Dolan, B. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01281 · Cited by 101 · see citing works
Object-and-action aware model for visual language navigation
Qi, Y., Pan, Z., Zhang, S., van den Hengel, A., & Wu, Q. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58607-2_18 · Cited by 100 · see citing works
Self-citationRobust Navigation with Language Pretraining and Stochastic Sampling
Li, X., Li, C., Xia, Q., Bisk, Y., Çelikyılmaz, A., Gao, J., Smith, N. A., & Choi, Y. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1159 · Cited by 97 · see citing works
Language and Visual Entity Relationship Graph for Agent NavigationPreprint
Hong, Y., Rodríguez-Opazo, C., Qi, Y., Wu, Q., & Gould, S. J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.09304 · Cited by 97 · see citing works
Self-citationTowards Scalable Multi-Domain Conversational Agents: The Schema-Guided Dialogue Dataset
Rastogi, A., Zang, X., Sunkara, S., Gupta, R., & Khaitan, P. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6394 · Cited by 87 · see citing works
Luo, H., Ji, L., Shi, B., Huang, H., Duan, N., Li, T., Chen, X., & Zhou, M. (2020) · arXiv (Cornell University) · Cited by 82 · see citing works
Counterfactual Vision-and-Language Navigation via Adversarial Path Sampler
Fu, T.-J., Wang, X. E., Peterson, M., Grafton, S. T., Eckstein, M. P., & Wang, W. Y. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58539-6_5 · Cited by 79 · see citing works
The Dialogue Dodecathlon: Open-Domain Knowledge and Image Grounded Conversational Agents
Shuster, K., Da, J. U., Roller, S., Dinan, E., Boureau, Y.-L., & Weston, J. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.222 · Cited by 72 · see citing works
Thomason, J., Murray, M. H., Çakmak, M., & Zettlemoyer, L. (2019) · Cited by 68 · see citing works
BabyWalk: Going Farther in Vision-and-Language Navigation by Taking Baby Steps
Zhu, W., Hu, H., Chen, J., Deng, Z., Jain, V., Ie, E., & Sha, F. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.229 · Cited by 60 · see citing works
Sub-Instruction Aware Vision-and-Language Navigation
Hong, Y., Rodríguez, C., Wu, Q., & Gould, S. J. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.271 · Cited by 60 · see citing works
Self-citationMulti-Task Learning of Hierarchical Vision-Language Representation
Nguyen, D.-K., & Okatani, T. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01074 · Cited by 55 · see citing works
Environment-Agnostic Multitask Learning for Natural Language Grounded Navigation
Wang, X. E., Jain, V., Ie, E., Wang, W. Y., Kozareva, Z., & Ravi, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58586-0_25 · Cited by 54 · see citing works
CLEVR-Dialog: A Diagnostic Dataset for Multi-Round Reasoning in Visual DialogPreprint
Kottur, S., Moura, J. M. F., Parikh, D., Batra, D., & Rohrbach, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.03166 · Cited by 51 · see citing works
Soft Expert Reward Learning for Vision-and-Language Navigation
Wang, H., Wu, Q., & Shen, C. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58545-7_8 · Cited by 41 · see citing works
Self-citationVIENA $$^2$$ : A Driving Anticipation Dataset
Aliakbarian, M. S., Saleh, F. S., Salzmann, M., Fernando, B., Petersson, L., & Andersson, L. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-20887-5_28 · Cited by 40 · see citing works
OmniNet: A unified architecture for multi-modal multi-task learningPreprint
Pramanik, S., Agrawal, P., & Hussain, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.07804 · Cited by 36 · see citing works
Multi-Domain Task-Completion Dialog Challenge
Lee, S., Schulz, H., Atkinson, A., Gao, J., Suleman, K., Asri, L. E., Adada, M., minlie-huang, Sharma, S., Tay, W. W., et al. (2019) · Cited by 28 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805
Flagged first, then by citations
1 of the 69 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Multimodal learning with transformers: A survey
Xu, P., Zhu, X., & Clifton, D. A. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3275156 · Cited by 1,072 · see citing works
LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Models
Song, C. H., Sadler, B. M., Wu, J., Chao, W.-L., Washington, C., & Su, Y. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00280 · Cited by 435 · see citing works
“Recent work [11,16,21,23] employs BERT and its variants [7,20] to get better language understanding.”
MethodA Comprehensive Review of Recent Research Trends on Unmanned Aerial Vehicles (UAVs)
Telli, K., Kraa, O., Himeur, Y., Ouamane, A., Boumehraz, M., Atalla, S., & Mansoor, W. (2023) · Systems · doi:10.3390/systems11080400 · Cited by 335 · see citing works
Diffusion-based Generation, Optimization, and Planning in 3D Scenes
Huang, S., Wang, Z., Li, P., Jia, B., Liu, T., Zhu, Y., Liang, W., & Zhu, S.-C. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01607 · Cited by 192 · see citing works
Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigation
Chen, S., Guhur, P.-L., Tapaswi, M., Schmid, C., & Laptev, I. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01604 · Cited by 194 · see citing works
Vision-Language Pre-Training: Basics, Recent Advances, and Future Trends
Gan, Z., Li, L., Li, C., Wang, L., Liu, Z., & Gao, J. (2022) · Foundations and Trends® in Computer Graphics and Vision · doi:10.1561/0600000105 · Cited by 155 · see citing works
Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directions
Gu, J., Stefani, E., Wu, Q., Thomason, J., & Wang, X. (2022) · Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2022.acl-long.524 · Cited by 216 · see citing works
Self-citationETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environments
An, D., Wang, H., Wang, W., Wang, Z., Huang, Y., He, K., & Wang, L. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3386695 · Cited by 103 · see citing works
HOP: History-and-Order Aware Pretraining for Vision-and-Language Navigation
Qiao, Y., Qi, Y., Hong, Y., Yu, Z., Wang, P., & Wu, Q. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01498 · Cited by 82 · see citing works
Self-citationCross-modal Map Learning for Vision and Language Navigation
Georgakis, G., Schmeckpeper, K., Wanchoo, K., Dan, S., Miltsakaki, E., Roth, D., & Daniilidis, K. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01502 · Cited by 81 · see citing works
On Transforming Reinforcement Learning With Transformers: The Development Trajectory
Hu, S., Shen, L., Zhang, Y., Chen, Y., & Tao, D. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3408271 · Cited by 81 · see citing works
GridMM: Grid Memory Map for Vision-and-Language Navigation
Wang, Z., Li, X., Yang, J., Liu, Y., & Jiang, S. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01432 · Cited by 93 · see citing works
VELMA: Verbalization Embodiment of LLM Agents for Vision and Language Navigation in Street View
Schumann, R., Zhu, W., Feng, W., Fu, T.-J., Riezler, S., & Wang, W. Y. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29858 · Cited by 78 · see citing works
EDA: Explicit Text-Decoupling and Dense Alignment for 3D Visual Grounding
Wu, Y., Cheng, X., Zhang, R., Cheng, Z., & Zhang, J. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01843 · Cited by 73 · see citing works
Bird’s-Eye-View Scene Graph for Vision-Language Navigation
Liu, R., Wang, X., Wang, W., & Yang, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01007 · Cited by 69 · see citing works
Towards Learning a Generalist Model for Embodied Navigation
Zheng, D., Huang, S., Zhao, L., Zhong, Y., & Wang, L. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01293 · Cited by 74 · see citing works
Discuss Before Moving: Visual Language Navigation via Multi-expert Discussions
Long, Y., Li, X., Cai, W., & Dong, H. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10611565 · Cited by 73 · see citing works
Vision-language navigation: a survey and taxonomy
Wu, W., Chang, T., Li, X., Yin, Q., & Hu, Y. (2023) · Neural Computing and Applications · doi:10.1007/s00521-023-09217-1 · Cited by 65 · see citing works
KERM: Knowledge Enhanced Reasoning for Vision-and-Language Navigation
Li, X., Wang, Z., Yang, J., Wang, Y., & Jiang, S. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00254 · Cited by 61 · see citing works
Navgpt-2: Unleashing navigational reasoning capability for large vision-language models
Zhou, G., Hong, Y., Wang, Z., Wang, X. E., & Wu, Q. (2025) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-72667-5_15 · Cited by 63 · see citing works
Self-citationA survey of transformer-based multimodal pre-trained modals
Han, X., Wang, Y.-T., Feng, J.-L., Deng, C., Chen, Z.-H., Huang, Y.-A., Su, H., Hu, L., & Hu, P.-W. (2023) · Neurocomputing · doi:10.1016/j.neucom.2022.09.136 · Cited by 59 · see citing works
Scaling Data Generation in Vision-and-Language Navigation
Wang, Z., Li, J., Hong, Y., Wang, Y., Wu, Q., Bansal, M., Gould, S., Tan, H., & Qiao, Y. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01103 · Cited by 64 · see citing works
“Compared to previous approaches of addressing data scarcity by augmenting agent’s observations [47, 74] or employing large visionlinguistic models pre-trained with image-text data from the web [27, 31, 54, 71, 72], utilizing additional traversable environments allows the agents to learn from in-domain visual-language…”
Self-citationWang, H., Liang, W. G., Shen, J., Gool, L. V., & Wang, W. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01503 · Cited by 57 · see citing works
Sim-2-Sim Transfer for Vision-and-Language Navigation in Continuous Environments
Krantz, J., & Lee, S. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-19842-7_34 · Cited by 56 · see citing works
NavCoT: Boosting LLM-Based Vision-and-Language Navigation via Learning Disentangled Reasoning
Lin, B., Nie, Y., Wei, Z., Chen, J., Ma, S., Han, J., Xu, H., Chang, X., & Liang, X. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3554559 · Cited by 64 · see citing works
Computer Vision Meets Educational Robotics
Sophokleous, A., Christodoulou, P., Doitsidis, L., & Chatzichristofis, S. A. (2021) · Electronics · doi:10.3390/electronics10060730 · Cited by 55 · see citing works
SceneVerse: Scaling 3D Vision-Language Learning for Grounded Scene Understanding
Jia, B., Chen, Y., Yu, H., Wang, Y., Niu, X., Liu, T., Li, Q., & Huang, S. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72673-6_16 · Cited by 52 · see citing works
Deng, H. F., Ou, Z., Zhang, G., Deng, Y., & Tian, M. (2021) · Sensors · doi:10.3390/s21113851 · Cited by 51 · see citing works
Prompt-Based Learning for Unpaired Image Captioning
Zhu, P., Wang, X., Zhu, L., Sun, Z., Zheng, W., Wang, Y., & Chen, C. W. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3265842 · Cited by 52 · see citing works
ADAPT: Vision-Language Navigation with Modality-Aligned Action Prompts
Lin, B., Zhu, Y., Chen, Z., Liang, X., Liu, J., & Liang, X. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01496 · Cited by 52 · see citing works
Dreamwalker: Mental Planning for Continuous Vision-Language Navigation
Wang, H., Liang, W. G., Gool, L. V., & Wang, W. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00998 · Cited by 50 · see citing works
MAP: Multimodal Uncertainty-Aware Vision-Language Pre-training Model
Ji, Y., Wang, J., Gong, Y., Zhang, L., Zhu, Y., Wang, H., Zhang, J., Sakai, T., & Yang, Y. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02228 · Cited by 48 · see citing works
Vision-and-Language Navigation via Causal Learning
Wang, L., He, Z., Dang, R., Shen, M., Liu, C., & Chen, Q. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01248 · Cited by 51 · see citing works
Vision-and-Language Pretrained Models: A Survey
Long, S., Cao, F., Han, S. C., & Yang, H. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/773 · Cited by 46 · see citing works
Housekeep: Tidying Virtual Households Using Commonsense Reasoning
Kant, Y., Ramachandran, A., Yenamandra, S., Gilitschenski, I., Batra, D., Szot, A., & Agrawal, H. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19842-7_21 · Cited by 45 · see citing works
Adaptive Zone-aware Hierarchical Planner for Vision-Language Navigation
Gao, C., Peng, X., Yan, M., Wang, H., Yang, L., Ren, H., Li, H., & Liu, S. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01432 · Cited by 43 · see citing works
Learning Navigational Visual Representations with Semantic Map Supervision
Hong, Y., Zhou, Y., Zhang, R., Dernoncourt, F., Bui, T., Gould, S., & Tan, H. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00284 · Cited by 42 · see citing works
Self-citationVolumetric Environment Representation for Vision-Language Navigation
Liu, R., Wang, W., & Yang, Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01544 · Cited by 43 · see citing works
Multimodal fusion and vision–language models: A survey for robot vision
Han, X., Chen, S., Fu, Z., Feng, Z., Fan, L., An, D., Wang, C., Guo, L., Meng, W., Zhang, X., et al. (2026) · Information Fusion · doi:10.1016/j.inffus.2025.103652 · Cited by 41 · see citing works
Layout-based Causal Inference for Object Navigation
Zhang, S., Song, X., Li, W., Bai, Y., Yu, X., & Jiang, S. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01039 · Cited by 41 · see citing works
LANA: A Language-Capable Navigator for Instruction Following and Generation
Wang, X., Wang, W., Shao, J., & Yang, Y. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01826 · Cited by 37 · see citing works
Kamath, A., Anderson, P., Wang, S., Koh, J. Y., Ku, A., Waters, A., Yang, Y., Baldridge, J., & Parekh, Z. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01041 · Cited by 39 · see citing works
Wang, L., He, Z., Dang, R., Chen, H., Liu, C., & Chen, Q. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3233554 · Cited by 36 · see citing works
March in Chat: Interactive Prompting for Remote Embodied Referring Expression
Qiao, Y., Qi, Y., Zheng, Y., Liu, J., & Wu, Q. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01444 · Cited by 35 · see citing works
Self-citationLearning from Unlabeled 3D Environments for Vision-and-Language Navigation
Chen, S., Guhur, P.-L., Tapaswi, M., Schmid, C., & Laptev, I. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-19842-7_37 · Cited by 35 · see citing works
Embodied navigation with multi-modal information: A survey from tasks to methodology
Wu, Y., Zhang, P., Gu, M., Zheng, J., & Bai, X. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102532 · Cited by 34 · see citing works
Multimodal Transformer with Variable-Length Memory for Vision-and-Language Navigation
Lin, C., Jiang, Y., Cai, J., Qu, L., Haffari, G., & Yuan, Z. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20059-5_22 · Cited by 35 · see citing works
Towards Natural Language-Guided Drones: GeoText-1652 Benchmark with Spatial Relation Matching
Chu, M., Zheng, Z., Ji, W., Wang, T., & Chua, T. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73247-8_13 · Cited by 36 · see citing works
Several Categories of Large Language Models (LLMs): A Short Survey
Pahune, S. A., & Chandrasekharan, M. (2023) · International Journal for Research in Applied Science and Engineering Technology · doi:10.22214/ijraset.2023.54677 · Cited by 40 · see citing works
Language and Sketching: An LLM-driven Interactive Multimodal Multitask Robot Navigation Framework
Zu, W., Song, W., Chen, R., Guo, Z., Sun, F., Tian, Z., Pan, W., & Wang, J. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10611462 · Cited by 31 · see citing works
Learning Vision-and-Language Navigation from YouTube Videos
Lin, K., Chen, P., Huang, D.-W., Li, T. H., Tan, M., & Gan, C. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00764 · Cited by 30 · see citing works
Yue, L., Zhou, D., Xie, L., Zhang, F., Yan, Y., & Yin, E. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3387171 · Cited by 27 · see citing works
Improving Vision-and-Language Navigation by Generating Future-View Image Semantics
Li, J., & Bansal, M. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01040 · Cited by 28 · see citing works
Room-Object Entity Prompting and Reasoning for Embodied Referring Expression
Gao, C., Liu, S., Chen, J., Wang, L., Wu, Q., Li, B. H., & Tian, Q. C. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3326851 · Cited by 26 · see citing works
Self-citationShen, K., & Kejriwal, M. (2023) · Expert Syst. J. Knowl. Eng. · doi:10.1111/exsy.13243 · Cited by 19 · see citing works
Memory-Adaptive Vision-and-Language Navigation
He, K., Jing, Y., Huang, Y., Lu, Z., An, D. U., & Wang, L. (2024) · Pattern Recognition · doi:10.1016/j.patcog.2024.110511 · Cited by 26 · see citing works
Visual-Language Navigation Pretraining via Prompt-based Environmental Self-exploration
Liang, X., Zhu, F., Lingling, L., Xu, H., & Liang, X. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.332 · Cited by 27 · see citing works
Classification of Arabic Poetry Emotions Using Deep Learning
Shahriar, S., Roken, N. A., & Zualkernan, I. A. (2023) · Computers · doi:10.3390/computers12050089 · Cited by 23 · see citing works
Probabilistic Prompt Learning for Dense Prediction
Kwon, H., Song, T., Jeong, S., Kim, J., Jang, J., & Sohn, K. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00654 · Cited by 23 · see citing works
Self-Supervised 3-D Semantic Representation Learning for Vision-and-Language Navigation
Tan, S., Sima, K., Wang, D., Ge, M., Guo, D., & Liu, H. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3395633 · Cited by 24 · see citing works
Hwang, M., Jeong, J., Kim, M., Oh, Y., & Oh, S. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00646 · Cited by 26 · see citing works
Huo, J., Sun, Q., Jiang, B., Lin, H., & Fu, Y. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02223 · Cited by 24 · see citing works
Correctable Landmark Discovery via Large Models for Vision-Language Navigation
Lin, B., Nie, Y., Wei, Z., Zhu, Y., Xu, H., Ma, S., Liu, J., & Liang, X. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3407759 · Cited by 21 · see citing works
Tem-adapter: Adapting Image-Text Pretraining for Video Question Answer
Chen, G., Liu, X., Wang, G., Zhang, K., Torr, P. H. S., Zhang, X.-P. S., & Tang, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01282 · Cited by 21 · see citing works
Augmented Commonsense Knowledge for Remote Object Grounding
Mohammadi, B., Hong, Y., Qi, Y., Wu, Q., Pan, S., & Shi, Q. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i5.28223 · Cited by 21 · see citing works
Self-citationA Dual Semantic-Aware Recurrent Global-Adaptive Network for Vision-and-Language Navigation
Wang, L., He, Z., Tang, J., Dang, R., Wang, N., Liu, C., & Chen, Q. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/164 · Cited by 23 · see citing works
VLN-PETL: Parameter-Efficient Transfer Learning for Vision-and-Language Navigation
Qiao, Y., Zheng, Y. H., & Wu, Q. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01416 · Cited by 20 · see citing works
Self-citationSpectral Prompt Tuning: Unveiling Unseen Classes for Zero-Shot Semantic Segmentation
Xu, W., Xu, R., Wang, C., Xu, S., Guo, L., Zhang, M., & Zhang, X. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i6.28456 · Cited by 20 · see citing works
Grounded Entity-Landmark Adaptive Pre-training for Vision-and-Language Navigation
Cui, Y., Xie, L., Zhang, Y., Zhang, M., Yan, Y., & Yin, E. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01106 · Cited by 19 · see citing works
UAV Swarm Objectives: A Critical Analysis and Comprehensive Review
Kumar, P. A., Manoj, N., Sudheer, N., Bhat, P. P., Arya, A., & Sharma, R. (2024) · SN Computer Science · doi:10.1007/s42979-024-03156-x · Cited by 18 · see citing works
Boosting Efficient Reinforcement Learning for Vision-and-Language Navigation With Open-Sourced LLM
Wang, J., Wang, T., Cai, W., Xu, L., & Sun, C. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3511402 · Cited by 19 · see citing works
Echo-Enhanced Embodied Visual Navigation
Yu, Y., Cao, L., Sun, F., Yang, C., Lai, H., & Huang, W. (2023) · Neural Computation · doi:10.1162/neco_a_01579 · Cited by 18 · see citing works
Vision-and-Language Navigation via Latent Semantic Alignment Learning
Wu, S., Fu, X., Wu, F. H., & Zha, Z.-J. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2024.3358112 · Cited by 16 · see citing works
Enhancing Vision and Language Navigation With Prompt-Based Scene Knowledge
Zhan, Z., Qin, J., Zhuo, W., & Tan, G. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3401451 · Cited by 16 · see citing works
Hu, Y., Zhou, Y., Zhu, Z., Yang, X., Zhang, H., Bian, K., & Han, H. W. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.114190 · Cited by 17 · see citing works
Constraint-Aware Zero-Shot Vision-Language Navigation in Continuous Environments
Chen, K., An, D. U., Huang, Y., Xu, R., Su, Y., Ling, Y., Reid, I. D., & Wang, L. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3594204 · Cited by 16 · see citing works
Cross-Modality Pyramid Alignment for Visual Intention Understanding
Ye, M., Shi, Q., Su, K., & Du, B. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3261743 · Cited by 17 · see citing works
ESceme: Vision-and-Language Navigation with Episodic Scene Memory
Zheng, Q., Liu, D., Wang, C., Zhang, J., Wang, D., & Tao, D. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-024-02159-8 · Cited by 15 · see citing works
PASTS: Progress-aware spatio-temporal transformer speaker for vision-and-language navigation
Wang, L., Liu, C., He, Z., Li, S., Yan, Q., Chen, H., & Chen, Q. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.107487 · Cited by 16 · see citing works
Deep Learning for Embodied Vision Navigation: A SurveyPreprint
Zhu, F., Zhu, Y., Lee, V. C., Liang, X., & Chang, X. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.04097 · Cited by 19 · see citing works
Watson, E. ‘., Viana, T., & Zhang, S. (2023) · AI · doi:10.3390/ai4010007 · Cited by 13 · see citing works
VLN-Trans: Translator for the Vision and Language Navigation Agent
Zhang, Y., & Kordjamshidi, P. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.737 · Cited by 13 · see citing works
WebVLN: Vision-and-Language Navigation on Websites
Chen, Q., Pitawela, D., Zhao, C. K., Zhou, G., Chen, H., & Wu, Q. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i2.27878 · Cited by 13 · see citing works
Self-citationQuestion-conditioned debiasing with focal visual context fusion for visual question answering
Liu, J., Wang, G., Fan, C., Zhou, F., & Xu, H. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.110879 · Cited by 13 · see citing works
Omnidirectional Information Gathering for Knowledge Transfer-based Audio-Visual Navigation
Chen, J., Wang, W., Liu, S., Li, H., & Yang, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01009 · Cited by 12 · see citing works
Aggregating Global and Local Representations via Hybrid Transformer for Video Deraining
Mao, D., Gao, S., Li, Z., Dai, H., Zhang, Y., & Zhou, Y. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3372944 · Cited by 12 · see citing works
DAP: Domain-Aware Prompt Learning for Vision-and-Language Navigation
Liu, T., Hu, Y., Wu, W., Wang, Y.-K., Xu, K., & Yin, Q. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10446504 · Cited by 12 · see citing works
Wang, S., Zhou, D., Xie, L., Xu, C., Yan, Y., & Yin, E. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.107320 · Cited by 11 · see citing works
To Boost Zero-Shot Generalization for Embodied Reasoning With Vision-Language Pre-Training
Su, K., Zhang, X., Zhang, S., Zhu, J., & Zhang, B. (2024) · IEEE Transactions on Image Processing · doi:10.1109/tip.2024.3459800 · Cited by 11 · see citing works
Video Dialog as Conversation About Objects Living in Space-Time
Pham, H.-A., Le, T. M., Le, V., Phuong, T. M., & Tran, T. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19842-7_41 · Cited by 11 · see citing works
MLNet: a multi-level multimodal named entity recognition architecture
Zhai, H., Lv, X., Hou, Z., Tong, X., & Bu, F. (2023) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2023.1181143 · Cited by 10 · see citing works
Enhancing Scene Understanding for Vision-and-Language Navigation by Knowledge Awareness
Gao, F., Tang, J., Wang, J., Li, S., & Yu, J. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3483042 · Cited by 10 · see citing works
Adaptive Cross-Modal Experts Network with Uncertainty-Driven Fusion for Vision–Language Navigation
Wu, J., Wu, C., Shen, X., & Wang, L. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112735 · Cited by 11 · see citing works
Source-Free Elastic Model Adaptation for Vision-and-Language Navigation
Tan, M., Chen, P., Zhi, H., Mai, J., Rosman, B., Ji, D., & Zeng, R. (2025) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2025.3535356 · Cited by 10 · see citing works
A Dehazing Method for UAV Remote Sensing Based on Global and Local Feature Collaboration
Li, C., Zhou, S., Wu, T., Shi, J., & Guo, F. (2025) · Remote Sensing · doi:10.3390/rs17101688 · Cited by 10 · see citing works
Learning Disentanglement with Decoupled Labels for Vision-Language Navigation
Cheng, W., Dong, X., Khan, S. H., & Shen, J. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20059-5_18 · Cited by 10 · see citing works
Tina: Think, Interaction, and Action Framework for Zero-Shot Vision Language Navigation
Li, D., Chen, W., & Lin, X. A. (2024) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme57554.2024.10687470 · Cited by 9 · see citing works
Wen, S., Gong, S., Zhang, Z., Yu, F. R., & Wang, Z. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112610 · Cited by 9 · see citing works
Enhancing Large Language Models with RAG for Visual Language Navigation in Continuous Environments
Bao, X., Lv, Z., & Wu, B. (2025) · Electronics · doi:10.3390/electronics14050909 · Cited by 9 · see citing works
AeroDuo: Aerial Duo for UAV-based Vision and Language Navigation
Wu, R., Zhang, Y., Chen, J., Huang, L., Zhang, S., Xu, Z. Y., Wang, L., & Liu, S. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3754498 · Cited by 10 · see citing works
- Authors
- Hong, Y., Wu, Q., Qi, Y., Rodriguez-Opazo, C., & Gould, S.
- Affiliations
- Australian National University (AU)
- Australian Centre for Robotic Vision (AU)
- Venue
- 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2021-06
- Pages
- 1643-1653
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W3172675210
- Cited by
- 264
- References
- 69
- Citations by year
- 2026: 46 · 2025: 60 · 2024: 63 · 2023: 62 · 2022: 27 · 2021: 6
- FWCI
- 12.92
- Type
- conference paper
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- ANU Open Research (Australian National University) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
- Keywords
- vision-and-language navigation, partially observable Markov decision processes, pre-training, transformer models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
