The DOI matches its Crossref record; 54 of the 55 references it cites were found in a registry or an index.
Airbert: In-domain Pretraining for Vision-and-Language Navigation
Open accessAbstract
Vision-and-language navigation (VLN) aims to enable embodied agents to navigate in realistic environments using natural language instructions. Given the scarcity of domain-specific training data and the high diversity of image and language inputs, the generalization of VLN agents to unseen environments remains challenging. Recent methods explore pretraining to improve generalization, however, the use of generic image-caption datasets or existing small-scale VLN environments is suboptimal and results in limited improvements. In this work, we introduce BnB1, a large-scale and diverse in-domain VLN dataset. We first collect image-caption (IC) pairs from hundreds of thousands of listings from online rental marketplaces. Using IC pairs we next propose automatic strategies to generate millions of VLN path-instruction (PI) pairs. We further propose a shuffling loss that improves the learning of temporal order inside PI pairs. We use BnB to pretrain our Airbert2model that can be adapted to discriminative and generative settings and show that it outperforms state of the art for Room-to-Room (R2R) navigation and Remote Referring Expression (REVERIE) benchmarks. Moreover, our in-domain pretraining significantly increases performance on a challenging few-shot VLN evaluation, where we train the model only on VLN instructions from a few houses.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“In contrast, GemBench [289], built upon RLBench, focuses on task generalization, incorporating zero-shot task setups and vision-language extensions to assess robotic performance in previously unseen tasks and objects.”
From Multimodal fusion and vision–language models: A survey for robot vision (2026) · cites this paper as [289]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“While GemBench [289] presents unique advantages in multimodal fusion and generalization, it also remains constrained to simulation and lacks real-world validation.”
From Multimodal fusion and vision–language models: A survey for robot vision (2026) · cites this paper as [289]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Unlike VLMbench [290] and Gem-Bench [289], which primarily evaluate vision-language models (VLMs) in robotic control, VIMA [291] further integrates multi-modal task execution and precise instruction following, making it distinct in its benchmarking approach.”
From Multimodal fusion and vision–language models: A survey for robot vision (2026) · cites this paper as [289]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Jakubův, J., Chvalovský, K., Goertzel, Z., et al. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,648
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293
Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering
Anderson, P., He, X., Buehler, C., Teney, D., Johnson, M., Gould, S. J., & Zhang, L. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00636 · Cited by 5,416
Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations
Krishna, R., Zhu, Y., Groth, O., Johnson, J., Hata, K., Kravitz, J., Chen, S., Kalantidis, Y., Li, L.-J., Shamma, D. A., et al. (2017) · International Journal of Computer Vision · doi:10.1007/s11263-016-0981-7 · Cited by 5,107
Self-citationVQA: Visual Question Answering
Antol, S., Agrawal, A., Lu, J., Mitchell, M., Batra, D., Zitnick, C. L., & Parikh, D. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.279 · Cited by 4,487
Places: A 10 Million Image Database for Scene Recognition
Zhou, B., Lapedriza, A., Khosla, A., Oliva, A., & Torralba, A. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2723009 · Cited by 4,042
Pointer NetworksPreprint
Vinyals, O., Fortunato, M., & Jaitly, N. (2015) · Neural Information Processing Systems · doi:10.48550/arxiv.1506.03134 · Cited by 3,824
LXMERT: Learning Cross-Modality Encoder Representations from Transformers
Tan, H., & Bansal, M. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1514 · Cited by 2,319
ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks
Lu, J., Batra, D., Parikh, D., & Lee, S. (2019) · arXiv · doi:10.48550/arxiv.1908.02265 · Cited by 2,109
UNITER: UNiversal Image-TExt Representation Learning
Chen, Y.-C., Li, L., Yu, L., El Kholy, A., Ahmed, F., Gan, Z., Cheng, Y., & Liu, J. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58577-8_7 · Cited by 2,055
Conceptual Captions: A Cleaned, Hypernymed, Image Alt-text Dataset For Automatic Image Captioning
Sharma, P., Ding, N., Goodman, S., & Soricut, R. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1238 · Cited by 1,879
Matterport3D: Learning from RGB-D Data in Indoor Environments
Chang, A., Dai, A., Funkhouser, T., Halber, M., Niebner, M., Savva, M., Song, S., Zeng, A., & Zhang, Y. (2017) · 2017 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2017.00081 · Cited by 1,858
Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks
Li, X., Yin, X. P., Li, C., Zhang, P., Hu, X., Zhang, L., Wang, L., Hu, H., Li, D., Wei, F., et al. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58577-8_8 · Cited by 1,791
Anderson, P., Wu, Q., Teney, D., Bruce, J., Johnson, M., Sunderhauf, N., Reid, I., Gould, S., & van den Hengel, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00387 · Cited by 1,187
ReferItGame: Referring to Objects in Photographs of Natural Scenes
Kazemzadeh, S., Ordóñez, V., Matten, M., & Berg, T. L. (2014) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1086 · Cited by 1,082
HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips
Miech, A., Zhukov, D., Alayrac, J.-B., Tapaswi, M., Laptev, I., & Sivic, J. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00272 · Cited by 986
Self-citationVL-BERT: Pre-training of Generic Visual-Linguistic Representations
Su, W., Zhu, X., Cao, Y., Li, B., Lu, L., Wei, F., & Dai, J. (2019) · arXiv · doi:10.48550/arxiv.1908.08530 · Cited by 950
Im2Text: Describing Images Using 1 Million Captioned Photographs
Ordóñez, V., Kulkarni, G. A., & Berg, T. L. (2011) · Cited by 879
Show and Tell: Lessons Learned from the 2015 MSCOCO Image Captioning Challenge
Vinyals, O., Toshev, A., Bengio, S., & Erhan, D. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2587640 · Cited by 875
Learning Deep Structure-Preserving Image-Text Embeddings
Wang, L., Li, Y., & Lazebnik, S. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.541 · Cited by 795
On evaluation of embodied navigation agents
Anderson, P., Chang, A., Chaplot, D. S., Dosovitskiy, A., Gupta, S., Koltun, V., Kosecka, J., Malik, J., Mottaghi, R., Savva, M., et al. (2018) · arXiv · doi:10.48550/arxiv.1807.06757 · Cited by 622
ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Tasks
Shridhar, M., Thomason, J., Gordon, D., Bisk, Y., Han, W., Mottaghi, R., Zettlemoyer, L., & Fox, D. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01075 · Cited by 540
Wang, X., Huang, Q., Celikyilmaz, A., Gao, J., Shen, D., Wang, Y.-F., Wang, W. Y., & Zhang, L. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00679 · Cited by 522
12-in-1: Multi-Task Vision and Language Representation Learning
Lu, J., Goswami, V., Rohrbach, M., Parikh, D., & Lee, S. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01045 · Cited by 420
TOUCHDOWN: Natural Language Navigation and Spatial Reasoning in Visual Street Environments
Chen, H., Suhr, A., Misra, D. K., Snavely, N., & Artzi, Y. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01282 · Cited by 371
REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environments
Qi, Y., Wu, Q., Anderson, P., Wang, X., Wang, W. Y., Shen, C., & van den Hengel, A. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01000 · Cited by 331
Learning to Navigate Unseen Environments: Back Translation with Environmental Dropout
Tan, H., Yu, L., & Bansal, M. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1268 · Cited by 293
Speaker-Follower Models for Vision-and-Language Navigation
Fried, D., Hu, R., Cirik, V., Rohrbach, A., Andreas, J., Morency, L.-P., Berg-Kirkpatrick, T., Saenko, K., Klein, D., & Darrell, T. (2018) · arXiv · doi:10.48550/arxiv.1806.02724 · Cited by 291
VL-BERT: Pre-training of Generic Visual-Linguistic Representations
Su, W., Zhu, X., Cao, Y., Li, B., Lu, L., Wei, F., & Dai, J. (2020) · International Conference on Learning Representations · Cited by 276
Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Grounding
Ku, A., Anderson, P., Patel, R., Ie, E., & Baldridge, J. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.356 · Cited by 274
Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environments
Krantz, J., Wijmans, E., Majumdar, A., Batra, D., & Lee, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58604-1_7 · Cited by 266
Towards Learning a Generic Agent for Vision-and-Language Navigation via Pre-Training
Hao, W., Li, C., Li, X., Carin, L., & Gao, J. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01315 · Cited by 264
Vision-language navigation with self-supervised auxiliary reasoning tasks
Zhu, F., Zhu, Y., Chang, X., & Liang, X. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01003 · Cited by 223
Improving Vision-and-Language Navigation with Image-Text Pairs from the Web
Majumdar, A., Shrivastava, A., Lee, S., Anderson, P., Parikh, D., & Batra, D. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58539-6_16 · Cited by 201
Self-Monitoring Navigation Agent via Auxiliary Progress Estimation
Ma, C.-Y., Lu, J., Wu, Z., AlRegib, G., Kira, Z., Socher, R., & Xiong, C. (2019) · arXiv · doi:10.48550/arxiv.1901.03035 · Cited by 200
Fusion of Detected Objects in Text for Visual Question Answering
Alberti, C., Ling, J., Collins, M., & Reitter, D. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1219 · Cited by 188
Wang, X. E., Xiong, W., Wang, H., & Wang, W. Y. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01270-0_3 · Cited by 187
Thomason, J., Murray, M., Cakmak, M., & Zettlemoyer, L. (2019) · arXiv · doi:10.48550/arxiv.1907.04957 · Cited by 173
The Regretful Agent: Heuristic-Aided Navigation Through Progress Estimation
Ma, C., Wu, Z., AlRegib, G., Xiong, C., & Kira, Z. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00689 · Cited by 166
Contrastive Learning for Weakly Supervised Phrase Grounding
Gupta, T., Vahdat, A., Chechik, G., Yang, X., Kautz, J., & Hoiem, D. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58580-8_44 · Cited by 125
Robust Navigation with Language Pretraining and Stochastic Sampling
Li, X., Li, C., Xia, Q., Bisk, Y., Çelikyılmaz, A., Gao, J., Smith, N. A., & Choi, Y. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1159 · Cited by 116
Nguyen, K., & Daumé III, H. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1063 · Cited by 115
Nguyen, K., Dey, D., Brockett, C., & Dolan, B. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01281 · Cited by 112
Extending a Parser to Distant Domains Using a Few Dozen Partially Annotated Examples
Joshi, V., Peters, M. E., & Hopkins, M. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-1110 · Cited by 109
Object-and-action aware model for visual language navigation
Qi, Y., Pan, Z., Zhang, S., van den Hengel, A., & Wu, Q. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58607-2_18 · Cited by 100
Transferable Representation Learning in Vision-and-Language Navigation
Huang, H., Jain, V., Mehta, H., Ku, A., Magalhaes, G., Baldridge, J., & Ie, E. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00750 · Cited by 82
Thomason, J., Murray, M. H., Çakmak, M., & Zettlemoyer, L. (2019) · Cited by 68
Sub-Instruction Aware Vision-and-Language Navigation
Hong, Y., Rodríguez, C., Wu, Q., & Gould, S. J. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.271 · Cited by 60
Diagnosing the Environment Bias in Vision-and-Language Navigation
Zhang, Y., Tan, H., & Bansal, M. (2020) · Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/124 · Cited by 51
Soft Expert Reward Learning for Vision-and-Language Navigation
Wang, H., Wu, Q., & Shen, C. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58545-7_8 · Cited by 41
A Recurrent Vision-and-Language BERT for NavigationPreprint
Hong, Y., Wu, Q., Qi, Y., Rodríguez-Opazo, C., & Gould, S. J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.13922 · Cited by 38
Cited by
Sorted by citations
Multimodal learning with transformers: A survey
Xu, P., Zhu, X., & Clifton, D. A. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3275156 · Cited by 1,072
Visual Language Maps for Robot Navigation
Huang, C., Mees, O., Zeng, A., & Burgard, W. (2023) · 2023 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra48891.2023.10160969 · Cited by 376
Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigation
Chen, S., Guhur, P.-L., Tapaswi, M., Schmid, C., & Laptev, I. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01604 · Cited by 228
Self-citationVision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directions
Gu, J., Stefani, E., Wu, Q., Thomason, J., & Wang, X. E. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.524 · Cited by 169
ETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environments
An, D., Wang, H., Wang, W., Wang, Z., Huang, Y., He, K., & Wang, L. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3386695 · Cited by 103
Metadata
- Authors
- Guhur, P.-L., Tapaswi, M., Chen, S., Laptev, I., & Schmid, C.
- Affiliations
- Centre National de la Recherche Scientifique (FR)
- International Institute of Information Technology, Hyderabad (IN)
- PSL Research University,Inria, École normale supérieure, CNRS,Paris,France
- Venue
- 2021 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2021-10
- Pages
- 1614-1623
- OpenAlex
- W3195026654
- Cited by
- 139
- References
- 55
- Citations by year
- 2026: 11 · 2025: 39 · 2024: 38 · 2023: 29 · 2022: 22
- FWCI
- 7.28
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Funders
- Agence Nationale de la Recherche (ANR-19-P3IA-0001)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
- Keywords
- vision-and-language navigation, few-shot generalization, embodied agents
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
