CoChat
Sign Up
Cited by 140Multimodal Machine Learning ApplicationsReinforcement Learning in RoboticsRobot Manipulation and Learning

Airbert: In-domain Pretraining for Vision-and-Language Navigation

Open access
Guhur, Pierre-Louis, Tapaswi, Makarand, Chen, Shizhe, Laptev, Ivan, Schmid, Cordelia·2021·2021 IEEE/CVF International Conference on Computer Vision (ICCV)
DOI 10.1109/iccv48922.2021.00166
Open this paper in CoChat — free accountOpen PDF

Abstract

Vision-and-language navigation (VLN) aims to enable embodied agents to navigate in realistic environments using natural language instructions. Given the scarcity of domain-specific training data and the high diversity of image and language inputs, the generalization of VLN agents to unseen environments remains challenging. Recent methods explore pretraining to improve generalization, however, the use of generic image-caption datasets or existing small-scale VLN environments is suboptimal and results in limited improvements. In this work, we introduce BnB1, a large-scale and diverse in-domain VLN dataset. We first collect image-caption (IC) pairs from hundreds of thousands of listings from online rental marketplaces. Using IC pairs we next propose automatic strategies to generate millions of VLN path-instruction (PI) pairs. We further propose a shuffling loss that improves the learning of temporal order inside PI pairs. We use BnB to pretrain our Airbert2model that can be adapted to discriminative and generative settings and show that it outperforms state of the art for Room-to-Room (R2R) navigation and Remote Referring Expression (REVERIE) benchmarks. Moreover, our in-domain pretraining significantly increases performance on a challenging few-shot VLN evaluation, where we train the model only on VLN instructions from a few houses.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/iccv48922.2021.00166

Cites

53 of 53 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • GemBench, built on RLBench, focuses on task generalization with zero-shot setups and vision-language extensions to assess performance on previously unseen tasks and objects.

    “In contrast, GemBench [289], built upon RLBench, focuses on task generalization, incorporating zero-shot task setups and vision-language extensions to assess robotic performance in previously unseen tasks and objects.”

    From Multimodal fusion and vision–language models: A survey for robot vision (2026) · cites this paper as [289]

    The Introduction and Section 3 describe Airbert and its BnB dataset for vision-and-language navigation. Section 5 evaluates navigation on R2R and REVERIE, not GemBench. Searches for GemBench, RLBench, zero-shot task setups, and unseen robotic tasks or objects found no passage addressing the attributed description. (LLM, conf=0.78)

    Experimental — accuracy not yet measured

  • GemBench has advantages in multimodal fusion and generalization but remains simulation-constrained and lacks real-world validation.

    “While GemBench [289] presents unique advantages in multimodal fusion and generalization, it also remains constrained to simulation and lacks real-world validation.”

    From Multimodal fusion and vision–language models: A survey for robot vision (2026) · cites this paper as [289]

    The Experimental Results (Section 5) and Conclusion (Section 6) discuss Airbert’s performance on vision-and-language navigation tasks, not GemBench. A search for GemBench and for simulation or real-world validation found no passage in our copy assessing GemBench’s advantages or limitations. (LLM, conf=0.78)

    Experimental — accuracy not yet measured

  • Unlike VLMbench and GemBench, VIMA integrates multi-modal task execution and precise instruction following.

    “Unlike VLMbench [290] and Gem-Bench [289], which primarily evaluate vision-language models (VLMs) in robotic control, VIMA [291] further integrates multi-modal task execution and precise instruction following, making it distinct in its benchmarking approach.”

    From Multimodal fusion and vision–language models: A survey for robot vision (2026) · cites this paper as [289]

    The Introduction describes Airbert as a model for vision-and-language navigation, and Figure 3 describes its path-instruction pretraining. Searches for VIMA, VLMbench, Gem-Bench, and the claimed benchmarking distinction found no passage in our copy addressing them. (LLM, conf=0.76)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

2 of the 55 references couldn’t be matched to a known paper.

Research works printed but not linked: 2

  1. [16]Haoshuo Huang, Vihan Jain, Harsh Mehta, Alexander Ku, Gabriel Magalhaes, Jason Baldridge, and Eugene Ie. Trans- ferable representation learning in vision-and-language navi- gation. In ICCV, 2019. 1, 2, 4
  2. [43]Oriol Vinyals, Meire Fortunato, and Navdeep Jaitly. Pointer networks. arXiv preprint arXiv:1506.03134, 2015. 12 · arXiv

Cited by

Sorted by citations

Related

Authors
Guhur, P.-L., Tapaswi, M., Chen, S., Laptev, I., & Schmid, C.
Affiliations
  • Centre National de la Recherche Scientifique (FR)
  • International Institute of Information Technology, Hyderabad (IN)
  • PSL Research University,Inria, École normale supérieure, CNRS,Paris,France
ORCID
Venue
2021 IEEE/CVF International Conference on Computer Vision (ICCV)
Publisher
IEEE
Published
2021-10
Pages
1614-1623
DOI
10.1109/iccv48922.2021.00166
OpenAlex
W3195026654
Cited by
139
References
55
Citations by year
2026: 11 · 2025: 39 · 2024: 38 · 2023: 29 · 2022: 22
FWCI
7.28
Type
conference paper
Language
en
Open access
Licence
Locations
  • IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
Funders
Agence Nationale de la Recherche (ANR-19-P3IA-0001)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
  • Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
Keywords
vision-and-language navigation, few-shot generalization, embodied agents

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review