THDA: Treasure Hunt Data Augmentation for Semantic Navigation
The DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.
Abstract
Can general-purpose neural models learn to navigate? For PointGoal navigation (‘go to Δx, Δy’), the answer is a clear ‘yes’ – mapless neural models composed of task-agnostic components (CNNs and RNNs) trained with large-scale model-free reinforcement learning achieve near-perfect performance [27]. However, for ObjectGoal navigation (‘find a TV’), this is an open question; one we tackle in this paper. The current best-known result on ObjectNav with general-purpose models is 6% success rate [25].First, we show that the key problem is overfitting. Large-scale training results in 94% success rate on training environments and only 8% in validation. We observe that this stems from agents memorizing environment layouts during training – sidestepping the need for exploration and directly learning shortest paths to nearby goal objects. We show that this is a natural consequence of optimizing for the task metric (which in fact penalizes exploration), is enabled by powerful observation encoders, and is possible due to the finite set of training environment configurations.Informed by our findings, we introduce Treasure Hunt Data Augmentation (THDA) to address overfitting in ObjectNav. THDA inserts 3D scans of household objects at arbitrary scene locations and uses them as ObjectNav goals – augmenting and greatly expanding the set of training layouts. Taken together with our other proposed changes, we improve the state of art on the Habitat ObjectGoal Navigation benchmark by 90% (from 14% success rate to 27%) and path efficiency by 48% (from 7.5 SPL to 11.1 SPL).
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
SUN RGB-D: A RGB-D scene understanding benchmark suite
Song, S., Lichtenberg, S. P., & Xiao, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298655 · Cited by 1,943 · cited 1×
“We fine-tune RedNet using publicly available pre-trained weights (learned on the SUN-RGBD dataset [23]) on our dataset.”
MethodMatterport3D: Learning from RGB-D Data in Indoor Environments
Chang, A., Dai, A., Funkhouser, T., Halber, M., Niebner, M., Savva, M., Song, S., Zeng, A., & Zhang, Y. (2017) · 2017 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2017.00081 · Cited by 1,858
Habitat: A Platform for Embodied AI Research
Savva, M., Kadian, A., Maksymets, O., Zhao, Y., Wijmans, E., Jain, B., Straub, J., Liu, J., Koltun, V., Malik, J., et al. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00943 · Cited by 1,237 · cited 5×
“This reward works well for PointGoal Navigation and thus it has also been adapted to ObjectNav by calculating the geodesic distance with respect to the closest object [9, 20].”
BackgroundMethodInfluentialSelf-citationThe YCB object and model set: Towards common benchmarks for manipulation research
Calli, B., Singh, A., Walsman, A., Srinivasa, S., Abbeel, P., & Dollar, A. M. (2015) · 2015 International Conference on Advanced Robotics (ICAR) · doi:10.1109/icar.2015.7251504 · Cited by 838 · cited 2×
“We use the YCB dataset objects [4] as inserted objects.”
MethodGibson Env: Real-World Perception for Embodied Agents
Xia, F., Zamir, A. R., He, Z., Sax, A., Malik, J., & Savarese, S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00945 · Cited by 763
AI2-THOR: An Interactive 3D Environment for Visual AI
Kolve, E., Mottaghi, R., Han, W., VanderBilt, E., Weihs, L., Herrasti, A., Deitke, M., Ehsani, K., Gordon, D., Zhu, Y., et al. (2017) · arXiv · doi:10.48550/arxiv.1712.05474 · Cited by 578 · cited 1×
“Photorealistic 3D indoor environments [29, 7, 18] are well suited for ObjectNav because they capture the distribution of layouts of real houses with a high-quality visual rendering, which might be needed for sim2real transfer [3].”
BackgroundObject Goal Navigation using Goal-Oriented Semantic Exploration
Chaplot, D. S., Gandhi, D., Gupta, A., & Salakhutdinov, R. (2020) · arXiv · doi:10.48550/arxiv.2007.00643 · Cited by 321
Learning to explore using active neural SLAM
Chaplot, D. S., Gandhi, D., Gupta, S., Gupta, A., & Salakhutdinov, R. (2020) · arXiv · doi:10.48550/arxiv.2004.05155 · Cited by 318
Cited by
Simple but Effective: CLIP Embeddings for Embodied AI
Khandelwal, A., Weihs, L., Mottaghi, R., & Kembhavi, A. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01441 · Cited by 172
“augmenting scenes with 3D object scans [19], predicting semantic segmentations as additional inputs [37], and crowdsourcing human demonstrations for imitation learning [24]).”
BackgroundPONI: Potential Functions for ObjectGoal Navigation with Interaction-free Learning
Ramakrishnan, S. K., Chaplot, D. S., Al-Halah, Z., Malik, J., & Grauman, K. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01832 · Cited by 154
“Recent work on end-to-end RL for ObjectNav has proposed improved visual representations [40, 62], auxiliary tasks [63], and data augmentation techniques [37] to improve generalization to novel scenes.”
BackgroundMethodInfluentialNavigating to objects in the real world
Gervet, T., Chintala, S., Batra, D., Malik, J., & Chaplot, D. S. (2023) · Science Robotics · doi:10.1126/scirobotics.adf6991 · Cited by 125
“Most relevant to us, [41, 42, 43] propose end-to-end policies to navigate to object goals.”
BackgroundSelf-citationHabitat-Web: Learning Embodied Object-Search Strategies from Human Demonstrations at Scale
Ramrakhya, R., Undersander, E., Batra, D., & Das, A. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00511 · Cited by 90
Self-citationKhanna, M., Mao, Y., Jiang, H., Haresh, S., Shacklett, B., Batra, D., Clegg, A., Undersander, E., Chang, A. X., & Savva, M. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01550 · Cited by 63
Self-citation
Metadata
- Authors
- Maksymets, O., Cartillier, V., Gokaslan, A., Wijmans, E., Galuba, W., Lee, S., & Batra, D.
- Venue
- 2021 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2021-10
- Pages
- 15354-15363
- OpenAlex
- W3210395242
- Cited by
- 65
- References
- 12
- FWCI
- 5.06
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: THDA: Treasure Hunt Data Augmentation for Semantic Navigation. https://app.cochat.ai/papers/10.1109/iccv48922.2021.01509
- The paper
- Maksymets, O., Cartillier, V., Gokaslan, A., Wijmans, E., Galuba, W., Lee, S., & Batra, D. (2021). THDA: Treasure Hunt Data Augmentation for Semantic Navigation. 2021 IEEE/CVF International Conference on Computer Vision (ICCV). https://doi.org/10.1109/iccv48922.2021.01509
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.