SplitNet: Sim2Sim and Task2Task Transfer for Embodied Visual Navigation
Open accessAbstract
We propose SplitNet, a method for decoupling visual perception and policy learning. By incorporating auxiliary tasks and selective learning of portions of the model, we explicitly decompose the learning objectives for visual navigation into perceiving the world and acting on that perception. We show improvements over baseline models on transferring between simulators, an encouraging step towards Sim2Real. Additionally, SplitNet generalizes better to unseen environments from the same simulator and transfers faster and more effectively to novel embodied navigation tasks. Further, given only a small sample from a target domain, SplitNet can match the performance of traditional end-to-end pipelines which receive the entire dataset1.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Policies learned in simulation often transfer poorly to reality because agents overfit to visual specifics of the simulator.
“Most policies trained in simulation generalize poorly to the real-world due to agents adapting to specific visual details of the simulator.”
From Language-Based Augmentation to Address Shortcut Learning in Object-Goal Navigation (2023) · cites this paper as [25]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · see citing works
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834 · see citing works
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684 · see citing works
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545 · see citing works
Unpaired image-to-image translation using cycle-consistent adversarial networks
Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.244 · Cited by 23,602 · see citing works
Unsupervised representation learning with deep convolutional generative adversarial networks
Radford, A., Metz, L., & Chintala, S. (2015) · arXiv · doi:10.48550/arxiv.1511.06434 · Cited by 12,235 · see citing works
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625 · see citing works
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132 · see citing works
Unsupervised Domain Adaptation by BackpropagationPreprint
Ganin, Y., & Lempitsky, V. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1409.7495 · Cited by 5,360 · see citing works
Adversarial Discriminative Domain Adaptation
Tzeng, E., Hoffman, J., Saenko, K., & Darrell, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.316 · Cited by 5,154 · see citing works
Self-citationLearning Transferable Features with Deep Adaptation Networks
Long, M., Cao, Y., Wang, J., & Jordan, M. I. (2015) · arXiv · doi:10.48550/arxiv.1502.02791 · Cited by 4,505 · see citing works
Deep CORAL: Correlation Alignment for Deep Domain Adaptation
Sun, B., & Saenko, K. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-49409-8_35 · Cited by 3,757 · see citing works
Domain randomization for transferring deep neural networks from simulation to the real world
Tobin, J., Fong, R., Ray, A., Schneider, J., Zaremba, W., & Abbeel, P. (2017) · 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8202133 · Cited by 3,124 · cited 2× · see citing works
“Transferring learned policies from simulation to the real-world (Sim2Real) has become an area of broad interest [24, 26, 33] yet there still exists a sizable performance gap for most algorithms.”
BackgroundCyCADA: Cycle-Consistent Adversarial Domain Adaptation
Hoffman, J., Tzeng, E., Park, T., Zhu, J.-Y., Isola, P., Saenko, K., Efros, A. A., & Darrell, T. (2017) · arXiv · doi:10.48550/arxiv.1711.03213 · Cited by 2,656 · see citing works
Self-citationMaking the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering
Goyal, Y., Khot, T., Summers-Stay, D., Batra, D., & Parikh, D. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.670 · Cited by 2,287 · see citing works
Self-citationEnd-to-end training of deep visuomotor policies
Levine, S., Finn, C., Darrell, T., & Abbeel, P. (2016) · Journal of Machine Learning Research · Cited by 2,192 · see citing works
Levine, S., Pastor, P., Krizhevsky, A., Ibarz, J., & Quillen, D. (2018) · The International Journal of Robotics Research · doi:10.1177/0278364917710318 · Cited by 1,971 · see citing works
Matterport3D: Learning from RGB-D Data in Indoor Environments
Chang, A., Dai, A., Funkhouser, T., Halber, M., Niebner, M., Savva, M., Song, S., Zeng, A., & Zhang, Y. (2017) · 2017 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2017.00081 · Cited by 1,858 · see citing works
Curiosity-Driven Exploration by Self-Supervised Prediction
Pathak, D., Agrawal, P., Efros, A. A., & Darrell, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.70 · Cited by 1,815 · see citing works
Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks
Bousmalis, K., Silberman, N., Dohan, D., Erhan, D., & Krishnan, D. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.18 · Cited by 1,603 · see citing works
Target-driven visual navigation in indoor scenes using deep reinforcement learning
Zhu, Y., Mottaghi, R., Kolve, E., Lim, J. J., Gupta, A., Fei-Fei, L., & Farhadi, A. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989381 · Cited by 1,533 · see citing works
Coupled Generative Adversarial NetworksPreprint
Liu, M.-Y., & Tuzel, O. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.07536 · Cited by 1,433 · see citing works
Sim-to-Real Transfer of Robotic Control with Dynamics Randomization
Peng, X. B., Andrychowicz, M., Zaremba, W., & Abbeel, P. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8460528 · Cited by 1,309 · see citing works
Semantic Scene Completion from a Single Depth Image
Song, S., Yu, F., Zeng, A., Chang, A. X., Savva, M., & Funkhouser, T. A. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.28 · Cited by 1,244 · see citing works
Habitat: A Platform for Embodied AI Research
Savva, M., Kadian, A., Maksymets, O., Zhao, Y., Wijmans, E., Jain, B., Straub, J., Liu, J., Koltun, V., Malik, J., et al. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00943 · Cited by 1,237 · see citing works
Self-citationTaskonomy: Disentangling Task Transfer Learning
Zamir, A. R., Sax, A., Shen, W., Guibas, L., Malik, J., & Savarese, S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00391 · Cited by 1,128 · see citing works
Unsupervised Cross-Domain Image GenerationPreprint
Taigman, Y., Polyak, A., & Wolf, L. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.02200 · Cited by 901 · see citing works
Reinforcement Learning with Unsupervised Auxiliary TasksPreprint
Jaderberg, M., Mnih, V., Czarnecki, W. M., Schaul, T., Leibo, J. Z., Silver, D., & Kavukcuoglu, K. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.05397 · Cited by 826 · see citing works
Gibson Env: Real-World Perception for Embodied Agents
Xia, F., Zamir, A. R., He, Z., Sax, A., Malik, J., & Savarese, S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00945 · Cited by 763 · see citing works
FCNs in the Wild: Pixel-level Adversarial and Constraint-based AdaptationPreprint
Hoffman, J., Wang, D., Yu, F., & Darrell, T. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.02649 · Cited by 692 · see citing works
Self-citationLearning to Navigate in Complex EnvironmentsPreprint
Mirowski, P., Pascanu, R., Viola, F., Soyer, H., Ballard, A. J., Banino, A., Denil, M., Goroshin, R., Sifre, L., Kavukcuoglu, K., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.03673 · Cited by 679 · see citing works
On evaluation of embodied navigation agents
(2018) · doi:10.48550/arxiv.1807.06757 · Cited by 622 · see citing works
CAD2RL: Real Single-Image Flight Without a Single Real Image
Sadeghi, F., & Levine, S. (2017) · Robotics: Science and Systems XIII · doi:10.15607/rss.2017.xiii.034 · Cited by 618 · see citing works
AI2-THOR: An Interactive 3D Environment for Visual AI
Kolve, E., Mottaghi, R., Han, W., VanderBilt, E., Weihs, L., Herrasti, A., Deitke, M., Ehsani, K., Gordon, D., Zhu, Y., et al. (2017) · arXiv · doi:10.48550/arxiv.1712.05474 · Cited by 578 · see citing works
Self-citationCognitive Mapping and Planning for Visual Navigation
Gupta, S., Davidson, J., Levine, S., Sukthankar, R., & Malik, J. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.769 · Cited by 521 · see citing works
Curriculum Domain Adaptation for Semantic Segmentation of Urban Scenes
Zhang, Y., David, P., & Gong, B. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.223 · Cited by 504 · see citing works
Adversarial Discriminative Domain AdaptationPreprint
Tzeng, E., Hoffman, J., Saenko, K., & Darrell, T. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.05464 · Cited by 485 · see citing works
Self-citationSim-to-Real Robot Learning from Pixels with Progressive Nets
Rusu, A. A., Vecerik, M., Rothörl, T., Heess, N., Pascanu, R., & Hadsell, R. (2016) · arXiv · doi:10.48550/arxiv.1610.04286 · Cited by 385 · see citing works
Matterport3D: Learning from RGB-D Data in Indoor EnvironmentsPreprint
Chang, A., Dai, A., Funkhouser, T. A., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.06158 · Cited by 332 · see citing works
Deep reinforcement learning with successor features for navigation across similar environments
Zhang, J.-W., Springenberg, J., Boedecker, J., & Burgard, W. (2016) · IEEE/RJS International Conference on Intelligent RObots and Systems · doi:10.1109/iros.2017.8206049 · Cited by 331 · see citing works
Cognitive Mapping and Planning for Visual Navigation
Gupta, S., Tolani, V., Davidson, J., Levine, S., Sukthankar, R., & Malik, J. (2019) · International Journal of Computer Vision · doi:10.1007/s11263-019-01236-7 · Cited by 275 · see citing works
Driving Policy Transfer via Modularity and AbstractionPreprint
Müller, M., Dosovitskiy, A., Ghanem, B., & Koltun, V. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.09364 · Cited by 167 · see citing works
Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement LearningPreprint
Zhu, Y., Mottaghi, R., Kolve, E., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1609.05143 · Cited by 163 · see citing works
Sim2Real Viewpoint Invariant Visual Servoing by Recurrent Control
Sadeghi, F., Toshev, A., Jang, E., & Levine, S. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00493 · Cited by 133 · see citing works
Visual Semantic Planning Using Deep Successor Representations
Zhu, Y., Gordon, D., Kolve, E., Fox, D., Fei-Fei, L., Gupta, A., Mottaghi, R., & Farhadi, A. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.60 · Cited by 129 · see citing works
Self-citationDomain Transfer Through Deep Activation Matching
Huang, H., Qi-xing Huang, A., & Krähenbühl, P. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01270-0_36 · Cited by 110 · see citing works
CHALET: Cornell House Agent Learning EnvironmentPreprint
Yan, C., Misra, D. K., Bennnett, A., Walsman, A., Bisk, Y., & Artzi, Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.07357 · Cited by 71 · see citing works
Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial NetworksPreprint
Bousmalis, K., Silberman, N., Dohan, D., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.05424 · Cited by 71 · see citing works
(Sasha), A. S., Emi, B., Zamir, A., Guibas, L. J., Savarese, S., & Malik, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.11971 · Cited by 51 · see citing works
Semantic Scene Completion from a Single Depth ImagePreprint
Song, S., Yu, F., Zeng, A., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.08974 · Cited by 43 · see citing works
DIViS: Domain Invariant Visual Servoing for Collision-Free Goal Reaching
Sadeghi, F. (2019) · Robotics: Science and Systems · doi:10.15607/rss.2019.xv.055 · Cited by 32 · see citing works
(Sasha), A. S., Emi, B., Zamir, A., Guibas, L. J., Savarese, S., & Malik, J. (2018) · arXiv (Cornell University) · Cited by 13 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- [25]25. Alec Radford, Luke Metz, and Soumith Chintala. Unsuper- vised representation learning with deep convolutional gener- ative adversarial networks. In International Conference in Learning Representations (ICLR), 2016.
Cited by
Sorted by citations
A Survey of Embodied AI: From Simulators to Research Tasks
Duan, J., Yu, S., Tan, H. L., Zhu, H., & Tan, C. (2022) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2022.3141105 · Cited by 420 · see citing works
End-to-End Model-Free Reinforcement Learning for Urban Driving Using Implicit Affordances
Toromanoff, M., Wirbel, É., & Moutarde, F. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00718 · Cited by 219 · see citing works
Sim2Real Predictivity: Does Evaluation in Simulation Predict Real-World Performance?
Kadian, A., Truong, J., Gokaslan, A., Clegg, A., Wijmans, E., Lee, S., Savva, M., Chernova, S., & Batra, D. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.3013848 · Cited by 201 · see citing works
Self-citationSoundSpaces: Audio-Visual Navigation in 3D Environments
Chen, C., Jain, U., Schissler, C., Gari, S. V. A., Al-Halah, Z., Ithapu, V. K., Robinson, P., & Grauman, K. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58539-6_2 · Cited by 202 · see citing works
DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion FramesPreprint
Wijmans, E., Kadian, A., Morcos, A. S., Lee, S., Essa, I. A., Parikh, D., Savva, M., & Batra, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.00357 · Cited by 253 · see citing works
Self-citationCoWs on Pasture: Baselines and Benchmarks for Language-Driven Zero-Shot Object Navigation
Gadre, S. Y., Wortsman, M., Ilharco, G., Schmidt, L., & Song, S. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02219 · Cited by 192 · see citing works
THDA: Treasure Hunt Data Augmentation for Semantic Navigation
Maksymets, O., Cartillier, V., Gokaslan, A., Wijmans, E., Galuba, W., Lee, S., & Batra, D. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01509 · Cited by 66 · see citing works
Self-citationZero Experience Required: Plug & Play Modular Transfer Learning for Semantic Visual Navigation
Al-Halah, Z., Ramakrishnan, S. K., & Grauman, K. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01652 · Cited by 58 · see citing works
Sim-2-Sim Transfer for Vision-and-Language Navigation in Continuous Environments
Krantz, J., & Lee, S. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-19842-7_34 · Cited by 56 · see citing works
GRI: General Reinforced Imitation and Its Application to Vision-Based Autonomous Driving
Chekroun, R., Toromanoff, M., Hornauer, S., & Moutarde, F. (2023) · Robotics · doi:10.3390/robotics12050127 · Cited by 50 · see citing works
Auxiliary Tasks Speed Up Learning PointGoal NavigationPreprint
Ye, J., Batra, D., Wijmans, E., & Das, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.04561 · Cited by 47 · see citing works
Self-citationVisual language navigation: a survey and open challenges
Park, S., & Kim, Y. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-022-10174-9 · Cited by 45 · see citing works
Learning Navigational Visual Representations with Semantic Map Supervision
Hong, Y., Zhou, Y., Zhang, R., Dernoncourt, F., Bui, T., Gould, S., & Tan, H. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00284 · Cited by 42 · see citing works
Embodied Visual Navigation With Automatic Curriculum Learning in Real Environments
Morad, S., Mecca, R., Poudel, R. P. K., Liwicki, S., & Cipolla, R. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.3048662 · Cited by 42 · see citing works
Monocular Camera-Based Complex Obstacle Avoidance via Efficient Deep Reinforcement Learning
Ding, J., Gao, L., Liu, W., Piao, H., Pan, J., Du, Z., Yang, X. S., & Yin, B. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3203974 · Cited by 40 · see citing works
(2019) · arXiv (Cornell University) · Cited by 38 · see citing works
Self-citationRobustNav: Towards Benchmarking Robustness in Embodied Navigation
Chattopadhyay, P., Hoffman, J., Mottaghi, R., & Kembhavi, A. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01540 · Cited by 37 · see citing works
Self-citationScene Graph Contrastive Learning for Embodied Navigation
Singh, K. P., Salvador, J., Weihs, L., & Kembhavi, A. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00999 · Cited by 23 · see citing works
XVO: Generalized Visual Odometry via Cross-Modal Self-Training
Lai, L., Shangguan, Z., Zhang, J., & Ohn-Bar, E. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00926 · Cited by 21 · see citing works
A Vision-based Irregular Obstacle Avoidance Framework via Deep Reinforcement Learning
Gao, L., Ding, J., Liu, W., Piao, H., Wang, Y., Yang, X. S., & Yin, B. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636512 · Cited by 21 · see citing works
NeoNav: Improving the Generalization of Visual Navigation via Generating Next Expected Observations
Wu, Q., Manocha, D., Wang, J., & Xu, K. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i06.6556 · Cited by 18 · see citing works
Audio-Visual Embodied Navigation.Preprint
Chen, C., Jain, U., Schissler, C., Garí, S. V. A., Al-Halah, Z., Ithapu, V. K., Robinson, P. W., & Grauman, K. L. (2019) · arXiv (Cornell University) · Cited by 17 · see citing works
A Few Shot Adaptation of Visual Navigation Skills to New Observations using Meta-Learning
Luo, Q., Sorokin, M., & Ha, S. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9561056 · Cited by 16 · see citing works
Egocentric Activity Recognition and Localization on a 3D Map
Liu, M., Ma, L., Somasundaram, K. K., Li, Y., Grauman, K. L., Rehg, J. M., & Li, C. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19778-9_36 · Cited by 16 · see citing works
Deep Learning for Embodied Vision Navigation: A SurveyPreprint
Zhu, F., Zhu, Y., Lee, V. C., Liang, X., & Chang, X. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.04097 · Cited by 19 · see citing works
Omnidirectional Information Gathering for Knowledge Transfer-based Audio-Visual Navigation
Chen, J., Wang, W., Liu, S., Li, H., & Yang, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01009 · Cited by 12 · see citing works
Kuo, C.-W., Ma, C., Hoffman, J., & Kira, Z. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00116 · Cited by 11 · see citing works
Self-citationLobos-Tsunekawa, K., & Harada, T. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9341771 · Cited by 11 · see citing works
How to Train PointGoal Navigation Agents on a (Sample and Compute) Budget
Wijmans, E., Essa, I. A., & Batra, D. (2022) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/dejb9801 · Cited by 8 · see citing works
Self-citationSkill Fusion in Hybrid Robotic Framework for Visual Object Goal Navigation
Staroverov, A., Muravyev, K., Yakovlev, K., & Panov, A. I. (2023) · Robotics · doi:10.3390/robotics12040104 · Cited by 8 · see citing works
Safe-Nav: learning to prevent PointGoal navigation failure in unknown environments
Jin, S., Meng, Q., Dai, X.-Y., & Hou, H.-R. (2022) · Complex & Intelligent Systems · doi:10.1007/s40747-022-00648-2 · Cited by 9 · see citing works
Towards AI Aesthetics: Human-AI Collaboration in Creating Chinese Landscape Painting
Chang, R., & Huang, Y. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-77411-0_15 · Cited by 7 · see citing works
Unsupervised Domain Adaptation for Visual NavigationPreprint
Li, S., Chaplot, D. S., Tsai, Y.-H. H., Wu, Y., Morency, L., & Salakhutdinov, R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.14543 · Cited by 9 · see citing works
Neighbor-Environment Observer: An Intelligent Agent for Immersive Working Companionship
Sun, Z., Liang, Q., Wang, M., & Zhang, Z. (2023) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3586183.3606728 · Cited by 6 · see citing works
Self-Supervised Domain Adaptation for Visual Navigation with Global Map Consistency
Lee, E. S., Kim, J., & Kim, Y. M. (2022) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv51458.2022.00193 · Cited by 6 · see citing works
Auto-Navigator: Decoupled Neural Architecture Search for Visual Navigation
Tang, T., Yu, X., Dong, X., & Yang, Y. (2021) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv48630.2021.00379 · Cited by 6 · see citing works
An A* Curriculum Approach to Reinforcement Learning for RGBD Indoor Robot NavigationPreprint
Balakrishnan, K., Chakravarty, P., & Shrivastava, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.01774 · Cited by 8 · see citing works
Multiple Visual Features in Topological Map for Vision-and-Language Navigation
Liu, R., Kong, P., & Zhang, W. (2024) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros58592.2024.10803061 · Cited by 5 · see citing works
Generalization in Reinforcement Learning for Radio Access Networks
Demirel, B., Wang, Y., Tatino, C., & Soldati, P. (2025) · IEEE Transactions on Machine Learning in Communications and Networking · doi:10.1109/tmlcn.2026.3655335 · Cited by 6 · see citing works
MANGA: Method Agnostic Neural-policy Generalization and Adaptation
Bharadhwaj, H., Yamaguchi, S., & Maeda, S. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9197398 · Cited by 5 · see citing works
Environment Predictive Coding for Embodied AgentsPreprint
Ramakrishnan, S. K., Nagarajan, T., Al-Halah, Z., & Grauman, K. L. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.02337 · Cited by 5 · see citing works
TVENet: Transformer-Based Visual Exploration Network for Mobile Robot in Unseen Environment
Zhang, T., Hu, X., Xiao, J., & Zhang, G. (2022) · IEEE Access · doi:10.1109/access.2022.3181989 · Cited by 4 · see citing works
Hong, Z. (2025) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2024.3524882 · Cited by 3 · see citing works
Language-Based Augmentation to Address Shortcut Learning in Object-Goal Navigation
Hoftijzer, D., Burghouts, G., & Spreeuwers, L. (2023) · 2023 Seventh IEEE International Conference on Robotic Computing (IRC) · doi:10.1109/irc59093.2023.00007 · Cited by 2 · see citing works
“While ProcTHOR shows incredibly powerful results, such augmentations might not be available for all simulators, and more often than not, difficult to apply post-hoc.”
Audio-Visual Navigation with Anti-Backtracking
Zhao, Z., Tang, H., & Yan, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-78456-9_23 · Cited by 2 · see citing works
Interactive Semantic Map Representation for Skill-Based Visual Object Navigation
Zemskova, T., Staroverov, A., Muravyev, K., Yudin, D. A., & Panov, A. I. (2024) · IEEE Access · doi:10.1109/access.2024.3380450 · Cited by 2 · see citing works
Audio-Visual Waypoints for NavigationPreprint
Chen, C., Majumder, S., Al-Halah, Z., Gao, R., Ramakrishnan, S. K., & Grauman, K. L. (2020) · arXiv (Cornell University) · Cited by 3 · see citing works
Language-Based Augmentation to Mitigate Shortcut Learning in Object-Goal Navigation
Hoftijzer, D., Burghouts, G. J., & Spreeuwers, L. (2025) · International Journal of Semantic Computing · doi:10.1142/s1793351x25410077 · Cited by 1 · see citing works
Robot Navigation and Obstacle Avoidance
Yang, X. S., Yin, B., & Wei, X. (2025) · Springer eBooks · doi:10.1007/978-981-96-3177-3_4 · Cited by 1 · see citing works
Takayama, Y., Ratsamee, P., & Mashita, T. (2021) · Robotics · doi:10.3390/robotics10040131 · Cited by 1 · see citing works
Decentralized Distributed PPO: Mastering PointGoal NavigationPreprint
Wijmans, E., Kadian, A., Morcos, A. S., Lee, S., Essa, I. A., Parikh, D., Savva, M., & Batra, D. (2020) · arXiv (Cornell University) · Cited by 1 · see citing works
Self-citationA Few Shot Adaptation of Visual Navigation Skills to New Observations using Meta-LearningPreprint
Luo, Q., Sorokin, M., & Ha, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.03609
VSGM -- Enhance robot task understanding ability through visual semantic graphPreprint
Tsai, C. Y., & Su, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.08959
Zhao, X., Wang, T., Liu, K., Zhang, M., Shi, P., & Snoussi, H. (2021) · China Automation Congress (CAC) · doi:10.1109/cac53003.2021.9728402
Model Adaptation for Time Constrained Embodied Control
Song, J., Yoo, M., & Woo, H. (2024) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr52733.2024.01561
DSSNav: dual-stream slot-based fusion via Soft Actor-Critic for audio–visual navigation
Li, L., Duan, L., Zhao, K., & Si, B. (2026) · Multimedia Systems · doi:10.1007/s00530-026-02363-1
- Authors
- Gordon, D., Kadian, A., Parikh, D., Hoffman, J., & Batra, D.
- Affiliations
- University of Washington (US)
- Meta (United States) (US)
- Georgia Institute of Technology (US)
- Venue
- 2019 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2019-10
- Pages
- 1022-1031
- OpenAlex
- W2944849990
- Semantic Scholar
- 980d6820831d5cadfd0f1699b06145efa975f9fd
- Cited by
- 57
- References
- 49
- Citations by year
- 2026: 2 · 2025: 3 · 2024: 4 · 2023: 10 · 2022: 10 · 2021: 13 · 2020: 12 · 2019: 3
- FWCI
- 3.56
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Engineering › Aerospace Engineering › Robotics and Sensor-Based Localization
- Keywords
- sim-to-real transfer, auxiliary tasks, selective learning, domain generalization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
