Transformer Memory for Interactive Visual Navigation in Cluttered Environments
The DOI matches its Crossref record; 27 of the 31 references it cites were found in a registry or an index.
Abstract
Substantial progress has been achieved in embodied visual navigation based on reinforcement learning (RL). These studies presume that the environment is stationary where all the obstacles are static. However, in real cluttered scenes, interactable objects (e.g. shoes and boxes) blocking the way of robots makes the environment non-stationary. Accordingly, the ego-centric visual agent will easily get stuck in the dilemma of finding the next waypoint as it struggles to decide whether to push the obstacles ahead. To handle the predicament, we formulate this interactive visual navigation as Partial Observed Markov Decision Process (POMDP). As the transformer encoder has demonstrated its superior ability to capture the spatial-temporal dependencies in natural language processing. We propose a transformer-based memory to empower the agents utilizing the historical interactive information. However, directly leveraging the transformer architecture in the RL settings is highly unstable. We further propose a surrogate objective to predict the next waypoint as the auxiliary task, which facilitates the representation learning and bootstraps the RL. We demonstrate our method in the iGibson environment and experimental results show a significant improvement over the interactive Gibson benchmark and the related recurrent RL policy both in the validation seen scenes and the test unseen scenes.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545
Soft Actor-Critic Algorithms and Applications
Haarnoja, T., Zhou, A., Hartikainen, K., Tucker, G., Ha, S., Tan, J., Kumar, V., Zhu, H., Gupta, A., Abbeel, P., et al. (2018) · arXiv · doi:10.48550/arxiv.1812.05905 · Cited by 1,994 · cited 1×
“We implement distributed soft actor-critic algorithm based on vanilla SAC [24], and the model is updated by the L sac and surrogate objective simultaneously.”
MethodHabitat: A Platform for Embodied AI Research
Savva, M., Kadian, A., Maksymets, O., Zhao, Y., Wijmans, E., Jain, B., Straub, J., Liu, J., Koltun, V., Malik, J., et al. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00943 · Cited by 1,237 · cited 5×
“We also analyze the performance of an RL Policy [12] with a convolutional network followed by a GRU, as it is considered the most common memory-based method.”
BackgroundMethodInfluentialOn evaluation of embodied navigation agents
Anderson, P., Chang, A., Chaplot, D. S., Dosovitskiy, A., Gupta, S., Koltun, V., Kosecka, J., Malik, J., Mottaghi, R., Savva, M., et al. (2018) · arXiv · doi:10.48550/arxiv.1807.06757 · Cited by 622
Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixels
Yarats, D., Kostrikov, I., & Fergus, R. (2021) · International Conference on Learning Representations · Cited by 509
Cognitive Mapping and Planning for Visual Navigation
Gupta, S., Davidson, J., Levine, S., Sukthankar, R., & Malik, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.769 · Cited by 500
DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion FramesPreprint
Wijmans, E., Kadian, A., Morcos, A. S., Lee, S., Essa, I. A., Parikh, D., Savva, M., & Batra, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.00357 · Cited by 485
Learning to Navigate in Complex Environments
Mirowski, P., Pascanu, R., Viola, F., Soyer, H., Ballard, A. J., Banino, A., Denil, M., Goroshin, R., Sifre, L., Kavukcuoglu, K., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.03673 · Cited by 323
Cited by
VME-Transformer: Enhancing Visual Memory Encoding for Navigation in Interactive Environments
Shen, J., Lou, P., Yuan, L., Lyu, S., & Lu, Y. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2023.3333238 · Cited by 26
“While direct learning of control policies from raw observations is possible, recent research highlights the advantages of joint supervision approaches, which not only enhance performance but also facilitate model scaling [7].”
BackgroundMethodInfluentialSelf-citationThe visual communication using generative artificial intelligence in the context of new media
Liu, W., & Kim, H.-G. (2025) · Scientific Reports · doi:10.1038/s41598-025-96869-9 · Cited by 28
Leveraging Predictions of Task-Related Latents for Interactive Visual Navigation
Shen, J., Yuan, L., Lu, Y., & Lyu, S. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3335416 · Cited by 15
Self-citationYang, B., Yuan, X., Ying, Z., Zhang, J., Song, B., Song, Y. H., Zhou, F., & Sheng, W. (2024) · Advanced Engineering Informatics · doi:10.1016/j.aei.2024.102671 · Cited by 15
UAV autonomous obstacle avoidance via causal reinforcement learning
Sun, T., Gu, J., & Mou, J. (2025) · Displays · doi:10.1016/j.displa.2025.102966 · Cited by 14
Metadata
- Authors
- Li, W., Hong, R., Shen, J., Yuan, L., & Lu, Y.
- Affiliations
- East China Normal University (CN)
- Beijing University of Chemical Technology (CN)
- Venue
- IEEE Robotics and Automation Letters
- Published
- 2023-03
- Volume
- 8
- Issue
- 3
- Pages
- 1731-1738
- ISSN
- 2377-3766 (ISSN-L)
- 2377-3774
- OpenAlex
- W4319990479
- Cited by
- 26
- References
- 31
- Citations by year
- 2026: 3 · 2025: 10 · 2024: 9 · 2023: 4
- FWCI
- 1.85
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Robotics and Automation Letters (publishedVersion)
- Funders
- Science and Technology Commission of Shanghai Municipality (22DZ2229004)
- Shanghai Trusted Industry Internet Software Collaborative Innovation Center
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Robotics and Sensor-Based Localization
- Keywords
- cluttered environments, non-stationary environments, auxiliary tasks, representation learning, embodied agents
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Transformer Memory for Interactive Visual Navigation in Cluttered Environments. https://app.cochat.ai/papers/10.1109/lra.2023.3241803
- The paper
- Li, W., Hong, R., Shen, J., Yuan, L., & Lu, Y. (2023). Transformer Memory for Interactive Visual Navigation in Cluttered Environments. IEEE Robotics and Automation Letters. https://doi.org/10.1109/lra.2023.3241803
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.