Neural Map: Structured Memory for Deep Reinforcement Learning
Open accessAbstract
A critical component to enabling intelligent reasoning in partially observable environments is memory. Despite this importance, Deep Reinforcement Learning (DRL) agents have so far used relatively simple memory architectures, with the main methods to overcome partial observability being either a temporal convolution over the past k frames or an LSTM layer. More recent work (Oh et al., 2016) has went beyond these architectures by using memory networks which can allow more sophisticated addressing schemes over the past k frames. But even these architectures are unsatisfactory due to the reason that they are limited to only remembering information from the last k frames. In this paper, we develop a memory system with an adaptable write operator that is customized to the sorts of 3D environments that DRL agents typically interact with. This architecture, called the Neural Map, uses a spatially structured 2D memory image to learn to store arbitrary information about the environment over long time lags. We demonstrate empirically that the Neural Map surpasses previous DRL memories on a set of challenging 2D and 3D maze environments and show that it is capable of generalizing to environments that were not seen during training.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
MemoryRepository mimics the human memory process, allowing AI NPC to forget short-term memory, thereby freeing up memory space.
“This mechanism mimics the human memory process, allowing AI NPC to forget short-term memory, thereby freeing up memory space [16].”
From MemoryRepository for AI NPC (2024) · cites this paper as [16]
The Introduction (pp. 1–2) describes Neural Map’s selective writes and bounded memory size, not a MemoryRepository or a mechanism for forgetting short-term memories to free space. A search for human-memory mimicry and short-term forgetting found no passage addressing those claims. (LLM, conf=0.75)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357 · see citing works
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,263 · see citing works
Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1412.3555 · Cited by 11,008 · see citing works
Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Machine Learning · doi:10.1007/bf00992696 · Cited by 7,936 · see citing works
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132 · see citing works
End-to-end training of deep visuomotor policies
Levine, S., Finn, C., Darrell, T., & Abbeel, P. (2016) · Journal of Machine Learning Research · Cited by 2,192 · see citing works
A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning
Ross, S., Gordon, G. J., & Bagnell, J. A. (2018) · Carnegie Mellon University · doi:10.1184/r1/6550949 · Cited by 1,941 · see citing works
Hybrid computing using a neural network with dynamic external memory
Graves, A., Wayne, G., Reynolds, M., Harley, T., Danihelka, I., Grabska-Barwińska, A., Colmenarejo, S. G., Grefenstette, E., Ramalho, T., Agapiou, J., et al. (2016) · Nature · doi:10.1038/nature20101 · Cited by 1,412 · see citing works
Deep Recurrent Q-Learning for Partially Observable MDPsPreprint
Hausknecht, M., & Stone, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1507.06527 · Cited by 1,224 · see citing works
(2016) · doi:10.18653/v1/d16-1147 · Cited by 1,035 · see citing works
Tamar, A., Wu, Y., Thomas, G., Levine, S., & Abbeel, P. (2017) · Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/700 · Cited by 484 · see citing works
Playing FPS Games with Deep Reinforcement Learning
Lample, G., & Chaplot, D. S. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10827 · Cited by 475 · see citing works
Cognitive Mapping and Planning for Visual Navigation
Gupta, S., Tolani, V., Davidson, J., Levine, S., Sukthankar, R., & Malik, J. (2019) · International Journal of Computer Vision · doi:10.1007/s11263-019-01236-7 · Cited by 275 · see citing works
Neural GPUs Learn AlgorithmsPreprint
Kaiser, Ł., & Sutskever, I. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.08228 · Cited by 224 · see citing works
Control of Memory, Active Perception, and Action in MinecraftPreprint
Oh, J., Chockalingam, V., Singh, S. P., & Lee, H. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.09128 · Cited by 213 · see citing works
Playing FPS Games with Deep Reinforcement LearningPreprint
Lample, G., & Chaplot, D. S. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1609.05521 · Cited by 103 · see citing works
Playing Doom with SLAM-Augmented Deep Reinforcement LearningPreprint
Bhatti, S., Desmaison, A., Mikšík, O., Nardelli, N., Siddharth, N., & Torr, P. H. S. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.00380 · Cited by 48 · see citing works
Flagged first, then by citations
5 of the 22 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Deep Reinforcement Learning: A Brief Survey
Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A. (2017) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2743240 · Cited by 4,733 · see citing works
On evaluation of embodied navigation agents
(2018) · doi:10.48550/arxiv.1807.06757 · Cited by 622 · see citing works
Deep reinforcement learning based mobile robot navigation: A review
Zhu, K., & Zhang, T. (2021) · Tsinghua Science & Technology · doi:10.26599/tst.2021.9010012 · Cited by 516 · see citing works
Attention, please! A survey of neural attention models in deep learning
Correia, A. D. S., & Colombini, E. L. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-022-10148-x · Cited by 330 · see citing works
Object Goal Navigation using Goal-Oriented Semantic Exploration
Chaplot, D. S., Gandhi, D., Gupta, A., & Salakhutdinov, R. (2020) · arXiv · doi:10.48550/arxiv.2007.00643 · Cited by 321 · see citing works
Self-citationDeep Reinforcement Learning-Based Automatic Exploration for Navigation in Unknown Environment
Li, H., Zhang, Q., & Zhao, D. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2927869 · Cited by 262 · see citing works
Learning to Navigate in Cities Without a MapPreprint
Mirowski, P., Grimes, M. K., Malinowski, M., Hermann, K. M., Anderson, K., Teplyashin, D., Simonyan, K., Kavukcuoglu, K., Zisserman, A., & Hadsell, R. T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.00168 · Cited by 248 · see citing works
Neural Topological SLAM for Visual Navigation
Singh Chaplot, D., Salakhutdinov, R., Gupta, A., & Gupta, S. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01289 · Cited by 241 · see citing works
Self-citationCognitive Mapping and Planning for Visual Navigation
Gupta, S., Tolani, V., Davidson, J., Levine, S., Sukthankar, R., & Malik, J. (2019) · International Journal of Computer Vision · doi:10.1007/s11263-019-01236-7 · Cited by 275 · see citing works
Stabilizing Transformers for Reinforcement Learning
Parisotto, E., Song, H. F., Rae, J. W., Pascanu, R., Gulcehre, C., Jayakumar, S. M., Jaderberg, M., Kaufman, R. L., Clark, A., Noury, S., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.06764 · Cited by 199 · see citing works
Self-citationPerceive, Predict, and Plan: Safe Motion Planning Through Interpretable Semantic Representations
Sadat, A., Casas, S., Ren, M., Wu, X., Dhawan, P., & Urtasun, R. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58592-1_25 · Cited by 183 · see citing works
iGibson 1.0: A Simulation Environment for Interactive Tasks in Large Realistic Scenes
Shen, B., Xia, F., Li, C., Martín-Martín, R., Fan, L., Wang, G., Pérez-D’Arpino, C., Buch, S., Srivastava, S., Tchapmi, L. P., et al. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636667 · Cited by 182 · see citing works
Scene Memory Transformer for Embodied Agents in Long-Horizon Tasks
Fang, K., Toshev, A., Fei-Fei, L., & Savarese, S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00063 · Cited by 178 · see citing works
Spatial Memory for Context Reasoning in Object Detection
Chen, X., & Gupta, A. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.440 · Cited by 172 · see citing works
A Survey of Deep Reinforcement Learning in Video GamesPreprint
Shao, K., Tang, Z., Zhu, Y., Li, N., & Zhao, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.10944 · Cited by 224 · see citing works
Occupancy Anticipation for Efficient Exploration and Navigation
Ramakrishnan, S. K., Al-Halah, Z., & Grauman, K. L. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58558-7_24 · Cited by 151 · see citing works
MapNet: An Allocentric Spatial Memory for Mapping Environments
Henriques, J. F., & Vedaldi, A. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00884 · Cited by 148 · see citing works
Learning Exploration Policies for NavigationPreprint
Chen, T., Gupta, S., & Gupta, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.01959 · Cited by 159 · see citing works
Combining Optimal Control and Learning for Visual Navigation in Novel EnvironmentsPreprint
Bansal, S., Tolani, V., Gupta, S., Malik, J., & Tomlin, C. J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.02531 · Cited by 141 · see citing works
Structured Scene Memory for Vision-Language Navigation
Wang, H., Wang, W., Liang, W., Xiong, C., & Shen, J. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00835 · Cited by 123 · see citing works
Topological Planning with Transformers for Vision-and-Language Navigation
Chen, K., Chen, J. K., Chuang, J., Vázquez, M., & Savarese, S. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01112 · Cited by 120 · see citing works
Value Prediction NetworkPreprint
Oh, J., Singh, S. P., & Lee, H. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.03497 · Cited by 119 · see citing works
Neural SLAMPreprint
Zhang, J.-W., Tai, L., Boedecker, J., Burgard, W., & Liu, M. (2017) · arXiv.org · doi:10.48550/arxiv.1706.09520 · Cited by 68 · see citing works
A survey on deep reinforcement learning for audio-based applications
Latif, S., Cuayáhuitl, H., Pervez, F., Shamshad, F., Ali, H. S., & Cambria, E. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-022-10224-2 · Cited by 116 · see citing works
Local-Global Context Aware Transformer for Language-Guided Video Segmentation
Liang, C., Wang, W., Zhou, T., Miao, J., Luo, Y., & Yang, Y. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3262578 · Cited by 106 · see citing works
A survey on visual navigation for artificial agents with deep reinforcement learning
Zeng, F., Wang, C., & Ge, S. S. (2020) · IEEE Access · doi:10.1109/access.2020.3011438 · Cited by 98 · see citing works
Bayesian Relational Memory for Semantic Visual Navigation
Wu, Y., Wu, Y., Tamar, A., Russell, S., Gkioxari, G., & Tian, Y. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00286 · Cited by 97 · see citing works
ViZDoom Competitions: Playing Doom From Pixels
Wydmuch, M., Kempka, M., & Jaśkowski, W. (2018) · IEEE Transactions on Games · doi:10.1109/tg.2018.2877047 · Cited by 92 · see citing works
Global Pose Estimation with an Attention-Based Recurrent Network
Parisotto, E., Chaplot, D. S., Zhang, J., & Salakhutdinov, R. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2018.00061 · Cited by 88 · see citing works
Self-citationA Behavioral Approach to Visual Navigation with Graph Localization Networks
Chen, K., de Vicente, J. P., Sepulveda, G., Xia, F., Soto, Á., Vzquez, M., & Savarese, S. (2019) · Robotics: Science and Systems · doi:10.15607/rss.2019.xv.010 · Cited by 86 · see citing works
Energy-Efficient UAV Crowdsensing with Multiple Charging Stations by Deep Learning
Liu, C. H., Piao, C., & Tang, J. (2020) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom41043.2020.9155535 · Cited by 79 · see citing works
Choi, J., Park, K.-S., Kim, M., & Seok, S. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8793979 · Cited by 78 · see citing works
Xiang, X., & Foo, S. Y. (2021) · Machine Learning and Knowledge Extraction · doi:10.3390/make3030029 · Cited by 72 · see citing works
Virtual-to-Real: Learning to Control in Visual Semantic Segmentation
Hong, Z.-W., Chen, Y., Yang, H.-K., Su, S.-Y., Shann, T.-Y., Chang, Y.-H., Ho, B. H.-L., Tu, C.-C., Hsiao, T.-C., Hsiao, H.-W., et al. (2018) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/682 · Cited by 73 · see citing works
Semantic MapNet: Building Allocentric Semantic Maps and Representations from Egocentric Views
Cartillier, V., Ren, Z., Jain, N., Lee, S., Essa, I. A., & Batra, D. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i2.16180 · Cited by 68 · see citing works
A Survey of Deep Network Solutions for Learning Control in Robotics: From Reinforcement to ImitationPreprint
Tai, L., Zhang, J., Liu, M., Boedecker, J., & Burgard, W. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.07139 · Cited by 95 · see citing works
Ravichandran, Z., Peng, L., Hughes, N., Griffith, J., & Carlone, L. (2022) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra46639.2022.9812179 · Cited by 63 · see citing works
Unifying Map and Landmark Based Representations for Visual NavigationPreprint
Gupta, S., Fouhey, D. F., Levine, S., & Malik, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.08125 · Cited by 62 · see citing works
Memory-Augmented Reinforcement Learning for Image-Goal Navigation
Mezghan, L., Sukhbaatar, S., Lavril, T., Maksymets, O., Batra, D., Bojanowski, P., & Alahari, K. (2022) · 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9981090 · Cited by 61 · see citing works
Active Neural LocalizationPreprint
Chaplot, D. S., Parisotto, E., & Salakhutdinov, R. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.08214 · Cited by 59 · see citing works
Self-citationVTNet: Visual Transformer Network for Object Goal NavigationPreprint
Du, H., Yu, X., & Zheng, L. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.09447 · Cited by 69 · see citing works
Situational Fusion of Visual Representation for Visual Navigation
Shen, W., Xu, D., Zhu, Y., Fei-Fei, L., Guibas, L. J., & Savarese, S. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00297 · Cited by 57 · see citing works
Benchmarking Classic and Learned Navigation in Complex 3D EnvironmentsPreprint
Mishkin, D., Dosovitskiy, A., & Koltun, V. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.10915 · Cited by 57 · see citing works
Renderable Neural Radiance Map for Visual Navigation
Kwon, O., Park, J., & Oh, S. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00878 · Cited by 56 · see citing works
3D-Aware Object Goal Navigation via Simultaneous Exploration and Identification
Zhang, J., Liu, D., Meng, F., Fan, Q., Chen, X., Xu, K., & Wang, H. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00645 · Cited by 55 · see citing works
Chasing Ghosts: Instruction Following as Bayesian State Tracking
Anderson, P., Shrivastava, A., Parikh, D., Batra, D., & Lee, S. (2019) · arXiv · doi:10.48550/arxiv.1907.02022 · Cited by 55 · see citing works
Evolving Graphical Planner: Contextual Global Planning for Vision-and-Language NavigationPreprint
Deng, Z., Narasimhan, K., & Russakovsky, O. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.05655 · Cited by 55 · see citing works
Learning Spatial Common Sense With Geometry-Aware Recurrent Networks
Tung, H.-Y. F., Cheng, R., & Fragkiadaki, K. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00270 · Cited by 54 · see citing works
Memory Augmented Control NetworksPreprint
Khan, A., Zhang, C., Atanasov, N., Karydis, K., Kumar, V., & Lee, D. D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.05706 · Cited by 53 · see citing works
Coupling Vision and Proprioception for Navigation of Legged Robots
Fu, Z., Kumar, A., Agarwal, A., Qi, H., Malik, J., & Pathak, D. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01676 · Cited by 51 · see citing works
Following High-level Navigation Instructions on a Simulated Quadcopter with Imitation Learning
Blukis, V., Brukhim, N., Bennett, A., Knepper, R. A., & Artzi, Y. (2018) · Robotics: Science and Systems · doi:10.15607/rss.2018.xiv.066 · Cited by 47 · see citing works
Learning to Map for Active Semantic Goal NavigationPreprint
Georgakis, G., Bucher, B., Schmeckpeper, K., Singh, S., & Daniilidis, K. (2021) · International Conference on Learning Representations · doi:10.48550/arxiv.2106.15648 · Cited by 114 · see citing works
Scene-Intuitive Agent for Remote Embodied Visual Grounding
Lin, X., Li, G., & Yu, Y. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00696 · Cited by 48 · see citing works
Graph Element Networks: adaptive, structured computation and memoryPreprint
Alet, F., Jeewajee, A. K., Bauzá, M., Rodríguez, A., Lozano‐Pérez, T., & Kaelbling, L. P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.09019 · Cited by 44 · see citing works
Volumetric Environment Representation for Vision-Language Navigation
Liu, R., Wang, W., & Yang, Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01544 · Cited by 43 · see citing works
Irshad, M. Z., Mithun, N. C., Seymour, Z., Chiu, H.-P., Samarasekera, S., & Kumar, R. (2022) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr56361.2022.9956561 · Cited by 47 · see citing works
Du, H., Li, L., Huang, Z. H., & Yu, X. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00252 · Cited by 39 · see citing works
Memory Augmented Deep Generative models for Forecasting the Next Shot Location in Tennis
Fernando, T., Denman, S., Sridharan, S., & Fookes, C. (2020) · QUT ePrints (Queensland University of Technology) · Cited by 38 · see citing works
DRLViz: Understanding Decisions and Memory in Deep Reinforcement Learning
Jaunet, T., Vuillemot, R., & Wolf, C. (2020) · Computer Graphics Forum · doi:10.1111/cgf.13962 · Cited by 38 · see citing works
Deep Reinforcement Learning for Navigation in AAA Video Games
Alonso, E., Peter, M., Goumard, D., & Romoff, J. (2021) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/294 · Cited by 37 · see citing works
Revisiting EmbodiedQA: A Simple Baseline and Beyond
Wu, Y., Jiang, L., & Yang, Y. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.2967584 · Cited by 36 · see citing works
Lyapunov-Based Long Short-Term Memory (Lb-LSTM) Neural Network-Based Control
Griffis, E. J., Patil, O. S., Bell, Z. I., & Dixon, W. E. (2023) · IEEE Control Systems Letters · doi:10.1109/lcsys.2023.3291328 · Cited by 34 · see citing works
Learning to Plan with Uncertain Topological Maps
Beeching, E., Dibangoye, J., Simonin, O., & Wolf, C. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58580-8_28 · Cited by 32 · see citing works
KERMIT: Knowledge-EmpoweRed Model In harmful meme deTection
Grasso, B., Gatta, V. L., Moscato, V., & Sperlí, G. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102269 · Cited by 34 · see citing works
Machine Learning Techniques used for the Histopathological Image Analysis of Oral Cancer-A Review
Panigrahi, S., & Swarnkar, T. (2020) · The Open Bioinformatics Journal · doi:10.2174/1875036202013010106 · Cited by 33 · see citing works
A Novel Neural Multi-Store Memory Network for Autonomous Visual Navigation in Unknown Environment
Sang, H., Jiang, R., Wang, Z., Zhou, Y., & He, B. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3140795 · Cited by 31 · see citing works
Hybrid Online POMDP Planning and Deep Reinforcement Learning for Safer Self-Driving Cars
Pusse, F., & Klusch, M. (2019) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2019.8814125 · Cited by 30 · see citing works
A Critical Investigation of Deep Reinforcement Learning for NavigationPreprint
Dhiman, V., Banerjee, S., Griffin, B., Siskind, J. M., & Corso, J. J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.02274 · Cited by 30 · see citing works
Reinforcement Learning for Computer Vision and Robot Navigation
Bernstein, A., Burnaev, E. V., & Kachan, O. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-96133-0_20 · Cited by 29 · see citing works
Detection of Fake and Fraudulent Faces via Neural Memory Networks
Fernando, T., Fookes, C., Denman, S., & Sridharan, S. (2020) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2020.3047768 · Cited by 47 · see citing works
Deep Visual Odometry With Adaptive Memory
Xue, F., Wang, X., Wang, J., & Zha, H. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3014100 · Cited by 25 · see citing works
Wang, Y., Liu, C. H., Piao, C., Yuan, Y., Han, R., Wang, G., & Tang, J. (2022) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde53745.2022.00039 · Cited by 25 · see citing works
Play it by Ear: Learning Skills amidst Occlusion through Audio-Visual Imitation Learning
Du, M., Lee, O. Y., Nair, S., & Finn, C. (2022) · Robotics: Science and Systems · doi:10.15607/rss.2022.xviii.009 · Cited by 24 · see citing works
Task-Agnostic Dynamics Priors for Deep Reinforcement LearningPreprint
Du, Y., & Narasimhan, K. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.04819 · Cited by 27 · see citing works
Deep Reinforcement Learning With Modulated Hebbian Plus Q-Network Architecture
Ładosz, P., Ben-Iwhiwhu, E., Dick, J., Ketz, N., Kolouri, S., Krichmar, J. L., Pilly, P. K., & Soltoggio, A. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3110281 · Cited by 24 · see citing works
VME-Transformer: Enhancing Visual Memory Encoding for Navigation in Interactive Environments
Shen, J., Lou, P., Yuan, L., Lyu, S., & Lu, Y. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2023.3333238 · Cited by 23 · see citing works
Integrating Egocentric Localization for More Realistic Point-Goal Navigation AgentsPreprint
Datta, S., Maksymets, O., Hoffman, J., Lee, S., Batra, D., & Parikh, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.03231 · Cited by 24 · see citing works
Vision-Based Topological Mapping and Navigation With Self-Organizing Neural Networks
Hu, Y., Subagdja, B., Tan, A., & Yin, Q. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3084212 · Cited by 21 · see citing works
EgoMap: Projective Mapping and Structured Egocentric Memory for Deep RL
Beeching, E., Dibangoye, J., Simonin, O., & Wolf, C. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-67661-2_31 · Cited by 21 · see citing works
Memory Augmented Neural Network Adaptive Controllers: Performance and Stability
Muthirayan, D. R. P., & Khargonekar, P. P. (2022) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2022.3144382 · Cited by 20 · see citing works
Collaborative Visual NavigationPreprint
Wang, H., Wang, W., Zhu, X., Dai, J., & Wang, L. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.01151 · Cited by 24 · see citing works
MaAST: Map Attention with Semantic Transformers for Efficient Visual Navigation
Seymour, Z., Thopalli, K., Mithun, N. C., Chiu, H.-P., Samarasekera, S., & Kumar, R. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9561058 · Cited by 21 · see citing works
Deep Reinforcement Learning on a Budget: 3D Control and Reasoning Without a Supercomputer
Beeching, E., Debangoye, J., Simonin, O., & Wolf, C. (2021) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr48806.2021.9412212 · Cited by 18 · see citing works
Teaching Agents how to Map: Spatial Reasoning for Multi-Object Navigation
Marza, P., Matignon, L., Simonin, O., & Wolf, C. (2022) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9982216 · Cited by 18 · see citing works
Pedestrian Trajectory Prediction with Structured Memory Hierarchies
Fernando, T., Denman, S., Sridharan, S., & Fookes, C. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-10925-7_15 · Cited by 18 · see citing works
Chen, B., Song, S., Lipson, H., & Vondrick, C. (2020) · Conference on Artificial Life (ALIFE) · doi:10.1162/isal_a_00269 · Cited by 17 · see citing works
Improving Target-driven Visual Navigation with Attention on 3D Spatial RelationshipsPreprint
Lv, Y., Xie, N., Shi, Y., Wang, Z., & Shen, H. T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.02153 · Cited by 18 · see citing works
Multi-Object Navigation with dynamically learned neural implicit representations
Marza, P., Matignon, L., Simonin, O., & Wolf, C. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01010 · Cited by 17 · see citing works
Long Range Neural Navigation Policies for the Real World
Wahid, A., Toshev, A., Fišer, M., & Lee, T.-W. E. (2019) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8968004 · Cited by 17 · see citing works
EMPNet: Neural Localisation and Mapping Using Embedded Memory Points
Avraham, G., Zuo, Y., Dharmasiri, T., & Drummond, T. W. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00821 · Cited by 17 · see citing works
Breaking Neural Reasoning Architectures With Metamorphic Relation-Based Adversarial Examples
Chan, A., Ma, L., Juefei-Xu, F., Ong, Y.-S., Xie, X., Xue, M., & Liu, Y. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3072166 · Cited by 16 · see citing works
Learning models for visual 3D localization with implicit mappingPreprint
Rosenbaum, D., Besse, F., Viola, F., Rezende, D. J., & Eslami, S. M. A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.03149 · Cited by 16 · see citing works
Zheng, S., He, K., Yang, L., & Xiong, J. (2024) · IEEE Access · doi:10.1109/access.2024.3393485 · Cited by 18 · see citing works
Learning Object-conditioned Exploration using Distributed Soft Actor CriticPreprint
Wahid, A., Stone, A., Chen, K., Ichter, B., & Toshev, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.14545 · Cited by 15 · see citing works
Deep Learning for Embodied Vision Navigation: A SurveyPreprint
Zhu, F., Zhu, Y., Lee, V. C., Liang, X., & Chang, X. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.04097 · Cited by 19 · see citing works
Approximate Bayesian Inference in Spatial Environments
Mirchev, A., Kayalibay, B., Soelch, M., van der Smagt, P., & Bayer, J. (2019) · Robotics: Science and Systems · doi:10.15607/rss.2019.xv.083 · Cited by 14 · see citing works
Deep Context Maps: Agent Trajectory Prediction Using Location-Specific Latent Maps
Gilitschenski, I., Rosman, G., Gupta, A., Karaman, S., & Rus, D. L. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.3004800 · Cited by 14 · see citing works
Attend and Segment: Attention Guided Active Semantic Segmentation
Seifi, S., & Tuytelaars, T. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58595-2_19 · Cited by 14 · see citing works
A Short Survey On Memory Based Reinforcement LearningPreprint
Ramani, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.06736 · Cited by 25 · see citing works
Gupta, K., Mukherjee, D., & Najjaran, H. (2021) · SN Computer Science · doi:10.1007/s42979-021-00934-9 · Cited by 18 · see citing works
- Authors
- Parisotto, E., & Salakhutdinov, R.
- Affiliations
- University of Toronto (CA)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2017
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1702.08360
- OpenAlex
- W2594903727
- Cited by
- 178
- References
- 22
- Citations by year
- 2025: 4 · 2024: 12 · 2023: 16 · 2022: 18 · 2021: 32 · 2020: 37 · 2019: 34 · 2018: 18 · 2017: 6 · 2016: 1
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Keywords
- neural maps, structured memory, memory networks, deep reinforcement learning, partial observability, long-term memory, 3D environments, maze navigation, memory addressing, generalization to unseen environments
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
