Brain-inspired spatial intelligence for embodied agents
Open accessThe DOI matches its Crossref record; 77 of the 77 references it cites were found in a registry or an index; 22 of the 89 sources its claims cite were checked for support.
Abstract
Spatial cognition enables adaptive goal-directed behavior through structured internal models of space. Robust biological systems consolidate spatial knowledge into three interconnected forms: landmarks for salient cues, route knowledge for movement trajectories, and survey knowledge for map-like representations. While recent advances in multi-modal large language models (MLLMs) have enabled visual-language reasoning in embodied agents, these efforts lack structured spatial memory and instead operate reactively, limiting their generalization and adaptability in complex real-world environments. Here we present Brain-inspired Spatial Cognition for Navigation (BSC-Nav), a unified framework for constructing and leveraging structured spatial memory in embodied agents. BSC-Nav builds allocentric cognitive maps from egocentric trajectories and contextual cues, and dynamically retrieves spatial knowledge aligned with semantic goals. Integrated with powerful MLLMs, BSC-Nav achieves state-of-the-art efficacy and efficiency across diverse embodied navigation tasks (e.g., improving success weighted by path length from 17.6% to 44.9% in instance-level navigation and from 42.7% to 53.1% in zero-shot long-horizon instruction-following), while also supporting versatile embodied behaviors in the real physical world. These results highlight a scalable path toward general-purpose spatial intelligence.
Overview
CoChat Summary
TL;DR
BSC-Nav is a brain-inspired framework that builds structured spatial memory—landmark, route, and survey knowledge—for embodied agents and couples it with MLLMs for goal-directed navigation. It reports state-of-the-art efficacy and efficiency across simulated and real-world navigation, instruction-following, embodied question answering, and mobile manipulation tasks.
Key findings
- BSC-Nav improves success weighted by path length from 17.6% to 44.9% in instance-level navigation and from 42.7% to 53.1% in zero-shot long-horizon instruction-following.
- In simulation, it achieves 78.5% SR on HM3D OGN and 56.5% on MP3D OGN, surpassing UniGoal by 24.0% and 15.5%; it reaches 40.2% and 38.9% SR on MP3D OVON unseen and seen validation sets, 71.4% SR on IIN, and 38.5% SR with 53.1% SPL on VLN-CE R2R.
- On A-EQA, BSC-Nav achieves the highest LLM-Match score of 54.6; in real-world tests across 75 navigation episodes, it reaches at least 3/5 successful trials per target, with IIN reaching 100% SR on 4/5 targets, final distance below 2.5 m in all cases, and mean velocity 0.76 m/s.
- In a real-world TIN ablation, the landmark-memory-only variant achieved 52% success versus 78% for full BSC-Nav.
Methods
- BSC-Nav constructs structured spatial memory via a landmark memory module, a cognitive map module that voxelizes egocentric trajectories into allocentric representations, and a working memory module for hierarchical retrieval and planning; it uses DINOv2, YOLO-World, CLIP, Stable Diffusion 3.5, and GPT-4/GPT-4o/o3-mini.
- Simulation evaluation covers 8195 episodes in 62 indoor scenes from MP3D and HM3D in Habitat across OGN, OVON, TIN, and IIN, plus VLN-CE R2R with 1000 episodes and OpenEQA A-EQA with 184 questions; metrics are SR, SPL, and LLM-Match.
- Real-world deployment uses a custom mobile robotic platform in a two-story indoor space of around 200 m², with 75 navigation episodes across OGN, TIN, and IIN, and demonstrations of language-guided mobile manipulation.
Limitations
- The paper states that BSC-Nav is not intended as a cognitively faithful model of human cognition; MLLMs are used as engineering modules, and the theoretical contribution remains primarily architectural and computational, with more formal analyses needed.
- The paper states scalability challenges: compact environments require representational economy, while larger or open-ended environments face memory capacity, map consistency, and retrieval efficiency bottlenecks; dynamic environments require continual memory revision.
- The text makes evident additional constraints: real-world memory construction relies on manual teleoperation, real-world initial pose is manually specified, simulation uses ground-truth odometry, real-world evaluation is limited to one 200 m² environment with 75 episodes, and the surprise-driven update is explicitly a heuristic rather than a formal free-energy optimization.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures






Verification
Claims the paper makes
15 Valid · 5 Not valid · 24 Not checked · 117 Own results and interpretations
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“Most existing methods, whether based on end-to-end reinforcement learning18–20 or modular pipelines with powerful MLLMs21,22, process observations in a reactive and stateless manner.”
§ Brain-inspired spatial intelligence for embodied agents > Introduction
What the sources say
- Disputedcites [18] Chaplot, D. S., Gandhi, D. P., Gupta, A. & Salakhutdinov, R. R. Object goal navigation using goal-oriented semantic exploration. Adv. Neural Inf. Process. Syst. 33, 4247 (2020). sourceExperimental — accuracy not yet measured
The Introduction (p. 1) describes end-to-end methods that use recurrent networks to build episodic memory and a modular system that maintains memory with explicit maps—contrary to calling these methods stateless. The paper does not address MLLM-based pipelines. (LLM, conf=0.88)
Checked by a reasoning model — may be wrong.
- Not matchedcites [19] Shah, D. et al. Offline reinforcement learning for visual navigation. In: Conference on Robot Learning, pp.44–54 (PMLR, 2023).
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites [20] Shah, D. et al. Vint: A foundation model for visual navigation. In: Conference on Robot Learning, pp. 711–733 (PMLR, 2023).
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites [21] OpenAI GPT-4V(ision) System Card https://cdn.openai.com/papers/GPTV_System_Card.pdf (2023).
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Could not verifycites [22] Yang, H. et al. Qwen-VL: a versatile vision-language model. arXiv preprint arXiv:2308.12966 (2023) sourceExperimental — accuracy not yet measured
The Introduction and Related Work describe Qwen-VL and other vision-language models, but do not characterize existing methods as processing observations reactively or without state. Searches for “reactive,” “stateless,” memory, and reinforcement-learning pipelines found no passage addressing that claim. (LLM, conf=0.70)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“Furthermore, they heavily depend on task-specifically trained cross-modal Transformers or policies for memory attention and planning, which inherently constrain their generalization to unseen environments.”
§ Brain-inspired spatial intelligence for embodied agents > Methods > Experimental details
What the sources say
- Disputedcites [36] Chang, M. et al. GOAT: GO to Any Thing. In Proc. Robotics: Science and Systems (2024). sourceExperimental — accuracy not yet measured
The Discussion (p. 10) says GOAT’s modular design avoids the overfitting to training scenes that can affect end-to-end methods, and reports no generalization issues across nine visually diverse homes. The Discussion (p. 8) describes its mapping and planning as classical methods paired with pretrained components, not as heavily dependent on task-specific cross-modal Transformers or learned planning policies. (LLM, conf=0.90)
Checked by a reasoning model — may be wrong.
- Not checked — no full text of the sourcecites [54] Krantz, J. et al. Navigating to objects specified by images. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10916–10925 (IEEE, 2023). source
- Not matchedcites 13,
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Despite the rapid progress in multi-modal large language models (MLLMs) and growing efforts to equip embodied agents with visual-language reasoning11–13, current artificial systems remain fundamentally limited in large-scale spatial cognition14, especially in tasks requiring long-horizon navigation and mobile manipulation.”
§ Brain-inspired spatial intelligence for embodied agents > Introduction
What the sources say
- Partly supportedcites [14] Ramakrishnan, S.K., Wijmans, E., Kraehenbuehl, P., Koltun, V. Does spatial cognition emerge in frontier models? In:International Conference on Learning Representations (ICLR, 2024). sourceExperimental — accuracy not yet measured
The Abstract (p. 1) reports that contemporary frontier models fall short on spatial-cognition tests, and the paper includes large-scale navigation tasks such as route retracing and shortcut discovery (p. 4). This supports the claim about limitations in large-scale spatial cognition, but the cited passages do not establish a specific limitation in long-horizon navigation or mobile manipulation. (LLM, conf=0.86)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Structured spatial memory is essential for guiding goal-directed behavior in complex real-world environments3–5, yet remains largely absent from current embodied agents.”
§ Brain-inspired spatial intelligence for embodied agents > Results > Construction and exploitation of structured spatial memory in embodied agents
What the sources say
- Not checked — no full text of the sourcecites [3] Bellmund, J. L., Gärdenfors, P., Moser, E. I. & Doeller, C. F. Navigating cognition: spatial codes for human thinking. Science 362, 6766 (2018). source
- Supportedcites [4] Epstein, R. A., Patai, E. Z., Julian, J. B. & Spiers, H. J. The cognitive map in humans: spatial navigation and beyond. Nat. Neurosci. 20, 1504 (2017). source
The Abstract describes a structured representation of the spatial environment that supports memory and guides future action, and says spatial codes are used to plan routes. The Introduction explains how spatial knowledge lets a navigator flexibly choose a path to a goal. The claim about current embodied agents is the citing authors’ own contrast, not a claim attributed to this source. (LLM, conf=0.88)
- Not checked — no full text of the sourcecites [5] Denis, M. & Loomis, J. M. Perspectives on human spatial cognition: memory, navigation, and environmental learning. Psychol. Res. 71, 235 (2007). source
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“In our demonstrations, natural language instructions are interpreted by GPT-464 to generate waypoint-action sequences.”
§ Brain-inspired spatial intelligence for embodied agents > Results > Real-world navigation and mobile manipulation
What the sources say
- Partly supportedcites [64] Wu, Z. et al. Embodied task planning with large language models. arXiv preprint arXiv:2307.01848 (2023). sourceExperimental — accuracy not yet measured
The Conclusion (section 5; page unavailable) says TaPA generates action steps from human instructions for robot navigation and manipulation, supporting the general instruction-to-plan idea. It says the dataset was generated with GPT-3.5, while the Experiment section (section 4; page unavailable) identifies a finetuned LLaMA-7B model as the task planner. These passages do not support the specific attribution to GPT-4 or waypoint-action sequences in BSC-Nav demonstrations. (LLM, conf=0.85)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
72 more claims · 2 with their cited sources checked
Create a free account to see all 161 claims and to verify the 2 references still open.
- Probably not a substantial claim. 51
- It may state two separate claims as one. 47
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“For the cognitive map, we set a voxel resolution of δ = 0.1 m3, a grid dimension of G = 1000, and a buffer capacity of B = 10 per voxel.”
§ Brain-inspired spatial intelligence for embodied agents > Methods > Experimental details
Reported 0.1 · recomputed 0.001
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Brain-inspired spatial intelligence for embodied agents > Methods > Experimental details
“the predefined landmark category set is reduced from 21 to 16.”
§ Brain-inspired spatial intelligence for embodied agents > Methods > Experimental details
“YOLO-World-X (YOLOv8 Extra-Large, WorldV2)73 with a confidence threshold of 0.55 across 20 predefined object categories.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Brain-inspired spatial intelligence for embodied agents > Results > Higher-level spatially-aware skills
“we equip BSC-Nav with GPT-o357, a powerful MLLM with advanced reasoning capabilities. GPT-o3 decomposes complex instructions into spatially grounded subgoals”
§ Brain-inspired spatial intelligence for embodied agents > Methods > Experimental details
“and o3-mini57 for complex instruction decomposition in long-horizon navigation tasks.”
Citations
Cites
High-Resolution Image Synthesis with Latent Diffusion Models
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01042 · Cited by 20,302 · cited 1×
“Theseenricheddescriptionsare then used to “ imagine ” potential visualappearances of targetsthrough a text-to-image generation model, defaulting to Stable Diffusion 3.5 75 .”
InfluentialA Formal Basis for the Heuristic Determination of Minimum Cost Paths
Hart, P., Nilsson, N., & Raphael, B. (1968) · IEEE Transactions on Systems Science and Cybernetics · doi:10.1109/tssc.1968.300136 · Cited by 12,738
The free-energy principle: a unified brain theory?
Friston, K. (2010) · Nature Reviews Neuroscience · doi:10.1038/nrn2787 · Cited by 8,120 · cited 3×
“Motivated by the broad idea in free-energy 37,38 and predictive-processing 39,40 accounts, that biological systems update internal representations in response to mismatches between observation and expectation, we implement a surprise-driven update strategy that selectively integrates novel or…”
Working Memory: Theories, Models, and Controversies
Baddeley, A. (2012) · Annual Review of Psychology · doi:10.1146/annurev-psych-120710-100422 · Cited by 4,271
Matterport3D: Learning from RGB-D Data in Indoor Environments
Chang, A., Dai, A., Funkhouser, T., Halber, M., Niebner, M., Savva, M., Song, S., Zeng, A., & Zhang, Y. (2017) · 2017 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2017.00081 · Cited by 1,858
The free-energy principle: a rough guide to the brain?
Friston, K. (2009) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2009.04.005 · Cited by 1,831 · cited 3×
“Motivated by the broad idea in free-energy 37,38 and predictive-processing 39,40 accounts, that biological systems update internal representations in response to mismatches between observation and expectation, we implement a surprise-driven update strategy that selectively integrates novel or…”
The Development of Spatial Representations of Large-Scale Environments
Siegel, A. W., & White, S. H. (1975) · Advances in Child Development and Behavior · doi:10.1016/s0065-2407(08)60007-5 · Cited by 1,830 · cited 3×
“Decades of neuroscience research revealed that organisms consolidate spatial knowledge into three distinct yet interconnected forms 16,17 (Fig.”
Habitat: A Platform for Embodied AI Research
Savva, M., Kadian, A., Maksymets, O., Zhao, Y., Wijmans, E., Jain, B., Straub, J., Liu, J., Koltun, V., Malik, J., et al. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00943 · Cited by 1,237 · cited 2×
“We conduct systematic evaluations across 8195 episodes in 62 indoor scenes from physically reconstructed environments (MP3D 44 and HM3D 45 datasets) in the Habitat simulator 46 , covering four representative navigation tasks (Fig.”
Overlapping mechanisms of attention and spatial working memory
Awh, E., & Jonides, J. (2001) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(00)01593-x · Cited by 1,229 · cited 2×
“Analogous to visual-spatial working memory in biological systems 28,29 , this module dynamically composes spatial representations based on task demands and environmental familiarity.”
Anderson, P., Wu, Q., Teney, D., Bruce, J., Johnson, M., Sunderhauf, N., Reid, I., Gould, S., & van den Hengel, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00387 · Cited by 1,187 · cited 3×
“We evaluate BSC-Nav on a representative task termed Long-horizon Instruction-based Navigation (LIN) 55,56 , which requires agents to understand and execute complex instructions containing multiple intermediate goals and spatialconstraints.”
Research works printed but not linked: 9
- Shah, D. et al. Offline reinforcement learning for visual navigation. In: Conference on Robot Learning, pp.44–54 (PMLR, 2023).
- Richter, K.-F. & Winter, S. Landmarks. Springer. Cham Heidelb. N. Y. Dordr. Lond. doi 10, 1 (2014).
- Oquab, M. et al. DINOv2: learning robust visual features without supervision. Trans. Mach. Learn. Res. (2024).
- Esser, P. et al. Scaling rectified flow transformers for high-resolution image synthesis. In: International Conference on Machine Learning, pp. 12606–12633 (2024). PMLR
- Kim, M.J. et al. OpenVLA: an open-source vision-language-action model. In: Conference on Robot Learning, pp. 2679–2713 (PMLR, 2025).
- Liu, S. et al. RDT-1B: a diffusion foundation model for bimanual manipulation. In Proc. International Conference on Learning Representations (2025).
- Rösmann, C., Feiten, W., Wösch, T., Hoffmann, F., Bertram, T. Trajectory modification considering dynamic constraints of autonomous robots. In: German Conference on Robotics, pp. 1–6 (GCR, 2012).
- Puig, X. et al. Habitat 3.0: a co-habitat for humans, avatars and robots. In Proc. International Conference on Learning Representations (2024).
- Radford, A. et al. Learning transferable visual models from natural language supervision. In: Proceedings of the International Conference on Machine Learning, pp. 8748–8763 (PMLR, 2021).
Cited by
PreAct-Nav: Agentic Reasoning Before Action for Urban NavigationPreprint
Xie, J., Ruan, S., Wang, Y., Zhang, Y., Yang, J., Jin, S., & Shi, D. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.04916
“Recent language and vision-language models (VLMs) have enabled navigation agents to reason explicitly about instructions, observations, and possible actions (Zhou et al., 2024; Long et al., 2024; Ruan et al., 2026).”
Self-citationActive Spatial Inspection for Effective and Efficient Embodied ExplorationPreprint
Wang, W., Bai, X., Wang, Y., Sun, H., Ren, D., Qin, J., Li, Q., & Wang, B. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.22385
“Memory-centric zero-shot methods, including 3D-Mem [1], GSMem [49], and BSC-NAV [65], maintain structured spatial or persistent 3D memories for VLM reasoning.”
Ieong, I.-T., Feng, R., Lu, Z., Cao, Y., Zhao, J., Li, L., Zhu, S., & Ding, W. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.14561
“We evaluate the resulting navigation system against a reproduced BSC-Nav baseline (Ruan et al., 2026) on the same HM3D-ObjectNav evaluation subset.”
Influential
Metadata
- Authors
- Ruan, S., Wang, L., Kang, C., Zhu, Q., Liu, S., Wei, X., & Su, H.
- Affiliations
- Beihang University (CN)
- Tsinghua University (CN)
- Corresponding author
- Wei, Xingxing
- Su, Hang
- Venue
- Nature Communications
- Published
- 2026-06-27
- Volume
- 17
- Issue
- 1
- ISSN
- 2041-1723 (ISSN-L)
- PMC
- PMC13454140
- OpenAlex
- W7166300381
- Semantic Scholar
- ea1c67338943ce4266f544b26dee255ad6963c70
- Cited by
- 1
- References
- 62
- Citations by year
- 2026: 1
- FWCI
- 1.76
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by-nc-nd
- creativecommons.org/licenses/by-nc-nd/4.0 (tdm)
- creativecommons.org/licenses/by-nc-nd/4.0 (vor, 44 d)
- Locations
- Nature Communications (publishedVersion, cc-by-nc-nd)
- Research Square (acceptedVersion, cc-by)
- PubMed (publishedVersion)
- PubMed Central (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Spatial Cognition and Navigation
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Robotics and Sensor-Based Localization
- Keywords
- spatial cognition, embodied agents, spatial memory, landmark-based navigation, route knowledge, survey knowledge, multimodal large language models, navigation, instruction following, zero-shot generalization, brain-inspired models
- MeSH terms
- Large Language Models, Artificial Intelligence, Brain, Cognition, Cues, Humans, Spatial Memory, Spatial Navigation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-01, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Brain-inspired spatial intelligence for embodied agents. https://app.cochat.ai/papers/10.1038/s41467-026-74358-5
- The paper
- Ruan, S., Wang, L., Kang, C., Zhu, Q., Liu, S., Wei, X., & Su, H. (2026). Brain-inspired spatial intelligence for embodied agents. Nature Communications. https://doi.org/10.1038/s41467-026-74358-5
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.