CoChat

Brain-inspired spatial intelligence for embodied agents

Open access

The DOI matches its Crossref record; 77 of the 77 references it cites were found in a registry or an index; 22 of the 89 sources its claims cite were checked for support.

Abstract

Spatial cognition enables adaptive goal-directed behavior through structured internal models of space. Robust biological systems consolidate spatial knowledge into three interconnected forms: landmarks for salient cues, route knowledge for movement trajectories, and survey knowledge for map-like representations. While recent advances in multi-modal large language models (MLLMs) have enabled visual-language reasoning in embodied agents, these efforts lack structured spatial memory and instead operate reactively, limiting their generalization and adaptability in complex real-world environments. Here we present Brain-inspired Spatial Cognition for Navigation (BSC-Nav), a unified framework for constructing and leveraging structured spatial memory in embodied agents. BSC-Nav builds allocentric cognitive maps from egocentric trajectories and contextual cues, and dynamically retrieves spatial knowledge aligned with semantic goals. Integrated with powerful MLLMs, BSC-Nav achieves state-of-the-art efficacy and efficiency across diverse embodied navigation tasks (e.g., improving success weighted by path length from 17.6% to 44.9% in instance-level navigation and from 42.7% to 53.1% in zero-shot long-horizon instruction-following), while also supporting versatile embodied behaviors in the real physical world. These results highlight a scalable path toward general-purpose spatial intelligence.

Overview

CoChat Summary

TL;DR

BSC-Nav is a brain-inspired framework that builds structured spatial memory—landmark, route, and survey knowledge—for embodied agents and couples it with MLLMs for goal-directed navigation. It reports state-of-the-art efficacy and efficiency across simulated and real-world navigation, instruction-following, embodied question answering, and mobile manipulation tasks.

Key findings

  • BSC-Nav improves success weighted by path length from 17.6% to 44.9% in instance-level navigation and from 42.7% to 53.1% in zero-shot long-horizon instruction-following.
  • In simulation, it achieves 78.5% SR on HM3D OGN and 56.5% on MP3D OGN, surpassing UniGoal by 24.0% and 15.5%; it reaches 40.2% and 38.9% SR on MP3D OVON unseen and seen validation sets, 71.4% SR on IIN, and 38.5% SR with 53.1% SPL on VLN-CE R2R.
  • On A-EQA, BSC-Nav achieves the highest LLM-Match score of 54.6; in real-world tests across 75 navigation episodes, it reaches at least 3/5 successful trials per target, with IIN reaching 100% SR on 4/5 targets, final distance below 2.5 m in all cases, and mean velocity 0.76 m/s.
  • In a real-world TIN ablation, the landmark-memory-only variant achieved 52% success versus 78% for full BSC-Nav.

Methods

  • BSC-Nav constructs structured spatial memory via a landmark memory module, a cognitive map module that voxelizes egocentric trajectories into allocentric representations, and a working memory module for hierarchical retrieval and planning; it uses DINOv2, YOLO-World, CLIP, Stable Diffusion 3.5, and GPT-4/GPT-4o/o3-mini.
  • Simulation evaluation covers 8195 episodes in 62 indoor scenes from MP3D and HM3D in Habitat across OGN, OVON, TIN, and IIN, plus VLN-CE R2R with 1000 episodes and OpenEQA A-EQA with 184 questions; metrics are SR, SPL, and LLM-Match.
  • Real-world deployment uses a custom mobile robotic platform in a two-story indoor space of around 200 m², with 75 navigation episodes across OGN, TIN, and IIN, and demonstrations of language-guided mobile manipulation.

Limitations

  • The paper states that BSC-Nav is not intended as a cognitively faithful model of human cognition; MLLMs are used as engineering modules, and the theoretical contribution remains primarily architectural and computational, with more formal analyses needed.
  • The paper states scalability challenges: compact environments require representational economy, while larger or open-ended environments face memory capacity, map consistency, and retrieval efficiency bottlenecks; dynamic environments require continual memory revision.
  • The text makes evident additional constraints: real-world memory construction relies on manual teleoperation, real-world initial pose is manually specified, simulation uses ground-truth odometry, real-world evaluation is limited to one 200 m² environment with 75 episodes, and the surprise-driven update is explicitly a heuristic rather than a formal free-energy optimization.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1038/s41467-026-74358-5

Cites

77 of 77 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Fig. 1 The BSC-Nav framework for cognitive spatial intelligence. a Structured spatial memory in biological brains, comprising landmarks, route knowledge, and survey knowledge. b The BSC-Nav framework instantiates structured spatial memory in embodied agents. Environment observations (RGB-D images and agent poses) are processed by (i) a landmark memory module encodes and retrieves durable associations of multi-modal environmental cues as the landmarks; and (ii) a cognitive map module accumulates and organizes movement trajectories as the route knowledge into allocentric, map-like representations as the survey knowledge. Upon task invocation, (iii) a working memory module dynamically composes relevant spatial knowledge for adaptive planning and reasoning. c Structured spatial memory enables not only universal navigation but also higher-level spatial-aware skills. The human and brain illustrations in (a) were adapted (with added layers and annotations) from SciDraw artwork, licensed under CC-BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Sources: 10.5281/zenodo.3925989,10.5281/zenodo.3926189.
Fig. 1 The BSC-Nav framework for cognitive spatial intelligence. a Structured spatial memory in biological brains, comprising landmarks, route knowledge, and survey knowledge. b The BSC-Nav framework instantiates structured spatial memory in embodied agents. Environment observations (RGB-D images and agent poses) are processed by (i) a landmark memory module encodes and retrieves durable associations of multi-modal environmental cues as the landmarks; and (ii) a cognitive map module accumulates and organizes movement trajectories as the route knowledge into allocentric, map-like representations as the survey knowledge. Upon task invocation, (iii) a working memory module dynamically composes relevant spatial knowledge for adaptive planning and reasoning. c Structured spatial memory enables not only universal navigation but also higher-level spatial-aware skills. The human and brain illustrations in (a) were adapted (with added layers and annotations) from SciDraw artwork, licensed under CC-BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Sources: 10.5281/zenodo.3925989,10.5281/zenodo.3926189.CC BY-NC-ND · PubMed Central
Fig. 2 Precise localization via hierarchical retrieval in working memory. a For simple category-level goals, working memory prioritizes retrieval from the landmark memory module for rapid matching and candidate coordinate generation. b For complex instance-level goals, working memory employs association-enhanced retrieval, converting text instructions into visual features to query the cognitive map module. For image goals, the cognitive map is queried directly using extracted visual features. A more detailed illustration of candidate generation, ranking, and verification is provided in Supplementary Fig. 2.
Fig. 2 Precise localization via hierarchical retrieval in working memory. a For simple category-level goals, working memory prioritizes retrieval from the landmark memory module for rapid matching and candidate coordinate generation. b For complex instance-level goals, working memory employs association-enhanced retrieval, converting text instructions into visual features to query the cognitive map module. For image goals, the cognitive map is queried directly using extracted visual features. A more detailed illustration of candidate generation, ranking, and verification is provided in Supplementary Fig. 2.CC BY-NC-ND · PubMed Central
Fig. 3 Goal-oriented multi-modal navigation. a Performance comparison against baseline methods across different navigation tasks. Evaluations encompass category-level tasks, including object goal navigation (OGN) and open-vocabulary object navigation (OVON), as well as instance-level tasks, including text instance navigation (TIN) and image instance navigation (IIN). Quantitative results are averaged across different trial numbers: 2195 and 2000 episodes for OGN on HM3D and MP3D, respectively; 1000 episodes each for OVON, TIN, and IIN. For each task, efficacy and efficiency are quantified by Success Rate (SR, %) and Success weighted by Path Length (SPL, %), respectively, as indicated by the paired bar charts. Source data are provided as a Source Data file. b Visualization of navigation trajectories executed by BSC-Nav, illustrating the agent’s egocentric observations and the target verification process facilitated by MLLM interactions.
Fig. 3 Goal-oriented multi-modal navigation. a Performance comparison against baseline methods across different navigation tasks. Evaluations encompass category-level tasks, including object goal navigation (OGN) and open-vocabulary object navigation (OVON), as well as instance-level tasks, including text instance navigation (TIN) and image instance navigation (IIN). Quantitative results are averaged across different trial numbers: 2195 and 2000 episodes for OGN on HM3D and MP3D, respectively; 1000 episodes each for OVON, TIN, and IIN. For each task, efficacy and efficiency are quantified by Success Rate (SR, %) and Success weighted by Path Length (SPL, %), respectively, as indicated by the paired bar charts. Source data are provided as a Source Data file. b Visualization of navigation trajectories executed by BSC-Nav, illustrating the agent’s egocentric observations and the target verification process facilitated by MLLM interactions.CC BY-NC-ND · PubMed Central
Fig. 4 Higher-level spatially-aware skills enabled by BSC-Nav. a Comparison of BSC-Nav against baseline methods on long-horizon instruction-following tasks using the VLN-CE R2R benchmark. Quantitative results are averaged over 1000 episodes. Bar charts report Success Rate (SR, %) to quantify efficacy and Success weighted by Path Length (SPL, %) to quantify efficiency. Source data are provided as a Source Data file. b Performance comparison on active embodied question answering tasks using the A-EQA benchmark, with results averaged over 184 test samples and evaluated by the LLM-match score. The right panel provides a fine-grained performance breakdown of BSC-Nav, baselines, and human references across seven EQA question categories: object recognition (OR), object localization (OL), attribute recognition (AR), spatial understanding (SU), object state recognition (OSR), functional reasoning (FR), and world knowledge (WK). Source data are provided as a Source Data file. c,d Representative visualizations of BSC-Nav’s navigation trajectories during human instruction navigation (c) and embodied question answering (d) tasks.
Fig. 4 Higher-level spatially-aware skills enabled by BSC-Nav. a Comparison of BSC-Nav against baseline methods on long-horizon instruction-following tasks using the VLN-CE R2R benchmark. Quantitative results are averaged over 1000 episodes. Bar charts report Success Rate (SR, %) to quantify efficacy and Success weighted by Path Length (SPL, %) to quantify efficiency. Source data are provided as a Source Data file. b Performance comparison on active embodied question answering tasks using the A-EQA benchmark, with results averaged over 184 test samples and evaluated by the LLM-match score. The right panel provides a fine-grained performance breakdown of BSC-Nav, baselines, and human references across seven EQA question categories: object recognition (OR), object localization (OL), attribute recognition (AR), spatial understanding (SU), object state recognition (OSR), functional reasoning (FR), and world knowledge (WK). Source data are provided as a Source Data file. c,d Representative visualizations of BSC-Nav’s navigation trajectories during human instruction navigation (c) and embodied question answering (d) tasks.CC BY-NC-ND · PubMed Central
Fig. 5 Universal navigation capabilities in real-world scenarios. a The custom-built robotic platform integrates perception, navigation, and manipulation capabilities. b The indoor experimental environment spans 200 m2 and includes diverse functional zones (e.g., office, lounge, reception, and kitchen). c–g Real-world navigation performance across Object-Goal Navigation (OGN), Text-Instance Navigation (TIN), and Image-Instance Navigation (IIN) tasks with 15 distinct targets (c). For each target, we conducted n = 5 independent experimental trials with randomized starting positions. We report the success rate (SR) (d), final distance to goal (e), and mean navigation velocity (f). In (e) and (f), data are presented as box plots with individual data points overlaid. For all box plots, the center line indicates the median, the lower and upper bounds of the box represent the 25th and 75th percentiles, respectively, and the whiskers extend to the minimum and maximum values. Source data are provided as a Source Data file. g Representative examples of real-world navigation. Each shows a top-down trajectory with timestamps (left) and corresponding egocentric-allocentric views (right).
Fig. 5 Universal navigation capabilities in real-world scenarios. a The custom-built robotic platform integrates perception, navigation, and manipulation capabilities. b The indoor experimental environment spans 200 m2 and includes diverse functional zones (e.g., office, lounge, reception, and kitchen). c–g Real-world navigation performance across Object-Goal Navigation (OGN), Text-Instance Navigation (TIN), and Image-Instance Navigation (IIN) tasks with 15 distinct targets (c). For each target, we conducted n = 5 independent experimental trials with randomized starting positions. We report the success rate (SR) (d), final distance to goal (e), and mean navigation velocity (f). In (e) and (f), data are presented as box plots with individual data points overlaid. For all box plots, the center line indicates the median, the lower and upper bounds of the box represent the 25th and 75th percentiles, respectively, and the whiskers extend to the minimum and maximum values. Source data are provided as a Source Data file. g Representative examples of real-world navigation. Each shows a top-down trajectory with timestamps (left) and corresponding egocentric-allocentric views (right).CC BY-NC-ND · PubMed Central
Fig. 6 Real-world mobile manipulation. The coordination between spatial memory-driven navigation and manipulation primitives enables the execution of long-horizon tasks specified through human instructions. a Single-waypoint task, cleaning stains on a marble table next to a shredder. b Object transfer task, relocating a cookie box from table to the kitchen island, requiring navigation between manipulation actions. c Complex multi-step task, preparing breakfast by sequentially navigating to and manipulating three open-vocabulary targets (oatmeal jar, coco balls, and milk bottle) to assemble ingredients on a plate. Full demonstrations are provided in Supplementary Movies 8--10.
Fig. 6 Real-world mobile manipulation. The coordination between spatial memory-driven navigation and manipulation primitives enables the execution of long-horizon tasks specified through human instructions. a Single-waypoint task, cleaning stains on a marble table next to a shredder. b Object transfer task, relocating a cookie box from table to the kitchen island, requiring navigation between manipulation actions. c Complex multi-step task, preparing breakfast by sequentially navigating to and manipulating three open-vocabulary targets (oatmeal jar, coco balls, and milk bottle) to assemble ingredients on a plate. Full demonstrations are provided in Supplementary Movies 8--10.CC BY-NC-ND · PubMed Central

Verification

Claims the paper makes

15 Valid · 5 Not valid · 24 Not checked · 117 Own results and interpretations

72 more claims · 2 with their cited sources checked

Create a free account to see all 161 claims and to verify the 2 references still open.

See all claims — free account
  • Probably not a substantial claim. 51
  • It may state two separate claims as one. 47

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

3 of 3

Citations

Cites

Show all 77 references

Research works printed but not linked: 9

  1. Shah, D. et al. Offline reinforcement learning for visual navigation. In: Conference on Robot Learning, pp.44–54 (PMLR, 2023).
  2. Richter, K.-F. & Winter, S. Landmarks. Springer. Cham Heidelb. N. Y. Dordr. Lond. doi 10, 1 (2014).
  3. Oquab, M. et al. DINOv2: learning robust visual features without supervision. Trans. Mach. Learn. Res. (2024).
  4. Esser, P. et al. Scaling rectified flow transformers for high-resolution image synthesis. In: International Conference on Machine Learning, pp. 12606–12633 (2024). PMLR
  5. Kim, M.J. et al. OpenVLA: an open-source vision-language-action model. In: Conference on Robot Learning, pp. 2679–2713 (PMLR, 2025).
  6. Liu, S. et al. RDT-1B: a diffusion foundation model for bimanual manipulation. In Proc. International Conference on Learning Representations (2025).
  7. Rösmann, C., Feiten, W., Wösch, T., Hoffmann, F., Bertram, T. Trajectory modification considering dynamic constraints of autonomous robots. In: German Conference on Robotics, pp. 1–6 (GCR, 2012).
  8. Puig, X. et al. Habitat 3.0: a co-habitat for humans, avatars and robots. In Proc. International Conference on Learning Representations (2024).
  9. Radford, A. et al. Learning transferable visual models from natural language supervision. In: Proceedings of the International Conference on Machine Learning, pp. 8748–8763 (PMLR, 2021).

Cited by

See all 3

Metadata

Authors
Ruan, S., Wang, L., Kang, C., Zhu, Q., Liu, S., Wei, X., & Su, H.
Affiliations
  • Beihang University (CN)
  • Tsinghua University (CN)
Corresponding author
  • Wei, Xingxing
  • Su, Hang
Venue
Nature Communications
Published
2026-06-27
Volume
17
Issue
1
ISSN
2041-1723 (ISSN-L)
OpenAlex
W7166300381
Cited by
1
References
62
Citations by year
2026: 1
FWCI
1.76
Type
journal article
Language
en
Open access
Licence
Locations
  • Nature Communications (publishedVersion, cc-by-nc-nd)
  • Research Square (acceptedVersion, cc-by)
  • PubMed (publishedVersion)
  • PubMed Central (publishedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
Topics
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Spatial Cognition and Navigation
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Robotics and Sensor-Based Localization
Keywords
spatial cognition, embodied agents, spatial memory, landmark-based navigation, route knowledge, survey knowledge, multimodal large language models, navigation, instruction following, zero-shot generalization, brain-inspired models
MeSH terms
Large Language Models, Artificial Intelligence, Brain, Cognition, Cues, Humans, Spatial Memory, Spatial Navigation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-01, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Brain-inspired spatial intelligence for embodied agents. https://app.cochat.ai/papers/10.1038/s41467-026-74358-5
The paper
Ruan, S., Wang, L., Kang, C., Zhu, Q., Liu, S., Wei, X., & Su, H. (2026). Brain-inspired spatial intelligence for embodied agents. Nature Communications. https://doi.org/10.1038/s41467-026-74358-5

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.