CoChat
Sign Up
Cited by 2Multimodal Machine Learning ApplicationsAI-based Problem Solving and PlanningRobot Manipulation and Learning

3DLLM-Mem: Long-Term Spatial-Temporal Memory for Embodied 3D Large Language Model

Open access
Hu, Wenbo, Hong, Yining, Wang, Yanjun, Gao, Leison, Wei, Zibu, Yao, Xingcheng, Peng, Nanyun, Bitton, Yonatan +2 more·2025·Advances in Neural Information Processing Systems 38
DOI 10.52202/085713-2278
Open this paper in CoChat — free accountOpen PDF

Abstract

Humans excel at performing complex tasks by leveraging long-term memory across temporal and spatial experiences. In contrast, current Large Language Models (LLMs) struggle to effectively plan and act in dynamic, multi-room 3D environments. We posit that part of this limitation is due to the lack of proper 3D spatial-temporal memory modeling in LLMs. To address this, we first introduce 3DMem-Bench, a comprehensive benchmark comprising over 26,000 trajectories and 2,892 embodied tasks, question-answering and captioning, designed to evaluate an agent's ability to reason over long-term memory in 3D environments. Second, we propose 3DLLM-Mem, a novel dynamic memory management and fusion model for embodied spatial-temporal reasoning and actions in LLMs. Our model uses working memory tokens, which represents current observations, as queries to selectively attend to and fuse the most useful spatial and temporal features from episodic memory, which stores past observations and interactions. Our approach allows the agent to focus on task-relevant information while maintaining memory efficiency in complex, long-horizon environments. Experimental results demonstrate that 3DLLM-Mem achieves state-of-the-art performance across various tasks, outperforming the strongest baselines by 16.5% in success rate on 3DMem-Bench's most challenging in-the-wild embodied tasks.

AI snapshot

Written from the paper’s full text.

TL;DR

The paper introduces 3DMEM-BENCH, a benchmark for long-term spatial-temporal memory in embodied 3D environments, and 3DLLM-MEM, a memory-enhanced 3D LLM with working and episodic memory fusion. 3DLLM-MEM reports state-of-the-art results across embodied tasks, EQA, and captioning, including a 32.1% average in-the-wild embodied success rate.

Key findings

  • 3DLLM-MEM outperforms the strongest baselines by 16.5% in success rate on 3DMEM-BENCH’s most challenging in-the-wild embodied tasks, with an average in-the-wild success rate of 32.1% and sub-success rate of 55.0%.
  • On hard in-the-wild embodied tasks, existing approaches degrade to about 5% success rate, while 3DLLM-MEM maintains 27.8%.
  • On EQA and captioning, 3DLLM-MEM reports spatial 62.8, navigation 40.6, comparative 41.4, layout 39.9, count 26.3, BLEU1 58.2, BLEU4 18.8, and METEOR 37.3.
  • Ablation shows initializing the memory-fusion query with working memory tokens is best: 37.6 average in-domain SR and 32.1 in-the-wild SR, versus 32.4/22.5 for most-recent episodic memory initialization and 32.9/23.3 for learnable zero parameters.

Methods

  • 3DMEM-BENCH is built on HM3D semantics, filtered to 182 3D spaces and 2,602 rooms, with interactive objects added from Objaverse; it includes embodied tasks, long-term memory EQA, and captioning, with simple/medium/hard settings using 3, 5, and 10 multi-room scenes.
  • The paper reports collecting 51K trajectories (31K simple, 10K medium, 10K hard), while the benchmark/table reports 26,276 train trajectories and 1,860 test tasks; the abstract also mentions over 26,000 trajectories and 2,892 embodied tasks.
  • 3DLLM-MEM builds on LLaVA-3D, expands context to 8192 tokens, stores past observations in an episodic memory bank, and uses working memory tokens as queries in a memory-fusion attention module; training uses supervised fine-tuning on 8 Google Cloud TPU v5p cores with batch size 256.

Limitations

  • The paper states that 3DLLM-MEM does not involve low-level navigation and control policy, instead using high-level pre-defined simulator policies for actions.
  • The paper also notes inherited risks from LLaMA, Vicuna, and CLIP, including hallucination and biases.
  • Evident from the text: benchmark trajectories and QA/captioning data are generated by prompting Gemini, with automatic validation yielding about a 24% validation rate and additional manual review by four student experts; 3D-Mem is evaluated only on EQA and captioning because it does not support embodied interaction or action execution.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.52202/085713-2278

Cites

0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

11 of 24 cited sources checked · 3 supported · 0 flagged · accuracy not yet measured

  • Premisecitation

    Existing objects in HM3D scene are not interactive in Habitat-sim.

    “However, existing objects in HM3D scene are not interactive in Habitat-sim (Szot et al., 2021).”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Szot et al., 2021 source

      The simulator comparison table describes earlier Habitat as using 3D scans with no physics support, consistent with scanned objects not being interactive. But the paper does not identify HM3D scenes or specifically state that their existing objects are non-interactive in Habitat-sim. (LLM, conf=0.82)

      Experimental — accuracy not yet measured
  • Premisecitation

    Most existing approaches construct an experience pool or memory bank and focus on improving the retrieval of useful past information.

    “Most existing approaches construct an experience pool or memory bank and focus on improving the retrieval of useful past information (Zhao et al., 2024; Gao et al., 2024; Xu et al., 2025b).”
    Quote in the paperSubstantial claim2 of 3 sources checked
    • cites Zhao et al., 2024 source

      Section 4.2 describes ExpeL’s experience pool and its retrieval of relevant successful trajectories. But the paper does not establish that most existing approaches use an experience pool or memory bank and focus on improving retrieval; ExpeL itself also emphasizes extracting insights from experiences. The indexed passages provide section labels but no page numbers. (LLM, conf=0.83)

      Experimental — accuracy not yet measured
    • cites Gao et al., 2024 source

      Section 3.1 describes an experience memory, and Section 3.3 describes retrieving similar tasks from it, so the paper is an example of an approach that stores and retrieves past experience. It does not establish that most existing approaches do this or that improving retrieval is their main focus; its own framework also emphasizes experience transfer and induction. The supplied passages have no page numbers. (LLM, conf=0.84)

      Experimental — accuracy not yet measured
    • cites Xu et al., 2025b

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Premisecitation

    Some existing benchmarks focus on high-level planning tasks with short trajectories that can often be completed within single-room settings, thereby requiring minimal spatial-temporal memory.

    “Grounded in these environments, some existing benchmarks focus on high-level planning tasks, typically involving short trajectories that can often be completed within single-room settings, thereby requiring minimal spatial-temporal memory (Shridhar et al., 2020, 2021; Li et al., 2024a; Szot et al., 2024; Li et al., 2024b; Yang et al., 2025a).”
    Quote in the paperSubstantial claim1 of 3 sources checked
    • cites Shridhar et al., 2020 source

      Section 3.1 reports that ALFRED demonstrations average 50 action steps, and the Vision & Language Task Completion section contrasts ALFRED’s complex, partially observable tasks with simpler existing benchmarks. This supports the comparison with shorter, less complex tasks. The paper does not establish that those benchmarks can often be completed in a single room or require minimal spatial-temporal memory. (LLM, conf=0.82)

      Experimental — accuracy not yet measured
    • cites Li et al., 2024a

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Szot et al., 2024

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Findingown data

    3DLLM-MEM outperforms baselines by a large margin on embodied tasks, question-answering, and captioning.

    “Experimental results on embodied tasks, question-answering, and captioning demonstrate that 3DLLM-MEM outperforms baselines by a large margin.”
    Quote in the paperSubstantial claim
  • Findingreasoning

    3D-Mem falls short on spatial relation, navigation and object counting tasks.

    “it falls short on spatial relation, navigation and object counting tasks”
    Quote in the paperSubstantial claim
  • Classified as probably not a substantial claim. 40
  • May combine two claims. 32
  • The quote only partly states this claim. 2

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    LLaVA-3D directly builds on 2D-LLM with multi-view images as input and utilizing the 3D position embeddings to bring the 2D patches within a 3D spatial context to construct 3D patches.

    “LLaVA-3D directly builds on 2D-LLM with multi-view images as input and utilizing the 3D position embeddings to bring the 2D patches within a 3D spatial context to construct 3D patches.”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Zhu et al., 2024 source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Long-term memory can be categorized into explicit memory and implicit memory.

    “Long-term memory (Camina and Güell, 2017; Friedman et al., 2018; Zlotnik and Vansintjan, 2019) can be categorized into explicit memory and implicit memory.”
    Quote in the paperNot a substantial claim1 of 3 sources checked
    • cites Camina and Güell, 2017 source
    • cites Friedman et al., 2018

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Zlotnik and Vansintjan, 2019

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    HM3D semantics dataset has 1000 3D spaces and 10,600 rooms.

    “We build our scenes on top of the Habitat-Matterport 3D (HM3D) semantics dataset (Ramakrishnan et al., 2021), which has 1000 3D spaces and 10,600 rooms within those spaces.”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Ramakrishnan et al., 2021 source

      The Introduction (p. 2) reports 1,000 building-scale reconstructions and more than 10,600 rooms, supporting the approximate counts. But the Limitations section (p. 13) says this HM3D release does not include semantic annotations, so the cited paper does not support calling it the HM3D semantics dataset. (LLM, conf=0.90)

      Experimental — accuracy not yet measured

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Some EQA benchmarks include embodied memory QA evaluation, such as OpenEQA, which includes an episodic memory QA split, and Yang et al., which focuses on spatial memory QA.

    “Some EQA benchmarks also include embodied memory QA evaluation, such as OpenEQA (Majumdar et al., 2024), which includes an episodic memory QA split, and Yang et al. ( 2024), which focuses on spatial memory QA.”
    Quote in the paperNot a substantial claim1 of 2 sources checked
    • cites Majumdar et al., 2024 source
    • cites Yang et al. ( 2024)

  • May combine two claims.

    Premisecitation

    Other benchmarks emphasize long-term scene exploration with extended trajectories, but are primarily centered around navigation tasks and often lack embodied interaction support.

    “Other benchmarks emphasize long-term scene exploration with extended trajectories, but are primarily centered around navigation tasks and often lack embodied interaction support (Deitke et al., 2020; Ramakrishnan et al., 2021; Krantz et al., 2022; Khanna et al., 2024).”
    Quote in the paperSubstantial claim3 of 3 sources checked
    • cites Deitke et al., 2020 source

      The Visual Semantic Navigation section (p. 5) describes a navigation benchmark in which agents explore until they find a target; human trajectories had a median of 49.5 steps. Its listed benchmark actions are movement, looking, and ending an episode, which supports the navigation-centered characterization. But the same section says RoboTHOR enables other embodied tasks, including task completion, so the broader claim that the platform lacks embodied interaction support is not established. (LLM, conf=0.85)

      Experimental — accuracy not yet measured
    • cites Ramakrishnan et al., 2021 source

      The Experimental setup (p. 7) benchmarks PointGoal navigation with movement-only actions, and the Limitations (p. 13) say HM3D supports only geometric tasks in static environments. The Scale comparison (p. 5) describes large, navigation-complex scenes, but the paper does not establish that HM3D emphasizes long-term scene exploration or extended trajectories. (LLM, conf=0.86)

      Experimental — accuracy not yet measured
    • cites Krantz et al., 2022 source

      The Abstract (p. 1) identifies this as a navigation benchmark in an unfamiliar environment, and the benchmark description (p. 5) limits actions to moving, turning, and stopping, supporting the navigation focus and lack of object-interaction actions. The cited passages do not establish that the benchmark emphasizes long-term scene exploration or extended trajectories. (LLM, conf=0.86)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Findingnone given

    3DLLM-MEM achieves state-of-the-art performance across various tasks, outperforming the strongest baselines by 16.5% in success rate on 3DMEM-BENCH's most challenging in-the-wild embodied tasks.

    “demonstrate that 3DLLM-MEM achieves state-of-the-art performance across various tasks, outperforming the strongest baselines by 16.5% in success rate on 3DMEM-BENCH's most challenging in-the-wild embodied tasks.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Findingown data

    3DMEM-BENCH includes three evaluation categories: embodied tasks, spatial-temporal EQA, and long-term scene captioning.

    “The benchmark includes three evaluation categories: (1) embodied tasks requiring extended action sequences across multiple rooms, (2) spatial-temporal embodied question answering (EQA) that evaluates understanding of spatial relationships over time, and (3) long-term scene captioning that tests memorization of previously observed environments.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    3DMEM-BENCH includes 26,000+ trajectory examples spanning 182+ unique scenes with an average of 18 rooms per scene.

    “Our dataset includes 26,000+ trajectory examples spanning 182+ unique scenes with an average of 18 rooms per scene.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingreasoning

    We filter HM3D to 182 3D spaces and 2,602 rooms.

    “we filter to 182 3D spaces and 2,602 rooms.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    We collect 51K trajectories, with 31K in the simple setting, 10K in the medium, and 10K in the hard.

    “In total, we collect 51K trajectories, with 31K in the simple setting, 10K in the medium, and 10K in the hard.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    3DLLM-MEM significantly outperforms both Most Recent Memory and RAG Memory.

    “3DLLM-MEM significantly outperforms both Most Recent Memory and RAG Memory”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    3DLLM-MEM is capable of maintaining long-term memory and executing complex tasks in embodied environments.

    “demonstrating that 3DLLM-MEM is capable of maintaining long-term memory and executing complex tasks in embodied environments.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Findingown data

    3DMEM-BENCH defines fine-grained embodied tasks across varying levels of difficulty from simple to hard.

    “we define fine-grained embodied tasks across varying levels of difficulty-from simple to hard”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Both Most Recent Memory and Retrieval-Augmented Memory (RAG) baselines perform poorly in this setting, with RAG showing only a slight improvement.

    “Both the Most Recent Memory and Retrieval-Augmented Memory (RAG) baselines perform poorly in this setting, with RAG showing only a slight improvement”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingreasoning

    3DLLM-MEM can maintain a long-term memory and perform complex tasks in embodied environments.

    “It demonstrates that 3DLLM-MEM can maintain a long-term memory and perform complex tasks in the embodied environments.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingreasoning

    Using the most recent memory for fusion query initialization outperforms zero initialization in the simple setting but underperforms it in the hard setting.

    “Interestingly, using the most recent memory outperforms zero initialization in the simple setting but underperforms in the hard setting.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    In the example, the agent successfully arranged the flower vase, box, and printer from small to large on the desk.

    “The objects are now arranged from small to large: flower vase(0), box(0) and printer(0) on the desk(0). Task Complete.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). The quote only partly states this claim.

    Findingown data

    In the in-domain example, the agent successfully completed the task of preparing for cooking stir-fried tomato and scrambled eggs.

    “The setup in kitchen has been prepared.Task Complete.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The model was trained on a training data split containing about 26K trajectories.

    “We train our model on our training data split which contains about 26K trajectories.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The filtering process yields a validation rate of approximately 24%, ensuring the correctness and feasibility of the generated trajectories.

    “On average, our filtering process yields a validation rate of approximately 24%, ensuring the correctness and feasibility of the generated trajectories.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    The fine-grained complexity of 3DMEM-BENCH's embodied tasks spans from simple to medium to hard.

    “Fine-grained complexity indicates our embodied task spans from simple to medium to hard.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    The training time for 3DLLM-MEM is less than one day, making energy consumption not a primary concern.

    “The training time only takes less than one day, which makes energy consumption not a primary concern.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Analysis of the 3D-LLM model evaluation output indicated that the primary struggle was retaining long-term memory of semantic observations in the scene, so the authors prioritized aligning 3D-LLM with high-level long-term memory representation over low-level spatial understanding.

    “Analysis of the 3D-LLM model evaluation output, indicated the primary struggle for the model was retaining long term memory of semantic observations in the scene, so we prioritized aligning 3D-LLM with the high level long-term memory representation in our data over low level spatial understanding of the scene.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The automatic check involves re-running the trajectory simulation validation pipeline.

    “The automatic check involves re-running the trajectory simulation validation pipeline”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    3DLLM-MEM demonstrates scalability and effectiveness in managing longer-term memory representations.

    “demonstrating its scalability and effectiveness in managing longer-term memory representations.”
    • Also stated in: § Results on embodied tasks As shown in · boosted
      “demonstrating its scalability and effectiveness in managing longer-term memory representations.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationreasoning

    The fusion process preserves critical spatial relationships while accounting for their evolvement through agent interactions over time.

    “The fusion process preserves critical spatial relationships while accounting for their evolvement through agent interactions over time.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Objaverse consists of 800K 3D objects spanning rich categories.

    “we add interactive objects from Objaverse (Deitke et al., 2023) which consists of 800K 3D objects spanning rich categories.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Deitke et al., 2023 source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    Four student experts manually inspect each benchmark example.

    “four student experts in the field manually inspect each benchmark example.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    We render multi-view images of the entire scene using the simulator and verify whether the benchmark annotations accurately correspond to the simulated environment.

    “We render multi-view images of the entire scene using the simulator and verify whether the benchmark annotations accurately correspond to the simulated environment.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationreasoning

    This allows the model to leverage the benefits of dense 3D representations while mitigating their computational demands.

    “This allows the model to leverage the benefits of dense 3D representations while mitigating their computational demands”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    We implement two quality control procedures: automatic validation using trajectory simulation rules and a manual review of each benchmark instance.

    “we implement two quality control procedures: automatic validation using trajectory simulation rules and a manual review of each benchmark instance.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    The memory bank remains dynamic and reflects the latest state of the explored environments.

    “Thus, the memory bank remains dynamic and reflects the latest state of the explored environments.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    3DLLM-MEM allows the agent to focus on task-relevant information while maintaining memory efficiency in complex, long-horizon environments.

    “Our approach allows the agent to focus on task-relevant information while maintaining memory efficiency in complex, long-horizon environments.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    The long-term memory EQA tasks in 3DMEM-BENCH are designed to require reasoning over multiple pieces of memory and their changes across time and space.

    “For specific comparison on EQA, our long-term memory EQA tasks are designed to require reasoning over multiple”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    The longer task data input required truncation to fit within the 512 token context length of 3D-LLM's t5-flanxl backbone.

    “Our longer task data input also required truncation to fit within the 512 token context length of 3D-LLM's t5-flanxl backbone.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductreasoning

    Due to compute limitations, training used 15 epochs for captioning, 20 epochs for question-answering, and 75 epochs for the embodied task.

    “Due to compute limitations, we trained on captioning task for 15 epochs, question-answering task for 20 epochs, and allocated most of the compute time on the embodied task, which we trained on for 75 epochs.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    In its native setting, the 3D-Mem agent navigates an unfamiliar scene by selecting the frontier view most likely to advance its task and then answers visual questions using the most relevant memory snapshots.

    “In its native setting the agent navigates an unfamiliar scene by selecting the frontier view most likely to advance its task and then answers visual questions using the most relevant memory snapshots.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Yang et al., 2025b

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    Rooms for which both floor and ceiling surfaces are absent are discarded because no reliable vertical evidence is present.

    “Rooms for which both surfaces are absent are discarded because no reliable vertical evidence is present.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: To support navigation-centric interaction, the agent requires precise knowledge of the traversable layout of each scene and the exact locations of all movable objects.

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductreasoning

    Because the evaluation focuses on post-exploration reasoning rather than active exploration, the frontier component of 3D-Mem is disabled and only the memory snapshots are retained.

    “Because our evaluation focuses on post-exploration reasoning rather than active exploration, we disable the frontier component and retain only the memory snapshots.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductown data

    The trajectory simulation pipeline validates each command based on the agent's location, the referenced object, and the correctness of pick-up and put-down actions.

    “The pipeline validates each command based on these criteria: (1) the agent's location; (2) the referenced object and (3) the correctness of pick-up and put-down actions.”
    • Also stated in: § Room 8 Room 9 · plain
      “At every step, the simulator checks: (1) the correctness of the agent's location, (2) the existence and validity of referenced objects, and (3) the correctness of pick-up and put-down actions.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    After automatic trajectory validation, four student experts in the field manually inspect each benchmark example for human validation.

    “After automatic trajectory validation, we further conduct human validation, in which four student experts in the field manually inspect each benchmark example.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

    Interpretationreasoning

    The deployment and release of 3DLLM-MEM carry both potential benefits and risks.

    “The deployment and release of 3DLLM-MEM carry both potential benefits and risks.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationnone given

    Dense 3D memory representations are critical for real-world embodied scenarios, where task execution depends heavily on maintaining and reasoning over long-term spatial-temporal memory.

    “In contrast, our work focuses on dense 3D memory representations that are critical for real-world embodied scenarios, where task execution depends heavily on maintaining and reasoning over long-term spatial-temporal memory.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: Existing models face significant challenges when performing long-horizon embodied tasks in densely populated 3D environments that require reasoning over long-term spatial-temporal memory.

  • May combine two claims.

    Interpretationnone given

    3DLLM-MEM employs a memory fusion mechanism that efficiently retrieves and learns task-relevant information, resulting in enhanced performance on complex embodied tasks.

    “Our model employs a memory fusion mechanism that efficiently retrieves and learns task-relevant information, resulting in enhanced performance on complex embodied tasks.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: Existing models face significant challenges when performing long-horizon embodied tasks in densely populated 3D environments that require reasoning over long-term spatial-temporal memory.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    Biases in 3DLLM-MEM may result in biased outcomes or unfair representations of diverse content.

    “This may result in biased outcomes or unfair representations of diverse content.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Recent work such as 3D-Mem has investigated 3D scene memory for exploration and reasoning by prompting vision-language models.

    “Recent work such as 3D-Mem (Yang et al., 2025b) has investigated 3D scene memory for exploration and reasoning by prompting vision-language models.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Yang et al., 2025b

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    3DLLM-MEM might produce outputs that are not based on factual information or input data.

    “3DLLM-MEM might produce outputs that are not based on factual information or input data.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning

    Biases present in the base models can be brought to 3DLLM-MEM, stemming from both the vision encoder (CLIP) and the language decoder (LLaMA / Vicuna).

    “Biases present in the base models can be brought to 3DLLM-MEM, stemming from both the vision encoder (CLIP) and the language decoder (LLaMA / Vicuna).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationnone given

    3DMEM-BENCH jointly targets both spatial and episodic memory, especially their changes over time, while also supporting embodied action tasks, EQA and captioning.

    “In contrast, our benchmark, 3DMEM-BENCH jointly targets both spatial and episodic memory, especially their changes over time, while also supporting embodied action tasks, EQA and captioning.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductcitation

    High-level actions can be executed in the simulator.

    “Finally, we ensure that high-level actions can be executed in the simulator, following (Szot et al., 2024; Yang et al., 2025a).”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites Szot et al., 2024

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Yang et al., 2025a

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    3D-Mem is a snapshot-based 3D memory architecture originally developed for embodied exploration and reasoning; it keeps two complementary stores: memory snapshots and frontier snapshots.

    “3D-Mem is a snapshot-based 3D memory architecture originally developed for embodied exploration and reasoning; it keeps two complementary stores-memory snapshots, a compact set of multi-view RGB-D frames with per-object bounding boxes summarizing the areas the agent has inspected, and frontier snapshots, boundary views that suggest where useful new information may be found next.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Yang et al., 2025b

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Premisenone given

    3DLLM-MEM implements a dual-memory system consisting of a limited-capacity working memory for current observations and an expandable episodic memory.

    “3DLLM-MEM implements a dual-memory system: a limited-capacity working memory for current observations and an expandable episodicComparison with related benchmarks.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationown data

    3DMEM-BENCH is a comprehensive benchmark designed to evaluate an agent's ability to reason over long-term memory in 3D environments.

    “we first introduce 3DMEM-BENCH, a comprehensive benchmark comprising over 26,000 trajectories and 2,892 embodied tasks, question-answering and captioning, designed to evaluate an agent's ability to reason over long-term memory in 3D environments.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisenone given

    3DLLM-MEM is built on LLaMA, Vicuna, and CLIP.

    “Since 3DLLM-MEM is built on LLaMA, Vicuna, and CLIP, it inherits certain challenges associated with LLMs and vision encoders.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationreasoning

    Hallucination in 3DLLM-MEM raises concerns about the accuracy of inferences, particularly in critical applications such as medical fields.

    “This raises concerns about the accuracy of inferences, particularly in critical applications such as medical fields.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    3DLLM-MEM inherits certain challenges associated with LLMs and vision encoders.

    “it inherits certain challenges associated with LLMs and vision encoders.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductnone given

    3DLLM-MEM does not involve low-level navigation and control policy, but utilizes high-level pre-defined policies in the simulator for carrying out actions.

    “One limitation of our model is that currently 3DLLM-MEM does not involve low-level navigation and control policy, but utilizes high-level pre-defined policies in simulator for carrying out the actions.”
    Quote in the paperSubstantial claim

  • May combine two claims. The quote only partly states this claim.

    Premiseown data

    3DMEM-BENCH focuses on spatial-temporal memory through fine-grained embodied tasks and EQA that span multiple 'pieces' of long-term memory, distinguishing it from prior benchmarks that typically target single-step or short-horizon reasoning.

    “3DMEM-BENCH focus on spatial-temporal memory through fine-grained embodied tasks and EQA that span multiple”
    Quote in the paperSubstantial claim
58 of 58

  • The paper relies on this without support.

    Humans excel at performing complex tasks by leveraging long-term memory across temporal and spatial experiences.

    “Humans excel at performing complex tasks by leveraging long-term memory across temporal and spatial experiences.”

  • The paper relies on this without support.

    Current Large Language Models (LLMs) struggle to effectively plan and act in dynamic, multi-room 3D environments.

    “In contrast, current Large Language Models (LLMs) struggle to effectively plan and act in dynamic, multi-room 3D environments.”

  • The authors state this as an assumption.

    Part of the limitation of current LLMs is due to the lack of proper 3D spatial-temporal memory modeling in LLMs.

    “We posit that part of this limitation is due to the lack of proper 3D spatial-temporal memory modeling in LLMs.”

  • The paper relies on this without support.

    Real-world 3D physical scenes are remarkably vast and information-dense, where every detail can matter for long-horizon embodied tasks.

    “Real-world 3D physical scenes are remarkably vast and information-dense, where every detail can matter for long-horizon embodied tasks”

  • The paper relies on this without support.

    Dense 3D representations are particularly valuable as they capture comprehensive spatial information, preserving intricate geometric relationships and environmental details that sparse or object-centric approaches might miss.

    “Dense 3D representations are particularly valuable as they capture comprehensive spatial information, preserving intricate geometric relationships and environmental details that sparse or object-centric approaches might miss.”

  • The paper relies on this without support.

    Retrieving the entire history would overwhelm the model's context limits.

    “retrieving the entire history would overwhelm the model's context limits”

  • The paper relies on this without support.

    As environments evolve, maintaining coherent representations of previously seen spaces while incorporating new information continues to exceed the capabilities of current embodied AI models.

    “As environments evolve, maintaining coherent representations of previously seen spaces while incorporating new information continues to exceed the capabilities of current embodied AI models.”

  • The paper relies on this without support.

    For humans, current observations are held in working memory, while longer-term observations and experiences are stored in episodic memory.

    “For humans, current observations are held in working memory, while longer-term observations and experiences are stored in episodic memory.”

  • The paper relies on this without support.

    Recent work on 3D Large Language Models (3D-LLMs) has showcased robust capabilities.

    “Recent work on 3D Large Language Models (3D-LLMs) has showcased robust capabilities.”

  • The paper relies on this without support.

    Existing models face significant challenges when performing long-horizon embodied tasks in densely populated 3D environments that require reasoning over long-term spatial-temporal memory.

    “Yet, existing models face significant challenges when performing long-horizon embodied tasks in densely populated 3D environments that require reasoning over long-term spatial-temporal memory.”

  • The paper relies on this without support.

    To support navigation-centric interaction, the agent requires precise knowledge of the traversable layout of each scene and the exact locations of all movable objects.

    “To support navigation-centric interaction, the agent requires precise knowledge of two things: the traversable layout of each scene and the exact locations of all movable objects.”
11 of 11

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Stated learning rate exceeds the warmup's ending rate

    Minor

    The implementation details specify a learning rate of 2e-5, but say the warmup increases only to 1e-5 before cosine decay. As described, the schedule never reaches the stated 2e-5 rate.

1 of 1

Cites

The publisher hasn’t deposited this paper’s reference list.

Research works printed but not linked: 20

  1. Eduardo Camina and Francisco Güell. 2017. The neuroanatomical, neurophysiological and psycho- logical basis of memory: Current models and their origins. Frontiers in Pharmacology, 8:438. · DOI · Paper page
  2. Boyuan Chen, Zhuo Xu, Sean Kirmani, Brian Ichter, Dorsa Sadigh, Leonidas J. Guibas, and Fei Xia. 2024a. Spatialvlm: Endowing vision-language models with spatial reasoning capabilities. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2024, Seattle, WA, USA, June 16-22, 2024, pages 14455-14465. IEEE. · DOI · Paper page
  3. Abhishek Das, Samyak Datta, Georgia Gkioxari, Stefan Lee, Devi Parikh, and Dhruv Batra. 2018. Embodied question answering. In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018, pages 1-10. IEEE Computer Society. · DOI · Paper page
  4. Matt Deitke, Winson Han, Alvaro Herrasti, Aniruddha Kembhavi, Eric Kolve, Roozbeh Mottaghi, Jordi Salvador, Dustin Schwenk, Eli VanderBilt, Matthew Wallingford, Luca Weihs, Mark Yatskar, and Ali Farhadi. 2020. Robothor: An open simulation-to-real embodied AI platform. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020, pages 3161-3171. IEEE. · DOI · Paper page
  5. Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. 2023. Objaverse: A universe of annotated 3d objects. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023, Vancouver, BC, Canada, June 17-24, 2023, pages 13142-13153. IEEE. · DOI · Paper page
  6. Gary N Friedman, Luke Johnson, and Zachary M Williams. 2018. Long-term visual memory and its role in learning suppression. Frontiers in Psychology, 9:1896. · DOI
  7. Qiao Gu, Ali Kuwajerwala, Sacha Morin, Krishna Murthy Jatavallabhula, Bipasha Sen, Aditya Agarwal, Corban Rivera, William Paul, Kirsty Ellis, Rama Chellappa, et al. 2024. Conceptgraphs: Open-vocabulary 3d scene graphs for perception and planning. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pages 5021-5028. IEEE.
  8. Yining Hong, Chunru Lin, Yilun Du, Zhenfang Chen, Joshua B. Tenenbaum, and Chuang Gan. 2023a. 3d concept learning and reasoning from multi-view images. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023, Vancouver, BC, Canada, June 17-24, 2023, pages 9202-9212. IEEE. 15 · DOI
  9. Yining Hong, Zishuo Zheng, Peihao Chen, Yian Wang, Junyan Li, and Chuang Gan. 2024. Multiply: A multisensory object-centric embodied large language model in 3d world. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2024, Seattle, WA, USA, June 16-22, 2024, pages 26396-26406. IEEE. · DOI
  10. Jiangyong Huang, Silong Yong, Xiaojian Ma, Xiongkun Linghu, Puhao Li, Yan Wang, Qing Li, Song-Chun Zhu, Baoxiong Jia, and Siyuan Huang. 2024b. An embodied generalist agent in 3d world. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024. OpenReview.net. 2, 8 Physical Intelligence, Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Manuel Y. Galliker, Dibya Ghosh, Lachy Groom, Karol Hausman, Brian Ichter, Szymon Jakubczak, Tim Jones, Liyiming Ke, Devin LeBlanc, Sergey Levine, Adrian Li-Bell, Mohith Mothukuri, Suraj Nair, Karl Pertsch, Allen Z. Ren, Lucy Xiaoyang Shi, Laura Smith, Jost Tobias Springenberg, Kyle Stachowicz, James Tanner, Quan Vuong, Homer Walke, Anna Walling, Haohuan Wang, Lili Yu, and Ury Zhilinsky. 2025. π 0.5 : a vision-language-action model with open-world generalization.
  11. Mukul Khanna, Ram Ramrakhya, Gunjan Chhablani, Sriram Yenamandra, Théophile Gervet, Matthew Chang, Zsolt Kira, Devendra Singh Chaplot, Dhruv Batra, and Roozbeh Mottaghi. 2024. Goat-bench: A benchmark for multi-modal lifelong navigation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2024, Seattle, WA, USA, June 16-22, 2024, pages 16373-16383. IEEE. · DOI · Paper page
  12. Manling Li, Shiyu Zhao, Qineng Wang, Kangrui Wang, Yu Zhou, Sanjana Srivastava, Cem Gokmen, Tony Lee, Li Erran Li, Ruohan Zhang, Weiyu Liu, Percy Liang, Li Fei-Fei, Jiayuan Mao, and Jiajun Wu. 2024b. Embodied agent interface: Benchmarking llms for embodied decision making. In Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 -15, 2024.
  13. Arjun Majumdar, Anurag Ajay, Xiaohan Zhang, Pranav Putta, Sriram Yenamandra, Mikael Henaff, Sneha Silwal, Paul McVay, Oleksandr Maksymets, Sergio Arnaud, Karmesh Yadav, Qiyang Li, Ben Newman, Mohit Sharma, Vincent-Pierre Berges, Shiqi Zhang, Pulkit Agrawal, Yonatan Bisk, Dhruv Batra, Mrinal Kalakrishnan, Franziska Meier, Chris Paxton, Alexander Sax, and Aravind Rajeswaran. 2024. Openeqa: Embodied question answering in the era of foundation models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2024, Seattle, WA, USA, June 16-22, 2024, pages 16488-16498. IEEE. · DOI · Paper page
  14. Mohit Shridhar, Jesse Thomason, Daniel Gordon, Yonatan Bisk, Winson Han, Roozbeh Mottaghi, Luke Zettlemoyer, and Dieter Fox. 2020. ALFRED: A benchmark for interpreting grounded instructions for everyday tasks. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020, pages 10737-10746. IEEE. · DOI · Paper page
  15. Erik Wijmans, Samyak Datta, Oleksandr Maksymets, Abhishek Das, Georgia Gkioxari, Stefan Lee, Irfan Essa, Devi Parikh, and Dhruv Batra. 2019. Embodied question answering in photorealistic environments with point cloud perception. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, pages 6659-6668. Computer Vision Foundation / IEEE. · DOI
  16. Runsen Xu, Xiaolong Wang, Tai Wang, Yilun Chen, Jiangmiao Pang, and Dahua Lin. 2025a. Pointllm: Empowering large language models to understand point clouds. In Computer Vision -ECCV 2024, pages 131-147. Springer Nature Switzerland. 2, 8
  17. Licheng Yu, Xinlei Chen, Georgia Gkioxari, Mohit Bansal, Tamara L. Berg, and Dhruv Batra. 2019. Multi-target embodied question answering. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, pages 6309-6318. Computer Vision Foundation / IEEE. · DOI
  18. Andrew Zhao, Daniel Huang, Quentin Xu, Matthieu Lin, Yong-Jin Liu, and Gao Huang. 2024. Expel: LLM agents are experiential learners. In Thirty-Eighth AAAI Conference on Artificial Intelligence, AAAI 2024, Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence, IAAI 2024, Fourteenth Symposium on Educational Advances in Artificial Intelligence, EAAI 2014, February 20-27, 2024, Vancouver, Canada, pages 19632-19642. AAAI Press. · DOI · Paper page
  19. Guillermo Zlotnik and Aaron Vansintjan. 2019. Memory: An extended definition. Frontiers in Psychology, 10:2523. · DOI
  20. Xueyan Zou, Yuchen Song, Ri-Zhao Qiu, Xuanbin Peng, Jianglong Ye, Sifei Liu, and Xiaolong Wang. 2025. M3: 3d-spatial multimodal memory. In ICLR. 9 in other rooms, so I'll head back to room 8 to pick up the vase first.", "<GO TO ROOM(8)>", "I need to pick up flower vase(0) in this room first. I pick up flower vase(0).", "<PICK UP flower vase(0) from room(8) in room(8)>", "I'm carrying flower vase(0) to room 10 where I saw printer and box.", "<GO TO ROOM(10)>", "Now I am in room 10. I will put flower vase(0) on the desk(0) for now so I can pick up box(0).", "<PUT DOWN flower vase(0) from room(8) on desk(0) in room(10)>", "Now I will pick up box(0) from cabinet(0).", "<PICK UP box(0) from room(10) in room(10)>", "Now I will put box(0) on the desk(0) so I can pick up printer(0).", "<PUT DOWN box(0) from room(10) on desk(0) in room(10)>", "Now I will pick up printer(0) from beside the desk(0).", "<PICK UP printer(0) from room(10) in room(10)>", "I need to compare the sizes of the vase, box and printer now to determine the rearranging order.", "I recall that printer(0) in this room had the largest bounding box dimensions based on my earlier observation. I need to put down printer(0).", "<PUT DOWN printer(0) from room(10) on desk(0) in room(10)>", "Now I will pick up box(0).", "<PICK UP box(0) from room(10) in room(10)>", "I remember that box(0) from cabinet(0) had medium bounding box dimensions, so I will put it on the desk.", "<PUT DOWN box(0) from room(10) on desk(0) in room(10)>", "Now I will pick up flower vase(0).", "<PICK UP flower vase(0) from room(8) in room(10)>", "I remember that flower vase(0) had the smallest bounding box dimensions, so I will put it on the desk.", "<PUT DOWN flower vase(0) from room(8) on desk(0) in room(10)>", "I will now arrange the objects from small to large: flower vase(0), box(0) and printer(0) on the desk(0).", "Now I pick up the medium object, box(0).", "<PICK UP box(0) from room(10) in room(10)>", "I will put the medium object on the left of the largest object, printer(0).", "<PUT DOWN box(0) from room(10) on desk(0) in room(10)>", "Now I pick up the smallest object, flower vase(0).", "<PICK UP flower vase(0) from room(8) in room(10)>", "I will put the smallest object on the left of the medium object, box(0).", "<PUT DOWN flower vase(0) from room(8) on desk(0) in room(10)>", "The objects are now arranged from small to large: flower vase(0), box(0) and printer(0) on the desk(0). Task Complete." Here is an example of in-domain task: Task: Preparing for Cooking Stir-fried Tomato and Scrambled Eggs. You need to prepare the ingredients, seasonings and cooking tools. Trajectory: "I'm at room 4. It seems to be living room, I need to prepare ingredients and seasonings and cooking tools and setup them in the kitchen", "There is seasonings on the table.", "<PICK UP seasonings(0) from room(4) in room(4)>", "No other things found for my task", "<GO TO NEW ROOM>", "I'm at room 5. It seems to be dining room. I saw an apron on dining table.", "I need the apron but I have seasonings(0) on my hand. Lets find kitchen to put the seasonings first and go back for apron", "<GO TO NEW ROOM>", "I'm at room 8. It seems to be kitchen with two cabinets. I found tomatoes and eggs on the cabinet(1) that is far away from me and cooking pan in the sink.", "There are two countertops in this room. countertop(1) is closer to stove(0). I should setup on countertop(1).", "<PUT DOWN seasonings(0) from room(4) on countertop(1) in room(8)>", "<PICK UP tomatoes(0) from room(8) in room(8)>", "<PUT DOWN tomatoes(0) from room(8) on countertop(1) in room(8)>", "<PICK UP eggs(0) from room(8) in room(8)>",

Cited by

Sorted by citations

Authors
Hu, W., Hong, Y., Wang, Y., Gao, L., Wei, Z., Yao, X., Peng, N., Bitton, Y., Szpektor, I., & Chang, K.-W.
Affiliations
  • University of California, Los Angeles (US)
  • Google (United States) (US)
ORCID
Venue
Advances in Neural Information Processing Systems 38
Publisher
Neural Information Processing Systems Foundation, Inc. (NeurIPS)
Published
2025
Pages
75620-75648
ISSN
1049-5258 (ISSN-L)
DOI
10.52202/085713-2278
OpenAlex
W4416048748
Cited by
2
References
0
Citations by year
2026: 2
FWCI
1.73
Type
conference paper
Language
en
Open access
Licence
cc-by
Locations
  • neural information processing systems (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
  • Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
Keywords
long-term memory, episodic memory, 3D navigation, spatio-temporal reasoning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-08.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-08.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review