CoChat

Language Models Meet World Models: Embodied Experiences Enhance Language Models

Open access
Xiang, Jiannan, Tao, Tianhua, Gu, Yi, Shu, Tianmin, Wang, Zirui, Yang, Zichao, Hu, Zhiting · 2023 · Advances in Neural Information Processing Systems 36

The DOI matches its Crossref record; 58 of the 58 references it cites were found in a registry or an index; none of the 5 sources its claims cite is checked for support yet.

Abstract

While large language models (LMs) have shown remarkable capabilities across numerous tasks, they often struggle with simple reasoning and planning in physical environments, such as understanding object permanence or planning household activities. The limitation arises from the fact that LMs are trained only on written text and miss essential embodied knowledge and skills. In this paper, we propose a new paradigm of enhancing LMs by finetuning them with world models, to gain diverse embodied knowledge while retaining their general language capabilities. Our approach deploys an embodied agent in a world model, particularly a simulator of the physical world (VirtualHome), and acquires a diverse set of embodied experiences through both goal-oriented planning and random exploration. These experiences are then used to finetune LMs to teach diverse abilities of reasoning and acting in the physical world, e.g., planning and completing goals, object permanence and tracking, etc. Moreover, it is desirable to preserve the generality of LMs during finetuning, which facilitates generalizing the embodied knowledge across tasks rather than being tied to specific simulations. We thus further introduce the classical (EWC) for selective weight updates, combined with low-rank adapters (LoRA) for training efficiency. Extensive experiments show our approach substantially improves base LMs on 18 downstream tasks by 64.28% on average. In particular, the small LMs (1.3B, 6B, and 13B) enhanced by our approach match or even outperform much larger LMs (e.g., ChatGPT).

Overview

CoChat Summary

TL;DR

The paper proposes E2WM, a paradigm that finetunes language models on embodied experiences collected from a world model (VirtualHome) to improve physical reasoning and planning while preserving general language ability. Using goal-oriented planning and random exploration plus EWC-LoRA, it reports large gains on 18 downstream tasks and competitive performance with ChatGPT for small LMs.

Key findings

  • E2WM improves base LMs on 18 downstream tasks by 64.28% on average, with small LMs (1.3B, 6B, 13B) matching or outperforming much larger LMs such as ChatGPT.
  • Specific reported gains include plan generation from 34.31 to 51.23 Rouge-L and counting from 30.41% to 67.01% accuracy; GPT-J object path tracking reaches 98.67 LCS versus 33.86 for base GPT-J and 59.53 for ChatGPT.
  • Language modeling ability is largely preserved: Pile test subset perplexity changes from 3.443 to 3.537 for GPT-J, and EWC-LoRA achieves the best trade-off between downstream performance and perplexity compared with EWC, LoRA, and KL regularization.
  • On bAbI, the approach outperforms base LMs on all selected tasks except the Two Supporting Fact task, and GPT-J surpasses ChatGPT on Three Supporting Fact and Lists/Sets.

Methods

  • The approach collects embodied experiences from VirtualHome using goal-oriented planning with Monte Carlo Tree Search and random exploration with one or multiple agents.
  • Experiences are compiled into supervised finetuning tasks: plan generation, activity recognition, counting, and object path tracking, trained with cross-entropy loss and task weights.
  • Models finetuned include GPT-Neo-1.3B, GPT-J-6B, OPT-13B, and LLaMA-13B, using EWC-LoRA for parameter-efficient updates and regularization; evaluation covers constructed downstream tasks, bAbI, Pile perplexity, and SuperGLUE, with metrics including Rouge-L, accuracy, and LCS.

Limitations

  • The paper states the work is limited to a single household environment as the world model, and future work should integrate embodied experiences from different world models and generalize across domains.
  • The paper states broader-impact risks: the trained language model may produce unintended output from harmful or adversarial input, and the authors advise against deployment outside controlled research environments until risks are mitigated.
  • Evident from the text: evaluation relies heavily on constructed tasks and VirtualHome-derived data, and the human evaluation is limited to GPT-J plan generation with 3 annotators on 150 sampled tasks.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.52202/075280-3295

Cites

58 of 58 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

0 Valid · 0 Not valid · 6 Not checked · 70 Own results and interpretations

42 more claims

Create a free account to see all 76 claims and every source verdict.

See all claims — free account
  • It may state two separate claims as one. 17
  • The quote only partly states this claim. 3
  • Probably not a substantial claim. 11

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

Research works printed but not linked: 4

  1. Matthias De Lange, Rahaf Aljundi, Marc Masana, Sarah Parisot, Xu Jia, Aleš Leonardis, Gregory Slabaugh, and Tinne Tuytelaars. A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence, 44(7):3366- 3385, 2021.
  2. Shuang Li, Xavier Puig, Chris Paxton, Yilun Du, Clinton Wang, Linxi Fan, Tao Chen, De-An Huang, Ekin Akyürek, Anima Anandkumar, et al. Pre-trained language models for interactive decision-making. Advances in Neural Information Processing Systems, 35:31199-31212, 2022.
  3. Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74-81, 2004.
  4. Ximing Lu, Sean Welleck, Jack Hessel, Liwei Jiang, Lianhui Qin, Peter West, Prithviraj Ammanabrolu, and Yejin Choi. Quark: Controllable text generation with reinforced unlearning. Advances in neural information processing systems, 35:27591-27609, 2022.

Cited by

See all 75

Metadata

Authors
Xiang, J., Tao, T., Gu, Y., Shu, T., Wang, Z., Yang, Z., & Hu, Z.
Venue
Advances in Neural Information Processing Systems 36
Published
2023
Pages
75392-75412
ISSN
1049-5258 (ISSN-L)
OpenAlex
W7133231263
Cited by
34
References
27
Citations by year
2026: 5 · 2025: 8 · 2024: 14 · 2023: 7
FWCI
17.53
Type
conference paper
Language
en
Open access
Locations
  • neural information processing systems (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
Defense Advanced Research Projects Agency
Topics
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Embodied and Extended Cognition
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
  • Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
Keywords
language models, world models, embodied experience, embodied AI, language model training

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-11-01, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-11-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.