CoChat

Dynamics-Aligned Latent Imagination in Contextual World Models for Zero-Shot Generalization

Open access
Röder, Frank, Benad, Jan, Eppe, Manfred, Banerjee, Pradeep · 2025 · Advances in Neural Information Processing Systems 38

The DOI matches its Crossref record; 30 of the 30 references it cites were found in a registry or an index; 7 of the 9 sources its claims cite were checked for support.

Abstract

Real-world reinforcement learning demands adaptation to unseen environmental conditions without costly retraining. Contextual Markov Decision Processes (cMDP) model this challenge, but existing methods often require explicit context variables (e.g., friction, gravity), limiting their use when contexts are latent or hard to measure. We introduce Dynamics-Aligned Latent Imagination (DALI), a framework integrated within the Dreamer architecture that infers latent context representations from agent-environment interactions. By training a self-supervised encoder to predict forward dynamics, DALI generates actionable representations conditioning the world model and policy, bridging perception and control. We theoretically prove this encoder is essential for efficient context inference and robust generalization. DALI's latent space enables counterfactual consistency: Perturbing a gravity-encoding dimension alters imagined rollouts in physically plausible ways. On challenging cMDP benchmarks, DALI achieves significant gains over context-unaware baselines, often surpassing context-aware baselines in extrapolation tasks, enabling zero-shot generalization to unseen contextual variations.

Overview

CoChat Summary

TL;DR

DALI extends DreamerV3 with a self-supervised context encoder that infers latent environmental context from short interaction histories by predicting forward dynamics, enabling zero-shot generalization in contextual MDPs. The paper reports large extrapolation gains over context-unaware baselines and sometimes over ground-truth context-aware baselines, plus physically consistent counterfactual latent perturbations.

Key findings

  • DALI’s context encoder is theoretically shown to capture near-optimal context information, \(I(c;z_t)\ge(1-\delta)h(c)\), using \(N=O(1/\delta^2)\) windows of \(K=\Omega(\log(1/\delta)/\lambda)\) transitions, with a sample-complexity gain of \(O(T/K)\) over DreamerV3’s full-episode recurrent state.
  • On DMC Ball-in-Cup Extrapolation, DALI-Sχ outperforms context-unaware Dreamer-DR by \(87.9\%\) (Featurized, IQM \(0.3720\)) to \(96.4\%\) (Pixel, IQM \(0.2730\)), and surpasses context-aware cRSSM-S by \(63.9\%\) (Featurized) and \(45.9\%\) (Pixel), and cRSSM-D by \(33.8\%\) (Featurized) and \(12.8\%\) (Pixel).
  • On Walker Walk Extrapolation, DALI-S achieves \(4.0\%\) over Dreamer-DR (Featurized, IQM \(0.7810\)), \(11.3\%\) over cRSSM-S, and \(4.3\%\) over cRSSM-D.
  • Perturbing the top-ranked Ball-in-Cup latent dimension \(z_6\) produces counterfactual rollouts with shorter string length and higher acceleration; perturbing Walker’s top-ranked \(z_3\) enables recovery and locomotion consistent with increased actuator strength.

Methods

  • The paper formalizes zero-shot generalization in contextual MDPs with latent, episode-fixed context and fixed reward across contexts. DALI adds a transformer-based context encoder \(g_\phi\) mapping \(K\)-length observation-action histories to \(z_t\in\mathbb{R}^8\), trained with a forward dynamics loss and optional cross-modal regularization, then integrated into DreamerV3 via Shallow or Deep Integration.
  • Experiments use contextualized DMC Ball-in-Cup (gravity, string length) and Walker Walk (gravity, actuator strength) from CARL, in Featurized and Pixel modalities, with 10 seeds, 200K timesteps for Ball-in-Cup and 500K for Walker, comparing against Dreamer-DR and cRSSM-S/D; results use IQM and Probability of Improvement with stratified bootstrap 95% confidence intervals.
  • Counterfactual validation perturbs each of 8 latent dimensions by its standard deviation, rolls out \(H=50\) imagined steps, generates \(N=2500\) baseline/perturbed trajectory pairs per dimension, and ranks dimensions using an ensemble classifier’s AUC with bootstrap confidence intervals and permutation tests.

Limitations

  • Paper-stated: the information-theoretic result does not address downstream policy performance, which depends on joint optimization of encoder, world model, and actor-critic; reliance on an exploratory policy may falter in sparse-reward or high-dimensional settings; the \(\beta\)-mixing assumption may not hold for slow-mixing dynamics or restricted exploration; training instability, hyperparameter sensitivity, and overfitting in high-dimensional observation spaces are beyond its scope.
  • Evident from the text: the main empirical evaluation is limited to two DMC tasks, with an additional Quadruped experiment in the appendix, and no real-robot or broader benchmark validation is reported.
  • Evident from the text: the main reported results use Shallow Integration exclusively, even though Deep Integration is proposed; the theory assumes Lipschitz dynamics, observation noise, bounded log-likelihood ratios, and a universal approximator.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.52202/085713-2438

Cites

0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Figure 1 Interquartile Mean (IQM) scores and Probability of Improvement (PoI) for the DMC Ballin-Cup and Walker Walk tasks under contextual variations: gravity and string length for Ball-in-Cup, and gravity and actuator strength for Walker Walk. Results are shown for Featurized and Pixel observations in each environment. Scores aggregate across single and combined contexts. Shaded intervals represent 95% stratified bootstrap confidence intervals over seeds and aggregated contexts. The rightmost panel in each plot displays PoI in the Extrapolation regime for DALI-Sχ (Ball-in-Cup) and DALI-S (Walker Walk), relative to baseline methods.
Figure 1 Interquartile Mean (IQM) scores and Probability of Improvement (PoI) for the DMC Ballin-Cup and Walker Walk tasks under contextual variations: gravity and string length for Ball-in-Cup, and gravity and actuator strength for Walker Walk. Results are shown for Featurized and Pixel observations in each environment. Scores aggregate across single and combined contexts. Shaded intervals represent 95% stratified bootstrap confidence intervals over seeds and aggregated contexts. The rightmost panel in each plot displays PoI in the Extrapolation regime for DALI-Sχ (Ball-in-Cup) and DALI-S (Walker Walk), relative to baseline methods.From the PDF
Figure 2 (a) (Pixel Modality) Counterfactual Trajectories in Pixel Space : Top: Original imagined trajectory of the Ball-in-Cup under default gravity and string length. Middle: Perturbed trajectory after adding noise ∆ to the top-ranked latent dimension z 6 . Bottom: Pixel-wise differences ( δ = | ˆ o t -ˆ o ′ t | ). The perturbed trajectory (blue) exhibits a shorter string (frame 40) and faster acceleration (overtaking the original trajectory in frames 15 and 45), aligning with increased gravitational effects. Rollouts use zero actions to isolate passive dynamics. (b) (Featurized Modality) Ball Z-Position Under Latent Perturbation : Comparison of original (blue) and counterfactual (orange) ball height trajectories. The perturbed z 6 reduces oscillation amplitude (lower peak Z-position) and accelerates descent, consistent with shorter string length and higher gravity. (c) (Featurized Modality) Ball Z-Velocity Under Latent Perturbation : Velocity profiles for original (blue) and counterfactual (orange) trajectories. The perturbed z 6 induces earlier and higher velocity peaks, confirming faster swing dynamics. This mirrors the pixel-based evidence of increased gravitational acceleration.
Figure 2 (a) (Pixel Modality) Counterfactual Trajectories in Pixel Space : Top: Original imagined trajectory of the Ball-in-Cup under default gravity and string length. Middle: Perturbed trajectory after adding noise ∆ to the top-ranked latent dimension z 6 . Bottom: Pixel-wise differences ( δ = | ˆ o t -ˆ o ′ t | ). The perturbed trajectory (blue) exhibits a shorter string (frame 40) and faster acceleration (overtaking the original trajectory in frames 15 and 45), aligning with increased gravitational effects. Rollouts use zero actions to isolate passive dynamics. (b) (Featurized Modality) Ball Z-Position Under Latent Perturbation : Comparison of original (blue) and counterfactual (orange) ball height trajectories. The perturbed z 6 reduces oscillation amplitude (lower peak Z-position) and accelerates descent, consistent with shorter string length and higher gravity. (c) (Featurized Modality) Ball Z-Velocity Under Latent Perturbation : Velocity profiles for original (blue) and counterfactual (orange) trajectories. The perturbed z 6 induces earlier and higher velocity peaks, confirming faster swing dynamics. This mirrors the pixel-based evidence of increased gravitational acceleration.From the PDF
Figure 3 DALI architecture overview.
Figure 3 DALI architecture overview.From the PDF
Figure 4 Learning curves for DMC Ball-in-Cup and Walker Walk tasks under Featurized and Pixel-based observation modalities. Results show mean episode returns with 25 th75 th percentile confidence intervals.
Figure 4 Learning curves for DMC Ball-in-Cup and Walker Walk tasks under Featurized and Pixel-based observation modalities. Results show mean episode returns with 25 th75 th percentile confidence intervals.From the PDF
Figure 5 AUC ± 95% CI per context dimension z j for the Ball-in-Cup task.
Figure 5 AUC ± 95% CI per context dimension z j for the Ball-in-Cup task.From the PDF
Figure 6 (a) (Pixel Modality) Counterfactual Trajectories in Pixel Space : Top: Original imagined trajectory of the Walker Walk under default gravity and actuator strength. Middle: Perturbed trajectory after adding noise ∆ to the top-ranked latent dimension z 3 . Original trajectory shows the Walker failing to recover after a fall. The perturbed trajectory with enhanced actuator strength enables recovery and forward locomotion. Bottom: Pixel-wise differences ( δ = | ˆ o t -ˆ o ′ t | ). The counterfactual agent refines leg kinematics (frames 30 -45 ) and stabilizes upright (frames 50 -64 ). (b) (Featurized Modality) Torso Z-Position Under Latent Perturbation : Comparison of original (blue) and counterfactual (orange) torso heights. The perturbed z 3 trajectory maintains higher elevation after time step 20, reflecting successful stabilization and locomotion.
Figure 6 (a) (Pixel Modality) Counterfactual Trajectories in Pixel Space : Top: Original imagined trajectory of the Walker Walk under default gravity and actuator strength. Middle: Perturbed trajectory after adding noise ∆ to the top-ranked latent dimension z 3 . Original trajectory shows the Walker failing to recover after a fall. The perturbed trajectory with enhanced actuator strength enables recovery and forward locomotion. Bottom: Pixel-wise differences ( δ = | ˆ o t -ˆ o ′ t | ). The counterfactual agent refines leg kinematics (frames 30 -45 ) and stabilizes upright (frames 50 -64 ). (b) (Featurized Modality) Torso Z-Position Under Latent Perturbation : Comparison of original (blue) and counterfactual (orange) torso heights. The perturbed z 3 trajectory maintains higher elevation after time step 20, reflecting successful stabilization and locomotion.From the PDF

Verification

Claims the paper makes

4 Valid · 1 Not valid · 0 Not checked · 113 Own results and interpretations

67 more claims

Create a free account to see all 118 claims and every source verdict.

See all claims — free account
  • It may state two separate claims as one. 35
  • The quote only partly states this claim. 5
  • Probably not a substantial claim. 6

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

The publisher hasn’t deposited this paper’s reference list.

Research works printed but not linked: 9

  1. Hamid Eghbal-zadeh, Florian Henkel, and Gerhard Widmer. Context-adaptive reinforcement learning using unsupervised learning of context variables. In NeurIPS 2020 Workshop on Pre-registration in Machine Learning , pages 236-254, 2021.
  2. Luckeciano C Melo. Transformers are meta-reinforcement learners. In International Conference on Machine Learning , pages 15340-15359, 2022.
  3. Yao Mu, Yuzheng Zhuang, Fei Ni, Bin Wang, Jianyu Chen, Jianye Hao, and Ping Luo. Domino: Decomposed mutual information optimization for generalized context in meta-reinforcement learning. Advances in Neural Information Processing Systems , 35:27563-27575, 2022.
  4. Kate Rakelly, Aurick Zhou, Chelsea Finn, Sergey Levine, and Deirdre Quillen. Efficient off-policy metareinforcement learning via probabilistic context variables. In International Conference on Machine Learning , pages 5331-5340, 2019.
  5. Emmanuel Rio. Asymptotic Theory of Weakly Dependent Random Processes . Springer, 2017.
  6. Seyed Kamyar Seyed Ghasemipour, Shixiang Shane Gu, and Richard Zemel. Smile: Scalable meta inverse reinforcement learning through context-conditional policies. Advances in Neural Information Processing Systems , 32, 2019.
  7. Joy A. Thomas and Thomas M. Cover. Elements of Information Theory . John Wiley & Sons, 1999.
  8. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in Neural Information Processing Systems , 30, 2017.
  9. Tianhe Yu, Deirdre Quillen, Zhanpeng He, Ryan Julian, Karol Hausman, Chelsea Finn, and Sergey Levine. Meta-world: A benchmark and evaluation for multi-task and meta reinforcement learning. In Conference on Robot Learning , pages 1094-1100, 2020.

Cited by

See all 4

Metadata

Authors
Röder, F., Benad, J., Eppe, M., & Banerjee, P.
Affiliations
Hamburg University of Technology (DE)
Venue
Advances in Neural Information Processing Systems 38
Published
2025
Pages
80885-80920
ISSN
1049-5258 (ISSN-L)
OpenAlex
W4414447571
Cited by
0
References
0
FWCI
0.00
Type
conference paper
Language
en
Open access
Locations
  • neural information processing systems (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
Keywords
zero-shot generalization, world models, unknown environments

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Dynamics-Aligned Latent Imagination in Contextual World Models for Zero-Shot Generalization. https://app.cochat.ai/papers/10.52202/085713-2438
The paper
Röder, F., Benad, J., Eppe, M., & Banerjee, P. (2025). Dynamics-Aligned Latent Imagination in Contextual World Models for Zero-Shot Generalization. Advances in Neural Information Processing Systems 38. https://doi.org/10.52202/085713-2438

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.