CoChat

Generalizable Causal Reinforcement Learning for Out-of-Distribution Environments

Huang, Sili, Hu, Jifeng, Chen, Hechang, Cui, Peng, Piao, Haiyin, Sun, Lichao, Yang, Bo · 2025 · IEEE Transactions on Industrial Informatics

The DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index.

Abstract

Out-of-distribution (OOD) generalization is critical for applying reinforcement learning algorithms to real-world applications. To address the OOD problem, recent works focus on learning an OOD adaptation policy by capturing the causal factors affecting the environmental dynamics. However, these works recover the causal factors with only an entangled or binary form, resulting in a limited generalization of the policy that requires extra data from the testing environments. To break this limitation, we propose generalizable causal reinforcement learning (GCRL) to learn a disentangled representation of causal factors, on the basis of which we learn a policy that achieves the OOD generalization without extra training. For capturing the causal factors, GCRL deploys a weakly supervised signal with a two-stage constraint to ensure that all factors can be disentangled. Then, to achieve the OOD generalization through causal factors, we establish the dependence of actions on the learned representation and optimize the policy model across multiple environments. Experimental results show that the established dependence recovers the correct relationship between causal factors and actions when the learned policy could address the target tasks in training environments. Benefiting from the recovered relationship, GCRL achieves the OOD generalization on eight benchmarks from Causal World and Mujoco. Moreover, the policy learned by our model is more explainable and can be controlled to generat…

Overview

Integrity signals

Registry

Crossref · 10.1109/tii.2025.3556029

Cites

17 of 17 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Citations

Cites

Show all 17 references

Research works printed but not linked: 27

  1. Fujimoto, 2024, For SALE: State-action representation learning for deep reinforcement learning, 61573
  2. Bhatt, 2024, CrossQ: Batch normalization in deep reinforcement learning for greater sample efficiency and simplicity
  3. Hiraoka, 2022, Dropout Q-functions for doubly efficient reinforcement learning
  4. Sontakke, 2021, Causal curiosity: Rl agents discovering self-supervised experiments for causal representation learning, 9848
  5. Zintgraf, 2020, VariBAD: A very good method for Bayes-adaptive deep RL via meta-learning
  6. Yao, Direct policy transfer via hidden parameter Markov decision processes
  7. Higgins, 2017, beta-VAE: Learning basic visual concepts with a constrained variational framework
  8. Kim, 2018, Disentangling by factorising, 2649
  9. Locatello, 2020, Weakly-supervised disentanglement without compromises, 6348
  10. Shu, 2020, Weakly-supervised disentanglement with guarantees, 1094
  11. Zhang, 2021, Learning robust state abstractions for hidden parameter block
  12. Bica, 2021, Invariant causal imitation learning for generalizable policies, 34, 3952
  13. Thomas, 2017, Disentangling the independently controllable factors of variation by interacting with the world
  14. Haarnoja, 2018, Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor, 1861
  15. Ahmed, 2021, CausalWorld: A robotic manipulation benchmark for causal structure and transfer learning
  16. Ding, 2022, Generalizing goal-conditioned reinforcement learning with variational causal reasoning, 26532
  17. Truble, The role of pretained representations for the OOD generalization of RL agents, 2022
  18. Agarwal, 2021, Deep reinforcement learning at the edge of the statistical precipice, 29304
  19. Seitzer, 2021, Causal influence detection for improving efficiency in reinforcement learning, 22905
  20. Lee, 2020, Characterizing optimal mixed policies: Where to intervene and what to observe, 8565
  21. Sonar, 2021, Invariant policy optimization: Towards stronger generalization in reinforcement learning, 21
  22. Ahuja, 2020, Invariant risk minimization games, 145
  23. Zhang, 2020, Invariant causal prediction for block MDPs, 11214
  24. Turchetta, 2021, Safe reinforcement learning via curriculum induction
  25. Zhang, 2021, Learning invariant representations for reinforcement learning without reconstruction
  26. Sodhani, 2021, Multi-task reinforcement learning with context-based representations, 9767
  27. Yarats, 2021, Reinforcement learning with prototypical representations, 11920

Cited by

See all 4

Metadata

Authors
Huang, S., Hu, J., Chen, H., Cui, P., Piao, H., Sun, L., & Yang, B.
Affiliations
  • Jilin University (CN)
  • Tsinghua University (CN)
  • Lehigh University (US)
Venue
IEEE Transactions on Industrial Informatics
Published
2025-07
Volume
21
Issue
7
Pages
5514-5524
ISSN
  • 1551-3203 (ISSN-L)
  • 1941-0050
OpenAlex
W4409683812
Cited by
4
References
16
Citations by year
2026: 3 · 2025: 1
FWCI
1.62
Type
journal article
Language
en
Open access
closed
Locations
IEEE Transactions on Industrial Informatics (publishedVersion)
Funders
  • National Key R&D Program of China (2021ZD0112500)
  • National Natural Science Foundation of China (U22A2098, U2341229, 62476110, 62172185, 61976102, 62206105, 62202200)
  • Key R&D Project of Jilin Province (20240304200SF)
  • International Cooperation Project of Jilin Province (20220402009GH)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
Keywords
out-of-distribution generalization, causal reinforcement learning, disentangled representation learning, causal factors, weakly supervised learning, MuJoCo

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Generalizable Causal Reinforcement Learning for Out-of-Distribution Environments. https://app.cochat.ai/papers/10.1109/tii.2025.3556029
The paper
Huang, S., Hu, J., Chen, H., Cui, P., Piao, H., Sun, L., & Yang, B. (2025). Generalizable Causal Reinforcement Learning for Out-of-Distribution Environments. IEEE Transactions on Industrial Informatics. https://doi.org/10.1109/tii.2025.3556029

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.