CoChat
Sign Up
Cited by 1,170Bayesian Modeling and Causal InferenceAdvanced Causal Inference TechniquesMachine Learning and Algorithms

Toward Causal Representation Learning

Open access
Scholkopf, Bernhard, Locatello, Francesco, Bauer, Stefan, Ke, Nan Rosemary, Kalchbrenner, Nal, Goyal, Anirudh, Bengio, Yoshua·2021·Proceedings of the IEEE
DOI 10.1109/jproc.2021.3058954
Open this paper in CoChat — free accountOpen PDF

Abstract

The two fields of machine learning and graphical causality arose and are developed separately. However, there is, now, cross-pollination and increasing interest in both fields to benefit from the advances of the other. In this article, we review fundamental concepts of causal inference and relate them to crucial open problems of machine learning, including transfer and generalization, thereby assaying how causality can contribute to modern machine learning research. This also applies in the opposite direction: we note that most work in causality starts from the premise that the causal variables are given. A central problem for AI and causality is, thus, causal representation learning, that is, the discovery of high-level causal variables from low-level observations. Finally, we delineate some implications of causality for machine learning and propose key research areas at the intersection of both communities.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/jproc.2021.3058954

Cites

20 of 20 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Learning causal representations is necessary for achieving strong robustness guarantees.

    “This provides further theoretical justification for causal representation learning (Schölkopf et al., 2021), showing that learning causal representations is necessary for achieving strong robustness guarantees.”

    From Robust agents learn causal world models (2024) · cites this paper as Schölkopf et al., 2021

    In “Robustness and Strong Generalization” (page unavailable), the paper suggests that causal features can make a predictor more robust and discusses using a causal model to optimize worst-case out-of-distribution risk. It does not establish that learning causal representations is necessary for strong robustness guarantees; its claim about robustness from causal structure is explicitly speculative. (LLM, conf=0.87)

    Experimental — accuracy not yet measured

  • Many causal discovery algorithms assume independent causal mechanisms, which is equivalent to assuming no agents are present in the data generating process.

    “Also, many causal discovery algorithms assume independent causal mechanisms (Schölkopf et al., 2021), which is equivalent to assuming no agents are present in the data generating process (Kenton et al., 2023).”

    From Robust agents learn causal world models (2024) · cites this paper as Schölkopf et al., 2021

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 257

  1. O. Ahmed et al., "Causalworld: A robotic manipulation benchmark for causal structure and transfer learning," in Proc. Int. Conf. Learn. Represent., 2021.
  2. OpenAI et al., "Solving Rubik's cube with a robot hand," 2019, arXiv:1910.07113. [Online]. Available: http://arxiv.org/abs/1910.07113 · arXiv
  3. A. Alaa and M. Schaar, "Limits of estimating heterogeneous treatment effects: Guidelines for practical algorithm design," in Proc. Int. Conf. Mach. Learn., 2018, pp. 129-138.
  4. J. Aldrich, "Autonomy," Oxford Econ. Papers, vol. 41, no. 1, pp. 15-34, 1989.
  5. M. Arjovsky, L. Bottou, I. Gulrajani, and D. Lopez-Paz, "Invariant risk minimization," 2019, arXiv:1907.02893. [Online]. Available: http://arxiv.org/abs/1907.02893 · arXiv
  6. O. Atan, J. Jordon, and M. van der Schaar, "Deep-treat: Learning optimal personalized treatments from observational data using neural networks," in Proc. 32nd AAAI Conf. Artif. Intell., 2018.
  7. A. L. Rezaabad and S. Vishwanath, "Learning representations by maximizing mutual information in variational autoencoders," in Proc. IEEE Int. Symp. Inf. Theory (ISIT), Jun. 2020, pp. 15535-15545.
  8. D. Bahdanau, S. Murty, M. Noukhovitch, T. H. Nguyen, H. de Vries, and A. Courville, "Systematic generalization: What is required and can it be learned," 2018, arXiv:1811.12889. [Online]. Available: https://arxiv.org/ abs/1811.12889 · arXiv
  9. H. Baird, "Document image defect models," in Proc. IAPR Workshop Syntactic Structural Pattern Recognit., Murray Hill, NJ, USA, 1990, pp. 38-46.
  10. V. Bapst et al., "Structured agents for physical construction," in Proc. Int. Conf. Mach. Learn., 2019, pp. 464-474.
  11. A. Barbu et al., "Objectnet: A large-scale bias-controlled dataset for pushing the limits of object recognition models," in Proc. Adv. Neural Inf. Process. Syst., 2019, pp. 9448-9458.
  12. E. Bareinboim, A. Forney, and J. Pearl, "Bandits with unobserved confounders: A causal approach," in Proc. Adv. Neural Inf. Process. Syst., 2015, pp. 1342-1350.
  13. E. Bareinboim and J. Pearl, "Transportability from multiple environments with limited experiments: Completeness results," in Proc. Adv. Neural Inf. Process. Syst., 2014, pp. 280-288.
  14. P. Battaglia et al., "Interaction networks for learning about objects, relations and physics," in Proc. Adv. Neural Inf. Process. Syst., 2016, pp. 4502-4510.
  15. P. W. Battaglia et al., "Relational inductive biases, deep learning, and graph networks," 2018, arXiv:1806.01261. [Online]. Available: http://arxiv.org/abs/1806.01261 · arXiv
  16. P. W. Battaglia, J. B. Hamrick, and J. B. Tenenbaum, "Simulation as an engine of physical scene understanding," Proc. Nat. Acad. Sci. USA, vol. 110, no. 45, pp. 18327-18332, Nov. 2013.
  17. S. Bauer, B. Schölkopf, and J. Peters, "The arrow of time in multivariate time series," in Proc. 33rd Int. Conf. Mach. Learn., vol. 48, 2016, pp. 2043-2051.
  18. S. Beery, G. Van Horn, and P. Perona, "Recognition in terra incognita," in Proc. Eur. Conf. Comput. Vis. (ECCV), 2018, pp. 456-473.
  19. S. Ben-David, T. Lu, T. Luu, and D. Pál, "Impossibility theorems for domain adaptation," in Proc. Int. Conf. Artif. Intell. Statist. (AISTATS), 2010, pp. 129-136.
  20. E. Bengio, V. Thomas, J. Pineau, D. Precup, and Y. Bengio, "Independently controllable features," 2017, arXiv:1703.07718. [Online]. Available: http://arxiv.org/abs/1703.07718 · arXiv
  21. Y. Bengio, S. Bengio, and J. Cloutier, "Learning a synaptic learning rule," in Proc. Seattle Int. Joint Conf. Neural Netw. (IJCNN), vol. 2. IEEE, Jul. 1991, p. 969.
  22. Y. Bengio, A. Courville, and P . Vincent, "Representation learning: A review and new perspectives," 2012, arXiv:1206.5538. [Online]. Available: http://arxiv.org/abs/1206.5538 · arXiv
  23. Y. Bengio et al., "A meta-transfer objective for learning to disentangle causal mechanisms," 2019, arXiv:1901.10912. [Online]. Available: http://arxiv.org/abs/1901.10912 · arXiv
  24. B. Benneke et al., "Water vapor on the habitable-zone exoplanet K2-18b," 2019, arXiv:1909.04642. [Online]. Available: https://arxiv.org/abs/1909.04642 · arXiv
  25. M. Besserve, N. Shajarisales, B. Schölkopf, and D. Janzing, "Group invariance principles for causal generative models," in Proc. 21st Int. Conf. Artif. Intell. Statist. (AISTATS), 2018, pp. 557-565.
  26. M. Besserve, R. Sun, D. Janzing, and B. Schölkopf, "A theory of independent mechanisms for extrapolation in generative models," in Proc. 35th AAAI Conf. Artif. Intell. Virtual Conf., Feb. 2021.
  27. M. Besserve, A. Mehrjou, R. Sun, and B. Schölkopf, "Counterfactuals uncover the modular structure of deep generative models," 2018, arXiv:1812.03253. [Online]. Available: http://arxiv.org/abs/1812.03253 · arXiv
  28. I. Bica, A. M. Alaa, and M. van der Schaar, "Time series deconfounder: Estimating treatment effects over time in the presence of hidden confounders," 2019, arXiv:1902.00450. [Online]. Available: http://arxiv.org/abs/1902.00450 · arXiv
  29. P. Blöbaum, T. Washio, and S. Shimizu, "Error asymmetry in causal and anticausal regression," 2016, arXiv:1610.03263. [Online]. Available: http://arxiv.org/abs/1610.03263 · arXiv
  30. A. Blum and T. Mitchell, "Combining labeled and unlabeled data with co-training," in Proc. 11th Annu. Conf. Comput. Learn. Theory, New York, NY, USA, 1998, pp. 92-100.
  31. B. Bonet and H. Geffner, "Learning first-order symbolic representations for planning from the structure of the state space," 2019, arXiv:1909.05546. [Online]. Available: http://arxiv.org/abs/1909.05546 · arXiv
  32. L. Bottou et al., "Counterfactual reasoning and learning systems: The example of computational advertising," J. Mach. Learn. Res., vol. 14, no. 1, pp. 3207-3260, Jan. 2013.
  33. T. B. Brown et al., "Language models are few-shot learners," 2020, arXiv:2005.14165. [Online]. Available: https://arxiv.org/abs/2005.14165 · arXiv
  34. K. Budhathoki and J. Vreeken, "Causal inference by compression," in Proc. IEEE 16th Int. Conf. Data Mining (ICDM), Dec. 2016, pp. 41-50.
  35. L. Buesing et al., "Woulda, coulda, shoulda: Counterfactually-guided policy search," 2018, arXiv:1811.06272. [Online]. Available: http://arxiv.org/abs/1811.06272 · arXiv
  36. C. J. C. Burges and B. Schölkopf, "Improving the accuracy and speed of support vector learning machines," in Advances in Neural Information Processing Systems, vol. 9, M. Mozer, M. Jordan, and T. Petsche, Eds. Cambridge, MA, USA: MIT Press, 1997, pp. 375-381.
  37. C. P. Burgess et al., "MONet: Unsupervised scene decomposition and representation," 2019, arXiv:1901.11390. [Online]. Available: http://arxiv.org/abs/1901.11390 · arXiv
  38. K. Chalupka, P. Perona, and F. Eberhardt, "Multi-level cause-effect systems," 2015, arXiv:1512.07942. [Online]. Available: http://arxiv.org/abs/1512.07942 · arXiv
  39. K. Chalupka, P. Perona, and F. Eberhardt, "Fast conditional independence test for vector variables with large sample sizes," 2018, arXiv:1804.02747. [Online]. Available: http://arxiv.org/abs/1804.02747 · arXiv
  40. M. B. Chang, T. Ullman, A. Torralba, and J. B. Tenenbaum, "A compositional object-based approach to learning physical dynamics," in Proc. 5th Int. Conf. Learn. Represent. (ICLR), 2017.
  41. O. Chapelle, B. Schölkopf, and A. Zien, Eds., Semi-Supervised Learning. Cambridge, MA, USA: MIT Press, 2006.
  42. M. Chen et al., "Generative pretraining from pixels," in Proc. 37th Int. Conf. Mach. Learn., 2020, pp. 1691-1703.
  43. T. Chen, S. Kornblith, M. Norouzi, and G. Hinton, "A simple framework for contrastive learning of visual representations," 2020, arXiv:2002.05709. [Online]. Available: http://arxiv.org/ abs/2002.05709 · arXiv
  44. S. Chiappa, S. Racaniere, D. Wierstra, and S. Mohamed, "Recurrent environment simulators," in Proc. 5th Int. Conf. Learn. Represent. (ICLR), 2017.
  45. E. D. Cubuk, B. Zoph, D. Mane, V. Vasudevan, and Q. V. Le, "AutoAugment: Learning augmentation strategies from data," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2019, pp. 113-123.
  46. P. Daniušis et al., "Inferring deterministic causal relations," in Proc. 26th Annu. Conf. Uncertainty Artif. Intell. (UAI), 2010, pp. 143-150.
  47. I. Dasgupta et al., "Causal reasoning from meta-reinforcement learning," 2019, arXiv:1901.08162. [Online]. Available: http://arxiv.org/abs/1901.08162 · arXiv
  48. A. P. Dawid, "Conditional independence in statistical theory," J. Roy. Stat. Soc. B, Stat. Methodol., vol. 41, no. 1, pp. 1-31, 1979.
  49. S. Dehaene, How We Learn: Why Brains Learn Better Than Any Machine ... for Now. Baltimore, MD, USA: Penguin, 2020.
  50. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, "ImageNet: A large-scale hierarchical image database," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2009, pp. 248-255.
  51. P. Dhariwal, H. Jun, C. Payne, J. W. Kim, A. Radford, and I. Sutskever, "Jukebox: A generative model for music," 2020, arXiv:2005.00341. [Online]. Available: http://arxiv.org/abs/2005.00341 · arXiv
  52. A. Dittadi et al., "On the transfer of disentangled representations in realistic settings," in Proc. Int. Conf. Learn. Represent., 2021.
  53. C. Diuk, A. Cohen, and M. L. Littman, "An object-oriented representation for efficient reinforcement learning," in Proc. 25th Int. Conf. Mach. Learn. (ICML), 2008, pp. 240-247.
  54. J. Djolonga et al., "On robustness and transferability of convolutional neural networks," 2020, arXiv:2007.08558. [Online]. Available: https://arxiv.org/abs/2007.08558 · arXiv
  55. G. Doran, K. Muandet, K. Zhang, and B. Schölkopf, "A permutation-based kernel conditional independence test," in Proc. 30th Conf. Uncertainty Artif. Intell., N. L. Zhang and J. Tian, Eds. Corvallis, OR, USA: AUAI Press, 2014, pp. 132-141.
  56. C. Eastwood and C. K. Williams, "A framework for the quantitative evaluation of disentangled representations," in Proc. Int. Conf. Learn. Represent., 2018.
  57. D. Eaton and K. Murphy, "Exact Bayesian structure learning from uncertain interventions," in Proc. Artif. Intell. Statist., 2007, pp. 107-114.
  58. L. Engstrom, B. Tran, D. Tsipras, L. Schmidt, and A. Madry, "Exploring the landscape of spatial robustness," 2017, arXiv:1712.02779. [Online]. Available: http://arxiv.org/abs/1712.02779 · arXiv
  59. K. Epstude and N. J. Roese, "The functional theory of counterfactual thinking," Personality Social Psychol. Rev., vol. 12, no. 2, pp. 168-192, May 2008.
  60. A. Farago and G. Lugosi, "Strong universal consistency of neural network classifiers," IEEE Trans. Inf. Theory, vol. 39, no. 4, pp. 1146-1151, Jul. 1993.
  61. C. Finn, P. Abbeel, and S. Levine, "Model-agnostic meta-learning for fast adaptation of deep networks," 2017, arXiv:1703.03400. [Online]. Available: http://arxiv.org/abs/1703.03400 · arXiv
  62. J. N. Foerster, G. Farquhar, T. Afouras, N. Nardelli, and S. Whiteson, "Counterfactual multi-agent policy gradients," in Proc. 32nd AAAI Conf. Artif. Intell., 2018.
  63. D. Foreman-Mackey, B. T. Montet, D. W. Hogg, T. D. Morton, D. Wang, and B. Schölkopf, "A systematic search for transiting planets in the K2 data," Astrophys. J., vol. 806, no. 2, p. 215, 2015.
  64. S. Fujimoto, D. Meger, and D. Precup, "Off-policy deep reinforcement learning without exploration," in Proc. Int. Conf. Mach. Learn., 2019, pp. 2052-2062.
  65. K. Fukumizu, A. Gretton, X. Sun, and B. Schölkopf, "Kernel measures of conditional dependence," in Proc. Adv. Neural Inf. Process. Syst., 2008, pp. 489-496.
  66. D. Geiger and J. Pearl, "Logical and algorithmic properties of independence and their application to Bayesian networks," Ann. Math. Artif. Intell., vol. 2, nos. 1-4, pp. 165-178, Mar. 1990.
  67. R. Geirhos, P. Rubisch, C. Michaelis, M. Bethge, F. A. Wichmann, and W. Brendel, "ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness," 2018, arXiv:1811.12231. [Online]. Available: http://arxiv.org/abs/1811.12231 · arXiv
  68. M. W. Gondal et al., "On the transfer of inductive bias from simulation to the real world: A new disentanglement dataset," in Proc. Adv. Neural Inf. Process. Syst., 2019, pp. 15740-15751.
  69. M. Gong, K. Zhang, T. Liu, D. Tao, C. Glymour, and B. Schölkopf, "Domain adaptation with conditional transferable components," in Proc. 33rd Int. Conf. Mach. Learn., 2016, pp. 2839-2848.
  70. M. Gong, K. Zhang, B. Schölkopf, C. Glymour, and D. Tao, "Causal discovery from temporally aggregated time series," in Proc. 33rd Conf. Uncertainty Artif. Intell. (UAI), 2017, p. 269.
  71. A. Gopnik, C. Glymour, D. M. Sobel, L. E. Schulz, T. Kushnir, and D. Danks, "A theory of causal learning in children: Causal maps and Bayes nets," Psychol. Rev., vol. 111, no. 1, pp. 3-32, 2004.
  72. O. Gottesman et al., "Evaluating reinforcement learning algorithms in observational health settings," 2018, arXiv:1805.12298. [Online]. Available: http://arxiv.org/abs/1805.12298 · arXiv
  73. O. Goudet, D. Kalainathan, P. Caillou, I. Guyon, D. Lopez-Paz, and M. Sebag, "Causal generative neural networks," 2017, arXiv:1711.08936. [Online]. Available: http://arxiv.org/abs/1711.08936 · arXiv
  74. A. Goyal et al., "Object files and schemata: Factorizing declarative and procedural knowledge in dynamical systems," 2020, arXiv:2006.16225. [Online]. Available: http://arxiv.org/abs/2006.16225 · arXiv
  75. A. Goyal, A. Lamb, J. Hoffmann, S. Sodhani, S. Levine, Y. Bengio, and B. Schölkopf, "Recurrent independent mechanisms," in Proc. Int. Conf. Learn. Represent., 2021.
  76. A. Graves, A.-R. Mohamed, and G. Hinton, "Speech recognition with deep recurrent neural networks," in Proc. IEEE Int. Conf. Acoust., Speech Signal Process., May 2013, pp. 6645-6649.
  77. K. Greff et al., "Multi-object representation learning with iterative variational inference," in Proc. Int. Conf. Mach. Learn., 2019, pp. 2424-2433.
  78. K. Greff, S. van Steenkiste, and J. Schmidhuber, "On the binding problem in artificial neural networks," 2020, arXiv:2012.05208. [Online]. Available: http://arxiv.org/abs/2012.05208 · arXiv
  79. K. Gregor, D. J. Rezende, F. Besse, Y. Wu, H. Merzic, and A. van den Oord, "Shaping belief states with generative environment models for RL," in Proc. Adv. Neural Inf. Process. Syst., 2019, pp. 13475-13487.
  80. L. Gresele, P. K. Rubenstein, A. Mehrjou, F. Locatello, and B. Schölkopf, "The incomplete Rosetta stone problem: Identifiability results for multi-view nonlinear ICA," 2019, arXiv:1905.06642. [Online]. Available: https://arxiv.org/abs/1905.06642 · arXiv
  81. A. Gretton, O. Bousquet, A. Smola, and B. Schölkopf, "Measuring statistical dependence with Hilbert-Schmidt norms," in Algorithmic Learning Theory. Springer-Verlag, 2005, pp. 63-78.
  82. A. Gretton, R. Herbrich, A. Smola, O. Bousquet, and B. Schölkopf, "Kernel methods for measuring independence," J. Mach. Learn. Res., vol. 6, pp. 2075-2129, Dec. 2005.
  83. J.-B. Grill et al., "Bootstrap your own latent: A new approach to self-supervised learning," 2020, arXiv:2006.07733. [Online]. Available: http://arxiv.org/abs/2006.07733 · arXiv
  84. R. Grzeszczuk, D. Terzopoulos, and G. Hinton, "NeuroAnimator: Fast neural network emulation and control of physics-based models," in Proc. 25th Annu. Conf. Comput. Graph. Interact. Techn., 1998, pp. 9-20.
  85. K. Gu, B. Yang, J. Ngiam, Q. Le, and J. Shlens, "Using videos to evaluate image model robustness," 2019, arXiv:1904.10076. [Online]. Available: http://arxiv.org/abs/1904.10076 · arXiv
  86. S. Gu and L. Rigazio, "Towards deep neural network architectures robust to adversarial examples," 2014, arXiv:1412.5068. [Online]. Available: http://arxiv.org/abs/1412.5068 · arXiv
  87. I. Guyon, D. Janzing, and B. Schölkopf, "Causality: Objectives and assessment," in Proc. JMLR Workshop Conf., vol. 6, I. Guyon, D. Janzing, and B. Schölkopf, Eds. Cambridge, MA, USA: MIT Press, 2010, pp. 1-42.
  88. D. Ha and J. Schmidhuber, "World models," 2018, arXiv:1803.10122. [Online]. Available: http://arxiv.org/abs/1803.10122 · arXiv
  89. T. Haavelmo, "The probability approach in econometrics," Econometrica, vol. 12, pp. S1-S115, Jul. 1944.
  90. H. Hälvä and A. Hyvärinen, "Hidden Markov nonlinear ICA: Unsupervised learning from nonstationary time series," 2020, arXiv:2006.12107. [Online]. Available: http://arxiv.org/abs/2006.12107 · arXiv
  91. K. He, H. Fan, Y. Wu, S. Xie, and R. Girshick, "Momentum contrast for unsupervised visual representation learning," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2020, pp. 9729-9738.
  92. S. He et al., "Learning to predict the cosmological structure formation," Proc. Nat. Acad. Sci. USA, vol. 116, no. 28, pp. 13825-13832, Jul. 2019.
  93. C. Heinze-Deml and N. Meinshausen, "Conditional variance penalties and domain shift robustness," 2017, arXiv:1710.11469. [Online]. Available: http://arxiv.org/abs/1710.11469 · arXiv
  94. C. Heinze-Deml, J. Peters, and N. Meinshausen, "Invariant causal prediction for nonlinear models," 2017, arXiv:1706.08576. [Online]. Available: http://arxiv.org/abs/1706.08576 · arXiv
  95. J. Henrich, The Secret our Success. Princeton, NJ, USA: Princeton Univ. Press, 2016.
  96. K. E. Henry, D. N. Hager, P. J. Pronovost, and S. Saria, "A targeted real-time early warning score (TREWScore) for septic shock," Sci. Transl. Med., vol. 7, no. 299, 2015, Art. no. 299ra122.
  97. I. Higgins et al., "Beta-VAE: Learning basic visual concepts with a constrained variational framework," in Proc. Int. Conf. Learn. Represent., 2016.
  98. K. D. Hoover, "Causality in economics and econometrics," in The New Palgrave Dictionary of Economics, S. N. Durlauf and L. E. Blume, Eds., 2nd ed. Basingstoke, U.K.: Palgrave Macmillan, 2008.
  99. J. Howard and S. Ruder, "Universal language model fine-tuning for text classification," 2018, arXiv:1801.06146. [Online]. Available: http://arxiv.org/abs/1801.06146 · arXiv
  100. P. O. Hoyer, D. Janzing, J. M. Mooij, J. Peters, and B. Schölkopf, "Nonlinear causal discovery with additive noise models," in Proc. Adv. Neural Inf. Process. Syst. (NIPS), 2009, pp. 689-696.
  101. B. Huang, K. Zhang, J. Zhang, J. Ramsey, R. Sanchez-Romero, C. Glymour, and B. Schölkopf, "Causal discovery from heterogeneous/nonstationary data," J. Mach. Learn. Res., vol. 21, no. 89, pp. 1-53, 2020.
  102. B. Huang, K. Zhang, J. Zhang, R. Sanchez-Romero, C. Glymour, and B. Schölkopf, "Behind distribution shift: Mining driving forces of changes and causal arrows," in Proc. IEEE Int. Conf. Data Mining (ICDM), Nov. 2017, pp. 913-918.
  103. A. Hyvarinen and H. Morioka, "Nonlinear ICA of temporally dependent stationary sources," in Proc. Mach. Learn. Res., 2017, pp. 460-469.
  104. A. Hyvärinen and P. Pajunen, "Nonlinear independent component analysis: Existence and uniqueness results," Neural Netw., vol. 12, no. 3, pp. 429-439, Apr. 1999.
  105. G. W. Imbens and D. B. Rubin, Causal Inference in Statistics, Social, and Biomedical Sciences. Cambridge, U.K.: Cambridge Univ. Press, 2015.
  106. D. Janzing, "Causal regularization," in Proc. Adv. Neural Inf. Process. Syst., 2019, pp. 12704-12714.
  107. D. Janzing, R. Chaves, and B. Schölkopf, "Algorithmic independence of initial condition and dynamical law in thermodynamics and causal inference," New J. Phys., vol. 18, no. 9, Sep. 2016, Art. no. 093052.
  108. D. Janzing, P. Hoyer, and B. Schölkopf, "Telling cause from effect based on high-dimensional observations," in Proc. 27th Int. Conf. Mach. Learn., J. Fürnkranz and T. Joachims, Eds., 2010, pp. 479-486.
  109. D. Janzing et al., "Information-geometric approach to inferring causal directions," Artif. Intell., vols. 182-183, pp. 1-31, May 2012.
  110. D. Janzing, J. Peters, J. M. Mooij, and B. Schölkopf, "Identifying confounders using additive noise models," in Proc. 25th Annu. Conf. Uncertainty Artif. Intell. (UAI), 2009, pp. 249-257.
  111. D. Janzing and B. Schölkopf, "Causal inference using the algorithmic Markov condition," IEEE Trans. Inf. Theory, vol. 56, no. 10, pp. 5168-5194, Oct. 2010.
  112. D. Janzing and B. Schölkopf, "Semi-supervised interpolation in an anticausal learning scenario," J. Mach. Learn. Res., vol. 16, no. 1, pp. 1923-1948, 2015.
  113. D. Janzing and B. Schölkopf, "Detecting non-causal artifacts in multivariate linear regression models," in Proc. 35th Int. Conf. Mach. Learn. (ICML), 2018, pp. 2250-2258.
  114. S. Karahan, M. K. Yildirum, K. Kirtac, F. S. Rende, G. Butun, and H. K. Ekenel, "How image degradations affect deep CNN-based face recognition," in Proc. Int. Conf. Biometrics Special Interest Group (BIOSIG), 2016, pp. 1-5.
  115. A.-H. Karimi, J. von Kügelgen, B. Schölkopf, and I. Valera, "Algorithmic recourse under imperfect causal knowledge: A probabilistic approach," 2020, arXiv:2006.06831. [Online]. Available: http://arxiv.org/abs/2006.06831 · arXiv
  116. N. R. Ke et al., "Learning neural causal models from unknown interventions," 2019, arXiv:1910.01075. [Online]. Available: http://arxiv.org/abs/1910.01075 · arXiv
  117. S. Tsirtsis, B. Tabibian, M. Khajehnejad, A. Singla, B. Schölkopf, and M. Gomez-Rodriguez, "Optimal decision making under strategic behavior," 2019, arXiv:1905.09239. [Online]. Available: http://arxiv.org/abs/1905.09239 · arXiv
  118. N. Kilbertus, G. Parascandolo, and B. Schölkopf, "Generalization in anti-causal learning," 2018, arXiv:1812.00524. [Online]. Available: http://arxiv.org/abs/1812.00524 · arXiv
  119. H. Kim and A. Mnih, "Disentangling by factorising," in Proc. Int. Conf. Mach. Learn., 2018, pp. 2649-2658.
  120. T. Kipf, E. Fetaya, K.-C. Wang, M. Welling, and R. Zemel, "Neural relational inference for interacting systems," in Proc. Int. Conf. Mach. Learn., 2018, pp. 2688-2697.
  121. A. Kolesnikov et al., "Big transfer (BiT): General visual representation learning," 2019, arXiv:1912.11370. [Online]. Available: http://arxiv.org/abs/1912.11370 · arXiv
  122. A. Kosiorek, H. Kim, Y. W. Teh, and I. Posner, "Sequential attend, infer, repeat: Generative modelling of moving objects," in Proc. Adv. Neural Inf. Process. Syst., vol. 31, 2018, pp. 8606-8616.
  123. S. Kpotufe, E. Sgouritsa, D. Janzing, and B. Schölkopf, "Consistency of causal inference under the additive noise model," in Proc. 31st Int. Conf. Mach. Learn., 2014, pp. 478-486.
  124. T. D. Kulkarni et al., "Unsupervised learning of object keypoints for perception and control," in Proc. Adv. Neural Inf. Process. Syst., 2019, pp. 10723-10733.
  125. M. J. Kusner, J. Loftus, C. Russell, and R. Silva, "Counterfactual fairness," in Advances in Neural Information Processing Systems. Red Hook, NY, USA: Curran Associates, 2017, pp. 4066-4076.
  126. L. Ladický, S. Jeong, B. Solenthaler, M. Pollefeys, and M. Gross, "Data-driven fluid simulations using regression forests," ACM Trans. Graph., vol. 34, no. 6, pp. 1-9, Nov. 2015.
  127. B. M. Lake, T. D. Ullman, J. B. Tenenbaum, and S. J. Gershman, "Building machines that learn and think like people," Behav. Brain Sci., vol. 40, p. e253, Jan. 2017.
  128. J. Landman, E. A. Vandewater, A. J. Stewart, and J. E. Malley, "Missed opportunities: Psychological ramifications of counterfactual thought in midlife women," J. Adult Develop., vol. 2, no. 2, pp. 87-97, Apr. 1995.
  129. S. Lange, T. Gabel, and M. Riedmiller, "Batch reinforcement learning," in Reinforcement Learning: State-of-the-Art, M. Wiering and M. van Otterlo, Eds. Berlin, Germany: Springer, 2012, pp. 45-73.
  130. S. L. Lauritzen, Graphical Models. New York, NY, USA: Oxford Univ. Press, 1996.
  131. F. Leeb, Y. Annadani, S. Bauer, and B. Schölkopf, "Structural autoencoders improve representations for generation and transfer," 2020, arXiv:2006.07796. [Online]. Available: http://arxiv.org/abs/2006.07796 · arXiv
  132. S. Levine, A. Kumar, G. Tucker, and J. Fu, "Offline reinforcement learning: Tutorial, review, and perspectives on open problems," 2020, arXiv:2005.01643. [Online]. Available: http://arxiv.org/abs/2005.01643 · arXiv
  133. D. Lewis, "Causation," J. Philosophy, vol. 70, no. 17, pp. 556-567, 1974.
  134. Y. Li, M. Gong, X. Tian, T. Liu, and D. Tao, "Domain generalization via conditional invariant representation," 2018, arXiv:1807.08479. [Online]. Available: http://arxiv.org/ abs/1807.08479 · arXiv
  135. Y. Li et al., "Deep domain generalization via conditional invariant adversarial networks," in Proc. Eur. Conf. Comput. Vis. (ECCV), 2018, pp. 624-639.
  136. S. Lim, I. Kim, T. Kim, C. Kim, and S. Kim, "Fast autoaugment," in Proc. Adv. Neural Inf. Process. Syst., 2019, pp. 6665-6675.
  137. Z. Lin et al., "Space: Unsupervised object-oriented scene representation via spatial attention and decomposition," in Proc. Int. Conf. Learn. Represent., 2019.
  138. Z. C. Lipton, Y.-X. Wang, and A. Smola, "Detecting and correcting for label shift with black box predictors," 2018, arXiv:1802.03916. [Online]. Available: http://arxiv.org/abs/1802.03916 · arXiv
  139. F. Locatello, G. Abbati, T. Rainforth, S. Bauer, B. Schölkopf, and O. Bachem, "On the fairness of disentangled representations," in Proc. Adv. Neural Inf. Process. Syst., 2019, pp. 14544-14557.
  140. F. Locatello et al., "Challenging common assumptions in the unsupervised learning of disentangled representations," in Proc. 36th Int. Conf. Mach. Learn., 2019, pp. 4114-4124.
  141. F. Locatello, B. Poole, G. Rätsch, B. Schölkopf, O. Bachem, and M. Tschannen, "Weakly-supervised disentanglement without compromises," in Proc. 37th Int. Conf. Mach. Learn. (ICML), 2020, pp. 6348-6359.
  142. F. Locatello et al., "Object-centric learning with slot attention," in Proc. Adv. Neural Inf. Process. Syst., 2020.
  143. D. Lopez-Paz, K. Muandet, B. Schölkopf, and I. Tolstikhin, "Towards a learning theory of cause-effect inference," in Proc. 32nd Int. Conf. Mach. Learn., 2015, pp. 1452-1461.
  144. D. Lopez-Paz, R. Nishihara, S. Chintala, B. Scholkopf, and L. Bottou, "Discovering causal signals in images," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jul. 2017, pp. 58-66.
  145. C. Lu, B. Huang, K. Wang, J. Miguel Hernández-Lobato, K. Zhang, and B. Schölkopf, "Sample-efficient reinforcement learning via counterfactual-based data augmentation," 2020, arXiv:2012.09092. [Online]. Available: http://arxiv.org/abs/2012.09092 · arXiv
  146. C. Lu, B. Schölkopf, and J. M. Hernández-Lobato, "Deconfounding reinforcement learning in observational settings," 2018, arXiv:1812.10576. [Online]. Available: http://arxiv.org/ abs/1812.10576 · arXiv
  147. A. S. Lundervold and A. Lundervold, "An overview of deep learning in medical imaging focusing on MRI," Zeitschrift für Medizinische Physik, vol. 29, no. 2, pp. 102-127, May 2019.
  148. S. Magliacane, T. van Ommen, T. Claassen, S. Bongers, P. Versteeg, and J. M. Mooij, "Domain adaptation by using causal inference to predict invariant conditional distributions," in Proc. NeurIPS, 2018, pp. 10869-10879.
  149. R. Matthews, "Storks deliver babies (p= 0.008)," Teaching Statist., vol. 22, no. 2, pp. 36-38, 2000.
  150. N. Meinshausen, "Causality from a distributional robustness point of view," in Proc. IEEE Data Sci. Workshop (DSW), Jun. 2018, pp. 6-10.
  151. C. Michaelis et al., "Benchmarking robustness in object detection: Autonomous driving when winter is coming," 2019, arXiv:1907.07484. [Online]. Available: http://arxiv.org/ abs/1907.07484 · arXiv
  152. V. Mnih et al., "Human-level control through deep reinforcement learning," Nature, vol. 518, no. 7540, pp. 529-533, 2015.
  153. B. T. Montet et al., "Stellar and planetary properties of K2 campaign 1 candidates and validation of 17 planets, including a planet receiving earth-like insolation," Astrophys. J., vol. 809, no. 1, p. 25, 2015.
  154. J. Mooij, D. Janzing, and B. Schölkopf, "From ordinary differential equations to structural causal models: The deterministic case," in Proc. 29th Conf. Annu. Conf. Uncertainty Artif. Intell., A. Nicholson and P. Smyth, Eds. Corvallis, OR, USA: AUAI Press, 2013, pp. 440-448.
  155. J. M. Mooij, D. Janzing, T. Heskes, and B. Schölkopf, "On causal discovery with cyclic additive noise models," in Proc. Adv. Neural Inf. Process. Syst. (NIPS), 2011, pp. 639-647.
  156. J. M. Mooij, D. Janzing, J. Peters, and B. Schölkopf, "Regression by dependence minimization and its application to causal inference," in Proc. 26th Int. Conf. Mach. Learn. (ICML), 2009, pp. 745-752.
  157. J. M. Mooij, J. Peters, D. Janzing, J. Zscheischler, and B. Schölkopf, "Distinguishing cause from effect using observational data: Methods and benchmarks," J. Mach. Learn. Res., vol. 17, no. 32, pp. 1-102, 2016.
  158. D. Mrowca et al., "Flexible neural representation for physics prediction," in Proc. Adv. Neural Inf. Process. Syst., 2018, pp. 8799-8810.
  159. J. Oh, X. Guo, H. Lee, R. L. Lewis, and S. Singh, "Action-conditional video prediction using deep networks in Atari games," in Proc. Adv. Neural Inf. Process. Syst., 2015, pp. 2863-2871.
  160. G. Parascandolo, N. Kilbertus, M. Rojas-Carulla, and B. Schölkopf, "Learning independent causal mechanisms," in Proc. 35th Int. Conf. Mach. Learn. (PMLR), vol. 80, 2018, pp. 4036-4044.
  161. G. Parascandolo, A. Neitz, A. ORVIETO, L. Gresele, and B. Schölkopf, "Learning explanations that are hard to vary," in Proc. Int. Conf. Learn. Represent., 2021.
  162. G. Parascandolo, M. Rojas-Carulla, N. Kilbertus, and B. Schölkopf, "Learning independent causal mechanisms," in Proc. Workshop Learn. Disentangled Represent. From Perception Control 31st Conf. Neural Inf. Process. Syst. (NIPS), 2017.
  163. J. Pearl and E. Bareinboim, "External validity: From do-calculus to transportability across populations," 2015, arXiv:1503.01603. [Online]. Available: http://arxiv.org/abs/1503.01603 · arXiv
  164. J. Peters, S. Bauer, and N. Pfister, "Causal models for dynamical systems," 2020, arXiv:2001.06208. [Online]. Available: http://arxiv.org/abs/ 2001.06208 · arXiv
  165. J. Peters, P. Bühlmann, and N. Meinshausen, "Causal inference by using invariant prediction: Identification and confidence intervals," J. Roy. Stat. Soc. B, Stat. Methodol., vol. 78, no. 5, pp. 947-1012, Nov. 2016.
  166. J. Peters, D. Janzing, and B. Schölkopf," Elements of Causal Inference-Foundations and Learning Algorithms. Cambridge, MA, USA: MIT Press, 2017.
  167. J. Peters, J. M. Mooij, D. Janzing, and B. Schölkopf, "Identifiability of causal graphs using functional models," in Proc. 27th Annu. Conf. Uncertainty Artif. Intell. (UAI), 2011, pp. 589-598.
  168. J. Peters, J. M. Mooij, D. Janzing, and B. Schölkopf, "Causal discovery with continuous additive noise models," J. Mach. Learn. Res., vol. 15, pp. 2009-2053, Jan. 2014.
  169. N. Pfister, S. Bauer, and J. Peters, "Learning stable and predictive structures in kinetic systems," Proc. Nat. Acad. Sci. USA, vol. 116, no. 51, pp. 25405-25411, Dec. 2019.
  170. N. Pfister, P. Bühlmann, and J. Peters, "Invariant causal prediction for sequential data," J. Amer. Stat. Assoc., vol. 114, no. 527, pp. 1264-1276, Jul. 2019.
  171. N. Pfister, P. Bühlmann, B. Schölkopf, and J. Peters, "Kernel-based tests for joint independence," J. Roy. Stat. Soc. B, Stat. Methodol., vol. 80, no. 1, pp. 5-31, Jan. 2018.
  172. R. L. Priol, R. B. Harikandeh, Y. Bengio, and S. Lacoste-Julien, "An analysis of the adaptation speed of causal models," 2020, arXiv:2005.09136. [Online]. Available: http://arxiv.org/abs/ 2005.09136 · arXiv
  173. N. Rahaman et al., "Spatially structured recurrent modules," in Proc. Int. Conf. Learn. Represent., 2021.
  174. H. Reichenbach, The Direction Time. Berkeley, CA, USA: Univ. of California Press, 1956.
  175. L. K. Reichert and J. R. Slate, "Reflective learning: The use of 'if only ...' statements to improve performance," Social Psychol. Educ., vol. 3, no. 4, pp. 261-275, 1999.
  176. D. J. Rezende et al., "Causally correct partial models for reinforcement learning," 2020, arXiv:2002.02836. [Online]. Available: http://arxiv.org/abs/2002.02836 · arXiv
  177. J. G. Richens, C. M. Lee, and S. Johri, "Improving the accuracy of medical diagnosis with causal machine learning," Nature Commun., vol. 11, no. 1, p. 3923, Dec. 2020.
  178. K. Ridgeway and M. C. Mozer, "Learning deep disentangled embeddings with the f-statistic loss," in Proc. Adv. Neural Inf. Process. Syst., 2018, pp. 185-194.
  179. N. J. Roese, "The functional basis of counterfactual thinking," J. Personality Social Psychol., vol. 66, no. 5, p. 805, 1994.
  180. M. Rojas-Carulla, B. Schölkopf, R. Turner, and J. Peters, "Invariant models for causal transfer learning," J. Mach. Learn. Res., vol. 19, no. 36, pp. 1-34, 2018.
  181. M. Rolinek, D. Zietlow, and G. Martius, "Variational autoencoders pursue PCA directions (by Accident)," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2019, pp. 12406-12415.
  182. P. Roy, S. Ghosh, S. Bhattacharya, and U. Pal, "Effects of degradations on deep neural network architectures," 2018, arXiv:1807.10108. [Online]. Available: http://arxiv.org/abs/1807.10108 · arXiv
  183. P. K. Rubenstein, S. Bongers, B. Schölkopf, and J. M. Mooij, "From deterministic ODEs to dynamic structural causal models," in Proc. 34th Conf. Uncertainty Artif. Intell. (UAI), 2018, pp. 114-123.
  184. P. K. Rubenstein et al., "Causal consistency of structural equation models," in Proc. Thirty-Third Conf. Uncertainty Artif. Intell., 2017, pp. 808-817.
  185. S. Ruder, "An overview of multi-task learning in deep neural networks," 2017, arXiv:1706.05098. [Online]. Available: http://arxiv.org/abs/ 1706.05098 · arXiv
  186. A. Sanchez-Gonzalez, J. Godwin, T. Pfaff, R. Ying, J. Leskovec, and P. W. Battaglia, "Learning to simulate complex physics with graph networks," 2020, arXiv:2002.09405. [Online]. Available: http://arxiv.org/abs/2002.09405 · arXiv
  187. A. Santoro et al., "A simple neural network module for relational reasoning," in Proc. Adv. Neural Inf. Process. Syst., 2017, pp. 4967-4976.
  188. J. Schmidhuber, "Evolutionary principles in self-referential learning, or on learning how to learn: The meta-meta-... hook," Ph.D. dissertation, Technische Universität München, München, Germany, 1987.
  189. J. Schmidhuber, "Curious model-building control systems," in Proc. IEEE Int. Joint Conf. Neural Netw., Nov. 1991, pp. 1458-1463.
  190. B. Schölkopf, "Artificial intelligence: Learning to see and act," Nature, vol. 518, no. 7540, pp. 486-487, 2015.
  191. B. Schölkopf, "Causal learning, 2017," in Proc. 34th Int. Conf. Mach. Learn. (ICML), 2017. [Online]. Available: https://vimeo. com/238274659
  192. B. Schölkopf, "Causality for machine learning," 2019, arXiv:1911.10500. [Online]. Available: https://arxiv.org/abs/1911.10500 · arXiv
  193. B. Schölkopf et al., "Modeling confounding by half-sibling regression," Proc. Nat. Acad. Sci. USA, vol. 113, no. 27, pp. 7391-7398, Jul. 2016.
  194. B. Schölkopf, D. Janzing, and D. Lopez-Paz, "Causal and statistical learning," Oberwolfach Rep., vol. 13, no. 3, pp. 1896-1899, 2016.
  195. B. Schölkopf, D. Janzing, J. Peters, E. Sgouritsa, K. Zhang, and J. M. Mooij, "On causal and anticausal learning," in Proc. 29th Int. Conf. Mach. Learn. (ICML), 2012, pp. 1255-1262.
  196. B. Schölkopf and A. J. Smola, Learning With Kernels. Cambridge, MA, USA: MIT Press, 2002.
  197. L. Schott, J. Rauber, M. Bethge, and W. Brendel, "Towards the first adversarially robust neural network model on MNIST," in Proc. Int. Conf. Learn. Represent., 2019.
  198. J. Schrittwieser et al., "Mastering Atari, go, chess and Shogi by planning with a learned model," 2019, arXiv:1911.08265. [Online]. Available: http://arxiv.org/abs/1911.08265 · arXiv
  199. P. Schulam and S. Saria, "Reliable decision support using counterfactual models," in Proc. Adv. Neural Inf. Process. Syst., 2017, pp. 1697-1708.
  200. R. D. Shah and J. Peters, "The hardness of conditional independence testing and the generalised covariance measure," 2018, arXiv:1804.07203. [Online]. Available: http://arxiv.org/abs/1804.07203 · arXiv

200 of 257 shown

Cited by

Sorted by citations

Authors
Scholkopf, B., Locatello, F., Bauer, S., Ke, N. R., Kalchbrenner, N., Goyal, A., & Bengio, Y.
Affiliations
  • Google (United States) (US)
  • Max Planck Institute for Intelligent Systems (DE)
  • Mila - Quebec Artificial Intelligence Institute (CA)
  • Canadian Institute for Advanced Research (CA)
ORCID
Venue
Proceedings of the IEEE
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Published
2021-05
Volume
109
Issue
5
Pages
612-634
ISSN
  • 0018-9219 (ISSN-L)
  • 1558-2256
DOI
10.1109/jproc.2021.3058954
OpenAlex
W3135588948
Cited by
1,170
References
20
Citations by year
2026: 294 · 2025: 278 · 2024: 201 · 2023: 177 · 2022: 123 · 2021: 93 · 2020: 2 · 2019: 1 · 2018: 1
FWCI
52.48
Type
journal article
Language
en
Open access
Licence
Locations
  • Proceedings of the IEEE (publishedVersion, cc-by)
  • Repository for Publications and Research Data (ETH Zurich) (submittedVersion, cc-by)
  • ETH Zürich Research Collection (cc-by)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Bayesian Modeling and Causal Inference
  • Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
Keywords
causal representation learning, causal inference, causal discovery, transfer learning, generalization

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review