CoChat
Sign Up
PreprintCited by 26Reinforcement Learning in RoboticsBayesian Modeling and Causal InferenceAI-based Problem Solving and Planning

Counterfactual Data Augmentation using Locally Factored Dynamics

Open access
Pitis, Silviu, Creager, Elliot, Garg, Animesh·2020·arXiv
DOI 10.48550/arxiv.2007.02863arXiv 2007.02863
Open this paper in CoChat — free accountOpen PDF

Abstract

Many dynamic processes, including common scenarios in robotic control and reinforcement learning (RL), involve a set of interacting subprocesses. Though the subprocesses are not independent, their interactions are often sparse, and the dynamics at any given time step can often be decomposed into locally independent causal mechanisms. Such local causal structures can be leveraged to improve the sample efficiency of sequence prediction and off-policy reinforcement learning. We formalize this by introducing local causal models (LCMs), which are induced from a global causal model by conditioning on a subset of the state space. We propose an approach to inferring these structures given an object-oriented state representation, as well as a novel algorithm for Counterfactual Data Augmentation (CoDA). CoDA uses local structures and an experience replay to generate counterfactual experiences that are causally valid in the global model. We find that CoDA significantly improves the performance of RL agents in locally factored tasks, including the batch-constrained and goal-conditioned settings.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2007.02863

Cites

8 of 8 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Offline data augmentation can be effective for generating larger dataset for robot manipulation.

    “Offline data augmentation can be effective for generating larger dataset for robot manipulation [7, 35-45]”

    From MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations (2023) · cites this paper as [41]

    The Experiments section (p. 8) reports that enlarging a HER agent’s dataset with CoDA approximately doubled sample efficiency in FetchPush and Slide2 manipulation tasks. It separately tests augmentation of a fixed, offline dataset in Pong (p. 8); the manipulation experiments collect new environment data during training, so they do not establish effectiveness in an offline robot-manipulation setting. (LLM, conf=0.85)

    Experimental — accuracy not yet measured

  • It can be difficult to generate plausible interactions without prior knowledge of physics or causal dependencies, especially for new scenes, objects, or robots.

    “however, it can be difficult to generate plausible interactions without prior knowledge of physics [35] or causal dependencies [41,42], especially for new scenes, objects, or robots.”

    From MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations (2023) · cites this paper as [41]

    Section 3 (p. 6) says generating counterfactual data generally requires either a domain expert’s approximate causal model or local causal structure learned from data. This supports the importance of causal dependencies, but the paper does not establish that generating plausible interactions is especially difficult for new scenes, objects, or robots; it also presents learning those dependencies as an alternative to prior knowledge. (LLM, conf=0.83)

    Experimental — accuracy not yet measured

  • Because data augmentation is offline, it can be difficult to generate plausible interactions without prior knowledge of physics or causal dependencies, especially for new scenes, objects, or robots.

    “However, because data augmentation is offline, it can be difficult to generate plausible interactions without prior knowledge of physics [35] or causal dependencies [41,42], especially for new scenes, objects, or robots.”

    From MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations (2023) · cites this paper as [41]

    The Introduction (p. 1) says knowledge of local interactions enables plausible counterfactual samples from observed transitions, supporting the role of causal dependencies. Section 3 (p. 6) says those dependencies can also be learned from data rather than known beforehand. The paper does not establish that offline augmentation is especially difficult for new scenes, objects, or robots. (LLM, conf=0.83)

    Experimental — accuracy not yet measured

  • Several works have used offline data augmentation to increase the dataset size for learning policies.

    “Several works have used offline data augmentation to increase the dataset size for learning policies [7, 35-45].”

    From MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations (2023) · cites this paper as [41]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 81

  1. [1][1] Marcin Andrychowicz, Filip Wolski, Alex Ray, Jonas Schneider, Rachel Fong, Peter Welinder, Bob McGrew, Josh Tobin, OpenAI Pieter Abbeel, and Wojciech Zaremba. Hindsight experience replay. In Advances in neural information processing systems , pages 5048-5058, 2017. -
  2. [2][2] OpenAI: Marcin Andrychowicz, Bowen Baker, Maciek Chociej, Rafal Jozefowicz, Bob McGrew, Jakub Pachocki, Arthur Petron, Matthias Plappert, Glenn Powell, Alex Ray, et al. Learning dexterous in-hand manipulation. The International Journal of Robotics Research , 39(1):3-20, 2020. -
  3. [3][3] Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein gan. arXiv preprint arXiv:1701.07875 , 2017. - · arXiv
  4. [4][4] Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. Layer normalization. arXiv preprint arXiv:1607.06450 , 2016. - · arXiv
  5. [5][5] Elias Bareinboim and Judea Pearl. Controlling selection bias in causal inference. In Artificial Intelligence and Statistics , pages 100-108, 2012. -
  6. [6][6] Elias Bareinboim, Jin Tian, and Judea Pearl. Recovering from selection bias in causal and statistical inference. In AAAI , pages 2410-2416. Citeseer, 2014. -
  7. [7][7] Marc G Bellemare, Yavar Naddaf, Joel Veness, and Michael Bowling. The arcade learning environment: An evaluation platform for general agents. Journal of Artificial Intelligence Research , 47:253-279, 2013. -
  8. [9][9] Craig Boutilier, Richard Dearden, Moises Goldszmidt, et al. Exploiting structure in policy construction. In IJCAI , volume 14, pages 1104-1113, 1995. -
  9. [10][10] Craig Boutilier, Nir Friedman, Moises Goldszmidt, and Daphne Koller. Context-specific independence in bayesian networks. arXiv preprint arXiv:1302.3562 , 2013. - · arXiv
  10. [11][11] Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. Openai gym. arXiv preprint arXiv:1606.01540 , 2016. - · arXiv
  11. [12][12] Lars Buesing, Theophane Weber, Yori Zwols, Sebastien Racaniere, Arthur Guez, Jean-Baptiste Lespiau, and Nicolas Heess. Woulda, coulda, shoulda: Counterfactually-guided policy search. International Conference on Learning Representations , 2019. -
  12. [13][13] Christopher P Burgess, Loic Matthey, Nicholas Watters, Rishabh Kabra, Irina Higgins, Matt Botvinick, and Alexander Lerchner. Monet: Unsupervised scene decomposition and representation. arXiv preprint arXiv:1901.11390 , 2019. - · arXiv
  13. [14][14] Karl Cobbe, Oleg Klimov, Chris Hesse, Taehoon Kim, and John Schulman. Quantifying generalization in reinforcement learning. arXiv preprint arXiv:1812.02341 , 2018. - · arXiv
  14. [15][15] Carlos Diuk, Andre Cohen, and Michael L Littman. An object-oriented representation for efficient reinforcement learning. In Proceedings of the 25th international conference on Machine learning , pages 240-247, 2008. -
  15. [16][16] Aysegul Dundar, Kevin J Shih, Animesh Garg, Robert Pottorf, Andrew Tao, and Bryan Catanzaro. Unsupervised disentanglement of pose, appearance and background from images and videos. arXiv preprint arXiv:2001.09518 , 2020. - · arXiv
  16. [17][17] Babak Esmaeili, Hao Wu, Sarthak Jain, Alican Bozkurt, N Siddharth, Brooks Paige, Dana H Brooks, Jennifer Dy, and Jan-Willem Meent. Structured disentangled representations. In The 22nd International Conference on Artificial Intelligence and Statistics , pages 2525-2534, 2019. -
  17. [18][18] Amir-massoud Farahmand, Andre Barreto, and Daniel Nikovski. Value-aware loss function for modelbased reinforcement learning. In Artificial Intelligence and Statistics , pages 1486-1494, 2017. -
  18. [19][19] Jakob N Foerster, Gregory Farquhar, Triantafyllos Afouras, Nantas Nardelli, and Shimon Whiteson. Counterfactual multi-agent policy gradients. In Thirty-second AAAI conference on artificial intelligence , 2018. -
  19. [20][20] Justin Fu, Aviral Kumar, Matthew Soh, and Sergey Levine. Diagnosing bottlenecks in deep q-learning algorithms. arXiv preprint arXiv:1902.10250 , 2019. - · arXiv
  20. [21][21] Scott Fujimoto, David Meger, and Doina Precup. Off-policy deep reinforcement learning without exploration. arXiv preprint arXiv:1812.02900 , 2018. - · arXiv
  21. [22][22] Scott Fujimoto, Herke Van Hoof, and David Meger. Addressing function approximation error in actor-critic methods. arXiv preprint arXiv:1802.09477 , 2018. - · arXiv
  22. [23][23] Sahaj Garg, Vincent Perot, Nicole Limtiaco, Ankur Taly, Ed H Chi, and Alex Beutel. Counterfactual fairness in text classification through robustness. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society , pages 219-226, 2019. -
  23. [25][25] Mathieu Germain, Karol Gregor, Iain Murray, and Hugo Larochelle. Made: Masked autoencoder for distribution estimation. In International Conference on Machine Learning , pages 881-889, 2015. -
  24. [26][26] Anirudh Goyal, Alex Lamb, Jordan Hoffmann, Shagun Sodhani, Sergey Levine, Yoshua Bengio, and Bernhard Schölkopf. Recurrent independent mechanisms. arXiv preprint arXiv:1909.10893 , 2019. - · arXiv
  25. [27][27] Clive WJ Granger. Investigating causal relations by econometric models and cross-spectral methods. Econometrica: journal of the Econometric Society , pages 424-438, 1969. -
  26. [28][28] Carlos Guestrin, Daphne Koller, Ronald Parr, and Shobha Venkataraman. Efficient solution algorithms for factored mdps. Journal of Artificial Intelligence Research , 19:399-468, 2003. -
  27. [29][29] Assaf Hallak, François Schnitzler, Timothy Mann, and Shie Mannor. Off-policy model-based learning under unknown factored dynamics. In International Conference on Machine Learning , pages 711-719, 2015. -
  28. [30][30] Dan Hendrycks and Kevin Gimpel. Gaussian error linear units (gelus). arXiv preprint arXiv:1606.08415 , 2016. - · arXiv
  29. [32][32] Irina Higgins, Nicolas Sonnerat, Loic Matthey, Arka Pal, Christopher P Burgess, Matko Bošnjak, Murray Shanahan, Matthew Botvinick, Demis Hassabis, and Alexander Lerchner. SCAN: Learning hierarchical compositional visual concepts. In International Conference on Learning Representations , 2018. URL https://openreview.net/forum?id=rkN2Il-RZ . -
  30. [33][33] Rodrigo Toro Icarte, Toryn Klassen, Richard Valenzano, and Sheila McIlraith. Using reward machines for high-level task specification and decomposition in reinforcement learning. In International Conference on Machine Learning , pages 2107-2116, 2018. -
  31. [34][34] Michael Janner, Justin Fu, Marvin Zhang, and Sergey Levine. When to trust your model: Model-based policy optimization. In Advances in Neural Information Processing Systems , pages 12498-12509, 2019. -
  32. [35][35] Matthew Johnson, Katja Hofmann, Tim Hutton, and David Bignell. The malmo platform for artificial intelligence experimentation. In IJCAI , pages 4246-4247, 2016. -
  33. [36][36] Leslie Pack Kaelbling. Learning to achieve goals. In IJCAI , pages 1094-1099. Citeseer, 1993. -
  34. [37][37] Michael Kearns and Daphne Koller. Efficient reinforcement learning in factored mdps. In IJCAI , volume 16, pages 740-747, 1999. -
  35. [39][39] Thomas Kipf, Elise van der Pol, and Max Welling. Contrastive learning of structured world models. arXiv preprint arXiv:1911.12247 , 2019. - · arXiv
  36. [41][41] Ilya Kostrikov, Denis Yarats, and Rob Fergus. Image augmentation is all you need: Regularizing deep reinforcement learning from pixels. arXiv preprint arXiv:2004.13649 , 2020. - · arXiv
  37. [42][42] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems , pages 1097-1105, 2012. -
  38. [43][43] Aviral Kumar, Abhishek Gupta, and Sergey Levine. Discor: Corrective feedback in reinforcement learning via distribution correction. arXiv preprint arXiv:2003.07305 , 2020. - · arXiv
  39. [44][44] Sébastien Lachapelle, Philippe Brouillard, Tristan Deleu, and Simon Lacoste-Julien. Gradient-based neural dag learning. arXiv preprint arXiv:1906.02226 , 2019. - · arXiv
  40. [45][45] Sameera Lanka and Tianfu Wu. Archer: Aggressive rewards to counter bias in hindsight experience replay. arXiv preprint arXiv:1809.02070 , 2018. - · arXiv
  41. [46][46] Michael Laskin, Kimin Lee, Adam Stooke, Lerrel Pinto, Pieter Abbeel, and Aravind Srinivas. Reinforcement learning with augmented data. arXiv preprint arXiv:2004.14990 , 2020. - · arXiv
  42. [48][48] Sergey Levine, Aviral Kumar, George Tucker, and Justin Fu. Offline reinforcement learning: Tutorial, review, and perspectives on open problems. arXiv preprint arXiv:2005.01643 , 2020. - · arXiv
  43. [49][49] Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy M Hospedales. Learning to generalize: Meta-learning for domain generalization. In Thirty-Second AAAI Conference on Artificial Intelligence , 2018. -
  44. [50][50] Richard Li, Allan Jabri, Trevor Darrell, and Pulkit Agrawal. Towards practical multi-object manipulation using relational reinforcement learning. arXiv preprint arXiv:1912.11032 , 2019. - · arXiv
  45. [51][51] Timothy P Lillicrap, Jonathan J Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971 , 2015. - · arXiv
  46. [52][52] Francesco Locatello, Stefan Bauer, Mario Lucic, Gunnar Rätsch, Sylvain Gelly, Bernhard Schölkopf, and Olivier Bachem. Challenging common assumptions in the unsupervised learning of disentangled representations. arXiv preprint arXiv:1811.12359 , 2018. - · arXiv
  47. [53][53] Francesco Locatello, Ben Poole, Gunnar Rätsch, Bernhard Schölkopf, Olivier Bachem, and Michael Tschannen. Weakly-supervised disentanglement without compromises. arXiv preprint arXiv:2002.02886 , 2020. - · arXiv
  48. [54][54] Ryan Lowe, Yi Wu, Aviv Tamar, Jean Harb, OpenAI Pieter Abbeel, and Igor Mordatch. Multi-agent actor-critic for mixed cooperative-competitive environments. In Advances in neural information processing systems , pages 6379-6390, 2017. -
  49. [55][55] Chaochao Lu, Bernhard Schölkopf, and José Miguel Hernández-Lobato. Deconfounding reinforcement learning in observational settings. arXiv preprint arXiv:1812.10576 , 2018. - · arXiv
  50. [56][56] Daoming Lyu, Fangkai Yang, Bo Liu, and Steven Gustafson. Sdrl: Interpretable and data-efficient deep reinforcement learning leveraging symbolic planning. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 33, pages 2970-2977, 2019. -
  51. [58][58] Travis Mandel, Yun-En Liu, Sergey Levine, Emma Brunskill, and Zoran Popovic. Offline policy evaluation across representations with applications to educational games. In AAMAS , pages 1077-1084, 2014. -
  52. [59][59] Alexander Mott, Daniel Zoran, Mike Chrzanowski, Daan Wierstra, and Danilo Jimenez Rezende. Towards interpretable reinforcement learning using attention augmented agents. In Advances in Neural Information Processing Systems , pages 12329-12338, 2019. -
  53. [60][60] Rémi Munos, Tom Stepleton, Anna Harutyunyan, and Marc Bellemare. Safe and efficient off-policy reinforcement learning. In Advances in Neural Information Processing Systems , pages 1054-1062, 2016. -
  54. [61][61] Ashvin V Nair, Vitchyr Pong, Murtaza Dalal, Shikhar Bahl, Steven Lin, and Sergey Levine. Visual reinforcement learning with imagined goals. In Advances in Neural Information Processing Systems , pages 9191-9200, 2018. -
  55. [62][62] Michael Oberst and David Sontag. Counterfactual off-policy evaluation with gumbel-max structural causal models. arXiv preprint arXiv:1905.05824 , 2019. - · arXiv
  56. [63][63] Junhyuk Oh, Satinder Singh, and Honglak Lee. Value prediction network. In Advances in Neural Information Processing Systems , pages 6118-6128, 2017. -
  57. [66][66] Luis Perez and Jason Wang. The effectiveness of data augmentation in image classification using deep learning. arXiv preprint arXiv:1712.04621 , 2017. - · arXiv
  58. [67][67] Jonas Peters, Dominik Janzing, and Bernhard Schölkopf. Elements of causal inference: foundations and learning algorithms . MIT press, 2017. -
  59. [68][68] Silviu Pitis, Harris Chan, and Jimmy Ba. Protoge: Prototype goal encodings for multi-goal reinforcement learning. The 4th Multidisciplinary Conference on Reinforcement Learning and Decision Making (RLDM2019) , 2019. -
  60. [70][70] Silviu Pitis, Harris Chan, Stephen Zhao, Bradly Stadie, and Jimmy Ba. Maximum entropy gain exploration for long horizon multi-goal reinforcement learning. In Proceedings of the Thirty-seventh International Conference on Machine Learning , 2020. -
  61. [71][71] Matthias Plappert, Marcin Andrychowicz, Alex Ray, Bob McGrew, Bowen Baker, Glenn Powell, Jonas Schneider, Josh Tobin, Maciek Chociej, Peter Welinder, et al. Multi-goal reinforcement learning: Challenging robotics environments and request for research. arXiv preprint arXiv:1802.09464 , 2018. - · arXiv
  62. [72][72] Martin L Puterman. Markov decision processes: discrete stochastic dynamic programming . John Wiley & Sons, 2014. -
  63. [73][73] Aniruddh Raghu, Omer Gottesman, Yao Liu, Matthieu Komorowski, Aldo Faisal, Finale Doshi-Velez, and Emma Brunskill. Behaviour policy estimation in off-policy policy evaluation: Calibration matters. arXiv preprint arXiv:1807.01066 , 2018. - · arXiv
  64. [74][74] Danilo J Rezende, Ivo Danihelka, George Papamakarios, Nan Rosemary Ke, Ray Jiang, Theophane Weber, Karol Gregor, Hamza Merzic, Fabio Viola, Jane Wang, et al. Causally correct partial models for reinforcement learning. arXiv preprint arXiv:2002.02836 , 2020. - · arXiv
  65. [75][75] Jakob Runge, Sebastian Bathiany, Erik Bollt, Gustau Camps-Valls, Dim Coumou, Ethan Deyle, Clark Glymour, Marlene Kretschmer, Miguel D Mahecha, Jordi Muñoz-Marí, et al. Inferring causation from time series in earth system sciences. Nature communications , 10(1):1-13, 2019. -
  66. [76][76] Tom Schaul, Daniel Horgan, Karol Gregor, and David Silver. Universal value function approximators. In International conference on machine learning , pages 1312-1320, 2015. -
  67. [77][77] Tom Schaul, John Quan, Ioannis Antonoglou, and David Silver. Prioritized experience replay. arXiv preprint arXiv:1511.05952 , 2015. - · arXiv
  68. [78][78] Yannick Schroecker and Charles Isbell. Universal value density estimation for imitation learning and goal-conditioned reinforcement learning. arXiv preprint arXiv:2002.06473 , 2020. - · arXiv
  69. [79][79] Aman Sinha, Hongseok Namkoong, and John Duchi. Certifying some distributional robustness with principled adversarial training. arXiv preprint arXiv:1710.10571 , 2017. - · arXiv
  70. [81][81] Peter Spirtes, Clark N Glymour, Richard Scheines, and David Heckerman. Causation, prediction, and search . MIT press, 2000. -
  71. [83][83] Richard S Sutton and Andrew G Barto. Reinforcement learning: An introduction . MIT press, 2018. -
  72. [84][84] Philip Thomas and Emma Brunskill. Data-efficient off-policy policy evaluation for reinforcement learning. In International Conference on Machine Learning , pages 2139-2148, 2016. -
  73. [85][85] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing systems , pages 5998-6008, 2017. -
  74. [87][87] Tingwu Wang, Xuchan Bao, Ignasi Clavera, Jerrick Hoang, Yeming Wen, Eric Langlois, Shunshi Zhang, Guodong Zhang, Pieter Abbeel, and Jimmy Ba. Benchmarking model-based reinforcement learning. arXiv preprint arXiv:1907.02057 , 2019. - · arXiv
  75. [88][88] Christopher JCH Watkins and Peter Dayan. Q-learning. Machine learning , 8(3-4):279-292, 1992. -
  76. [89][89] Nicholas Watters, Loic Matthey, Matko Bosnjak, Christopher P Burgess, and Alexander Lerchner. Cobra: Data-efficient model-based rl through unsupervised object discovery and curiosity-driven exploration. arXiv preprint arXiv:1905.09275 , 2019. - · arXiv
  77. [90][90] Théophane Weber, Nicolas Heess, Lars Buesing, and David Silver. Credit assignment techniques in stochastic computation graphs. arXiv preprint arXiv:1901.01761 , 2019. - · arXiv
  78. [91][91] Cathy Wu, Aravind Rajeswaran, Yan Duan, Vikash Kumar, Alexandre M Bayen, Sham Kakade, Igor Mordatch, and Pieter Abbeel. Variance reduction for policy gradient with action-dependent factorized baselines. arXiv preprint arXiv:1803.07246 , 2018. - · arXiv
  79. [92][92] Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabas Poczos, Russ R Salakhutdinov, and Alexander J Smola. Deep sets. In Advances in neural information processing systems , pages 3391-3401, 2017. -
  80. [93][93] Vinicius Zambaldi, David Raposo, Adam Santoro, Victor Bapst, Yujia Li, Igor Babuschkin, Karl Tuyls, David Reichert, Timothy Lillicrap, Edward Lockhart, et al. Relational deep reinforcement learning. arXiv preprint arXiv:1806.01830 , 2018. - · arXiv
  81. [94][94] Guangxiang Zhu, Zhiao Huang, and Chongjie Zhang. Object-oriented dynamics predictor. In Advances in Neural Information Processing Systems , pages 9804-9815, 2018.

Cited by

Sorted by citations

Authors
Pitis, S., Creager, E., & Garg, A.
Affiliations
University of Toronto (CA)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2020
Volume
33
Pages
3976-3990
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2007.02863
arXiv
2007.02863
OpenAlex
W3039671729
Cited by
26
References
8
Citations by year
2025: 2 · 2024: 11 · 2023: 4 · 2022: 7 · 2021: 2
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • Government of Canada
  • Natural Sciences and Engineering Research Council of Canada
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
  • Physical Sciences › Computer Science › Artificial Intelligence › Bayesian Modeling and Causal Inference
  • Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
Keywords
counterfactual data augmentation, causal discovery, off-policy learning, sample efficiency, experience replay, goal-conditioned reinforcement learning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review