CoChat

Causally Correct Partial Models for Reinforcement Learning

Open access
Danilo Jimenez Rezende, Ivo Danihelka, George Papamakarios, Nan Rosemary Ke, Ray Jiang, Théophane Weber, Karol Gregor, Hamza Merzić, Fabio Viola, Jane Wang, Jovana Mitrovic, Frederic Besse, Ioannis Antonoglou, Lars Buesing · 2020 · arXiv (Cornell University)
PreprintCited by 15

The DOI matches its OpenAlex record; 10 of the 10 references it cites were found in a registry or an index.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

OpenAlex · 10.48550/arxiv.2002.02836

Cites

10 of 10 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

  • Long Short-Term Memory

    Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318

  • A Survey of Monte Carlo Tree Search Methods

    Browne, C. B., Powley, E., Whitehouse, D., Lucas, S. M., Cowling, P. I., Rohlfshagen, P., Tavener, S., Perez, D., Samothrakis, S., & Colton, S. (2012) · IEEE Transactions on Computational Intelligence and AI in Games · doi:10.1109/tciaig.2012.2186810 · Cited by 2,970 · cited 1×

    “Additionally, we enhance the MCTS with chance nodes (Browne et al., 2012).”
    Method
  • Curiosity-Driven Exploration by Self-Supervised Prediction

    Pathak, D., Agrawal, P., Efros, A. A., & Darrell, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.70 · Cited by 1,815 · cited 1×

    “…generation of synthetic training data (Weber et al., 2017), better credit assignment (Heess et al., 2015), learning useful internal representations and belief states (Gregor et al., 2019; Guo et al., 2018), or exploration via quantification of uncertainty or information gain (Pathak et al., 2017).”
    Background
  • IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures

    Espeholt, L., Soyer, H., Munos, R., Simonyan, K., Mnih, V., Ward, T., Doron, Y., Firoiu, V., Harley, T., Dunning, I., et al. (2018) · arXiv · doi:10.48550/arxiv.1802.01561 · Cited by 1,708

  • Formal theory of creativity, fun, and intrinsic motivation (1990–2010)

    Schmidhuber, J. (2010) · IEEE Transactions on Autonomous Mental Development · doi:10.1109/tamd.2010.2056368 · Cited by 747 · cited 1×

    “Within MBRL, commonly explored methods include action-conditional, next-step models (Oh et al., 2015; Ha & Schmidhuber, 2018; Chiappa et al., 2017; Schmidhuber, 2010; Xie et al., 2016; Deisenroth & Rasmussen, 2011; Lin & Mitchell, 1992; Li et al., 2015; Diuk et al., 2008; Igl et al., 2018; Ebert et…”
    Method
  • Learning Latent Dynamics for Planning from Pixels

    Hafner, D., Lillicrap, T., Fischer, I., Villegas, R., Ha, D., Lee, H., & Davidson, J. (2018) · arXiv · doi:10.48550/arxiv.1811.04551 · Cited by 708

  • Deep visual foresight for planning robot motion

    Finn, C., & Levine, S. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989324 · Cited by 599 · cited 1×

    “…fact that conditioning on high-dimensional data such as images would require modeling and generating images in order to plan over several timesteps (Finn & Levine, 2017), and to the fact that modeling images is challenging and may un-necessarily focus on visual details which are not relevant for…”
    Background
  • Model-Based Reinforcement Learning for Atari

    Kaiser, L., Babaeizadeh, M., Milos, P., Osinski, B., Campbell, R. H., Czechowski, K., Erhan, D., Finn, C., Kozakowski, P., Levine, S., et al. (2019) · arXiv · doi:10.48550/arxiv.1903.00374 · Cited by 467 · cited 1×

    “…next-step models (Oh et al., 2015; Ha & Schmidhuber, 2018; Chiappa et al., 2017; Schmidhuber, 2010; Xie et al., 2016; Deisenroth & Rasmussen, 2011; Lin & Mitchell, 1992; Li et al., 2015; Diuk et al., 2008; Igl et al., 2018; Ebert et al., 2018; Kaiser et al., 2019; Janner et al., 2019).”
  • When to Trust Your Model: Model-Based Policy Optimization

    Janner, M., Fu, J., Zhang, M., & Levine, S. (2019) · arXiv · doi:10.48550/arxiv.1906.08253 · Cited by 351 · cited 1×

    “…next-step models (Oh et al., 2015; Ha & Schmidhuber, 2018; Chiappa et al., 2017; Schmidhuber, 2010; Xie et al., 2016; Deisenroth & Rasmussen, 2011; Lin & Mitchell, 1992; Li et al., 2015; Diuk et al., 2008; Igl et al., 2018; Ebert et al., 2018; Kaiser et al., 2019; Janner et al., 2019).”
  • World Models

    Ha, D., & Schmidhuber, J. (2018) · Zenodo · doi:10.5281/zenodo.1207631 · Cited by 187 · cited 1×

    “Within MBRL, commonly explored methods include action-conditional, next-step models (Oh et al., 2015; Ha & Schmidhuber, 2018; Chiappa et al., 2017; Schmidhuber, 2010; Xie et al., 2016; Deisenroth & Rasmussen, 2011; Lin & Mitchell, 1992; Li et al., 2015; Diuk et al., 2008; Igl et al., 2018; Ebert et…”
    Method

Cited by

See all 15

Metadata

Authors
Rezende, D. J., Danihelka, I., Papamakarios, G., Ke, N. R., Jiang, R., Weber, T., Gregor, K., Merzić, H., Viola, F., Wang, J., Mitrovic, J., Besse, F., Antonoglou, I., & Buesing, L.
Venue
arXiv (Cornell University)
Publisher
Cornell University
Published
2020-02-07
ISSN
2331-8422 (ISSN-L)
OpenAlex
W2996117162
Cited by
15
References
10
Type
posted content
Language
en
Open access
green

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.