Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Control
Open accessThe DOI matches its OpenAlex record; 11 of the 11 references it cites were found in a registry or an index.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Playing Atari with Deep Reinforcement Learning
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013) · arXiv · doi:10.48550/arxiv.1312.5602 · Cited by 9,471
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
Finn, C., Abbeel, P., & Levine, S. (2017) · arXiv · doi:10.48550/arxiv.1703.03400 · Cited by 8,512 · cited 1×
“We build upon model-agnostic meta-learning (MAML) [44], which learns initial parameters θ for model f that can efficiently adapt to a new task with one or a few steps of gradient descent.”
MethodSelf-citationConvolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting
Shi, X., Chen, Z., Wang, H., Yeung, D.-Y., Wong, W.-K., & Woo, W.-C. (2015) · arXiv · doi:10.48550/arxiv.1506.04214 · Cited by 7,709 · cited 2×
“The model uses a convolutional LSTM [37] to predict future frames.”
BackgroundMethodLevine, S., Pastor, P., Krizhevsky, A., Ibarz, J., & Quillen, D. (2018) · The International Journal of Robotics Research · doi:10.1177/0278364917710318 · Cited by 1,971
Self-citationTemporal difference learning and TD-Gammon
Tesauro, G. (1995) · Communications of the ACM · doi:10.1145/203330.203343 · Cited by 1,618 · cited 1×
“While there has been significant progress in developing deep reinforcement learning algorithms that learn complex skills and scale to high-dimensional observation spaces, such as pixels [1], [2], [3], [4], learning behaviors that generalize to new tasks and objects remains an open problem.”
BackgroundMastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm
Silver, D., Hubert, T., Schrittwieser, J., Antonoglou, I., Lai, M., Guez, A., Lanctot, M., Sifre, L., Kumaran, D., Graepel, T., et al. (2017) · arXiv · doi:10.48550/arxiv.1712.01815 · Cited by 1,102 · cited 1×
“While there has been significant progress in developing deep reinforcement learning algorithms that learn complex skills and scale to high-dimensional observation spaces, such as pixels [1], [2], [3], [4], learning behaviors that generalize to new tasks and objects remains an open problem.”
BackgroundUnsupervised Learning for Physical Interaction through Video Prediction
Finn, C., Goodfellow, I., & Levine, S. (2016) · arXiv · doi:10.48550/arxiv.1605.07157 · Cited by 759
Self-citationLearning Synergies Between Pushing and Grasping with Self-Supervised Deep Reinforcement Learning
Zeng, A., Song, S., Welker, S., Lee, J., Rodriguez, A., & Funkhouser, T. (2018) · 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8593986 · Cited by 606
Deep visual foresight for planning robot motion
Finn, C., & Levine, S. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989324 · Cited by 599
Self-citation
Cited by
Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems
Levine, S., Kumar, A., Tucker, G., & Fu, J. (2020) · arXiv · doi:10.48550/arxiv.2005.01643 · Cited by 824
Self-citationHow to train your robot with deep reinforcement learning: Lessons we have learned
Ibarz, J., Tan, J., Finn, C., Kalakrishnan, M., Pastor, P., & Levine, S. (2021) · The International Journal of Robotics Research · doi:10.1177/0278364920987859 · Cited by 570
Self-citationModel-Based Reinforcement Learning for Atari
Kaiser, L., Babaeizadeh, M., Milos, P., Osinski, B., Campbell, R. H., Czechowski, K., Erhan, D., Finn, C., Kozakowski, P., Levine, S., et al. (2019) · arXiv · doi:10.48550/arxiv.1903.00374 · Cited by 467
“…several recent works have incorporated images into real-world (Finn et al., 2016; Finn & Levine, 2017; Babaeizadeh et al., 2017a; Ebert et al., 2017; Piergiovanni et al., 2018; Paxton et al., 2019; Rybkin et al., 2018; Ebert et al., 2018) and simulated (Watter et al., 2015; Hafner et al., 2019) .”
BackgroundSelf-citationWhen to Trust Your Model: Model-Based Policy Optimization
Janner, M., Fu, J., Zhang, M., & Levine, S. (2019) · arXiv · doi:10.48550/arxiv.1906.08253 · Cited by 351
“…2018), are appealing because they allow for algorithms that combine the sample efficiency of a model-based approach with the asymptotic performance of high-capacity function approximators, even in domains with high-dimensional observations (Oh et al., 2015; Ebert et al., 2018; Kaiser et al., 2019).”
BackgroundSelf-citationA Review on Deep Learning Techniques for Video Prediction
Oprea, S., Martinez-Gonzalez, P., Garcia-Garcia, A., Castro-Vargas, J. A., Orts-Escolano, S., Garcia-Rodriguez, J., & Argyros, A. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3045007 · Cited by 301
Review of machine learning methods in soft robotics
Kim, D., Kim, S.-H., Kim, T., Kang, B. B., Lee, M., Park, W., Ku, S., Kim, D., Kwon, J., Lee, H., et al. (2021) · PLOS ONE · doi:10.1371/journal.pone.0246102 · Cited by 256
Open X-Embodiment: Robotic Learning Datasets and RT-X Models : Open X-Embodiment Collaboration 0
O’Neill, A., Rehman, A., Maddukuri, A., Gupta, A., Padalkar, A., Lee, A., Pooley, A., Gupta, A., Mandlekar, A., Jain, A., et al. (2024) · 2024 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra57147.2024.10611477 · Cited by 210
Self-citationGrasping in the Wild: Learning 6DoF Closed-Loop Grasping From Low-Cost Demonstrations
Song, S., Zeng, A., Lee, J., & Funkhouser, T. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.3004787 · Cited by 185
“This action-view representation is inspired by prior work, which use predictive models to improve the sample efficiency of reinforcement learning algorithms [33], [34].”
MethodDROID: A Large-Scale In-The-Wild Robot Manipulation Dataset
Khazatsky, A., Pertsch, K., Nair, S., Balakrishna, A., Dasari, S., Karamcheti, S., Nasiriany, S., Srirama, M., Chen, L., Ellis, K., et al. (2024) · Robotics: Science and Systems XX · doi:10.15607/rss.2024.xx.120 · Cited by 159
Self-citationSoftGym: Benchmarking Deep Reinforcement Learning for Deformable Object Manipulation
Lin, X., Wang, Y., Olkin, J., & Held, D. (2020) · arXiv · doi:10.48550/arxiv.2011.07215 · Cited by 140
Seita, D., Florence, P., Tompson, J., Coumans, E., Sindhwani, V., Goldberg, K., & Zeng, A. (2021) · 2021 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra48506.2021.9561391 · Cited by 130
Planning to Explore via Self-Supervised World Models
Sekar, R., Rybkin, O., Daniilidis, K., Abbeel, P., Hafner, D., & Pathak, D. (2020) · arXiv · doi:10.48550/arxiv.2005.05960 · Cited by 129
“…agents can achieve performance competitive with model-free agents while at-taining much higher data efficiency, and even scale to high-dimensional observations (Chua et al., 2018; Buesing et al., 2018; Ebert et al., 2018; Ha & Schmidhuber, 2018; Hafner et al., 2018; Nagabandi et al., 2019).”
BackgroundA neural network trained for prediction mimics diverse features of biological neurons and perception
Lotter, W., Kreiman, G., & Cox, D. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-0170-9 · Cited by 128
Learning Predictive Representations for Deformable Objects Using Contrastive Estimation
Yan, W., Vangipuram, A., Abbeel, P., & Pinto, L. (2020) · arXiv · doi:10.48550/arxiv.2003.05436 · Cited by 122
“Some approaches take a direct approach to learning complex dynamics models through pixel-space [19, 8].”
BackgroundLanguage Conditioned Imitation Learning Over Unstructured Data
Lynch*, C., & Sermanet*, P. (2021) · Robotics: Science and Systems XVII · doi:10.15607/rss.2021.xvii.047 · Cited by 110
AVID: Learning Multi-Stage Tasks via Pixel-Level Translation of Human Videos
Smith, L., Dhawan, N., Zhang, M., Abbeel, P., & Levine, S. (2020) · Robotics: Science and Systems XVI · doi:10.15607/rss.2020.xvi.024 · Cited by 102
Self-citationBridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasets
Ebert, F., Yang, Y., Schmeckpeper, K., Bucher, B., Georgakis, G., Daniilidis, K., Finn, C., & Levine, S. (2022) · Robotics: Science and Systems XVIII · doi:10.15607/rss.2022.xviii.063 · Cited by 94
Self-citationBridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasets
Ebert, F., Yang, Y., Schmeckpeper, K., Bucher, B., Georgakis, G., Daniilidis, K., Finn, C., & Levine, S. (2021) · arXiv · doi:10.48550/arxiv.2109.13396 · Cited by 94
Self-citationEveritt, T., Hutter, M., Kumar, R., & Krakovna, V. (2021) · Synthese · doi:10.1007/s11229-021-03141-4 · Cited by 80
VisuoSpatial Foresight for Multi-Step, Multi-Task Fabric Manipulation
Hoque, R., Seita, D., Balakrishna, A., Ganapathi, A., Tanwani, A., Jamali, N., Yamane, K., Iba, S., & Goldberg, K. (2020) · Robotics: Science and Systems XVI · doi:10.15607/rss.2020.xvi.034 · Cited by 74
You Only Demonstrate Once: Category-Level Manipulation from Single Visual Demonstration
Wen, B., Lian, W., Bekris, K., & Schaal, S. (2022) · Robotics: Science and Systems XVIII · doi:10.15607/rss.2022.xviii.044 · Cited by 70
Learning Visible Connectivity Dynamics for Cloth Smoothing
Lin, X., Wang, Y., Huang, Z., & Held, D. (2021) · arXiv · doi:10.48550/arxiv.2105.10389 · Cited by 62
DayDreamer: World Models for Physical Robot Learning
Wu, P., Escontrela, A., Hafner, D., Goldberg, K., & Abbeel, P. (2022) · arXiv · doi:10.48550/arxiv.2206.14176 · Cited by 76
Kadi, H. A., & Terzić, K. (2023) · Sensors · doi:10.3390/s23052389 · Cited by 27
Learning Periodic Tasks from Human Demonstrations
Yang, J., Zhang, J., Settle, C., Rai, A., Antonova, R., & Bohg, J. (2022) · 2022 International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra46639.2022.9812402 · Cited by 21
Behavior Retrieval: Few-Shot Imitation Learning by Querying Unlabeled Datasets
Du, M., Nair, S., Sadigh, D., & Finn, C. (2023) · Robotics: Science and Systems XIX · doi:10.15607/rss.2023.xix.011 · Cited by 20
Self-citationDisentangling Dense Multi-Cable Knots
Viswanath, V., Grannen, J., Sundaresan, P., Thananjeyan, B., Balakrishna, A., Novoseller, E., Ichnowski, J., Laskey, M., Gonzalez, J. E., & Goldberg, K. (2021) · 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636397 · Cited by 21
C-Learning: Learning to Achieve Goals via Recursive Classification
Eysenbach, B., Salakhutdinov, R., & Levine, S. (2020) · arXiv · doi:10.48550/arxiv.2011.08909 · Cited by 18
“, 2019), model-based approaches (Nair et al., 2020; Ebert et al., 2018), Q-learning (Kaelbling, 1993; Schaul et al.”
BackgroundSelf-citationLanguage Conditioned Imitation Learning over Unstructured Data
Lynch, C., & Sermanet, P. (2020) · arXiv · doi:10.48550/arxiv.2005.07648 · Cited by 18
“One way of acquiring this kind of control is to first learn a model of the environment through interaction (Oh et al., 2015; Hafner et al., 2018; Ebert et al., 2018; Ha and Schmidhuber, 2018) then use it for planning.”
BackgroundBag All You Need: Learning a Generalizable Bagging Strategy for Heterogeneous Objects
Bahety, A., Jain, S., Ha, H., Hager, N., Burchfiel, B., Cousineau, E., Feng, S., & Song, S. (2023) · 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros55552.2023.10341841 · Cited by 15
MoDem-V2: Visuo-Motor World Models for Real-World Robot Manipulation
Lancaster, P., Hansen, N., Rajeswaran, A., & Kumar, V. (2024) · 2024 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra57147.2024.10611121 · Cited by 8
“For continuous control, planning can be formalized as Model Predictive Control (MPC) [20], [6], [7], [9], whereas Monte-Carlo Tree Search (MCTS) is used for discrete action spaces [22], [23].”
BackgroundMethodOcclusion resistant learning of intuitive physics from videos
Riochet, R., Sivic, J., Laptev, I., & Dupoux, E. (2020) · arXiv · doi:10.48550/arxiv.2005.00069 · Cited by 7
“Forward modeling in videos has been studied for action planning (Ebert et al., 2018; Finn et al., 2016) and as a scheme for unsupervised learning of visual features (Lan et al.”
MethodArnold, S., Tanaka, D., & Yamazaki, K. (2023) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2022.1045747 · Cited by 6
V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning
Assran, M., Bardes, A., Fan, D., Garrido, Q., Howes, R., Mojtaba, Komeili, Muckley, M., Rizvi, A., Roberts, C., et al. (2025) · arXiv · doi:10.48550/arxiv.2506.09985 · Cited by 8
“…2018; Hafner et al., 2019b,a; Hansen et al., 2022, 2023; Hafner et al., 2023; Schrittwieser et al., 2020; Samsami et al., 2024), as well as real-world locomotion and manipulation tasks (Lee et al., 2020; Nagabandi et al., 2020; Finn et al., 2016; Ebert et al., 2017, 2018; Yen-Chen et al., 2020).”
Block Contextual MDPs for Continual Learning
Sodhani, S., Meier, F., Pineau, J., & Zhang, A. (2021) · arXiv · doi:10.48550/arxiv.2110.06972 · Cited by 3
“Several works have focused on modeling the environment context from high-level pixel observations (Pathak et al., 2017; Ebert et al., 2018; Chen et al., 2018; Xu et al., 2019).”
Background
Metadata
- Authors
- Ebert, F., Finn, C., Dasari, S., Xie, A., Lee, A. X., & Levine, S.
- Venue
- arXiv (Cornell University)
- Publisher
- Cornell University
- Published
- 2018-12-03
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1812.00568
- OpenAlex
- W2902125520
- Cited by
- 235
- References
- 11
- Type
- posted content
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.