CoChat
Sign Up
PreprintCited by 759Reinforcement Learning in RoboticsRobot Manipulation and LearningMultimodal Machine Learning Applications

Unsupervised Learning for Physical Interaction through Video Prediction

Open access
Finn, Chelsea, Goodfellow, Ian, Levine, Sergey·2016·arXiv
DOI 10.48550/arxiv.1605.07157arXiv 1605.07157
Open this paper in CoChat — free accountOpen PDF

Abstract

A core challenge for an agent learning to interact with the world is to predict how its actions affect objects in its environment. Many existing methods for learning the dynamics of physical interactions require labeled object information. However, to scale real-world interaction learning to a variety of scenes and objects, acquiring labeled data becomes increasingly impractical. To learn about physical object motion without labels, we develop an action-conditioned video prediction model that explicitly models pixel motion, by predicting a distribution over pixel motion from previous frames. Because our model explicitly predicts motion, it is partially invariant to object appearance, enabling it to generalize to previously unseen objects. To explore video prediction for real-world interactive agents, we also introduce a dataset of 59,000 robot interactions involving pushing motions, including a test set with novel objects. In this dataset, accurate prediction of videos conditioned on the robot's future actions amounts to learning a "visual imagination" of different futures based on different courses of action. Our experiments show that our proposed method produces more accurate video predictions both quantitatively and qualitatively, when compared to prior methods.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.1605.07157

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Video prediction is a much-studied task, including predicting how visual input will develop conditional on the system's outputs, but this is not sufficient for the kind of use in perception that includes making sense of exogenous changes not predictable from outputs.

    “For example, video prediction is a much-studied task (Oprea et al. 2020), and this includes predicting how visual input will develop conditional on the system's outputs (Finn et al. 2016).But this is not sufficient for the kind of use in perception that we have described, which also includes making sense of exogenous changes which are not predictable from outputs.”

    From Consciousness in Artificial Intelligence: Insights from the Science of Consciousness (2023) · cites this paper as Finn et al. 2016

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 27

  1. [2][2] P. W. Battaglia, J. B. Hamrick, and J. B. Tenenbaum. Simulation as an engine of physical scene understanding. Proceedings of the National Academy of Sciences , 110(45), 2013. -
  2. [3][3] S. Bengio, O. Vinyals, N. Jaitly, and N. Shazeer. Scheduled sampling for sequence prediction with recurrent neural networks. In Advances in Neural Information Processing Systems , pages 1171-1179, 2015. -
  3. [4][4] B. Boots, A. Byravan, and D. Fox. Learning predictive models of a depth camera & manipulator from raw execution traces. In International Conference on Robotics and Automation (ICRA) , 2014. -
  4. [5][5] M. A. Brubaker, L. Sigal, and D. J. Fleet. Estimating contact dynamics. In International Conference on Computer Vision (ICCV) , 2009. -
  5. [6][6] B. De Brabandere, X. Jia, T. Tuytelaars, and L. Van Gool. Dynamic filter networks. In Neural Information Processing Systems (NIPS) . 2016. -
  6. [7][7] S. Eslami, N. Heess, T. Weber, Y. Tassa, K. Kavukcuoglu, and G. E. Hinton. Attend, infer, repeat: Fast scene understanding with generative models. Neural Information Processing Systems (NIPS) , 2016. -
  7. [9][9] D.-A. Huang and K. M. Kitani. Action-reaction: Forecasting the dynamics of human interaction. In European Conference on Computer Vision (ECCV) . Springer, 2014. -
  8. [11][11] M. Jaderberg, K. Simonyan, A. Zisserman, et al. Spatial transformer networks. In Neural Information Processing Systems (NIPS) , 2015. -
  9. [12][12] A. Karpathy, G. Toderici, S. Shetty, T. Leung, R. Sukthankar, and L. Fei-Fei. Large-scale video classification with convolutional neural networks. In Compuer Vision and Pattern Recognition (CVPR) , 2014. -
  10. [13][13] D. Kinga and J. Ba. Adam: A method for stochastic optimization. International Conference on Learning Representations (ICLR) , 2015. -
  11. [14][14] S. Lange, M. Riedmiller, and A. Voigtlander. Autonomous reinforcement learning on raw visual input data in a real world application. In International Joint Conference on Neural Networks (IJCNN) , 2012. -
  12. [15][15] A. Lerer, S. Gross, and R. Fergus. Learning physical intuition of block towers by example. International Conference on Machine Learning (ICML) , 2016. -
  13. [16][16] W. Lotter, G. Kreiman, and D. Cox. Deep predictive coding networks for video prediction and unsupervised learning. arXiv preprint arXiv:1605.08104 , 2016. - · arXiv
  14. [17][17] M. Mathieu, C. Couprie, and Y. LeCun. Deep multi-scale video prediction beyond mean square error. International Conference on Learning Representations (ICLR) , 2016. -
  15. [18][18] R. Mottaghi, H. Bagherinezhad, M. Rastegari, and A. Farhadi. Newtonian image understanding: Unfolding the dynamics of objects in static images. Computer Vision and Pattern Recognition (CVPR) , 2015. -
  16. [19][19] R. Mottaghi, M. Rastegari, A. Gupta, and A. Farhadi. "What happens if..." learning to predict the effect of forces in images. European Conference on Computer Vision (ECCV) , 2016. -
  17. [20][20] J. Oh, X. Guo, H. Lee, R. L. Lewis, and S. Singh. Action-conditional video prediction using deep networks in atari games. In Neural Information Processing Systems (NIPS) , 2015. -
  18. [21][21] M. Ranzato, A. Szlam, J. Bruna, M. Mathieu, R. Collobert, and S. Chopra. Video (language) modeling: a baseline for generative models of natural videos. arXiv preprint arXiv:1412.6604 , 2014. - · arXiv
  19. [22][22] N. Srivastava, E. Mansimov, and R. Salakhutdinov. Unsupervised learning of video representations using lstms. International Conference on Machine Learning (ICML) , 2015. -
  20. [23][23] C. Vondrick, H. Pirsiavash, and A. Torralba. Anticipating the future by watching unlabeled video. CoRR , abs/1504.08023, 2015. - · arXiv
  21. [24][24] C. Vondrick, H. Pirsiavash, and A. Torralba. Generating videos with scene dynamics. In Neural Information Processing Systems (NIPS) . 2016. -
  22. [25][25] J. Walker, C. Doersch, A. Gupta, and M. Hebert. An uncertain future: Forecasting from static images using variational autoencoders. In European Conference on Computer Vision (ECCV) , 2016. -
  23. [26][26] J. Walker, A. Gupta, and M. Hebert. Patch to the future: Unsupervised visual prediction. In Computer Vision and Pattern Recognition (CVPR) , 2014. -
  24. [27][27] M. Watter, J. Springenberg, J. Boedecker, and M. Riedmiller. Embed to control: A locally linear latent dynamics model for control from raw images. In Neural Information Processing Systems (NIPS) , 2015. -
  25. [28][28] S. Xingjian, Z. Chen, H. Wang, D. Yeung, W. Wong, and W. Woo. Convolutional LSTM network: A machine learning approach for precipitation nowcasting. In Neural Information Processing Systems , 2015. -
  26. [29][29] T. Xue, J. Wu, K. L. Bouman, and W. T. Freeman. Visual dynamics: Probabilistic future frame synthesis via cross convolutional networks. In Neural Information Processing Systems (NIPS) . 2016. -
  27. [30][30] J. Yuen and A. Torralba. A data-driven approach for event prediction. In European Conference on Computer Vision (ECCV) , 2010.

Cited by

Sorted by citations

Authors
Finn, C., Goodfellow, I., & Levine, S.
ORCID
Venue
arXiv
Publisher
arXiv
Published
2016
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.1605.07157
arXiv
1605.07157
OpenAlex
W2400532028
Cited by
759
References
2
Citations by year
2025: 16 · 2024: 31 · 2023: 46 · 2022: 56 · 2021: 134 · 2020: 159 · 2019: 145 · 2018: 100 · 2017: 63 · 2016: 9
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
  • Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Keywords
physical interaction, video prediction, unsupervised learning, object dynamics, unseen objects, generalization, visual imagination

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review