CoChat
Sign Up
PreprintCited by 969Model Reduction and Neural NetworksReinforcement Learning in RoboticsNeural Networks and Applications

Interaction Networks for Learning about Objects, Relations and Physics

Open access
Battaglia, Peter W., Pascanu, Razvan, Lai, Matthew, Rezende, Danilo, Kavukcuoglu, Koray·2016·arXiv
DOI 10.48550/arxiv.1612.00222arXiv 1612.00222
Open this paper in CoChat — free accountOpen PDF

Abstract

Reasoning about objects, relations, and physics is central to human intelligence, and a key goal of artificial intelligence. Here we introduce the interaction network, a model which can reason about how objects in complex systems interact, supporting dynamical predictions, as well as inferences about the abstract properties of the system. Our model takes graphs as input, performs object- and relation-centric reasoning in a way that is analogous to a simulation, and is implemented using deep neural networks. We evaluate its ability to reason about several challenging physical domains: n-body problems, rigid-body collision, and non-rigid dynamics. Our results show it can be trained to accurately simulate the physical trajectories of dozens of objects over thousands of time steps, estimate abstract quantities such as energy, and generalize automatically to systems with different numbers and configurations of objects and relations. Our interaction network implementation is the first general-purpose, learnable physics engine, and a powerful general framework for reasoning about object and relations in a wide variety of complex real-world domains.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.1612.00222

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 26

  1. [1][1] J Andreas, M Rohrbach, T Darrell, and D Klein. Learning to compose neural networks for question answering. NAACL , 2016. -
  2. [2][2] DBaraff. Physically based modeling: Rigid body simulation. SIGGRAPH Course Notes, ACM SIGGRAPH , 2(1), 2001. -
  3. [3][3] PW Battaglia, JB Hamrick, and JB Tenenbaum. Simulation as an engine of physical scene understanding. Proceedings of the National Academy of Sciences , 110(45):18327-18332, 2013. -
  4. [4][4] Michael B. Chang, Tomer Ullman, Antonio Torralba, and Joshua B. Tenenbaum. A compositional object-based approach to learning physical dynamics. Under Review , 2016. -
  5. [5][5] K.J.W. Craik. The nature of explanation . Cambridge University Press, 1943. -
  6. [6][6] K Fragkiadaki, P Agrawal, S Levine, and J Malik. Learning visual predictive models of physics for playing billiards. ICLR , 2016. -
  7. [7][7] Z. Ghahramani. Probabilistic machine learning and artificial intelligence. Nature , 521(7553):452-459, 2015. -
  8. [8][8] R Grzeszczuk, D Terzopoulos, and G Hinton. Neuroanimator: Fast neural network emulation and control of physics-based models. In Proceedings of the 25th annual conference on Computer graphics and interactive techniques , pages 9-20. ACM, 1998. -
  9. [10][10] M. Hegarty. Mechanical reasoning by mental simulation. TICS , 8(6):280-285, 2004. -
  10. [11][11] MJaderberg, K Simonyan, and A Zisserman. Spatial transformer networks. In NIPS , pages 2008-2016, 2015. -
  11. [12][12] P.N. Johnson-Laird. Mental models: towards a cognitive science of language, inference, and consciousness , volume 6. Cambridge University Press, 1983. -
  12. [13][13] D. Kingma and J. Ba. Adam: A method for stochastic optimization. ICLR , 2015. -
  13. [14][14] L Ladický, S Jeong, B Solenthaler, M Pollefeys, and M Gross. Data-driven fluid simulations using regression forests. ACM Transactions on Graphics (TOG) , 34(6):199, 2015. -
  14. [15][15] B Lake, T Ullman, J Tenenbaum, and S Gershman. Building machines that learn and think like people. arXiv:1604.00289 , 2016. - · arXiv
  15. [17][17] ALerer, S Gross, and R Fergus. Learning physical intuition of block towers by example. arXiv:1603.01312 , 2016. - · arXiv
  16. [18][18] WLi, S Azimi, A Leonardis, and M Fritz. To fall or not to fall: A visual approach to physical stability prediction. arXiv:1604.00066 , 2016. - · arXiv
  17. [19][19] Yujia Li, Daniel Tarlow, Marc Brockschmidt, and Richard Zemel. Gated graph sequence neural networks. arXiv preprint arXiv:1511.05493 , 2015. - · arXiv
  18. [20][20] R Mottaghi, H Bagherinezhad, M Rastegari, and A Farhadi. Newtonian image understanding: Unfolding the dynamics of objects in static images. arXiv:1511.04048 , 2015. - · arXiv
  19. [21][21] R Mottaghi, M Rastegari, A Gupta, and A Farhadi. " what happens if..." learning to predict the effect of forces in images. arXiv:1603.05600 , 2016. - · arXiv
  20. [22][22] SE Reed and N de Freitas. Neural programmer-interpreters. ICLR , 2016. -
  21. [23][23] F. Scarselli, M. Gori, A.C. Tsoi, M. Hagenbuchner, and G. Monfardini. The graph neural network model. IEEE Trans. Neural Networks , 20(1):61-80, 2009. -
  22. [25][25] R Socher, E Huang, J Pennin, C Manning, and A Ng. Dynamic pooling and unfolding recursive autoencoders for paraphrase detection. In in NIPS , pages 801-809, 2011. -
  23. [26][26] E Spelke, K Breinlinger, J Macomber, and K Jacobson. Origins of knowledge. Psychol. Rev. , 99(4):605632, 1992. -
  24. [27][27] I Sutskever and GE Hinton. Using matrices to model symbolic relationship. In D. Koller, D. Schuurmans, Y. Bengio, and L. Bottou, editors, NIPS , pages 1593-1600. 2009. -
  25. [29][29] P Winston and B Horn. The psychology of computer vision , volume 73. McGraw-Hill New York, 1975. -
  26. [30][30] J Wu, I Yildirim, JJ Lim, B Freeman, and J Tenenbaum. Galileo: Perceiving physical object properties by integrating a physics engine with deep learning. In NIPS , pages 127-135, 2015.

Cited by

Sorted by citations

Authors
Battaglia, P. W., Pascanu, R., Lai, M., Rezende, D., & Kavukcuoglu, K.
Affiliations
  • Google DeepMind (United Kingdom) (GB)
  • Google (United States) (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2016
Volume
29
Pages
4509-4517
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.1612.00222
arXiv
1612.00222
OpenAlex
W2552391307
Cited by
969
References
2
Citations by year
2026: 3 · 2025: 43 · 2024: 87 · 2023: 106 · 2022: 107 · 2021: 191 · 2020: 206 · 2019: 154 · 2018: 49 · 2017: 22 · 2016: 1
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
  • neural information processing systems
Topics
  • Physical Sciences › Physics and Astronomy › Statistical and Nonlinear Physics › Model Reduction and Neural Networks
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
  • Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
Keywords
interaction networks, graph neural networks, physical simulation, N-body problem, rigid body collision, energy estimates

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review