Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Sign Up
PreprintCited by 1

The DOI matches its DataCite record.

Learning rigid dynamics with face interaction graph networks

Allen, Kelsey R., Rubanova, Yulia, Lopez-Guevara, Tatiana, Whitney, William, Sanchez-Gonzalez, Alvaro, Battaglia, Peter, Pfaff, Tobias·2022·arXiv
DOI 10.48550/arxiv.2212.03574arXiv 2212.03574
Open this paper in CoChat — free account

Abstract

Simulating rigid collisions among arbitrary shapes is notoriously difficult due to complex geometry and the strong non-linearity of the interactions. While graph neural network (GNN)-based models are effective at learning to simulate complex physical dynamics, such as fluids, cloth and articulated bodies, they have been less effective and efficient on rigid-body physics, except with very simple shapes. Existing methods that model collisions through the meshes' nodes are often inaccurate because they struggle when collisions occur on faces far from nodes. Alternative approaches that represent the geometry densely with many particles are prohibitively expensive for complex shapes. Here we introduce the Face Interaction Graph Network (FIGNet) which extends beyond GNN-based methods, and computes interactions between mesh faces, rather than nodes. Compared to learned node- and particle-based methods, FIGNet is around 4x more accurate in simulating complex shape interactions, while also 8x more computationally efficient on sparse, rigid meshes. Moreover, FIGNet can learn frictional dynamics directly from real-world data, and can be more accurate than analytical solvers given modest amounts of training data. FIGNet represents a key step forward in one of the few remaining physical domains which have seen little competition from learned simulators, and offers allied fields such as robotics, graphics and mechanical design a new tool for simulation and model-based planning.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2212.03574

Cites

Statistics

Figures

Image 1
CC-BY · From the PDF
Figure 1 FIGNet model with face-to-face edges. (a) Example meshes from the Kubric MOVi-B dataset (Greff et al., 2022). (b) FIGNet exploits three types of interactions: face-face collisions between objects, mesh-mesh interactions within an object, and interactions with the virtual perobject node. (c) Rollout from FIGNet trained on Kubric MOVi-B (Greff et al., 2022).
Figure 1 FIGNet model with face-to-face edges. (a) Example meshes from the Kubric MOVi-B dataset (Greff et al., 2022). (b) FIGNet exploits three types of interactions: face-face collisions between objects, mesh-mesh interactions within an object, and interactions with the virtual perobject node. (c) Rollout from FIGNet trained on Kubric MOVi-B (Greff et al., 2022).CC-BY · From the PDF
Figure 3 Schematic of how baselines treat interacting objects. If one object's vertex intersects another object's face, a small collision radius d c will not detect a collision (Naive MeshGraphNet). A dense particle representation fixes the problem but adds more edges and more nodes. Using a larger collision radius introduces quadratically more edges. By operating on faces, FIGNet can use a small collision radius without missing collisions.
Figure 3 Schematic of how baselines treat interacting objects. If one object's vertex intersects another object's face, a small collision radius d c will not detect a collision (Naive MeshGraphNet). A dense particle representation fixes the problem but adds more edges and more nodes. Using a larger collision radius introduces quadratically more edges. By operating on faces, FIGNet can use a small collision radius without missing collisions.CC-BY · From the PDF
Figure 4 Rollout examples on Movi-A dataset. The node-based MeshGraphNet model struggles to resolve the collision between the sides of the sparse cube meshes.
Figure 4 Rollout examples on Movi-A dataset. The node-based MeshGraphNet model struggles to resolve the collision between the sides of the sparse cube meshes.CC-BY · From the PDF
Figure 5 Performance for FIGNet and several baseline methods: root mean squared-error on translation and rotation after 50 timesteps, and number of collision edges for (a) MOVi-A and (b) MOVi-B datasets. FIGNet has significantly better translation and rotation error, while also creating fewer collision edges overall. Models with ∗ use an implicit floor representation, while models with + use a subdivided floor.
Figure 5 Performance for FIGNet and several baseline methods: root mean squared-error on translation and rotation after 50 timesteps, and number of collision edges for (a) MOVi-A and (b) MOVi-B datasets. FIGNet has significantly better translation and rotation error, while also creating fewer collision edges overall. Models with ∗ use an implicit floor representation, while models with + use a subdivided floor.CC-BY · From the PDF
Image 6
CC-BY · From the PDF
Figure 6 Rollout example on Kubric MoviC dataset (Greff et al., 2022) from a FIGNet model trained on the MOVi-B dataset. FIGNet is able to generalize to the complex geometries of the MoviC objects which it has never seen during training.
Figure 6 Rollout example on Kubric MoviC dataset (Greff et al., 2022) from a FIGNet model trained on the MOVi-B dataset. FIGNet is able to generalize to the complex geometries of the MoviC objects which it has never seen during training.CC-BY · From the PDF
Figure 7 MIT Pushing dataset results. (a) A demonstration of the setup from Yu et al. (2016). (b) Example rollouts from FIGNet (blue) vs. ground truth (gray). The dashed outline indicates the initial object position. (c) Performance on MIT Pushing dataset for FIGNet as compared to analytic simulators MuJoCo (Todorov et al., 2012), PyBullet (Coumans, 2015), and the planar pushing analytic model developed by (Lynch, 1992) and implemented for this dataset by Kloss et al. (2022).
Figure 7 MIT Pushing dataset results. (a) A demonstration of the setup from Yu et al. (2016). (b) Example rollouts from FIGNet (blue) vs. ground truth (gray). The dashed outline indicates the initial object position. (c) Performance on MIT Pushing dataset for FIGNet as compared to analytic simulators MuJoCo (Todorov et al., 2012), PyBullet (Coumans, 2015), and the planar pushing analytic model developed by (Lynch, 1992) and implemented for this dataset by Kloss et al. (2022).CC-BY · From the PDF
Figure B.1: Objects used in the MIT pushing dataset. Figure credit to Yu et al. (2016).
Figure B.1: Objects used in the MIT pushing dataset. Figure credit to Yu et al. (2016).CC-BY · From the PDF
Figure E.1: Model ablations on MOVi-A dataset. Replacing the face-face collisions with the nodenode collisions makes the performance deteriorate the most. Models with ∗ use an implicit floor representation, while models with + use a subdivided floor.
Figure E.1: Model ablations on MOVi-A dataset. Replacing the face-face collisions with the nodenode collisions makes the performance deteriorate the most. Models with ∗ use an implicit floor representation, while models with + use a subdivided floor.CC-BY · From the PDF
Figure E.2: Model ablations on MOVi-B dataset. Removing the object node and replacing face-face collisions with node-node drastically affects the performance, making it worse. Models with ∗ use an implicit floor representation, while models with + use a subdivided floor.
Figure E.2: Model ablations on MOVi-B dataset. Removing the object node and replacing face-face collisions with node-node drastically affects the performance, making it worse. Models with ∗ use an implicit floor representation, while models with + use a subdivided floor.CC-BY · From the PDF
Figure E.3: Model ablations towards DPI (Li et al., 2019b) on MOVi-A dataset. While the loss formulation has little effect, replacing the mesh with densely sampled particles creates huge issues with memory requirements, and makes overall translation and rotation error worse. Models with ∗ use an implicit floor representation.
Figure E.3: Model ablations towards DPI (Li et al., 2019b) on MOVi-A dataset. While the loss formulation has little effect, replacing the mesh with densely sampled particles creates huge issues with memory requirements, and makes overall translation and rotation error worse. Models with ∗ use an implicit floor representation.CC-BY · From the PDF
Figure E.4: Model ablations towards DPI on MOVi-B dataset. While the loss formulation has little effect, replacing the mesh with densely sampled particles creates huge issues with memory requirements, and makes overall translation and rotation error worse. Models with ∗ use an implicit floor representation.
Figure E.4: Model ablations towards DPI on MOVi-B dataset. While the loss formulation has little effect, replacing the mesh with densely sampled particles creates huge issues with memory requirements, and makes overall translation and rotation error worse. Models with ∗ use an implicit floor representation.CC-BY · From the PDF
Figure F.2: Demonstration of generalization rollouts from Kubric MOVi-B to a manually designed u-slide and imported asset ( hippo.obj asset imported from github.com/mmacklin/ tinsel.git ). Grey and blue are static and dynamic objects respectively.
Figure F.2: Demonstration of generalization rollouts from Kubric MOVi-B to a manually designed u-slide and imported asset ( hippo.obj asset imported from github.com/mmacklin/ tinsel.git ). Grey and blue are static and dynamic objects respectively.CC-BY · From the PDF
Image 15
CC-BY · From the PDF
Figure G.2: Rollout examples on MOVi-A dataset
Figure G.2: Rollout examples on MOVi-A datasetCC-BY · From the PDF
Figure G.3: Rollout examples on MOVi-B dataset
Figure G.3: Rollout examples on MOVi-B datasetCC-BY · From the PDF
Figure G.4: Generalization from MOVi-A to MOVi-B dataset
Figure G.4: Generalization from MOVi-A to MOVi-B datasetCC-BY · From the PDF
Figure G.5: Generalization from MOVi-B to Movi-C dataset
Figure G.5: Generalization from MOVi-B to Movi-C datasetCC-BY · From the PDF
Figure J.1: Ablation over collision radii on MOVi-A dataset for MGN, compared to FIGNet with collision radius 0.1. If the nodes (in MGN) or faces (in FIGNet) between different objects lie within the collision radius, we connect them with an edge in the graph.
Figure J.1: Ablation over collision radii on MOVi-A dataset for MGN, compared to FIGNet with collision radius 0.1. If the nodes (in MGN) or faces (in FIGNet) between different objects lie within the collision radius, we connect them with an edge in the graph.CC-BY · From the PDF

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

No references could be resolved yet.

Research works printed but not linked: 42

  1. Brian Acosta, William Yang, and Michael Posa. Validating robotics simulators on real-world im- pacts. IEEE Robotics and Automation Letters, 2022.
  2. A. Ajay, J. Wu, N. Fazeli, M. Bauza, L. P. Kaelbling, J. B. Tenenbaum, and A. Rodriguez. Aug- menting physical simulators with stochastic neural networks: Case study of planar pushing and bouncing. In International Conference on Intelligent Robots and Systems (IROS), 2018.
  3. Kelsey R. Allen, Kevin A. Smith, and Joshua B. Tenenbaum. Rapid trial-and-error learning with simulation supports flexible tool use and physical reasoning. Proceedings of the Na- tional Academy of Sciences, 117(47):29302-29310, 2020. doi: 10.1073/pnas.1912341117. URL https://www.pnas.org/doi/abs/10.1073/pnas.1912341117. · DOI
  4. Kelsey R Allen, Tatiana Lopez Guevara, Yulia Rubanova, Kim Stachenfeld, Alvaro Sanchez- Gonzalez, Peter Battaglia, and Tobias Pfaff. Graph network simulators can learn discontin- uous, rigid contact dynamics. In 6th Annual Conference on Robot Learning, 2022. URL https://openreview.net/forum?id=rbIzq-I84i_.
  5. M Anitescu and F A Potra. Formulating dynamic multi-rigid-body contact problems with friction as solvable linear complementarity problems. Nonlinear Dynamics, 14(3):231-247, 1997.
  6. Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. Layer normalization. arXiv preprint arXiv:1607.06450, 2016. · arXiv
  7. Jakob Andreas Bærentzen and Henrik Aanaes. Signed distance computation using the angle weighted pseudonormal. IEEE Transactions on Visualization and Computer Graphics, 11(3): 243-253, 2005.
  8. Peter Battaglia, Razvan Pascanu, Matthew Lai, Danilo Jimenez Rezende, and koray kavukcuoglu. Interaction networks for learning about objects, relations and physics. In Advances in Neural Information Processing Systems 29, pp. 4502-4510.
  9. Peter W Battaglia, Jessica B Hamrick, Victor Bapst, Alvaro Sanchez-Gonzalez, Vinicius Zambaldi, Mateusz Malinowski, Andrea Tacchetti, David Raposo, Adam Santoro, Ryan Faulkner, et al. Relational inductive biases, deep learning, and graph networks. arXiv preprint arXiv:1806.01261, 2018. · arXiv
  10. Maria Bauza and Alberto Rodriguez. A probabilistic data-driven model for planar pushing. In IEEE International Conference on Robotics and Automation (ICRA), pp. 3008-3015, 2017.
  11. Daniel M Bear, Elias Wang, Damian Mrowca, Felix J Binder, Hsiau-Yu Fish Tung, RT Pramod, Cameron Holdaway, Sirui Tao, Kevin Smith, Fan-Yun Sun, et al. Physion: Evaluating physical prediction from vision in humans and machines. arXiv preprint arXiv:2106.08261, 2021. · arXiv
  12. James H. Clark. Hierarchical geometric models for visible surface algorithms. Commun. ACM, 19(10):547-554, oct 1976. ISSN 0001-0782. doi: 10.1145/360349.360354. URL https: //doi.org/10.1145/360349.360354. · DOI
  13. Erwin Coumans. Bullet physics simulation. In ACM SIGGRAPH 2015 Courses, pp. 7, 2015.
  14. Laura Downs, Anthony Francis, Nate Koenig, Brandon Kinman, Ryan Hickman, Krista Reymann, Thomas B McHugh, and Vincent Vanhoucke. Google scanned objects: A high-quality dataset of 3d scanned household items. arXiv preprint arXiv:2204.11918, 2022. · arXiv
  15. Nima Fazeli, Elliott Donlon, Evan Drumwright, and Alberto Rodriguez. Empirical evaluation of common contact models for planar impact. In 2017 IEEE international conference on robotics and automation (ICRA), pp. 3418-3425. IEEE, 2017.
  16. Florian Golemo, Adrien Ali Taiga, Aaron Courville, and Pierre-Yves Oudeyer. Sim-to-real transfer with neural-augmented robot simulation. In Conference on Robot Learning, pp. 817-828. PMLR, 2018.
  17. Daniel Golovin, Benjamin Solnik, Subhodeep Moitra, Greg Kochanski, John Karro, and David Scul- ley. Google vizier: A service for black-box optimization. In Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1487-1495, 2017.
  18. Klaus Greff, Francois Belletti, Lucas Beyer, Carl Doersch, Yilun Du, Daniel Duckworth, David J. Fleet, Dan Gnanapragasam, Florian Golemo, Charles Herrmann, Thomas Kipf, Abhijit Kundu, Dmitry Lagun, Issam Laradji, Hsueh-Ti (Derek) Liu, Henning Meyer, Yishu Miao, Derek Nowrouzezahrai, Cengiz Oztireli, Etienne Pot, Noha Radwan, Daniel Rebain, Sara Sabour, Mehdi S. M. Sajjadi, Matan Sela, Vincent Sitzmann, Austin Stone, Deqing Sun, Suhani Vora, Ziyu Wang, Tianhao Wu, Kwang Moo Yi, Fangcheng Zhong, and Andrea Tagliasacchi. Kubric: A scalable dataset generator. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3749-3761, June 2022.
  19. Eric Heiden, David Millard, Erwin Coumans, Yizhou Sheng, and Gaurav S Sukhatme. Neuralsim: Augmenting differentiable simulators with neural networks. In 2021 IEEE International Confer- ence on Robotics and Automation (ICRA), pp. 9474-9481. IEEE, 2021.
  20. Yifeng Jiang, Jiazheng Sun, and C Karen Liu. Data-augmented contact model for rigid body simu- lation. In Learning for Dynamics and Control Conference, pp. 378-390. PMLR, 2022.
  21. Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014. · arXiv
  22. Alina Kloss, Stefan Schaal, and Jeannette Bohg. Combining learned and analytical models for predicting action effects. CoRR, abs/1710.04102, 2017. · arXiv
  23. Alina Kloss, Stefan Schaal, and Jeannette Bohg. Combining learned and analytical models for predicting action effects from sensory data. The International Journal of Robotics Research, 41 (8):778-797, 2022.
  24. Lei Lan, Danny M Kaufman, Minchen Li, Chenfanfu Jiang, and Yin Yang. Affine body dynamics: Fast, stable & intersection-free simulation of stiff materials. ACM Trans. Graph, 2022.
  25. Yunzhu Li, Jiajun Wu, Russ Tedrake, Joshua B. Tenenbaum, and Antonio Torralba. Learning par- ticle dynamics for manipulating rigid bodies, deformable objects, and fluids. In International Conference on Learning Representations, 2019a.
  26. Yunzhu Li, Jiajun Wu, Jun-Yan Zhu, Joshua B Tenenbaum, Antonio Torralba, and Russ Tedrake. Propagation networks for model-based control under partial observation. In 2019 International Conference on Robotics and Automation (ICRA), pp. 1205-1211. IEEE, 2019b.
  27. Kevin M Lynch. The mechanics of fine manipulation by pushing. In ICRA, pp. 2269-2276.
  28. Damian Mrowca, Chengxu Zhuang, Elias Wang, Nick Haber, Li F Fei-Fei, Josh Tenenbaum, and Daniel L Yamins. Flexible neural representation for physics prediction. Advances in neural information processing systems, 31, 2018.
  29. Matthias Müller, Bruno Heidelberger, Matthias Teschner, and Markus Gross. Meshless deformations based on shape matching. ACM transactions on graphics (TOG), 24(3):471-478, 2005.
  30. Mihir Parmar, Mathew Halm, and Michael Posa. Fundamental challenges in deep learning for stiff contact dynamics. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 5181-5188. IEEE, 2021.
  31. Deepak Pathak, Christopher Lu, Trevor Darrell, Phillip Isola, and Alexei A Efros. Learning to control self-assembling morphologies: A study of generalization via modularity. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 32.
  32. Tobias Pfaff, Meire Fortunato, Alvaro Sanchez-Gonzalez, and Peter Battaglia. Learning mesh-based simulation with graph networks. In International Conference on Learning Representations, 2021.
  33. Samuel Pfrommer, Mathew Halm, and Michael Posa. Contactnets: Learning discontinuous contact dynamics with smooth, implicit representations. Conference on Robot Learning (CoRL), 2020.
  34. Yulia Rubanova, Alvaro Sanchez-Gonzalez, Tobias Pfaff, and Peter W. Battaglia. Constraint-based graph network simulator. CoRR, abs/2112.09161, 2021. · arXiv
  35. Alvaro Sanchez-Gonzalez, Nicolas Heess, Jost Tobias Springenberg, Josh Merel, Martin Riedmiller, Raia Hadsell, and Peter Battaglia. Graph networks as learnable physics engines for inference and control. In International Conference on Machine Learning, pp. 4470-4479. PMLR, 2018.
  36. Alvaro Sanchez-Gonzalez, Jonathan Godwin, Tobias Pfaff, Rex Ying, Jure Leskovec, and Peter Battaglia. Learning to simulate complex physics with graph networks. In International Confer- ence on Machine Learning, pp. 8459-8468. PMLR, 2020.
  37. D Stewart and JC J.C. Trinkle. An implicit time-stepping scheme for rigid body dynamics with Coulomb friction. International Journal for Numerical Methods in Engineering, 39(15):2673- 2691, 1996.
  38. Emanuel Todorov, Tom Erez, and Yuval Tassa. Mujoco: A physics engine for model-based control. In 2012 IEEE/RSJ international conference on intelligent robots and systems, pp. 5026-5033. IEEE, 2012.
  39. Tingwu Wang, Renjie Liao, Jimmy Ba, and Sanja Fidler. Nervenet: Learning structured policy with graph neural networks. In International Conference on Learning Representations, 2018.
  40. Pierre-Brice Wieber, Russ Tedrake, and Scott Kuindersma. Modeling and control of legged robots. In Springer handbook of robotics, pp. 1203-1234. Springer, 2016.
  41. Kuan-Ting Yu, Maria Bauza, Nima Fazeli, and Alberto Rodriguez. More than a million ways to be pushed. a high-fidelity experimental dataset of planar pushing. In 2016 IEEE/RSJ international conference on intelligent robots and systems (IROS), pp. 30-37. IEEE, 2016.
  42. Andy Zeng, Shuran Song, Johnny Lee, Alberto Rodriguez, and Thomas Funkhouser. Tossingbot: Learning to throw arbitrary objects with residual physics. IEEE Transactions on Robotics, 36(4): 1307-1319, 2020.

Cited by

See all 1

Metadata

Authors
Allen, K. R., Rubanova, Y., Lopez-Guevara, T., Whitney, W., Sanchez-Gonzalez, A., Battaglia, P., & Pfaff, T.
Venue
arXiv
Publisher
arXiv
Published
2022
DOI
10.48550/arxiv.2212.03574
arXiv
2212.03574
References
0
Type
Preprint
Licence
creativecommons.org/licenses/by/4.0/legalcode

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Learning rigid dynamics with face interaction graph networks. https://app.cochat.ai/papers/10.48550/arxiv.2212.03574
The paper
Allen, K. R., Rubanova, Y., Lopez-Guevara, T., Whitney, W., Sanchez-Gonzalez, A., Battaglia, P., & Pfaff, T. (2022). Learning rigid dynamics with face interaction graph networks. arXiv. https://doi.org/10.48550/arxiv.2212.03574

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.