CoChat
Sign Up
Cited by 19Multimodal Machine Learning ApplicationsAdvanced Vision and ImagingModel Reduction and Neural Networks

The DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.

IntPhys 2019: A Benchmark for Visual Intuitive Physics Understanding

Open access
Riochet, Ronan, Castro, Mario Ynocente, Bernard, Mathieu, Lerer, Adam, Fergus, Rob, Izard, Veronique, Dupoux, Emmanuel·2022·IEEE Transactions on Pattern Analysis and Machine Intelligence
DOI 10.1109/tpami.2021.3083839
Open this paper in CoChat — free accountOpen PDF

Abstract

In order to reach human performance on complex visual tasks, artificial systems need to incorporate a significant amount of understanding of the world in terms of macroscopic objects, movements, forces, etc. Inspired by work on intuitive physics in infants, we propose an evaluation benchmark which diagnoses how much a given system understands about physics by testing whether it can tell apart well matched videos of possible versus impossible events constructed with a game engine. The test requires systems to compute a physical plausibility score over an entire video. To prevent perceptual biases, the dataset is made of pixel matched quadruplets of videos, enforcing systems to focus on high level temporal dependencies between frames rather than pixel-level details. We then describe two Deep Neural Networks systems aimed at learning intuitive physics in an unsupervised way, using only physically possible videos. The systems are trained with a future semantic mask prediction objective and tested on the possible versus impossible discrimination task. The analysis of their results compared to human data gives novel insights in the potentials and limitations of next frame prediction architectures.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/tpami.2021.3083839

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • IntPhys is the most carefully curated data source, with pairs of videos aligned at the pixel level thanks to simulator use and each frame individually stored.

    “IntPhys (Riochet et al., 2022) is the most carefully curated data source, with pairs of videos being aligned at the pixel level thanks to the use of a simulator and each frame being individually stored (avoiding compression artifacts).”

    From Intuitive physics understanding emerges from self-supervised pretraining on natural videos (2025) · cites this paper as Riochet et al., 2022

    Section 2.2 (p. 3) says possible and impossible clips are constructed in pixel-matched quadruplets and contain frames with exactly the same pixels; Section 2.6 (p. 4) says the clips were made with Unreal Engine. The copy does not establish that IntPhys is the most carefully curated data source, or say that each frame is individually stored to avoid compression artifacts. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • This suggests that core knowledge is not necessary for deep learning systems to understand intuitive physics concepts.

    “In contrast to previous work (Smith et al., 2019; Riochet et al., 2022), this suggests that core knowledge is not necessary for deep learning systems to understand intuitive physics concepts.”

    From Intuitive physics understanding emerges from self-supervised pretraining on natural videos (2025) · cites this paper as Riochet et al., 2022

    The citation identifies Riochet et al. as prior work on deep-learning models of intuitive physics, which Sections 3 and 6 support. The Discussion (p. 9) says abstract semantic-mask representations performed better than pixel-based systems, but it does not conclude that core knowledge is necessary. The claim that it is not necessary is framed as the citing authors’ own conclusion, not Riochet et al.’s. (LLM, conf=0.80)

    Experimental — accuracy not yet measured

  • Relative surprise is commonly used as it allows to precisely measure the effect of the physics breaking event.

    “Relative surprise (Piloto et al., 2022; Smith et al., 2019; Riochet et al., 2022), which looks at the difference between surprise on the impossible and possible videos, is commonly used as it allows to precisely measure the effect of the physics breaking event.”

    From Intuitive physics understanding emerges from self-supervised pretraining on natural videos (2025) · cites this paper as Riochet et al., 2022

    Section 2.5 (p. 3) describes a relative error rate that compares plausibility scores for matched possible and impossible videos; Section 2.2 (p. 3) says the matching minimizes low-level visual biases. The paper does not establish that a metric called “relative surprise” is commonly used or that it precisely measures the size of a physics-breaking event’s effect. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

  • AI systems often surpass human performance on high-level cognitive tasks like language, coding or mathematics, but paradoxically struggle in common sense physical understanding.

    “rapid progress has produced artificial intelligence (AI) systems that often surpass human performance on high-level cognitive tasks like language, coding or mathematics (OpenAI, 2024), but paradoxically struggle in common sense physical understanding (Riochet et al., 2022; Weihs et al., 2022; Jassim et al., 2024; Bisk et al., 2020; Benchekroun et al.,”

    From Intuitive physics understanding emerges from self-supervised pretraining on natural videos (2025) · cites this paper as Riochet et al., 2022

  • A V-JEPA model trained on natural videos achieves 98% zero-shot accuracy on the IntPhys benchmark and 62% zero-shot accuracy on the InfLevel benchmark.

    “a V-JEPA model trained on natural videos achieves 98% zero-shot accuracy on the IntPhys benchmark (Riochet et al., 2022) and 62% zero-shot accuracy on the InfLevel benchmark (Weihs et al., 2022).”

    From Intuitive physics understanding emerges from self-supervised pretraining on natural videos (2025) · cites this paper as Riochet et al., 2022

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Flagged first, then by citations

Research works printed but not linked: 40

  1. 10.1109/TPAMI.2008.79 · DOI
  2. 10.5555/2999134.2999257 · DOI
  3. Pinheiro, 2015, Learning to segment object candidates,, Adv. Neural Inf. Process. Syst., 1990
  4. 10.1111/j.1467-8624.2006.00975.x · DOI
  5. 10.1016/0010-0285(83)90017-8 · DOI
  6. 10.1111/j.2044-835X.1995.tb00669.x · DOI
  7. 10.1016/j.actpsy.2006.05.005 · DOI
  8. 10.1002/edp.2430010203 · DOI
  9. 10.1016/0885-2014(90)90011-H · DOI
  10. 10.1006/cogp.1996.0005 · DOI
  11. Battaglia, Interaction networks for learning about objects, relations and physics,, 4502
  12. Xue, 2016, Visual dynamics: Probabilistic future frame synthesis via cross convolutional networks,, Adv. Neural Inf. Process. Syst., 91
  13. Fraccaro, A disentangled recognition and nonlinear dynamics model for unsupervised learning,, 3604
  14. 10.1007/978-3-319-46484-8_38 · DOI
  15. 10.1109/TPAMI.2016.2516982 · DOI
  16. 10.1007/978-3-319-48881-3_56 · DOI · Paper page
  17. Finn, Unsupervised learning for physical interaction through video prediction,, 64
  18. Oh, Action-conditional video prediction using deep networks in Atari games,, 2863
  19. 10.1007/978-3-642-15561-1_2 · DOI
  20. 10.1109/CVPR.2013.387 · DOI
  21. 10.1016/0010-0277(85)90008-3 · DOI
  22. 10.1162/tacl_a_00115 · DOI
  23. 10.1016/S0010-0277(99)00035-9 · DOI
  24. 10.1002/wcs.157 · DOI
  25. Baillargeon, 2012, Core cognition and beyond,, 33
  26. 10.1163/156856888X00122 · DOI
  27. 10.1126/science.7350666 · DOI
  28. 10.3156/jsoft.29.5_177_2 · DOI
  29. Fragkiadaki, Learning visual predictive models of physics for playing billiards,
  30. 10.1007/s11263-015-0816-y · DOI · Paper page
  31. Smith, Modeling expectation violation in intuitive physics with coarse probabilistic object representations,, 8983
  32. 10.1073/pnas.1306572110 · DOI · Paper page
  33. Lerer, Learning physical intuition of block towers by example,, 430
  34. Zhang, A comparative evaluation of approximate probabilistic simulation and deep neural networks as accounts of human physical scene understanding,
  35. Mirza, Generalizable features from unsupervised learning,
  36. 10.5244/C.30.39 · DOI
  37. Battaglia, Interaction networks for learning about objects, relations and physics,, 4502
  38. 10.1007/springerreference_63882 · DOI
  39. Mottaghi, Proc. Eur. Conf. Comput. Vis, 269
  40. 10.1073/pnas.1912341117 · DOI

Cited by

Sorted by citations

Metadata

Authors
Riochet, R., Castro, M. Y., Bernard, M., Lerer, A., Fergus, R., Izard, V., & Dupoux, E.
Affiliations
  • Institut national de recherche en sciences et technologies du numérique (FR)
  • Meta (United States) (US)
  • CNRS, Université Paris Descartes, Paris, France
  • CoML, ENS/CNRS/EHESS/INRIA/PSL Research University, Paris, France
ORCID
Venue
IEEE Transactions on Pattern Analysis and Machine Intelligence
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Published
2022-09-01
Volume
44
Issue
9
Pages
5016-5025
ISSN
  • 0162-8828 (ISSN-L)
  • 2160-9292
  • 1939-3539
DOI
10.1109/tpami.2021.3083839
OpenAlex
W2971147709
Cited by
19
References
2
Citations by year
2026: 3 · 2025: 3 · 2024: 3 · 2023: 4
FWCI
0.90
Type
journal article
Language
en
Open access
Licence
Locations
  • IEEE Transactions on Pattern Analysis and Machine Intelligence (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • HAL (submittedVersion, other-oa)
Funders
  • Canadian Institute for Advanced Research
  • École des Hautes Etudes en Sciences Sociales
  • Agence Nationale de la Recherche (ANR-10-IDEX-0001, ANR-17-EURE-0017, ANR-19-P3IA-0001, ANR-18-IDEX-0001)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
  • Physical Sciences › Physics and Astronomy › Statistical and Nonlinear Physics › Model Reduction and Neural Networks
Keywords
intuitive physics, benchmark, physical plausibility, unsupervised learning, next-frame prediction, temporal dependencies, infant cognitive development
MeSH terms
Algorithms, Humans, Learning, Physics, Neural Networks, Computer, Benchmarking

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review