The DOI matches its Crossref record; 49 of the 49 references it cites were found in a registry or an index; none of the 6 sources its claims cite is checked for support yet.
Physion++: Evaluating Physical Scene Understanding that Requires Online Inference of Different Physical Properties
Open accessAbstract
General physical scene understanding requires more than simply localizing and recognizing objects -- it requires knowledge that objects can have different latent properties (e.g., mass or elasticity), and that those properties affect the outcome of physical events. While there has been great progress in physical and video prediction models in recent years, benchmarks to test their performance typically do not require an understanding that objects have individual physical properties, or at best test only those properties that are directly observable (e.g., size or color). This work proposes a novel dataset and benchmark, termed Physion++, that rigorously evaluates visual physical prediction in artificial systems under circumstances where those predictions rely on accurate estimates of the latent physical properties of objects in the scene. Specifically, we test scenarios where accurate prediction relies on estimates of properties such as mass, friction, elasticity, and deformability, and where the values of those properties can only be inferred by observing how objects move and interact with other objects or fluids. We evaluate the performance of a number of state-of-the-art prediction models that span a variety of levels of learning vs. built-in knowledge, and compare that performance to a set of human predictions. We find that models that have been trained using standard regimes and datasets do not spontaneously learn to make inferences about latent properties, but also that models that encode objectness and physical states tend to make better predictions. However, there is still a huge gap between all models and human performance, and all models' predictions correlate poorly with those made by humans, suggesting that no state-of-the-art model is learning to make physical predictions in a human-like way. Project page: https://dingmyu.github.io/physion_v2/
Overview
CoChat Summary
TL;DR
Physion++ is a new benchmark of 9 simulated physical scenarios across mass, friction, elasticity, and deformability, designed to test whether models can infer latent object properties online and use them for contact prediction. State-of-the-art video and dynamics models mostly perform near chance, fail to use property inference, and correlate poorly with human predictions.
Key findings
- Across the benchmark, most models perform only slightly above chance; unified training is typically worse than separate per-property training, and models with property inference perform similarly to those without it, indicating they are not using the inference phase.
- DPI-Net achieves the best overall performance (60.2% with property, 57.5% without, 92.2% fully observed) and is the only model showing clear mass-property inference (75.9% with property vs. 60.1% without).
- Humans perform reliably above chance but not especially well (60% accuracy); all video models perform slightly worse, and only DPI-Net is comparable. No model correlates well with humans: SlotFormer has the highest correlation (r=0.12 vs. split-half human correlation r=0.37).
- Models underperform humans on “easy” trials but outperform humans on “hard” trials; most models do better than humans on deformability but worse on mass, friction, and elasticity. Ablations show performance near random at 200 training trials and saturating by 1000–2000 trials; adding explicit property estimation as an auxiliary task barely changes performance.
Methods
- Stimuli were generated with the ThreeDWorld simulator across 9 scenarios covering mass, friction, elasticity, and deformability. Each property has a dynamics training set of 2,000 trials, a readout fitting set of 192 trials (96 pairs), and a testing set of 192 trials (96 pairs); paired test videos share the same first prediction-phase frame but differ in latent properties and outcomes.
- Models evaluated include MCVD, pRESNET-mlp, pVGG-mlp, pDEIT-mlp, SlotFormer, ALOE, and DPI-Net. Training protocols were “separate” per property or “unified” across properties; testing variants were “w/ property,” “w/o property,” and “fully observed.” Readout models map learned representations to YES/NO contact predictions.
- Human data came from 200 Prolific participants (50 per mechanical property), paid $15.50/hour. Each participant saw 32 stimuli from one property, balanced 16 YES and 16 NO; analyses compared model and human accuracy and Pearson correlation, including easy trials (>67% human accuracy) and hard trials (<33% human accuracy).
Limitations
- The paper states that property inference may be difficult, models were not designed for it, 2,000 scenarios may be insufficient for it to emerge, and models may struggle with multi-stage long-term dependencies; task videos last 13–20 seconds, longer than typical training videos. The checklist also reports no error bars across random seeds.
- The paper states the dataset can only be used to measure physical prediction based on latent property inference and that third-party contributions are not contemplated.
- Evident from the text: all stimuli are simulator-generated TDW scenes rather than real-world videos, human data are limited to 200 participants (50 per property), and the benchmark’s test set is 192 trials per property.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 Valid · 0 Not valid · 3 Not checked · 83 Own results and interpretations
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We generate 2000 trials for each mechanical property without YES/NO labels for dynamics pretraining.”
§ Download and unzip the three data files --Training_data ... --Readout_data ... --Testing_data --Folder organized by physical properties # folders named with copy0 and copy1 indicate two matched trials ( e .g ., the same video location from copy0 will have a different property / outcome in copy1 but the same initial conditions ) --Subfolder organized by scenarios and objects # e .g .bouncy_platformu s e _ b l o c k e r _ w i t h _ h o l e =0 -target = cube --[ id ]. json # object id and instance information over time --[ id ]. pkl # meta information of the physical event and the video --[ id ] _image .mp4 # raw input , RGB video --[ id ] _seg .mp4 # segmentation masks of the video --[ id ] _map .png # indicating the yellow and red objects # Below are lists of test filenames and ids (192 entries per physical property ) --Mass .csv --Friction .csv --Elasticity .csv --Deformability .csv > Training set.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“yet all video models are performing slightly worse than humans.”
§ Physion++: Evaluating Physical Scene Understanding that Requires Online Inference of Different Physical Properties > 4.2 Human experiments
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Only DPI-Net, which operates on ground truth states from the physics engine performs comparably to humans.”
§ Physion++: Evaluating Physical Scene Understanding that Requires Online Inference of Different Physical Properties > 4.2 Human experiments
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“DPI-Net achieves the best overall performance”
§ Physion++: Evaluating Physical Scene Understanding that Requires Online Inference of Different Physical Properties > 4.1 Model Performance
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Most models perform only slightly above chance”
§ Physion++: Evaluating Physical Scene Understanding that Requires Online Inference of Different Physical Properties > 4.1 Model Performance
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
35 more claims
Create a free account to see all 86 claims and every source verdict.
- The quote only partly states this claim. 3
- Probably not a substantial claim. 31
- It may state two separate claims as one. 22
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
§ Physion++: Evaluating Physical Scene Understanding that Requires Online Inference of Different Physical Properties > 4.1 Model Performance
“Based on these representations we train a readout model as logistic regression.”
§ Download and unzip the three data files --Training_data ... --Readout_data ... --Testing_data --Folder organized by physical properties # folders named with copy0 and copy1 indicate two matched trials ( e .g ., the same video location from copy0 will have a different property / outcome in copy1 but the same initial conditions ) --Subfolder organized by scenarios and objects # e .g .bouncy_platformu s e _ b l o c k e r _ w i t h _ h o l e =0 -target = cube --[ id ]. json # object id and instance information over time --[ id ]. pkl # meta information of the physical event and the video --[ id ] _image .mp4 # raw input , RGB video --[ id ] _seg .mp4 # segmentation masks of the video --[ id ] _map .png # indicating the yellow and red objects # Below are lists of test filenames and ids (192 entries per physical property ) --Mass .csv --Friction .csv --Elasticity .csv --Deformability .csv > D Pipelines and settings
“We implement a multilayer perceptron (MLP) with intermediate dimensions of 256 and 64 as our readout model, which is trained on rollout scene representations from the readout set to classify whether the two cued objects contact.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“| | | fully observed | 59.9 | 63.9 | 61.2 | 56.0 | 55.5 | 65.1 | 92.2 |”
§ Download and unzip the three data files --Training_data ... --Readout_data ... --Testing_data --Folder organized by physical properties # folders named with copy0 and copy1 indicate two matched trials ( e .g ., the same video location from copy0 will have a different property / outcome in copy1 but the same initial conditions ) --Subfolder organized by scenarios and objects # e .g .bouncy_platformu s e _ b l o c k e r _ w i t h _ h o l e =0 -target = cube --[ id ]. json # object id and instance information over time --[ id ]. pkl # meta information of the physical event and the video --[ id ] _image .mp4 # raw input , RGB video --[ id ] _seg .mp4 # segmentation masks of the video --[ id ] _map .png # indicating the yellow and red objects # Below are lists of test filenames and ids (192 entries per physical property ) --Mass .csv --Friction .csv --Elasticity .csv --Deformability .csv > G Checklist
Reported 56 · recomputed 59.65
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“We selected 192 trials from each scenario, then shuffle and split the 192 trials into 8 disjoint sets, where all sets have half of the trials containing contact between the red and yellow objects, and half do not.Each participant was shown 32 stimuli drawn from one of these disjoint sets of trials drawn from a single mechanical property”
§ Physion++: Evaluating Physical Scene Understanding that Requires Online Inference of Different Physical Properties > 4.2 Human experiments
Reported 32 · recomputed 24
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Vision meets robotics: The KITTI dataset
Geiger, A., Lenz, P., Stiller, C., & Urtasun, R. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913491297 · Cited by 9,913
Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments
Ionescu, C., Papava, D., Olaru, V., & Sminchisescu, C. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.248 · Cited by 3,653
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., III, H. D., & Crawford, K. (2021) · Communications of the ACM · doi:10.1145/3458723 · Cited by 2,469
Learning Latent Dynamics for Planning from Pixels
Hafner, D., Lillicrap, T., Fischer, I., Villegas, R., Ha, D., Lee, H., & Davidson, J. (2018) · arXiv · doi:10.48550/arxiv.1811.04551 · Cited by 708
Deep visual foresight for planning robot motion
Finn, C., & Levine, S. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989324 · Cited by 599
CLEVRER: CoLlision Events for Video REpresentation and ReasoningPreprint
Yi, K., Gan, C., Li, Y., Kohli, P., Wu, J., Torralba, A. B., & Tenenbaum, J. B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.01442 · Cited by 578
Self-citationDream to Control: Learning Behaviors by Latent Imagination
Hafner, D., Lillicrap, T., Ba, J., & Norouzi, M. (2019) · arXiv · doi:10.48550/arxiv.1912.01603 · Cited by 558
Learning to Poke by Poking: Experiential Learning of Intuitive PhysicsPreprint
Agrawal, P., Nair, A., Abbeel, P., Malik, J., & Levine, S. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.07419 · Cited by 404
DaViT: Dual Attention Vision Transformers
Ding, M., Xiao, B., Codella, N., Luo, P., Wang, J., & Yuan, L. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-20053-3_5 · Cited by 382
Self-citationRoboNet: Large-Scale Multi-Robot Learning
Dasari, S., Ebert, F., Tian, S., Nair, S., Bucher, B., Schmeckpeper, K., Singh, S., Levine, S., & Finn, C. (2019) · Digital Eprints Services at ISB (DESI) (Indian School of Business) · doi:10.48550/arxiv.1910.11215 · Cited by 332
ThreeDWorld: A Platform for Interactive Multi-Modal Physical SimulationPreprint
Gan, C., Schwartz, J., Alter, S., Mrowca, D., Schrimpf, M., Traer, J. A., Freitas, J. D., Kubilius, J., Bhandwaldar, A., Haber, N., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.04954 · Cited by 320
Self-citationMONet: Unsupervised Scene Decomposition and Representation
Burgess, C. P., Matthey, L., Watters, N., Kabra, R., Higgins, I., Botvinick, M., & Lerchner, A. (2019) · arXiv · doi:10.48550/arxiv.1901.11390 · Cited by 270
Physion: Evaluating Physical Prediction from Vision in Humans and MachinesPreprint
Bear, D. M., Wang, E., Mrowca, D., Binder, F. J., Tung, H.-Y. F., Pramod, R. T., Holdaway, C., Tao, S., Smith, K. A., Sun, F.-Y., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.08261 · Cited by 228
Self-citationTraining data-efficient image transformers & distillation through attention
Touvron, H., Cord, M., Douze, M., Massa, F., Sablayrolles, A., & Jégou, H. (2020) · arXiv · doi:10.48550/arxiv.2012.12877 · Cited by 182
Learning Visual Predictive Models of Physics for Playing BilliardsPreprint
Fragkiadaki, K., Agrawal, P., Levine, S., & Malik, J. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.07404 · Cited by 180
Simple Unsupervised Object-Centric Learning for Complex and Naturalistic Videos
Singh, G., Wu, Y., & Ahn, S. (2022) · neural information processing systems · doi:10.52202/068431-1322 · Cited by 172
Learning Particle Dynamics for Manipulating Rigid Bodies, Deformable Objects, and Fluids
Li, Y., Wu, J., Tedrake, R., Tenenbaum, J. B., & Torralba, A. B. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.01566 · Cited by 161
Self-citationMCVD: Masked Conditional Video Diffusion for Prediction, Generation, and InterpolationPreprint
Voleti, V., Jolicoeur‐Martineau, A., & Pal, C. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2205.09853 · Cited by 147
IntPhys: A Framework and Benchmark for Visual Intuitive Physics ReasoningPreprint
Riochet, R., Castro, M. Y., Bernard, M., Lerer, A. K., Fergus, R., Véronique Izard, A., & Emmanuel Dupoux, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.07616 · Cited by 132
SlotFormer: Unsupervised Visual Dynamics Simulation with Object-Centric ModelsPreprint
Wu, Z., Dvornik, N., Greff, K., Kipf, T., & Garg, A. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2210.05861 · Cited by 127
Inferring mass in complex scenes by mental simulation
Hamrick, J. B., Battaglia, P., Griffiths, T. L., & Tenenbaum, J. B. (2016) · Cognition · doi:10.1016/j.cognition.2016.08.012 · Cited by 121
Self-citationIntuitive physics learning in a deep-learning model inspired by developmental psychology
Piloto, L. S., Weinstein, A., Battaglia, P., & Botvinick, M. (2022) · Nature Human Behaviour · doi:10.1038/s41562-022-01394-8 · Cited by 118
FitVid: Overfitting in Pixel-Level Video PredictionPreprint
Babaeizadeh, M., Saffar, M., Nair, S., Levine, S., Finn, C., & Erhan, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.13195 · Cited by 109
Seeing liquids from visual motion
Kawabe, T., Maruya, K., Fleming, R. W., & Nishida, S. (2014) · Vision Research · doi:10.1016/j.visres.2014.07.003 · Cited by 106
Attention over learned object embeddings enables complex visual reasoningPreprint
Ding, D., Hill, F., Santoro, A., Reynolds, M., & Botvinick, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.08508 · Cited by 100
CoPhy: Counterfactual Learning of Physical DynamicsPreprint
Baradel, F., Neverova, N., Mille, J., Mori, G., & Wolf, C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.12000 · Cited by 90
gradSim: Differentiable simulation for system identification and visuomotor controlPreprint
Jatavallabhula, K. M., Macklin, M., Golemo, F., Voleti, V., Petrini, L., Weiss, M., Considine, B., Parent-Lévesque, J., Xie, K., Erleben, K., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.02646 · Cited by 85
Dynamic Visual Reasoning by Learning Differentiable Physics Models from Video and LanguagePreprint
Ding, M., Chen, Z., Du, T., Luo, P., Tenenbaum, J. B., & Gan, C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.15358 · Cited by 83
Self-citationShapeStacks: Learning Vision-Based Physical Intuition for Generalised Object Stacking
Groth, O., Fuchs, F. B., Posner, I., & Vedaldi, A. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01246-5_43 · Cited by 82
Visual Features in the Perception of Liquids
van Assen, J. J. R., Barla, P., & Fleming, R. W. (2018) · Current Biology · doi:10.1016/j.cub.2017.12.037 · Cited by 79
Shape, motion, and optical cues to stiffness of elastic objects
Paulun, V. C., Schmidt, F., van Assen, J. J. R., & Fleming, R. W. (2017) · Journal of Vision · doi:10.1167/17.1.20 · Cited by 75
VDT: General-purpose Video Diffusion Transformers via Mask ModelingPreprint
Lu, H., Yang, G., Fei, N., Huo, Y., Lu, Z., Luo, P., & Ding, M. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.13311 · Cited by 74
Self-citationCompositional Physical Reasoning of Objects and Events From Videos
Chen, Z., Dong, S., Yi, K., Li, Y., Ding, M., Torralba, A. B., Tenenbaum, J. B., & Gan, C. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3574322 · Cited by 69
Self-citationSeeing liquids from static snapshots
Paulun, V. C., Kawabe, T., Nishida, S., & Fleming, R. W. (2015) · Vision Research · doi:10.1016/j.visres.2015.01.023 · Cited by 68
Modeling human intuitions about liquid flow with particle-based simulation
Bates, C., Yildirim, I., Tenenbaum, J. B., & Battaglia, P. (2019) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1007210 · Cited by 53
Self-citationESPRIT: Explaining Solutions to Physical Reasoning Tasks
Rajani, N. F., Zhang, R., Tan, Y. C., Zheng, S., Weiss, J., Vyas, A., Gupta, A., Xiong, C., Socher, R., & Radev, D. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.706 · Cited by 26
Simple Unsupervised Object-Centric Learning for Complex and Naturalistic VideosPreprint
Singh, G., Wu, Y., & Ahn, S. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2205.14065 · Cited by 20
Visually inferring elasticity from the motion trajectory of bouncing cubes
Paulun, V. C., & Fleming, R. W. (2020) · Journal of Vision · doi:10.1167/jov.20.6.6 · Cited by 20
IntPhys 2019: A Benchmark for Visual Intuitive Physics Understanding
Riochet, R., Castro, M. Y., Bernard, M., Lerer, A., Fergus, R., Izard, V., & Dupoux, E. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3083839 · Cited by 19
ComPhy: Compositional Physical Reasoning of Objects and Events from VideosPreprint
Chen, Z., Yi, K., Li, Y., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2205.01089 · Cited by 14
Self-citationThe Visual Discrimination of Bending
Norman, J. F., Wiesemann, E. Y., Norman, H. F., et al. (2007) · Perception · doi:10.1068/p5641 · Cited by 14
Neurocomputational Modeling of Human Physical Scene Understanding
Yildirim, I., Smith, K. A., Belledonne, M., Wu, J., & Tenenbaum, J. B. (2018) · Conference on Cognitive Computational Neuroscience (CCN) · doi:10.32470/ccn.2018.1091-0 · Cited by 9
Self-citationWang, K., Aanjaneya, M., & Bekris, K. E. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.13859 · Cited by 8
Neural Material: Learning Elastic Constitutive Material and Damping Models from Sparse DataPreprint
Wang, B., Kry, P. G., Deng, Y., Ascher, U. M., Huang, H. F., & Chen, B. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.04931 · Cited by 4
CRAFT: A Benchmark for Causal Reasoning About Forces and inTeractions
Ates, T., Atesoglu, M. S., Yigit, C., Kesen, I., Kobas, M., Erdem, E., Erdem, A., Goksun, T., & Yuret, D. (2020) · arXiv · doi:10.48550/arxiv.2012.04293 · Cited by 2
Seeing jelly: judging elasticity of a transparent object
Kawabe, T., & Nishida, S. (2016) · Proceedings of the ACM Symposium on Applied Perception · doi:10.1145/2931002.2931008
ESPRIT: Explaining Solutions to Physical Reasoning Tasks
Rajani, N. F., Zhang, R., Tan, Y. C., Zheng, S., Weiss, J., Vyas, A., Gupta, A., XIong, C., Socher, R., & Radev, D. (2020) · arXiv · doi:10.48550/arxiv.2005.00730
Research works printed but not linked: 5
- Yichen Li, YingQiao Wang, Tal Boger, Kevin Smith, Samuel J Gershman, and Tomer Ullman. An approximate representation of objects underlies physical reasoning. PsyArXiv, 2022.
- Jiajun Wu, Ilker Yildirim, Joseph J Lim, Bill Freeman, and Josh Tenenbaum. Galileo: Perceiving physical object properties by integrating a physics engine with deep learning. Advances in neural information processing systems, 28, 2015.
- Yunzhu Li, Toru Lin, Kexin Yi, Daniel Bear, Daniel L.K. Yamins, Jiajun Wu, Joshua B. Tenenbaum, and Antonio Torralba. Visual grounding of learned physical models. In International Conference on Machine Learning, 2020.
- Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.
- Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [Yes] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [Yes]
Cited by
A Survey of Reasoning with Foundation ModelsPreprint
Sun, J., Zheng, C., Xie, E., Liu, Z., Chu, R., Liu, J., Xu, J., Ding, M., Li, H., Geng, M., et al. (2023) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/ac4sp · Cited by 154
Self-citationContPhy: Continuum Physical Concept Learning and Reasoning from VideosPreprint
Zheng, Z., Yan, X., Chen, Z., Wang, J., Lim, Q. Z. E., Tenenbaum, J. B., & Gan, C. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2402.06119 · Cited by 14
Self-citationDo Vision-Language Models Have Internal World Models? Towards an Atomic EvaluationPreprint
Gao, Q., Pi, X., Liu, K., Chen, J., Yang, R., Huang, X., Fang, X., Sun, L., Kishore, G., Ai, B., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2506.21876 · Cited by 12
Self-citationBenchmarks for Physical Reasoning AIPreprint
Melnik, A., Schiewer, R., Lange, M., Muresanu, A. I., Saeidi, M., Garg, A., & Ritter, H. J. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2312.10728 · Cited by 10
Do Vision-Language Models Have Internal World Models? Towards an Atomic Evaluation
Gao, Q., Pi, X., Liu, K., Chen, J., Yang, R., Huang, X., Fang, X., Sun, L., Kishore, G., Ai, B., et al. (2025) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2025.findings-acl.1342 · Cited by 14
Self-citation
Metadata
- Authors
- Tung, H.-Y., Ding, M., Chen, Z., Bear, D., Gan, C., Tenenbaum, J., Yamins, D., Fan, J., & Smith, K.
- Venue
- Advances in Neural Information Processing Systems 36
- Published
- 2023
- Pages
- 67048-67068
- ISSN
- 1049-5258 (ISSN-L)
- OpenAlex
- W7133212649
- Cited by
- 8
- References
- 16
- Citations by year
- 2026: 4 · 2025: 1 · 2024: 1 · 2023: 2
- FWCI
- 0.86
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- neural information processing systems (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Robotics and Sensor-Based Localization
- Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
- Keywords
- online inference, physical properties
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-11-01, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-11-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.