Do Neural Language Representations Learn Physical Commonsense?
Open accessAbstract
Humans understand language based on the rich background knowledge about how the physical world works, which in turn allows us to reason about the physical world through language. In addition to the properties of objects (e.g., boats require fuel) and their affordances, i.e., the actions that are applicable to them (e.g., boats can be driven), we can also reason about if-then inferences between what properties of objects imply the kind of actions that are applicable to them (e.g., that if we can drive something then it likely requires fuel). In this paper, we investigate the extent to which state-of-the-art neural language representations, trained on a vast amount of natural language text, demonstrate physical commonsense reasoning. While recent advancements of neural language models have demonstrated strong performance on various types of natural language inference tasks, our study based on a dataset of over 200k newly collected annotations suggests that neural language representations still only learn associations that are explicitly written down.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Probing-classifier studies have tended to be more positive than question-answering studies, often reporting results that suggest LLMs perform well above baseline, though still below humans.
“These studies have tended to be more positive than those which are based on question answering tasks, often reporting results that suggest the LLMs perform well above baseline, though still below humans.”
From Symbols and grounding in large language models (2023) · cites this paper as [94]
The Results (p. 5) report good performance on object–property compatibility and performance around human ability on object–affordance compatibility, but also report that models struggle on affordance–property compatibility and score well below humans there. The paper does not compare probing-classifier studies with question-answering studies, and its findings do not support a blanket characterization of model performance as well above baseline but below humans. (LLM, conf=0.85)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460 · see citing works
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071 · see citing works
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983 · see citing works
The Senses Considered as Perceptual Systems
West, C. K., & Gibson, J. J. (1969) · Journal of Aesthetic Education · doi:10.2307/3331482 · Cited by 6,344 · see citing works
Six views of embodied cognition
Wilson, M. (2002) · Psychonomic Bulletin & Review · doi:10.3758/bf03196322 · Cited by 4,782 · see citing works
(2013) · Cited by 1,951 · see citing works
Gaver, W. W. (1991) · Proceedings of the SIGCHI conference on Human factors in computing systems Reaching through technology - CHI '91 · doi:10.1145/108844.108856 · Cited by 1,536 · cited 1× · see citing works
“Coined by Gibson (1966), this term initially described animal-perceived uses for an object, but has since come to mean the perceived uses of an object in a given environment (Norman, 1988; Gaver, 1991).”
Dependency-Based Word Embeddings
Levy, O., & Goldberg, Y. (2014) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/v1/p14-2050 · Cited by 1,091 · see citing works
Semantic feature production norms for a large set of living and nonliving things
McRae, K., Cree, G. S., Seidenberg, M. S., & McNorgan, C. (2005) · Behavior Research Methods · doi:10.3758/bf03192726 · Cited by 1,070 · see citing works
Levesque, H. J., Davis, E., & Morgenstern, L. (2011) · Cited by 879 · see citing works
Simulation as an engine of physical scene understanding
Battaglia, P. W., Hamrick, J. B., & Tenenbaum, J. B. (2013) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1306572110 · Cited by 681 · see citing works
Galileo: Perceiving Physical Object Properties by Integrating a Physics Engine with Deep Learning
Wu, J., Yildirim, I., Lim, J. J., Freeman, W. T., & Tenenbaum, J. B. (2016) · Cited by 264 · see citing works
Situation Recognition: Visual Semantic Role Labeling for Image Understanding
Yatskar, M., Zettlemoyer, L., & Farhadi, A. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.597 · Cited by 253 · see citing works
The Centre for Speech, Language and the Brain (CSLB) concept property norms
Devereux, B. J., TYLER, L. K., Geertzen, J., & Randall, B. (2013) · Behavior Research Methods · doi:10.3758/s13428-013-0420-4 · Cited by 178 · see citing works
How Well Do Distributional Models Capture Different Types of Semantic Knowledge?
Rubinstein, D., Levi, E., Schwartz, R., & Rappoport, A. (2015) · doi:10.3115/v1/p15-2119 · Cited by 74 · see citing works
Lucy, L., & Gauthier, J. (2017) · RoboNLP@ACL · doi:10.18653/v1/w17-2810 · Cited by 68 · see citing works
Human few-shot learning of compositional instructionsPreprint
Lake, B. M., Linzen, T., & Baroni, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.04587 · Cited by 62 · see citing works
Extracting implicit knowledge from text
Durme, V., & Benjamin, D. J. (2009) · Cited by 19 · see citing works
The Embodied Mind: Cognitive Science and Human Experience (Book)
Gover, M. R. (1996) · Mind Culture and Activity · doi:10.1207/s15327884mca0304_9 · Cited by 16 · see citing works
Lucy, L., & Gauthier, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.11168 · Cited by 14 · see citing works
Faulty Towers: A hypothetical simulation model of physical support.
Gerstenberg, T., Zhou, L., Smith, K. A., & Tenenbaum, J. B. (2017) · eScholarship (California Digital Library) · Cited by 7 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805
Flagged first, then by citations
Research works printed but not linked: 1
- Herbelot, A., & Vecchi, E. M. (2015). From concepts to models: some issues in quantifying feature norms. In Lilt (Vol. 2).
Cited by
Sorted by citations
Pre-trained models: Past, present and future
Han, X., Zhang, Z., Ding, N., Gu, Y., Liu, X., Huo, Y., Qiu, J., Yao, Y., Zhang, A., Zhang, L., et al. (2021) · AI Open · doi:10.1016/j.aiopen.2021.08.002 · Cited by 994 · see citing works
Measuring and improving consistency in pretrained language models
Elazar, Y., Kassner, N., Ravfogel, S., Ravichander, A., Hovy, E., Schütze, H., & Goldberg, Y. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00410 · Cited by 220 · see citing works
Inducing Relational Knowledge from BERT
Bouraoui, Z., Camacho-Collados, J., & Schockaert, S. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6242 · Cited by 12 · see citing works
Amnesic Probing: Behavioral Explanation with Amnesic Counterfactuals
Elazar, Y., Ravfogel, S., Jacovi, A., & Goldberg, Y. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00359 · Cited by 126 · see citing works
Symbols and grounding in large language models
Pavlick, E. (2023) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2022.0041 · Cited by 95 · see citing works
Strong Prediction: Language Model Surprisal Explains Multiple N400 Effects
Michaelov, J. A., Bardolph, M. D., Petten, C. K. V., Bergen, B. K., & Seana Coulson, A. (2023) · Neurobiology of Language · doi:10.1162/nol_a_00105 · Cited by 74 · see citing works
Knowledgeable or Educated Guess? Revisiting Language Models as Knowledge Bases
Cao, B., Lin, H., Han, X., Sun, L., Yan, L., Liao, M., Xue, T., & Xu, J. (2021) · doi:10.18653/v1/2021.acl-long.146 · Cited by 90 · see citing works
Can Language Models Encode Perceptual Structure Without Grounding? A Case Study in Color
Abdou, M., Kulmizev, A., Hershcovich, D., Frank, S., Pavlick, E., & Søgaard, A. (2021) · Proceedings of the 25th Conference on Computational Natural Language Learning · doi:10.18653/v1/2021.conll-1.9 · Cited by 64 · see citing works
Commonsense Knowledge Reasoning and Generation with Pre-trained Language Models: A Survey
Bhargava, P., & Ng, V. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i11.21496 · Cited by 54 · see citing works
Do Neural Language Models Overcome Reporting Bias?
Shwartz, V., & Choi, Y. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.605 · Cited by 43 · see citing works
Self-citationBreaking Common Sense: WHOOPS! A Vision-and-Language Benchmark of Synthetic and Compositional Images
Bitton-Guetta, N., Bitton, Y., Hessel, J., Schmidt, L., Elovici, Y., Stanovsky, G., & Schwartz, R. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00247 · Cited by 37 · see citing works
BERTnesia: Investigating the capture and forgetting of knowledge in BERT
Wallat, J., Singh, J., & Anand, A. (2020) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/2020.blackboxnlp-1.17 · Cited by 33 · see citing works
Elazar, Y., Zhang, H., Goldberg, Y., & Roth, D. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.819 · Cited by 34 · see citing works
Language (Re)modelling: Towards Embodied Language Understanding
Tamari, R., Shani, C., Hope, T., Petruck, M. R. L., Abend, O., & Shahaf, D. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.559 · Cited by 28 · see citing works
Understanding Commonsense Inference Aptitude of Deep Contextual Representations
Da, J., & Kasai, J. (2019) · Proceedings of the First Workshop on Commonsense Inference in Natural Language Processing · doi:10.18653/v1/d19-6001 · Cited by 22 · see citing works
Context Matters: A Pragmatic Study of PLMs’ Negation Understanding
Gubelmann, R., & Handschuh, S. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.315 · Cited by 20 · see citing works
Acquiring and modeling abstract commonsense knowledge via conceptualization
He, M., Fang, T., Wang, W., & Song, Y. (2024) · Artificial Intelligence · doi:10.1016/j.artint.2024.104149 · Cited by 19 · see citing works
Misra, K., Rayz, J. T., & Ettinger, A. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.213 · Cited by 23 · see citing works
Distilling Relation Embeddings from Pretrained Language Models
Ushio, A., Camacho-Collados, J., & Schockaert, S. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.712 · Cited by 18 · see citing works
Can Language Models Understand Physical Concepts?
Li, L., Xu, J., Dong, Q., Zheng, C., Sun, X., Kong, L., & Liu, Q. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.726 · Cited by 18 · see citing works
Do Language Models Know the Way to Rome?
Liétard, B., Abdou, M., & Søgaard, A. (2021) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/2021.blackboxnlp-1.40 · Cited by 16 · see citing works
Tiered Reasoning for Intuitive Physics: Toward Verifiable Commonsense Language Understanding
Storks, S., Gao, Q., Zhang, Y., & Chai, J. Y. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.422 · Cited by 14 · see citing works
Apidianaki, M., & Soler, A. G. (2021) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/2021.blackboxnlp-1.7 · Cited by 14 · see citing works
Dove, G. O. (2024) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2023.0149 · Cited by 12 · see citing works
Semantic Novelty Detection in Natural Language Descriptions
Ma, N., Politowicz, A. L., Mazumder, S., Chen, J., Liu, B., Robertson, E., & Grigsby, S. S. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.66 · Cited by 12 · see citing works
Can a Gorilla Ride a Camel? Learning Semantic Plausibility from Text
Porada, I., Suleman, K., & Cheung, J. C. K. (2019) · doi:10.18653/v1/d19-6015 · Cited by 11 · see citing works
VASR: Visual Analogies of Situation Recognition
Bitton, Y., Yosef, R., Strugo, E., Shahaf, D., Schwartz, R., & Stanovsky, G. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i1.25096 · Cited by 10 · see citing works
Knowledgeable or Educated Guess? Revisiting Language Models as Knowledge Bases
Cao, B., Lin, H., Han, X., Sun, L., Yan, L., Liao, M., Xue, T., & Xu, J. (2021) · arXiv · doi:10.48550/arxiv.2106.09231 · Cited by 10 · see citing works
Open Relation Modeling: Learning to Define Relations between Entities
Huang, J., Chang, K. C., Xiong, J., & Hwu, W. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.26 · Cited by 10 · see citing works
Enriching Language Models with Visually-grounded Word Vectors and the Lancaster Sensorimotor Norms
Kennington, C. (2021) · Conference on Computational Natural Language Learning · doi:10.18653/v1/2021.conll-1.11 · Cited by 8 · see citing works
Language Models as a Knowledge Source for Cognitive Agents
Wray, R. E., III, Kirk, J. R., & Laird, J. E. (2021) · arXiv · doi:10.48550/arxiv.2109.08270 · Cited by 8 · see citing works
Alper, M., Fiman, M., & Averbuch‐Elor, H. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00655 · Cited by 8 · see citing works
Probing Pre-Trained Language Models for Disease Knowledge
Alghanmi, I., Espinosa-Anke, L., & Schockaert, S. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.266 · Cited by 7 · see citing works
How Pre-trained Word Representations Capture Commonsense Physical Comparisons
Goel, P., Feng, S., & Boyd-Graber, J. L. (2019) · doi:10.18653/v1/d19-6016 · Cited by 7 · see citing works
Does BERT Know that the IS-A Relation Is Transitive?
Lin, R., & Ng, H. T. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-short.11 · Cited by 6 · see citing works
Zhao, Z., Papalexakis, E. E., & Ma, X. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.266 · Cited by 7 · see citing works
Towards Semantics-Enhanced Pre-Training: Can Lexicon Definitions Help Learning Sentence Meanings?
Ren, X., Sun, X., Wang, H., & Liu, Q. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i15.17619 · Cited by 6 · see citing works
Modelling General Properties of Nouns by Selectively Averaging Contextualised Embeddings
Li, N., Bouraoui, Z., Camacho-Collados, J., Espinosa-Anke, L., Gu, Q., & Schockaert, S. (2021) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/530 · Cited by 5 · see citing works
Porada, I., Sordoni, A., & Cheung, J. C. K. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.337 · Cited by 3 · see citing works
Knowledge Acquired by Foundation Models
Paaß, G., & Giesselbach, S. (2023) · Artificial intelligence: foundations, theory, and algorithms/Artificial intelligence: Foundations, theory, and algorithms · doi:10.1007/978-3-031-23190-2_4 · Cited by 3 · see citing works
What do Toothbrushes do in the Kitchen? How Transformers Think our World is Structured
Henlein, A., & Mehler, A. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.425 · Cited by 4 · see citing works
Probing Physical Reasoning with Counter-Commonsense Context
Kondo, K., Sugawara, S., & Aizawa, A. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-short.53 · Cited by 3 · see citing works
Learning the Effects of Physical Actions in a Multi-modal Environment
Dagan, G., Keller, F., & Lascarides, A. (2023) · Findings of the Association for Computational Linguistics: EACL · doi:10.18653/v1/2023.findings-eacl.10 · Cited by 3 · see citing works
On Masked Language Models for Contextual Link Prediction
Brayne, A., Wiatrak, M., & Corneil, D. (2022) · Workshop on Knowledge Extraction and Integration for Deep Learning Architectures: Deep Learning Inside Out (DeeLIO) · doi:10.18653/v1/2022.deelio-1.9 · Cited by 2 · see citing works
McOmet: Multimodal Fusion Transformer for Physical Audiovisual Commonsense Reasoning
Zong, D., & Sun, S. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i5.25813 · Cited by 2 · see citing works
RetractedSommerauer, P. (2020) · Annual Meeting of the Association for Computational Linguistics: Student Research Workshop (ACL SRW) · doi:10.18653/v1/2020.acl-srw.18 · Cited by 3 · see citing works
Deriving Word Vectors from Contextualized Language Models using Topic-Aware Mention Selection
Wang, Y., Bouraoui, Z., Espinosa-Anke, L., & Schockaert, S. (2021) · Workshop on Representation Learning for NLP (RepL4NLP) · doi:10.18653/v1/2021.repl4nlp-1.19 · Cited by 2 · see citing works
Analyzing COVID-19 Tweets with Transformer-based Language ModelsPreprint
Feldman, P., Tiwari, S., Cheah, C. S. L., Foulds, J. R., & Pan, S. (2021) · arXiv (Cornell University) · doi:10.13016/m2j3r1-ktks · Cited by 3 · see citing works
Divergences between Language Models and Human BrainsPreprint
Zhou, Y., Liu, E., Neubig, G., Tarr, M. J., & Wehbe, L. (2023) · PubMed · doi:10.48550/arxiv.2311.09308 · Cited by 4 · see citing works
Anderson, M., & Camacho-Collados, J. (2022) · Joint Conference on Lexical and Computational Semantics · doi:10.18653/v1/2022.starsem-1.15 · Cited by 1 · see citing works
VIPHY: Probing “Visible” Physical Commonsense Knowledge
Singh, S., Qasemi, E., & Chen, M. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.473 · Cited by 3 · see citing works
Gajbhiye, A., Bouraoui, Z., Li, N., Chatterjee, U., Espinosa-Anke, L., & Schockaert, S. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.654 · Cited by 1 · see citing works
Chatterjee, U., Gajbhiye, A., & Schockaert, S. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.725 · Cited by 1 · see citing works
BERTnesia: Investigating the capture and forgetting of knowledge in BERTPreprint
Wallat, J., Singh, J., & Anand, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.09313 · Cited by 8 · see citing works
Toward context-based text-to-3D scene generation
Henlein, A. (2022) · doi:10.21248/gups.73448 · Cited by 1 · see citing works
Understanding Object Affordances Through Verb Usage Patterns.Preprint
Lam, K. C., Pereira, F., Vaziri-Pashkam, M., Woodard, K. M., & McMahon, E. (2020) · arXiv (Cornell University)
Interactive language learning - How to explore complex environments using natural language?
Petrache, T.-A., Rebedea, T. E., & Trăușan-Matu, Ș. (2020) · International Joural of User-System Interaction · doi:10.37789/ijusi.2020.13.1.2
Modelling Commonsense Commonalities with Multi-Facet Concept Embeddings
Kteich, H., Li, N., Chatterjee, U., Bouraoui, Z., & Schockaert, S. (2024) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2024.findings-acl.86
- Authors
- Forbes, M., Holtzman, A., & Choi, Y.
- Affiliations
- University of Washington (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2019
- Pages
- 1753-1759
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1908.02899
- OpenAlex
- W2964895772
- Cited by
- 60
- References
- 21
- Citations by year
- 2024: 3 · 2023: 14 · 2022: 11 · 2021: 19 · 2020: 9 · 2019: 4
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- eScholarship (California Digital Library) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (1703166, 1637479, 1524371)
- Defense Advanced Research Projects Agency
- Army Research Office (W911NF-15-1-, W911NF, W911NF-15)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- natural language inference, affordances, object properties, language models, prior knowledge
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
