Structured, flexible, and robust: benchmarking and improving large language models towards more human-like behavior in out-of-distribution reasoning tasks
Open accessAbstract
Human language offers a powerful window into our thoughts -- we tell stories, give explanations, and express our beliefs and goals through words. Abundant evidence also suggests that language plays a developmental role in structuring our learning. Here, we ask: how much of human-like thinking can be captured by learning statistical patterns in language alone? We first contribute a new challenge benchmark for comparing humans and distributional large language models (LLMs). Our benchmark contains two problem-solving domains (planning and explanation generation) and is designed to require generalization to new, out-of-distribution problems expressed in language. We find that humans are far more robust than LLMs on this benchmark. Next, we propose a hybrid Parse-and-Solve model, which augments distributional LLMs with a structured symbolic reasoning module. We find that this model shows more robust adaptation to out-of-distribution planning problems, demonstrating the promise of hybrid AI models for more human-like reasoning.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Several studies design question answering datasets such that answering correctly purportedly requires reasoning about the physical world (e.g. colour, shape, motion).
“For example, several studies design question answering datasets such that answering the question correctly purportedly requires reasoning about the physical world (e.g. colour, shape, motion) [10,11].”
From Symbols and grounding in large language models (2023) · cites this paper as [11]
Part I (p. 2) describes a benchmark of planning and causal-explanation prompts, and the Results (p. 5) discuss problems requiring physical reasoning, including colour, water, gravity, and materials. The paper does not describe its benchmark as a question-answering dataset with correct answers; its Part I prompts elicit open-ended responses. (LLM, conf=0.82)
Experimental — accuracy not yet measuredSuch question-answering studies reveal that LLMs perform very poorly compared to humans.
“Such studies reveal that LLMs perform very poorly compared to humans, and are thus taken to be evidence that the LLM requires grounding in the physical world in order to succeed.”
From Symbols and grounding in large language models (2023) · cites this paper as [11]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 13
- Bisk, Y., Zellers, R., Bras, R. L., Gao, J., & Choi, Y. (2019). PIQA: reasoning about physical commonsense in natural language. CoRR , abs/1911.11641 . Retrieved from http://arxiv.org/abs/1911.11641 · arXiv
- Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., . . . Amodei, D. (2020). Language models are few-shot learners. CoRR , abs/2005.14165 . Retrieved from https://arxiv.org/abs/2005.14165 · arXiv
- Carey, S. (2009). Where our number concepts come from. The Journal of philosophy , 106 (4), 220.
- Chen, M., Tworek, J., Jun, H., Yuan, Q., de Oliveira Pinto, H. P., Kaplan, J., . . . Zaremba, W. (2021). Evaluating large language models trained on code. CoRR , abs/2107.03374 . Retrieved from https://arxiv .org/abs/2107.03374 · arXiv
- Cobbe, K., Kosaraju, V., Bavarian, M., Hilton, J., Nakano, R., Hesse, C., & Schulman, J. (2021). Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168 . · arXiv
- Ellis, K., Wong, C., Nye, M. I., Sabl´ e-Meyer, M., Cary, L., Morales, L., . . . Tenenbaum, J. B. (2020). Dreamcoder: Growing generalizable, interpretable knowledge with wake-sleep bayesian program learning. CoRR , abs/2006.08381 . Retrieved from https:// arxiv.org/abs/2006.08381 · arXiv
- Fodor, J. A. (1975). The language of thought . Harvard University Press.
- Gupta, N., & Nau, D. S. (1992). On the complexity of blocks-world planning. Artificial Intelligence , 56 (2-3), 223-254.
- Gopnik, A., & Meltzoff, A. N. (1997). Words, thoughts, and theories . Mit Press.
- Harris, P. L., Koenig, M. A., Corriveau, K. H., & Jaswal, V. K. (2018). Cognitive foundations of learning from testimony. Annual Review of Psychology , 69 , 251-273.
- Nye, M. I., Tessler, M. H., Tenenbaum, J. B., & Lake, B. M. (2021). Improving coherence and consistency in neural sequence models with dual-system, neurosymbolic reasoning. CoRR , abs/2107.02794 . Retrieved from https://arxiv.org/abs/2107.02794 · arXiv
- Rae, J. W., Borgeaud, S., Cai, T., Millican, K., Hoffmann, J., Song, F., . . . others (2021). Scaling language models: Methods, analysis & insights from training gopher. arXiv preprint arXiv:2112.11446 . · arXiv
- Wong, C., Ellis, K., Tenenbaum, J. B., & Andreas, J. (2021). Leveraging language to learn program abstractions and search heuristics. CoRR , abs/2106.11053 . Retrieved from https://arxiv.org/abs/2106.11053 · arXiv
Cited by
Sorted by citations
A Survey of GPT-3 Family Large Language Models Including ChatGPT and GPT-4
Kalyan, K. S. (2024) · Natural Language Processing Journal · doi:10.1016/j.nlp.2023.100048 · Cited by 335 · see citing works
Eight Things to Know about Large Language Models
Bowman, S. R. (2023) · arXiv · doi:10.48550/arxiv.2304.00612 · Cited by 137 · see citing works
“…al., 2019; Stiennon et al., 2020; Ouyang et al., 2022; Bai et al., 2022a), integration with image processing systems, (Alayrac et al., 2022; OpenAI, 2023b), or integration with other software tools (Nakano et al., 2021; Menick et al., 2022; Collins et al., 2022; Schick et al., 2023; OpenAI, 2023a).”
Harms from Increasingly Agentic Algorithmic Systems
Chan, A., Salganik, R., Markelius, A., Pang, C., Rajkumar, N., Krasheninnikov, D., Langosco, L., He, Z., Duan, Y., Carroll, M., et al. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594033 · Cited by 149 · see citing works
“While it is crucial not to idealize or over-hype a model’s performance by ignoring model failures [23, 24, 28, 47, 120, 153, 187], it is also important not to understate (and thus fail to anticipate negative consequences of) what these models can do and may be capable of doing in the near future [31, 90, 103], especia…”
BackgroundSelf-citationEight Things to Know about Large Language Models
Bowman, S. R. (2024) · Critical AI · doi:10.1215/2834703x-11556011 · Cited by 198 · see citing works
AI for social science and social science of AI: A survey
Xu, R. F., Sun, Y., Ren, M., Guo, S., Pan, R., Lin, H., Sun, L., & Han, X. (2024) · Information Processing & Management · doi:10.1016/j.ipm.2024.103665 · Cited by 149 · see citing works
Symbols and grounding in large language models
Pavlick, E. (2023) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2022.0041 · Cited by 95 · see citing works
Generalized Planning in PDDL Domains with Pretrained Large Language Models
Silver, T., Dan, S., Srinivas, K., Tenenbaum, J. B., Kaelbling, L. P., & Katz, M. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i18.30006 · Cited by 90 · see citing works
Self-citationDemystifying large language models in second language development research
Cong, Y. (2024) · Computer Speech & Language · doi:10.1016/j.csl.2024.101700 · Cited by 21 · see citing works
Resilience Assessment of Large Language Models under Transient Hardware Faults
Agarwal, U. K., Chan, A., & Pattabiraman, K. (2023) · Proceedings/Proceedings - International Symposium on Software Reliability Engineering · doi:10.1109/issre59848.2023.00052 · Cited by 18 · see citing works
van Dijk, B., Kouwenhoven, T., Spruit, M. R., & van Duijn, M. J. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.779 · Cited by 22 · see citing works
Distilling Script Knowledge from Large Language Models for Constrained Language Planning
Yuan, S., Chen, J., Fu, Z., Ge, X., Shah, S., Jankowski, C., Xiao, Y., & Yang, D. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.236 · Cited by 23 · see citing works
Arora, R., Singh, S., Swaminathan, K., Datta, A., Banerjee, S., Bhowmick, B., Murthy, J. K., Sridharan, M., & Krishna, M. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10611164 · Cited by 11 · see citing works
MORPHeus: a Multimodal One-armed Robot-assisted Peeling System with Human Users In-the-loop
Ye, R., Hu, Y., Bian, Y. A., Kulm, L., & Bhattacharjee, T. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610050 · Cited by 8 · see citing works
Evaluating deep generative models on cognitive tasks: a case study
Tang, Z., & Kejriwal, M. (2023) · Discover Artificial Intelligence · doi:10.1007/s44163-023-00067-3 · Cited by 14 · see citing works
Replicating Reason: The Advent of Human-like Audit Judgment by Generative AIPreprint
Isack, M. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4716860 · Cited by 4 · see citing works
- Authors
- Collins, K. M., Wong, C., Feng, J., Wei, M., & Tenenbaum, J. B.
- Affiliations
- University of Cambridge (GB)
- Massachusetts Institute of Technology (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2022
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2205.05718
- OpenAlex
- W7071831927
- Cited by
- 13
- References
- 0
- Citations by year
- 2024: 7 · 2023: 6
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- eScholarship (California Digital Library) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Funders
- Defense Advanced Research Projects Agency
- Quest for Intelligence, Massachusetts Institute of Technology
- Office of Naval Research (FA9550)
- Air Force Office of Scientific Research (FA9550-, FA9550-19-1-0269)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- large language models, planning, explanation generation, human-like reasoning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
