CoChat
Sign Up
PreprintCited by 16Psychiatry, Mental Health, NeuroscienceTopic ModelingCognitive Science and Education Research

The DOI matches its DataCite record.

Large Language Models and Cognitive Science: A Comprehensive Review of Similarities, Differences, and Challenges

Open access
Niu, Qian, Liu, Junyu, Bi, Ziqian, Feng, Pohsun, Peng, Benji, Chen, Keyu, Li, Ming, Yan, Lawrence KQ, Zhang, Yichao, Yin, Caitlyn Heqi, Fei, Cheng, Wang, Tianyang, Wang, Yunze, Chen, Silin, Liu, Ming, Qin, Ziyuan, Bao, Riyang, Song, Xinyuan, Jiang, Zekun·2024·arXiv
DOI 10.48550/arxiv.2409.02387arXiv 2409.02387
Open this paper in CoChat — free accountOpen PDF

Abstract

This comprehensive review explores the intersection of Large Language Models (LLMs) and cognitive science, examining similarities and differences between LLMs and human cognitive processes. We analyze methods for evaluating LLMs cognitive abilities and discuss their potential as cognitive models. The review covers applications of LLMs in various cognitive fields, highlighting insights gained for cognitive science research. We assess cognitive biases and limitations of LLMs, along with proposed methods for improving their performance. The integration of LLMs with cognitive architectures is examined, revealing promising avenues for enhancing artificial intelligence (AI) capabilities. Key challenges and future research directions are identified, emphasizing the need for continued refinement of LLMs to better align with human cognition. This review provides a balanced perspective on the current state and future potential of LLMs in advancing our understanding of both artificial and human intelligence.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2409.02387

Cites

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

No references could be resolved yet.

Research works printed but not linked: 52

  1. A. Goldstein, Z. Zada, E. Buchnik, M. Schain, A. Price, S. A. Nastase, A. Feder, D. Emanuel, A. Cohen, A. Jansen, H. Gazula, G. Choe, A. Rao, C. Kim, C. Casto, A. Flinker, S. Devore, W. Doyle, D. Friedman, P. Dugan, A. Hassidim, M. Brenner, Y. Matias, K. A. Norman, O. Devinsky, and U. Hasson, "Thinking ahead: prediction in context as a keystone of language in humans and machines," arXiv [cs.CL], 2020.
  2. G. Mischler, Y. A. Li, S. Bickel, A. D. Mehta, and N. Mesgarani, "Contextual feature extraction hierarchies converge in large language models and the brain," arXiv [cs.CL], 2024.
  3. J. Shaki, S. Kraus, and M. Wooldridge, "Cognitive effects in large language models," in European Conference on Artificial Intelligence, 2023.
  4. I. Dasgupta, A. K. Lampinen, S. C. Y. Chan, H. R. Sheahan, A. Creswell, D. Kumaran, J. L. McClelland, and F. Hill, "Lan- guage models show human-like content effects on reasoning tasks," arXiv [cs.CL], 2022.
  5. R. Marjieh, I. Sucholutsky, P. van Rijn, N. Jacoby, and T. L. Griffiths, "Large language models predict human sensory judg- ments across six modalities," arXiv [cs.CL], 2023.
  6. K. M. Collins, C. Wong, J. Feng, M. Wei, and J. B. Tenenbaum, "Structured, flexible, and robust: benchmarking and improving large language models towards more human-like behavior in out-of-distribution reasoning tasks," arXiv [cs.CL], 2022.
  7. S. Lamprinidis, "LLM cognitive judgements differ from human," arXiv [cs.CL], 2023.
  8. K. Mahowald, A. A. Ivanova, I. A. Blank, N. Kanwisher, J. B. Tenenbaum, and E. Fedorenko, "Dissociating language and thought in large language models," arXiv [cs.CL], 2023.
  9. R. A. Janik, "Aspects of human memory and large language models," arXiv [cs.CL], 2023.
  10. S. Suresh, K. Mukherjee, X. Yu, W.-C. Huang, L. Padua, and T. Rogers, "Conceptual structure coheres in human cognition but not in large language models," in Conference on Empirical Methods in Natural Language Processing, 2023.
  11. J. Coda-Forno, M. Binz, J. X. Wang, and E. Schulz, "CogBench: a large language model walks into a psychology lab," arXiv [cs.CL], 2024.
  12. T. Hagendorff and S. Fabi, "Human-like intuitive behavior and reasoning biases emerged in language models -and disap- peared in GPT-4," ArXiv, 2023.
  13. Y. Zhang, X. Zhang, C. Li, S. Wang, and C. Zong, "MulCog- Bench: A multi-modal cognitive benchmark dataset for evaluat- ing chinese and english computational language models," arXiv [cs.CL], 2024.
  14. A. A. Ivanova, "Running cognitive evaluations on large language models: The do's and the don'ts," arXiv [cs.CL], 2023.
  15. R. Srinivasan, H. Inakoshi, and K. Uchino, "Leveraging cogni- tive science for testing large language models," in International Conference on Artificial Intelligence Testing, 2023.
  16. M. Binz and E. Schulz, "Using cognitive psychology to under- stand GPT-3," Proc. Natl. Acad. Sci. U. S. A., 2022.
  17. --, "Turning large language models into cognitive models," arXiv [cs.CL], 2023.
  18. L. Wong, G. Grand, A. K. Lew, N. D. Goodman, V. K. Mansinghka, J. Andreas, and J. B. Tenenbaum, "From word models to world models: Translating from natural language to the probabilistic language of thought," arXiv [cs.CL], 2023.
  19. S. T. Piantadosi and F. Hill, "Meaning without reference in large language models," arXiv [cs.CL], 2022.
  20. M. Schrimpf, I. Blank, G. Tuckute, C. Kauf, E. A. Hosseini, N. Kanwisher, J. Tenenbaum, and E. Fedorenko, "The neural architecture of language: Integrative modeling converges on predictive processing," Proc. Natl. Acad. Sci. U. S. A., 2020.
  21. A. K. Lampinen, "Can language models handle recursively nested grammatical structures? a case study on comparing models and humans," arXiv [cs.CL], 2022.
  22. S. Nolfi, "On the unexpected abilities of large language mod- els," arXiv [cs.CL], 2023.
  23. J. Li and J. Li, "Memory, consciousness and large language model," arXiv [cs.CL], 2024.
  24. E. Pavlick, "Symbols and grounding in large language models," Philosophical Transactions of the Royal Society A, 2023.
  25. Y. Ren, R. Jin, T. Zhang, and D. Xiong, "Do large language models mirror cognitive language processing?" arXiv [cs.CL], 2024.
  26. C. Veres, "A precis of language models are not models of language," arXiv [cs.CL], 2022.
  27. M. Shanahan, "Talking about large language models," arXiv [cs.CL], 2022.
  28. J. Grindrod, "Modelling language," arXiv [cs.CL], 2024.
  29. M. Mitchell and D. Krakauer, "The debate over understanding in AI's large language models," Proc. Natl. Acad. Sci. U. S. A., 2022.
  30. X. Liu, P. Xu, J. Wu, J. Yuan, Y. Yang, Y. Zhou, F. Liu, T. Guan, H. Wang, T. Yu, J. McAuley, W. Ai, and F. Huang, "Large language models and causal inference in collaboration: A comprehensive survey," arXiv [cs.CL], 2024.
  31. E. Kıcıman, R. Ness, A. Sharma, and C. Tan, "Causal reasoning and large language models: Opening a new frontier for causal- ity," arXiv [cs.CL], 2023.
  32. T. Chakrabarty, V. Padmakumar, F. Brahman, and S. Muresan, "Creativity support in the age of large language models: An empirical study involving emerging writers," arXiv [cs.CL], 2023.
  33. T. Ullman, "Large language models fail on trivial alterations to theory-of-mind tasks," arXiv [cs.CL], 2023.
  34. A. N. Talboy and E. Fuller, "Challenging the appearance of machine intelligence: Cognitive bias in LLMs and best practices for adoption," arXiv [cs.CL], 2023.
  35. D. Thorstad, "Cognitive bias in large language models: Cautious optimism meets anti-panglossian meliorism," arXiv [cs.CL], 2023.
  36. A. K. Singh, S. Devkota, B. Lamichhane, U. Dhakal, and C. Dhakal, "The confidence-competence gap in large language models: A cognitive study," arXiv [cs.CL], 2023.
  37. G. Marcus, E. Leivada, and E. Murphy, "A sentence is worth a thousand pictures: Can large language models understand human language?" arXiv [cs.CL], 2023.
  38. O. Macmillan-Scott and M. Musolesi, "(ir)rationality and cogni- tive biases in large language models," arXiv [cs.CL], 2024.
  39. E. Jones and J. Steinhardt, "Capturing failures of large language models via human cognitive biases," in Neural Information Processing Systems, 2022.
  40. K. Nguyen, "Language models are bounded pragmatic speak- ers: Understanding RLHF from a bayesian cognitive modeling perspective," arXiv [cs.CL], 2023.
  41. Y. Lv, H. Pan, R. Fu, M. Liu, Z. Wang, and B. Qin, "CogGPT: Unleashing the power of cognitive dynamics on large language models," arXiv [cs.CL], 2024.
  42. B. Prystawski, P. Thibodeau, C. Potts, and N. D. Good- man, "Psychologically-informed chain-of-thought prompts for metaphor understanding in large language models," arXiv [cs.CL], 2022.
  43. K. L. Aw and M. Toneva, "Training language models to summa- rize narratives improves brain alignment," arXiv [cs.CL], 2022.
  44. M. Du, F. He, N. Zou, D. Tao, and X. Hu, "Shortcut learning of large language models in natural language understanding," Commun. ACM, 2022.
  45. O. J. Romero, J. Zimmerman, A. Steinfeld, and A. Tomasic, "Synergistic integration of large language models and cognitive architectures for robust AI: An exploratory analysis," arXiv [cs.CL], 2023.
  46. J. R. Kirk, R. E. Wray, and J. E. Laird, "Exploiting language models as a source of knowledge for cognitive agents," arXiv [cs.CL], 2023.
  47. H. Joshi and V. Ustun, "Augmenting cognitive architectures with large language models," in Proceedings of the AAAI Symposium Series, 2024.
  48. M. A. Gonzalez-Santamarta, F. J. Rodríguez-Lera, A. M. Guerrero-Higueras, and V. Matellan-Olivera, "Integration of large language models within cognitive architectures for au- tonomous robots," arXiv [cs.CL], 2023.
  49. F. Zhu and R. Simmons, "Bootstrapping cognitive agents with a large language model," in AAAI Conference on Artificial Intelligence, 2024.
  50. C. Nakos and K. D. Forbus, "Using large language models in the companion cognitive architecture: A case study and future prospects," in Proceedings of the AAAI Symposium Series, 2024.
  51. R. Wray, J. R. Kirk, and J. Laird, "Language models as a knowledge source for cognitive agents," arXiv.org, 2021.
  52. Y. Zhou, Q. Zhu, J. Jin, and Z. Dou, "Cognitive personalized search integrating large language models with an efficient memory mechanism," arXiv [cs.CL], 2024.

Cited by

Sorted by citations

See all 16

Metadata

Authors
Niu, Q., Liu, J., Bi, Z., Feng, P., Peng, B., Chen, K., Li, M., Yan, L. K., Zhang, Y., Yin, C. H., Fei, C., Wang, T., Wang, Y., Chen, S., Liu, M., Qin, Z., Bao, R., Song, X., & Jiang, Z.
Affiliations
  • Kyoto University (JP)
  • Indiana University Bloomington (US)
  • National Taiwan Normal University (TW)
  • Georgia Institute of Technology (US)
  • Hong Kong University of Science and Technology (HK)
  • The University of Texas at Dallas (US)
  • University of Wisconsin–Madison (US)
  • Cornell University (US)
  • University of Liverpool (GB)
  • University of Edinburgh (GB)
  • Zhejiang University (CN)
  • Purdue University West Lafayette (US)
  • Emory University (US)
Corresponding author
Niu, Qian
ORCID
Venue
arXiv
Publisher
arXiv
Published
2024
Volume
7
Issue
1
ISSN
2712-0074 (ISSN-L)
DOI
10.48550/arxiv.2409.02387
arXiv
2409.02387
OpenAlex
W7137907728
Cited by
16
References
0
Citations by year
2026: 5 · 2025: 10 · 2024: 1
FWCI
17.32
Type
journal article
Language
en
Open access
Licence
Locations
  • BIO Integration (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion, cc-by-sa)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Psychiatry, Mental Health, Neuroscience
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Cognitive Science and Education Research
Keywords
large language models, human cognition, cognitive models, cognitive biases, cognitive architecture, LLM evaluation, human-AI comparison

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Large Language Models and Cognitive Science: A Comprehensive Review of Similarities, Differences, and Challenges. https://app.cochat.ai/papers/10.48550/arxiv.2409.02387
The paper
Niu, Q., Liu, J., Bi, Z., Feng, P., Peng, B., Chen, K., Li, M., Yan, L. K., Zhang, Y., Yin, C. H., Fei, C., Wang, T., Wang, Y., Chen, S., Liu, M., Qin, Z., Bao, R., Song, X., & Jiang, Z. (2024). Large Language Models and Cognitive Science: A Comprehensive Review of Similarities, Differences, and Challenges. arXiv. https://doi.org/10.48550/arxiv.2409.02387