CoChat
Sign Up
PreprintCited by 24Child and Animal Learning DevelopmentTopic ModelingLogic, Reasoning, and Knowledge

LLMs achieve adult human performance on higher-order theory of mind tasks

Open access
Street, Winnie, Siy, John Oliver, Keeling, Geoff, Baranes, Adrien, Barnett, Benjamin, McKibben, Michael, Kanyere, Tatenda, Lentz, Alison +2 more·2024·arXiv
DOI 10.48550/arxiv.2405.18870arXiv 2405.18870PMC PMC12808479
Open this paper in CoChat — free accountOpen PDF

Abstract

This paper examines the extent to which large language models (LLMs) have developed higher-order theory of mind (ToM); the human ability to reason about multiple mental and emotional states in a recursive manner (e.g. I think that you believe that she knows). This paper builds on prior work by introducing a handwritten test suite -- Multi-Order Theory of Mind Q&A -- and using it to compare the performance of five LLMs to a newly gathered adult human benchmark. We find that GPT-4 and Flan-PaLM reach adult-level and near adult-level performance on ToM tasks overall, and that GPT-4 exceeds adult performance on 6th order inferences. Our results suggest that there is an interplay between model size and finetuning for the realisation of ToM abilities, and that the best-performing LLMs have developed a generalised capacity for ToM. Given the role that higher-order ToM plays in a wide range of cooperative and competitive human behaviours, these findings have significant implications for user-facing LLM applications.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2405.18870

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Prior studies investigating self-consciousness in large language models lack functional definitions of self-consciousness.

    “However, these studies have two major limitations: (1) The absence of functional definitions of self-consciousness; and (2) The lack of exploration of the language model's internal state of self-consciousness (i.e., how the model represents self-consciousness, and whether it can be manipulated or acquired).”

    From From Imitation to Introspection: Probing Self-Consciousness in Language Models (2025) · cites this paper as Street et al., 2024

    The Introduction (page unavailable) defines theory of mind, and the Abstract (page unavailable) describes tests of higher-order theory-of-mind performance. Neither addresses whether prior LLM studies lacked functional definitions of self-consciousness or examined models’ internal representations of it. Searches for self-consciousness, functional definitions, and internal states found no passage establishing those limitations. (LLM, conf=0.74)

    Experimental — accuracy not yet measured

  • Prior studies investigating self-consciousness in large language models lack exploration of the language model's internal state of self-consciousness.

    “However, these studies have two major limitations: (1) The absence of functional definitions of self-consciousness; and (2) The lack of exploration of the language model's internal state of self-consciousness (i.e., how the model represents self-consciousness, and whether it can be manipulated or acquired).”

    From From Imitation to Introspection: Probing Self-Consciousness in Language Models (2025) · cites this paper as Street et al., 2024

    The Introduction (page unavailable) discusses gaps in understanding the computational processes behind LLM theory-of-mind performance. It does not address self-consciousness, how a model internally represents it, or whether it can be manipulated or acquired; searches for those concepts found no relevant passage. (LLM, conf=0.75)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 81

  1. Alvarez, R. M., Atkeson, L. R., Levin, I., and Li, Y. (2019). Paying attention to inattentive survey respondents. Polit. Anal. 27, 145-162. doi: 10.1017/pan.2018. · DOI
  2. Baron-Cohen, S., Leslie, A. M., and Frith, U. (1985). Does the autistic child have a "theory of mind"? Cognition 21, 37-46. doi: 10.1016/0010-0277(85)90022-8 · DOI · Paper page
  3. Begeer, S. M., Gevers, C., Clifford, P., and Moerbeek, M. (2011). Theory of mind training in children with autism: a randomized controlled trial. J. Autism Dev. Disord. 41, 1358-1366. doi: 10.1007/s10803-010-1121-9 · DOI
  4. Bishop-Fitzpatrick, L., Minshew, N. J., and Eack, S. M. (2013). A systematic review of psychosocial interventions for adults with autism spectrum disorders. J. Autism Dev. Disord. 43, 687-694. doi: 10.1007/s10803-012-1615-8 · DOI
  5. Bisong, E. (2019). "Google colaboratory, " in Building Machine Learning and Deep Learning Models on Google Cloud Platform: A Comprehensive Guide for Beginners. Berkeley, CA: Apress, 59-64. doi: 10.1007/978-1-4842-44 70-8_7 · DOI
  6. Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., et al. (2021). On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258. doi: 10.48550/arXiv.2108. 07258 · DOI · Paper page
  7. Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., et al. (2020). Language models are few-shot learners. Adv. Neural Inform. Process. Syst. 33, 1877-1901. doi: 10.5555/3495724.3495883 · DOI
  8. Bubeck, S., Chandrasekaran, V., Eldan, R., Gehrke, J., Horvitz, E., Kamar, E., et al. (2023). Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv preprint arXiv:2303.12712. doi: 10.48550/arXiv.2303.12712 · DOI · Paper page
  9. Cheng, L. S., Burgess, D., Vernooij, N., Solís-Barroso, C., McDermott, A., and Namboodiripad, S. (2021). The problematic concept of native speaker in psycholinguistics: replacing vague and harmful terminology with inclusive and accurate measures. Front. Psychol. 12:715843. doi: 10.3389/fpsyg.2021.715843 · DOI
  10. Chomsky, N. (2014). Aspects of the Theory of Syntax. Number 11. Cambridge, MA: MIT Press.
  11. Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., et al. (2023). Palm: scaling language modeling with pathways. J. Mach. Learn. Res. 24, 1-113. doi: 10.5555/3648699.3648939 · DOI
  12. Chung, H. W., Hou, L., Longpre, S., Zoph, B., Tay, Y., Fedus, W., et al. (2024). Scaling instruction-finetuned language models. J. Mach. Learn. Res. 25, 1-53. doi: 10.5555/3722577.3722647 · DOI
  13. De Weerd, H., Verbrugge, R., and Verheij, B. (2017). Negotiating with other minds: the role of recursive theory of mind in negotiation with incomplete information. Auton. Agents Multi-Agent Syst. 31, 250-287. doi: 10.1007/s10458-015-9317-1 · DOI
  14. De Weerd, H., Verbrugge, R., and Verheij, B. (2022). Higher-order theory of mind is especially useful in unpredictable negotiations. Auton. Agents Multi-Agent Syst. 36:30. doi: 10.1007/s10458-022-09558-6 · DOI
  15. Dunbar, R. I., Launay, J., and Curry, O. (2016). The complexity of jokes is limited by cognitive constraints on mentalizing. Hum. Nat. 27, 130-140. doi: 10.1007/s12110-015-9251-6 · DOI
  16. El-Sayed, S., Akbulut, C., McCroskery, A., Keeling, G., Kenton, Z., Jalan, Z., et al. (2024). A mechanism-based approach to mitigating harms from persuasive generative AI. arXiv preprint arXiv:2404.15058. doi: 10.48550/arXiv.2404.15058 · DOI · Paper page
  17. Gabriel, I., Manzini, A., Keeling, G., Hendricks, L. A., Rieser, V., Iqbal, H., et al. (2024). The ethics of advanced ai assistants. arXiv preprint arXiv:2404.16244. doi: 10.48550/arXiv.2404.16244 · DOI · Paper page
  18. Gandhi, K., Fränken, J.-P., Gerstenberg, T., and Goodman, N. (2024). Understanding social reasoning in language models with language models. Adv. Neural Inform. Process. Syst. 36, 13518-13529. doi: 10.5555/3666122.3666717 · DOI
  19. Golan, O., and Baron-Cohen, S. (2006). Systemizing empathy: teaching adults with autism spectrum conditions to recognize complex emotions using "the mind reading" web-based training program. J. Dev. Learn. Disord. 10, 1-19. doi: 10.1017/S0954579406060305 · DOI · Paper page
  20. Happé, F. G. (1994). An advanced test of theory of mind: understanding of story characters' thoughts and feelings by able autistic, mentally handicapped, and normal children and adults. J. Autism Dev. Disord. 24, 129-154. doi: 10.1007/BF021 72093 · DOI · Paper page
  21. Harrington, L., Siegert, R., and McClure, J. (2005). Theory of mind in schizophrenia: a critical review. Cogn. Neuropsychiatry 10, 249-286. doi: 10.1080/13546800444000056 · DOI
  22. Heider, F. (1946). Attitudes and cognitive organization. J. Psychol. 21, 107-112. doi: 10.1080/00223980.1946.9917275 · DOI · Paper page
  23. Henighan, T., Kaplan, J., Katz, M., Chen, M., Hesse, C., Jackson, J., et al. (2020). Scaling laws for autoregressive generative modeling. arXiv preprint arXiv:2010.14701. doi: 10.48550/arXiv.2010.14701 · DOI · Paper page
  24. Hooker, C. I., Verosky, S. C., Germine, L. T., Knight, R. T., and D'Esposito, M. (2008). Mentalizing about emotion and its relationship to empathy. Soc. Cogn. Affect. Neurosci. 3, 204-217. doi: 10.1093/scan/nsn019 · DOI · Paper page
  25. Hu, J., Sosa, F., and Ullman, T. (2025). Re-evaluating theory of mind evaluation in large language models. Phil. Trans. R. Soc. B. 380:20230499. doi: 10.1098/rstb.2023.0499 · DOI · Paper page
  26. Humphrey, N. K. (1976). "The social function of intellect, " in Growing Points in Ethology, eds. P. G. Bateson and R. A. Hinde (Cambridge: Cambridge University Press), 303-317.
  27. Hyde, J. S., and Linn, M. C. (1988). Gender differences in verbal ability: a meta- analysis. Psychol. Bull. 104:53. doi,: 10.1037/0033-2909.104.1.53 · DOI · Paper page
  28. Keysar, B., Lin, S., and Barr, D. J. (2003). Limits on theory of mind use in adults. Cognition 89, 25-41. doi: 10.1016/S0010-0277(03)00064-7 · DOI · Paper page
  29. Kinderman, P., Dunbar, R., and Bentall, R. P. (1998). Theory-of-mind deficits and causal attributions. Br. J. Psychol. 89, 191-204. doi: 10.1111/j.2044-8295.1998. · DOI
  30. tb02680.x Kordbache, E., Noubarian, M., Babaei, F., Eshraghi, M., and Tarnas, G. (2024). The effectiveness of therapeutic interventions based on theory of mind in improving social skills and adaptability in children with autism spectrum disorder. KMAN Counsel. Psychol. Nexus 2, 136-142. doi: 10.61838/kman.psynexus.2.1.20 · DOI
  31. Kosinski, M. (2024). Evaluating large language models in theory of mind tasks. Proc. Natl. Acad. Sci. U.S.A. 121:e2405460121. doi: 10.1073/pnas.2405460121 · DOI · Paper page
  32. Krems, J. A., Dunbar, R. I., and Neuberg, S. L. (2016). Something to talk about: are conversation sizes constrained by mental modeling abilities? Evol. Hum. Behav. 37, 423-428. doi: 10.1016/j.evolhumbehav.2016.05.005 · DOI
  33. Lane, J. D., Wellman, H. M., Olson, S. L., LaBounty, J., and Kerr, D. C. (2010). Theory of mind and emotion understanding predict moral development in early childhood. Br. J. Dev. Psychol. 28, 871-889. doi: 10.1348/026151009X483056 · DOI
  34. Launay, J., Pearce, E., Wlodarski, R., van Duijn, M., Carney, J., and Dunbar, R. I. (2015). Higher-order mentalising and executive functioning. Pers. Individ. Diff. 86, 6-14. doi: 10.1016/j.paid.2015.05.021 · DOI
  35. Le Berre, A.-P. (2019). Emotional processing and social cognition in alcohol use disorder. Neuropsychology 33:808. doi: 10.1037/neu0000572 · DOI
  36. Lecce, S., Bianco, F., Devine, R. T., Hughes, C., and Banerjee, R. T. (2014). The impact of theory of mind training on children's prosocial behavior: a longitudinal study. J. Exp. Child Psychol. 126, 1-13. doi: 10.1016/j.jecp.2014.03.002 · DOI
  37. Lei, J., and Ventola, P. (2018). Characterising the relationship between theory of mind and anxiety in children with autism spectrum disorder and typically developing children. Res. Autism Spectr. Disord. 49, 1-12. doi: 10.1016/j.rasd.2018.01.005 · DOI
  38. Lewis, P. A., Birch, A., Hall, A., and Dunbar, R. I. (2017). Higher order intentionality tasks are cognitively more demanding. Soc. Cogn. Affect. Neurosci. 12, 1063-1071. doi: 10.1093/scan/nsx034 · DOI
  39. Lewis, P. A., Rezaie, R., Brown, R., Roberts, N., and Dunbar, R. I. (2011). Ventromedial prefrontal volume predicts understanding of others and social network size. Neuroimage 57, 1624-1629. doi: 10.1016/j.neuroimage.2011.05.030 · DOI · Paper page
  40. Liddle, B., and Nettle, D. (2006). Higher-order theory of mind and social competence in school-age children. J. Cult. Evol. Psychol. 4, 231-244. doi: 10.1556/JCEP.4.2006.3-4. · DOI
  41. Antikythera J. 2025. doi: 10.1162/ANTI.5CZK · DOI
  42. Longpre, S., Hou, L., Vu, T., Webson, A., Chung, H. W., Tay, Y., et al. (2023). "The flan collection: Designing data and methods for effective instruction tuning, " in International Conference on Machine Learning (Honolulu, HI: PMLR), 22631-22648.
  43. Lu, Y., Bartolo, M., Moore, A., Riedel, S., and Stenetorp, P. (2022). "Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity, " in Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (Dublin: Association for Computational Linguistics), 8086-8098. doi: 10.18653/v1/2022.acl-long.556 · DOI · Paper page
  44. Malle, B. F. (2004). How the Mind Explains Behavior. Folk Explanation, Meaning and Social Interaction. Cambridge, MA: MIT Press. doi: 10.7551/mitpress/3586.001.0001 · DOI
  45. Mao, S.-Y., Chiu, H. M., Yu, Y.-T., and Chen, K.-L. (2023). The associations of theory of mind with both general and theory-of-mind-related social interaction in children with autism spectrum disorder. Res. Autism Spectr. Disord. 102:102107. doi: 10.1016/j.rasd.2023.102107 · DOI
  46. Millière, R., and Rathkopf, C. (2024). Anthropocentric bias and the possibility of artificial cognition. arXiv [Preprint]. arXiv-2407.
  47. Oesch, N., and Dunbar, R. I. (2017). The emergence of recursion in human language: mentalising predicts recursive syntax task performance. J. Neurolinguist. 43, 95-106. doi: 10.1016/j.jneuroling.2016.09.008 · DOI
  48. OpenAI (2023). Gpt-4 technical report. arXiv [Preprint]. arXiv:2303.08774. · arXiv
  49. Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., et al. (2022). "Training language models to follow instructions with human feedback, " in Advances in Neural Information Processing Systems, vol. 35 (New Orleans, LA: Neural Information Processing Systems Foundation, Inc.), 27730-27744.
  50. Park, J. S., O'Brien, J., Cai, C. J., Morris, M. R., Liang, P., and Bernstein, M. S. (2023). "Generative agents: interactive simulacra of human behavior, " in Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology (New York, NY: Association for Computing Machinery), 1-22. doi: 10.1145/3586183. 3606763 · DOI · Paper page
  51. Pham, T. M., Korbel, J., Hanel, R., and Thurner, S. (2022). Empirical social triad statistics can be explained with dyadic homophylic interactions. Proc. Natl. Acad. Sci. 119:e2121103119. doi: 10.1073/pnas.2121103119 · DOI
  52. Piantadosi, S. T., and Hill, F. (2022). Meaning without reference in large language models. arXiv [Preprint]. arXiv:2208.02957. · arXiv
  53. Powell, J. L., Lewis, P. A., Dunbar, R. I., García-Fi nana, M., and Roberts, N. (2010). Orbital prefrontal cortex volume correlates with social cognitive competence. Neuropsychologia 48, 3554-3562. doi: 10.1016/j.neuropsychologia.2010.08.004 · DOI
  54. Premack, D., and Woodruff, G. (1978). Does the chimpanzee have a theory of mind? Behav. Brain Sci. 1, 515-526. doi: 10.1017/S0140525X00076512 · DOI · Paper page
  55. Sap, M., Le Bras, R., Fried, D., and Choi, Y. (2022). "Neural theory-of-mind? on the limits of social intelligence in large lms, " in Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (Abu Dhabi: Association for Computational Linguistics), 3762-3780. doi: 10.18653/v1/2022.emnlp- main.248 · DOI · Paper page
  56. Schick, B., De Villiers, P., De Villiers, J., and Hoffmeister, R. (2007). Language and theory of mind: a study of deaf children. Child Dev. 78, 376-396. doi: 10.1111/j.1467-8624.2007.01004.x · DOI · Paper page
  57. Senju, A., Southgate, V., White, S., and Frith, U. (2009). Mindblind eyes: an absence of spontaneous theory of mind in asperger syndrome. Science 325, 883-885. doi: 10.1126/science.1176170 · DOI · Paper page
  58. Shamay-Tsoory, S. G., Aharon-Peretz, J., and Perry, D. (2009). Two systems for empathy: a double dissociation between emotional and cognitive empathy in inferior frontal gyrus versus ventromedial prefrontal lesions. Brain 132, 617-627. doi: 10.1093/brain/awn279 · DOI · Paper page
  59. Shapira, N., Levy, M., Alavi, S. H., Zhou, X., Choi, Y., Goldberg, Y., et al. (2024). "Clever hans or neural theory of mind? stress testing social reasoning in large language models, " in Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers) (St. Julian's: Association for Computational Linguistics), 2257- 2273.doi: 10.18653/v1/2024.eacl-long.138 · DOI · Paper page
  60. Silber, H., Roßsmann, J., and Gummer, T. (2022). The issue of noncompliance in attention check questions: false positives in instructed response items. Field Methods 34, 346-360. doi: 10.1177/1525822X221115830 · DOI
  61. Stiller, J., and Dunbar, R. I. (2007). Perspective-taking and memory capacity predict social network size. Soc. Netw. 29, 93-104. doi: 10.1016/j.socnet.2006.04.001 · DOI · Paper page
  62. Stinson, J., Wolfe, R., and Spaulding, W. (2022). Social connectedness in schizotypy: the role of cognitive and affective empathy. Behav. Sci. 12:253. doi: 10.3390/bs120 80253 · DOI
  63. Strachan, J. W., Albergo, D., Borghini, G., Pansardi, O., Scaliti, E., Gupta, S., et al. (2024). Testing theory of mind in large language models and humans. Nat. Hum. Behav. 8, 1285-1295. doi: 10.1038/s41562-024-01882-z · DOI · Paper page
  64. Street, W. (2024). LLM theory of mind and alignment: opportunities and risks. arXiv preprint arXiv:2405.08154. doi: 10.48550/arXiv.2405.08154 · DOI
  65. Street, W., Siy, J. O., Keeling, G., Baranes, A., Barnett, B., McKibben, M., et al. (2024). Llms achieve adult human performance on higher-order theory of mind tasks. arXiv preprint arXiv:2405.18870. doi: 10.48550/arXiv.2405.18870
  66. Sutton, J., Smith, P. K., and Swettenham, J. (1999a). Bullying and "theory of mind": a critique of the "social skills deficit" view of anti-social behaviour. Soc Dev. 8, 117-127. doi: 10.1111/1467-9507.00083 · DOI
  67. Sutton, J., Smith, P. K., and Swettenham, J. (1999b). Social cognition and bullying: social inadequacy or skilled manipulation? Br. J. Dev. Psychol. 17, 435-450. doi: 10.1348/026151099165384 · DOI
  68. Thoppilan, R., De Freitas, D., Hall, J., Shazeer, N., Kulshreshtha, A., Cheng, H.- T., et al. (2022). Lamda: language models for dialog applications. arXiv preprint arXiv:2201.08239. doi: 10.48550/arXiv.2201.08239 · DOI · Paper page
  69. Tversky, A., and Kahneman, D. (1974). Judgment under uncertainty: heuristics and biases: biases in judgments reveal some heuristics of thinking under uncertainty. Science 185, 1124-1131. doi: 10.1126/science.185.4157.1124 · DOI · Paper page
  70. Ullman, T. (2023). Large language models fail on trivial alterations to theory-of- mind tasks. arXiv preprint arXiv:2302.08399. doi: 10.48550/arXiv.2302.08399 · DOI · Paper page
  71. Van Duijn, M., Van Dijk, B., Kouwenhoven, T., De Valk, W., Spruit, M., and van der Putten, P. (2023). "Theory of mind in large language models: Examining performance of 11 state-of-the-art models vs. children aged 7-10 on advanced tests, " in Proceedings of the 27th Conference on Computational Natural Language Learning (CoNLL) (Singapore: Association for Computational Linguistics), 389-402. doi: 10.18653/v1/2023.conll-1. · DOI · Paper page
  72. van Neerven, T., Bos, D. J., and van Haren, N. E. (2021). Deficiencies in theory of mind in patients with schizophrenia, bipolar disorder, and major depressive disorder: a systematic review of secondary literature. Neurosci. Biobeh. Rev. 120, 249-261. doi: 10.1016/j.neubiorev.2020.11.011 · DOI
  73. Wang, L., Ma, C., Feng, X. et al. (2024). A survey on large language model based autonomous agents. Front. Comput. Sci. 18:186345. doi: 10.1007/s11704-024-40231-1 · DOI · Paper page
  74. Wellman, H. M., and Bartsch, K. (1988). Young children's reasoning about beliefs. Cognition 30, 239-277. doi: 10.1016/0010-0277(88)90021-2 · DOI
  75. Wellman, H. M., Cross, D., and Watson, J. (2001). Meta-analysis of theory- of-mind development: the truth about false belief. Child Dev. 72, 655-684. doi: 10.1111/1467-8624.00304 · DOI · Paper page
  76. Westby, C., and Robinson, L. (2014). A developmental perspective for promoting theory of mind. Top. Lang. Disord. 34, 362-382. doi: 10.1097/TLD.0000000000000035 · DOI · Paper page
  77. Wimmer, H., and Perner, J. (1983). Beliefs about beliefs: representation and constraining function of wrong beliefs in young children's understanding of deception. Cognition 13, 103-128. doi: 10.1016/0010-0277(83)90004-5 · DOI · Paper page
  78. Wu, Y., He, Y., Jia, Y., Mihalcea, R., Chen, Y., and Deng, N. (2023). "Hi-ToM: a benchmark for evaluating higher-order theory of mind reasoning in large language models, " in Findings of the Association for Computational Linguistics: EMNLP 2023, eds. H. Bouamor, J. Pino, and K. Bali (Singapore: Association for Computational Linguistics), 10691-10706. doi: 10.18653/v1/2023.findings-emnlp.717 · DOI
  79. Zhao, W. X., Zhou, K., Li, J., Tang, T., Wang, X., Hou, Y., et al. (2023). A survey of large language models. arXiv preprint arXiv:2303.18223. doi: 10.48550/arXiv.2303.18223 · DOI · Paper page
  80. Zhou, L., Moreno-Casares, P. A., Martínez-Plumed, F., Burden, J., Burnell, R., Cheke, L., et al. (2023). Predictable artificial intelligence. arXiv [Preprint]. arXiv:2310.06167. · arXiv
  81. Zhu, H., Neubig, G., and Bisk, Y. (2021). "Few-shot language coordination by modeling theory of mind, " in International Conference on Machine Learning (PMLR), 12901-12911.

Cited by

Sorted by citations

Authors
Street, W., Siy, J. O., Keeling, G., Baranes, A., Barnett, B., McKibben, M., Kanyere, T., Lentz, A., Arcas, B. A. Y., & Dunbar, R. I. M.
Affiliations
  • Google (United Kingdom) (GB)
  • Johns Hopkins University (US)
  • University of Oxford (GB)
Corresponding author
Street, Winnie
ORCID
Venue
arXiv
Publisher
arXiv
Published
2024
Volume
19
Pages
1633272-1633272
ISSN
1662-5161 (ISSN-L)
DOI
10.48550/arxiv.2405.18870
arXiv
2405.18870
PMC
PMC12808479
OpenAlex
W7117966739
Cited by
24
References
0
Citations by year
2026: 10 · 2025: 9 · 2024: 5
FWCI
140.07
Type
journal article
Language
en
Open access
Licence
Locations
  • Frontiers in Human Neuroscience (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • PubMed Central (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • arXiv (Cornell University) (cc-by)
Topics
  • Social Sciences › Psychology › Developmental and Educational Psychology › Child and Animal Learning Development
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Logic, Reasoning, and Knowledge
Keywords
large language models, recursive reasoning, mental state inference, GPT-4, model size, fine-tuning, social cognition, LLM applications

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review