Can AI language models replace human participants?
Abstract
Recent work suggests that language models such as GPT can make human-like judgments across a number of domains. We explore whether and when language models might replace human participants in psychological science. We review nascent research, provide a theoretical model, and outline caveats of using AI as a participant.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The more ambitious proposals of using AI to substitute for human participants or as surrogates for collective decision-making are clearly off the mark, at least right now.
“As a result, the more ambitious proposals of using AI to substitute for human participants or as surrogates for collective decision-making (Dillion et al., 2023) are clearly off the mark, at least right now.”
From Exploring the psychology of LLMs’ moral and legal reasoning (2024) · cites this paper as Dillion et al., 2023
LLMs can replicate classic experiments in cognitive science and the study of morality and replicate human behavior in the Prisoner's Dilemma and other behavioral games.
“Still other studies indicate LLMs can replicate classic experiments in cognitive science and the study of morality and replicate human behavior in the Prisoner’s Dilemma and other behavioral games (21–24).”
From Can Generative AI improve social science? (2024) · cites this paper as [22]
Cites
Out of One, Many: Using Language Models to Simulate Human Samples
Argyle, L. P., Busby, E. C., Fulda, N., Gubler, J. R., Rytting, C., & Wingate, D. (2023) · Political Analysis · doi:10.1017/pan.2023.2 · Cited by 783 · see citing works
(2019) · Cited by 771 · see citing works
Using cognitive psychology to understand GPT-3
Binz, M., & Schulz, E. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2218523120 · Cited by 559 · see citing works
Large Language Models as Simulated Economic Agents: What Can We Learn from Homo Silicus?
Horton, J., Filippas, A., & Manning, B. (2023) · National Bureau of Economic Research · doi:10.3386/w31122 · Cited by 329 · see citing works
Clifford, S., Iyengar, V., Cabeza, R., & Sinnott-Armstrong, W. (2015) · Behavior Research Methods · doi:10.3758/s13428-014-0551-2 · Cited by 328 · see citing works
Large pre-trained language models contain human-like biases of what is right and wrong to do
Schramowski, P., Turan, C., Andersen, N., Rothkopf, C. A., & Kersting, K. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00458-8 · Cited by 291 · see citing works
Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies
Aher, G., Arriaga, R. I., & Kalai, A. T. (2022) · arXiv · doi:10.48550/arxiv.2208.10264 · Cited by 161 · see citing works
Language Models and Cognitive Automation for Economic ResearchPreprint
Korinek, A. (2023) · Social Science Research Network · doi:10.2139/ssrn.4356243 · Cited by 129 · see citing works
Language Models and Cognitive Automation for Economic Research
Korinek, A. (2023) · doi:10.3386/w30957 · Cited by 129 · see citing works
Whose Opinions Do Language Models Reflect?
Santurkar, S., Durmus, E., Ladhak, F., Lee, C., Liang, P., & Hashimoto, T. (2023) · arXiv · doi:10.48550/arxiv.2303.17548 · Cited by 114 · see citing works
Grizzard, M., Matthews, N. L., Francemone, C. J., & Fitzgerald, K. S. (2021) · Human Communication Research · doi:10.1093/hcr/hqab011 · Cited by 41 · see citing works
The moral repetition effect: Bad deeds seem less unethical when repeatedly encountered.
Effron, D. A. (2022) · Journal of Experimental Psychology General · doi:10.1037/xge0001214 · Cited by 35 · see citing works
Cook, W. L., & Kuhn, K. M. (2020) · Journal of Business Ethics · doi:10.1007/s10551-020-04501-9 · Cited by 26 · see citing works
The Psychology of Robots and Artificial Intelligence
Gray, K. J., Yam, K. C., Eng, A., Dillion, D., & Waytz, A. G. (2025) · doi:10.70400/ikvm1257 · Cited by 15 · see citing works
Self-citationMickelberg, A. J., Walker, B., Ecker, U. K. H., Howe, P. D. L., Perfors, A., & Fay, N. (2022) · PLoS ONE · doi:10.1371/journal.pone.0269393 · Cited by 12 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Artificial intelligence and illusions of understanding in scientific research
Messeri, L., & Crockett, M. J. (2024) · Nature · doi:10.1038/s41586-024-07146-0 · Cited by 707 · see citing works
Can Generative AI improve social science?
Bail, C. A. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2314021121 · Cited by 369 · see citing works
GPT is an effective tool for multilingual psychological text analysis
Rathje, S., Mirea, D.-M., Sucholutsky, I., Marjieh, R., Robertson, C. E., & Van Bavel, J. J. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2308950121 · Cited by 297 · see citing works
Synthetic Replacements for Human Survey Data? The Perils of Large Language Models
Bisbee, J., Clinton, J. D., Dorff, C., Kenkel, B., & Larson, J. M. (2024) · Political Analysis · doi:10.1017/pan.2024.5 · Cited by 209 · see citing works
Generative AI in Higher Education: The ChatGPT Effect
Chan, C. K. Y., & Colloton, T. (2024) · Routledge · doi:10.4324/9781003459026 · Cited by 203 · see citing works
Atari, M., Xue, M. J., Park, P. S., Blasi, D. E., & Henrich, J. (2023) · doi:10.31234/osf.io/5b26t · Cited by 151 · see citing works
Sarstedt, M., Adler, S. J., Rau, L., & Schmitt, B. (2024) · Psychology & Marketing · doi:10.1002/mar.21982 · Cited by 149 · see citing works
AI for social science and social science of AI: A survey
Xu, R. F., Sun, Y., Ren, M., Guo, S., Pan, R., Lin, H., Sun, L., & Han, X. (2024) · Information Processing & Management · doi:10.1016/j.ipm.2024.103665 · Cited by 149 · see citing works
How large language models can reshape collective intelligence
Burton, J. W., Lopez-Lopez, E., Hechtlinger, S., Rahwan, Z., Aeschbach, S., Bakker, M. A., Becker, J. A., Berditchevskaia, A., Berger, J., Brinkmann, L. U. L., et al. (2024) · Nature Human Behaviour · doi:10.1038/s41562-024-01959-9 · Cited by 124 · see citing works
Wang, A., Morgenstern, J., & Dickerson, J. P. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-025-00986-z · Cited by 192 · see citing works
Frontiers: Can Large Language Models Capture Human Preferences?
Goli, A., & Singh, A. (2024) · Marketing Science · doi:10.1287/mksc.2023.0306 · Cited by 115 · see citing works
A foundation model to predict and capture human cognition
Binz, M., Akata, E., Bethge, M., Brändle, F., Callaway, F., Coda-Forno, J., Dayan, P., Demircan, C., Eckstein, M. K., Éltető, N., et al. (2025) · Nature · doi:10.1038/s41586-025-09215-4 · Cited by 119 · see citing works
Perils and opportunities in using large language models in psychological research
Abdurahman, S., Atari, M., Karimi-Malekabadi, F., Xue, M. J., Trager, J., Park, P. S., Golazizian, P., Omrani, A., & Dehghani, M. (2024) · PNAS Nexus · doi:10.1093/pnasnexus/pgae245 · Cited by 111 · see citing works
Reclaiming AI as a Theoretical Tool for Cognitive Science
van Rooij, I., Guest, O., Adolfi, F., de Haan, R., Kolokolova, A., & Rich, P. (2024) · Computational Brain & Behavior · doi:10.1007/s42113-024-00217-5 · Cited by 103 · see citing works
Generative AI in Marketing and Principles for Ethical Design and Deployment
Hermann, E., & Puntoni, S. (2024) · Journal of Public Policy & Marketing · doi:10.1177/07439156241309874 · Cited by 93 · see citing works
The Future of Marketing and Marketing Education
Grewal, D., Guha, A., Satornino, C. B., & Becker, M. (2024) · Journal of Marketing Education · doi:10.1177/02734753241269838 · Cited by 83 · see citing works
Do LLMs Exhibit Human-like Response Biases? A Case Study in Survey Design
Tjuatja, L., Chen, V., Wu, T., Talwalkwar, A., & Neubig, G. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00685 · Cited by 100 · see citing works
Performance and biases of large language models in public opinion simulation.
Qu, Y., & Wang, J. (2024) · Humanities and Social Sciences Communications · doi:10.1057/s41599-024-03609-x · Cited by 79 · see citing works
Diminished diversity-of-thought in a standard large language model
Park, P. S., Schoenegger, P., & Zhu, C. (2024) · Behavior Research Methods · doi:10.3758/s13428-023-02307-x · Cited by 83 · see citing works
How should the advancement of large language models affect the practice of science?
Binz, M., Alaniz, S., Roskies, A., Aczel, B., Bergstrom, C. T., Allen, C., Schad, D., Wulff, D., West, J. D., Zhang, Q., et al. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2401227121 · Cited by 74 · see citing works
Veselovsky, V., Ribeiro, M. H., & West, R. (2023) · arXiv · doi:10.48550/arxiv.2306.07899 · Cited by 73 · see citing works
“As previously mentioned, LLMs can act as effective proxies for human sub-populations (Argyle et al., 2022), leading to a series of studies using LLMs as “silicon samples” (Ar-gyle et al., 2022; Horton, 2023; Dillion et al., 2023).”
Friend or foe? Exploring the implications of large language models on the science system
Fecher, B., Hebing, M., Laufer, M., Pohle, J., & Sofsky, F. (2025) · AI & SOCIETY · doi:10.1007/s00146-023-01791-1 · Cited by 78 · see citing works
“They may even be used as a replacement for human participants in psychological experiments (Dillion et al. 2023).”
Exploring the psychology of LLMs’ moral and legal reasoning
Almeida, G. F. C. F., Nunes, J. L., Engelmann, N., Wiegmann, A., & Araújo, M. D. (2024) · Artificial Intelligence · doi:10.1016/j.artint.2024.104145 · Cited by 70 · see citing works
Do large language models have a legal duty to tell the truth?
Wachter, S., Mittelstadt, B., & Russell, C. (2024) · Royal Society Open Science · doi:10.1098/rsos.240197 · Cited by 85 · see citing works
Generative agent‐based modeling: an introduction and tutorial
Ghaffarzadegan, N., Majumdar, A., Williams, R., & Hosseinichimeh, N. (2024) · System Dynamics Review · doi:10.1002/sdr.1761 · Cited by 88 · see citing works
Large language models predict human sensory judgments across six modalities
Marjieh, R., Sucholutsky, I., van Rijn, P., Jacoby, N., & Griffiths, T. L. (2024) · Scientific Reports · doi:10.1038/s41598-024-72071-1 · Cited by 73 · see citing works
Exploring the frontiers of LLMs in psychological applications: a comprehensive review
Ke, L., Tong, S., Cheng, P., & Peng, K. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11297-5 · Cited by 92 · see citing works
Can Generative AI Improve Social Science?
Bail, C. A. (2023) · doi:10.31235/osf.io/rwtzs · Cited by 369 · see citing works
Lehr, S. A., Caliskan, A., Liyanage, S., & Banaji, M. R. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2404328121 · Cited by 66 · see citing works
Generative Artificial Intelligence in Qualitative Data Analysis: Analyzing—Or Just Chatting?
Nguyen, D. C., & Welch, C. (2025) · Organizational Research Methods · doi:10.1177/10944281251377154 · Cited by 70 · see citing works
Take caution in using LLMs as human surrogates
Gao, Y., Lee, D., Burtch, G., & Fazelpour, S. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2501660122 · Cited by 52 · see citing works
Knowledge of cultural moral norms in large language models
Ramezani, A., & Xu, Y. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.26 · Cited by 54 · see citing works
GPT and CLT: The impact of ChatGPT's level of abstraction on consumer recommendations
Kirshner, S. N. (2023) · Journal of Retailing and Consumer Services · doi:10.1016/j.jretconser.2023.103580 · Cited by 54 · see citing works
AI language models cannot replace human research participants
Harding, J., D’Alessandro, W., Laskowski, N. G., & Long, R. (2023) · AI & Society · doi:10.1007/s00146-023-01725-x · Cited by 53 · see citing works
Large language models show amplified cognitive biases in moral decision-making
Cheung, V., Maier, M., & Lieder, F. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2412015122 · Cited by 53 · see citing works
Automating the practice of science: Opportunities, challenges, and implications
Musslick, S., Bartlett, L., Chandramouli, S., Dubova, M., Gobet, F. R., Griffiths, T. L., Hullman, J. R., King, R. D., Kutz, J. N., Lucas, C. G., et al. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2401238121 · Cited by 56 · see citing works
AI language model rivals expert ethicist in perceived moral expertise
Dillion, D., Mondal, D., Tandon, N., & Gray, K. J. (2025) · Scientific Reports · doi:10.1038/s41598-025-86510-0 · Cited by 54 · see citing works
Self-citationAttributions toward artificial agents in a modified Moral Turing Test
Aharoni, E., Fernandes, S., Brady, D. J., Alexander, C., Criner, M., Queen, K., Rando, J., Nahmias, E., & Crespo, V. (2024) · Scientific Reports · doi:10.1038/s41598-024-58087-7 · Cited by 65 · see citing works
(Ir)rationality and cognitive biases in large language models
Macmillan-Scott, O., & Musolesi, M. (2024) · Royal Society Open Science · doi:10.1098/rsos.240255 · Cited by 55 · see citing works
The Challenge of Using LLMs to Simulate Human Behavior: A Causal Inference PerspectivePreprint
Gui, G., & Toubia, O. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4650172 · Cited by 48 · see citing works
Systematic Biases in LLM Simulations of Debates
Taubenfeld, A., Dover, Y., Reichart, R., & Goldstein, A. (2024) · Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2024.emnlp-main.16 · Cited by 47 · see citing works
Salah, M., Abdelfattah, F., & Halbusi, H. A. (2024) · Current Opinion in Psychology · doi:10.1016/j.copsyc.2024.101872 · Cited by 47 · see citing works
Large language models display human-like social desirability biases in Big Five personality surveys
Salecha, A., Ireland, M. E., Subrahmanya, S., Sedoc, J., Ungar, L. H., & Eichstaedt, J. C. (2024) · PNAS Nexus · doi:10.1093/pnasnexus/pgae533 · Cited by 55 · see citing works
A Reasoning and Value Alignment Test to Assess Advanced GPT Reasoning
Mcintosh, T. R., Liu, T., Susnjak, T., Watters, P., & Halgamuge, M. N. (2024) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3670691 · Cited by 42 · see citing works
Can large language models help augment English psycholinguistic datasets?
Trott, S. (2024) · Behavior Research Methods · doi:10.3758/s13428-024-02337-z · Cited by 45 · see citing works
Cognitive Biases and Artificial Intelligence
Wang, J., & Redelmeier, D. A. (2024) · NEJM AI · doi:10.1056/aics2400639 · Cited by 43 · see citing works
Cui, Z., Li, N., & Zhou, H. (2025) · Nature Computational Science · doi:10.1038/s43588-025-00840-7 · Cited by 46 · see citing works
The homogenizing effect of large language models on human expression and thought
Sourati, Z., Ziabari, A. S., & Dehghani, M. (2026) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2026.01.003 · Cited by 62 · see citing works
Start Generating: Harnessing Generative Artificial Intelligence for Sociological Research
Davidson, T. (2024) · Socius Sociological Research for a Dynamic World · doi:10.1177/23780231241259651 · Cited by 51 · see citing works
Studying and improving reasoning in humans and machines
Yax, N., Anlló, H., & Palminteri, S. (2024) · Communications Psychology · doi:10.1038/s44271-024-00091-8 · Cited by 40 · see citing works
A Primer for Evaluating Large Language Models in Social-Science Research
Abdurahman, S., Ziabari, A. S., Moore, A. K., Bartels, D. M., & Dehghani, M. (2025) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459251325174 · Cited by 51 · see citing works
Prompt architecture induces methodological artifacts in large language models
Brucks, M. S., & Toubia, O. (2025) · PLoS ONE · doi:10.1371/journal.pone.0319159 · Cited by 42 · see citing works
Synthetic Replacements for Human Survey Data? The Perils of Large Language Models
Bisbee, J., Clinton, J. D., Dorff, C., Kenkel, B., & Larson, J. (2023) · doi:10.31235/osf.io/5ecfa · Cited by 213 · see citing works
Toubia, O., Gui, G., Peng, T., Merlau, D. J., Li, A., & Chen, H. (2025) · Marketing Science · doi:10.1287/mksc.2025.0262 · Cited by 39 · see citing works
Can AI serve as a substitute for human subjects in software engineering research?
Gerosa, M. A., Trinkenreich, B., Steinmacher, I., & Sarma, A. (2024) · Automated Software Engineering · doi:10.1007/s10515-023-00409-6 · Cited by 35 · see citing works
Bian, Y., Küster, D., Liu, H., & Krumhuber, E. G. (2023) · Sensors · doi:10.3390/s24010126 · Cited by 34 · see citing works
More or less than human? Evaluating the role of AI-as-participant in online qualitative research
Gibson, A. F., & Beattie, A. (2024) · Qualitative Research in Psychology · doi:10.1080/14780887.2024.2311427 · Cited by 31 · see citing works
Generative AI lacks the human creativity to achieve scientific discovery from scratch
Ding, A. W., & Li, S. (2025) · Scientific Reports · doi:10.1038/s41598-025-93794-9 · Cited by 31 · see citing works
From text to map: a system dynamics bot for constructing causal loop diagrams
Hosseinichimeh, N., Majumdar, A., Williams, R., & Ghaffarzadegan, N. (2024) · System Dynamics Review · doi:10.1002/sdr.1782 · Cited by 30 · see citing works
Human-like object concept representations emerge naturally in multimodal large language models
Du, C., Fu, K., Wen, B., Sun, Y., Peng, J., Wei, W., Gao, Y., Wang, S., Zhang, C., Li, J., et al. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-025-01049-z · Cited by 30 · see citing works
Understanding AI-Generated Experiments in Tourism: Replications Using GPT Simulations
Xiong, X., Wong, I. A., Huang, G. I., & Peng, Y. (2024) · Journal of Travel Research · doi:10.1177/00472875241275945 · Cited by 30 · see citing works
Large Language Models based on historical text could offer informative tools for behavioral science
Varnum, M. E. W., Baumard, N., Atari, M., & Gray, K. J. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2407639121 · Cited by 30 · see citing works
Self-citationAmirova, A., Fteropoulli, T., Ahmed, N., Cowie, M., & Leibo, J. Z. (2024) · PLoS ONE · doi:10.1371/journal.pone.0300024 · Cited by 47 · see citing works
Llama 2: Early Adopters' Utilization of Meta's New Open-Source Pretrained ModelPreprint
Roumeliotis, K. I., Tselikas, N. D., & Nasiopoulos, D. Κ. (2023) · Preprints.org · doi:10.20944/preprints202307.2142.v2 · Cited by 29 · see citing works
Using cognitive psychology to understand GPT-like models needs to extend beyond human biases
Stella, M., Hills, T. T., & Kenett, Y. N. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2312911120 · Cited by 27 · see citing works
Multi-Agent Collaboration Mechanisms: A Survey of LLMs
Tran, K.-T., Dao, D., Nguyen, M.-D., Pham, Q.-V., O'Sullivan, B., & Nguyen, H. D. (2025) · arXiv · doi:10.48550/arxiv.2501.06322 · Cited by 26 · see citing works
“Authors in [5, 36] suggest that LLM-based agents can replace human participants in specific social science experiments, while [88] demonstrates their use in generating diverse natural language data with iterative cross-agent refinement.”
van Duijn, M., van Dijk, B., Kouwenhoven, T., de Valk, W., Spruit, M., & vanderPutten, P. (2023) · Proceedings of the 27th Conference on Computational Natural Language Learning (CoNLL) · doi:10.18653/v1/2023.conll-1.25 · Cited by 24 · see citing works
The use of synthetic data in tourism
Viglia, G., Adler, S. J., Miltgen, C. L., & Sarstedt, M. (2024) · Annals of Tourism Research · doi:10.1016/j.annals.2024.103819 · Cited by 23 · see citing works
Automatic recognition of second language speech-in-noise
Kim, S.-E., Chernyak, B. R., Seleznova, O., Keshet, J., Goldrick, M., & Bradlow, A. R. (2024) · JASA Express Letters · doi:10.1121/10.0024877 · Cited by 24 · see citing works
Influence of External Information on Large Language Models Mirrors Social Cognitive Patterns
Bian, N., Lin, H., Liu, P., Lu, Y., Zhang, C.-P., He, B., Han, X., & Sun, L. (2024) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2024.3476030 · Cited by 36 · see citing works
Eng, P. H., & Liu, R. L. (2024) · Studies in computational intelligence · doi:10.1007/978-3-031-61463-7_9 · Cited by 25 · see citing works
Machine Bias. How Do Generative Language Models Answer Opinion Polls?
Boelaert, J., Coavoux, S., Ollion, É., Petev, I. D., & Präg, P. (2025) · Sociological Methods & Research · doi:10.1177/00491241251330582 · Cited by 30 · see citing works
Wang, Z.-Y., Wang, D.-Q., Xu, Y., Zhou, L.-F., & Zhou, Y.-Q. (2024) · Journal of Chinese Political Science · doi:10.1007/s11366-025-09917-6 · Cited by 23 · see citing works
An integrated model of semantics and control.
Giallanza, T., Campbell, D. I., Cohen, J., & Rogers, T. T. (2024) · Psychological Review · doi:10.1037/rev0000485 · Cited by 24 · see citing works
Brysbaert, M., Martínez, G., & Reviriego, P. (2024) · Behavior Research Methods · doi:10.3758/s13428-024-02561-7 · Cited by 22 · see citing works
Using Large Language Models for Qualitative Analysis can Introduce Serious Bias
Ashwin, J., Chhabra, A., & Rao, V. (2026) · Sociological Methods & Research · doi:10.1177/00491241251338246 · Cited by 22 · see citing works
Beyond WEIRD: Can synthetic survey participants substitute for humans in global policy research?
Shrestha, P., Krpan, D., Koaik, F., Schnider, R., Sayess, D., & Binbaz, M. S. (2024) · Behavioral Science & Policy · doi:10.1177/23794607241311793 · Cited by 21 · see citing works
Moral stereotyping in large language models
Zewail, A., Figueroa, A., Graham, J., & Atari, M. (2026) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2519941123 · Cited by 23 · see citing works
Large Language Models and the Wisdom of Small Crowds
Trott, S. (2024) · Open Mind · doi:10.1162/opmi_a_00144 · Cited by 22 · see citing works
Bojić, L., Cinelli, M., Culibrk, D. R., & Delibašić, B. (2024) · European Journal of Futures Research · doi:10.1186/s40309-024-00238-0 · Cited by 21 · see citing works
Llama 2: Early Adopters' Utilization of Meta's New Open-Source Pretrained ModelPreprint
Roumeliotis, K. I., Tselikas, N. D., & Nasiopoulos, D. Κ. (2023) · Preprints.org · doi:10.20944/preprints202307.2142.v1 · Cited by 21 · see citing works
Wang, K., Mirosa, M., Hou, Y., & Bremer, P. J. (2025) · Trends in Food Science & Technology · doi:10.1016/j.tifs.2025.105050 · Cited by 20 · see citing works
A Survey of LLM Datasets: From Autoregressive Model to AI Chatbot
Du, F., Ma, X.-J., Yang, J., Liu, Y., Luo, C.-R., Wang, X., Jiang, H.-O., & Jing, X. (2024) · Journal of Computer Science and Technology · doi:10.1007/s11390-024-3767-3 · Cited by 22 · see citing works
Six Fallacies in Substituting Large Language Models for Human Participants
Lin, Z. (2025) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459251357566 · Cited by 20 · see citing works
Huang, G. I., Zhang, C., Torres, W. C. Z., Xiong, X., & Li, H. (2025) · Tourism Management · doi:10.1016/j.tourman.2025.105315 · Cited by 19 · see citing works
Large language models can predict the results of social science experiments
Ashokkumar, A., Hewitt, L., Ghezae, I., & Willer, R. (2026) · Nature · doi:10.1038/s41586-026-10742-x · Cited by 25 · see citing works
Studying large language models as compression algorithms for human culture
Buttrick, N. R. (2024) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2024.01.001 · Cited by 19 · see citing works
Mobilizing New Sources of Data: Opportunities and Recommendations
Grégoire, D. A., Wal, A. L. J. T., Little, L. M., Bermiss, Y. S., Kotha, R. R., & Gruber, M. B. (2024) · Academy of Management Journal · doi:10.5465/amj.2024.4002 · Cited by 19 · see citing works
Large language models (LLMs) as agents for augmented democracy
Gudiño, J., Grandi, U., & Hidalgo, C. A. (2024) · Philosophical Transactions of the Royal Society A Mathematical Physical and Engineering Sciences · doi:10.1098/rsta.2024.0100 · Cited by 19 · see citing works
Stylos, N., Okumus, F., & Onder, I. (2025) · Tourism Management · doi:10.1016/j.tourman.2025.105205 · Cited by 18 · see citing works
The Mixed Subjects Design: Treating Large Language Models as Potentially Informative Observations
Broska, D., Howes, M., & van Loon, A. (2025) · Sociological Methods & Research · doi:10.1177/00491241251326865 · Cited by 18 · see citing works
Integration of cognitive tasks into artificial general intelligence test for large models
Qu, Y., Wei, C., Du, P., Che, W., Zhang, C., Ouyang, W., Bian, Y., Xu, F., Hu, B., Du, K., et al. (2024) · iScience · doi:10.1016/j.isci.2024.109550 · Cited by 18 · see citing works
Martínez, G., Conde, J., Reviriego, P., & Brysbaert, M. (2024) · Quarterly Journal of Experimental Psychology · doi:10.1177/17470218241306694 · Cited by 18 · see citing works
Bickley, S. J., Chan, H. F., Dao, B., Torgler, B., Tran, S., & Zimbatu, A. (2024) · Journal of Services Marketing · doi:10.1108/jsm-11-2023-0441 · Cited by 17 · see citing works
Recommending With, Not For: Co-Designing Recommender Systems for Social Good
Ekstrand, M. D., Razi, A., Sarcevic, A., Pera, M. S., Burke, R., & Wright, K. L. (2025) · ACM Transactions on Recommender Systems · doi:10.1145/3759261 · Cited by 17 · see citing works
Demonstrations of the Potential of AI-based Political Issue Polling
Sanders, N. E., Ulinich, A., & Schneier, B. (2023) · Harvard Data Science Review · doi:10.1162/99608f92.1d3cf75d · Cited by 25 · see citing works
Artificial intelligence chatbots mimic human collective behaviour
He, J. K., Wallis, F. P. S., Gvirtz, A., & Rathje, S. (2024) · British Journal of Psychology · doi:10.1111/bjop.12764 · Cited by 34 · see citing works
Theorizing with Large Language Models
Tranchero, M., Brenninkmeijer, C.-F., Murugan, A., & Nagaraj, A. (2024) · National Bureau of Economic Research · doi:10.3386/w33033 · Cited by 17 · see citing works
Li, C., & Qi, Y. (2025) · Computers in Human Behavior · doi:10.1016/j.chb.2025.108687 · Cited by 16 · see citing works
The impact of ChatGPT on human data collection: A case study involving typicality norming data
Heyman, T., & Heyman, G. (2023) · Behavior Research Methods · doi:10.3758/s13428-023-02235-w · Cited by 16 · see citing works
- Authors
- Dillion, D., Tandon, N., Gu, Y., & Gray, K.
- Affiliations
- University of North Carolina at Chapel Hill (US)
- Allen Institute for Artificial Intelligence (US)
- Corresponding author
- Gray, Kurt
- Venue
- Trends in Cognitive Sciences
- Publisher
- Elsevier BV
- Published
- 2023-07
- Volume
- 27
- Issue
- 7
- Pages
- 597-600
- ISSN
- 1364-6613 (ISSN-L)
- OpenAlex
- W4376117416
- Cited by
- 405
- References
- 14
- Citations by year
- 2026: 112 · 2025: 147 · 2024: 120 · 2023: 26
- FWCI
- 101.34
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- Trends in Cognitive Sciences (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
- Social Sciences › General Social Sciences › Computational and Text Analysis Methods
- Keywords
- AI language models, GPT
- MeSH terms
- Artificial Intelligence, Humans, Judgment, Language
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review