AI and the transformation of social science research
Abstract
Careful bias management and data fidelity are key
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Researchers have noted GPT's ability to help with computational social science tasks, detect misinformation, infer politicians' ideologies, write persuasive political arguments, respond to patient questions, simulate human research participants, and model collective behavior.
“Researchers have also recently noted GPT’s ability to help with computational social science tasks (49–54), detect misinformation (55), infer politicians’ ideologies (56), write persuasive political arguments (57), respond to patient questions (58), simulate human research participants (59–61), and model collective behavior (56).”
From GPT is an effective tool for multilingual psychological text analysis (2024) · cites this paper as [53]
LLMs may be unwilling to impersonate biased groups because they have been fine-tuned according to the normative preferences of highly educated liberals, who may have more concern about protecting marginalized groups than others.
“Researchers who want to use LLMs to impersonate biased groups, for example, may discover these tools are unwilling to perform such roles because they have been fine-tuned according to the normative preferences of highly educated liberals who may have more concern about the protection of marginalized groups than others (12, 64).”
From Can Generative AI improve social science? (2024) · cites this paper as [12]
Guard-rails in Generative AI probably improve safety for public use but may impede social scientists' ability to leverage bias for research purposes.
“Though such guard-rails probably improve the safety of Generative AI tools for public use, they may impede the ability of social scientists to leverage bias for research purposes (12).”
From Can Generative AI improve social science? (2024) · cites this paper as [12]
Generative AI may help increase external validity but may come at the cost of internal validity-the capacity of different groups to reproduce or replicate each other's results.
“Though Generative AI may help researchers increase the external validity of their research designs, this may come at the cost of internal validity—or the capacity of different groups of researchers to reproduce or replicate each other’s results (2, 12).”
From Can Generative AI improve social science? (2024) · cites this paper as [12]
Cites
Pennycook, G., McPhetres, J., Zhang, Y., Lu, J. G., & Rand, D. G. (2020) · Psychological Science · doi:10.1177/0956797620939054 · Cited by 1,715 · see citing works
Taxonomy of Risks posed by Language Models
Weidinger, L., Uesato, J., Rauh, M., Griffin, C., Huang, P.-S., Mellor, J., Glaese, A., Cheng, M., Balle, B., Kasirzadeh, A., et al. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533088 · Cited by 764 · see citing works
The Science of Wisdom in a Polarized World: Knowns and Unknowns
Grossmann, I., Weststrate, N. M., Ardelt, M., Brienza, J. P., Dong, M., Ferrari, M., Fournier, M. A., Hu, C. S., Nusbaum, H. C., & Vervaeke, J. (2020) · Psychological Inquiry · doi:10.1080/1047840x.2020.1750917 · Cited by 330 · see citing works
Self-citationLocally noisy autonomous agents improve global human coordination in network experiments
Shirado, H., & Christakis, N. A. (2017) · Nature · doi:10.1038/nature22332 · Cited by 288 · see citing works
Self-citationAlgorithmic bias: Senses, sources, solutions
Fazelpour, S., & Danks, D. J. (2021) · Philosophy Compass · doi:10.1111/phc3.12760 · Cited by 274 · see citing works
Using large-scale experiments and machine learning to discover theories of human decision-making
Peterson, J. C., Bourgin, D., Agrawal, M., Reichman, D., & Griffiths, T. L. (2021) · Science · doi:10.1126/science.abe2629 · Cited by 251 · see citing works
Large language models associate Muslims with violence
Abid, A., Farooqi, M., & Zou, J. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00359-2 · Cited by 193 · see citing works
Integrating social and cognitive aspects of belief dynamics: towards a unifying framework
Galesic, M., Olsson, H., Dalege, J., van der Does, T., & Stein, D. (2020) · Journal of the Royal Society Interface · doi:10.1098/rsif.2020.0857 · Cited by 92 · see citing works
Psychological Perspectives on Nuclear Deterrence
Tetlock, P. E., McGuire, C., & Mitchell, G. (1991) · Annual Review of Psychology · doi:10.1146/annurev.ps.42.020191.001323 · Cited by 34 · see citing works
Self-citationAntosz, P., Bharwani, S., Borit, M., & Edmonds, B. (2022) · International Journal of Social Research Methodology · doi:10.1080/13645579.2022.2052651 · Cited by 12 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
A survey on large language model based autonomous agents
Wang, L., Ma, C., Feng, X., Zhang, Z., Yang, H., Zhang, J., Chen, Z., Tang, J., Chen, X., Lin, Y., et al. (2024) · Frontiers of Computer Science · doi:10.1007/s11704-024-40231-1 · Cited by 1,941 · see citing works
“ChemCrow [75], Boiko et al. [115], Grossmann et al. [122] Natural Science Education ChemCrow [75], CodeHelp [120], Boiko et al. [115], MathA-gent [117], Drori et al. [118] Engineering”
BackgroundArtificial intelligence and illusions of understanding in scientific research
Messeri, L., & Crockett, M. J. (2024) · Nature · doi:10.1038/s41586-024-07146-0 · Cited by 707 · see citing works
“…including (but not limited to) race, ethnicity, gender, class, disability and/or relevant experiences such as background, training and career stage. experiences and perspectives ... offering a more accurate portrayal of human behaviour and social dynamics than those from conventional methods” 4 .”
InfluentialThe rise and potential of large language model based agents: a survey
Xi, Z., Chen, W., Guo, X., He, W., Ding, Y., Hong, B., Zhang, M., Wang, J., Jin, S., Zhou, E., et al. (2025) · Science China Information Sciences · doi:10.1007/s11432-024-4222-0 · Cited by 970 · see citing works
“Recent research on simulated societies has followed two primary lines, namely, exploring the boundaries of the collective intelligence capabilities of LLM-based agents [90, 115, 327, 330, 332] and using them to accelerate discoveries in the social sciences [16,438,439].”
Durably reducing conspiracy beliefs through dialogues with AI
Costello, T. H., Pennycook, G., & Rand, D. G. (2024) · Science · doi:10.1126/science.adq1814 · Cited by 354 · see citing works
“Moreover, ouruseofGPT-4Turbo,afrontier,closed-source,pretrained,andfine-tunedlanguage model, presents challenges related to inter-pretability and replicability ( 69 – 71 ).”
Can Generative AI improve social science?
Bail, C. A. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2314021121 · Cited by 369 · see citing works
GPT is an effective tool for multilingual psychological text analysis
Rathje, S., Mirea, D.-M., Sucholutsky, I., Marjieh, R., Robertson, C. E., & Van Bavel, J. J. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2308950121 · Cited by 297 · see citing works
Atari, M., Xue, M. J., Park, P. S., Blasi, D. E., & Henrich, J. (2023) · doi:10.31234/osf.io/5b26t · Cited by 151 · see citing works
GPT is an effective tool for multilingual psychological text analysisPreprint
Rathje, S., Mirea, D.-M., Sucholutsky, I., Marjieh, R., Robertson, C., & Bavel, J. J. V. (2023) · doi:10.31234/osf.io/sekf5 · Cited by 129 · see citing works
E-waste challenges of generative artificial intelligence
Wang, P., Zhang, L., Tzachor, A., Masanet, E. R., & Chen, W. (2024) · Nature Computational Science · doi:10.1038/s43588-024-00712-6 · Cited by 122 · see citing works
E-waste Challenges of Generative Artificial Intelligence
Wang, P., Zhang, L.-Y., Tzachor, A., Masanet, E., & Chen, W.-Q. (2024) · doi:10.21203/rs.3.rs-3978528/v1 · Cited by 122 · see citing works
How large language models can reshape collective intelligence
Burton, J. W., Lopez-Lopez, E., Hechtlinger, S., Rahwan, Z., Aeschbach, S., Bakker, M. A., Becker, J. A., Berditchevskaia, A., Berger, J., Brinkmann, L. U. L., et al. (2024) · Nature Human Behaviour · doi:10.1038/s41562-024-01959-9 · Cited by 124 · see citing works
“181,182), LLMs are uniquely positioned to address these logistical issues and support CI research.”
Wang, A., Morgenstern, J., & Dickerson, J. P. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-025-00986-z · Cited by 192 · see citing works
“Prior work has speculated that LLMs’ vast training data enable it to perform such representation 8 .”
Perils and opportunities in using large language models in psychological research
Abdurahman, S., Atari, M., Karimi-Malekabadi, F., Xue, M. J., Trager, J., Park, P. S., Golazizian, P., Omrani, A., & Dehghani, M. (2024) · PNAS Nexus · doi:10.1093/pnasnexus/pgae245 · Cited by 111 · see citing works
Jiang, P., Sonne, C., Li, W., You, F., & You, S. (2024) · Engineering · doi:10.1016/j.eng.2024.04.002 · Cited by 109 · see citing works
LLMs for science: Usage for code generation and data analysis
Nejjar, M., Zacharias, L., Stiehle, F., & Weber, I. M. (2024) · Journal of Software Evolution and Process · doi:10.1002/smr.2723 · Cited by 95 · see citing works
“For example, Grossmann et al. 5 share their perspective on LLM’s transformative use for social science research practices and Li et al. 4 comment on the ethical concerns related to their use in medicine and medical research.”
LLMs and generative agent-based models for complex systems research
Lu, Y., Aleta, A., Du, C., Shi, L., & Moreno, Y. (2024) · Physics of Life Reviews · doi:10.1016/j.plrev.2024.10.013 · Cited by 82 · see citing works
Quantifying the use and potential benefits of artificial intelligence in scientific research
Gao, J., & Wang, D. (2024) · Nature Human Behaviour · doi:10.1038/s41562-024-02020-5 · Cited by 81 · see citing works
Cultural tendencies in generative AI
Lu, J. G., Song, L. L., & Zhang, L. D. (2025) · Nature Human Behaviour · doi:10.1038/s41562-025-02242-1 · Cited by 79 · see citing works
“In this way, generative AI can serve as a barometer of cultural tendencies in the world 34,44 , offering an additional methodological lens for studying culture.”
Diminished diversity-of-thought in a standard large language model
Park, P. S., Schoenegger, P., & Zhu, C. (2024) · Behavior Research Methods · doi:10.3758/s13428-023-02307-x · Cited by 83 · see citing works
“One way that LLMs have been studied before in the social sciences is by studying them with the methods of psychology as if they were human participants, and potentially even as “surrogates” (Grossmann et al., 2023, p. 1108) that directly supplant human participants.”
BackgroundHow should the advancement of large language models affect the practice of science?
Binz, M., Alaniz, S., Roskies, A., Aczel, B., Bergstrom, C. T., Allen, C., Schad, D., Wulff, D., West, J. D., Zhang, Q., et al. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2401227121 · Cited by 74 · see citing works
Generative agent‐based modeling: an introduction and tutorial
Ghaffarzadegan, N., Majumdar, A., Williams, R., & Hosseinichimeh, N. (2024) · System Dynamics Review · doi:10.1002/sdr.1761 · Cited by 88 · see citing works
“The opportunity mainly comes from the observation that some of the best performing LLMs (such as GPT models, and their renowned interface of ChatGPT) depict humanlike behavior when responding to user questions (Grossmann et al., 2023; Ziems et al., 2023).”
Envisioning Information Access Systems: What Makes for Good Tools and a Healthy Web?
Shah, C. S., & Bender, E. M. (2024) · ACM Transactions on the Web · doi:10.1145/3649468 · Cited by 72 · see citing works
Exploring the frontiers of LLMs in psychological applications: a comprehensive review
Ke, L., Tong, S., Cheng, P., & Peng, K. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11297-5 · Cited by 92 · see citing works
Lehr, S. A., Caliskan, A., Liyanage, S., & Banaji, M. R. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2404328121 · Cited by 66 · see citing works
“Several recent articles suggest, for example, that LLMs can mimic responses from human research subjects (34–36), with some even suggesting they may significantly supplant them (37).”
Large language models reduce public knowledge sharing on online Q&A platforms
del Rio-Chanona, R. M., Laurentsyeva, N., & Wachs, J. (2024) · PNAS Nexus · doi:10.1093/pnasnexus/pgae400 · Cited by 87 · see citing works
“poised to change how we do research (Grossmann et al., 2023) and present a strong competitor to search engines (Xu et al.”
Are Large Language Models a Threat to Digital Public Goods? Evidence from Activity on Stack Overflow
del Rio-Chanona, M., Laurentsyeva, N., & Wachs, J. (2023) · arXiv · doi:10.48550/arxiv.2307.07367 · Cited by 62 · see citing works
“poised to change how we do research (Grossmann et al., 2023) and present a strong competitor to search engines (Xu et al.”
Artificial Intelligence (AI)-driven approach to climate action and sustainable development
Cho, H., & Ackom, E. K. (2025) · Nature Communications · doi:10.1038/s41467-024-53956-1 · Cited by 56 · see citing works
“Advances in AI have substantially affected social science research and revolutionized how we understand and interpret human behavior, society, and culture 22 – 24 .”
Take caution in using LLMs as human surrogates
Gao, Y., Lee, D., Burtch, G., & Fazelpour, S. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2501660122 · Cited by 52 · see citing works
Tang, Q., Deng, W., Huang, Y., Wang, S., & Zhang, H. (2025) · Journal of Computer Assisted Learning · doi:10.1111/jcal.70027 · Cited by 55 · see citing works
Large language models show amplified cognitive biases in moral decision-making
Cheung, V., Maier, M., & Lieder, F. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2412015122 · Cited by 62 · see citing works
Large Language Models in Qualitative Research: Uses, Tensions, and Intentions
Schroeder, H., Quéré, M. A. L., Randazzo, C., Mimno, D., & Schoenebeck, S. Y. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713120 · Cited by 51 · see citing works
Large Language Models for Text Classification: From Zero-Shot Learning to Instruction-Tuning
Chae, Y., & Davidson, T. (2025) · Sociological Methods & Research · doi:10.1177/00491241251325243 · Cited by 57 · see citing works
AirGPT: Pioneering the convergence of conversational AI with atmospheric science
Song, J., Ma, C., & Ran, M. (2025) · npj Climate and Atmospheric Science · doi:10.1038/s41612-025-01070-4 · Cited by 48 · see citing works
“LLMs have found widespread adoptionin fi eldssuchas fi nanceandsocialscienceanalytics,wheretheyare employedtoextractinformationandinformdecision-makingprocesses 5,9,10 .”
Large language models can outperform humans in social situational judgments
Mittelstädt, J. M., Maier, J., Goerke, P., Zinn, F. D., & Hermes, M. (2024) · Scientific Reports · doi:10.1038/s41598-024-79048-0 · Cited by 48 · see citing works
“Given that the training datasets may originate from mostly English-speaking sources or from a disproportionately Western cultural background 51,52 and the fact that social norms vary across cultures, the question arises as to whether LLMs contain cultural bias in terms of social judgment 53 .”
BackgroundVisualizing the knowledge mapping of artificial intelligence in education: A systematic review
Qin, Q., & Zhang, S. (2024) · Education and Information Technologies · doi:10.1007/s10639-024-13076-1 · Cited by 53 · see citing works
“The social sciences could utilize AI in data analysis, modeling human behavior, and predicting social trends (Bircan & Salah, 2022; Grossmann et al., 2023).”
Survey of Cultural Awareness in Language Models: Text and Beyond
Pawar, S., Park, J., Jin, J., Arora, A., Myung, J., Yadav, S., Haznitrama, F. G., Song, I., Oh, A. H., & Augenstein, I. (2025) · Computational Linguistics · doi:10.1162/coli.a.14 · Cited by 77 · see citing works
Can ChatGPT Perform a Grounded Theory Approach to Do Risk Analysis? An Empirical Study
Zhou, Y., Yuan, Y., Huang, K., & Hu, X. (2024) · Journal of Management Information Systems · doi:10.1080/07421222.2024.2415772 · Cited by 43 · see citing works
“Due to the absence of existing validation methods in GT, further research is needed to explore more advanced validation methods [52, 80] and the possible collaboration between human and AI [96, 99] to improve the performance of our approach.”
The use of ChatGPT for personality research: Administering questionnaires using generated personas
de Winter, J., Driessen, T., & Dodou, D. (2024) · Personality and Individual Differences · doi:10.1016/j.paid.2024.112729 · Cited by 49 · see citing works
Start Generating: Harnessing Generative Artificial Intelligence for Sociological Research
Davidson, T. (2024) · Socius Sociological Research for a Dynamic World · doi:10.1177/23780231241259651 · Cited by 51 · see citing works
“When used with care, these technologies can foster methodological innovation across the social sciences and spur the formation of new sub-fields and specialties (Grossmann et al. 2023).”
InfluentialGenerative artificial intelligence–mediated confirmation bias in health information seeking
Lopez‐Lopez, E., Abels, C. M., Holford, D., Herzog, S. M., & Lewandowsky, S. (2025) · Annals of the New York Academy of Sciences · doi:10.1111/nyas.15413 · Cited by 38 · see citing works
“47 In domains like healthcare, 48,49 behavioral and social sciences, 23,50,51 and collective intelligence, 52 these models are being used to refine research, access information, enhance decision-making, and support complex problem-solving.”
Küçükuncular, A., & Ertugan, A. (2025) · Sustainability · doi:10.3390/su17167405 · Cited by 36 · see citing works
Leveraging large language model to generate a novel metaheuristic algorithm with CRISPE framework
Zhong, R., Xu, Y., Zhang, C., & Yu, J. (2024) · Cluster Computing · doi:10.1007/s10586-024-04654-6 · Cited by 35 · see citing works
“The emergence of LLM has swiftly created a wave of interest in interdisciplinary research, making ’AI for science’ a prominent term in contemporary discussions [36, 37].”
How developments in natural language processing help us in understanding human behaviour
Mihalcea, R. F., Biester, L., Boyd, R. L., Jin, Z., Pérez‐Rosas, V., Wilson, S. R., & Pennebaker, J. W. (2024) · Nature Human Behaviour · doi:10.1038/s41562-024-01938-0 · Cited by 34 · see citing works
“We are witnessing a paradigm shift in the computer and social sciences that is reverberating across societies all over the world 196 .”
Davidson, T., & Karell, D. (2025) · Sociological Methods & Research · doi:10.1177/00491241251339184 · Cited by 34 · see citing works
LLM-Augmented Agent-Based Modelling for Social Simulations: Challenges and Opportunities
Gürcan, Ö. (2024) · Frontiers in artificial intelligence and applications · doi:10.3233/faia240190 · Cited by 41 · see citing works
Understanding AI-Generated Experiments in Tourism: Replications Using GPT Simulations
Xiong, X., Wong, I. A., Huang, G. I., & Peng, Y. (2024) · Journal of Travel Research · doi:10.1177/00472875241275945 · Cited by 30 · see citing works
“This position is in line with Grossmann et al. (2023), who contend that simulated participants have the potential for academic research while results ‘‘could then be confirmed in human popula-tions’’ (p. 1,108).”
InfluentialLarge Language Models based on historical text could offer informative tools for behavioral science
Varnum, M. E. W., Baumard, N., Atari, M., & Gray, K. J. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2407639121 · Cited by 30 · see citing works
Yoo, K., Haenlein, M., & Hewett, K. (2025) · Journal of the Academy of Marketing Science · doi:10.1007/s11747-025-01097-2 · Cited by 31 · see citing works
“Lastly, ethical considerations are critical when utilizing LMMs for consumer research and offer a rich area for future research (Benmamoun, 2024; Grossmann et al., 2023; Hermann, 2022).”
Amirova, A., Fteropoulli, T., Ahmed, N., Cowie, M., & Leibo, J. Z. (2024) · PLoS ONE · doi:10.1371/journal.pone.0300024 · Cited by 47 · see citing works
“Large-Scale generative Language Models (LLMs) [1–3] may provide a new opportunity for capturing available knowledge and beliefs at scale and facilitate in silico research on human behavior and cognition [4–8].”
Li, Y., Fan, J., Cheng, X., Jin, H., Wang, Y., Wei, F., An, F., & Ma, S. (2024) · TrAC Trends in Analytical Chemistry · doi:10.1016/j.trac.2024.118023 · Cited by 28 · see citing works
Personal narrative and stream of consciousness: an AI approach
Blyler, A. P., & Seligman, M. (2023) · Journal of Positive Psychology · doi:10.1080/17439760.2023.2257666 · Cited by 26 · see citing works
“More recently, developments of large language models (LLMs) enable social science researchers to accurately simulate human behavioral responses and analyze complex relationships within language on a much greater scale than ever before (Grossmann et al., 2023).”
BackgroundWhy scientists trust AI too much — and what to do about it
(2024) · Nature · doi:10.1038/d41586-024-00639-y · Cited by 25 · see citing works
The use of synthetic data in tourism
Viglia, G., Adler, S. J., Miltgen, C. L., & Sarstedt, M. (2024) · Annals of Tourism Research · doi:10.1016/j.annals.2024.103819 · Cited by 23 · see citing works
Humans create more novelty than ChatGPT when asked to retell a story
Breithaupt, F. A., Otenen, E., Wright, D. R., Kruschke, J. K., Li, Y., & Tan, Y. (2024) · Scientific Reports · doi:10.1038/s41598-023-50229-7 · Cited by 24 · see citing works
Wang, Z.-Y., Wang, D.-Q., Xu, Y., Zhou, L.-F., & Zhou, Y.-Q. (2024) · Journal of Chinese Political Science · doi:10.1007/s11366-025-09917-6 · Cited by 23 · see citing works
“On the other hand, AI is increasingly employed in social science research to simulate social actors, analyze behavioral patterns, and address complex issues by leveraging its cognitive, reasoning, and linguistic capabilities.”
Toward LLM-agent-based modeling of transportation systems: A conceptual framework
Liu, T., Yang, J., & Yin, Y. (2025) · Artificial Intelligence for Transportation · doi:10.1016/j.ait.2025.100001 · Cited by 23 · see citing works
Zhao, Y. C., Lian, J., Zhang, Y., Song, S., & Yao, X. (2023) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.24862 · Cited by 22 · see citing works
“…such as metaverse, large language models, and conversational bots in the past 2 years has primarily put forward a series of new propositions for traditional human-computer interaction, including human – computer collaboration and the interactive co-creation of human and AI (Grossmann et al., 2023).”
BackgroundMoral stereotyping in large language models
Zewail, A., Figueroa, A., Graham, J., & Atari, M. (2026) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2519941123 · Cited by 23 · see citing works
“This is particularly pressing given that some researchers have argued that AI will transform social science research (1, 53), and some called for the use of LLMs as data generators, potentially replacing human participants (3, 54).”
Chen, S., Guo, H., Hu, S., Ши, Л., Wang, Z., & Tanimoto, J. (2023) · Europhysics Letters (EPL) · doi:10.1209/0295-5075/acfb34 · Cited by 21 · see citing works
Large Language Models and the Wisdom of Small Crowds
Trott, S. (2024) · Open Mind · doi:10.1162/opmi_a_00144 · Cited by 22 · see citing works
A comprehensive review of artificial intelligence for pharmacology research
Li, B., Tan, K., Lao, A. R., Wang, H., Zheng, H., & Zhang, L. (2024) · Frontiers in Genetics · doi:10.3389/fgene.2024.1450529 · Cited by 24 · see citing works
Machine learning in microscopy – insights, opportunities and challenges
Cunha, I., Latron, E., Bauer, S., Sage, D., & Griffié, J. (2024) · Journal of Cell Science · doi:10.1242/jcs.262095 · Cited by 20 · see citing works
Emergence of cooperation in the one-shot Prisoner’s dilemma through Discriminatory and Samaritan AIs
Zimmaro, F., Miranda, M., Fernández, J. M. R., López, J. A. M., Reddel, M., Widler, V., Antonioni, A., & Han, T. A. (2024) · Journal of The Royal Society Interface · doi:10.1098/rsif.2024.0212 · Cited by 20 · see citing works
Six Fallacies in Substituting Large Language Models for Human Participants
Lin, Z. (2025) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459251357566 · Cited by 20 · see citing works
Large language models can predict the results of social science experiments
Ashokkumar, A., Hewitt, L., Ghezae, I., & Willer, R. (2026) · Nature · doi:10.1038/s41586-026-10742-x · Cited by 25 · see citing works
“Beyond these applications, LLM predictions could support power analyses, serve as Bayesian priors or approximate results where human experiments are ethically constrained (for example, misinformation research) 2 .”
Lugo-Morín, D. R. (2024) · Societies · doi:10.3390/soc14070106 · Cited by 19 · see citing works
“Cultural diversity and bias [79]: AI can be influenced by biases inherent in the data and algorithms used to train systems.”
The Mixed Subjects Design: Treating Large Language Models as Potentially Informative Observations
Broska, D., Howes, M., & van Loon, A. (2025) · Sociological Methods & Research · doi:10.1177/00491241251326865 · Cited by 18 · see citing works
The power of bridging decision scales: Model coupling for advanced climate policy analysis
Filatova, T., Akkerman, J., Bosello, F., Chatzivasileiadis, T., Arbués, I. C., Ghorbani, A., Ivanova, O., Knittel, N., Kwakkel, J. H., Lamperti, F., et al. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2411592122 · Cited by 18 · see citing works
Public administration with , of , and through AI: toward a new paradigm in the era of intelligence
Zhu, X. (2025) · Journal of Chinese Governance · doi:10.1080/23812346.2025.2578589 · Cited by 28 · see citing works
Grossmann, I., Varnum, M. E. W., Hutcherson, C. A. C., & Mandel, D. R. (2023) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2023.10.005 · Cited by 20 · see citing works
Self-citationBickley, S. J., Chan, H. F., Dao, B., Torgler, B., Tran, S., & Zimbatu, A. (2024) · Journal of Services Marketing · doi:10.1108/jsm-11-2023-0441 · Cited by 19 · see citing works
Potential benefits of employing large language models in research in moral education and development
Han, H. (2023) · Journal of Moral Education · doi:10.1080/03057240.2023.2250570 · Cited by 17 · see citing works
“Interestingly, recent developments in LLMs demonstrate significant emerging features simulating human psychological processes, which were unavailable in the previous simulation models (Grossmann et al., 2023).”
BackgroundArtificial intelligence chatbots mimic human collective behaviour
He, J. K., Wallis, F. P. S., Gvirtz, A., & Rathje, S. (2024) · British Journal of Psychology · doi:10.1111/bjop.12764 · Cited by 34 · see citing works
“Recent advances in AI chatbots have led some researchers to propose using them as agents to improve ABM's ability to simulate complex human collective dynamics (Epstein et al., 2023; Grossmann et al., 2023; Park et al., 2023; Pastor-Galindo et al., 2023).”
Theorizing with Large Language Models
Tranchero, M., Brenninkmeijer, C.-F., Murugan, A., & Nagaraj, A. (2024) · National Bureau of Economic Research · doi:10.3386/w33033 · Cited by 17 · see citing works
Wang, X., & Xie, F. (2025) · Journal of Knowledge Management · doi:10.1108/jkm-01-2025-0006 · Cited by 21 · see citing works
Using deep learning to generate key variables in global mitigation scenarios
Li, P., Zhu, R., McJeon, H. C., Byers, E. A., Zhou, P., & Ou, Y. (2025) · Nature Climate Change · doi:10.1038/s41558-025-02352-8 · Cited by 15 · see citing works
AI Surrogates and illusions of generalizability in cognitive science
Crockett, M. J., & Messeri, L. R. (2025) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2025.09.012 · Cited by 15 · see citing works
Counterfeit judgments in large language models
Perc, M. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2528527122 · Cited by 14 · see citing works
A large-scale comparison of divergent creativity in humans and large language models
Wang, D., Huang, D., Shen, H., & Uzzi, B. (2025) · Nature Human Behaviour · doi:10.1038/s41562-025-02331-1 · Cited by 14 · see citing works
Rudd, M. A., Bratcher, L., Collins, S., Branscum, D., Carson, M., Connell, S., David, E., Gronowska, M., Hess, S., Mitchell, A., et al. (2023) · Challenges · doi:10.3390/challe14040047 · Cited by 14 · see citing works
AI Thinking: a framework for rethinking artificial intelligence in practice
Newman-Griffis, D. R. (2025) · Royal Society Open Science · doi:10.1098/rsos.241482 · Cited by 18 · see citing works
Examining the Expanding Role of Synthetic Data Throughout the AI Development Pipeline
Kapania, S., Ballard, S. A., Kessler, A., & Vaughan, J. W. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732005 · Cited by 15 · see citing works
Uses, opportunities and risks of artificial intelligence in participatory urban planning
Raymond, C. M., Nummi, P., von Wirth, T., Poom, A., Ahdekivi, A., Barthel, S., Delmelle, E., Dunkel, E., Fagerholm, N., Grêt-Regamey, A., et al. (2025) · Discover Cities · doi:10.1007/s44327-025-00137-4 · Cited by 15 · see citing works
“The large language learning models associated with Generative AI could be used to emulate, interpret and test public comments without using traditional social survey methods [46].”
Psychologically-Valid Generative Agents: A Novel Approach to Agent-Based Modeling in Social Sciences
Mitsopoulos, K., Bose, R., Mather, B., Bhatia, A., Gluck, K., Dorr, B., Lebiere, C., & Pirolli, P. (2024) · AAAI Fall Symposia · doi:10.1609/aaaiss.v2i1.27698 · Cited by 18 · see citing works
Intersectional race–gender stereotypes in natural language
Bao, H., & GRIES, P. H. (2024) · British Journal of Social Psychology · doi:10.1111/bjso.12748 · Cited by 13 · see citing works
A step-by-step method for cultural annotation by LLMs
Dubourg, E., Thouzeau, V., & Baumard, N. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1365508 · Cited by 15 · see citing works
“These LLMs, like Generative Pre-trained Transformers (GPT), appear for the first time to be capable of annotating and parsing cultural data on a vast scale, and at an abstract level, by leveraging the contextual understanding inherent in these models (Brown et al., 2020; Grossmann et al., 2023).”
MethodChatGPT usage in the Reactome curation processPreprint
Tiwari, K. K., Matthews, L. R., May, B., Shamovsky, V., Orlic-Milacic, M., Rothfels, K., Ragueneau, E., Gong, C., Stephan, R., Li, N. T., et al. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.11.08.566195 · Cited by 13 · see citing works
“5 to fields like social sciences [6], education [7], art [8], software [9], health care [10], clinical research [11], and medicine [12] provides a compelling motive to develop good benchmarking exercises to test the capabilities of current LLMs to streamline biological data curation and to track…”
BackgroundCapturing a moving target: Developing research on and with AI for Human Relations
Stollberger, J., Anand, S., & Dick, P. (2025) · Human Relations · doi:10.1177/00187267251332075 · Cited by 13 · see citing works
Stavropoulos, A., Crone, D. L., & Grossmann, I. (2024) · Behavior Research Methods · doi:10.3758/s13428-024-02441-0 · Cited by 12 · see citing works
“…often hidden in the “black box” of their complex architecture, and reliable tools have yet to be developed to interpret this (Demszky et al., 2023; Grossmann et al., 2023) Finally, the protection of data privacy is complicated by the fact that LLMs may store sensitive data on external servers and…”
InfluentialSelf-citationThe Fill-Mask Association Test (FMAT): Measuring propositions in natural language.
Bao, H. (2024) · Journal of Personality and Social Psychology · doi:10.1037/pspa0000396 · Cited by 13 · see citing works
Li, F., Su, H., Lee, C., Feng, S., Jiang, Z., & Sun, Z. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2024.2446511 · Cited by 15 · see citing works
“Specifically, large multi-modal models (LMMs) have demonstrated significant potential and broad application prospects in various fields such as industrial design and customer service [4], [5].”
Alonso, S. L. N., Ozili, P. K., Sastre-Hernández, B. M., Pacheco, L., & Central Bank of Nigeria, A. (2025) · Quantitative Finance and Economics · doi:10.3934/qfe.2025008 · Cited by 12 · see citing works
Alexiou, V. G., Sumpio, B. E., Vassiliou, A., Kakkos, S. K., & Geroulakos, G. (2024) · Annals of Vascular Surgery · doi:10.1016/j.avsg.2024.11.014 · Cited by 11 · see citing works
Artificial Intelligence-Assisted Case Management in Social Work Services: A Systematic Review
Li, L., Wang, M., & Jian, M. (2025) · Research on Social Work Practice · doi:10.1177/10497315251329531 · Cited by 11 · see citing works
“…work professionals and AI experts to explore potential applications in social work practice (Goldkind, 2021), including developing LLMs capable of understanding and simulating human behavior (Grossmann et al., 2023), and utilizing ML to optimize social services and volunteer management (Yin, 2021).”
Large language models in climate and sustainability policy: limits and opportunities
Larosa, F., Hoyas, S., Conejero, J. A., García‐Martínez, J., Nerini, F. F., & Vinuesa, R. (2025) · Environmental Research Letters · doi:10.1088/1748-9326/addd36 · Cited by 11 · see citing works
Algorithmic CooperationPreprint
Kasberger, B., Martin, S., Normann, H., & Werner, T. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4389647 · Cited by 11 · see citing works
Bodell, M. H., Magnusson, M., & Keuschnigg, M. (2024) · Sociological Methods & Research · doi:10.1177/00491241241268453 · Cited by 10 · see citing works
“If researchers fi nd ways to gain more control over labeling, replicability, and transparency (Grossmann et al., 2023), this transformative brand of text modeling will be in a good position to develop important alternatives to the seeded topic model.”
AI and Data Science for Public Policy
Benoit, K. R. (2024) · LSE Public Policy Review · doi:10.31389/lseppr.115 · Cited by 10 · see citing works
Generative Agents in Agent-Based Modeling: Overview, Validation, and Emerging Challenges
Adornetto, C., Mora, A., Hu, K., Garcia, L. I., Atchade-Adelomou, P., Greco, G., Alonso, L., & Larson, K. (2025) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2025.3566362 · Cited by 10 · see citing works
“…of large language models (LLMs) suggests we rethink the entire agent-based model (ABM) processes, from the design of new enhanced agents able to perceive, think, memorize, and act as likely as possible to humans [1], [2], to the methodologies and approaches by which we validate their behaviors.”
Leça, M. D. M., Valença, L., Santos, R., & Santos, R. D. S. (2025) · IEEE/ACM International Workshop on Methodological Issues with Empirical Studies in Software Engineering (WSESE) · doi:10.1109/wsese66602.2025.00013 · Cited by 14 · see citing works
“LLMs are now being used widely across research fields due to their potential to streamline text-heavy tasks [24], [27].”
- Authors
- Grossmann, I., Feinberg, M., Parker, D. C., Christakis, N. A., Tetlock, P. E., & Cunningham, W. A.
- Affiliations
- University of Waterloo (CA)
- University of Toronto (CA)
- Yale University (US)
- University of Pennsylvania (US)
- ORCID
- Venue
- Science
- Published
- 2023-06-16
- Volume
- 380
- Issue
- 6650
- Pages
- 1108-1109
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W4380763235
- Cited by
- 377
- References
- 10
- Citations by year
- 2026: 110 · 2025: 156 · 2024: 92 · 2023: 19
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Social Sciences › General Social Sciences › Computational and Text Analysis Methods
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
- Keywords
- artificial intelligence, social science research, research transformation, algorithmic bias, data fidelity, artificial intelligence in research
- MeSH terms
- Artificial Intelligence, Humans, Social Sciences, Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
