The DOI matches its Crossref record; 11 of the 11 references it cites were found in a registry or an index.
AI and the transformation of social science research
Abstract
Careful bias management and data fidelity are key
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Researchers have also recently noted GPT’s ability to help with computational social science tasks (49–54), detect misinformation (55), infer politicians’ ideologies (56), write persuasive political arguments (57), respond to patient questions (58), simulate human research participants (59–61), and model collective behavior (56).”
From GPT is an effective tool for multilingual psychological text analysis (2024) · cites this paper as [53]
In the citing paper
“Researchers who want to use LLMs to impersonate biased groups, for example, may discover these tools are unwilling to perform such roles because they have been fine-tuned according to the normative preferences of highly educated liberals who may have more concern about the protection of marginalized groups than others (12, 64).”
From Can Generative AI improve social science? (2024) · cites this paper as [12]
In the citing paper
“Though such guard-rails probably improve the safety of Generative AI tools for public use, they may impede the ability of social scientists to leverage bias for research purposes (12).”
From Can Generative AI improve social science? (2024) · cites this paper as [12]
In the citing paper
“Though Generative AI may help researchers increase the external validity of their research designs, this may come at the cost of internal validity—or the capacity of different groups of researchers to reproduce or replicate each other’s results (2, 12).”
From Can Generative AI improve social science? (2024) · cites this paper as [12]
Citations
Cites
Generative Agents: Interactive Simulacra of Human Behavior
Park, J. S., O'Brien, J. C., Cai, C. J., Morris, M. R., Liang, P., & Bernstein, M. S. (2023) · arXiv · doi:10.48550/arxiv.2304.03442 · Cited by 2,173
Pennycook, G., McPhetres, J., Zhang, Y., Lu, J. G., & Rand, D. G. (2020) · Psychological Science · doi:10.1177/0956797620939054 · Cited by 1,715
Taxonomy of Risks posed by Language Models
Weidinger, L., Uesato, J., Rauh, M., Griffin, C., Huang, P.-S., Mellor, J., Glaese, A., Cheng, M., Balle, B., Kasirzadeh, A., et al. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533088 · Cited by 764
Algorithmic bias: Senses, sources, solutions
Fazelpour, S., & Danks, D. J. (2021) · Philosophy Compass · doi:10.1111/phc3.12760 · Cited by 338
The Science of Wisdom in a Polarized World: Knowns and Unknowns
Grossmann, I., Weststrate, N. M., Ardelt, M., Brienza, J. P., Dong, M., Ferrari, M., Fournier, M. A., Hu, C. S., Nusbaum, H. C., & Vervaeke, J. (2020) · Psychological Inquiry · doi:10.1080/1047840x.2020.1750917 · Cited by 330
Self-citationLocally noisy autonomous agents improve global human coordination in network experiments
Shirado, H., & Christakis, N. A. (2017) · Nature · doi:10.1038/nature22332 · Cited by 288
Self-citationUsing large-scale experiments and machine learning to discover theories of human decision-making
Peterson, J. C., Bourgin, D., Agrawal, M., Reichman, D., & Griffiths, T. L. (2021) · Science · doi:10.1126/science.abe2629 · Cited by 273
Large language models associate Muslims with violence
Abid, A., Farooqi, M., & Zou, J. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00359-2 · Cited by 176
Integrating social and cognitive aspects of belief dynamics: towards a unifying framework
Galešić, M., Olsson, H., Dalege, J., van der Does, T., & Stein, D. L. (2021) · Journal of The Royal Society Interface · doi:10.1098/rsif.2020.0857 · Cited by 102
Psychological Perspectives on Nuclear Deterrence
Tetlock, P. E., McGuire, C., & Mitchell, G. (1991) · Annual Review of Psychology · doi:10.1146/annurev.ps.42.020191.001323 · Cited by 34
Self-citation
Research works printed but not linked: 3
- S. Bubeck arXiv:2303.12712 [cs.CL] (2023). · arXiv
- L. P. Argyle Polit. Anal. 10.1017/pan.2023.2 (2023). · DOI · Paper page
- Y. Bai arXiv:2212.08073 [cs.CL] (2022). · arXiv
Cited by
Sorted by citations
A survey on large language model based autonomous agents
Wang, L., Ma, C., Feng, X., Zhang, Z., Yang, H., Zhang, J., Chen, Z., Tang, J., Chen, X., Lin, Y., et al. (2024) · Frontiers of Computer Science · doi:10.1007/s11704-024-40231-1 · Cited by 1,941
“ChemCrow [75], Boiko et al. [115], Grossmann et al. [122] Natural Science Education ChemCrow [75], CodeHelp [120], Boiko et al. [115], MathA-gent [117], Drori et al. [118] Engineering”
BackgroundThe Rise and Potential of Large Language Model Based Agents: A Survey
Xi, Z., Chen, W., Guo, X., He, W., Ding, Y., Hong, B., Zhang, M., Wang, J., Jin, S., Zhou, E., et al. (2023) · arXiv · doi:10.48550/arxiv.2309.07864 · Cited by 972
The rise and potential of large language model based agents: a survey
Xi, Z., Chen, W., Guo, X., He, W., Ding, Y., Hong, B., Zhang, M., Wang, J., Jin, S., Zhou, E., et al. (2025) · Science China Information Sciences · doi:10.1007/s11432-024-4222-0 · Cited by 970
“Recent research on simulated societies has followed two primary lines, namely, exploring the boundaries of the collective intelligence capabilities of LLM-based agents [90, 115, 327, 330, 332] and using them to accelerate discoveries in the social sciences [16,438,439].”
Artificial intelligence and illusions of understanding in scientific research
Messeri, L., & Crockett, M. J. (2024) · Nature · doi:10.1038/s41586-024-07146-0 · Cited by 707
“…including (but not limited to) race, ethnicity, gender, class, disability and/or relevant experiences such as background, training and career stage. experiences and perspectives ... offering a more accurate portrayal of human behaviour and social dynamics than those from conventional methods” 4 .”
InfluentialDurably reducing conspiracy beliefs through dialogues with AI
Costello, T. H., Pennycook, G., & Rand, D. G. (2024) · Science · doi:10.1126/science.adq1814 · Cited by 354
“Moreover, ouruseofGPT-4Turbo,afrontier,closed-source,pretrained,andfine-tunedlanguage model, presents challenges related to inter-pretability and replicability ( 69 – 71 ).”
Metadata
- Authors
- Grossmann, I., Feinberg, M., Parker, D. C., Christakis, N. A., Tetlock, P. E., & Cunningham, W. A.
- Affiliations
- University of Waterloo (CA)
- University of Toronto (CA)
- Yale University (US)
- University of Pennsylvania (US)
- ORCID
- Venue
- Science
- Published
- 2023-06-16
- Volume
- 380
- Issue
- 6650
- Pages
- 1108-1109
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W4380763235
- Cited by
- 377
- References
- 10
- Citations by year
- 2026: 110 · 2025: 156 · 2024: 92 · 2023: 19
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › General Social Sciences › Computational and Text Analysis Methods
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
- Keywords
- artificial intelligence, social science research, research transformation, algorithmic bias, data fidelity, artificial intelligence in research
- MeSH terms
- Artificial Intelligence, Humans, Social Sciences, Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
