The DOI matches its Crossref record; 98 of the 98 references it cites were found in a registry or an index.
Can Generative AI Improve Social Science?
Open accessAbstract
Artificial intelligence that can produce realistic text,images, and other human-like outputs is currently transforming many different industries. Yet it is not yet known how such tools might transform social science research. In the first section of this article, I assess the potential of Generative AI to improve online experiments, agent-based models, and automated content analyses.I also discuss whether these tools may help social scientists perform literature reviews, identify novel research questions, and develop hypotheses to explain them. Next, I evaluate whether Generative AI can help social scientists with more mundane tasks such as acquiring advancedprogramming skills or writing more effective prose. In the second section of this article I discuss the limitations of Generative AI as well as how these tools might be employed by researchers in an ethical manner. I discuss how bias in the processes and data used to train these tools can negatively impact social science research as well as a rangeof other challenges related to accuracy, reproducibility,interpretability, and efficiency. I conclude by highlighting the need for increased collaboration between social scientists and artificial intelligence researchers--- not only to ensure that such tools are used in a safe and ethical manner, but also because the progress of artificial intelligence may require deeper understanding of theories of human behavior
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Research assistant: Beyond their application in specialized domains, LLM-based agents are increasingly adopted as versatile assistants in the broad field of social science research [105,114].”
From A survey on large language model based autonomous agents (2024) · cites this paper as [114]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Dissecting racial bias in an algorithm used to manage the health of populations
Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019) · Science · doi:10.1126/science.aax2342 · Cited by 7,137
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988
Agendas and instability in American politics
(1993) · Choice Reviews Online · doi:10.5860/choice.31-0574 · Cited by 5,981
Experimental evidence of massive-scale emotional contagion through social networks
Kramer, A. D. I., Guillory, J. E., & Hancock, J. T. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1320040111 · Cited by 3,687
Energy and Policy Considerations for Deep Learning in NLP
Strubell, E., Ganesh, A., & McCallum, A. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1355 · Cited by 3,684
(2018) · Cited by 3,510
Grandmaster level in StarCraft II using multi-agent reinforcement learning
Vinyals, O., Babuschkin, I., Czarnecki, W. M., Mathieu, M., Dudzik, A., Chung, J., Choi, D. H., Powell, R., Ewalds, T., Georgiev, P., et al. (2019) · Nature · doi:10.1038/s41586-019-1724-z · Cited by 3,455
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
Growing Artificial Societies: Social Science from the Bottom Up
Epstein, J. M., & Axtell, R. L. (1996) · doi:10.7551/mitpress/3374.001.0001 · Cited by 2,441
Research works printed but not linked: 3
- [15]15. Daneshjou, Roxana et al. 2021. "Lack of Transparency and Potential Bias in Artificial Intelligence Data Sets and Algorithms: A Scoping Review." JAMA Dermatology 157(11): 1362-69.
- [39]39. Princeton, N.J.: Princeton University Press.
- [48]48. Yang, Kai-Cheng, and Filippo Menczer. 2023. "Large Language Models Can Rate News Outlet Credibility." http://arxiv.org/abs/2304.00228 (April 25, 2023). · arXiv
Cited by
A survey on large language model based autonomous agents
Wang, L., Ma, C., Feng, X., Zhang, Z., Yang, H., Zhang, J., Chen, Z., Tang, J., Chen, X., Lin, Y., et al. (2024) · Frontiers of Computer Science · doi:10.1007/s11704-024-40231-1 · Cited by 1,941
Multi-Agent Collaboration Mechanisms: A Survey of LLMs
Tran, K.-T., Dao, D., Nguyen, M.-D., Pham, Q.-V., O'Sullivan, B., & Nguyen, H. D. (2025) · arXiv · doi:10.48550/arxiv.2501.06322 · Cited by 680
Can Large Language Models Transform Computational Social Science?
Ziems, C., Held, W., Shaikh, O., Chen, J., Zhang, Z., & Yang, D. (2024) · Computational Linguistics · doi:10.1162/coli_a_00502 · Cited by 585
Exploring Large Language Model‐Based Intelligent Agents: Definitions, Methods, and Prospects
Cheng, Y., Zhang, C., Zhang, Z., Meng, X., Hong, S., Li, W., Wang, Z., Wang, Z., Yin, F., Zhao, J., et al. (2026) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.70111 · Cited by 334
GPT is an effective tool for multilingual psychological text analysis
Rathje, S., Mirea, D.-M., Sucholutsky, I., Marjieh, R., Robertson, C. E., & Van Bavel, J. J. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2308950121 · Cited by 297
Metadata
- Authors
- Bail, C. A.
- Affiliations
- Duke University (US)
- Corresponding author
- Bail, Christopher A.
- ORCID
- Bail, Christopher A.
- Venue
- Proceedings of the National Academy of Sciences
- Publisher
- Center for Open Science
- Published
- 2023-05-12
- Volume
- 121
- Issue
- 21
- Pages
- e2314021121-e2314021121
- ISSN
- 0027-8424 (ISSN-L)
- PMC
- PMC11127003
- OpenAlex
- W4396781199
- Cited by
- 369
- References
- 90
- Citations by year
- 2026: 143 · 2025: 160 · 2024: 55 · 2023: 11
- FWCI
- 148.38
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Proceedings of the National Academy of Sciences (publishedVersion, cc-by-nc-nd)
- PubMed Central (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- SocArXiv (OSF Preprints) (submittedVersion, cc-by-nd)
- SocArXiv (OSF Preprints) (submittedVersion)
- Funders
- Eunice Kennedy Shriver National Institute of Child Health and Human Development (P2CHD065563)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › General Social Sciences › Computational and Text Analysis Methods
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
- Keywords
- generative AI, social science research, survey research, online experiments, computational text analysis, agent-based modeling, training data bias, research ethics, replication, environmental impact of AI, human behavior
- MeSH terms
- Artificial Intelligence, Humans, Social Sciences
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Can Generative AI Improve Social Science? https://app.cochat.ai/papers/10.31235/osf.io/rwtzs
- The paper
- Bail, C. A. (2023). Can Generative AI Improve Social Science? Proceedings of the National Academy of Sciences. https://doi.org/10.31235/osf.io/rwtzs
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.