Scientific production in the era of large language models
Open accessAbstract
With the production process rapidly evolving, science policy must consider how institutions could evolve
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Existing work has documented that LLM adoption is associated with changes in productivity and referencing behavior in manuscripts and published papers, and more broadly that AI-augmented research can contract the collective focus of science.
“Existing work has documented that LLM adoption is associated with changes in productivity and referencing behavior in manuscripts and published papers, and more broadly that AI-augmented research can contract the collective focus of science (16, 26).”
From The rise of large language models and the direction and impact of US federal research funding (2026) · cites this paper as [16]
Split into 2 atoms — 0 confirmed, 1 partially supported, 0 disputed, 1 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
Experimental — accuracy not yet measuredThe productivity patterns align with and qualify recent evidence of large output gains following LLM diffusion in manuscript production.
“These productivity patterns both align with and qualify recent evidence of large output gains following LLM diffusion in manuscript production (16).”
From The rise of large language models and the direction and impact of US federal research funding (2026) · cites this paper as [16]
Higher LLM involvement is positively associated with greater writing complexity (Flesch Reading Ease score) in both proposals and awards.
“Consistent with prior work (16), we find that higher LLM involvement is positively associated with greater writing complexity in both proposals and awards”
From The rise of large language models and the direction and impact of US federal research funding (2026) · cites this paper as [16]
Recent large-scale evidence shows that the diffusion of LLMs has already been accompanied by sharp increases in scientific production across major preprint platforms.
“Recent large-scale evidence shows that the diffusion of LLMs has already been accompanied by sharp increases in scientific production across major preprint platforms, underscoring their potential to substantially raise output at scale (16).”
From The rise of large language models and the direction and impact of US federal research funding (2026) · cites this paper as [16]
Little is known about how the rise of LLMs is reshaping the public funding landscape.
“Despite growing attention to the use of AI in science (15–27), little is known about how the rise of large language models (LLMs) is reshaping the public funding landscape.”
From The rise of large language models and the direction and impact of US federal research funding (2026) · cites this paper as [16]
Prior work has documented large productivity increases at the manuscript level.
“recent large-scale evidence that LLM diffusion has been accompanied by sharp increases in manuscript production (16).”
From The rise of large language models and the direction and impact of US federal research funding (2026) · cites this paper as [16]
LLMs represent a potentially transformative development for scientific work.
“LLMs represent a potentially transformative development for scientific work (15–21).”
From The rise of large language models and the direction and impact of US federal research funding (2026) · cites this paper as [16]
Recent large-scale evidence shows that LLM diffusion has been accompanied by sharp increases in manuscript production.
“recent large-scale evidence that LLM diffusion has been accompanied by sharp increases in manuscript production (16).”
From The rise of large language models and the direction and impact of US federal research funding (2026) · cites this paper as [16]
Authors who incorporate LLMs into their writing publish more preprints per month than nonadopters.
“Adoption also appears to translate into measurable output gains, with authors who incorporate LLMs into their writing publishing more preprints per month than nonadopters 28 .”
From Reimagining biomedical science workflows in the age of large language models (2026) · cites this paper as [28]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Experimental evidence on the productivity effects of generative artificial intelligence
Noy, S., & Zhang, W. (2023) · Science · doi:10.1126/science.adh2586 · Cited by 2,150 · see citing works
Brynjolfsson, E., Li, D., & Raymond, L. (2025) · The Quarterly Journal of Economics · doi:10.1093/qje/qjae044 · Cited by 1,083 · see citing works
The Burden of Knowledge and the “Death of the Renaissance Man”: Is Innovation Getting Harder?
JONES, B. F. (2009) · Review of Economic Studies · doi:10.1111/j.1467-937x.2008.00531.x · Cited by 1,058 · see citing works
The Virtual Lab of AI agents designs new SARS-CoV-2 nanobodies
Swanson, K., Wu, W., Bulaong, N. L., Pak, J. E., & Zou, J. (2025) · Nature · doi:10.1038/s41586-025-09442-9 · Cited by 169 · see citing works
Quantifying large language model usage in scientific papers
Liang, W., Zhang, Y., Wu, Z., Lepp, H., Ji, W., Zhao, X., Cao, H., Liu, S., He, S., Huang, Z., et al. (2025) · Nature Human Behaviour · doi:10.1038/s41562-025-02273-8 · Cited by 80 · see citing works
Mapping the Increasing Use of LLMs in Scientific Papers
Liang, W., Zhang, Y., Wu, Z., Lepp, H., Ji, W., Zhao, X., Cao, H., Liu, S., He, S., Huang, Z., et al. (2024) · arXiv · doi:10.48550/arxiv.2404.01268 · Cited by 55 · see citing works
Flagged first, then by citations
Research works printed but not linked: 18
- [1]1. N. Rasmussen, Picture Control: The Electron Microscope and the Transformation of Biology in America, 1940-1960 . (Stanford University Press, 1999).
- [6]6. H. Bao, M. Sun, M. Teplitskiy, Where there's a will there's a way: ChatGPT is used more for science in countries where it is prohibited. arXiv preprint arXiv:2406.11583 , (2024). · arXiv
- [10]10. Q. Hao, F. Xu, Y. Li, J. Evans, AI Expands Scientists' Impact but Contracts Science's Focus. arXiv preprint arXiv:2412.07727 , (2024). · arXiv
- [16]16. C. Si, D. Yang, T. Hashimoto, Can LLMs Generate Novel Research Ideas? A LargeScale Human Study with 100+ NLP Researchers. arXiv preprint arXiv:2409.04109 , (2024). · arXiv
- [17]17. J. Kim, B. Lee, AI-Augmented Surveys: Leveraging Large Language Models and Surveys for Opinion Prediction. arXiv preprint arXiv:2305.09620 , (2023). · arXiv
- [19]19. S. Lai, J. Kim, N. Kunievsky, Y. Potter, J. Evans, Based AI improves human decisionmaking but reduces trust. arXiv preprint arXiv:2508.09297 , (2025). · arXiv
- [20]20. C. Xie et al. , Can large language model agents simulate human trust behavior? Advances in neural information processing systems 37 , 15674-15729 (2024).
- [22]22. S. Peng, E. Kalliamvakou, P. Cihon, M. Demirer, The impact of ai on developer productivity: Evidence from github copilot. arXiv preprint arXiv:2302.06590 , (2023). · arXiv
- [24]24. C. Bazerman, Shaping Written Lnowledge: The Genre and Activity of the Experimental Article in Science . (The University of Wisconsin Press, 1988).
- [25]25. D. N. McCloskey, The Rhetoric of Economics. Journal of Economic Literature 21 , 481517 (1983).
- [28]28. D. D. Belcher, Seeking acceptance in an English-only research world. Journal of Second Language Writing 16 , 1-22 (2007).
- [31]31. H. Hu, D. Wang, S. Deng, Analysis of the scientific literature's abstract writing style and citations. Online Information Review 45 , 1290-1305 (2021).
- [32]32. J. Feld, C. Lines, L. Ross, Writing matters. Journal of Economic Behavior & Organization 217 , 378-397 (2024).
- [33]33. H. Peng, H. S. Qiu, H. B. Fosse, B. Uzzi, Promotional language and the adoption of innovative ideas in science. Proceedings of the National Academy of Sciences 121 , e2320066121 (2024).
- [36]36. Z. Wojtowicz, S. DeDeo, Undermining mental proof: How ai can make cooperation harder by making thinking easier. Proceedings of the AAAI Conference on Artificial Intelligence 39 , 1592-1600 (2025).
- [37]37. Y. Tian, Y. Liu, Y. Bu, J. Liu, Who Gets Recommended? Investigating Gender, Race, and Country Disparities in Paper Recommendations from Large Language Models. arXiv preprint arXiv:2501.00367 , (2024). · arXiv
- [39]39. A. Algaba et al. , Large Language Models Reflect Human Citation Patterns with a Heightened Citation Bias. arXiv preprint arXiv:2405.15739 , (2024). · arXiv
- [42]42. D. Crane, Invisible colleges; diffusion of knowledge in scientific communities. (1972). Acknowledgments: We thank Mor Naaman, Will Cong, Wu Zhu, as well as seminar participants at Complexity Science Hub (Vienna), UCLA Price Center, Haas MRL seminar, and Columbia MAD conference for helpful discussions, and Alex Cui for providing academic access to the GPTZero API. This work is supported by National Science Foundation under grant numbers 2311521, 2404035, and 2412389.
Cited by
Sorted by citations
Generative AI use and misuse call for assessment reform in higher education
Chirikov, I., Smirnov, I., & Kizilcec, R. F. (2026) · Science · doi:10.1126/science.aec5115 · Cited by 16 · see citing works
The rise of large language models and the direction and impact of US federal research funding
Qian, Y., Wen, Z., Furnas, A. C., Bai, Y., Shao, E., & Wang, D. (2026) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2601439123 · Cited by 5 · see citing works
LLM hallucinations in the wild: Large-scale evidence from non-existent citations
Zhao, Z., Wang, Y., Stuart, T., De Vaan, M., Ginsparg, P., & Yin, Y. (2026) · arXiv · doi:10.48550/arxiv.2605.07723 · Cited by 3 · see citing works
“These results appear consistent with recent evidence that LLM use boosts scientific output [11], [26], especially for less-experienced researchers [27] who also may have fewer resources at their disposal.”
Self-citationReimagining biomedical science workflows in the age of large language models
Roberts, K. F., Koutarapu, S., Melendez, J., Landsness, E. C., Reese, J. T., Elbert, D. L., & Bateman, R. J. (2026) · npj Dementia · doi:10.1038/s44400-026-00126-3
- Authors
- Kusumegi, K., Yang, X., Ginsparg, P., de Vaan, M., Stuart, T., & Yin, Y.
- Affiliations
- Cornell University (US)
- Haas School of Business, University of California, Berkeley, Berkeley, CA, USA.
- University of California, Berkeley (US)
- Venue
- Science
- Published
- 2025-12-18
- Volume
- 390
- Issue
- 6779
- Pages
- 1240-1243
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W4417474777
- Cited by
- 83
- References
- 6
- Citations by year
- 2026: 82 · 2025: 1
- FWCI
- 83.08
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Science (publishedVersion)
- arXiv (Cornell University) (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University)
- Topics
- Social Sciences › Decision Sciences › Information Systems and Management › Research, Science, and Academia
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › scientometrics and bibliometrics research
- Social Sciences › Decision Sciences › Information Systems and Management › Academic Publishing and Open Access
- Keywords
- large language models, research output, science policy, research institutions, artificial intelligence in science
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
