Delving into LLM-assisted writing in biomedical publications through excess vocabulary
Open accessPublished version of a preprint
The DOI matches its Crossref record; 62 of the 62 references it cites were found in a registry or an index; 26 of the 38 sources its claims cite were checked for support.
Abstract
Large language models (LLMs) like ChatGPT can generate and revise text with human-level performance. These models come with clear limitations, can produce inaccurate information, and reinforce existing biases. Yet, many scientists use them for their scholarly writing. But how widespread is such LLM usage in the academic literature? To answer this question for the field of biomedical research, we present an unbiased, large-scale approach: We study vocabulary changes in more than 15 million biomedical abstracts from 2010 to 2024 indexed by PubMed and show how the appearance of LLMs led to an abrupt increase in the frequency of certain style words. This excess word analysis suggests that at least 13.5% of 2024 abstracts were processed with LLMs. This lower bound differed across disciplines, countries, and journals, reaching 40% for some subcorpora. We show that LLMs have had an unprecedented impact on scientific writing in biomedical research, surpassing the effect of major world events such as the COVID pandemic.
Overview
CoChat Summary
TL;DR
This paper uses excess word frequencies in >15 million PubMed abstracts (2010–2024) to estimate LLM-assisted writing in biomedical publications. It finds an unprecedented post-ChatGPT rise in style words, implying at least 13.5% of 2024 abstracts were processed with LLMs.
Key findings
- At least 13.5% of 2024 PubMed abstracts were processed with LLMs, based on averaging two independent marker-word estimates: Δrare = 0.136 (291 rare style words) and Δcommon = 0.134 (10 common style words).
- Excess vocabulary rose sharply after ChatGPT: 454 excess words in 2024 versus 190 in 2021; counting unique lemmas, 343 in 2024 versus 180 in 2021.
- The 2024 excess vocabulary was mostly style words, not content words: of 379 excess style words in 2024, 66% were verbs and 14% adjectives.
- Lower bounds varied widely by subcorpus, reaching Δ = 0.41 for computation papers from China, Δ = 0.34 for South Korea papers in Sensors, Δ = 0.25 for Sensors, Δ = 0.21 for MDPI, Δ = 0.20 for Frontiers and Cureus, and Δ = 0.07 for Nature/Science/Cell. Gender differences were minor (0.11 male vs 0.10 female).
Methods
- Used the 2025 PubMed annual snapshot, keeping complete English-language abstracts of 250–4000 characters from 2010–2024, yielding 15,103,888 abstracts.
- Built a sparse binary word-occurrence matrix and focused on 273,112 words of at least four English letters; main excess analysis used 26,657 words with p > 10−4 in both 2023 and 2024.
- Computed counterfactual 2024 word frequencies by linear extrapolation from 2021–2022, then measured excess frequency gap δ = p − q and ratio r = p/q; combined excess style words into rare and common marker sets to estimate lower bounds on LLM usage.
Limitations
- The paper states the analysis is corpus-level and cannot identify individual abstracts processed by an LLM; it also cannot separate direct LLM usage from authors adopting LLM-preferred words, nor distinguish effects of different LLMs.
- The paper states the estimates are lower bounds because abstracts not containing selected marker words do not contribute; heterogeneity may reflect detection differences, author editing/censoring of LLM style, and faster publication timelines in some fields/journals.
- The paper states gender inference is approximate, based on first names, with limited US-based training data and only 55% of authors inferable; evident from the text, the method depends on style-word markers and may miss well-disguised LLM use.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures





Verification
Claims the paper makes
11 Valid · 3 Not valid · 5 Not checked · 66 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“This makes LLM outputs less diverse and novel than human-written text (43, 44).”
§ Delving into LLM-assisted writing in biomedical publications through excess vocabulary > DISCUSSION > Implications and policies
What the sources say
- Partly supportedcites [43] V. Padmakumar, H. He, Does writing with language models reduce content diversity? arXiv:2309.05196 [cs.CL] (2023). sourceExperimental — accuracy not yet measured
The Introduction and conclusion report reduced content diversity in essays co-written with InstructGPT compared with essays written without model help; Section 6 also finds InstructGPT suggestions less diverse than GPT3 suggestions. The paper does not establish that LLM outputs generally are less diverse than human-written text: it found no statistically significant diversity reduction for GPT3-assisted writing. It also does not separately establish the claim about novelty. (LLM, conf=0.87)
- Partly supportedcites [44] A. J. Alvero, J. Lee, A. Regla-Vargas, R. F. Kizilec, T. Joachims, A. L. Antonio, Large language models, social demography, and hegemony: Comparing authorship in human and synthetic text. J. Big Data 11, 138 (2024). sourceExperimental — accuracy not yet measured
The Direct comparisons section (page unavailable) reports that AI-generated essays had lower variation across many writing-style features than human essays, supporting a narrower stylistic range. It does not establish that LLM outputs are less novel, and its comparison is limited to essays generated for college-admissions prompts. (LLM, conf=0.86)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“This setup can introduce biases (16), as it requires assumptions on which models scientists use for their LLM-assisted writing, and how exactly they prompt them.”
§ Delving into LLM-assisted writing in biomedical publications through excess vocabulary > INTRODUCTION
What the sources say
- Partly supportedcites [16] R. Tang, Y.-N. Chuang, X. Hu, The science of detecting LLM-generated text. Commun. ACM 67, 50– 59 (2024). sourceExperimental — accuracy not yet measured
The “Limitations of black-box detection” section says data collection can introduce biases that harm a detector’s generalization. “Data acquisition” also says researchers must choose generation models and prompts when collecting LLM-generated text. The paper does not specifically discuss assumptions about which models scientists use for LLM-assisted writing or how those scientists prompt them, so that application is an extension of its point. (LLM, conf=0.85)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Computational fields like computation and bioinformatics showed Δ ≈ 0.20 (Fig. 5B).”
§ Delving into LLM-assisted writing in biomedical publications through excess vocabulary > RESULTS > Lower bounds differed between subcorpora
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on 2 assumptions not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In 2020–2022, words like coronavirus, covid, lockdown, and pandemic showed very large excess usages (up to r > 1000 and δ = 0.06), in agreement with the observation that the COVID pandemic had an unprecedented effect on biomedical publishing (30).”
§ Delving into LLM-assisted writing in biomedical publications through excess vocabulary > RESULTS > Excess words indicate widespread LLM usage
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Content words were predominantly nouns (79.2%), and hence most excess words before 2024 were nouns.”
§ Delving into LLM-assisted writing in biomedical publications through excess vocabulary > RESULTS > Excess words indicate widespread LLM usage
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
28 more claims
Create a free account to see all 85 claims and to verify the 1 reference still open.
- Probably not a substantial claim. 30
- It may state two separate claims as one. 33
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“LLM text exhibits systematic lexical and syntactic patterns (Shaib et al., 2026; Kobak et al., 2025; Shaib et al., 2025)”
From StoryScope: Investigating idiosyncrasies in AI fiction (2026) · cites this paper as Kobak et al., 2025
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Another study which estimated LLM usage in scientific writing by quantitating vocabulary shifts suggested that 13.5% of all 2024 PubMed abstracts were processed with LLMs 27 .”
From Reimagining biomedical science workflows in the age of large language models (2026) · cites this paper as [27]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Kobak et al. [16] reported that no clear increase in LLM-associated “excess vocabulary” was observed during the pre-ChatGPT period (e.g., 2020–2022) compared with two years earlier.”
From Delving Into PubMed Records: How AI-Influenced Vocabulary has Transformed Medical Writing since ChatGPT (2025) · cites this paper as [16]
The cited source says what the claim attributes to it.
In the citing paper
“One study that analyzed over 15 million abstracts from the biomedical literature documented the emergence of a bona fide LLM lexicon, with specific model-favored words rising at rates that cannot be explained by natural linguistic drift 2 .”
From Fine-Grained Detection of AI-Generated Writing in the Biomedical Literature (2026) · cites this paper as [2]
The cited source says what the claim attributes to it.
In the citing paper
“their analysis revealed an unprecedented post-2022 surge in stylistic words including ‘delve’ and ‘underscore’ [16].”
From Delving Into PubMed Records: How AI-Influenced Vocabulary has Transformed Medical Writing since ChatGPT (2025) · cites this paper as [16]
3 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Delving into LLM-assisted writing in biomedical publications through excess vocabulary > RESULTS > Lower bounds differed between subcorpora
“Among countries, some English-speaking countries like United Kingdom and Australia showed Δ ≈ 0.05, while countries like China, South Korea, and Taiwan showed Δ ≈ 0.20 (Fig. 5B).”
§ Delving into LLM-assisted writing in biomedical publications through excess vocabulary > RESULTS > Combining excess words puts a lower bound on LLM usage
“(B) Frequency gap estimates for various fields identified based on journal names (30). (C) Frequency gap estimates for various countries.”
Citations
Cites
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016
(2008) · Journal of Machine Learning Research · Cited by 33,051
Scikit-learn: Machine Learning in Python
(2011) · Journal of Machine Learning Research · doi:10.5555/1953048.2078195 · Cited by 8,539
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988 · cited 1×
“Furthermore, LLMs can mimic biases and other deficiencies from their training data (Bender et al., 2021; Nav-igli et al., 2023; Bai et al., 2024; Choudhury, 2023), or even outright plagiarise (McCoy et al., 2023).”
Survey of Hallucination in Natural Language Generation
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023) · ACM Computing Surveys · doi:10.1145/3571730 · Cited by 5,025 · cited 2×
“…LLMs are infamous for making up references (Walters and Wilder, 2023), providing inaccurate summaries (Tang et al., 2024a; Kim et al., 2024), and making false claims that sound authoritative and convincing (Mittelstadt et al., 2023; Ji et al., 2023; Zhang et al., 2023; Zheng and Zhan, 2023).”
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-T., Rocktäschel, T., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.11401 · Cited by 3,101 · cited 1×
“Some have suggested to use retrieval-augmented LLMs that provide verifiable facts from trusted sources (Lewis et al., 2020; Borgeaud et al., 2022; Ahmed et al., 2023) or let the user provide all relevant facts to the LLM in order to protect scientific literature from accumulating subtle…”
Domain-specific language model pretraining for biomedical natural language processing
Gu, Y., Tinn, R., Cheng, H., Lucas, M., Usuyama, N., Liu, X., Naumann, T., Gao, J., & Poon, H. (2022) · ACM Transactions on Computing for Healthcare · doi:10.1145/3458754 · Cited by 2,170
ChatGPT is fun, but not an author
Thorp, H. H. (2023) · Science · doi:10.1126/science.adg7879 · Cited by 1,437 · cited 1×
“As a result, publishers and funding agencies have put out various policies, banning LLMs in peer review (Kaiser, 2023; Brainard, 2023), as co-authors (Thorp, 2023), or undis-closed resource of any kind (Brainard, 2023).”
The WHO estimates of excess mortality associated with the COVID-19 pandemic
Msemburi, W., Karlinsky, A., Knutson, V., Aleshin-Guendel, S., Chatterji, S., & Wakefield, J. (2023) · Nature · doi:10.1038/s41586-022-05522-2 · Cited by 1,064
Adapted large language models can outperform medical experts in clinical text summarization
Van Veen, D., Van Uden, C., Blankemeier, L., Delbrouck, J.-B., Aali, A., Bluethgen, C., Pareek, A., Polacin, M., Reis, E. P., Seehofnerová, A., et al. (2024) · Nature Medicine · doi:10.1038/s41591-024-02855-5 · Cited by 891 · cited 1×
“Scientists use LLM-assisted writing because LLMs can improve grammar, rhetoric, and overall readability of their texts, help translate to English, and quickly generate summaries (Van Veen et al., 2024; Zhang et al., 2024).”
ChatGPT listed as author on research papers: many scientists disapprove
Stokel-Walker, C. (2023) · Nature · doi:10.1038/d41586-023-00107-z · Cited by 868 · cited 1×
“Many researchers have since integrated LLMs in their daily writing tasks (Van Noorden and Perkel, 2023) and even co-authored papers with LLMs (Stokel-Walker, 2023).”
Karlinsky, A., & Kobak, D. (2021) · eLife · doi:10.7554/elife.69336 · Cited by 772 · cited 1×
“We were inspired by studies of excess mortality (Islam et al., 2021; Karlinsky and Kobak, 2021; Msemburi et al., 2023) that looked at the excess of fatalities during the Covid pandemic compared to pre-Covid mortality.”
InfluentialSelf-citationSiren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models
Zhang, Y., Li, Y., Cui, L., Cai, D., Liu, L., Fu, T., Huang, X., Zhao, E., Zhang, Y., Xu, C., et al. (2023) · arXiv · doi:10.48550/arxiv.2309.01219 · Cited by 544
Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models
Zhang, Y., Li, Y., Cui, L., Cai, D., Liu, L., Fu, T., Huang, X., Zhao, E., Zhang, Y., Chen, Y., et al. (2025) · Computational Linguistics · doi:10.1162/coli.a.16 · Cited by 534 · cited 2×
“…LLMs are infamous for making up references (Walters and Wilder, 2023), providing inaccurate summaries (Tang et al., 2024a; Kim et al., 2024), and making false claims that sound authoritative and convincing (Mittelstadt et al., 2023; Ji et al., 2023; Zhang et al., 2023; Zheng and Zhan, 2023).”
Fabrication and errors in the bibliographic citations generated by ChatGPT
Walters, W. H., & Wilder, E. I. (2023) · Scientific Reports · doi:10.1038/s41598-023-41032-5 · Cited by 499 · cited 2×
“However, LLMs are infamous for making up references (Walters and Wilder, 2023), providing inaccurate summaries (Tang et al., 2024a; Kim et al., 2024), and making false claims that sound authoritative and convincing (Mittelstadt et al., 2023; Ji et al., 2023; Zhang et al., 2023; Zheng and Zhan,…”
Islam, N., Shkolnikov, V. M., Acosta, R. J., Klimkin, I., Kawachi, I., Irizarry, R. A., Alicandro, G., Khunti, K., Yates, T., Jdanov, D. A., et al. (2021) · BMJ · doi:10.1136/bmj.n1137 · Cited by 468 · cited 1×
“We were inspired by studies of excess mortality (Islam et al., 2021; Karlinsky and Kobak, 2021; Msemburi et al., 2023) that looked at the excess of fatalities during the Covid pandemic compared to pre-Covid mortality.”
InfluentialBenchmarking Large Language Models for News Summarization
Zhang, T., Ladhak, F., Durmus, E., Liang, P., McKeown, K., & Hashimoto, T. B. (2023) · arXiv · doi:10.48550/arxiv.2301.13848 · Cited by 415
Benchmarking Large Language Models for News Summarization
Zhang, T., Ladhak, F., Durmus, E., Liang, P., McKeown, K., & Hashimoto, T. B. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00632 · Cited by 415
AI and science: what 1,600 researchers think
Van Noorden, R., & Perkel, J. M. (2023) · Nature · doi:10.1038/d41586-023-02980-0 · Cited by 383 · cited 1×
“Many researchers have since integrated LLMs in their daily writing tasks (Van Noorden and Perkel, 2023) and even co-authored papers with LLMs (Stokel-Walker, 2023).”
Improving language models by retrieving from trillions of tokens
Borgeaud, S., Mensch, A., Hoffmann, J., Cai, T., Rutherford, E., Millican, K., Driessche, G. V. D., Lespiau, J.-B., Damoc, B., Clark, A., et al. (2021) · arXiv · doi:10.48550/arxiv.2112.04426 · Cited by 382 · cited 1×
“Some have suggested to use retrieval-augmented LLMs that provide verifiable facts from trusted sources (Lewis et al., 2020; Borgeaud et al., 2022; Ahmed et al., 2023) or let the user provide all relevant facts to the LLM in order to protect scientific literature from accumulating subtle…”
Biases in Large Language Models: Origins, Inventory, and Discussion
Navigli, R., Conia, S., & Ross, B. (2023) · Journal of Data and Information Quality · doi:10.1145/3597307 · Cited by 364
Studying the history of ideas using topic models
Hall, D., Jurafsky, D., & Manning, C. D. (2008) · Proceedings of the Conference on Empirical Methods in Natural Language Processing - EMNLP '08 · doi:10.3115/1613715.1613763 · Cited by 191 · cited 1×
“The rise and fall of scientific disciplines is traceable in scholarly writing (Hall et al., 2008; Bizzoni et al., 2020).”
Desaire, H., Chua, A. E., Isom, M., Jarošová, R., & Hua, D. (2023) · Cell Reports Physical Science · doi:10.1016/j.xcrp.2023.101426 · Cited by 166
Empirical evidence of Large Language Model's influence on human spoken communicationPreprint
Yakura, H., Lopez-Lopez, E., Brinkmann, L. U. L., Serna, I., Kirfel, L., Gupta, P., Soraperra, I., Eisenmann, T. F., Wulff, D. U., & Rahwan, I. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2409.01754 · Cited by 137
Brinkmann, L., Baumann, F., Bonnefon, J.-F., Derex, M., Müller, T. F., Nussberger, A.-M., Czaplicka, A., Acerbi, A., Griffiths, T. L., Henrich, J., et al. (2023) · Nature Human Behaviour · doi:10.1038/s41562-023-01742-2 · Cited by 132 · cited 1×
“Our analysis can inform the necessary debate around LLM policies by providing a measurement method for LLM usage that is urgently needed (Brinkmann et al., 2023; Heersmink, 2024).”
The Science of Detecting LLM-Generated Text
Tang, R., Chuang, Y.-N., & Hu, X. (2024) · Communications of the ACM · doi:10.1145/3624725 · Cited by 125 · cited 3×
“However, LLMs are infamous for making up references (Walters and Wilder, 2023), providing inaccurate summaries (Tang et al., 2024a; Kim et al., 2024), and making false claims that sound authoritative and convincing (Mittelstadt et al., 2023; Ji et al., 2023; Zhang et al., 2023; Zheng and Zhan,…”
Jane, John … Leslie? A Historical Method for Algorithmic Gender Prediction
Blevins, C., & Mullen, L. A. (2015) · Digital humanities quarterly · doi:10.63744/2h8m9cv8t9y4 · Cited by 109
ChatGPT in Scientific Writing: A Cautionary Tale
Zheng, H., & Zhan, H. (2023) · The American Journal of Medicine · doi:10.1016/j.amjmed.2023.02.011 · Cited by 104 · cited 1×
“…LLMs are infamous for making up references (Walters and Wilder, 2023), providing inaccurate summaries (Tang et al., 2024a; Kim et al., 2024), and making false claims that sound authoritative and convincing (Mittelstadt et al., 2023; Ji et al., 2023; Zhang et al., 2023; Zheng and Zhan, 2023).”
Kendall, G., & Teixeira da Silva, J. A. (2024) · Learned Publishing · doi:10.1002/leap.1578 · Cited by 100 · cited 2×
“Even worse, it is likely that malign actors such as paper mills will employ LLMs to produce fake publications (Kendall and Teixeira da Silva, 2024).”
The landscape of biomedical research
González-Márquez, R., Schmidt, L., Schmidt, B. M., Berens, P., & Kobak, D. (2024) · Patterns · doi:10.1016/j.patter.2024.100968 · Cited by 89 · cited 5×
“We assigned papers to the 39 fields taken from Gonz´alez-M´arquez et al. (2024) based on the journal names: for example, all papers from all journals containing the word neuroscience (such as The Journal of Neuroscience or Nature Neuro-science ) were assigned to the ‘neuroscience’ field.”
InfluentialSelf-citationAI tools can improve equity in science
Berdejo-Espinola, V., & Amano, T. (2023) · Science · doi:10.1126/science.adg9714 · Cited by 83
Liang, W., Izzo, Z., Zhang, Y., Lepp, H., Cao, H., Zhao, X., Chen, L., Ye, H., Liu, S., Huang, Z., et al. (2024) · arXiv · doi:10.48550/arxiv.2403.07183 · Cited by 73
Universals versus historical contingencies in lexical evolution
Bochkarev, V. V., Solovyev, V. D., & Wichmann, S. (2014) · Journal of The Royal Society Interface · doi:10.1098/rsif.2014.0841 · Cited by 70 · cited 1×
“Major events like wars and revolutions affect word frequency distributions in text corpora (Bochkarev et al., 2014).”
McCoy, R. T., Smolensky, P., Linzen, T., Gao, J., & Celikyilmaz, A. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00567 · Cited by 70
McCoy, R. T., Smolensky, P., Linzen, T., Gao, J., & Celikyilmaz, A. (2021) · arXiv · doi:10.48550/arxiv.2111.09509 · Cited by 70
(In)citing Action to Realize an Equitable Future
Dworkin, J., Zurn, P., & Bassett, D. S. (2020) · Neuron · doi:10.1016/j.neuron.2020.05.011 · Cited by 69 · cited 1×
“For instance, all LLM-generated introductions on a certain topic might sound the same and would contain the same set of ideas and references, thereby missing out on innovations (Nakadai et al., 2023) and exacerbating citation injustice (Dworkin et al., 2020).”
Journals take up arms against AI-written text
Brainard, J. (2023) · Science · doi:10.1126/science.adh2762 · Cited by 67 · cited 1×
“As a result, publishers and funding agencies have put out various policies, banning LLMs in peer review (Kaiser, 2023; Brainard, 2023), as co-authors (Thorp, 2023), or undis-closed resource of any kind (Brainard, 2023).”
To protect science, we must use LLMs as zero-shot translators
Mittelstadt, B., Wachter, S., & Russell, C. (2023) · Nature Human Behaviour · doi:10.1038/s41562-023-01744-0 · Cited by 58 · cited 3×
“…LLMs are infamous for making up references (Walters and Wilder, 2023), providing inaccurate summaries (Tang et al., 2024a; Kim et al., 2024), and making false claims that sound authoritative and convincing (Mittelstadt et al., 2023; Ji et al., 2023; Zhang et al., 2023; Zheng and Zhan, 2023).”
Mapping the Increasing Use of LLMs in Scientific Papers
Liang, W., Zhang, Y., Wu, Z., Lepp, H., Ji, W., Zhao, X., Cao, H., Liu, S., He, S., Huang, Z., et al. (2024) · arXiv · doi:10.48550/arxiv.2404.01268 · Cited by 55 · cited 2×
“For example, Liang et al. (2024b) estimated the fraction of LLM-assisted papers in early 2024 to vary between 7% for Nature Portfolio papers and 17% for computer science preprints.”
Linguistic Variation and Change in 250 Years of English Scientific Writing: A Data-Driven Approach
Bizzoni, Y., Degaetano‐Ortlieb, S., Fankhauser, P., & Teich, E. (2020) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2020.00073 · Cited by 50 · cited 1×
“The rise and fall of scientific disciplines is traceable in scholarly writing (Hall et al., 2008; Bizzoni et al., 2020).”
Alvero, A. J., Lee, J., Regla-Vargas, A., Kizilcec, R. F., Joachims, T., & Antonio, A. L. (2024) · Journal of Big Data · doi:10.1186/s40537-024-00986-7 · Cited by 48 · cited 1×
“This makes LLM outputs less diverse and novel than human-written text (Padmakumar and He, 2023; Alvero et al., 2024).”
Analysing the impact of ChatGPT in research
Picazo-Sanchez, P., & Ortiz-Martin, L. (2024) · Applied Intelligence · doi:10.1007/s10489-024-05298-0 · Cited by 46 · cited 1×
“Some studies have reported differences in estimated LLM usage between English-and non-English-speaking countries (Cheng et al., 2024; Liu and Bu, 2024; Picazo-Sanchez and Ortiz-Martin, 2024), academic fields (Akram, 2024), and publishing venues.”
Use of large language models might affect our cognitive skills
Heersmink, R. (2024) · Nature Human Behaviour · doi:10.1038/s41562-024-01859-y · Cited by 44 · cited 1×
“Our analysis can inform the necessary debate around LLM policies by providing a measurement method for LLM usage that is urgently needed (Brinkmann et al., 2023; Heersmink, 2024).”
The future of academic publishing
Ahmed, A., Al-Khatib, A., Boum, Y., Debat, H., Gurmendi Dunkelberg, A., Hinchliffe, L. J., Jarrad, F., Mastroianni, A., Mineault, P., Pennington, C. R., et al. (2023) · Nature Human Behaviour · doi:10.1038/s41562-023-01637-2 · Cited by 42 · cited 2×
“Some have suggested to use retrieval-augmented LLMs that provide verifiable facts from trusted sources (Lewis et al., 2020; Borgeaud et al., 2022; Ahmed et al., 2023) or let the user provide all relevant facts to the LLM in order to protect scientific literature from accumulating subtle…”
Generative AI has a language problem
Choudhury, M. (2023) · Nature Human Behaviour · doi:10.1038/s41562-023-01716-4 · Cited by 33 · cited 1×
“Furthermore, LLMs can mimic biases and other deficiencies from their training data (Bender et al., 2021; Nav-igli et al., 2023; Bai et al., 2024; Choudhury, 2023), or even outright plagiarise (McCoy et al., 2023).”
ChatGPT "contamination": estimating the prevalence of LLMs in the scholarly literature
Gray, A. (2024) · arXiv · doi:10.48550/arxiv.2403.16887 · Cited by 31 · cited 1×
“Gray (2024) described a 2-fold increase in frequency for the words intricate and meticulously in 2023, while Liang et al. (2024b) identified pivotal , intricate , showcasing , and realm as the top LLM-preferred words based on a corpus of LLM-generated text.”
TofuEval: Evaluating Hallucinations of LLMs on Topic-Focused Dialogue Summarization
Tang, L., Shalyminov, I., Wong, A., Burnsky, J., Vincent, J., Yang, Y., Singh, S., Feng, S., Song, H., Su, H., et al. (2024) · Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) · doi:10.18653/v1/2024.naacl-long.251 · Cited by 26
Detecting LLM-assisted writing in scientific communication: Are we there yet?
Lazebnik, T., & Rosenfeld, A. (2024) · Journal of Data and Information Science · doi:10.2478/jdis-2024-0020 · Cited by 23
Does Writing with Language Models Reduce Content Diversity?
Padmakumar, V., & He, H. (2023) · arXiv · doi:10.48550/arxiv.2309.05196 · Cited by 21
AI language tools risk scientific diversity and innovation
Nakadai, R., Nakawake, Y., & Shibasaki, S. (2023) · Nature Human Behaviour · doi:10.1038/s41562-023-01652-3 · Cited by 20 · cited 1×
“For instance, all LLM-generated introductions on a certain topic might sound the same and would contain the same set of ideas and references, thereby missing out on innovations (Nakadai et al., 2023) and exacerbating citation injustice (Dworkin et al., 2020).”
Funding agencies say no to AI peer review
Kaiser, J. (2023) · Science · doi:10.1126/science.adj8309 · Cited by 18 · cited 1×
“As a result, publishers and funding agencies have put out various policies, banning LLMs in peer review (Kaiser, 2023; Brainard, 2023), as co-authors (Thorp, 2023), or undis-closed resource of any kind (Brainard, 2023).”
Measuring Implicit Bias in Explicitly Unbiased Large Language Models
Bai, X., Wang, A., Sucholutsky, I., & Griffiths, T. L. (2024) · arXiv · doi:10.48550/arxiv.2402.04105 · Cited by 17
Cheng, H., Sheng, B., Lee, A., Chaudary, V., Atanasov, A. G., Liu, N., Qiu, Y., Wong, T. Y., Tham, Y.-C., & Zheng, Y. (2024) · doi:10.1101/2024.03.25.586710 · Cited by 15 · cited 1×
“Some studies have reported differences in estimated LLM usage between English-and non-English-speaking countries (Cheng et al., 2024; Liu and Bu, 2024; Picazo-Sanchez and Ortiz-Martin, 2024), academic fields (Akram, 2024), and publishing venues.”
Alvero, A., Lee, J., Regla-Vargas, A., et al. (2024) · doi:10.31235/osf.io/qfx4a · Cited by 11
LLMs are not ready for editorial work
Lindsay, G. W. (2023) · Nature Human Behaviour · doi:10.1038/s41562-023-01730-6 · Cited by 8 · cited 1×
“Others think that for certain tasks like peer reviewing, LLMs are ill-suited and should not be used at all (Lindsay, 2023).”
FABLES: Evaluating faithfulness and content selection in book-length summarization
Kim, Y., Chang, Y., Karpinska, M., Garimella, A., Manjunatha, V., Lo, K., Goyal, T., & Iyyer, M. (2024) · arXiv · doi:10.48550/arxiv.2404.01261 · Cited by 4 · cited 1×
“However, LLMs are infamous for making up references (Walters and Wilder, 2023), providing inaccurate summaries (Tang et al., 2024a; Kim et al., 2024), and making false claims that sound authoritative and convincing (Mittelstadt et al., 2023; Ji et al., 2023; Zhang et al., 2023; Zheng and Zhan,…”
Liu, J., & Bu, Y. (2024) · arXiv · doi:10.48550/arxiv.2404.15799 · Cited by 2
TofuEval: Evaluating Hallucinations of LLMs on Topic-Focused Dialogue SummarizationPreprint
Tang, L., Shalyminov, I., Wong, A. W., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2402.13249 · Cited by 2
Detecting LLM-assisted writing in scientific communication: Are we there yet?
Lazebnik, T., & Rosenfeld, A. (2024) · arXiv · doi:10.48550/arxiv.2401.16807
Delving into the Utilisation of ChatGPT in Scientific Publications in Astronomy
Astarita, S., Kruk, S., Reerink, J., et al. (2024) · arXiv · doi:10.48550/arxiv.2406.17324
Research works printed but not linked: 1
- M. Geng R. Trotta “Is ChatGPT transforming academics’ writing style?” in Next Generation of AI Safety Workshop at ICML 2024 Vienna Austria 26 July 2024.
Cited by
AI and Its Impact on Creativity and Diversity: An Empirical Study of LLM-Generated Product Ideas
Terwiesch, C., Meincke, L., Girotra, K., Mollick, E. R., Nave, G., & Ulrich, K. T. (2023) · doi:10.2139/ssrn.4526071 · Cited by 174
Artificial intelligence tools expand scientists' impact but contract science's focus
Hao, Q., Xu, F., Li, Y., & Evans, J. (2026) · Nature · doi:10.1038/s41586-025-09922-y · Cited by 117
“Recent developments in large language models have also become increasingly used to assist scientific writing 23–26 and facilitate the distillation of scientific findings, but they raise concerns about weakened confidence in AI-generated content 21,22,27 .”
Resisting the Algorithmic Management of Science: Craft and Community After Generative AI
Bechky, B. A., & Davis, G. F. (2024) · Administrative Science Quarterly · doi:10.1177/00018392241304403 · Cited by 111
“One article opened with ‘‘Certainly, here is a possible introduction to your topic’’ (Coddington, 2024; see Kobak et al., 2024 for a systematic assessment in the medical literature).”
‘Stamp out paper mills’ — science sleuths on how to fight fake research
Abalkina, A., Aquarius, R., Bik, E., Bimler, D., Bishop, D., Byrne, J., Cabanac, G., Day, A., Labbé, C., & Wise, N. (2025) · Nature · doi:10.1038/d41586-025-00212-1 · Cited by 59
A Primer for Evaluating Large Language Models in Social-Science Research
Abdurahman, S., Ziabari, A. S., Moore, A. K., Bartels, D. M., & Dehghani, M. (2025) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459251325174 · Cited by 98
Metadata
- Authors
- Kobak, D., González-Márquez, R., Horvát, E.-Á., & Lause, J.
- Affiliations
- University of Tübingen (DE)
- Northwestern University (US)
- Corresponding author
- Kobak, Dmitry
- González-Márquez, Rita
- Horvát, Emőke-Ágnes
- Lause, Jan
- Venue
- Science Advances
- Published
- 2025-07-04
- Volume
- 11
- Issue
- 27
- Pages
- eadt3813-eadt3813
- ISSN
- 2375-2548 (ISSN-L)
- PMC
- PMC12219543
- OpenAlex
- W4411948293
- Cited by
- 213
- References
- 47
- Citations by year
- 2026: 145 · 2025: 54
- FWCI
- 126.21
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Science Advances (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by-sa)
- arXiv (Cornell University)
- Universitätsbibliothek Tübingen
- Funders
- National Science Foundation (1943506, CAREER)
- International Max Planck Research School for Environmental, Cellular and Molecular Microbiology
- Deutsche Forschungsgemeinschaft (390727645, KO6282/2-1)
- Gemeinnützige Hertie-Stiftung
- International Max Planck Research School for Advanced Methods in Process and Systems Engineering
- Leibniz-Gemeinschaft
- Topics
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Social Sciences › Library and Information Sciences › Academic Writing and Publishing
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Keywords
- large language models, ChatGPT, biomedical abstracts, scientific writing, scholarly publishing
- MeSH terms
- Humans, Language, Publications, Vocabulary, Writing, Biomedical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.