The state of artificial intelligence in medical research: A survey of corresponding authors from top medical journals
Open accessAbstract
Natural Language Processing (NLP) is a subset of artificial intelligence that enables machines to understand and respond to human language through Large Language Models (LLMs)‥ These models have diverse applications in fields such as medical research, scientific writing, and publishing, but concerns such as hallucination, ethical issues, bias, and cybersecurity need to be addressed. To understand the scientific community’s understanding and perspective on the role of Artificial Intelligence (AI) in research and authorship, a survey was designed for corresponding authors in top medical journals. An online survey was conducted from July 13 th , 2023, to September 1 st , 2023, using the SurveyMonkey web instrument, and the population of interest were corresponding authors who published in 2022 in the 15 highest-impact medical journals, as ranked by the Journal Citation Report. The survey link has been sent to all the identified corresponding authors by mail. A total of 266 authors answered, and 236 entered the final analysis. Most of the researchers (40.6%) reported having moderate familiarity with artificial intelligence, while a minority (4.4%) had no associated knowledge. Furthermore, the vast majority (79.0%) believe that artificial intelligence will play a major role in the future of research. Of note, no correlation between academic metrics and artificial intelligence knowledge or confidence was found. The results indicate that although researchers have varying degrees of familiarity with artificial intelligence, its use in scientific research is still in its early phases. Despite lacking formal AI training, many scholars publishing in high-impact journals have started integrating such technologies into their projects, including rephrasing, translation, and proofreading tasks. Efforts should focus on providing training for their effective use, establishing guidelines by journal editors, and creating software applications that bundle multiple integrated tools into a single platform.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
24% of authors in the medical sciences (n = 229) used LLMs for rephrasing, proofreading, or translation.
“while 24% of authors in the medical sciences (n = 229) used LLMs for rephrasing, proofreading, or translation (Salvagno et al., 2024).”
From How much are LLMs changing the language of academic papers after ChatGPT? A multi-database and full text analysis (2026) · cites this paper as Salvagno et al., 2024
The Results report that 55 of 229 respondents (24.02%) used AI tools in scientific content creation. They also report rephrasing, proofreading, and translation among the uses of those tools, but only subsets of the 55 reported each task; the paper does not establish that all 24% used LLMs for those purposes. (LLM, conf=0.88)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Dwivedi, Y. K., Hughes, L., Ismagilova, E., Aarts, G., Coombs, C., Crick, T., Duan, Y., Dwivedi, R., Edwards, J., Eirug, A., et al. (2021) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2019.08.002 · Cited by 4,611 · see citing works
Sallam, M. (2023) · Healthcare · doi:10.3390/healthcare11060887 · Cited by 3,126 · cited 2× · see citing works
“However, AI output is far from being perfect, as AI hallucination has been well described and documented in the current literature [8, 9].”
BackgroundAyers, J. W., Poliak, A., Dredze, M., Leas, E. C., Zhu, Z., Kelley, J. B., Faix, D. J., Goodman, A. M., Longhurst, C. A., Hogarth, M., et al. (2023) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2023.1838 · Cited by 2,613 · cited 1× · see citing works
“Even though literature tells us that AI could be more empathetic than human doctors [41], for the moment, everyone agrees.”
BackgroundAli, S., Abuhmed, T., El-Sappagh, S., Muhammad, K., Alonso-Moral, J. M., Confalonieri, R., Guidotti, R., Del Ser, J., Díaz-Rodríguez, N., & Herrera, F. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101805 · Cited by 1,762 · cited 1× · see citing works
“However, even though the basic use appears straightforward, a lack of comprehension may lead individuals to commit unnoticed errors with these tools, stemming from an unawareness of their own knowledge gaps [11].”
BackgroundPrivacy and artificial intelligence: challenges for protecting health information in a new era
Murdoch, B. (2021) · BMC Medical Ethics · doi:10.1186/s12910-021-00687-3 · Cited by 991 · see citing works
The current and future state of AI interpretation of medical images
Rajpurkar, P., & Lungren, M. P. (2023) · New England Journal of Medicine · doi:10.1056/nejmra2301725 · Cited by 443 · see citing works
Bias in AI-based models for medical applications: challenges and mitigation strategies
Mittermaier, M., Raza, M. M., & Kvedar, J. C. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00858-z · Cited by 372 · cited 1× · see citing works
“This is particularly concerning in social sciences and medicine, where biased conclusions can have far-reaching implications [27].”
BackgroundKiseleva, A., Kotzinos, D., & De Hert, P. (2022) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2022.879603 · Cited by 278 · see citing works
The ethics of disclosing the use of artificial intelligence tools in writing scholarly manuscripts
Hosseini, M., Resnik, D. B., & Holmes, K. (2023) · Research Ethics · doi:10.1177/17470161231180449 · Cited by 253 · see citing works
Temsah, M.-H., Aljamaan, F., Malki, K. H., Alhasan, K., Altamimi, I., Aljarbou, R., Bazuhair, F., Alsubaihin, A., Abdulmajeed, N., Alshahrani, F. S., et al. (2023) · Healthcare · doi:10.3390/healthcare11131812 · Cited by 179 · see citing works
Artificial intelligence hallucinations
Salvagno, M., Taccone, F. S., & Gerli, A. G. (2023) · Critical Care · doi:10.1186/s13054-023-04473-y · Cited by 136 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 30
- [1]De Simone B, Di Saverio S. Invited Commentary: Artificial Intelligence in Surgical Care: We Must Overcome Ethical Boundaries. J Am Coll Surg 2022; 235: 275– 7. doi: 10.1097/XCS.0000000000000227 35839402 · DOI
- [2]Gebran A, Vapsi A, Maurer LR, et al. POTTER-ICU: An artificial intelligence smartphone-accessible tool to predict the need for intensive care after emergency surgery. Surgery 2022; 172: 470– 5. doi: 10.1016/j.surg.2022.03.023 35489978 · DOI
- [3]Elhazmi A, Al-Omari A, Sallam H, et al. Machine learning decision tree algorithm role for predicting mortality in critically ill adult COVID-19 patients admitted to the ICU. J Infect Public Health 2022; 15: 826– 34. doi: 10.1016/j.jiph.2022.06.008 35759808 PMC9212964 · DOI
- [4]Maciąg TT, van Amsterdam K, Ballast A, et al. Machine learning in anesthesiology: Detecting adverse events in clinical practice. Health Informatics J 2022; 28. 10.1177/14604582221112855. 35801667 · DOI
- [5]Salvagno M, ChatGPT, Taccone FS, et al. Can artificial intelligence help for scientific writing? Crit Care 2023; 27: 75. doi: 10.1186/s13054-023-04380-2 36841840 PMC9960412 · DOI · Paper page
- [6]Huang J, Tan M. The role of ChatGPT in scientific communication: writing better scientific review articles. Am J Cancer Res 2023; 13: 1148. 37168339 PMC10164801
- [8]De Cassai A, Dost B. Concerns regarding the uncritical use of ChatGPT: a critical analysis of AI-generated references in the context of regional anesthesia. Reg Anesth Pain Med 2023. 10.1136/RAPM-2023-104771. 37380197 · DOI
- [10]Peterson R A.. Constructing Effective Questionnaires. Constructing Effective Questionnaires 2000. 10.4135/9781483349022. · DOI
- [12]Huang J, Tan M. The role of ChatGPT in scientific communication: writing better scientific review articles. Am J Cancer Res 2023; 13: 1148. 37168339 PMC10164801
- [13]Flanagin A, Kendall-Taylor J, Bibbins-Domingo K. Guidance for Authors, Peer Reviewers, and Editors on Use of AI, Language Models, and Chatbots. JAMA 2023; 330: 702– 3. doi: 10.1001/jama.2023.12500 37498593 · DOI · Paper page
- [14]Artificial Intelligence (AI) | Nature Portfolio n.d. https://www.nature.com/nature-portfolio/editorial-policies/ai (accessed April 15, 2024).
- [15]Shickel B, Silva B, Ozrazgat-Baslanti T, et al. Multi-dimensional patient acuity estimation with longitudinal EHR tokenization and flexible transformer networks. Front Digit Health 2022; 4: 1029191. doi: 10.3389/fdgth.2022.1029191 36440460 PMC9682245 · DOI
- [16]Bhandari M, Reddiboina M. Augmented intelligence: A synergy between man and the machine. Indian J Urol 2019; 35: 89. doi: 10.4103/iju.IJU_74_19 31000911 PMC6458810 · DOI
- [17]De Felice F, Petrillo A, De Luca C, et al. Artificial Intelligence or Augmented Intelligence? Impact on our lives, rights and ethics. Procedia Comput Sci 2022; 200: 1846– 56. 10.1016/J.PROCS.2022.01.385. · DOI
- [18]Bazoukis G, Hall J, Loscalzo J, et al. The inclusion of augmented intelligence in medicine: A framework for successful implementation. Cell Rep Med 2022; 3. 10.1016/J.XCRM.2021.100485. PMC8784713 35106506 · DOI
- [20]Azamfirei R, Kudchadkar SR, Fackler J. Large language models and the perils of their hallucinations. Crit Care 2023; 27: 1– 2. 10.1186/S13054-023-04393-X/METRICS. 36945051 PMC10032023 · DOI
- [22]Oosterhoff JHF, Doornberg JN. Artificial intelligence in orthopaedics: false hope or not? A narrative review along the line of Gartner’s hype cycle. EFORT Open Rev 2020; 5: 593– 603. doi: 10.1302/2058-5241.5.190092 33204501 PMC7608572 · DOI
- [23]Wadden JJ. Defining the undefinable: the black box problem in healthcare artificial intelligence. J Med Ethics 2021; 48: 764– 8. doi: 10.1136/medethics-2021-107529 34290113 · DOI · Paper page
- [26]Salvagno M, Taccone FS. Disrupting the “Animal Farm” of scientific research and publishing: AI as an impartial sidekick. Crit Care 2023; 27: 365. doi: 10.1186/s13054-023-04647-8 37740219 PMC10517559 · DOI
- [28]Naik N, Hameed BMZ, Shetty DK, et al. Legal and Ethical Consideration in Artificial Intelligence in Healthcare: Who Takes Responsibility? Front Surg 2022; 9: 862322. doi: 10.3389/fsurg.2022.862322 35360424 PMC8963864 · DOI · Paper page
- [29]Elliott D, Soifer E. AI Technologies, Privacy, and Security. Front ArtifIntell 2022; 5: 826737. doi: 10.3389/frai.2022.826737 35493613 PMC9044077 · DOI
- [32]Bouza L, Bugeau A, Lannelongue L. How to estimate carbon footprint when training deep learning models? A guide and review. Environ Res Commun2023; 5: 115014. doi: 10.1088/2515-7620/acf81b 38022395 PMC10661046 · DOI
- [33]Tse G, Lee Q, Chou OHI, et al. Healthcare Big Data in Hong Kong: Development and Implementation of Artificial Intelligence-Enhanced Predictive Models for Risk Stratification. CurrProblCardiol 2024; 49. 10.1016/J.CPCARDIOL.2023.102168. 37871712 · DOI
- [34]Bajjad AA, Gupta S, Agarwal S, et al. Use of artificial intelligence in determination of bone age of the healthy individuals: A scoping review. J World Fed Orthod 2024; 13. doi: 10.1016/j.ejwf.2023.10.001 37968159 · DOI
- [36]Dash S, Shakyawar SK, Sharma M, et al. Big data in healthcare: management, analysis and future prospects. J Big Data 2019; 6: 1– 25. 10.1186/S40537-019-0217-0/FIGURES/6. · DOI
- [37]Boeken T, Feydy J, Lecler A, et al. Artificial intelligence in diagnostic and interventional radiology: Where are we now? DiagnInterv Imaging 2023; 104: 1– 5. doi: 10.1016/j.diii.2022.11.004 36494290 · DOI
- [38]Lind Plesner L, Müller FC, Brejnebøl MW, et al. Commercially Available Chest Radiograph AI Tools for Detecting Airspace Disease, Pneumothorax, and Pleural Effusion. Radiology 2023; 308: e231236. doi: 10.1148/radiol.231236 37750768 · DOI
- [39]Piorkowski A, Obuchowicz R, Najjar R. Redefining Radiology: A Review of Artificial Intelligence Integration in Medical Imaging. Diagnostics 2023, Vol 13, Page 2760 2023;13:2760. 10.3390/DIAGNOSTICS13172760. PMC10487271 37685300 · DOI
- [40]Kulkarni PA, Singh H. Artificial Intelligence in Clinical Diagnosis: Opportunities, Challenges, and Hype. JAMA 2023; 330: 317– 8. doi: 10.1001/jama.2023.11440 37410477 · DOI · Paper page
- [42]Barnhart BJ, Reddy SG, Arnold GK. Remind Me Again: Physician Response to Web Surveys: The Effect of Email Reminders Across 11 Opinion Survey Efforts at the American Board of Internal Medicine from 2017 to 2019. Eval Health Prof 2021; 44: 245– 59. doi: 10.1177/01632787211019445 34008437 · DOI
Cited by
Sorted by citations
Kousha, K., & Thelwall, M. (2026) · Scientometrics · doi:10.1007/s11192-026-05601-5 · Cited by 10 · see citing works
Hallucinations in large language models within academic contexts: A systematic review.
Lai, K. E., Mustaffa, N., Preece, C. N., & Wong, F. Y. (2026) · Journal of Science and Technology Policy Management · doi:10.1108/jstpm-01-2026-0015 · Cited by 1 · see citing works
- Authors
- Salvagno, M., Cassai, A. D., Zorzi, S., Zaccarelli, M., Pasetto, M., Sterchele, E. D., Chumachenko, D., Gerli, A. G., Azamfirei, R., & Taccone, F. S.
- Affiliations
- Ospedale Sant Antonio (IT)
- University of Waterloo (CA)
- University of Milan (IT)
- Corresponding author
- Cassai, Alessandro De
- ORCID
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2024-08-23
- Volume
- 19
- Issue
- 8
- Pages
- e0309208
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC11343420
- OpenAlex
- W4401806632
- Cited by
- 32
- References
- 11
- Citations by year
- 2026: 15 · 2025: 12 · 2024: 5
- FWCI
- 2.98
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Padua Research Archive (University of Padova) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
- Social Sciences › Arts and Humanities › History and Philosophy of Science › Academic Writing and Publishing
- Keywords
- large language models, natural language processing, corresponding authors, authorship, AI familiarity, AI in scientific writing, AI in scientific publishing, journal editorial guidelines, AI hallucination
- MeSH terms
- Artificial Intelligence, Authorship, Humans, Natural Language Processing, Periodicals as Topic, Surveys and Questionnaires, Biomedical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
