Using ChatGPT-4 to Create Structured Medical Notes From Audio Recordings of Physician-Patient Encounters: Comparative Study
Open accessPublished version of a preprint
The DOI matches its Crossref record; 28 of the 29 references it cites were found in a registry or an index, and 1 are not checked yet.
Abstract
Background: Medical documentation plays a crucial role in clinical practice, facilitating accurate patient management and communication among health care professionals. However, inaccuracies in medical notes can lead to miscommunication and diagnostic errors. Additionally, the demands of documentation contribute to physician burnout. Although intermediaries like medical scribes and speech recognition software have been used to ease this burden, they have limitations in terms of accuracy and addressing provider-specific metrics. The integration of ambient artificial intelligence (AI)–powered solutions offers a promising way to improve documentation while fitting seamlessly into existing workflows. Objective: This study aims to assess the accuracy and quality of Subjective, Objective, Assessment, and Plan (SOAP) notes generated by ChatGPT-4, an AI model, using established transcripts of History and Physical Examination as the gold standard. We seek to identify potential errors and evaluate the model’s performance across different categories. Methods: We conducted simulated patient-provider encounters representing various ambulatory specialties and transcribed the audio files. Key reportable elements were identified, and ChatGPT-4 was used to generate SOAP notes based on these transcripts. Three versions of each note were created and compared to the gold standard via chart review; errors generated from the comparison were categorized as omissions, incorrect information, or additions. We compared the accuracy of data elements across versions, transcript length, and data categories. Additionally, we assessed note quality using the Physician Documentation Quality Instrument (PDQI) scoring system. Results: Although ChatGPT-4 consistently generated SOAP-style notes, there were, on average, 23.6 errors per clinical case, with errors of omission (86%) being the most common, followed by addition errors (10.5%) and inclusion of incorrect facts (3.2%). There was significant variance between replicates of the same case, with only 52.9% of data elements reported correctly across all 3 replicates. The accuracy of data elements varied across cases, with the highest accuracy observed in the “Objective” section. Consequently, the measure of note quality, assessed by PDQI, demonstrated intra- and intercase variance. Finally, the accuracy of ChatGPT-4 was inversely correlated to both the transcript length (P=.05) and the number of scorable data elements (P=.05). Conclusions: Our study reveals substantial variability in errors, accuracy, and note quality generated by ChatGPT-4. Errors were not limited to specific sections, and the inconsistency in error types across replicates complicated predictability. Transcript length and data complexity were inversely correlated with note accuracy, raising concerns about the model’s effectiveness in handling complex medical cases. The quality and reliability of clinical notes produced by ChatGPT-4 do not meet the standards required for clinical use. Although AI holds promise in health care, caution should be exercised before widespread adoption. Further research is needed to address accuracy, variability, and potential errors. ChatGPT-4, while valuable in various applications, should not be considered a safe alternative to human-generated clinical documentation at this time.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures






Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Our simulation approach facilitates objective, replicable evaluation of the quality, accuracy, and safety of both ADS-generated transcripts and clinical notes.”
From Evaluating the Quality and Safety of Ambient Digital Scribe Platforms Using Simulated Ambulatory Encounters (2025) · cites this paper as [16]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“When compared to previous literature AI-driven documentation varied between “bad (29.7)” and “excellent (35.6)” highlighting the variability in documentation produced by AI systems [23, 25, 28].”
From Artificial Intelligence (AI) – Powered Documentation Systems in Healthcare: A Systematic Review (2025) · cites this paper as [25]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“To assess clinical note accuracy and completeness, we utilized standardized scoring rubrics of key clinical elements previously developed by 2 clinical experts based on simulated encounter recordings and clinical context.16”
From Evaluating the Quality and Safety of Ambient Digital Scribe Platforms Using Simulated Ambulatory Encounters (2025) · cites this paper as [16]
The cited source says what the claim attributes to it.
In the citing paper
“In our previous work using ChatGPT to generate clinical notes using uploaded transcripts, we identified potential accuracy and consistency concerns, with only 53% of clinical elements reported correctly across 3 replicates.16”
From Evaluating the Quality and Safety of Ambient Digital Scribe Platforms Using Simulated Ambulatory Encounters (2025) · cites this paper as [16]
The cited source says what the claim attributes to it.
In the citing paper
“We have previously demonstrated the utility of this method in assessing the quality and accuracy of notes generated by human scribes and ChatGPT4.16,34”
From Evaluating the Quality and Safety of Ambient Digital Scribe Platforms Using Simulated Ambulatory Encounters (2025) · cites this paper as [16]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The impact of artificial intelligence in medicine on the future role of the physician
Ahuja, A. S. (2019) · PeerJ · doi:10.7717/peerj.7702 · Cited by 789 · cited 1×
“With the rapid and exponential growth in computing capacity, artificial intelligence (AI) is being increasingly used in health care, holding the promise of revolutionizing medical documentation, thus potentially alleviating the burden on physicians [6].”
Johnson, D., Goodman, R., Patrinely, J., Stone, C., Zimmerman, E., Donald, R., Chang, S., Berkowitz, S., Finn, A., Jahangir, E., et al. (2023) · doi:10.21203/rs.3.rs-2566942/v1 · Cited by 598 · cited 1×
“In the realm of ChatGPT-4, the study conducted by Johnson et al [15] delved into its performance in giving precise and comprehensive medical information.”
The Association of Standardized Patient Educators (ASPE) Standards of Best Practice (SOBP)
Lewis, K. L., Bohnert, C. A., Gammon, W. L., Hölzer, H., Lyman, L., Smith, C., Thompson, T. M., Wallace, A., & Gliva-McConvey, G. (2017) · Advances in Simulation · doi:10.1186/s41077-017-0043-4 · Cited by 580 · cited 1×
“Briefly, for each case, a storyboard was created by subject matter experts and used for training the standardized patient to ensure standard content delivery according to best practices [11].”
Walker, H. L., Ghani, S., Kuemmerli, C., Nebiker, C. A., Müller, B. P., Raptis, D. A., & Staubli, S. M. (2023) · Journal of Medical Internet Research · doi:10.2196/47479 · Cited by 363 · cited 1×
“However, in another study by Walker et al [17] aimed at evaluating the reliability of medical information provided by ChatGPT-4, multiple iterations of their queries executed through the model yielded a remarkable 100% internal consistency among the generated outputs [17].”
Hirosawa, T., Harada, Y., Yokose, M., Sakamoto, T., Kawamura, R., & Shimizu, T. (2023) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph20043378 · Cited by 343 · cited 1×
“Health care providers envision numerous applications for ChatGPT-4, such as answering patient questions, automated insurance prior authorizations, and creating differential diagnoses [9,10].”
Association of Electronic Health Record Design and Use Factors With Clinician Stress and Burnout
Kroth, P. J., Morioka-Douglas, N., Veres, S., Babbott, S., Poplau, S., Qeadan, F., Parshall, C., Corrigan, K., & Linzer, M. (2019) · JAMA Network Open · doi:10.1001/jamanetworkopen.2019.9609 · Cited by 333 · cited 1×
“Acknowledging the increasing documentation burden contributing to physician burnout, generative AI technology for clinical note documentation may save time [2,19].”
Mobile, social, and wearable computing and the evolution of psychological practice.
Morris, M. E., & Aguilera, A. (2012) · Professional Psychology: Research and Practice · doi:10.1037/a0029041 · Cited by 298
Towards Making the Most of ChatGPT for Machine TranslationPreprint
Peng, K., Ding, L., Zhong, Q., Shen, L., Liu, X., Zhang, M., Ouyang, Y., & Tao, D. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4390455 · Cited by 264 · cited 1×
“With this in mind, ChatGPT-4’s temperature allows for variability, but this setting is not accessible to the end user [25,26].”
An Empirical Study of the Non-Determinism of ChatGPT in Code Generation
Ouyang, S., Zhang, J. M., Harman, M., & Wang, M. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3697010 · Cited by 258
Frequency and Types of Patient-Reported Errors in Electronic Health Record Ambulatory Care Notes
Bell, S. K., Delbanco, T., Elmore, J. G., Fitzgerald, P. S., Fossa, A., Harcourt, K., Leveille, S. G., Payne, T. H., Stametz, R. A., Walker, J., et al. (2020) · JAMA Network Open · doi:10.1001/jamanetworkopen.2020.5867 · Cited by 238 · cited 1×
“In recent years, it has become increasingly evident that inaccuracies in medical notes lead to miscommunication, diagnostic discrepancies, and patients’ perceptions of subpar medical care [1].”
Research works printed but not linked: 2
- Florig, ST, 2021, AMIA Annu Symp Proc, 2021, 457
- Rule, A, 2021, AMIA Annu Symp Proc, 2021, 1059
Cited by
Biswas, A., & Talukdar, W. (2024) · International Journal of Innovative Science and Research Technology (IJISRT) · doi:10.38124/ijisrt/ijisrt24may1483 · Cited by 1,028
Artificial Intelligence (AI) – Powered Documentation Systems in Healthcare: A Systematic Review
Bracken, A., Reilly, C., Feeley, A., Sheehan, E., Merghani, K., & Feeley, I. (2025) · Journal of Medical Systems · doi:10.1007/s10916-025-02157-4 · Cited by 113
“…has been reported with the use of the physician documentation quality instrument (PDQI-9 tool) [34] When compared to previous literature AI-driven documentation varied between “bad (29.7)” and “excellent (35.6)” highlighting the variability in documentation produced by AI systems [23, 25, 28].”
InfluentialThe Impact of AI Scribes on Streamlining Clinical Documentation: A Systematic Review
Sasseville, M., Yousefi, F., Ouellet, S., Naye, F., Stefan, T., Carnovale, V., Bergeron, F., Ling, L., Gheorghiu, B., Hagens, S., et al. (2025) · Healthcare · doi:10.3390/healthcare13121447 · Cited by 122
“One study featured simulated patient-provider encounters across various ambulatory specialties [34].”
InfluentialAI Scribes in Health Care: Balancing Transformative Potential With Responsible Integration
Leung, T. I., Coristine, A. J., & Benis, A. (2025) · JMIR Medical Informatics · doi:10.2196/80898 · Cited by 98
“Other evaluations of stand-alone tools for audio recordings suggest potentially high rates of errors, including incorrect information, omissions, and hallucinations [11,29,38].”
The effect of ambient artificial intelligence notes on provider burnout.
Misurac, J., Knake, L. A., & Blum, J. M. (2025) · Applied Clinical Informatics · doi:10.1055/a-2461-4576 · Cited by 58
“23 Thus, AI technologies must continue to evolve and adapt to the different clinical use cases and needs.”
Lee, C., Britto, S. S., & Diwan, K. (2024) · Cureus · doi:10.7759/cureus.73994 · Cited by 100
Ha, E., Choon-Kon-Yune, I., Murray, L., Luan, S., Montague, E. N. H., Bhattacharyya, O. K., & Agarwal, P. (2025) · JMIR Human Factors · doi:10.2196/71434 · Cited by 50
“Performance also differed significantly across different LLMs [46].”
Sarraf, B., & Ghasempour, A. (2025) · Frontiers in Public Health · doi:10.3389/fpubh.2025.1628831 · Cited by 40
Zhu, S., Hu, W., Zhi, Y., Yan, J., & Zhang, F. (2024) · JMIR Medical Informatics · doi:10.2196/63731 · Cited by 42
Croxford, E., Gao, Y., Pellegrino, N., Wong, K., Wills, G. J., First, E., Schnier, M., Burton, K. A., Ebby, C. G., Gorski, J. K., et al. (2025) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaf068 · Cited by 68
Stephan, D., Bertsch, A., Schumacher, S., Puladi, B., Burwinkel, M., Al‐Nawas, B., Kämmerer, P. W., & Thiem, D. G. E. (2025) · Journal of Medical Internet Research · doi:10.2196/73337 · Cited by 44
“One study [44], for instance, highlighted those errors of omission accounted for 86% of inaccuracies in AI-generated subjective, objective, assessment, and plan notes, underscoring the necessity of refining input frameworks to enhance data completeness and consistency.”
The use of large language models in clinical documentation: A scoping review
Woo, B., Cato, K., Cho, H., You, S. B., & Song, J. (2025) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2025.105322 · Cited by 43
Perspectives and Experiences With Large Language Models in Health Care: Survey Study
Sumner, J., Wang, Y., Tan, S. Y., Chew, E. H. H., & Yip, A. W. (2025) · Journal of Medical Internet Research · doi:10.2196/67383 · Cited by 34
Bellanda, V. C. F., dos Santos, M. L., Ferraz, D., Jorge, R., & Melo, G. B. (2024) · International Journal of Retina and Vitreous · doi:10.1186/s40942-024-00595-9 · Cited by 27
“In versions that accept audio inputs, ChatGPT can process clinical consultation data to generate organized reports and notes, potentially reducing paperwork and freeing resources for other aspects of medical care [29–33].”
BackgroundMethodSaadat, S., Darounkolaei, M. K., Qorbani, M., Hemmat, A., & Hariri, S. (2025) · InfoScience Trends · doi:10.61186/ist.202502.01.01 · Cited by 25
Khanna, A., Wolf, T., Frank, I., et al. (2025) · Journal of the American College of Surgeons · doi:10.1097/xcs.0000000000001352 · Cited by 21
Al-hammuri, K., Gebali, F., & Kanan, A. (2024) · AI · doi:10.3390/ai5030055 · Cited by 38
“At the same time, an AI-based context-aware system can be used to validate data integrity [12,13].”
MethodAnderson, T. N., Mohan, V., Dorr, D. A., Ratwani, R. M., Biro, J. M., & Gold, J. A. (2025) · Mayo Clinic proceedings. Digital health · doi:10.1016/j.mcpdig.2025.100292 · Cited by 17
“16,34 Other authors have utilized similar simulation-based approaches for ADS performance testing, with inconsistent methods for evaluating the resultant output.”
Self-citationvan Breugel, M., Greenhawt, M., Eguíluz‐Gracia, I., Torres, M. J., Anagnostou, A., & Koppelman, G. H. (2025) · Journal of Allergy and Clinical Immunology · doi:10.1016/j.jaci.2025.08.022 · Cited by 22
Atiku, S. O., Olakotan, O. O., & Owolanke, K. (2026) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.70365 · Cited by 17
Woo, B., Song, J., Middleton, E., Fijačko, N., & Cato, K. (2025) · International Nursing Review · doi:10.1111/inr.70102 · Cited by 13
Zhang, Z., Nezhad, M. J. M., Hosseini, S. M. B., Zolnour, A., Zonour, Z., Hosseini, S. M., Topaz, M., & Zolnoori, M. (2025) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2025.106231 · Cited by 18
Yuan, L., Huang, S., Chou, D., & Tsai, C.-Y. (2024) · The World Journal of Men s Health · doi:10.5534/wjmh.240173 · Cited by 14
“As a result, employing simulated patient data to test and evaluate AI models is a commonly used research method in healthcare and medical research [13,14,25,26].”
Nyongesa, C. A., Hogarth, M. A., & Pa, J. (2025) · Journal of Alzheimer s Disease · doi:10.1177/13872877251339756 · Cited by 15
General practitioners’ opinions of generative artificial intelligence in the UK: An online survey
Kharko, A., Sanchez, C. G., Hagström, J., Gaab, J., Locher, C., McMillan, B., Sundemo, D. S., & Blease, C. (2025) · Digital Health · doi:10.1177/20552076251360863 · Cited by 22
“26 Few studies, however, have consulted health-care professionals, especially frontline clinicians, on their use and opinions about AI in clinical practice.”
Nofal, H. A. E., Mohamed, A. E., Almadani, N. A., Mahfouz, R., Bahri, H. A., Ali, H. T., & Elrafey, D. S. (2025) · Frontiers in Public Health · doi:10.3389/fpubh.2025.1639333 · Cited by 19
Impact of acoustic and informational noise on AI-generated clinical summaries
Draper, T. C., Leake, J., Lamb-Riddell, K., Cox, T. M., McCormick, J., Trowell, S., Kiely, J., & Luxton, R. W. (2025) · BMJ Digital Health & AI · doi:10.1136/bmjdhai-2025-000057 · Cited by 10
“…approaches, for example, ROUGE (Recall-Oriented Understudy for Gisting Evaluation) and BERT (Bidirectional Encoder Representations from Transformers) 13–16 and also manual approaches, 12 in which the clinical notes are compared with the original audio file, often by clinically trained personnel.”
InfluentialClinical AI Scribes in primary care: accuracy, error severity and implications for clinical practice
Draper, T. C., Cox, T., Lamb-Riddell, K., Moretti, L. A., McCormick, J., Trowell, S., Kiely, J., & Luxton, R. (2025) · BMJ Digital Health & AI · doi:10.1136/bmjdhai-2025-000092 · Cited by 10
“12 These include automated approaches, eg, ROUGE (Recall-Oriented Understudy for Gisting Evaluation) and BERT (Bidirectional Encoder Repre-sentations from Transformers), 22–25 and also manual approaches 26–28 in which the clinical notes are compared with the ground-truth original audio file.”
Suhail, D., Wong, Z. Y., Ubhi, J. S., Kungwengwe, G., Faderani, R., & Mosahebi, A. (2025) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2025.106113 · Cited by 15
Stanton, N., Aziz, A., Jakhra, S., Wong, S., Morganstein, L., Bassett, P., Brewerton, M., Golchinheydari, S., Brogan, D., & Pushparajah, D. (2025) · BJPsych Bulletin · doi:10.1192/bjb.2025.10186 · Cited by 15
Iannone, S., Kaur, A., & Johnson, K. B. (2026) · Journal of General Internal Medicine · doi:10.1007/s11606-025-09938-0 · Cited by 19
Speech recognition technology in prehospital documentation: A scoping review
Hedderson, D., Courtney, K. L., Monkman, H., & Blanchard, I. E. (2024) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2024.105662 · Cited by 11
Artificial intelligence and physician burnout: A productivity paradox
Goodson, D. A., Garcia, B., Hogarth, M. A., & Tu, S.-P. (2025) · Learning Health Systems · doi:10.1002/lrh2.70013 · Cited by 15
“30,31 Humans signing AI documents must remain vigilant.”
AI for Personalized Learning in Medical Education
Niazai, H., & Monib, W. K. (2025) · IGI Global eBooks · doi:10.4018/979-8-3373-1519-5.ch004 · Cited by 8
Dahlberg, A., Käenniemi, T., Winther-Jensen, T., Tapiola, O., Luisto, R., Puranen, T., Gordon, M., Sanmark, E., & Vartiainen, V. A. (2026) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2026.103421 · Cited by 15
Toal, M. P., Hill, C. J., Quinn, M., O’Neill, C., & Maxwell, A. P. (2025) · Journal of Medical Internet Research · doi:10.2196/73603 · Cited by 10
Chiang, Y.-L., Yang, K.-F., Su, P.-Y., Tsai, S., & Liang, K.-L. (2025) · Applied Clinical Informatics · doi:10.1055/a-2707-2959 · Cited by 7
Hahne, J., & Carpenter, B. D. (2026) · Patient Education and Counseling · doi:10.1016/j.pec.2026.109492 · Cited by 13
Transforming Healthcare through Generative and Agentic AI: A Systematic Review
Assayed, S. K. (2025) · International Conference on Information Technology Trends (ITT) · doi:10.1109/itt69610.2025.11352935 · Cited by 6
Tscholl, D. W., Ebensperger, M., RahrischRahrisch, A., Wang, H. J., Heckel, H., Thomasius, M., Kaserer, A., Grande, B., Seelandt, J., & Kolbe, M. (2026) · Advances in Simulation · doi:10.1186/s41077-026-00407-0 · Cited by 12
“These concerns relate to critiques of AI tools in health-care, where hallucinations and context-insensitive outputs pose risks [70].”
Rahrisch, A., Braun, J., Ganter, M. T., Roche, T. R., Spahn, D. R., Tscholl, D. W., & Kaserer, A. (2026) · Journal of Clinical Anesthesia · doi:10.1016/j.jclinane.2026.112163 · Cited by 10
Vasquez, V. M., McCabe, M. M., McKee, J. C., Siby, S., Hussain, U., Faizuddin, F., Sheikh, A., Nguyen, T., Mayer, G., Grier, J. T., et al. (2025) · Journal of Clinical Medicine · doi:10.3390/jcm14155346 · Cited by 10
“While current AI capture methods for clinical notes remain imperfect [44], the available tools and accuracy continue to improve and can enhance the clinical experience for both the patient and the provider.”
Etagiuri, A., Chaudhry, D., van der Merwe, J., Nickol, M., & van der Merwe, J. M. (2026) · The Journal of Arthroplasty · doi:10.1016/j.arth.2026.03.069 · Cited by 11
Alharbi, F., & Asiri, S. N. (2024) · Applied Sciences · doi:10.3390/app142210323 · Cited by 5
Impact of Ambient Artificial Intelligence Notes on Provider BurnoutPreprint
Misurac, J. M., Knake, L. A., & Blum, J. (2024) · medRxiv · doi:10.1101/2024.07.18.24310656 · Cited by 5
Azmakan, H., & Hashemian, F. (2026) · Journal of the American Pharmacists Association · doi:10.1016/j.japh.2026.103019 · Cited by 4
Sanchez, C. G., Kharko, A., Hägglund, M., Riggare, S. S., & Blease, C. (2025) · JMIR Research Protocols · doi:10.2196/73602 · Cited by 4
Maas, J., Franssen, S., Petković, M., Cano, S. C., Dingemans, A. E., van Oosterzee, A., Landt, M., Talavera, E., Vreeswijk, C. M. J. M., & Simeunovic-Ostojic, M. (2025) · International Journal of Eating Disorders · doi:10.1002/eat.24579 · Cited by 32
HPB SmartNotes: The impact of artificial intelligence on surgeon workload in the outpatient office
Gasque, R. A., Zaietta, N., Mollard, L., Beltrame, M. C., Virreira, M. E. L., Quiñonez, E. G., & Mattera, F. J. (2025) · EngMedicine · doi:10.1016/j.engmed.2025.100101 · Cited by 6
MacKenzie, J., Coric, C., Kim, Y. W., Lee, C., & Grant-Kels, J. M. (2025) · Clinics in Dermatology · doi:10.1016/j.clindermatol.2025.06.002 · Cited by 6
Towards Intelligent Virtual Clerks: AI-Driven Automation for Clinical Data Entry in Dialysis Care
Worragin, P., Chernbumroong, S., Puritat, K., Julrode, P., & Intawong, K. (2025) · Technologies · doi:10.3390/technologies13110530 · Cited by 7
Ambient Scribe Technology in Simulated Patient Encounters Across Specialties
Brunner, J., Morrissey, S. E., Stevens, E. M., Payne, C. M., Wiltz, S. A., Cutrona, S. L., & Rinne, S. T. (2026) · JAMA Network Open · doi:10.1001/jamanetworkopen.2025.52870 · Cited by 3
The Growing Use of Copy-Paste and Its Impact on Creation and Propagation of Documentation Errors
Grattan, R., Florig, S. T., Devra, T., Mohan, V., & Gold, J. A. (2026) · CHEST Pulmonary · doi:10.1016/j.chpulm.2026.100259 · Cited by 3
Self-citationAtiku, S. O., & Olakotan, O. O. (2026) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.70516 · Cited by 5
Kaddari, Z., Hachmi, I. E., Berrich, J., Amrani, R., & Bouchentouf, T. (2024) · Applied Computer Science · doi:10.35784/acs-2024-44 · Cited by 5
Vidal‐Alaball, J., Alonso, C. V., Heinisch, D. H., Castaño, A., Sánchez-Freire, E., Serrano, M. L. B., Pascual, C. F., Menacho, I., Acosta‐Rojas, R., Gubert, O. C., et al. (2025) · JMIR Research Protocols · doi:10.2196/66232 · Cited by 5
“For example, a 2024 study assessed the accuracy of AI-generated clinical notes using ChatGPT-4 (OpenAI), finding that such tools can produce notes comparable in quality to those taken by physicians [23].”
Application and Challenges of Large Language Models in Clinical Nursing: A Systematic Review
Song, J., Liu, W., Wang, Y., Hu, X., Chen, L., Wu, X., Zheng, C., & Gu, Q. (2025) · CIN Computers Informatics Nursing · doi:10.1097/cin.0000000000001328 · Cited by 11
Zhang, Z., Scroggins, J. K., Harkins, S., Hulchafo, I. I., Moen, H., Tadiello, M., Barcelona, V., & Topaz, M. (2025) · Nursing Outlook · doi:10.1016/j.outlook.2025.102472 · Cited by 4
Hershenhouse, J. S., Gill, K. S., & Nabhani, J. A. (2025) · Current Opinion in Urology · doi:10.1097/mou.0000000000001282 · Cited by 4
AI-generated clinical summaries: errors and susceptibility to speech and speaker variability
Draper, T. C., Leake, J., Cox, T., Lamb-Riddell, K., Johns, B. E., McCormick, J., Trowell, S., Kiely, J., & Luxton, R. (2025) · medRxiv · doi:10.1136/bmjhci-2025-101918 · Cited by 4
“Following prior work, 8 10 23 these are defined as: ► Hallucinations: instances where the CAIS tool generated content that was not based on the original audio input, causing false information to be included in the summary.”
Bertin, F., Lawrence, J., Niessen, S., Pinard, C. J., Reagan, K. L., & Rentko, V. T. (2026) · Journal of Veterinary Internal Medicine · doi:10.1093/jvimsj/aalaf036 · Cited by 3
Menz, B. D., Scarfo, N. L., Modi, N. D., Cornelisse, E., Li, L. X., Tan, E., Gandhi, J., Maher, D., Kousa, D., Daniel, K., et al. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-02494-9 · Cited by 6
“Illustrating this limitation, a recent study evaluated 14 consultation notes generated by an audio-only AI scribe 26 .”
Tandon, U. (2026) · Frontiers in Digital Health · doi:10.3389/fdgth.2026.1772345 · Cited by 2
“These inaccuracies may emerge from missing data, transcription errors, and low-quality inputs (29, 30).”
Validating Digital Scribes: A Scoping Review of Evaluation Practices and Clinical Use
Kerimoğlu, E., Notermans, F. V., Silkens, M., de Mul, M., Ahaus, K., & van der Boon, R. M. A. (2026) · Journal of Medical Systems · doi:10.1007/s10916-026-02392-3 · Cited by 3
“…clarity and detail, coherence and flow, structural differences, stylistic variations, most common deletions & insertions [1 ]; Errors: Additions [ 15 ]; Errors: Incorrect statements, Omissions [ 15, 21] Capture surface-level differences by counting character-or word-level transformations needed…”
InfluentialEvaluating large language models in pediatric nephrology
Filler, G., & Niel, O. (2025) · Pediatric Nephrology · doi:10.1007/s00467-025-06729-x · Cited by 2
Round table discussion. Integration of artificial intelligence into daily practice
Miller, R., Kedgley, A. E., Farnebo, S. J., Stockmans, F., Zlotolow, D. A., & Horwitz, M. D. (2025) · Journal of Hand Surgery (European Volume) · doi:10.1177/17531934251342418 · Cited by 1
“…see several projects implementing natural language processing tools in automated real-time transcription of recorded patient/doctor consultations (Kernberg et al., 2024), AI facilitating medical charts and discharge messages (Sa´nchez-Rosenberg et al., 2024) to patients as well as triaging of…”
Rattananan, W. (2025) · Medicina · doi:10.3390/medicina61111931 · Cited by 1
Bains, B., Rapuri, S., Robitaille, E., Wang, J., Khera, A., Gómez, C., Reyes, E., Perry, C., Wilson, J., & Tracey, E. (2025) · JMIR Medical Informatics · doi:10.2196/80205 · Cited by 2
“Kernberg et al [39] reported that 58% of structured medical notes from patient-physician interactions omitted important information.”
Artificial Intelligence-Assisted Scribing for Chronic Pain Care: A Narrative Review
Yost, C. D., Jumreornvong, O., Hasoon, J. J., Ruan, Q. Z., Ang, S. P., Palumbo, P., Bianco, G. L., Yong, R. J., Zetter, G., Christo, P. J., et al. (2025) · Current Pain and Headache Reports · doi:10.1007/s11916-025-01439-3 · Cited by 4
“This premise is supported by research revealing that AI-generated notes using ambient AI tools scored up to 26% higher on critical metrics like completeness, conciseness, and accuracy than physician-written notes [20].”
del Barrio, M., Laos, K., Lara, M. J. V., Goicoechea, C., & Menéndez, H. D. (2026) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2026.103379 · Cited by 3
Klein, A., Song, J., Chim, J., Keren, L., Triantafyllopoulos, A., Schuller, B. W., Liakata, M., & Atzil–Slonim, D. (2026) · JMIR Formative Research · doi:10.2196/71230 · Cited by 1
“Beyond summarization, LLMs have also been used to generate structured medical notes from physician-patient encounters [16], condense electronic health records [17], and extract key information from medical texts while addressing hallucination risks [18].”
A Review on Artificial Intelligence as a Solution to Burnout in Interventional Radiology
Zhang, H., Torkpour, A., Zeidaabadi, B., Ashraf, N., Yazdabadi, A., Iqbal, S. I., Asadi, H., Cazzato, R. L., Morgan, R., & Shaygi, B. (2026) · CardioVascular and Interventional Radiology · doi:10.1007/s00270-026-04440-4 · Cited by 1
“A study on the reliability of ChatGPT-4-generated medical notes showed high numbers of errors (23.6 per case) and a modest note quality (Overall PDQI score = 29.7) [45].”
InfluentialApplications of ChatGPT in the Health Sector: A Systematic Literature Review
Martínez‐Burgos, A., Enamorado-Díaz, E., Gutiérrez, J., & García-García, J. A. (2026) · Expert Systems · doi:10.1111/exsy.70296 · Cited by 1
Implications of artificial intelligence scribe usage
Klufas, T., Hanggodo, S., Lee, C., Negrini, E., Zhou, A. E., & Grant-Kels, J. M. (2025) · Journal of the American Academy of Dermatology · doi:10.1016/j.jaad.2025.04.014 · Cited by 1
Modeling Clinician Preferences in AI-Assisted Drafting: A Human-in-the-Loop Adaptation Theory
Rahman, S., Zaki, A., & Huda, N. (2024) · Journal of Health Informatics and Digital Systems · doi:10.68159/z142477894 · Cited by 1
McCradden, M. D., Tng, S., Jeyaseelan, D., Leane, C., Campbell, M., Braund, T., Earle-Bandaralage, L., Ebrahimi, M., Sharma, A. K., & TANG, J. C. Y. (2025) · medRxiv · doi:10.1101/2025.09.30.25336928 · Cited by 1
Liang, Z., Chen, Y., Madi, N. A., & Liu, C. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791209
Anand, A. S., Ganesan, D., & Karkar, R. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791866 · Cited by 1
“Foundation models (FMs) trained on web-scale data are rapidly entering clinical practice, with systems like OpenAI’s ChatGPT, Anthropic’s Claude, and Google’s Gemini being deployed for tasks ranging from clinical documentation to diagnostic support [39, 50, 85, 104].”
Chow, J. C., Yeh, Y.-T., & Chou, W. (2024) · doi:10.2196/preprints.64656
Ha, E., Choon-Kon-Yune, I., Murray, L., Luan, S., Montague, E. N. H., Bhattacharyya, O. K., & Agarwal, P. (2025) · doi:10.2196/preprints.71434
Yan, Y., Chen, Q., Nasir, R., Griffin, P. M., Bone, C., & Tuan, W. (2025) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2025.109870 · Cited by 2
Rattananan, W. (2025) · medRxiv · doi:10.1101/2025.06.12.25329503
del Barrio, M., Laos, K., Lara, M. J. V., Garcia, C. G., & Menéndez, H. D. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5392429
The Impact of AI Implementation on Healthcare Workers’ Satisfaction and Performance
Galdames, I. S., & Gil, A. M. C. (2025) · Lecture notes in networks and systems · doi:10.1007/978-3-031-99699-3_20
Padovano, M., Longo, F., Padovano, A., Frascio, M., & Nardo, B. (2025) · Procedia Computer Science · doi:10.1016/j.procs.2025.12.117 · Cited by 1
ChatGPT in liver transplantation: Current applications, limitations, and future directions
Avramidou, E., Kougianos, N., Chiotis-Miehe, G., Vasileiadou, S., Katsanos, G., & Tsoulfas, G. (2026) · World Journal of Transplantation · doi:10.5500/wjt.v16.i1.110485 · Cited by 2
Devereux, C., Murphy, C. F., Pallin, N. D., Mulligan, B., Araz, K., Jacob, B. M., & Redmond, P. (2026) · HRB Open Research · doi:10.12688/hrbopenres.14302.1
Khan, U. T., Khan, A. T., Aljaadi, W., Alhadlaq, R., Baqashmer, Z. A. A., Alsafi, Y., Alomran, Y. A., Rusaiyes, M. A., Radif, M., Khan, T. N., et al. (2026) · JMIR Medical Informatics · doi:10.2196/83335 · Cited by 3
“Kernberg et al [11] further noted that the AI-generated notes frequently omitted clinically important information, introduced hallucinated content, or contained factual inaccuracies and that note generation was not consistently reproducible.”
InfluentialTrust Me, I Might be a Medical Device: The Problem with AI ScribesPreprint
Dady, C., POTTS, H. W. W., Stevenson, F. A., & Schartau, P. (2026) · Research Square · doi:10.21203/rs.3.rs-8994700/v1
Zou, M., Srinivasan, S., Chin, V., Lo, T. W. S., Loke, W. T., Yang, G. D., Lim, Z. W., Li, K. Z., Wong, W., Lam, J. S. H., et al. (2026) · Eye · doi:10.1038/s41433-026-04325-3 · Cited by 1
“This safety analysis was performed based on the following categories of error types: omissions (missing critical information), incorrect information (inaccurate details), and extraneous content (repetitive or hallucinatory additions) [8].”
InfluentialBurstein, D. S., Schwartz, N., Ganguly, A., Hanak, M. A., Zimmermann, L. J., & Keshavarzian, A. (2026) · Diagnosis · doi:10.1515/dx-2025-0088
Dey, N., Rahman, M. M., Joy, S. D., Tanvir, K., Biswas, R., & Gomes, D. J. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-5906-9.ch001
Vivekanantha, P., Bouchard, M. D., Kay, J., De, D., & Ayeni, O. R. (2026) · Journal of Medical Education and Curricular Development · doi:10.1177/23821205261441470
Artificial Intelligence Language Models in Oncology: A Cross-Sectional Analysis of Published Studies
Edwards, N., Kannout, S., Zhang, D., Wong, H. C. Y., Leigh, J., Simone, C. B., & Chow, R. (2026) · Cureus · doi:10.7759/cureus.106983
Eixenberger, C., & O'Reilly-Shah, V. (2026) · Springer eBooks · doi:10.1007/978-3-032-18058-2_39 · Cited by 1
AI Clinical Scribe Limitations: Scoping Review with ☸️SAIMSARA
SAIMSARA (2026) · SAIMSARA Journal · doi:10.62487/saimsara5a879885
The Challenges and Promise of AI for Clinical Documentation in Pediatric Inpatient Care
Levitties, N., Sinha, A., Cullion, K., Kelly, D. P., Pandiyan, P., Agus, M. S. D., & Cifra, C. L. (2026) · Hospital Pediatrics · doi:10.1542/hpeds.2025-008972
Deep Learning in Medical Speech to Text: Methods and Challenges
Sztabinski, M., & Weichbroth, P. (2026) · Symmetry · doi:10.3390/sym18060885
Gelman, D., Akler, D., Shimon, G., Ogen, S., Ketko, I., Leibovitz, M., Almog, O., Benov, A., Beer, Z. A., & Nadler, R. (2026) · The Journal of Trauma: Injury, Infection, and Critical Care · doi:10.1097/ta.0000000000005059
“7,13,14 However, despite significant advancements in the field, concerns have been raised regarding the risk of less-accurate records that may lack the critical human reasoning essential for quality care.”
Metadata
- Authors
- Kernberg, A., Gold, J. A., & Mohan, V.
- Affiliations
- Oregon Health & Science University (US)
- Corresponding author
- Mohan, Vishnu
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2024-04-22
- Volume
- 26
- Pages
- e54419
- ISSN
- 1438-8871 (ISSN-L)
- PMC
- PMC11074889
- OpenAlex
- W4392632345
- Semantic Scholar
- 78dee4159add8e48b50a2100104fbfae99d2185b
- Cited by
- 120
- References
- 27
- Citations by year
- 2026: 55 · 2025: 52 · 2024: 13
- FWCI
- 11.24
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Journal of Medical Internet Research (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Nursing › Research and Theory › Nursing Diagnosis and Documentation
- Keywords
- ChatGPT-4, SOAP notes, clinical documentation, physician-patient encounters, audio recordings, note quality, medical scribes, physician burnout, large language models, history and physical examination, outpatient care
- MeSH terms
- Artificial Intelligence, Documentation, Humans, Physician-Patient Relations, Electronic Health Records
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.