The DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index; 25 of the 44 sources its claims cite were checked for support.
Artificial Intelligence (AI) – Powered Documentation Systems in Healthcare: A Systematic Review
Open accessAbstract
This systematic review of 11 studies found AI documentation systems improved efficiency, though quality varied and reliability concerns remained.
Overview
CoChat Summary
## TL;DR
This systematic review found that AI-driven documentation systems, especially ChatGPT and ambient AI, consistently improved documentation efficiency across the 11 included studies. However, documentation quality was variable, and concerns about hallucinations, reliability, and validity limit confidence in widespread clinical use.
## Key findings
- The review identified 3,969 studies and included 11 eligible studies after screening.
- Efficiency was assessed in 5 of 11 studies, and all five found AI technologies made documentation more efficient; the review notes significant improvement in mean documentation time in hospital and primary care settings.
- Documentation quality was assessed in 9 studies; 6 of 9 reported AI-generated documents meeting or surpassing traditional documentation standards, but 3 of those studies did not screen for hallucinations.
- Stakeholder experience was assessed in 4 studies, and HCPs reported improved ease of documentation with ChatGPT and ambient AI in all four; user opinion was assessed in 3 studies and was generally positive.
- Quality appraisal using MMAT found 5 of 11 studies met 100% of assessed criteria, 4 of 11 met 60%, and no study scored below 60%.
## Methods
- Systematic review conducted according to PRISMA; databases searched were PubMed, Embase, and Cochrane Library, with no publication-year limits.
- Studies were selected using PICOSS criteria; duplicates were removed in Rayyan, with two independent reviewers screening titles/abstracts and then full texts.
- Data extracted included publication year, design, population, AI type, setting, document type, quality, timing, stakeholder opinion, and hallucinations/false information; study quality was assessed with the Mixed Methods Appraisal Tool (MMAT).
## Limitations
- The paper states that the included technologies are relatively new, so their capabilities and limitations may change rapidly as AI evolves.
- The included studies used a wide range of tools and methods to assess outcomes, creating heterogeneity that complicates comparison and interpretation.
- The paper notes that some studies did not screen for hallucinations at all, which may have led to an underestimation of risks.
- Evident from the text: the evidence base is small (11 studies) and concentrated in 2024 (9 of 11 studies), which limits maturity and breadth of the review’s conclusions.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
19 Valid · 1 Not valid · 8 Not checked · 46 Own results and interpretations
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“Hallucination rates have been documented to vary between 3% – 28% in current literature [30, 31].”
§ Artificial Intelligence (AI) – Powered Documentation Systems in Healthcare: A Systematic Review > Documentation Quality
What the sources say
- Disputedcites [30] Chelli M, Descamps J, Lavoue V, Trojani C, Azar M, Deckert M, et al. Hallucination Rates and Reference Accuracy of ChatGPT and Bard for Systematic Reviews: Comparative Analysis. J Med Internet Res. 2024;26:e53164. sourceExperimental — accuracy not yet measured
The Results (page unavailable) report hallucination rates of 28.6%, 39.6%, and 91.4% for the models tested. The paper therefore documents rates well above the claimed 28% upper bound; it does not report a 3% rate. (LLM, conf=0.88)
Checked by a reasoning model — may be wrong.
- Not matchedcites [31] Metz C. Chatbots May ‘Hallucinate’ More Often Than Many Realize. The New York Times. 2023 06/11/2023.
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
Every source the paper cites for this claim was checked and supports it.
In the paper
“Clinical documentation burden and excessive bureaucratic tasks have been cited as leading contributors to HCP burnout [2].”
§ Artificial Intelligence (AI) – Powered Documentation Systems in Healthcare: A Systematic Review > Introduction
What the sources say
- Supportedcites [2] Wu Y, Wu M, Wang C, Lin J, Liu J, Liu S. Evaluating the Prevalence of Burnout Among Health Care Professionals Related to Electronic Health Record Use: Systematic Review and Meta-Analysis. JMIR Medical Informatics. 2024;12:e54811. source
The Results section, under “Main Causes of Burnout and Proposed Solutions,” identifies administrative documentation tasks among workload factors contributing to burnout and says workload factors appeared in 18 of 32 reviewed studies. The Discussion section, under “Solutions,” also reports a robust relationship between workload, time spent using EHRs, and burnout, and recommends reducing clerical and documentation burdens. (LLM, conf=0.88)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“AI based tools are poised to improve clinical documentation workflow and efficiency [2].”
§ Artificial Intelligence (AI) – Powered Documentation Systems in Healthcare: A Systematic Review > Introduction
What the sources say
- Supportedcites [2] Wu Y, Wu M, Wang C, Lin J, Liu J, Liu S. Evaluating the Prevalence of Burnout Among Health Care Professionals Related to Electronic Health Record Use: Systematic Review and Meta-Analysis. JMIR Medical Informatics. 2024;12:e54811. source
The Discussion’s “Artificial Intelligence–Based Solutions” section says AI-enabled dictation and note-taking can streamline medical-document creation and increase operational efficiency. It also says GPT-4 and intelligent templates can speed up documentation. These passages support the claim’s prospective wording. (LLM, conf=0.93)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“The administrative burden associated with clinical documentation has been identified as a major contributor to health care professional (HCP) burnout.”
- Also stated in: § Introduction · plain
“Clinical documentation burden and excessive bureaucratic tasks have been cited as leading contributors to HCP burnout [2].”
§ Artificial Intelligence (AI) – Powered Documentation Systems in Healthcare: A Systematic Review > Abstract Artificial Intelligence (AI) driven documentation systems are positioned to enhance documentation efficiency and reduce documentation burden in the healthcare setting. The administrative burden associated with clinical documentation has been identified as a major contributor to health care professional (HCP) burnout. The current systematic review aims to evaluate the efficiency, quality, and stakeholder opinion regarding the use of AI-driven documentation systems. Using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines a comprehensive search was conducted across PubMed, Embase and Cochrane library. Two independent reviewers applied inclusion and exclusion criteria to identify eligible studies. Details of AI technology, document type, document quality and stakeholder experience were extracted. The review included 11 studies. All included studies utilised Chat generated pretrained transformer (Chat GPT, Open AI, CA, USA) or an ambient AI technology. Both forms of AI demonstrated significant potential to improve documentation efficiency. Despite efficiency gains, the quality of AI-generated documentation varied across studies. The heterogeneity of methods utilised to assess document quality influenced interpretation of results. HCP opinion was generally positive, users highlighted ease of use and reduced task load as primary benefits. However, HCPs also expressed concerns about the reliability and validity of AI-generated documentation. Chat GPT and ambient AI show promise in enhancing the efficiency and quality of clinical documentation. While the efficiency benefits are clear, the challenges associated with accuracy and consistency need to be addressed. HCP experiences indicate a cautious optimism towards AI integration, however reliability will depend on continued refinement and validation of the technology.
What the sources say
- Supportedcites [2] Wu Y, Wu M, Wang C, Lin J, Liu J, Liu S. Evaluating the Prevalence of Burnout Among Health Care Professionals Related to Electronic Health Record Use: Systematic Review and Meta-Analysis. JMIR Medical Informatics. 2024;12:e54811. source
The Results section identifies administrative documentation tasks as a workload factor contributing to burnout among health care professionals. The Discussion also describes reducing documentation and EHR workload as a strategy for reducing burnout. (LLM, conf=0.90)
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Introduction · plain
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“evidence suggests that HCPs spend on average two hours outside the official working day on documentation tasks [8].”
§ Artificial Intelligence (AI) – Powered Documentation Systems in Healthcare: A Systematic Review > Introduction
What the sources say
- Partly supportedcites [8] Gaffney A, Woolhandler S, Cai C, Bor D, Himmelstein J, McCormick D, et al. Medical Documentation Burden Among US Office-Based Physicians in 2019: A National Study. JAMA Intern Med. 2022;182(5):564-6. sourceExperimental — accuracy not yet measured
The results passage and Table 2 report a mean of 1.77 hours per day documenting outside office hours, reasonably rounded to two hours. The study concerns US office-based physicians, however, not HCPs generally, so the claim extends beyond the population studied. (LLM, conf=0.86)
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
37 more claims · 7 with their cited sources checked
Create a free account to see all 74 claims and to verify the 1 reference still open.
- It may state two separate claims as one. 14
- Probably not a substantial claim. 19
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Ambient Listening technology, or Ambient AI, may reduce documentation burden by leveraging automated speech recognition (ASR) and large language models (LLMs) to transform real-time clinical conversations into drafted EHR documentation records [11].”
From Protocol for an EHR-embedded pragmatic randomized control trial of Ambient AI to Reduce Nursing Staff Documentation Time (2026) · cites this paper as [11]
Experimental — accuracy not yet measured
Citations
Cites
The PRISMA 2020 statement: An updated guideline for reporting systematic reviews
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., et al. (2021) · BMJ · doi:10.1136/bmj.n71 · Cited by 108,283
The Mixed Methods Appraisal Tool (MMAT) version 2018 for information professionals and researchers
Hong, Q. N., Fàbregues, S., Bartlett, G., Boardman, F., Cargo, M., Dagenais, P., Gagnon, M.-P., Griffiths, F., Nicolau, B., O’Cathain, A., et al. (2018) · Education for Information · doi:10.3233/efi-180221 · Cited by 4,737 · cited 1×
“1 The mixed methods appraisal tool (MMAT) [16] was used to assess the quality of eligible studies.”
The potential for artificial intelligence in healthcare
Davenport, T., & Kalakota, R. (2019) · Future Healthcare Journal · doi:10.7861/futurehosp.6-2-94 · Cited by 3,800 · cited 2×
“Many barriers have been identified to the widespread implementation of AI technologies in healthcare, however previous literature has emphasised the inevitability of its adoption [3].”
Shanafelt, T. D., Dyrbye, L. N., Sinsky, C., Hasan, O., Satele, D., Sloan, J., & West, C. P. (2016) · Mayo Clinic Proceedings · doi:10.1016/j.mayocp.2016.05.007 · Cited by 1,086 · cited 1×
“This administrative load not only contributes to burnout but also detracts from time that could be spent on direct patient care [9].”
ChatGPT: the future of discharge summaries?
Patel, S. B., & Lam, K. (2023) · The Lancet Digital Health · doi:10.1016/s2589-7500(23)00021-3 · Cited by 728
Boscardin, C. K., Gin, B., Golde, P. B., & Hauer, K. E. (2024) · Academic Medicine · doi:10.1097/acm.0000000000005439 · Cited by 620 · cited 1×
“Chat generated pretrained transformer (Chat GPT) (Open AI, San Francisco, CA, USA) is a form of generative AI which utilises large language model (LLM) algorithms and advanced neural networks to generate human like text responses [4].”
Chelli, M., Descamps, J., Lavoué, V., Trojani, C., Azar, M., Deckert, M., Raynier, J.-L., Clowez, G., Boileau, P., & Ruetsch-Chelli, C. (2024) · Journal of Medical Internet Research · doi:10.2196/53164 · Cited by 602 · cited 1×
“Hallucination rates have been documented to vary between 3% – 28% in current literature [30, 31].”
Johnson, D., Goodman, R., Patrinely, J., Stone, C., Zimmerman, E., Donald, R., Chang, S., Berkowitz, S., Finn, A., Jahangir, E., et al. (2023) · doi:10.21203/rs.3.rs-2566942/v1 · Cited by 598 · cited 1×
“The output generated by Chat GPT is dependent on the complexity and clarity of the input, continuous evaluation of generated output will be required to ensure safe implementation in clinical settings [32, 33].”
Ambient artificial intelligence scribes to alleviate the burden of clinical documentation.
Tierney, A. A., Gayre, G., Hoberman, B., Mattern, B., Ballesca, M., Kipnis, P., Liu, V., & Lee, K. (2024) · NEJM Catalyst · doi:10.1056/cat.23.0404 · Cited by 400 · cited 2×
“Both Chat GPT and ambient AI have shown promise in streamlining clinical documentation in a variety of settings [5–7].”
How to formulate research recommendations
Brown, P., Brunnhuber, K., Chalkidou, K., Chalmers, I., Clarke, M., Fenton, M., Forbes, C., Glanville, J., Hicks, N. J., Moody, J., et al. (2006) · BMJ · doi:10.1136/bmj.38987.492014.94 · Cited by 381 · cited 1×
“Studies that met the population, intervention, comparator, outcome, study design and setting (PICOSS) criteria were included in the review (Table 2) [14].”
Cited by
AI Scribes in Health Care: Balancing Transformative Potential With Responsible Integration
Leung, T. I., Coristine, A. J., & Benis, A. (2025) · JMIR Medical Informatics · doi:10.2196/80898 · Cited by 86
“Results suggest that there is risk for hallucinations or fictitious information [4].”
InfluentialMache, S., Bernburg, M., Würtenberger, A., & Groneberg, D. A. (2025) · Clinics and Practice · doi:10.3390/clinpract15080138 · Cited by 29
“Consistent with previous research [27,41–43], many participants in this study identified tangible benefits of AI, particularly in automating administrative tasks such as documentation and scheduling.”
Cheungpasitporn, W., Athavale, A. M., Ghazi, L., KASHANI, K. B., Colicchio, T. K., Koyner, J. L., Chen, J., Ix, J. H., Nadkarni, G. N., & Neyra, J. A. (2026) · Clinical Kidney Journal · doi:10.1093/ckj/sfag004 · Cited by 42
“As highlighted in Table 2 rigorous validation and real-world testing are foundational requirements for responsible AI deployment in nephrology rather than optional extensions [1 ,3 ,4 ,7 ,11 ,13 ,31 ,37 ,40 –44 , 51 –54 , 65 –67 , 84 , 85 , 107 –152 ].”
Tiem, J. M. V., Cramer, E., Iverson, C., Kennelty, K. A., Andrys, N. R. R., Lee, J., Knake, L. A., Misurac, J. M., Blum, J. M., & Reisinger, H. S. (2025) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaf214 · Cited by 35
“5,8,9,12,23,26,27 In prioritizing efficiency, clinicians may be pointing to what Tracey et al describe as the “logistification of documentation,” or how documentation can function to expedite the movement of people, resources, and/or information, through an infrastructure.”
Ohde, J. W., Thompson, A., Liu, Z., Rost, L. M., Overgaard, J. D., Tan, J. Y. E., Overgaard, S. M., Sarmiento, R. F., Ke, Y., Liew, J. C. K., et al. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-02554-0 · Cited by 23
Metadata
- Authors
- Bracken, A., Reilly, C., Feeley, A., Sheehan, E., Merghani, K., & Feeley, I.
- Affiliations
- Royal College of Surgeons in Ireland (IE)
- University College Dublin (IE)
- University of Limerick (IE)
- Midland Regional Hospital at Tullamore (IE)
- Corresponding author
- Bracken, Aisling
- Venue
- Journal of Medical Systems
- Published
- 2025-02-18
- Volume
- 49
- Issue
- 1
- Pages
- 28-28
- ISSN
- 0148-5598 (ISSN-L)
- 1573-689X
- PMC
- PMC11835907
- OpenAlex
- W4407695342
- Cited by
- 113
- References
- 33
- Citations by year
- 2026: 82 · 2025: 31
- FWCI
- 6.67
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Journal of Medical Systems (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Funders
- Royal College of Surgeons in Ireland
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Nursing › Research and Theory › Nursing Diagnosis and Documentation
- Keywords
- documentation burden, ChatGPT, ambient AI, stakeholder acceptance, workload
- MeSH terms
- Artificial Intelligence, Documentation, Humans
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Artificial Intelligence (AI) – Powered Documentation Systems in Healthcare: A Systematic Review. https://app.cochat.ai/papers/10.1007/s10916-025-02157-4
- The paper
- Bracken, A., Reilly, C., Feeley, A., Sheehan, E., Merghani, K., & Feeley, I. (2025). Artificial Intelligence (AI) – Powered Documentation Systems in Healthcare: A Systematic Review. Journal of Medical Systems. https://doi.org/10.1007/s10916-025-02157-4
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
