CoChat

Impact of a Digital Scribe System on Clinical Documentation Time and Quality: Usability Study

Open access
van Buchem, Marieke Meija, Kant, Ilse M J, King, Liza, Kazmaier, Jacqueline, Steyerberg, Ewout W, Bauer, Martijn P · 2024 · JMIR AI

The DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.

Abstract

Background: Physicians spend approximately half of their time on administrative tasks, which is one of the leading causes of physician burnout and decreased work satisfaction. The implementation of natural language processing–assisted clinical documentation tools may provide a solution. Objective: This study investigates the impact of a commercially available Dutch digital scribe system on clinical documentation efficiency and quality. Methods: Medical students with experience in clinical practice and documentation (n=22) created a total of 430 summaries of mock consultations and recorded the time they spent on this task. The consultations were summarized using 3 methods: manual summaries, fully automated summaries, and automated summaries with manual editing. We then randomly reassigned the summaries and evaluated their quality using a modified version of the Physician Documentation Quality Instrument (PDQI-9). We compared the differences between the 3 methods in descriptive statistics, quantitative text metrics (word count and lexical diversity), the PDQI-9, Recall-Oriented Understudy for Gisting Evaluation scores, and BERTScore. Results: The median time for manual summarization was 202 seconds against 186 seconds for editing an automatic summary. Without editing, the automatic summaries attained a poorer PDQI-9 score than manual summaries (median PDQI-9 score 25 vs 31, P<.001, ANOVA test). Automatic summaries were found to have higher word counts but lower lexical diversity than manual summaries (P<.001, independent t test). The study revealed variable impacts on PDQI-9 scores and summarization time across individuals. Generally, students viewed the digital scribe system as a potentially useful tool, noting its ease of use and time-saving potential, though some criticized the summaries for their greater length and rigid structure. Conclusions: This study highlights the potential of digital scribes in improving clinical documentation processes by offering a first summary draft for physicians to edit, thereby reducing documentation time without compromising the quality of patient records. Furthermore, digital scribes may be more beneficial to some physicians than to others and could play a role in improving the reusability of clinical documentation. Future studies should focus on the impact and quality of such a system when used by physicians in clinical practice.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.2196/60020

Cites

29 of 29 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 Flowchart showing the 3 different summarization methods and consecutive evaluation.
Figure 1 Flowchart showing the 3 different summarization methods and consecutive evaluation.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 29 references

Research works printed but not linked: 3

  1. 2019, Taking Action Against Clinician Burnout: A Systems Approach to Professional Well-Being
  2. LinCYROUGE: a package for automatic evaluation of summariesText Summarization Branches Out Barcelona2004Text Summarization Branches Out Barcelona2004-07-25BarcelonaSpainAssociation for Computational Linguistics7481
  3. Kittinger, BJ, 2016, Perspect Health Inf Manag, 13, 1f

Cited by

See all 70

Metadata

Authors
van Buchem, M. M., Kant, I. M. J., King, L., Kazmaier, J., Steyerberg, E. W., & Bauer, M. P.
Affiliations
  • University Medical Center Utrecht (NL)
  • Leiden University Medical Center (NL)
Venue
JMIR AI
Published
2024-09-23
Volume
3
Pages
e60020
ISSN
2817-1705 (ISSN-L)
OpenAlex
W4402732546
Cited by
70
References
27
Citations by year
2026: 42 · 2025: 26 · 2024: 2
FWCI
21.19
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • JMIR AI (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Leiden Repository (Leiden University) (submittedVersion, cc-by)
  • Utrecht University Repository (Utrecht University) (submittedVersion, other-oa)
Topics
  • Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
  • Health Sciences › Nursing › Research and Theory › Nursing Diagnosis and Documentation
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Keywords
clinical documentation, natural language processing, clinical summarization, documentation time, documentation quality, physician burnout, usability, medical students, lexical diversity

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Impact of a Digital Scribe System on Clinical Documentation Time and Quality: Usability Study. https://app.cochat.ai/papers/10.2196/60020
The paper
van Buchem, M. M., Kant, I. M. J., King, L., Kazmaier, J., Steyerberg, E. W., & Bauer, M. P. (2024). Impact of a Digital Scribe System on Clinical Documentation Time and Quality: Usability Study. JMIR AI. https://doi.org/10.2196/60020

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.