CoChat
Sign Up
Cited by 162Electronic Health Records SystemsArtificial Intelligence in Healthcare and EducationNursing Diagnosis and Documentation

Challenges of developing a digital scribe to reduce clinical documentation burden

Open access
Quiroz, Juan C., Laranjo, Liliana, Kocaballi, Ahmet Baki, Berkovsky, Shlomo, Rezazadegan, Dana, Coiera, Enrico·2019·npj Digital Medicine
DOI 10.1038/s41746-019-0190-1PMC PMC6874666
Open this paper in CoChat — free accountOpen PDF

Abstract

Clinicians spend a large amount of time on clinical documentation of patient encounters, often impacting quality of care and clinician satisfaction, and causing physician burnout. Advances in artificial intelligence (AI) and machine learning (ML) open the possibility of automating clinical documentation with digital scribes, using speech recognition to eliminate manual documentation by clinicians or medical scribes. However, developing a digital scribe is fraught with problems due to the complex nature of clinical environments and clinical conversations. This paper identifies and discusses major challenges associated with developing automated speech-based documentation in clinical settings: recording high-quality audio, converting audio to transcripts using speech recognition, inducing topic structure from conversation data, extracting medical concepts, generating clinically meaningful summaries of conversations, and obtaining clinical data for AI and ML algorithms.

AI snapshot

Written from the paper’s full text.

TL;DR

This perspective paper identifies and discusses the major technical and data-related challenges to developing a "digital scribe" that automatically documents clinician–patient conversations using speech recognition and NLP. It argues that progress is best achieved by solving each pipeline challenge individually rather than pursuing a holistic solution.

Key findings

The paper is a challenge-identification piece rather than an empirical study, so it reports no primary outcomes of its own; its claims are framed around cited evidence and the authors' analysis.

  • A cited study found word error rates of 35% or higher for simulated medical conversations using commercial ASR engines, and the authors note these were best-case results from controlled, near-ideal acoustic conditions.
  • The authors enumerate challenges across five pipeline stages: audio recording/ASR, topic structuring, medical concept extraction, summarization, and data collection (summarized in Table 1).
  • Existing tools such as MetaMap and cTAKES map text to UMLS concepts but were designed for written text; with default settings, "I am feeling fine" maps "I" to "blood group antibody I," "feeling" to "emotions," and "fine" to "qualitative concept" or "legal fine."
  • One cited effort developed its own corpus of 3000 conversations annotated by medical scribes because no public corpus was available, described as a costly investment.

Methods

This is a conceptual/perspective review, not an empirical study; it identifies and discusses challenges via narrative synthesis of prior literature and a summary table (Table 1) mapping tasks to challenges. No dataset, sample, or statistical analysis is reported. The paper draws on cited prior work (e.g., ASR engine comparisons, diarization research, UMLS mapping tools) to illustrate each challenge.

Limitations

The paper does not present a formal limitations section; as a perspective piece it offers no empirical validation of the challenges it describes. The text makes evident that the challenges are largely illustrated through cited studies and hypothetical examples rather than systematic evidence, and the authors note that evaluation of digital scribes in clinical environments (quality of care, patient satisfaction, clinician efficiency, documentation time, organizational dynamics) remains to be conducted. The authors also state that collecting sufficient data at the scale needed for AI/ML could alone take years, and that the closed data-sharing environment slows progress.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1038/s41746-019-0190-1

Cites

47 of 47 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 39 cited sources checked · 0 supported · 0 flagged

  • Premisenone given

    UMLS was designed for written text, not for spoken medical conversations.

    “However, UMLS was designed for written text, not for spoken medical conversations.”
    Quote in the paperSubstantial claim
  • Premisecitation

    ASR produces a transcript of the clinician-patient conversation that lacks clear boundaries and structure due to the unconstrained nature of conversations.

    “ASR produces a transcript of the clinician–patient conversation that lacks clear boundaries and structure due to the unconstrained nature of conversations.22”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [22] Lacson, R. C., Barzilay, R. & Long, W. J. Automatic analysis of medical dialogue in the home hemodialysis domain: structure induction and summarization. J. Biomed. Inform. 39, 541–555 (2006). source

      Not checked in the free run.

  • Premisecitation

    Spontaneous, conversational speech is not linguistically well-formed.

    “Spontaneous, conversational speech is not linguistically well-formed.21”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [21] Xiong, W. et al. Toward human parity in conversational speech recognition. IEEE/ACM Trans. Audio Speech Lang. Proc. 25, 2410–2423 (2017). source

      Not checked in the free run.

  • Premisecitation

    The word error rate of simulated medical conversations with commercial ASR engines was 35% or higher.

    “A recent study found that the word error rate of simulated medical conversations with commercial ASR engines was 35% or higher.17”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [17] Kodish-Wachs, J., Agassi, E., Kenny, P. & Overhage, J. M. A systematic comparison of contemporary automatic speech recognition engines for conversational clinical speech. AMIA Annu. Symp. Proc. 2018, 683–689 (2018).

      Not checked in the free run.

  • Premisecitation

    Tools like MetaMap must have their parameters tuned, as using them with default settings may result in the extraction of irrelevant terms.

    “Tools like MetaMap must also have their parameters tuned, as using them with default settings may result in the extraction of irrelevant terms.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [29] Reátegui, R. & Ratté, S. Comparison of MetaMap and cTAKES for entity extraction in clinical notes. BMC Med. Inform. Decis. Mak. 18, 74 (2018). source

      Not checked in the free run.

  • Classified as probably not a substantial claim. 17
  • May combine two claims. 11

  • Classified as probably not a substantial claim. May combine two claims.

    Premisenone given

    A digital scribe must be able to record the clinician-patient conversation, convert the audio to text, and extract salient information from the text and summarize the information.

    “To generate medical notes for the clinician–patient encounter, a digital scribe must be able to: (1) record the clinician–patient conversation, (2) convert the audio to text, and (3) extract salient information from the text and summarize the information (Fig. 1).”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given

    Advances in AI, machine learning, NLP, natural language understanding, and automatic speech recognition have raised the prospect of deploying effective and reliable digital scribes in clinical practice.

    “Recently, advances in AI, machine learning (ML), NLP, natural language understanding, and automatic speech recognition (ASR), have raised the prospect of deploying effective and reliable digital scribes in clinical practice.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Clinical encounters do not necessarily follow a linear order of their components, which exacerbates summarization or information extraction.

    “However, clinical encounters do not necessarily follow a linear order of their components,27,28 which exacerbates summarization or information extraction.22”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [27] Waitzkin, H. A critical theory of medical discourse: ideology, social control, and the processing of social context in medical encounters. J. Health Soc. Behav. 30, 220–239 (1989). source

      Not checked in the free run.

    • cites [28] Kocaballi, A. B. et al. A network model of activities in primary care consultations. J. Am. Med. Inform. Assoc. 26, 1074–1082 (2019). source

      Not checked in the free run.

    • cites [22] Lacson, R. C., Barzilay, R. & Long, W. J. Automatic analysis of medical dialogue in the home hemodialysis domain: structure induction and summarization. J. Biomed. Inform. 39, 541–555 (2006). source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Medical conversations have different statistical properties than medical dictations, meaning that ASR trained with dictations is likely to underperform with medical conversations.

    “Medical conversations have different statistical properties than medical dictations, meaning that ASR trained with dictations is likely to underperform with medical conversations.7”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [7] Finley, G. et al. An automated medical scribe for documenting clinical encounters. In Proc. 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations, 11–15 (Association for Computational Linguistics, 2018). source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    High-quality audio minimizes errors across the processing pipeline of the digital scribe.

    “High-quality audio minimizes errors across the processing pipeline of the digital scribe.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Clinical documentation is associated with clinician burnout, increased cognitive load, information loss, and distractions.

    “Clinical documentation is found to be associated with clinician burnout,1 increased cognitive load,2 information loss,3 and distractions.4”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [1] Friedberg, M. W. et al. Factors affecting physician professional satisfaction and their implications for patient care, health systems, and health policy. RAND Health Q. 3, 1 (2014).

      Not checked in the free run.

    • cites [2] Wachter, R. & Goldsmith, J. To combat physician burnout and improve care, fix the electronic health record. Harvard Bus. Rev. (2018) https://hbr.org/2018/03/to-combat-physician-burnout-and-improve-care-fix-the-electronic-health-record#comment-section.

      Not checked in the free run.

    • cites [3] Shachak, A., Hadas-Dayagi, M., Ziv, A. & Reis, S. Primary care physicians’ use of an electronic medical record system: a cognitive task analysis. J. Gen. Intern. Med. 24, 341–348 (2009). source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationnone given

    Developing a digital scribe is fraught with problems due to the complex nature of clinical environments and clinical conversations.

    “However, developing a digital scribe is fraught with problems due to the complex nature of clinical environments and clinical conversations.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationnone given

    Any digital scribe solution that requires ongoing input and supervision throughout the consultation will distract clinicians from patients and replace the distractions and disruptions of using an EHR with those of a digital scribe.

    “Any digital scribe solution that requires ongoing input and supervision throughout the consultation will (1) distract clinicians from patients and (2) replace the distractions and disruptions of using an EHR with those of a digital scribe.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationreasoning

    To generate effective medical notes, the summarization may need to draw on medical knowledge and capture nonverbal information during a consultation.

    “To generate effective medical notes, the summarization may need to draw on medical knowledge and capture nonverbal information during a consultation.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    The main concern raised against automating documentation is that manual documentation allows clinicians to structure their thoughts, think critically, reflect, and practice medicine effectively, such that removing it would adversely affect the way clinicians practice medicine.

    “The main concern raised is that manual documentation allows clinicians to structure their thoughts, think critically, reflect, and practice medicine effectively, such that removing it would adversely affect the way clinicians practice medicine.9”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [9] Willis, M. & Jarrahi, M. H. Automating documentation: a critical perspective into the role of artificial intelligence in clinical documentation. In Information in Contemporary Society (eds Taylor, N. G., Christian-Lamb, C., Martin, M. H. & Nardi, B.) 200–209 (Springer International Publishing, 2019). source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Digital scribe research is hindered by limited well-annotated large-scale data for modeling human-human spoken dialogs and even scarcer conversation data in healthcare due to privacy issues.

    “In general, digital scribe research is hindered by (1) limited well-annotated large-scale data for modeling human–human spoken dialogs and (2) even scarcer conversation data in healthcare due to privacy issues.14”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [14] Liu, Z. et al. Fast prototyping a dialogue comprehension system for nurse-patient conversations on symptom monitoring. In Proc. 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Vol. 2 (Industry Papers), 24–31 (Association for Computational Linguistics, 2019).

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Obtaining the gold standard summary of each conversation is costly because of the medical expertize required to complete the task and the high variability in clinician notes' content, style, organization, and quality.

    “However, obtaining the gold standard summary of each conversation is costly because of the medical expertize required to complete the task14 and the high variability in clinician notes’ content, style, organization, and quality.36”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [14] Liu, Z. et al. Fast prototyping a dialogue comprehension system for nurse-patient conversations on symptom monitoring. In Proc. 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Vol. 2 (Industry Papers), 24–31 (Association for Computational Linguistics, 2019).

      Not checked in the free run.

    • cites [36] Edwards, S. T., Neri, P. M., Volk, L. A., Schiff, G. D. & Bates, D. W. Association of note quality and quality of care: a cross-sectional study. BMJ Qual. Saf. 23, 406–413 (2014). source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    A dataset of medical conversations along with the corresponding summaries would allow far-reaching advances in the digital scribe and clinical documentation space.

    “A dataset of medical conversations along with the corresponding summaries would allow far-reaching advances in the digital scribe and clinical documentation space.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Conversations do not fit a command-like structure, which makes it difficult to perform intent recognition and to apply NLP techniques.

    “In addition, conversations do not fit a command-like structure, which makes it difficult to perform intent recognition—identifying a user’s intent from an utterance31—and to apply NLP techniques.19”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [31] Bhargava, A., Celikyilmaz, A., Hakkani-Tur, D. & Sarikaya, R. Easy contextual intent prediction and slot detection. In 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, 8337–8341 (IEEE, 2013). source

      Not checked in the free run.

    • cites [19] Ram, A. et al. Conversational AI: the science behind the Alexa prize. Preprint at arXiv:1801.03604 [cs] (2018).

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

    Interpretationreasoning

    The currently closed environment for sharing sensitive data means that the few research teams with access to data are the only ones that can make advances, further slowing progress by impeding open science.

    “The currently closed environment for sharing sensitive data means that the few research teams with access to data are the only ones that can make advances, further slowing progress by impeding open science.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    The goal of digital scribes is the formation of a "human-AI symbiosis" that augments the clinician-patient experience and improves quality of care.

    “Rather than replacement of clinicians as depicted in many dystopic AI futures, the goal of digital scribes is the formation of a “human–AI symbiosis” that augments the clinician–patient experience and improves quality of care.6,8,9”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [6] Lin, S. Y., Shanafelt, T. D. & Asch, S. M. Reimagining clinical documentation with artificial intelligence. Mayo Clin. Proc. 93, 563–565 (2018). source

      Not checked in the free run.

    • cites [8] Coiera, E., Kocaballi, B., Halamka, J. & Laranjo, L. The digital scribe. npj Digit. Med. 1, 58 (2018). source

      Not checked in the free run.

    • cites [9] Willis, M. & Jarrahi, M. H. Automating documentation: a critical perspective into the role of artificial intelligence in clinical documentation. In Information in Contemporary Society (eds Taylor, N. G., Christian-Lamb, C., Martin, M. H. & Nardi, B.) 200–209 (Springer International Publishing, 2019). source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationreasoning

    Due to the complexity of each task, research may reap the greatest benefits by focusing on solving the challenges individually, as opposed to seeking to build a holistic solution.

    “Due to the complexity of each task, we posit that research may reap the greatest benefits by focusing on solving the challenges individually, as opposed to seeking to build a holistic solution.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationreasoning

    Rich and accurate clinical data has become one of the most valuable intellectual property assets for industry and academia.

    “As a result, rich and accurate clinical data has become one of the most valuable intellectual property assets for industry and academia.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationreasoning

    Clinicians will only be convinced of embracing digital scribes if they believe that any reviewing and revising of summaries will be less time consuming than writing a summary from scratch.

    “Nevertheless, clinicians will only be convinced of embracing digital scribes if they believe that any reviewing and revising of summaries will be less time consuming than writing a summary from scratch.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisecitation

    Current advocates of replacing the entire documentation process with AI tend to overlook the complexities of healthcare sociotechnical systems.

    “Current advocates of replacing the entire documentation process with AI also tend to overlook the complexities of healthcare sociotechnical systems.9”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [9] Willis, M. & Jarrahi, M. H. Automating documentation: a critical perspective into the role of artificial intelligence in clinical documentation. In Information in Contemporary Society (eds Taylor, N. G., Christian-Lamb, C., Martin, M. H. & Nardi, B.) 200–209 (Springer International Publishing, 2019). source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationreasoning

    Conversational clinical data, transcripts, and summaries are needed to apply the recent advances in ML and AI to digital scribe development.

    “In particular, conversational clinical data, transcripts, and summaries are needed to apply the recent advances in ML and AI to digital scribe development.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationnone given

    Collecting sufficient data at the scale needed for AI and ML algorithms could alone take years to complete.

    “Collecting sufficient data at the scale needed for AI and ML algorithms could alone take years to complete.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given

    During a clinical encounter, it is common for a clinician to change their assessments or revise some observations.

    “During a clinical encounter, it is common for a clinician to change their assessments or revise some observations.”
    Quote in the paperNot a substantial claim
23 of 23

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • There are no sufficiently large clinic dialogue-note datasets publicly available to benchmark the task of AI-assisted note generation from visit dialogue.

    “Unfortunately as clinic encounter conversations are not routinely recorded and are difficult to ethically share due to patient confidentiality, there are no sufficiently large clinic dialogue-note datasets to benchmark this task.”

    From Aci-bench: a Novel Ambient Clinical Intelligence Dataset for Benchmarking Automatic Visit Note Generation (2023) · cites this paper as [12]

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Related

Authors
Quiroz, J. C., Laranjo, L., Kocaballi, A. B., Berkovsky, S., Rezazadegan, D., & Coiera, E.
Affiliations
Macquarie University (AU)
Corresponding author
Quiroz, Juan C.
ORCID
Venue
npj Digital Medicine
Publisher
Springer Science and Business Media LLC
Published
2019-11-22
Volume
2
Issue
1
Pages
114-114
ISSN
2398-6352 (ISSN-L)
DOI
10.1038/s41746-019-0190-1
PMC
PMC6874666
OpenAlex
W2989925457
Semantic Scholar
bfcfdf78801e224e88016c04e973130e5944ee8f
Cited by
162
References
47
Citations by year
2026: 26 · 2025: 37 · 2024: 26 · 2023: 24 · 2022: 18 · 2021: 16 · 2020: 12 · 2019: 1 · 2018: 1 · 2013: 1
FWCI
2.21
Type
journal article
Language
en
Open access
Licence
Locations
  • npj Digital Medicine (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
  • UTS ePRESS (University of Technology Sydney) (submittedVersion, other-oa)
  • Swinburne figshare (Swinburne University of Technology) (submittedVersion, cc-by)
Funders
Department of Health | National Health and Medical Research Council (APP1134919)
Topics
  • Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Health Sciences › Nursing › Issues, ethics and legal aspects › Nursing Diagnosis and Documentation
Keywords
clinical documentation burden, speech recognition, medical scribes, speech-to-text, topic modeling, medical concept extraction, physician burnout

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-06.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-06.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review