CoChat

Explainable AI decision support improves accuracy during telehealth strep throat screening

Open access
Gomez, Catalina, Smith, Brittany-Lee, Zayas, Alisa, Unberath, Mathias, Canares, Therese · 2024 · Communications Medicine

The DOI matches its Crossref record; 68 of the 68 references it cites were found in a registry or an index.

Abstract

Background: Artificial intelligence-based (AI) clinical decision support systems (CDSS) using unconventional data, like smartphone-acquired images, promise transformational opportunities for telehealth; including remote diagnosis. Although such solutions’ potential remains largely untapped, providers’ trust and understanding are vital for effective adoption. This study examines how different human–AI interaction paradigms affect clinicians’ responses to an emerging AI CDSS for streptococcal pharyngitis (strep throat) detection from smartphone throat images. Methods: In a randomized experiment, we tested explainable AI strategies using three AI-based CDSS prototypes for strep throat prediction. Participants received clinical vignettes via an online survey to predict the disease state and offer clinical recommendations. The first set included a validated CDSS prediction (Modified Centor Score) and the second introduced an explainable AI prototype randomly. We used linear models to assess explainable AI’s effect on clinicians’ accuracy, confirmatory testing rates, and perceived trust and understanding of the CDSS. Results: The study, involving 121 telehealth providers, shows that compared to using the Centor Score, AI-based CDSS can improve clinicians’ predictions. Despite higher agreement with AI, participants report lower trust in its advice than in the Centor Score, leading to more requests for in-person confirmatory testing. Conclusions: Effectively integrating AI is crucial in the telehealth-based diagnosis of infectious diseases, given the implications of antibiotic over-prescriptions. We demonstrate that AI-based CDSS can improve the accuracy of remote strep throat screening yet underscores the necessity to enhance human–machine collaboration, particularly in trust and intelligibility. This ensures providers and patients can capitalize on AI interventions and smartphones for virtual healthcare.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s43856-024-00568-x

Cites

68 of 68 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1 Flowchart of participant allocation and study task completion. Participant allocation and task completion flowchart in the study. XAI-1–XAI-3 represents the different explainable AI (XAI) elements. Each case corresponds to a clinical vignette for strep pharyngitis screening.
Fig. 1 Flowchart of participant allocation and study task completion. Participant allocation and task completion flowchart in the study. XAI-1–XAI-3 represents the different explainable AI (XAI) elements. Each case corresponds to a clinical vignette for strep pharyngitis screening.CC BY · PubMed Central
Fig. 2 User interface design and experimental conditions for CDSS types. Visualization of the user interface for strep throat screening using different types of support. From left to right: CDSS with the Centor Score, XAI-1: AI-based CDSS with performance characteristics (PC), XAI-2: AI-based CDSS with Explain by Feature Contribution (EFC), and XAI-3: AI-based CDSS with explain by examples (EE).
Fig. 2 User interface design and experimental conditions for CDSS types. Visualization of the user interface for strep throat screening using different types of support. From left to right: CDSS with the Centor Score, XAI-1: AI-based CDSS with performance characteristics (PC), XAI-2: AI-based CDSS with Explain by Feature Contribution (EFC), and XAI-3: AI-based CDSS with explain by examples (EE).CC BY · PubMed Central
Fig. 3 Comparative analysis of CDSS types on outcomes and perceptions. Plots of measures for different types of support information in the CDSS. CS corresponds to Centor Score, PC to Performance Characteristics, EFC to Explain by Feature Contribution, and EE to Explain by Examples. a Sensitivity (N = 242 observations), b fraction of negative cases that were suggested to order a confirmatory strep test (N = 242 observations), c perceived trust (N = 968 observations), and d perceived understanding of the CDSS (N = 968 observations). The error bars shown in the plots represent the standard deviation, and significant results are emphasized with gray lines. *p ≤ 0.05, **p ≤ 0.01, ***p ≤ 0.001, ****p ≤ 0.0001.
Fig. 3 Comparative analysis of CDSS types on outcomes and perceptions. Plots of measures for different types of support information in the CDSS. CS corresponds to Centor Score, PC to Performance Characteristics, EFC to Explain by Feature Contribution, and EE to Explain by Examples. a Sensitivity (N = 242 observations), b fraction of negative cases that were suggested to order a confirmatory strep test (N = 242 observations), c perceived trust (N = 968 observations), and d perceived understanding of the CDSS (N = 968 observations). The error bars shown in the plots represent the standard deviation, and significant results are emphasized with gray lines. *p ≤ 0.05, **p ≤ 0.01, ***p ≤ 0.001, ****p ≤ 0.0001.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

1 more claim about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 68 references

Research works printed but not linked: 3

  1. Soberanis-Mukul, R. et al. A novel method to identify streptococcus pharyngitis using smart-phone images and artificial intelligence. In American Academy of Pediatrics National Conference (2022).
  2. Canares, T. Machine learning systems and related aspects for the detection of disease states. US Patent No. 63,375,978 (2022).
  3. Tonekaboni, S., Joshi, S., McCradden, M. D. & Goldenberg, A. What clinicians want: contextualizing explainable machine learning for clinical end use. In Machine Learning for Healthcare Conference 359–380 (Proceedings of Machine Learning Research, 2019).

Cited by

See all 30

Metadata

Authors
Gomez, C., Smith, B.-L., Zayas, A., Unberath, M., & Canares, T.
Affiliations
Johns Hopkins University (US)
Corresponding author
Unberath, Mathias
Venue
Communications Medicine
Published
2024-07-24
Volume
4
Issue
1
Pages
149-149
ISSN
2730-664X (ISSN-L)
OpenAlex
W4400935096
Cited by
30
References
68
Citations by year
2026: 17 · 2025: 13
FWCI
2.75
Type
journal article
Language
en
Open access
Licence
Locations
  • Communications Medicine (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • Johns Hopkins University internal funds
  • Bisciotti Foundation Translational Fund at Johns Hopkins University
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
Keywords
explainable AI, clinical decision support systems, streptococcal pharyngitis, telemedicine, human-AI interaction, clinician trust in AI, Centor score, confirmatory testing, antibiotic overprescription, remote diagnosis

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Explainable AI decision support improves accuracy during telehealth strep throat screening. https://app.cochat.ai/papers/10.1038/s43856-024-00568-x
The paper
Gomez, C., Smith, B.-L., Zayas, A., Unberath, M., & Canares, T. (2024). Explainable AI decision support improves accuracy during telehealth strep throat screening. Communications Medicine. https://doi.org/10.1038/s43856-024-00568-x

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.