CoChat

The influence of explainable vs non-explainable clinical decision support systems on rapid triage decisions: a mixed methods study

Open access

The DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.

Abstract

Background: During the COVID-19 pandemic, a variety of clinical decision support systems (CDSS) were developed to aid patient triage. However, research focusing on the interaction between decision support systems and human experts is lacking. Methods: Thirty-two physicians were recruited to rate the survival probability of 59 critically ill patients by means of chart review. Subsequently, one of two artificial intelligence systems advised the physician of a computed survival probability. However, only one of these systems explained the reasons behind its decision-making. In the third step, physicians reviewed the chart once again to determine the final survival probability rating. We hypothesized that an explaining system would exhibit a higher impact on the physicians’ second rating (i.e., higher weight-on-advice). Results: The survival probability rating given by the physician after receiving advice from the clinical decision support system was a median of 4 percentage points closer to the advice than the initial rating. Weight-on-advice was not significantly different (p = 0.115) between the two systems (with vs without explanation for its decision). Additionally, weight-on-advice showed no difference according to time of day or between board-qualified and not yet board-qualified physicians. Self-reported post-experiment overall trust was awarded a median of 4 out of 10 points. When asked after the conclusion of the experiment, overall trust was 5.5/10 (non-explaining median 4 (IQR 3.5–5.5), explaining median 7 (IQR 5.5–7.5), p = 0.007). Conclusions: Although overall trust in the models was low, the median (IQR) weight-on-advice was high (0.33 (0.0–0.56)) and in line with published literature on expert advice. In contrast to the hypothesis, weight-on-advice was comparable between the explaining and non-explaining systems. In 30% of cases, weight-on-advice was 0, meaning the physician did not change their rating. The median of the remaining weight-on-advice values was 50%, suggesting that physicians either dismissed the recommendation or employed a “meeting halfway” approach. Newer technologies, such as clinical reasoning systems, may be able to augment the decision process rather than simply presenting unexplained bias.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1186/s12916-023-03068-2

Cites

35 of 35 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1 The explanation provided. This type of graphic was shown to the examiner by the explaining CDSS system. It was presented alongside the rated survival probability. The figure shows true and fictional influential factors (left) rated by their relative importance. The distribution and dot size are intended to represent some degree of variability. All values were generated randomly
Fig. 1 The explanation provided. This type of graphic was shown to the examiner by the explaining CDSS system. It was presented alongside the rated survival probability. The figure shows true and fictional influential factors (left) rated by their relative importance. The distribution and dot size are intended to represent some degree of variability. All values were generated randomlyCC BY · PubMed Central
Fig. 2 The x-axis shows the progress of the experiment; the y-axis shows the number of seconds spent on each individual patient
Fig. 2 The x-axis shows the progress of the experiment; the y-axis shows the number of seconds spent on each individual patientCC BY · PubMed Central
Fig. 3 Distance of initial rating and rating after advice. The x-axis shows the progress of the experiment; the y-axis shows the median distance to the advice for each patient with IQR error bars—both the initial rating and the rating after advice are displayed. The mean change between the initial rating and the rating after advice is 5.38% absolute (33.8% relative)
Fig. 3 Distance of initial rating and rating after advice. The x-axis shows the progress of the experiment; the y-axis shows the median distance to the advice for each patient with IQR error bars—both the initial rating and the rating after advice are displayed. The mean change between the initial rating and the rating after advice is 5.38% absolute (33.8% relative)CC BY · PubMed Central
Fig. 4 Weight-on-advice by physician. Each vertical lane represents the calculated weight-of-advice values for a single physician. Boxes contain Q1 through Q3; whiskers extend to the farthest observation within 1.5 times the IQR. Outliers are shown as circles. Weight-of-advice values were limited to between 0 and 1
Fig. 4 Weight-on-advice by physician. Each vertical lane represents the calculated weight-of-advice values for a single physician. Boxes contain Q1 through Q3; whiskers extend to the farthest observation within 1.5 times the IQR. Outliers are shown as circles. Weight-of-advice values were limited to between 0 and 1CC BY · PubMed Central
Fig. 5 Weight-on-advice over the course of the day. No statistically significant trend can be observed
Fig. 5 Weight-on-advice over the course of the day. No statistically significant trend can be observedCC BY · PubMed Central
Fig. 6 The probability of admission correlates with predicted survival. The survival rating after advice is shown on the x-axis, while the probability of admission (as estimated by the generalized linear mixed model) is displayed on the y-axis
Fig. 6 The probability of admission correlates with predicted survival. The survival rating after advice is shown on the x-axis, while the probability of admission (as estimated by the generalized linear mixed model) is displayed on the y-axisCC BY · PubMed Central
Fig. 7 Influential factors in decision-making
Fig. 7 Influential factors in decision-makingCC BY · PubMed Central
Fig. 8 Final questionnaire
Fig. 8 Final questionnaireCC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 35 references

Research works printed but not linked: 1

  1. Asimov I. I, robot. New York: Doubleday; 1950.

Cited by

See all 19

Metadata

Authors
Laxar, D., Eitenberger, M., Maleczek, M., Kaider, A., Hammerle, F. P., & Kimberger, O.
Affiliations
  • Ludwig Boltzmann Gesellschaft (AT)
  • Medical University of Vienna (AT)
Corresponding author
Maleczek, Mathias
Venue
BMC Medicine
Published
2023-09-19
Volume
21
Issue
1
Pages
359-359
ISSN
1741-7015 (ISSN-L)
OpenAlex
W4386857282
Cited by
19
References
35
Citations by year
2026: 4 · 2025: 7 · 2024: 7 · 2023: 1
FWCI
0.76
Type
journal article
Language
en
Open access
Licence
Locations
  • BMC Medicine (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
Keywords
clinical decision support systems, explainable AI, rapid triage, COVID-19 pandemic, survival probability estimation, critically ill patients, human-AI interaction, physician decision-making
MeSH terms
COVID-19, Artificial Intelligence, Humans, Triage, Decision Support Systems, Clinical, Pandemics

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: The influence of explainable vs non-explainable clinical decision support systems on rapid triage decisions: a mixed methods study. https://app.cochat.ai/papers/10.1186/s12916-023-03068-2
The paper
Laxar, D., Eitenberger, M., Maleczek, M., Kaider, A., Hammerle, F. P., & Kimberger, O. (2023). The influence of explainable vs non-explainable clinical decision support systems on rapid triage decisions: a mixed methods study. BMC Medicine. https://doi.org/10.1186/s12916-023-03068-2

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.