The influence of explainable vs non-explainable clinical decision support systems on rapid triage decisions: a mixed methods study
Open accessThe DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
Abstract
Background: During the COVID-19 pandemic, a variety of clinical decision support systems (CDSS) were developed to aid patient triage. However, research focusing on the interaction between decision support systems and human experts is lacking. Methods: Thirty-two physicians were recruited to rate the survival probability of 59 critically ill patients by means of chart review. Subsequently, one of two artificial intelligence systems advised the physician of a computed survival probability. However, only one of these systems explained the reasons behind its decision-making. In the third step, physicians reviewed the chart once again to determine the final survival probability rating. We hypothesized that an explaining system would exhibit a higher impact on the physicians’ second rating (i.e., higher weight-on-advice). Results: The survival probability rating given by the physician after receiving advice from the clinical decision support system was a median of 4 percentage points closer to the advice than the initial rating. Weight-on-advice was not significantly different (p = 0.115) between the two systems (with vs without explanation for its decision). Additionally, weight-on-advice showed no difference according to time of day or between board-qualified and not yet board-qualified physicians. Self-reported post-experiment overall trust was awarded a median of 4 out of 10 points. When asked after the conclusion of the experiment, overall trust was 5.5/10 (non-explaining median 4 (IQR 3.5–5.5), explaining median 7 (IQR 5.5–7.5), p = 0.007). Conclusions: Although overall trust in the models was low, the median (IQR) weight-on-advice was high (0.33 (0.0–0.56)) and in line with published literature on expert advice. In contrast to the hypothesis, weight-on-advice was comparable between the explaining and non-explaining systems. In 30% of cases, weight-on-advice was 0, meaning the physician did not change their rating. The median of the remaining weight-on-advice values was 50%, suggesting that physicians either dismissed the recommendation or employed a “meeting halfway” approach. Newer technologies, such as clinical reasoning systems, may be able to augment the decision process rather than simply presenting unexplained bias.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures








Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Notably, Laxar et al. [61] discovered a disconnect between low self-reported trust and high behavioral reliance (WoA) on an AI-based triage support tool, suggesting that clinicians' actual reliance on AI may not directly align with what they say about trust.”
From Explainable AI for Clinical Decision Support Systems: Literature Review, Key Gaps, and Research Synthesis (2025) · cites this paper as [61]
The cited source says what the claim attributes to it.
In the citing paper
“Notably, Laxar et al. [61] discovered a disconnect between low self-reported trust and high behavioral reliance (WoA) on an AI-based triage support tool, suggesting that clinicians' actual reliance on AI may not directly align with what they say about trust.”
From Explainable AI for Clinical Decision Support Systems: Literature Review, Key Gaps, and Research Synthesis (2025) · cites this paper as [61]
The cited source says what the claim attributes to it.
In the citing paper
“Findings from Laxar et al. (difference between self-reported trust and real reliance) and Gomez et al. (explanation type and reliance) further strengthen the need for user-aligned method selection and interface design-and for complex, multimodal evaluation.”
From Explainable AI for Clinical Decision Support Systems: Literature Review, Key Gaps, and Research Synthesis (2025) · cites this paper as [61]
The cited source says what the claim attributes to it.
In the citing paper
“Transparency of the underlying model was also a concern (echoed in Laxar et al. [61]).”
From Explainable AI for Clinical Decision Support Systems: Literature Review, Key Gaps, and Research Synthesis (2025) · cites this paper as [61]
The cited source says what the claim attributes to it.
In the citing paper
“As with Laxar et al. [61], actual reliance and self-reported trust may diverge-suggesting the importance of multifaceted metrics.”
From Explainable AI for Clinical Decision Support Systems: Literature Review, Key Gaps, and Research Synthesis (2025) · cites this paper as [61]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Behavioral Study of obedience.
Milgram, S. (1963) · The Journal of Abnormal and Social Psychology · doi:10.1037/h0040525 · Cited by 4,918 · cited 1×
“Against the background of experiments suggesting obedient human behaviour in stressful situations influenced by robots and machines [12–14], it is vital to understand the interactions between CDSS and professional healthcare providers.”
BackgroundFair Allocation of Scarce Medical Resources in the Time of Covid-19
Emanuel, E. J., Persad, G., Upshur, R., Thome, B., Parker, M., Glickman, A., Zhang, C., Boyle, C., Smith, M., & Phillips, J. P. (2020) · New England Journal of Medicine · doi:10.1056/nejmsb2005114 · Cited by 3,137 · cited 1×
“The outbreak of the SARS-CoV-2 pandemic spiked both public and healthcare professional interests in the process of triage [6, 7].”
BackgroundWynants, L., Van Calster, B., Bonten, M. M., Collins, G. S., Debray, T. P., De Vos, M., Haller, M. C., Heinze, G., Moons, K. G., Riley, R. D., et al. (2020) · doi:10.1101/2020.03.24.20041020 · Cited by 3,070 · cited 2×
“Numerous decision support systems have been developed, validated or modified specifically for COVID-19, many of which are based on novel machine-learning technologies and algorithms [2].”
BackgroundHartigan, J. A., & Hartigan, P. M. (1985) · The Annals of Statistics · doi:10.1214/aos/1176346577 · Cited by 1,973 · cited 1×
“Indeed, the premise of WoA bimodality, computed using Hartigan’s dip test [33], holds in our dataset ( p -value 0.001) [34].”
MethodInfluentialKnight, S. R., Ho, A., Pius, R., Buchan, I., Carson, G., Drake, T. M., Dunning, J., Fairfield, C. J., Gamble, C., Green, C. A., et al. (2020) · BMJ · doi:10.1136/bmj.m3339 · Cited by 1,029 · cited 1×
“This was achieved by showing the most influential factors in the scoring formula which was based on the 4C score by Knight et al. [16] and is shown in Fig.”
MethodChatGPT and other large language models are double-edged swords
Shen, Y., Heacock, L., Elias, J., Hentel, K. D., Reig, B., Shih, G., & Moy, L. (2023) · Radiology · doi:10.1148/radiol.230163 · Cited by 1,008 · cited 1×
“Most recently, newer machine learning models have fuelled public discussion about the ethical use of these systems in medicine [10, 11].”
BackgroundA roadmap for national action on clinical decision support
Osheroff, J. A., Teich, J. M., Middleton, B., Steen, E. B., Wright, A., & Detmer, D. E. (2007) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2334 · Cited by 707 · cited 2×
“In healthcare, the term clinical decision support systems (CDSS) is used to refer to a system that “provides clinicians, staff, patients or other individuals with knowledge and person-specific information, intelligently filtered or presented at appropriate times, to enhance health and health care” [1].”
BackgroundAdvice taking in decision making: egocentric discounting and reputation formation
Yaniv, I., & Kleinberger, E. (2000) · Organizational Behavior and Human Decision Processes · doi:10.1006/obhd.2000.2909 · Cited by 701 · cited 1×
“Despite this, studies show that when taking advice, regardless of the experiment, the mean WoA is approximately 30%, slightly lower than in our observations [17, 30].”
ResultReceiving other people’s advice: Influence and benefit
Yaniv, I. (2004) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2003.08.002 · Cited by 652
Abstracts written by ChatGPT fool scientists
Else, H. (2023) · Nature · doi:10.1038/d41586-023-00056-7 · Cited by 611 · cited 1×
“Most recently, newer machine learning models have fuelled public discussion about the ethical use of these systems in medicine [10, 11].”
Background
Research works printed but not linked: 1
- Asimov I. I, robot. New York: Doubleday; 1950.
Cited by
Benk, M., Kerstan, S., von Wangenheim, F., & Ferrario, A. (2024) · AI & Society · doi:10.1007/s00146-024-02059-y · Cited by 122
“For example, Laxar et al. (2023) explore the influence of explainable clinical decision support systems on physicians’ trust in decision support systems in the context of rapid triage decisions, such as those that impacted COVID-19.”
Salimparsa, M., Sedig, K., Lizotte, D. J., Abdullah, S. S., Chalabianloo, N., & Muanda, F. T. (2025) · Informatics · doi:10.3390/informatics12040119 · Cited by 39
“Transparency of the underlying model was also a concern (echoed in Laxar et al. [61]).”
InfluentialKomasawa, N., & Yokohira, M. (2025) · Cureus · doi:10.7759/cureus.86316 · Cited by 64
“Despite these advancements, generative AI raises several ethical and professional challenges that must be addressed [10].”
Learner-Centered Experience-Based Medical Education in an AI-Driven Society: A Literature Review
Komasawa, N., & Yokohira, M. (2023) · Cureus · doi:10.7759/cureus.46883 · Cited by 44
“The collaborative integration of AI with the medical field holds immense promise, paving the way for a future where healthcare is more personalized, efficient, and effective [9].”
BackgroundSchmidt, E., Putora, P. M., & Fijten, R. (2025) · Philosophy & Technology · doi:10.1007/s13347-024-00834-9 · Cited by 14
Metadata
- Authors
- Laxar, D., Eitenberger, M., Maleczek, M., Kaider, A., Hammerle, F. P., & Kimberger, O.
- Affiliations
- Ludwig Boltzmann Gesellschaft (AT)
- Medical University of Vienna (AT)
- Corresponding author
- Maleczek, Mathias
- Venue
- BMC Medicine
- Published
- 2023-09-19
- Volume
- 21
- Issue
- 1
- Pages
- 359-359
- ISSN
- 1741-7015 (ISSN-L)
- PMC
- PMC10510231
- OpenAlex
- W4386857282
- Cited by
- 19
- References
- 35
- Citations by year
- 2026: 4 · 2025: 7 · 2024: 7 · 2023: 1
- FWCI
- 0.76
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- BMC Medicine (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
- Keywords
- clinical decision support systems, explainable AI, rapid triage, COVID-19 pandemic, survival probability estimation, critically ill patients, human-AI interaction, physician decision-making
- MeSH terms
- COVID-19, Artificial Intelligence, Humans, Triage, Decision Support Systems, Clinical, Pandemics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The influence of explainable vs non-explainable clinical decision support systems on rapid triage decisions: a mixed methods study. https://app.cochat.ai/papers/10.1186/s12916-023-03068-2
- The paper
- Laxar, D., Eitenberger, M., Maleczek, M., Kaider, A., Hammerle, F. P., & Kimberger, O. (2023). The influence of explainable vs non-explainable clinical decision support systems on rapid triage decisions: a mixed methods study. BMC Medicine. https://doi.org/10.1186/s12916-023-03068-2
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.