From Oracular to Judicial: Enhancing Clinical Decision Making through Contrasting Explanations and a Novel Interaction Protocol
Open accessAbstract
Clinical Decision Support Systems (CDSS) utilizing machine learning (ML) classifiers have demonstrated substantial potential for improving diagnostic accuracy across various medical domains. However, concerns regarding automation bias, diminished sense of agency, and over-reliance on these systems remain, particularly in clinical settings where decision-making autonomy is critical.To address these challenges, we propose "Judicial AI,"an innovative interaction protocol aimed at reducing automation bias and preserving a sense of agency. This system presents contrasting explanations to medical professionals rather than definitive recommendations, encouraging user engagement and critical evaluation.Before adopting interaction protocols that avoid definitive recommendations, it is important to assess whether such an approach impacts diagnostic accuracy, and if so, how. This paper reports an exploratory study investigating the efficacy of a Judicial CDSS in the diagnosis of vertebral fractures from X-ray images. Sixteen medical professionals, comprising spine surgeons and radiologists, participated in the diagnosis of 18 X-ray images, which were carefully selected to represent particularly difficult and complex cases. Diagnosticians first recorded their decisions independently and then with support from the Judicial AI, which provided activation maps for opposing diagnoses.Our findings show a significant improvement in diagnostic accuracy for complex cases among experienced users (…
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Clinicians may become over-reliant on automation, placing excessive trust in system recommendations and neglecting their own clinical judgment, which increases the risk of errors when the system's recommendations are incorrect.
“Clinicians may also become over-reliant on automation, placing excessive trust in system recommendations and neglecting their own clinical judgment, which increases the risk of errors when the system’s recommendations are incorrect [230,231].”
From Toward Safer Diagnoses: A SEIPS-Based Narrative Review of Diagnostic Errors (2026) · cites this paper as [231]
Cites
Larian Studios Dev Team (2024) · doi:10.1145/3292500.3330701 · Cited by 8,618 · see citing works
Complacency and Bias in Human Use of Automation: An Attentional Integration
Parasuraman, R., & Manzey, D. H. (2010) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720810376055 · Cited by 1,710 · cited 1× · see citing works
“This phenomenon is a case of automation bias, but it differs substantially from the well-documented instances in the literature [22, 43].”
Automation bias: a systematic review of frequency, effect mediators, and mitigators
Goddard, K., Roudsari, A., & Wyatt, J. C. (2012) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000089 · Cited by 1,162 · cited 2× · see citing works
“This phenomenon is a case of automation bias, but it differs substantially from the well-documented instances in the literature [22, 43].”
Buçinca, Z., Malaya, M. B., & Gajos, K. Z. (2021) · arXiv · doi:10.48550/arxiv.2102.09692 · Cited by 1,124 · see citing works
Zhang, Y., Liao, Q. V., & Bellamy, R. K. E. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372852 · Cited by 895 · see citing works
Automation bias and verification complexity: a systematic review
Lyell, D., & Coiera, E. (2017) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocw105 · Cited by 451 · see citing works
The Perceived Utility of Human and Automated Aids in a Visual Detection Task
Dzindolet, M. T., Pierce, L. G., Beck, H. P., & Dawe, L. A. (2002) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/0018720024494856 · Cited by 386 · see citing works
Randomized Clinical Trials of Machine Learning Interventions in Health Care: A Systematic Review
Plana, D., Shung, D. L., Grimshaw, A. A., Saraf, A., Sung, J. J. Y., & Kann, B. H. (2022) · JAMA Network Open · doi:10.1001/jamanetworkopen.2022.33946 · Cited by 236 · see citing works
How to obtain nnt from cohen’s D: comparison of two methods
Furukawa, T. A., & Leucht, S. (2011) · PLoS ONE · doi:10.1371/journal.pone.0019070 · Cited by 179 · see citing works
Miller, T. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594001 · Cited by 172 · see citing works
Aquino, Y. S. J., Rogers, W. A., Braunack-Mayer, A., Frazer, H., Win, K. T., Houssami, N., Degeling, C., Semsarian, C., & Carter, S. M. (2023) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2022.104903 · Cited by 131 · see citing works
Meyners, M. (2012) · Food Quality and Preference · doi:10.1016/j.foodqual.2012.05.003 · Cited by 130 · cited 1× · see citing works
“A bootstrap analysis based on the BCa method was conducted to ensure the reliability and precision of both effect size estimates and for the Equivalence Test [36].”
Harnessing the potential of large language models in medical education: promise and pitfalls
Benítez, T. M., Xu, Y., Boudreau, J. D., Kow, A. W. C., Bello, F., Van Phuoc, L., Wang, X., Sun, X., Leung, G. K.-K., Lan, Y., et al. (2024) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad252 · Cited by 100 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Natali, C., Marconi, L., Dias Duran, L. D., & Cabitza, F. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11352-1 · Cited by 143 · see citing works
Self-citationToward Safer Diagnoses: A SEIPS-Based Narrative Review of Diagnostic Errors
Yen, C., Epling, J. W., Rockwell, M., & Vaughn-Cooke, M. (2026) · Diagnostics · doi:10.3390/diagnostics16020347 · Cited by 6 · see citing works
Raees, M., Khan, V.-J., Lykourentzou, I., & Papangelis, K. (2026) · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791467 · Cited by 5 · see citing works
Pe, S., Bergomi, L., Nicora, G., Simonelli, C. A., Kour, P., Diaz, E., Alendal, G., Hernáiz Ferrer, A. I., Corso, V., Bortolotto, C., et al. (2026) · Machine Learning and Knowledge Extraction · doi:10.3390/make8070216
“…considerations discussed in Section 1.2; while previous studies have shown that adjunct-based interaction protocols may negatively affect decision quality [41], Cabitza et al. [38] demonstrated that combining adjunct protocols with judicial principles can produce more favorable outcomes.”
InfluentialSelf-citationFrom Oracles to Productive Friction: Designing for Meaningful Human Oversight in Clinical AI
Rubegni, E., Natali, C., Valsecchi, C., Rizzo, S., Ayoub, O., & Facchini, A. (2026) · Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1609/aies.v9i3.43300
- Authors
- Cabitza, F., Famiglini, L., Fregosi, C., Pe, S., Parimbelli, E., La Maida, G. A., & Gallazzi, E.
- Affiliations
- IRCCS Ospedale Galeazzi - Sant'Ambrogio (IT)
- University of Milano-Bicocca (IT)
- University of Pavia (IT)
- ASST G. Pini - CTO, Milan, Italy,
- Istituto Ortopedico Gaetano Pini (IT)
- ORCID
- Venue
- Proceedings of the 30th International Conference on Intelligent User Interfaces
- Publisher
- ACM
- Published
- 2025-03-24
- Pages
- 745-754
- OpenAlex
- W4408615376
- Cited by
- 11
- References
- 13
- Citations by year
- 2026: 10 · 2025: 1
- FWCI
- 11.51
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM International Conference on Intelligent User Interfaces (IUI) (publishedVersion, cc-by-nc-sa)
- BOA (University of Milano-Bicocca) (submittedVersion)
- Funders
- Italian Ministry of Reaserch and University (H53D23008090001)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Keywords
- clinical decision support systems, automation bias, sense of agency, activation maps, X-ray images, diagnostic accuracy, user confidence, cognitive load, radiologists, spine surgeons, human-AI interaction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
