CoChat
Sign Up
PreprintCited by 0Artificial Intelligence in Healthcare and EducationMedical Image Segmentation TechniquesAdversarial Robustness in Machine Learning

Failure detection in biomedical image classification under realistic distribution shifts: insights from a large-scale evaluation

Open access
Steinmetz, Paul, Frouin, Frédérique, Morard, Vincent, Buvat, Irène·2026·medRxiv
DOI 10.64898/2026.05.04.26350496
Open this paper in CoChat — free account

Abstract

Biomedical images (BMI) exhibit variability due to different acquisition protocols, devices, and patient populations, making failure detection at inference time essential for reliable deployment of clinical classifiers. As existing evaluations of failure detection methods use different settings, it is difficult to compare results and identify the best strategy, if any. We present a comprehensive evaluation of eight confidence scoring functions and two score-aggregation strategies across eight BMI tasks spanning diverse modalities, backbone architectures, training setups, and failure sources. The confidence ranking ability and classification error mitigation are jointly evaluated. While no single method systematically dominated across settings, aggregation of confidence scores consistently matched or approached the best individual method and substantially reduced silent failure rate. The failure detection performance was strongly correlated with classifier accuracy for all tested settings. These findings provide large-scale evidence of the strengths and limitations of confidence scoring strategies and offer actionable guidance for mitigating silent failures under realistic distribution shifts in BMI. Graphical Abstract: Highlights: Failure detection performance is independent of biomedical image modality but strongly correlated with classifier accuracy. Monte Carlo Dropout does not outperform simple margin-based scores despite substantially higher computational costs. Aggregating standardized confidence scores provides more robust performance than individual methods and reduces mean silent failure rates from 24% down to 8% for realistic distribution shifts. Cross-validation ensembling improves both classification accuracy and failure detection performance.

Integrity signals

Registry

Crossref · 10.64898/2026.05.04.26350496

Cites

30 of 30 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Cites

Flagged first, then by citations

Research works printed but not linked: 17

  1. 2019, Skin lesion analysis toward melanoma detection 2018: A challenge hosted by the international skin imaging collaboration (isic), arXiv preprint
  2. 2024, Medmnist-c: Comprehensive benchmark and improved classifier robustness by simulating realistic image corruptions, arXiv preprint
  3. Gal, Y. , Ghahramani, Z. , 2016. Dropout as a bayesian approximation: Representing model uncertainty in deep learning, in: international conference on machine learning, PMLR. pp. 1050–1059.
  4. Guo, C. , Pleiss, G. , Sun, Y. , Weinberger, K.Q. , 2017. On calibration of modern neural networks, in: International conference on machine learning, PMLR. pp. 1321–1330.
  5. 2019, Scaling out-of-distribution detection for real-world settings, arXiv preprint
  6. 2016, A baseline for detecting misclassified and out-of-distribution examples in neural networks, arXiv preprint
  7. 2022, A call to reflect on evaluation practices for failure detection in image classification, arXiv preprint
  8. 2022, Amos: A large-scale abdominal multi-organ benchmark for versatile medical image segmentation, arXiv preprint
  9. Lakshminarayanan, B. , Pritzel, A. , Blundell, C. , 2017. Simple and scalable predictive uncertainty estimation using deep ensembles. Advances in neural information processing systems 30.
  10. 2017, Leveraging uncertainty information from deep neural networks for disease detection, Scientific reports, 7, 1
  11. Lyzhov, A. , Molchanova, Y. , Ashukha, A. , Molchanov, D. , Vetrov, D. , 2020. Greedy policy search: A simple baseline for learnable test-time augmentation, in: Conference on uncertainty in artificial intelligence, PMLR. pp. 1308–1317.
  12. 2025, A systematic analysis of out-of-distribution detection under representation and training paradigm shifts, arXiv preprint
  13. Sun, Y. , Ming, Y. , Zhu, X. , Li, Y. , 2022. Out-of-distribution detection with deep nearest neighbors, in: International conference on machine learning, PMLR. pp. 20827–20840.
  14. 2024, Overcoming common flaws in the evaluation of selective classification systems, Advances in Neural Information Processing Systems, 37, 2323
  15. 2018, The ham10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions, Scientific data, 5, 1
  16. 2021, A fine-grained analysis on distribution shift, arXiv preprint
  17. 2022, Openood: Benchmarking generalized out-of-distribution detection, Advances in Neural Information Processing Systems, 35, 32598

Cited by

Still being checked.

Authors
Steinmetz, P., Frouin, F., Morard, V., & Buvat, I.
Affiliations
  • Centre National de la Recherche Scientifique (FR)
  • Université Paris-Saclay (FR)
  • GE HealthCare, Buc, 78530, France
ORCID
Venue
medRxiv
Publisher
openRxiv
Published
2026-05-05
DOI
10.64898/2026.05.04.26350496
OpenAlex
W7160306460
Semantic Scholar
bb46cc54507e46389150c9dc442fc464e1c2a6d8
Cited by
0
References
30
Type
posted content
Open access
Licence
Locations
medRxiv (acceptedVersion, cc-by-nc)
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
Keywords
fault detection, biomedical image classification, distribution shift, confidence scores, silent failures, Monte Carlo dropout, evaluation benchmark

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review