Uncertainty Aware Segmentation Quality Assessment in Medical Images
Abstract
Image segmentation is a fundamental step in most computational biomedical image analysis pipelines. During model training and validation, we can measure segmentation performance using well-established similarity metrics like the Dice coefficient. However, once the model is deployed in a clinical scenario, this is no longer possible as manual annotations are not available. In addition, segmentation models that produce a solution with no indication of its reliability result in harder adoption by end-users. To approach these two challenges, this paper introduces a segmentation quality prediction framework that does not rely on manual annotations in test time. This framework integrates uncertainty estimates on the underlying segmentation model, which we show to be advantageous for quality scoring purposes. We validate our approach on a popular skin lesion segmentation dataset, carefully analyzing the impact of different uncertainty modeling and estimation techniques on the performance of segmentation quality prediction performance.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
FPN with ResNet50 has been widely used for skin cancer segmentation, with Esteva et al. and Codella et al. highlighting its effectiveness in capturing fine-grained details in dermoscopic images.
“Similarly, FPN with ResNet50 has been widely used for skin cancer segmentation, with Esteva et al. [30] and Codella et al. [31] highlighting its effective- ness in capturing fine-grained details in dermoscopic im- ages.”
Cites
nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation
Isensee, F., Jaeger, P. F., Kohl, S. A. A., Petersen, J., & Maier-Hein, K. H. (2021) · Nature Methods · doi:10.1038/s41592-020-01008-z · Cited by 10,477 · see citing works
Dropout as a bayesian approximation: Representing model uncertainty in deep learning
Gal, Y., & Ghahramani, Z. (2015) · arXiv · doi:10.48550/arxiv.1506.02142 · Cited by 6,471 · cited 1× · see citing works
“We illustrate our approach with various popular UMs: 1) Monte Carlo Dropout (MCD, [12]), 2) Ensemble models [13], 3) Test Time Augmentation (TTA, [14]), and UE strategies, e.g. maximum confidence, entropy, mutual information, and expected pair-wise Kullback-Leibler (KL) divergence [15].”
MethodTschandl, P., Rosendahl, C., & Kittler, H. (2018) · Scientific Data · doi:10.1038/sdata.2018.161 · Cited by 3,628 · see citing works
UNETR: Transformers for 3D Medical Image Segmentation
Hatamizadeh, A., Tang, Y., Nath, V., Yang, D., Myronenko, A., Landman, B., Roth, H. R., & Xu, D. (2022) · 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv51458.2022.00181 · Cited by 3,586 · cited 1× · see citing works
“Although Deep Neural Networks have significantly advanced the state of the art on many medical image segmentation problems over the last years [2], their integration into clinical practice remains limited.”
BackgroundA review of uncertainty quantification in deep learning: Techniques, applications and challenges
Abdar, M., Pourpanah, F., Hussain, S., Rezazadegan, D., Liu, L., Ghavamzadeh, M., Fieguth, P., Cao, X., Khosravi, A., Acharya, U. R., et al. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.05.008 · Cited by 2,720 · see citing works
Wang, G., Li, W., Aertsen, M., Deprest, J., Ourselin, S., & Vercauteren, T. (2019) · Neurocomputing · doi:10.1016/j.neucom.2019.01.103 · Cited by 659 · see citing works
Understanding metric-related pitfalls in image analysis validation
Reinke, A., Tizabi, M. D., Baumgartner, M., Eisenmann, M., Heckmann-Nötzel, D., Kavur, A. E., Rädsch, T., Sudre, C. H., Acion, L., Antonelli, M., et al. (2024) · Nature Methods · doi:10.1038/s41592-023-02150-0 · Cited by 241 · see citing works
Reverse Classification Accuracy: Predicting Segmentation Performance in the Absence of Ground Truth
Valindria, V. V., Lavdas, I., Bai, W., Kamnitsas, K., Aboagye, E. O., Rockall, A. G., Rueckert, D., & Glocker, B. (2017) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2017.2665165 · Cited by 152 · see citing works
Leveraging Uncertainty Estimates for Predicting Segmentation QualityPreprint
DeVries, T., & Taylor, G. W. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.00502 · Cited by 117 · see citing works
Reverse KL-Divergence Training of Prior Networks: Improved Uncertainty and Adversarial RobustnessPreprint
Gales, M., & Malinin, A. (2019) · arXiv (Cornell University) · doi:10.17863/cam.45323 · Cited by 117 · see citing works
Estimating Uncertainty in Neural Networks for Cardiac MRI Segmentation: A Benchmark Study
Ng, M., Guo, F., Biswas, L., Petersen, S. E., Piechnik, S., Neubauer, S., & Wright, G. (2020) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2022.3232730 · Cited by 59 · see citing works
Uncertainty Estimation in Autoregressive Structured Prediction
Gales, M., & Malinin, A. (2021) · Apollo - University of Cambridge Repository · doi:10.17863/cam.63497 · Cited by 48 · see citing works
QualityNet: Segmentation quality evaluation with deep convolutional networks
Huang, C., Wu, Q., & Meng, F. (2016) · 2016 Visual Communications and Image Processing (VCIP) · doi:10.1109/vcip.2016.7805585 · Cited by 40 · see citing works
Galdrán, A., Carneiro, G., & Ballester, M. Á. G. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-26354-5_4 · Cited by 36 · see citing works
A Novel Quality Control Algorithm for Medical Image Segmentation Based on Fuzzy Uncertainty
Qiao, L., Chen, X., Chen, C., & Garibaldi, J. M. (2022) · IEEE Transactions on Fuzzy Systems · doi:10.1109/tfuzz.2022.3228332 · Cited by 32 · see citing works
A No-Reference Quality Metric for Retinal Vessel Tree Segmentation
Galdrán, A., Costa, P., Bria, A., Araújo, T., Mendonça, A. M., & Campilho, A. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00928-1_10 · Cited by 21 · see citing works
Robust Image Segmentation Quality AssessmentPreprint
Zhou, L., Deng, W., & Wu, X. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.08773 · Cited by 12 · see citing works
Subject-level Prediction of Segmentation Failure using Real-Time Convolutional Neural Nets
Robinson, R. H., Oktay, O., Bai, W., Valindria, V. V., Sanghvi, M. M., Aung, N., Paiva, J. M., Zemrak, F., Fung, K., Lukaschuk, E. I., et al. (2018) · Cited by 11 · see citing works
Molchanova, N., Raina, V., Malinin, A., Rosa, F. L., Müller, H., Gales, M., Granziera, C., Graziani, M., & Cuadra, M. B. (2023) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi53787.2023.10230563 · Cited by 9 · see citing works
Flagged first, then by citations
1 of the 20 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
O.K., S., Riera-Marín, M., Galdran, A., López, J. G., Rodríguez-Comas, J., Piella, G., & Ballester, M. A. G. (2025) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2025.102547 · Cited by 11 · see citing works
Self-citationSikha, O. K., Stone, A. L. B., & Ballester, M. Á. G. (2025) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-025-03490-2 · Cited by 3 · see citing works
Ribeiro, M. A. O., Gutierrez, M. A., & Nunes, F. L. S. (2026) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-026-01940-0
Reducing Diagnostic Ambiguity Through Adaptive Uncertainty Modeling in Medical Imaging Systems
VanithaLakshmi, M., Vignesh, G. D., Samara, G., Meenatchi, N., Chinnaiah, K., & Ramachandran, R. P. (2026) · doi:10.1109/icstem68144.2026.11735467
- Authors
- O K, S., Galdran, A., Riera-Marin, M., García, J., Rodríguez-Comas, J., Piella, G., & González Ballester, M. A.
- Affiliations
- Universitat Pompeu Fabra,BCN Medtech,Dept. of Information and Communication Technologies,Barcelona,Spain
- Sycai Technologies S.L.,Barcelona,Spain
- Universitat Pompeu Fabra (ES)
- Venue
- 2024 IEEE International Symposium on Biomedical Imaging (ISBI)
- Publisher
- IEEE
- Published
- 2024-05-27
- Pages
- 1-5
- OpenAlex
- W4401753205
- Cited by
- 4
- References
- 20
- Citations by year
- 2026: 2 · 2025: 2
- FWCI
- 0.83
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE International Symposium on Biomedical Imaging (ISBI) (publishedVersion)
- TECNALIA Publications (Fundación TECNALIA Research & Innovation) (submittedVersion, public-domain)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- segmentation quality assessment, uncertainty estimation, medical image segmentation, skin lesion segmentation, uncertainty modeling
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
