Analyzing the quality and challenges of uncertainty estimations for brain tumor segmentation
Open accessAbstract
Automatic segmentation of brain tumors has the potential to enable volumetric measures and high-throughput analysis in the clinical setting. Reaching this potential seems almost achieved, considering the steady increase in segmentation accuracy. However, despite segmentation accuracy, the current methods still do not meet the robustness levels required for patient-centered clinical use. In this regard, uncertainty estimates are a promising direction to improve the robustness of automated segmentation systems. Different uncertainty estimation methods have been proposed, but little is known about their usefulness and limitations for brain tumor segmentation. In this study, we present an analysis of the most commonly used uncertainty estimation methods in regards to benefits and challenges for brain tumor segmentation. We evaluated their quality in terms of calibration, segmentation error localization, and segmentation failure detection. Our results show that the uncertainty methods are typically well-calibrated when evaluated at the dataset level. Evaluated at the subject level, we found notable miscalibrations and limited segmentation error localization (e.g., for correcting segmentations), which hinder the direct use of the voxel-wise uncertainties. Nevertheless, voxel-wise uncertainty showed value to detect failed segmentations when uncertainty estimates are aggregated at the subject level. Therefore, we suggest a careful usage of voxel-wise uncertainty measures and highlight the importance of developing solutions that address the subject-level requirements on calibration and segmentation error localization.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Disagreement among CV models is increasingly reused as a proxy for epistemic uncertainty in downstream applications such as quality control and human-in-the-loop review.
“Increasingly, disagreement among these CV models is reused as a proxy for epistemic uncertainty in downstream applications such as quality control and human-in-the-loop review [11,18,15].”
From Lost in the Folds: When Cross-Validation Is Not a Deep Ensemble for Uncertainty Estimation (2026) · cites this paper as [11]
The Introduction describes uncertainty estimates as useful for visual feedback, guided corrections, and patient-level segmentation failure detection. Section 2.3.4 uses an ensemble of differently trained models to estimate uncertainty, but measures entropy of their averaged probabilities rather than explicitly measuring model disagreement. The paper does not establish that this practice is increasingly reused across computer-vision applications or identify the resulting estimate specifically as epistemic uncertainty; its Discussion also cautions against using voxel-wise uncertainty for guided corrections without better calibration. (LLM, conf=0.84)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
The citation lists haven’t been fetched yet
References 50 · Cited by 115
Research works printed but not linked: 12
- Gal Y. Hron J. Kendall A. (2017). “Concrete dropout,” in Advances in Neural Information Processing Systems 30 (Curran Associates, Inc.), 3581– 3590.
- Glorot X. Bordes A. Bengio Y. (2011). “Deep sparse rectifier neural networks,” in Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, Vol. 15 (Fort Lauderdale, FL: PMLR), 315– 323.
- Hernández-Lobato J. M. Adams R. (2015). “Probabilistic backpropagation for scalable learning of bayesian neural networks,” in Proceedings of the 32nd International Conference on Machine Learning, Vol. 37 (Lille: PMLR), 1861– 1869.
- Ioffe S. Szegedy C. (2015). “Batch normalization: accelerating deep network training by reducing internal covariate shift,” in Proceedings of the 32nd International Conference on Machine Learning, Vol. 37 (Lille: PMLR), 448– 456.
- Jiang H. Kim B. Guan M. Gupta M. (2018). “To trust or not to trust a classifier,” in Advances in Neural Information Processing Systems 31 (Curran Associates, Inc.), 5541– 5552.
- Jungo A. Meier R. Ermis E. Herrmann E. Reyes M. (2018b). “Uncertainty-driven sanity check: application to postoperative brain tumor cavity segmentation,” in International Conference on Medical Imaging with Deep Learning.
- Kendall A. Badrinarayanan V. Cipolla R. (2015). Bayesian segnet: model uncertainty in deep convolutional encoder-decoder architectures for scene understanding. arXiv: 1511.02680. 28060704 · arXiv
- Kendall A. Gal Y. (2017). “What uncertainties do we need in bayesian deep learning for computer vision?,” in Advances in Neural Information Processing Systems 30 (Curran Associates, Inc.), 5574– 5584.
- Lakshminarayanan B. Pritzel A. Blundell C. (2017). “Simple and scalable predictive uncertainty estimation using deep ensembles,” in Advances in Neural Information Processing Systems 30 (Curran Associates, Inc.), 6402– 6413.
- Neal R. M. (1995). Bayesian learning for neural networks. (Ph.D. thesis). University of Toronto, Toronto.
- Neumann L. Zisserman A. Vedaldi A. (2018). “Relaxed softmax: efficient confidence auto-calibration for safe pedestrian detection,” in NIPS Workshop on Machine Learning for Intelligent Transportation Systems.
- Nikolov S. Blackwell S. Mendes R. De Fauw J. Meyer C. Hughes C. (2018). Deep learning to achieve clinically applicable segmentation of head and neck anatomy for radiotherapy. arXiv: 1809.04430. PMC8314151 34255661 · arXiv
- Authors
- Jungo, A., Balsiger, F., & Reyes, M.
- Affiliations
- University of Bern (CH)
- Corresponding author
- Jungo, Alain
- Venue
- Frontiers in Neuroscience
- Publisher
- Frontiers Media SA
- Published
- 2020-04-08
- Volume
- 14
- Pages
- 282-282
- ISSN
- 1662-453X (ISSN-L)
- PMC
- PMC7156850
- OpenAlex
- W3015910149
- Cited by
- 115
- References
- 50
- Citations by year
- 2026: 22 · 2025: 24 · 2024: 17 · 2023: 15 · 2022: 19 · 2021: 14 · 2020: 4
- FWCI
- 5.03
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Frontiers in Neuroscience (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Bern Open Repository and Information System (University of Bern) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Open Access CRIS of the University of Bern
- Funders
- Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung (169607)
- Topics
- Life Sciences › Neuroscience › Neurology › Brain Tumor Detection and Classification
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- brain tumor segmentation, uncertainty estimation, uncertainty calibration
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
