Cited by 428Medical Image Segmentation TechniquesAdvanced Neural Network ApplicationsDomain Adaptation and Few-Shot Learning

The DOI matches its Crossref record; 56 of the 56 references it cites were found in a registry or an index.

Confidence Calibration and Predictive Uncertainty Estimation for Deep Medical Image Segmentation

Open access
Mehrtash, Alireza, Wells, William M., Tempany, Clare M., Abolmaesumi, Purang, Kapur, Tina·2020·IEEE Transactions on Medical Imaging

Abstract

Fully convolutional neural networks (FCNs), and in particular U-Nets, have achieved state-of-the-art results in semantic segmentation for numerous medical imaging applications. Moreover, batch normalization and Dice loss have been used successfully to stabilize and accelerate training. However, these networks are poorly calibrated i.e. they tend to produce overconfident predictions for both correct and erroneous classifications, making them unreliable and hard to interpret. In this paper, we study predictive uncertainty estimation in FCNs for medical image segmentation. We make the following contributions: 1) We systematically compare cross-entropy loss with Dice loss in terms of segmentation quality and uncertainty estimation of FCNs; 2) We propose model ensembling for confidence calibration of the FCNs trained with batch normalization and Dice loss; 3) We assess the ability of calibrated FCNs to predict segmentation quality of structures and detect out-of-distribution test examples. We conduct extensive experiments across three medical image segmentation applications of the brain, the heart, and the prostate to evaluate our contributions. The results of this study offer considerable insight into the predictive uncertainty estimation and out-of-distribution detection in medical image segmentation and provide practical recipes for confidence calibration. Moreover, we consistently demonstrate that model ensembling improves confidence calibration.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/tmi.2020.3006437

Cites

56 of 56 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Flagged first, then by citations

Show all 55 references

Research works printed but not linked: 10

  1. 2018, Tgs Salt Identification Challenge
  2. 2018, Grand Challenge on MR Brain Segmentation at MICCAI 2018
  3. 10.5555/3045118.3045167 · DOI
  4. 10.21236/ADA459827 · DOI
  5. 10.5555/3045390.3045502 · DOI
  6. Liao, 2019, On modelling label uncertainty in deep neural networks: Automatic estimation of intra-observer variability in 2D echocardiography quality assessment, arXiv:1911.00674 · arXiv
  7. 10.1109/CVPR.2018.00464 · DOI
  8. 10.1201/9781420002690 · DOI
  9. 10.5555/3295222.3295309 · DOI
  10. 10.1007/978-3-030-32251-9_73 · DOI

Cited by

Sorted by citations

Items 201–300 of 428

Metadata

Authors
Mehrtash, A., Wells, W. M., Tempany, C. M., Abolmaesumi, P., & Kapur, T.
Affiliations
  • Brigham and Women's Hospital (US)
  • University of British Columbia (CA)
Venue
IEEE Transactions on Medical Imaging
Published
2020-12
Volume
39
Issue
12
Pages
3868-3878
ISSN
  • 0278-0062 (ISSN-L)
  • 1558-254X
OpenAlex
W2991523350
Cited by
428
References
55
Citations by year
2026: 91 · 2025: 92 · 2024: 71 · 2023: 71 · 2022: 56 · 2021: 41 · 2020: 6
FWCI
16.91
Type
journal article
Language
en
Open access
Locations
  • IEEE Transactions on Medical Imaging (publishedVersion)
  • PubMed (publishedVersion)
  • PubMed Central (acceptedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • U.S. National Institutes of Health (P41EB015898)
  • Natural Sciences and Engineering Research Council (NSERC) of Canada
  • Canadian Institutes of Health Research
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
Keywords
confidence calibration, predictive uncertainty estimation, medical image segmentation, fully convolutional network, U-Net, Dice loss, cross-entropy loss, model ensembling, out-of-distribution detection, batch normalization, brain segmentation, cardiac image segmentation, prostate segmentation
MeSH terms
Brain, Calibration, Humans, Image Processing, Computer-Assisted, Male, Neural Networks, Computer, Uncertainty

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review