Cited by 857Medical Image Segmentation TechniquesBrain Tumor Detection and ClassificationMedical Imaging and Analysis

Evaluation of prostate segmentation algorithms for mri: The promise12 challenge

Open access
Litjens, Geert, Toth, Robert, van de Ven, Wendy, Hoeks, Caroline, Kerkstra, Sjoerd, van Ginneken, Bram, Vincent, Graham, Guillard, Gwenael +20 more·2014·Medical Image Analysis

Abstract

Prostate MRI image segmentation has been an area of intense research due to the increased use of MRI as a modality for the clinical workup of prostate cancer. Segmentation is useful for various tasks, e.g. to accurately localize prostate boundaries for radiotherapy or to initialize multi-modal registration algorithms. In the past, it has been difficult for research groups to evaluate prostate segmentation algorithms on multi-center, multi-vendor and multi-protocol data. Especially because we are dealing with MR images, image appearance, resolution and the presence of artifacts are affected by differences in scanners and/or protocols, which in turn can have a large influence on algorithm accuracy. The Prostate MR Image Segmentation (PROMISE12) challenge was setup to allow a fair and meaningful comparison of segmentation methods on the basis of performance and robustness. In this work we will discuss the initial results of the online PROMISE12 challenge, and the results obtained in the live challenge workshop hosted by the MICCAI2012 conference. In the challenge, 100 prostate MR cases from 4 different centers were included, with differences in scanner manufacturer, field strength and protocol. A total of 11 teams from academic research groups and industry participated. Algorithms showed a wide variety in methods and implementation, including active appearance models, atlas registration and level sets. Evaluation was performed using boundary and volume based metrics which were combined into a single score relating the metrics to human expert performance. The winners of the challenge where the algorithms by teams Imorphics and ScrAutoProstate, with scores of 85.72 and 84.29 overall. Both algorithms where significantly better than all other algorithms in the challenge (p<0.05) and had an efficient implementation with a run time of 8min and 3s per case respectively. Overall, active appearance model based approaches seemed to outperform other approaches like multi-atlas registration, both on accuracy and computation time. Although average algorithm performance was good to excellent and the Imorphics algorithm outperformed the second observer on average, we showed that algorithm combination might lead to further improvement, indicating that optimal performance for prostate segmentation is not yet obtained. All results are available online at http://promise12.grand-challenge.org/.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.media.2013.12.002

Cites

53 of 53 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • PROMISE12 is the dataset made for the MICCAI prostate segmentation challenge, used by 11 academic teams for various segmentation algorithms and showed promising results.

    “PROMISE12 as the example in the prostate dataset is the dataset made for MICCAI prostate segmentation challenge, which was used by 11 academic teams for various segmentation algorithms and showed promising results”

    From A review of deep learning-based deformable medical image registration (2022) · cites this paper as [55]

    The Introduction describes the PROMISE12 dataset and its use in a challenge associated with the MICCAI2012 workshop. The Abstract reports 11 participating teams, varied segmentation methods, and good-to-excellent average algorithm performance. However, the teams came from both academic research groups and industry, not exclusively academia. (LLM, conf=0.90)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

3 of the 56 references couldn’t be matched to a known paper.

Research works printed but not linked: 8

  1. Birkbeck N Zhang J Requardt M Kiefer B Gall P Kevin Zhou S Region-Specific Hierarchical Segmentation of MR Prostate Using Discriminative Learning MICCAI Grand Challenge: Prostate MR Image Segmentation 2012 2012
  2. Gao Q Rueckert D Edwards P An automatic multi-atlas based prostate segmentation using local appearance-specific atlases and patchbased voxel weighting MICCAI Grand Challenge: Prostate MR Image Segmentation 2012 2012a
  3. Ghose S Mitra J Oliver A Martí R Lladó X Freixenet J Vilanova JC Sidibé D Meriaudeau F A Random Forest Based Classification Approach to Prostate Segmentation in MRI MICCAI Grand Challenge: Prostate MR Image Segmentation 2012 2012
  4. Kirschner M Jung F Wesarg S Automatic Prostate Segmentation in MR Images with a Probabilistic Active Shape Model MICCAI Grand Challenge: Prostate MR Image Segmentation 2012 2012
  5. Litjens GJS Karssemeijer N Huisman HJ A multi-atlas approach for prostate segmentation in MR images MICCAI Grand Challenge: Prostate MR Image Segmentation 2012 2012
  6. Toth R Madabhushi A Deformable Landmark-Free Active Appearance Models: Application to Segmentation of Multi-Institutional Prostate MRI Data MICCAI Grand Challenge: Prostate MR Image Segmentation 2012 2012a 22665505
  7. Vincent G Guillard G Bowes M Fully Automatic Segmentation of the Prostate using Active Appearance Models MICCAI Grand Challenge: Prostate MR Image Segmentation 2012 2012
  8. Yuan J Qiu W Ukwatta E Rajchl M Sun Y Fenster A An Efficient Convex Optimization Approach to 3D Prostate MRI Segmentation with Generic Star Shape Prior MICCAI Grand Challenge: Prostate MR Image Segmentation 2012 2012

Cited by

Sorted by citations

Authors
Litjens, G., Toth, R., van de Ven, W., Hoeks, C., Kerkstra, S., van Ginneken, B., Vincent, G., Guillard, G., Birbeck, N., Zhang, J., Strand, R., Malmberg, F., Ou, Y., Davatzikos, C., Kirschner, M., Jung, F., Yuan, J., Qiu, W., Gao, Q., Edwards, P. “., Maan, B., van der Heijden, F., Ghose, S., Mitra, J., Dowling, J., Barratt, D., Huisman, H., & Madabhushi, A.
Affiliations
  • Radboud University Medical Center (NL)
  • Siemens (United States) (US)
  • Uppsala University (SE)
  • University of Pennsylvania (US)
  • Technische Universität Darmstadt (DE)
  • Robarts Research Institute (CA)
  • Imperial College London (GB)
  • University of Twente (NL)
  • Commonwealth Scientific and Industrial Research Organisation (AU)
  • University College London (GB)
  • Case Western Reserve University (US)
Corresponding author
Litjens, Geert
Venue
Medical Image Analysis
Publisher
Elsevier BV
Published
2014-02
Volume
18
Issue
2
Pages
359-373
ISSN
1361-8415 (ISSN-L)
OpenAlex
W2106033751
Cited by
857
References
56
Citations by year
2026: 107 · 2025: 127 · 2024: 95 · 2023: 106 · 2022: 92 · 2021: 87 · 2020: 71 · 2019: 60 · 2018: 35 · 2017: 25 · 2016: 26 · 2015: 16 · 2014: 10
FWCI
19.49
Type
journal article
Language
en
Open access
Locations
  • Medical Image Analysis (publishedVersion)
  • PubMed (publishedVersion)
  • University of Twente Research Information (publishedVersion)
  • PubMed Central (acceptedVersion)
  • UCL Discovery (University College London) (submittedVersion)
  • Publikationsdatenbank der Fraunhofer-Gesellschaft (Fraunhofer-Gesellschaft) (submittedVersion)
  • Fraunhofer-Publica (Fraunhofer-Gesellschaft) (submittedVersion)
  • Radboud Repository (Radboud University) (submittedVersion)
  • Radboud Repository (Radboud University) (submittedVersion, other-oa)
  • University of Twente Research Information (submittedVersion)
Funders
  • Dutch Cancer Society (KUN2007-3971)
  • National Cancer Institute of the National Institutes of Health
  • National Institute of Biomedical Imaging and Bioengineering of the National Institutes of Health
  • National Science Foundation
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
  • Life Sciences › Neuroscience › Neurology › Brain Tumor Detection and Classification
  • Physical Sciences › Engineering › Biomedical Engineering › Medical Imaging and Analysis
Keywords
prostate segmentation, prostate MRI, active appearance models, multi-atlas registration, level sets, segmentation algorithm comparison, prostate cancer, algorithm combination
MeSH terms
Algorithms, Humans, Image Enhancement, Image Interpretation, Computer-Assisted, Magnetic Resonance Imaging, Male, Prostatic Neoplasms, Reference Standards, Sensitivity and Specificity, Reproducibility of Results, Artifacts, Imaging, Three-Dimensional

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review