The DOI matches its Crossref record; 53 of the 56 references it cites were found in a registry or an index.
Evaluation of prostate segmentation algorithms for MRI: The PROMISE12 challenge
Open accessAbstract
Prostate MRI image segmentation has been an area of intense research due to the increased use of MRI as a modality for the clinical workup of prostate cancer. Segmentation is useful for various tasks, e.g. to accurately localize prostate boundaries for radiotherapy or to initialize multi-modal registration algorithms. In the past, it has been difficult for research groups to evaluate prostate segmentation algorithms on multi-center, multi-vendor and multi-protocol data. Especially because we are dealing with MR images, image appearance, resolution and the presence of artifacts are affected by differences in scanners and/or protocols, which in turn can have a large influence on algorithm accuracy. The Prostate MR Image Segmentation (PROMISE12) challenge was setup to allow a fair and meaningful comparison of segmentation methods on the basis of performance and robustness. In this work we will discuss the initial results of the online PROMISE12 challenge, and the results obtained in the live challenge workshop hosted by the MICCAI2012 conference. In the challenge, 100 prostate MR cases from 4 different centers were included, with differences in scanner manufacturer, field strength and protocol. A total of 11 teams from academic research groups and industry participated. Algorithms showed a wide variety in methods and implementation, including active appearance models, atlas registration and level sets. Evaluation was performed using boundary and volume based metrics which were combined into a single score relating the metrics to human expert performance. The winners of the challenge where the algorithms by teams Imorphics and ScrAutoProstate, with scores of 85.72 and 84.29 overall. Both algorithms where significantly better than all other algorithms in the challenge (p<0.05) and had an efficient implementation with a run time of 8min and 3s per case respectively. Overall, active appearance model based approaches seemed to outperform other approaches like multi-atlas registration, both on accuracy and computation time. Although average algorithm performance was good to excellent and the Imorphics algorithm outperformed the second observer on average, we showed that algorithm combination might lead to further improvement, indicating that optimal performance for prostate segmentation is not yet obtained. All results are available online at http://promise12.grand-challenge.org/.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“PROMISE12 as the example in the prostate dataset is the dataset made for MICCAI prostate segmentation challenge, which was used by 11 academic teams for various segmentation algorithms and showed promising results”
From A review of deep learning-based deformable medical image registration (2022) · cites this paper as [55]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Robust Real-Time Face Detection
Viola, P., & Jones, M. J. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000013087.49260.fb · Cited by 14,097
Marching cubes: A high resolution 3D surface construction algorithm
Lorensen, W. E., & Cline, H. E. (1987) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/37401.37422 · Cited by 10,320
Marching cubes: A high resolution 3D surface construction algorithm
Lorensen, W. E., & Cline, H. E. (1987) · ACM SIGGRAPH Computer Graphics · doi:10.1145/37402.37422 · Cited by 8,928
Cootes, T. F., Edwards, G. J., & Taylor, C. J. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.927467 · Cited by 5,409
ESUR prostate MR guidelines 2012
Barentsz, J. O., Richenberg, J., Clements, R., Choyke, P. L., Verma, S., Villeirs, G. M., Rouvière, O., Løgager, V., & Fütterer, J. J. (2012) · European Radiology · doi:10.1007/s00330-011-2377-y · Cited by 2,444
Robust real-time object detection.
Viola, P. A. (2001) · Cited by 2,340
Warfield, S. K., Zou, K. H., & Wells, W. M. (2004) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2004.828354 · Cited by 2,072
Pérez, P., Gangnet, M., & Blake, A. (2003) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/1201775.882269 · Cited by 2,041
Pérez, P., Gangnet, M., & Blake, A. (2003) · ACM Transactions on Graphics · doi:10.1145/882262.882269 · Cited by 1,769
Comparison and evaluation of methods for liver segmentation from CT datasets
Heimann, T., van Ginneken, B., Styner, M. A., Arzhaeva, Y., Aurich, V., Bauer, C., Beck, A., Becker, C., Beichel, R., Bekes, G., et al. (2009) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2009.2013851 · Cited by 1,135
Research works printed but not linked: 8
- Birkbeck, N., Zhang, J., Requardt, M., Kiefer, B., Gall, P., Kevin Zhou, S., 2012. Region-specific hierarchical segmentation of MR prostate using discriminative learning. MICCAI Grand Challenge: Prostate MR Image Segmentation 2012.
- Gao, Q., Rueckert, D., Edwards, P., 2012. An automatic multi-atlas based prostate segmentation using local appearance-specific atlases and patch-based voxel weighting. MICCAI Grand Challenge: Prostate MR Image Segmentation 2012.
- Ghose, S., Mitra, J., Oliver, A., Martí, R., Lladó, X., Freixenet, J., Vilanova, J.C., Sidibé, D., Meriaudeau, F., 2012. A random forest based classification approach to prostate segmentation in MRI. MICCAI Grand Challenge: Prostate MR Image Segmentation 2012.
- Kirschner, M., Jung, F., Wesarg, S., 2012. Automatic prostate segmentation in MR images with a probabilistic active shape model. MICCAI Grand Challenge: Prostate MR Image Segmentation 2012.
- Litjens, G.J.S., Karssemeijer, N., Huisman, H.J., 2012. A multi-atlas approach for prostate segmentation in MR images. MICCAI Grand Challenge: Prostate MR Image Segmentation 2012.
- Toth, 2012, Multifeature landmark-free active appearance models: application to prostate MRI segmentation, IEEE Trans. Med. Imaging, 31, 1638, 10.1109/TMI.2012.2201498 · DOI
- Vincent, G., Guillard, G., Bowes, M., 2012. Fully automatic segmentation of the prostate using active appearance models. MICCAI Grand Challenge: Prostate MR Image Segmentation 2012.
- Yuan, J., Qiu, W., Ukwatta, E., Rajchl, M., Sun, Y., Fenster, A., 2012. An efficient convex optimization approach to 3D prostate MRI segmentation with generic star shape prior. MICCAI Grand Challenge: Prostate MR Image Segmentation 2012.
Cited by
Sorted by citations
V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation
Milletari, F., Navab, N., & Ahmadi, S.-A. (2016) · 2016 Fourth International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2016.79 · Cited by 11,931
nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation
Isensee, F., Jaeger, P. F., Kohl, S. A. A., Petersen, J., & Maier-Hein, K. H. (2021) · Nature Methods · doi:10.1038/s41592-020-01008-z · Cited by 10,477
Metrics for evaluating 3D medical image segmentation: analysis, selection, and tool
Taha, A. A., & Hanbury, A. (2015) · BMC Medical Imaging · doi:10.1186/s12880-015-0068-x · Cited by 3,247
Medical image segmentation using deep learning: A survey
Wang, R., Lei, T., Cui, R., Zhang, B., Meng, H., & Nandi, A. K. (2022) · IET Image Processing · doi:10.1049/ipr2.12419 · Cited by 1,012
Yeung, M., Sala, E., Schönlieb, C., & Rundo, L. (2021) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2021.102026 · Cited by 795
Metadata
- Authors
- Litjens, G., Toth, R., van de Ven, W., Hoeks, C., Kerkstra, S., van Ginneken, B., Vincent, G., Guillard, G., Birbeck, N., Zhang, J., Strand, R., Malmberg, F., Ou, Y., Davatzikos, C., Kirschner, M., Jung, F., Yuan, J., Qiu, W., Gao, Q., Edwards, P. “., Maan, B., van der Heijden, F., Ghose, S., Mitra, J., Dowling, J., Barratt, D., Huisman, H., & Madabhushi, A.
- Affiliations
- Radboud University Medical Center (NL)
- Siemens (United States) (US)
- Uppsala University (SE)
- University of Pennsylvania (US)
- Technische Universität Darmstadt (DE)
- Robarts Research Institute (CA)
- Imperial College London (GB)
- University of Twente (NL)
- Commonwealth Scientific and Industrial Research Organisation (AU)
- University College London (GB)
- Case Western Reserve University (US)
- Corresponding author
- Litjens, Geert
- ORCID
- Venue
- Medical Image Analysis
- Publisher
- Elsevier BV
- Published
- 2014-02
- Volume
- 18
- Issue
- 2
- Pages
- 359-373
- ISSN
- 1361-8415 (ISSN-L)
- PMC
- PMC4137968
- OpenAlex
- W2106033751
- Cited by
- 857
- References
- 56
- Citations by year
- 2026: 107 · 2025: 127 · 2024: 95 · 2023: 106 · 2022: 92 · 2021: 87 · 2020: 71 · 2019: 60 · 2018: 35 · 2017: 25 · 2016: 26 · 2015: 16 · 2014: 10
- FWCI
- 19.49
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Medical Image Analysis (publishedVersion)
- PubMed (publishedVersion)
- University of Twente Research Information (publishedVersion)
- PubMed Central (acceptedVersion)
- UCL Discovery (University College London) (submittedVersion)
- Publikationsdatenbank der Fraunhofer-Gesellschaft (Fraunhofer-Gesellschaft) (submittedVersion)
- Fraunhofer-Publica (Fraunhofer-Gesellschaft) (submittedVersion)
- Radboud Repository (Radboud University) (submittedVersion)
- Radboud Repository (Radboud University) (submittedVersion, other-oa)
- University of Twente Research Information (submittedVersion)
- Funders
- Dutch Cancer Society (KUN2007-3971)
- National Cancer Institute of the National Institutes of Health
- National Institute of Biomedical Imaging and Bioengineering of the National Institutes of Health
- National Science Foundation
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Brain Tumor Detection and Classification
- Physical Sciences › Engineering › Biomedical Engineering › Medical Imaging and Analysis
- Keywords
- prostate segmentation, prostate MRI, active appearance models, multi-atlas registration, level sets, segmentation algorithm comparison, prostate cancer, algorithm combination
- MeSH terms
- Algorithms, Humans, Image Enhancement, Image Interpretation, Computer-Assisted, Magnetic Resonance Imaging, Male, Prostatic Neoplasms, Reference Standards, Sensitivity and Specificity, Reproducibility of Results, Artifacts, Imaging, Three-Dimensional
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
