Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Sign Up
Cited by 524

The DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.

ISLES 2015 - A public evaluation benchmark for ischemic stroke lesion segmentation from multispectral MRI

Open access
Maier, Oskar, Menze, Bjoern H., von der Gablentz, Janina, Häni, Levin, Heinrich, Mattias P., Liebrand, Matthias, Winzeck, Stefan, Basit, Abdul, Bentley, Paul, Chen, Liang, Christiaens, Daan, Dutil, Francis, Egger, Karl, Feng, Chaolu, Glocker, Ben, Götz, Michael, Haeck, Tom, Halme, Hanna-Leena, Havaei, Mohammad, Iftekharuddin, Khan M., Jodoin, Pierre-Marc, Kamnitsas, Konstantinos, Kellner, Elias, Korvenoja, Antti, Larochelle, Hugo, Ledig, Christian, Lee, Jia-Hong, Maes, Frederik, Mahmood, Qaiser, Maier-Hein, Klaus H., McKinley, Richard, Muschelli, John, Pal, Chris, Pei, Linmin, Rangarajan, Janaki Raman, Reza, Syed M.S., Robben, David, Rueckert, Daniel, Salli, Eero, Suetens, Paul, Wang, Ching-Wei, Wilms, Matthias, Kirschke, Jan S., Krämer, Ulrike M., Münte, Thomas F., Schramm, Peter, Wiest, Roland, Handels, Heinz, Reyes, Mauricio·2017·Medical Image Analysis
DOI 10.1016/j.media.2016.07.009
Open this paper in CoChat — free account

Abstract

&NA; Ischemic stroke is the most common cerebrovascular disease, and its diagnosis, treatment, and study relies on non‐invasive imaging. Algorithms for stroke lesion segmentation from magnetic resonance imaging (MRI) volumes are intensely researched, but the reported results are largely incomparable due to different datasets and evaluation schemes. We approached this urgent problem of comparability with the Ischemic Stroke Lesion Segmentation (ISLES) challenge organized in conjunction with the MICCAI 2015 conference. In this paper we propose a common evaluation framework, describe the publicly available datasets, and present the results of the two sub‐challenges: Sub‐Acute Stroke Lesion Segmentation (SISS) and Stroke Perfusion Estimation (SPES). A total of 16 research groups participated with a wide range of state‐of‐the‐art automatic segmentation algorithms. A thorough analysis of the obtained data enables a critical evaluation of the current state‐of‐the‐art, recommendations for further developments, and the identification of remaining challenges. The segmentation of acute perfusion lesions addressed in SPES was found to be feasible. However, algorithms applied to sub‐acute lesion segmentation in SISS still lack accuracy. Overall, no algorithmic characteristic of any method was found to perform superior to the others. Instead, the characteristics of stroke lesion appearances, their evolution, and the observed challenges should be studied in detail. The annotated ISLES image datasets continue to be publicly available through an online evaluation system to serve as an ongoing benchmarking resource (www.isles‐challenge.org). HighlightsEvaluation framework for automatic stroke lesion segmentation from MRIPublic multi‐center, multi‐vendor, multi‐protocol databases releasedOngoing fair and automated benchmark with expert created ground truth setsComparison of 14+7 groups who responded to an open challenge in MICCAISegmentation feasible in acute and unsolved in sub‐acute cases Graphical abstract Figure. No caption available.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.media.2016.07.009

Cites

10 of 10 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 10 references

Research works printed but not linked: 52

  1. Albers, 2006, Magnetic resonance imaging profiles predict clinical response to early reperfusion: the diffusion and perfusion imaging evaluation for understanding stroke evolution (DEFUSE) study, Ann. Neurol., 60, 508, 10.1002/ana.20976 · DOI · Paper page
  2. Artzi, 2013, FLAIR lesion segmentation: application in patients with brain tumors and acute ischemic stroke, Eur. J. Radiol., 82, 1512, 10.1016/j.ejrad.2013.05.029 · DOI
  3. Avants, 2008, Symmetric diffeomorphic image registration with cross-correlation: evaluating automated labeling of elderly and neurodegenerative brain, Med. Image Anal., 12, 26, 10.1016/j.media.2007.06.004 · DOI · Paper page
  4. Bauer, 2013, A skull-stripping filter for ITK, Insight J, 10.54294/dp4mfp · DOI
  5. Crimi, A., Maier, O., Menze, B., Reyes, M., Handels, H. (Eds.), 2016. In: LNCS Brainlesion: Glioma, MS, Stroke and Traumatic Brain Injuries - First International BrainLes Workshop MICCAI 2015. Springer. · DOI
  6. Criminisi, A., Shotton, J. (Eds.), 2013. In: Decision forests for computer vision and medical image analysis. Springer. · DOI · Paper page
  7. Dastidar, 2000, Volumetric measurements of right cerebral hemisphere infarction: use of a semiautomatic MRI segmentation technique, Comput. Biol. Med., 30, 41, 10.1016/S0010-4825(99)00022-0 · doi:10.1016/s0010-4825(99)00022-0
  8. Fiez, 2000, Lesion segmentation and manual warping to a reference brain: intra- and interobserver reliability, Hum. Brain Mapp., 9, 192, 10.1002/(SICI)1097-0193(200004)9:4<192::AID-HBM2>3.0.CO;2-Y · DOI
  9. Forkert, 2013, Comparison of 10 TTP and Tmax estimation techniques for MR perfusion-diffusion mismatch quantification in acute stroke, Am. J. Neuroradiol., 34, 1697, 10.3174/ajnr.A3460 · doi:10.3174/ajnr.a3460
  10. Galar, 2013, Dynamic classifier selection for One-vs-One strategy: avoiding non-competent classifiers, Pattern Recognit., 46, 3412, 10.1016/j.patcog.2013.04.018 · DOI
  11. Gao, 2014, Non-locally regularized segmentation of multiple sclerosis lesion from multi-channel MRI data, Magn. Reson. Imaging, 32, 1058, 10.1016/j.mri.2014.03.006 · DOI
  12. Geurts, 2006, Extremely randomized trees, Mach. Learn., 63, 3, 10.1007/s10994-006-6226-1 · DOI · Paper page
  13. Ghosh, 2014, Automated detection of brain abnormalities in neonatal hypoxia ischemic injury from MR images, Med. Image Anal., 18, 1059, 10.1016/j.media.2014.05.002 · DOI
  14. Goldstein, 2009, Geometric applications of the split Bregman method: segmentation and surface reconstruction, J. Sci. Comput., 45, 272
  15. Išgum, 2015, Evaluation of automatic neonatal brain segmentation algorithms: the NeoBrainS12 challenge, Med. Image Anal., 20, 135, 10.1016/j.media.2014.11.001 · DOI
  16. Islam, 2013, Multifractal texture estimation for detection and segmentation of brain tumors, IEEE Trans. Biomed. Eng., 60, 3204, 10.1109/TBME.2013.2271383 · DOI · Paper page
  17. James, 2006, A supervised method for calculating perfusion/diffusion mismatch volume in acute ischemic stroke, Comput. Biol. Med., 36, 1268, 10.1016/j.compbiomed.2005.05.007 · DOI
  18. Jovicich, 2009, MRI-derived measurements of human subcortical, ventricular and intracranial brain volumes: reliability effects of scan sessions, acquisition sequences, data analyses, scanner upgrade, scanner vendors and field strengths, Neuroimage, 46, 177, 10.1016/j.neuroimage.2009.02.010 · DOI · Paper page
  19. Kemmling, 2015, Multivariate dynamic prediction of ischemic infarction and tissue salvage as a function of time and degree of recanalization, J. Cereb. Blood Flow Metab., 35, 1397, 10.1038/jcbfm.2015.144 · DOI
  20. Kistler, 2013, The virtual skeleton database: an open access repository for biomedical research and collaboration, J. Med. Internet Res., 15, e245, 10.2196/jmir.2930 · DOI · Paper page
  21. Konukoglu, 2013, Neighbourhood approximation using randomized forests, Med. Image Anal., 17, 790, 10.1016/j.media.2013.04.013 · DOI
  22. Langerak, 2010, Label fusion in atlas-based segmentation using a selective and iterative method for performance level estimation (SIMPLE), Med. Imag. IEEE Trans., 29, 2000, 10.1109/TMI.2010.2057442 · DOI · Paper page
  23. Lansberg, 2012, MRI profile and response to endovascular reperfusion after stroke (DEFUSE 2): a prospective cohort study, Lancet. Neurol., 11, 860, 10.1016/S1474-4422(12)70203-X · DOI · Paper page
  24. Li, 2004, Robust unsupervised segmentation of infarct lesion from diffusion tensor MR images using multiscale statistical classification and partial volume voxel reclassification, Neuroimage, 23, 1507, 10.1016/j.neuroimage.2004.08.009 · DOI
  25. Mah, 2014, A new method for automated high-dimensional lesion segmentation evaluated in vascular injury and applied to the human occipital lobe, Cortex, 56, 51, 10.1016/j.cortex.2012.12.008 · DOI
  26. Maier, O., Reyes, M., Menze, B., Handels, H. (Eds.), 2015. In: ISLES 2015: Ischemic Stroke Lesion Segmentation - Proceedings.
  27. Maier, 2015, Extra tree forests for sub-acute ischemic stroke lesion segmentation in MR sequences, J. Neurosci. Methods, 240, 89, 10.1016/j.jneumeth.2014.11.011 · DOI · Paper page
  28. Menze, 2015, The multimodal brain tumor image segmentation benchmark (BRATS), IEEE Trans. Med. Imag., 34, 1993, 10.1109/TMI.2014.2377694 · DOI · Paper page
  29. Muda, 2015, Brain lesion segmentation using fuzzy C-means on diffusion-weighted imaging, ARPN J. Eng. Appl. Sci., 10
  30. Murphy, 2011, Evaluation of registration methods on thoracic CT: the EMPIRE10 challenge, IEEE Trans. Med. Imag., 30, 1901, 10.1109/TMI.2011.2158349 · DOI · Paper page
  31. Nabizadeh, 2014, Histogram-based gravitational optimization algorithm on single MR modality for automatic brain lesion detection and segmentation, Expert Syst. Appl., 41, 7820, 10.1016/j.eswa.2014.06.043 · DOI
  32. Neumann, 2009, Interrater agreement for final infarct MRI lesion delineation, Stroke, 40, 3768, 10.1161/STROKEAHA.108.545368 · doi:10.1161/strokeaha.108.545368
  33. Olivot, 2009, Optimal Tmax threshold for predicting penumbral tissue in acute stroke, Stroke, 40, 469, 10.1161/STROKEAHA.108.526954 · DOI · Paper page
  34. Olivot, 2009, Perfusion MRI (Tmax and MTT) correlation with xenon CT cerebral blood flow in stroke patients, Neurology, 72, 1140, 10.1212/01.wnl.0000345372.49233.e3 · doi:10.1212/01.wnl.0000345372.49233.e3
  35. Petitjean, 2015, Right ventricle segmentation from cardiac MRI: a collation study, Med. Image Anal., 19, 187, 10.1016/j.media.2014.10.004 · DOI · Paper page
  36. Porz, 2014, Multi-modal glioblastoma segmentation: man versus machine, PLoS One, 9, e96873, 10.1371/journal.pone.0096873 · DOI
  37. Prakash, 2006, Identification, segmentation, and image property study of acute infarcts in diffusion-weighted images by using a probabilistic neural network and adaptive Gaussian mixture model, Acad. Radiol., 13, 1474, 10.1016/j.acra.2006.09.045 · doi:10.1016/j.acra.2006.09.045
  38. Rekik, 2012, Medical image analysis methods in MR/CT-imaged acute-subacute ischemic stroke lesion: segmentation, prediction and insights into dynamic evolution simulation models. a critical appraisal, NeuroImage Clin., 1, 164, 10.1016/j.nicl.2012.10.003 · DOI
  39. Rudyanto, 2014, Comparing algorithms for automated vessel segmentation in computed tomography scans of the lung: the VESSEL12 study, Med. Image Anal., 18, 1217, 10.1016/j.media.2014.07.003 · doi:10.1016/j.media.2014.07.003
  40. Salli, 2001, Contextual clustering for analysis of functional MRI data, IEEE Trans. Med. Imag., 20, 403, 10.1109/42.925293 · doi:10.1109/42.925293
  41. Seghier, 2008, Lesion identification using unified segmentation-normalisation models and fuzzy clustering, Neuroimage, 41, 1253, 10.1016/j.neuroimage.2008.03.028 · DOI · Paper page
  42. Shattuck, 2009, Online resource for validation of brain segmentation methods, Neuroimage, 45, 431, 10.1016/j.neuroimage.2008.10.066 · DOI · Paper page
  43. Soltanian-Zadeh, 2007, Multiparametric iterative self-organizing data analysis of ischemic lesions using pre- or post-Gd T1 MRI, Cerebrovasc. Dis., 23, 91, 10.1159/000097044 · doi:10.1159/000097044
  44. Straka, 2010, Real-time diffusion-perfusion mismatch analysis in acute stroke, J. Magn. Reson. Imag., 32, 1024, 10.1002/jmri.22338 · DOI · Paper page
  45. Styner, 2008, 3D segmentation in the clinic: a grand challenge II: MS lesion segmentation, Midas J, 10.54294/lmkqvm · DOI · Paper page
  46. Takasawa, 2008, How reliable is perfusion MR in acute stroke? Validation and determination of the penumbra threshold against quantitative PET, Stroke, 39, 870, 10.1161/STROKEAHA.107.500090 · doi:10.1161/strokeaha.107.500090
  47. Tobon-Gomez, 2013, Benchmarking framework for myocardial tracking and deformation algorithms: an open access database, Med. Image Anal., 17, 632, 10.1016/j.media.2013.03.008 · DOI · Paper page
  48. Tsai, 2014, Automatic detection and quantification of acute cerebral infarct by fuzzy clustering and histographic characterization on diffusion weighted MR imaging and apparent diffusion coefficient map, Biomed. Res. Int., 2014, 13, 10.1155/2014/963032 · doi:10.1155/2014/963032
  49. Wang, 2015, Fully automatic and robust 3D registration of serial-section microscopic images, Sci. Rep., 5, 15051, 10.1038/srep15051 · DOI
  50. Wheeler, 2013, Early diffusion-weighted imaging and perfusion-weighted imaging lesion volumes forecast final infarct size in DEFUSE 2, Stroke, 44, 681, 10.1161/STROKEAHA.111.000135 · DOI
  51. Woloszynski, 2011, A probabilistic model of classifier competence for dynamic ensemble selection, Pattern Recognit., 44, 2656, 10.1016/j.patcog.2011.03.020 · DOI · Paper page
  52. Xu, 1992, Methods of combining multiple classifiers and their applications to handwriting recognition, Syst. Man Cybern. IEEE Trans., 22, 418, 10.1109/21.155943 · DOI · Paper page

Cited by

See all 524

Metadata

Authors
Maier, O., Menze, B. H., von der Gablentz, J., Häni, L., Heinrich, M. P., Liebrand, M., Winzeck, S., Basit, A., Bentley, P., Chen, L., Christiaens, D., Dutil, F., Egger, K., Feng, C., Glocker, B., Götz, M., Haeck, T., Halme, H.-L., Havaei, M., Iftekharuddin, K. M., Jodoin, P.-M., Kamnitsas, K., Kellner, E., Korvenoja, A., Larochelle, H., Ledig, C., Lee, J.-H., Maes, F., Mahmood, Q., Maier-Hein, K. H., McKinley, R., Muschelli, J., Pal, C., Pei, L., Rangarajan, J. R., Reza, S. M. S., Robben, D., Rueckert, D., Salli, E., Suetens, P., Wang, C.-W., Wilms, M., Kirschke, J. S., Krämer, U. M., Münte, T. F., Schramm, P., Wiest, R., Handels, H., & Reyes, M.
Venue
Medical Image Analysis
Publisher
Elsevier BV
Published
2017-01
Volume
35
Pages
250-269
ISSN
1361-8415 (ISSN-L)
DOI
10.1016/j.media.2016.07.009
OpenAlex
W2484736472
Cited by
524
References
10
FWCI
33.77
Type
journal article
Language
en
Open access
green
Licence
www.elsevier.com/tdm/userlicense/1.0/ (tdm)
Funders
Northeastern University (20150310)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: ISLES 2015 - A public evaluation benchmark for ischemic stroke lesion segmentation from multispectral MRI. https://app.cochat.ai/papers/10.1016/j.media.2016.07.009
The paper
Maier, O., Menze, B. H., von der Gablentz, J., Häni, L., Heinrich, M. P., Liebrand, M., Winzeck, S., Basit, A., Bentley, P., Chen, L., Christiaens, D., Dutil, F., Egger, K., Feng, C., Glocker, B., Götz, M., Haeck, T., Halme, H.-L., Havaei, M., . . . Reyes, M. (2017). ISLES 2015 - A public evaluation benchmark for ischemic stroke lesion segmentation from multispectral MRI. Medical Image Analysis. https://doi.org/10.1016/j.media.2016.07.009

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.