Multi-Centre, Multi-Vendor and Multi-Disease Cardiac Segmentation: The M&Ms Challenge
Open accessAbstract
The emergence of deep learning has considerably advanced the state-of-the-art in cardiac magnetic resonance (CMR) segmentation. Many techniques have been proposed over the last few years, bringing the accuracy of automated segmentation close to human performance. However, these models have been all too often trained and validated using cardiac imaging samples from single clinical centres or homogeneous imaging protocols. This has prevented the development and validation of models that are generalizable across different clinical centres, imaging conditions or scanner vendors. To promote further research and scientific benchmarking in the field of generalizable deep learning for cardiac segmentation, this paper presents the results of the Multi-Centre, Multi-Vendor and Multi-Disease Cardiac Segmentation (M&Ms) Challenge, which was recently organized as part of the MICCAI 2020 Conference. A total of 14 teams submitted different solutions to the problem, combining various baseline models, data augmentation strategies, and domain adaptation techniques. The obtained results indicate the importance of intensity-driven data augmentation, as well as the need for further research to improve generalizability towards unseen scanner vendors or new imaging protocols. Furthermore, we present a new resource of 375 heterogeneous CMR datasets acquired by using four different scanner vendors in six hospitals and three different countries (Spain, Canada and Germany), which we provide as open-access for the community to enable future research in the field.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
M&Ms Challenge is a publicly available cardiac MRI dataset containing 375 participants from 6 hospitals in Spain, Canada, and Germany, acquired by 4 different scanners.
“Multi-Center, Multi-Vendor, and Multi-Disease Cardiac Segmentation (M&Ms) Challenge [112] 10 is a publicly available cardiac MRI dataset.”
From Federated learning for medical image analysis: A survey (2024) · cites this paper as [112]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation
Isensee, F., Jaeger, P. F., Kohl, S. A. A., Petersen, J., & Maier-Hein, K. H. (2021) · Nature Methods · doi:10.1038/s41592-020-01008-z · Cited by 10,477 · see citing works
Self-citationAttention U-Net: Learning Where to Look for the Pancreas
Oktay, O., Schlemper, J., Folgoc, L. L., Lee, M., Heinrich, M., Misawa, K., Mori, K., McDonagh, S., Hammerla, N. Y., Kainz, B., et al. (2018) · arXiv · doi:10.48550/arxiv.1804.03999 · Cited by 5,458 · see citing works
Arbitrary Style Transfer in Real-Time with Adaptive Instance Normalization
Huang, X., & Belongie, S. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.167 · Cited by 5,076 · see citing works
Self-citationBernard, O., Lalande, A., Zotti, C., Cervenansky, F., Yang, X., Heng, P.-A., Cetin, I., Lekadir, K., Camara, O., Gonzalez Ballester, M. A., et al. (2018) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2018.2837502 · Cited by 2,431 · see citing works
Self-citationDeep Learning for Cardiac Image Segmentation: A Review
Chen, C., Qin, C., Qiu, H., Tarroni, G., Duan, J., Bai, W., & Rueckert, D. (2020) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2020.00025 · Cited by 915 · see citing works
Automated cardiovascular magnetic resonance image analysis with fully convolutional networks
Bai, W., Sinclair, M., Tarroni, G., Oktay, O., Rajchl, M., Vaillant, G., Lee, A. M., Aung, N., Lukaschuk, E., Sanghvi, M. M., et al. (2018) · Journal of Cardiovascular Magnetic Resonance · doi:10.1186/s12968-018-0471-x · Cited by 773 · see citing works
Self-citationKhened, M., Kollerathu, V. A., & Krishnamurthi, G. (2018) · Medical Image Analysis · doi:10.1016/j.media.2018.10.004 · Cited by 376 · see citing works
A Fully Convolutional Neural Network for Cardiac Segmentation in Short-Axis MRI
Tran, P. V. (2016) · arXiv · doi:10.48550/arxiv.1604.00494 · Cited by 323 · see citing works
Isensee, F., Jaeger, P. F., Full, P. M., Ivo Wolf, A., Engelhardt, S., & Maier-Hein, K. H. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-75541-0_13 · Cited by 313 · see citing works
Self-citationImage-Based Cardiac Diagnosis With Machine Learning: A Review
Martín-Isla, C., Campello, V. M., Izquierdo, C., Raisi‐Estabragh, Z., Baeßler, B., Petersen, S. E., & Lekadir, K. (2020) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2020.00001 · Cited by 250 · see citing works
Self-citationRight ventricle segmentation from cardiac MRI: A collation study
Petitjean, C., Zuluaga, M. A., Bai, W., Dacher, J., Grosgeorge, D., Caudron, J., Ruan, S., Ayed, I. B., Cardoso, M. J., Chen, H.-C., et al. (2014) · Medical Image Analysis · doi:10.1016/j.media.2014.10.004 · Cited by 242 · see citing works
Tao, Q., Yan, W., Wang, Y., Paiman, E. H. M., Shamonin, D. P., Garg, P., Plein, S., Huang, L., Xia, L., Šramko, M., et al. (2018) · Radiology · doi:10.1148/radiol.2018180513 · Cited by 209 · see citing works
Suinesiaputra, A., Cowan, B. R., Al‐Agamy, A. O., Elattar, M., Ayache, N., Fahmy, A. S., Khalifa, A. M., Medrano−Gracia, P., Jolly, M., Kadish, A. H., et al. (2013) · Medical Image Analysis · doi:10.1016/j.media.2013.09.001 · Cited by 206 · see citing works
Self-citationDisentangled representation learning in cardiac image analysis
Chartsias, A., Joyce, T., Papanastasiou, G., Semple, S. I. K., Williams, M. C., Newby, D. E., Dharmakumar, R., & Tsaftaris, S. A. (2019) · Medical Image Analysis · doi:10.1016/j.media.2019.101535 · Cited by 195 · see citing works
Self-citationMulti-atlas segmentation with augmented features for cardiac MR images
Bai, W., Shi, W., Ledig, C., & Rueckert, D. (2014) · Medical Image Analysis · doi:10.1016/j.media.2014.09.005 · Cited by 159 · see citing works
Improving the Generalizability of Convolutional Neural Network-Based Segmentation on CMR Images
Chen, C., Bai, W., Davies, R. H., Bhuva, A., Manisty, C., Augusto, J. B., Moon, J., Aung, N., Lee, A. M., Sanghvi, M. M., et al. (2020) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2020.00105 · Cited by 126 · see citing works
Self-citationAutomatic initialization and quality control of large-scale cardiac MRI segmentations
Albà, X., Lekadir, K., Pereañez, M., Medrano−Gracia, P., Young, A. A., & Frangi, A. F. (2017) · Medical Image Analysis · doi:10.1016/j.media.2017.10.001 · Cited by 69 · see citing works
Self-citationRadiomics Signatures of Cardiovascular Risk Factors in Cardiac MRI: Results From the UK Biobank
Cetin, I., Raisi‐Estabragh, Z., Petersen, S. E., Napel, S. A., Piechnik, S. K., Neubauer, S., Ballester, M. Á. G., Cámara, Ó., & Lekadir, K. (2020) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2020.591368 · Cited by 61 · see citing works
Self-citationStudying Robustness of Semantic Segmentation Under Domain Shift in Cardiac MRI
Full, P. M., Isensee, F., Jäger, P. F., & Maier-Hein, K. H. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-68107-4_24 · Cited by 59 · see citing works
Self-citationA Radiomics Approach to Computer-Aided Diagnosis with Cardiac Cine-MRI
Cetin, I., Sanroma, G., Petersen, S. E., Napel, S. A., Cámara, Ó., Ballester, M.-A. G., & Lekadir, K. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-75541-0_9 · Cited by 55 · see citing works
Self-citationMa, J. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-68107-4_18 · Cited by 40 · see citing works
Self-citationRemove Appearance Shift for Ultrasound Image Segmentation via Fast and Universal Style Transfer
Liu, Z., Yang, X., Gao, R., Liu, S., Dou, H., He, S., Huang, Y., Huang, Y., Luo, H., Zhang, Y., et al. (2020) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi45749.2020.9098457 · Cited by 34 · see citing works
Self-citationSemi-supervised Cardiac Image Segmentation via Label Propagation and Style Transfer
Zhang, Y., Yang, J., Hou, F., Liu, Y., Wang, Y., Tian, J., Zhang, Y., Zhang, Y., & He, Z. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-68107-4_22 · Cited by 31 · see citing works
Self-citationScannell, C. M., Chiribiri, A., & Veta, M. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-68107-4_23 · Cited by 30 · see citing works
Self-citationDisentangled Representations for Domain-Generalized Cardiac Segmentation
Liu, X., Thermos, S., Chartsias, A., et al. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-68107-4_19 · Cited by 26 · see citing works
Self-citationRandom Style Transfer Based Domain Generalization Networks Integrating Shape and Spatial Information
Li, L., Zimmer, V. A., Ding, W., et al. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-68107-4_21 · Cited by 25 · see citing works
Self-citationKong, F., & Shadden, S. C. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-68107-4_29 · Cited by 22 · see citing works
Self-citationAcero, J. C., Sundaresan, V., Dinsdale, N. K., et al. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-68107-4_20 · Cited by 18 · see citing works
Self-citationStyle-Invariant Cardiac Image Segmentation with Test-Time Augmentation
Huang, X., Chen, Z., Yang, X., et al. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-68107-4_31 · Cited by 16 · see citing works
Self-citationGeneralisable Cardiac Structure Segmentation via Attentional and Stacked Image Adaptation
Li, H., Zhang, J., & Menze, B. H. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-68107-4_30 · Cited by 14 · see citing works
Self-citationDeidentifying MRI Data Domain by Iterative Backpropagation
Parreño, M., Paredes, R., & Albiol, A. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-68107-4_28 · Cited by 14 · see citing works
Self-citationCarscadden, A., Noga, M., & Punithakumar, K. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-68107-4_25 · Cited by 14 · see citing works
Self-citationAdaptive Preprocessing for Generalization in Cardiac MR Image Segmentation
Khader, F., Schock, J., Truhn, D., et al. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-68107-4_27 · Cited by 11 · see citing works
Self-citationSaber, M., Abdelrauof, D., & Elattar, M. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-68107-4_26 · Cited by 10 · see citing works
Self-citationAutomatic Brain Structures Segmentation Using Deep Residual Dilated U-NetPreprint
Li, H., Zhygallo, A., & Menze, B. H. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.04312 · Cited by 2 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 3
- P. Radau, Y. Lu, K. Connelly, G. Paul, A. J. W. G. Dick, and G. Wright, "Evaluation framework for algorithms segmenting short axis cardiac MRI," MIDAS J.-Cardiac MR Left Ventricle Segmentation Challenge, 2009. [Online]. Available: http://hdl.handle.net/10380/3070
- O. Ronneberger, P. Fischer, and T. Brox, "U-Net: Convolutional net- works for biomedical image segmentation," in Proc. Int. Conf. Med. Image Comput. Comput.-Assist. Intervent., in Lecture Notes in Computer Science. Cham, Switzerland: Springer, 2015, pp. 234-241.
- D. Shanmugam, D. Blalock, G. Balakrishnan, and J. Guttag, "When and why test-time augmentation works," 2020, arXiv:2011.11156. [Online]. Available: http://arxiv.org/abs/2011.11156 · arXiv
Cited by
Sorted by citations
Lekadir, K., Frangi, A. F., Porras, A. R., Glocker, B., Cintas, C., Langlotz, C. P., Weicken, E., Asselbergs, F. W., Prior, F., Collins, G. S., et al. (2025) · BMJ · doi:10.1136/bmj-2024-081554 · Cited by 957 · see citing works
Self-citationSalahuddin, Z., Woodruff, H. C., Chatterjee, A., & Lambin, P. (2022) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2021.105111 · Cited by 508 · see citing works
Segment anything model for medical images?
Huang, Y., Yang, X., Liu, L., Zhou, H., Chang, A., Zhou, X., Chen, R., Yu, J., Chen, J., Chen, C., et al. (2024) · Medical Image Analysis · doi:10.1016/j.media.2023.103061 · Cited by 536 · see citing works
“Colonoscopy Kvasir-SEG [58] Gastrointestinal polyp colonoscopy LiVScar [59] Infarct segmentation in the left ventricle CMR LUNA16 [60] Lungs, heart and trachea CT M&Ms [61] Left and right ventricle and left ventricular myocardium CMR MALBCV-Abdomen [62] Abdominal multi-organ segmentation CT Montgomery County CXR Set […”
Self-citationFederated learning for medical image analysis: A survey
Guan, H., Yap, P.-T., Bozoki, A., & Liu, M. (2024) · Pattern Recognition · doi:10.1016/j.patcog.2024.110424 · Cited by 518 · see citing works
Linardos, A., Kushibar, K., Walsh, S., Gkontra, P., & Lekadir, K. (2022) · Scientific Reports · doi:10.1038/s41598-022-07186-4 · Cited by 130 · see citing works
Self-citationA systematic review on federated learning in medical image analysis
Sohan, M. F., & Basalamah, A. (2023) · IEEE Access · doi:10.1109/access.2023.3260027 · Cited by 114 · see citing works
Reviewing Federated Machine Learning and Its Use in Diseases Prediction
Moshawrab, M., Adda, M., Bouzouane, A., Ibrahim, H., & Raad, A. (2023) · Sensors · doi:10.3390/s23042112 · Cited by 114 · see citing works
Dynamic Memory to Alleviate Catastrophic Forgetting in Continual Learning with Medical Imaging
Perkonigg, M., Hofmanninger, J., Herold, C. J., Brink, J. A., Pianykh, O., Prosch, H., & Langs, G. (2021) · Nature Communications · doi:10.1038/s41467-021-25858-z · Cited by 106 · see citing works
Minimising multi-centre radiomics variability through image normalisation: a pilot study
Campello, V. M., Martín-Isla, C., Izquierdo, C., Guala, A., Palomares, J. F. R., Viladés, D., Descalzo, M. L., Karakas, M., Çavuş, E., Raisi-Estabragh, Z., et al. (2022) · Scientific Reports · doi:10.1038/s41598-022-16375-0 · Cited by 24 · see citing works
Self-citation
- Authors
- Campello, V. M., Gkontra, P., Izquierdo, C., Martin-Isla, C., Sojoudi, A., Full, P. M., Maier-Hein, K., Zhang, Y., He, Z., Ma, J., Parreno, M., Albiol, A., Kong, F., Shadden, S. C., Acero, J. C., Sundaresan, V., Saber, M., Elattar, M., Li, H., Menze, B., Khader, F., Haarburger, C., Scannell, C. M., Veta, M., Carscadden, A., Punithakumar, K., Liu, X., Tsaftaris, S. A., Huang, X., Yang, X., Li, L., Zhuang, X., Vilades, D., Descalzo, M. L., Guala, A., Mura, L. L., Friedrich, M. G., Garg, R., Lebel, J., Henriques, F., Karakas, M., Cavus, E., Petersen, S. E., Escalera, S., Segui, S., Rodriguez-Palomares, J. F., & Lekadir, K.
- Affiliations
- Universitat de Barcelona (ES)
- Circle Cardiovascular Imaging (CA)
- German Cancer Research Center (DE)
- Chinese Academy of Sciences (CN)
- Lenovo (China) (CN)
- Nanjing University of Science and Technology (CN)
- Universitat Politècnica de València (ES)
- University of California, Berkeley (US)
- University of Oxford (GB)
- Nile University (EG)
- Technical University of Munich (DE)
- King's College London (GB)
- Eindhoven University of Technology (NL)
- University of Alberta (CA)
- University of Edinburgh (GB)
- The Alan Turing Institute (GB)
- Shenzhen University (CN)
- Shanghai Jiao Tong University (CN)
- Fudan University (CN)
- Universitat Autònoma de Barcelona (ES)
- University of Naples Federico II (IT)
- McGill University (CA)
- German Centre for Cardiovascular Research (DE)
- Queen Mary University of London (GB)
- ORCID
- Campello, Victor M.
- Gkontra, Polyxeni
- Full, Peter M.
- Zhang, Yao
- Ma, Jun
- Parreno, Mario
- Albiol, Alberto
- Kong, Fanwei
- Shadden, Shawn C.
- Acero, Jorge Corral
- Sundaresan, Vaanathi
- Elattar, Mustafa
- Li, Hongwei
- Menze, Bjoern
- Scannell, Cian M.
- Veta, Mitko
- Punithakumar, Kumaradevan
- Tsaftaris, Sotirios A.
- Yang, Xin
- Zhuang, Xiahai
- Vilades, David
- Descalzo, Martin L.
- Guala, Andrea
- Mura, Lucia La
- Friedrich, Matthias G.
- Garg, Ria
- Petersen, Steffen E.
- Escalera, Sergio
- Segui, Santi
- Rodriguez-Palomares, Jose F.
- Lekadir, Karim
- Venue
- IEEE Transactions on Medical Imaging
- Published
- 2021-12
- Volume
- 40
- Issue
- 12
- Pages
- 3543-3554
- ISSN
- 0278-0062 (ISSN-L)
- 1558-254X
- OpenAlex
- W3172614365
- Cited by
- 481
- References
- 35
- Citations by year
- 2026: 73 · 2025: 128 · 2024: 102 · 2023: 84 · 2022: 79 · 2021: 15
- FWCI
- 25.18
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- IEEE Transactions on Medical Imaging (publishedVersion, cc-by)
- PubMed (publishedVersion)
- TU/e Research Portal (publishedVersion, cc-by)
- Dipòsit Digital de Documents de la UAB (Universitat Autònoma de Barcelona) (submittedVersion, cc-by)
- ePrints Soton (University of Southampton) (submittedVersion)
- Research Portal (King's College London) (submittedVersion, cc-by)
- Zurich Open Repository and Archive (University of Zurich) (submittedVersion, cc-by)
- TU/e Research Portal (submittedVersion, cc-by)
- Funders
- European Union’s Horizon 2020 Research and Innovation Program (825903)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Cardiac Imaging and Diagnostics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- domain adaptation, benchmarking
- MeSH terms
- Heart, Humans, Magnetic Resonance Imaging, Cardiac Imaging Techniques
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
