MedMNIST Classification Decathlon: A Lightweight AutoML Benchmark for Medical Image Analysis
Open accessAbstract
We present MedMNIST, a collection of 10 pre-processed medical open datasets. MedMNIST is standardized to perform classification tasks on lightweight 28 x 28 images, which requires no background knowledge. Covering the primary data modalities in medical image analysis, it is diverse on data scale (from 100 to 100,000) and tasks (binary/multi-class, ordinal regression and multi-label). MedMNIST could be used for educational purpose, rapid prototyping, multi-modal machine learning or AutoML in medical image analysis. Moreover, MedMNIST Classification Decathlon is designed to benchmark AutoML algorithms on all 10 datasets; We have compared several baseline methods, including open-source or commercial AutoML tools. The datasets, evaluation code and baseline methods for MedMNIST are publicly available at https://medmnist.github.io/.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
MedMNIST v2 is extended from MedMNIST v1, with 10 2D datasets for medical image classification.
“MedMNIST v2 is extended from our preliminary version, MedMNIST v19, with 10 2D datasets for medical image classification.”
From MedMNIST v2 - A large-scale lightweight benchmark for 2D and 3D biomedical image classification (2023) · cites this paper as [9]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
Identifying Medical Diagnoses and Treatable Diseases by Image-Based Deep Learning
Kermany, D. S., Goldbaum, M., Cai, W., Valentim, C. C. S., Liang, H., Baxter, S. L., McKeown, A., Yang, G., Wu, X., Yan, F., et al. (2018) · Cell · doi:10.1016/j.cell.2018.02.010 · Cited by 4,842 · see citing works
The Liver Tumor Segmentation Benchmark (LiTS)
Bilic, P., Christ, P., Li, H. B., Vorontsov, E., Ben-Cohen, A., Kaissis, G., Szeskin, A., Jacobs, C., Mamani, G. E. H., Chartrand, G., et al. (2023) · Medical Image Analysis · doi:10.1016/j.media.2022.102680 · Cited by 1,412 · see citing works
Kather, J. N., Krisam, J., Charoentong, P., Luedde, T., Herpel, E., Weis, C.-A., Gaiser, T., Marx, A., Valous, N. A., Ferber, D., et al. (2019) · PLOS Medicine · doi:10.1371/journal.pmed.1002730 · Cited by 1,082 · see citing works
Flagged first, then by citations
Research works printed but not linked: 13
- [1][1] Geert Litjens, Thijs Kooi, et al., 'A survey on deep learning in medical image analysis,' Medical image analysis , vol. 42, pp. 60-88, 2017. -
- [2][2] Dinggang Shen, Guorong Wu, and Heung-Il Suk, 'Deep learning in medical image analysis,' Annual review of biomedical engineering , vol. 19, pp. 221-248, 2017. -
- [3][3] Livia Faes, Siegfried K Wagner, et al., 'Automated deep learning design for medical image classification by health-care professionals with no coding experience: a feasibility study,' The Lancet Digital Health , vol. 1, no. 5, pp. e232-e242, 2019. -
- [4][4] Yann LeCun, Corinna Cortes, and CJ Burges, 'Mnist handwritten digit database,' ATT Labs [Online]. Available: http://yann.lecun.com/exdb/mnist , vol. 2, 2010. -
- [5][5] Amber L Simpson, Michela Antonelli, et al., 'A large annotated medical image dataset for the development and evaluation of segmentation algorithms,' arXiv preprint arXiv:1902.09063 , 2019. - · arXiv
- [7][7] Matthias Feurer, Aaron Klein, et al., 'Efficient and robust automated machine learning,' in Advances in Neural Information Processing Systems 28 , C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett, Eds., pp. 2962-2970. Curran Associates, Inc., 2015. -
- [8][8] Haifeng Jin, Qingquan Song, and Xia Hu, 'Auto-keras: An efficient neural architecture search system,' in Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining . ACM, 2019, pp. 1946-1956. -
- [10][10] Xiaosong Wang, Yifan Peng, et al., 'Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases,' in CVPR , 2017, pp. 34623471. -
- [11][11] Philipp Tschandl, Cliff Rosendahl, and Harald Kittler, 'The ham10000 dataset, a large collection of multisource dermatoscopic images of common pigmented skin lesions,' Scientific data , vol. 5, pp. 180161, 2018. -
- [12][12] Noel Codella, Veronica Rotemberg, Philipp Tschandl, M Emre Celebi, Stephen Dusza, David Gutman, Brian Helba, Aadi Kalloo, Konstantinos Liopyris, Michael Marchetti, et al., 'Skin lesion analysis toward melanoma detection 2018: A challenge hosted by the international skin imaging collaboration (isic),' arXiv preprint arXiv:1902.03368 , 2019. - · arXiv
- [15][15] Walid Al-Dhabyani, Mohammed Gomaa, Hussien Khaled, and Aly Fahmy, 'Dataset of breast ultrasound images,' Data in Brief , vol. 28, pp. 104863, 2020. -
- [17][17] X. Xu, F. Zhou, et al., 'Efficient multiple organ localization in ct image using 3d region proposal network,' IEEE Transactions on Medical Imaging , vol. 38, no. 8, pp. 1885-1898, 2019. -
- [18][18] Fabian Pedregosa, Ga¨ el Varoquaux, et al., 'Scikit-learn: Machine learning in python,' the Journal of machine Learning research , vol. 12, pp. 2825-2830, 2011. -
Cited by
Sorted by citations
MedMNIST v2 - A large-scale lightweight benchmark for 2D and 3D biomedical image classification
Yang, J., Shi, R., Wei, D., Liu, Z., Zhao, L., Ke, B., Pfister, H., & Ni, B. (2023) · Scientific Data · doi:10.1038/s41597-022-01721-8 · Cited by 1,043 · see citing works
“16, 17 Colon Pathology MC (9) 107,180 89,996 / 10,004 / 7,180 ChestMNIST Wang et al.”
Self-citationBorys, K., Schmitt, Y. A., Nauta, M., Seifert, C., Krämer, N., Friedrich, C. M., & Nensa, F. (2023) · European Journal of Radiology · doi:10.1016/j.ejrad.2023.110786 · Cited by 233 · see citing works
Federated learning for healthcare applications
Chaddad, A., Wu, Y., & Desrosiers, C. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3325822 · Cited by 136 · see citing works
Kalapaaking, A. P., Khalil, I., & Yi, X. (2024) · IEEE Transactions on Emerging Topics in Computing · doi:10.1109/tetc.2023.3268186 · Cited by 122 · see citing works
PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answering
Zhang, X., Wu, C., Zhao, Z., Lin, W., Zhang, Y., Wang, Y., & Xie, W. (2023) · arXiv · doi:10.48550/arxiv.2305.10415 · Cited by 66 · see citing works
“Same with MedVInT-TE, it also encodes the question input into embedding features q and can perform multi-choice or blank through different prompts.”
MethodDi Salvo, F., Doerrich, S., & Ledig, C. (2024) · arXiv · doi:10.48550/arxiv.2406.17536 · Cited by 3 · see citing works
“We first introduce MedMNIST-C , a comprehensive image corruption benchmark for 12 datasets and 9 imaging modalities, leveraging the publicly available MedMNIST+ [29] dataset collection (resolution: 224 × 224 ).”
Method
- Authors
- Yang, J., Shi, R., & Ni, B.
- Affiliations
- Shanghai Jiao Tong University (CN)
- Venue
- 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI)
- Publisher
- IEEE
- Published
- 2021-04-13
- Pages
- 191-195
- OpenAlex
- W3095948197
- Cited by
- 414
- References
- 4
- Citations by year
- 2026: 62 · 2025: 101 · 2024: 96 · 2023: 81 · 2022: 50 · 2021: 24
- FWCI
- 28.99
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE International Symposium on Biomedical Imaging (ISBI) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Funders
- National Science Foundation
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- MedMNIST, medical image classification, ordinal regression, multi-label classification, multimodal learning, medical image analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
