The DOI matches its Crossref record; 212 of the 212 references it cites were found in a registry or an index.
A Review of Deep Learning in Medical Imaging: Imaging Traits, Technology Trends, Case Studies With Progress Highlights, and Future Promises
Open accessAbstract
Since its renaissance, deep learning has been widely used in various medical imaging tasks and has achieved remarkable success in many medical imaging applications, thereby propelling us into the so-called artificial intelligence (AI) era. It is known that the success of AI is mostly attributed to the availability of big data with annotations for a single task and the advances in high performance computing. However, medical imaging presents unique challenges that confront deep learning approaches. In this survey paper, we first present traits of medical imaging, highlight both clinical needs and technical challenges in medical imaging, and describe how emerging trends in deep learning are addressing these issues. We cover the topics of network architecture, sparse and noisy labels, federating learning, interpretability, uncertainty quantification, etc. Then, we present several case studies that are commonly found in clinical practice, including digital pathology and chest, brain, cardiovascular, and abdominal imaging. Rather than presenting an exhaustive literature survey, we instead describe some prominent research highlights related to these case study applications. We conclude with a discussion and presentation of promising future directions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Squeeze-and-Excitation Networks
Hu, J., Shen, L., & Sun, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00745 · Cited by 35,667
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., & Adam, H. (2017) · arXiv · doi:10.48550/arxiv.1704.04861 · Cited by 19,015
A survey on deep learning in medical image analysis
Litjens, G., Kooi, T., Bejnordi, B. E., Setio, A. A. A., Ciompi, F., Ghafoorian, M., van der Laak, J. A. W. M., van Ginneken, B., & Sánchez, C. I. (2017) · Medical Image Analysis · doi:10.1016/j.media.2017.07.005 · Cited by 15,332
Squeeze-and-Excitation Networks
Hu, J., Shen, L., Albanie, S., Sun, G., & Wu, E. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2913372 · Cited by 12,157
V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation
Milletari, F., Navab, N., & Ahmadi, S.-A. (2016) · 2016 Fourth International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2016.79 · Cited by 11,931 · cited 1×
“Specialized networks, such as U-Nets [38], V-Nets [120], and GANs [44], are also popular in neuroimaging and have been leveraged for a variety of segmentation and synthesis tasks.”
BackgroundFully convolutional networks for semantic segmentation
Shelhamer, E., Long, J., & Darrell, T. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2572683 · Cited by 11,898
Research works printed but not linked: 39
- campello, 2020, Multi-centre, multi-vendor & multi-disease cardiac image segmentation challenge (M&MS), Proc Med Image Comput Comput Assist Intervent (MICCAI)
- yang, 2019, Left ventricle full quantification challenge, Proc Int Conf Med Image Comput Comput -Assist Intervent
- simonyan, 2014, Very deep convolutional networks for large-scale image recognition, arXiv 1409 1556
- ronneberger, 2015, U-net: Convolutional networks for biomedical image segmentation, Proc Int Conf Med Image Comput Comput Assist Intervent (MICCAI), 234
- krizhevsky, 2012, ImageNet classification with deep convolutional neural networks, Proc Adv Neural Inf Process Syst, 1097
- 2013, MIT Tech Rev 2013
- 10.1109/JPROC.2019.2956422 · doi:10.1109/jproc.2019.2956422
- simpson, 2019, A large annotated medical image dataset for the development and evaluation of segmentation algorithms, arXiv 1902 09063
- 10.1016/j.media.2019.03.009 · DOI · Paper page
- zhou, 2017, Deep learning for medical image analysis
- yang, 2020, Smile-GANs: Semi-supervised clustering via GANs for dissecting brain disease heterogeneity from medical images, arXiv 2006 15255
- xu, 2017, 200x low-dose PET reconstruction using deep learning, arXiv 1712 04119
- tamada, 2018, Motion artifact reduction in abdominal MR imaging using the U-NET network, Proc ICMRM Sci Meeting KSMRM
- chen, 2019, Med3D: Transfer learning for 3D medical image analysis, arXiv 1904 00625
- raghu, 2019, Transfusion: Understanding transfer learning for medical imaging, Proc Adv Neural Inf Process Syst, 3347
- wolterink, 2017, Automatic segmentation and disease classification using cardiac cine MR images, Statistical Atlases and Computational Models of the Heart ACDC and MMWHS Challenges, 101
- isensee, 2017, Automatic cardiac disease assessment on cine-MRI via time-series segmentation and domain specific features, Statistical Atlases and Computational Models of the Heart ACDC and MMWHS Challenges, 120
- börgers, 2012, Computational Radiology and Imaging Therapy and Diagnostics, 110
- beutel, 2000, Handbook of Medical Imaging Medical Image Processing and Analysis, 2
- prince, 2006, Medical Imaging Signals and Systems
- goodfellow, 2014, Generative adversarial nets, Proc Adv Neural Inf Process Syst, 2672
- lee, 2015, Deeply-supervised nets, Proc Artif Intell Statist, 562
- huang, 2019, 3D U2-net: A 3D universal U-net for multi-domain medical image segmentation, Proc Int Conf Med Image Comput Comput Assist Intervent (MICCIAI), 291
- shi, 2020, Marginal loss and exclusion loss for partially supervised multi-organ segmentation, arXiv 2007 03868
- tang, 2020, E2Net: An edge enhanced network for accurate liver and tumor segmentation on CT scans, Proc Int Conf Med Image Comput Comput Assist Intervent (MICCAI), 512
- zhao, 2018, Federated learning with non-IID data, arXiv 1806 00582
- sheller, 2018, Multi-institutional deep learning modeling without sharing patient data: A feasibility study on brain tumor segmentation, Proc Int MICCAI Brainlesion Workshop, 92
- roth, 2015, CT lymph nodes, The Cancer Imaging Archive
- 10.1016/0090-4295(85)90280-8 · doi:10.1016/0090-4295(85)90280-8
- li, 2017, Non-rigid image registration using fully convolutional networks with deep self-supervision, arXiv 1709 00799
- yao, 2020, A hierarchical feature constraint to camouflage medical adversarial attacks, arXiv 2012 09501
- liu, 2014, Early diagnosis of Alzheimer’s disease with deep learning, Proc Int Symp Biomed Imag (ISBI), 1015
- lu, 2018, Multimodal and multiscale deep neural networks for the early diagnosis of Alzheimer’s disease using structural MR and FDG-PET images, Sci Rep, 8, 1
- beutel, 2000, Handbook of Medical Imaging, 1
- gozes, 2020, Rapid AI development cycle for the coronavirus (COVID-19) pandemic: Initial results for automated detection & patient monitoring using deep learning CT image analysis, arXiv 2003 05037
- 2019, Lung CT Screening Reporting & Data System v1 1
- bankman, 2008, Handbook of Medical Image Processing and Analysis
- zhou, 2019, Med Image Comput Comput Assist Interv
- 10.1007/978-3-030-59713-9_5 · doi:10.1007/978-3-030-59713-9_5
Cited by
Transformers in medical imaging: A survey
Shamshad, F., Khan, S., Zamir, S. W., Khan, M. H., Hayat, M., Khan, F. S., & Fu, H. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.102802 · Cited by 1,296
Edge artificial intelligence for 6g: vision, enabling technologies, and applications
Letaief, K. B., Shi, Y., Lu, J., & Lu, J. (2022) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3126076 · Cited by 829
“This is achieved by emerging advanced technologies, including DL [311], [312], Tactile Internet, IoT, edge AI, and wireless communications.”
MethodA Study of CNN and Transfer Learning in Medical Imaging: Advantages, Challenges, Future Scope
Salehi, A. W., Khan, S., Gupta, G., Alabduallah, B. I., Almjally, A., Alsolai, H., Siddiqui, T., & Mellit, A. (2023) · Sustainability · doi:10.3390/su15075930 · Cited by 570
AbdomenCT-1K: Is Abdominal Organ Segmentation a Solved Problem?
Ma, J., Zhang, Y., Gu, S., Zhu, C., Ge, C., Zhang, Y., An, X., Wang, C., Wang, Q., Liu, X., et al. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3100536 · Cited by 426
HiFuse: Hierarchical multi-scale feature fusion network for medical image classification
Huo, X., Sun, G., Tian, S., Wang, Y., Yu, L., Long, J., Zhang, W., & Li, A. (2024) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2023.105534 · Cited by 410
Li, J., Chen, J., Tang, Y., Wang, C., Landman, B. A., & Zhou, S. K. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.102762 · Cited by 389
Self-citationShallow Attention Network for Polyp Segmentation
Wei, J., Hu, Y., Zhang, R., Li, Z., Zhou, S. K., & Cui, S. (2021) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-87193-2_66 · Cited by 383
Self-citationSegment anything model for medical image segmentation: Current applications and future directions
Zhang, Y., Shen, Z., & Jiao, R. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.108238 · Cited by 332
Vision Transformers in medical computer vision—A contemplative retrospection
Parvaiz, A., Khalid, M. A., Zafar, R., Ameer, H., Ali, M., & Fraz, M. M. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.106126 · Cited by 276
Chakraborty, C., Bhattacharya, M., Pal, S., & Lee, S.-S. (2024) · Current Research in Biotechnology · doi:10.1016/j.crbiot.2023.100164 · Cited by 274
Moustafa, N., Koroniotis, N., Keshk, M., Zomaya, A. Y., & Tari, Z. (2023) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2023.3280465 · Cited by 266
Robust and data-efficient generalization of self-supervised machine learning for diagnostic imaging
Azizi, S., Culp, L., Freyberg, J., Mustafa, B., Baur, S., Kornblith, S., Chen, T., Tomasev, N., Mitrović, J., Strachan, P., et al. (2023) · Nature Biomedical Engineering · doi:10.1038/s41551-023-01049-7 · Cited by 250
Self-supervised learning methods and applications in medical imaging analysis: a survey
Shurrab, S., & Duwairi, R. (2022) · PeerJ Computer Science · doi:10.7717/peerj-cs.1045 · Cited by 254
Radiology report generation with a learned knowledge base and multi-modal alignment
Yang, S., Wu, X., Ge, S., Zheng, Z., Zhou, S. K., & Xiao, L. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.102798 · Cited by 199
Self-citationCombining Human Expertise with Artificial Intelligence: Experimental Evidence from Radiology
Agarwal, N., Moehring, A., Rajpurkar, P., & Salz, T. (2023) · doi:10.2139/ssrn.4505053 · Cited by 161
Combining Human Expertise with Artificial Intelligence: Experimental Evidence from Radiology
Agarwal, N., Moehring, A., Rajpurkar, P., & Salz, T. (2023) · doi:10.3386/w31422 · Cited by 161
A medical multimodal large language model for future pandemics
Liu, F., Zhu, T., Wu, X., Yang, B., You, C., Wang, C., Lu, L., Liu, Z., Zheng, Y., Sun, X., et al. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00952-2 · Cited by 155
Linardos, A., Kushibar, K., Walsh, S., Gkontra, P., & Lekadir, K. (2022) · Scientific Reports · doi:10.1038/s41598-022-07186-4 · Cited by 130
Sheu, R.-K., & Pardeshi, M. S. (2022) · Sensors · doi:10.3390/s22208068 · Cited by 129
Saad, M. B., Hong, L., Aminu, M., Vokes, N. I., Chen, P., Salehjahromi, M., Qin, K., Sujit, S. J., Lu, X., Young, E., et al. (2023) · The Lancet Digital Health · doi:10.1016/s2589-7500(23)00082-1 · Cited by 121
Generating synthetic data for medical imaging
Koetzier, L. R., Wu, J., Mastrodicasa, D., Lutz, A., Chung, M., Koszek, W. A., Pratap, J., Chaudhari, A. S., Rajpurkar, P., Lungren, M. P., et al. (2024) · Radiology · doi:10.1148/radiol.232471 · Cited by 111
Guha Roy, A., Ren, J., Azizi, S., Loh, A., Natarajan, V., Mustafa, B., Pawlowski, N., Freyberg, J., Liu, Y., Beaver, Z., et al. (2022) · Medical Image Analysis · doi:10.1016/j.media.2021.102274 · Cited by 77
A survey on deep learning in medical ultrasound imaging
Song, K., Feng, J., & Chen, D. (2024) · Frontiers in Physics · doi:10.3389/fphy.2024.1398393 · Cited by 59
Hsu, S.-T., Su, Y.-J., Hung, C.-H., Chen, M.-J., Lu, C.-H., & Kuo, C.-E. (2022) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-022-02047-6 · Cited by 58
Sendra-Balcells, C., Campello, V. M., Torrents-Barrena, J., Ahmed, Y. A., Elattar, M., Ohene-Botwe, B., Nyangulu, P., Stones, W., Ammar, M., Benamer, L. N., et al. (2023) · Scientific Reports · doi:10.1038/s41598-023-29490-3 · Cited by 57
Background selection schema on deep learning-based classification of dermatological disease
Zhou, J., Wu, Z., Jiang, Z., Huang, K., Guo, K., & Zhao, S. (2022) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2022.105966 · Cited by 53
Agrawal, A., Khatri, G. D., Khurana, B., Sodickson, A. D., Liang, Y., & Dreizin, D. (2023) · Emergency Radiology · doi:10.1007/s10140-023-02121-0 · Cited by 41
Large language model for medical images: a survey of taxonomy, systematic review, and future trends
Wang, P., Lu, W., Lu, C., Zhou, R., Li, M., & Qin, L. (2025) · Big Data Mining and Analytics · doi:10.26599/bdma.2024.9020090 · Cited by 21
Federated Learning for Multi-Center Imaging Diagnostics: A Study in Cardiovascular Disease
Linardos, A., Kushibar, K., Walsh, S., Gkontra, P., & Lekadir, K. (2021) · doi:10.21203/rs.3.rs-688924/v1 · Cited by 9
[Research progress on electronic health records multimodal data fusion based on deep learning].
(2024) · PubMed Central · doi:10.7507/1001-5515.202310011 · Cited by 1
Gao, W., You, S., Wu, Y., Su, J., Li, Z., Shi, B., & Jiang, X. (2026) · Placenta and Reproductive Medicine · doi:10.54844/prm.2026.1183
Metadata
- Authors
- Zhou, S. K., Greenspan, H., Davatzikos, C., Duncan, J. S., Van Ginneken, B., Madabhushi, A., Prince, J. L., Rueckert, D., & Summers, R. M.
- Venue
- Proceedings of the IEEE
- Published
- 2021-05
- Volume
- 109
- Issue
- 5
- Pages
- 820-838
- ISSN
- 0018-9219 (ISSN-L)
- 1558-2256
- OpenAlex
- W3074741277
- Cited by
- 1,118
- References
- 212
- FWCI
- 31.18
- Type
- journal article
- Language
- en
- Open access
- green
- Funders
- National Institutes of Health (1U24CA199374-01, R01CA202752-01A1, R01CA208236-01A1, R01CA216579-01A1, R01CA220581-01A1, 1U01CA239055-01, 1U54CA254566-01, 1U01CA248226-01, 1R43EB028736-01)
- VA Merit Review (IBX004121A)
- Biomedical Laboratory Research and Development Service of the United States Department of Veterans Affairs
- Israeli Science Foundation
- Ministry of Science & Technology
- National Institutes of Health Clinical Center
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.