Novel Transfer Learning Approach for Medical Imaging with Limited Labeled Data
Open accessThe DOI matches its Crossref record; 62 of the 62 references it cites were found in a registry or an index.
Abstract
Deep learning requires a large amount of data to perform well. However, the field of medical image analysis suffers from a lack of sufficient data for training deep learning models. Moreover, medical images require manual labeling, usually provided by human annotators coming from various backgrounds. More importantly, the annotation process is time-consuming, expensive, and prone to errors. Transfer learning was introduced to reduce the need for the annotation process by transferring the deep learning models with knowledge from a previous task and then by fine-tuning them on a relatively small dataset of the current task. Most of the methods of medical image classification employ transfer learning from pretrained models, e.g., ImageNet, which has been proven to be ineffective. This is due to the mismatch in learned features between the natural image, e.g., ImageNet, and medical images. Additionally, it results in the utilization of deeply elaborated models. In this paper, we propose a novel transfer learning approach to overcome the previous drawbacks by means of training the deep learning model on large unlabeled medical image datasets and by next transferring the knowledge to train the deep learning model on the small amount of labeled medical images. Additionally, we propose a new deep convolutional neural network (DCNN) model that combines recent advancements in the field. We conducted several experiments on two challenging medical imaging scenarios dealing with skin and breast cancer classification tasks. According to the reported results, it has been empirically proven that the proposed approach can significantly improve the performance of both classification scenarios. In terms of skin cancer, the proposed model achieved an F1-score value of 89.09% when trained from scratch and 98.53% with the proposed approach. Secondly, it achieved an accuracy value of 85.29% and 97.51%, respectively, when trained from scratch and using the proposed approach in the case of the breast cancer scenario. Finally, we concluded that our method can possibly be applied to many medical imaging problems in which a substantial amount of unlabeled image data is available and the labeled image data is limited. Moreover, it can be utilized to improve the performance of medical imaging tasks in the same domain. To do so, we used the pretrained skin cancer model to train on feet skin to classify them into two classes—either normal or abnormal (diabetic foot ulcer (DFU)). It achieved an F1-score value of 86.0% when trained from scratch, 96.25% using transfer learning, and 99.25% using double-transfer learning.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
1 retracted
Monisha, M. L., Suresh, A., Bapu, B. R. T., & Rashmi, M. R. (2018) · Cluster Computing · doi:10.1007/s10586-018-1798-7 · Cited by 27
RetractedSung, H., Ferlay, J., Siegel, R. L., Laversanne, M., Soerjomataram, I., Jemal, A., & Bray, F. (2021) · CA: A Cancer Journal for Clinicians · doi:10.3322/caac.21660 · Cited by 108,699
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Bray, F., Ferlay, J., Soerjomataram, I., Siegel, R. L., Torre, L. A., & Jemal, A. (2018) · CA: A Cancer Journal for Clinicians · doi:10.3322/caac.21492 · Cited by 84,109
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
Dermatologist-level classification of skin cancer with deep neural networks
Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017) · Nature · doi:10.1038/nature21056 · Cited by 14,362
Gulshan, V., Peng, L., Coram, M., Stumpe, M. C., Wu, D., Narayanaswamy, A., Venugopalan, S., Widner, K., Madams, T., Cuadros, J., et al. (2016) · JAMA · doi:10.1001/jama.2016.17216 · Cited by 7,697
Tschandl, P., Rosendahl, C., & Kittler, H. (2018) · Scientific Data · doi:10.1038/sdata.2018.161 · Cited by 3,628
A survey of the recent architectures of deep convolutional neural networks
Khan, A., Sohail, A., Zahoora, U., & Qureshi, A. S. (2020) · Artificial Intelligence Review · doi:10.1007/s10462-020-09825-6 · Cited by 3,137
Codella, N. C. F., Gutman, D., Celebi, M. E., Helba, B., Marchetti, M. A., Dusza, S. W., Kalloo, A., Liopyris, K., Mishra, N., Kittler, H., et al. (2018) · 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018) · doi:10.1109/isbi.2018.8363547 · Cited by 2,620
Deep learning applications and challenges in big data analytics
Najafabadi, M. M., Villanustre, F., Khoshgoftaar, T. M., Seliya, N., Wald, R., & Muharemagic, E. (2015) · Journal of Big Data · doi:10.1186/s40537-014-0007-7 · Cited by 2,594
Review of Deep Learning Algorithms and Architectures
Shrestha, A., & Mahmood, A. (2019) · IEEE Access · doi:10.1109/access.2019.2912200 · Cited by 1,883 · cited 3×
“Furthermore, these layers efficiently decrease the network sensitivity to image distortions, scales, and shifts along with shared weights and local receptive fields (inside the same filter) [38].”
BackgroundMethodA Dataset for Breast Cancer Histopathological Image Classification
Spanhol, F. A., Oliveira, L. S., Petitjean, C., & Heutte, L. (2016) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2015.2496264 · Cited by 1,832
A State-of-the-Art Survey on Deep Learning Theory and Architectures
Alom, M. Z., Taha, T. M., Yakopcic, C., Westberg, S., Sidike, P., Nasrin, M. S., Hasan, M., Van Essen, B. C., Awwal, A. A. S., & Asari, V. K. (2019) · Electronics · doi:10.3390/electronics8030292 · Cited by 1,620
CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning
Rajpurkar, P., Irvin, J., Zhu, K., Yang, B., Mehta, H., Duan, T., Ding, D. Y., Bagul, A., Langlotz, C. P., Shpanskaya, K. S., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.05225 · Cited by 1,219
Classification of breast cancer histology images using Convolutional Neural Networks
Araújo, T., Aresta, G., Castro, E., Rouco, J., Aguiar, P., Eloy, C., Polónia, A., & Campilho, A. (2017) · PLOS ONE · doi:10.1371/journal.pone.0177544 · Cited by 1,013
Stacked Sparse Autoencoder (SSAE) for Nuclei Detection on Breast Cancer Histopathology Images
Xu, J., Xiang, L., Liu, Q., Gilmore, H., Wu, J., Tang, J., & Madabhushi, A. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2015.2458702 · Cited by 854 · cited 1×
“The third source is the breast cancer dataset, which is composed of 537 H&E-stained histopathological images with a size of 2200 × 2200 pixels [54].”
BackgroundA deep learning system for differential diagnosis of skin diseases
Liu, Y., Jain, A., Eng, C., Way, D. H., Lee, K., Bui, P., Kanada, K., de Oliveira Marinho, G., Gallegos, J., Gabriele, S., et al. (2020) · Nature Medicine · doi:10.1038/s41591-020-0842-3 · Cited by 853
Deep learning for computational chemistry
Goh, G. B., Hodas, N. O., & Vishnu, A. (2017) · Journal of Computational Chemistry · doi:10.1002/jcc.24764 · Cited by 809
Khan, S., Islam, N., Jan, Z., Ud Din, I., & Rodrigues, J. J. P. C. (2019) · Pattern Recognition Letters · doi:10.1016/j.patrec.2019.03.022 · Cited by 785
BACH: Grand challenge on breast cancer histology images
Aresta, G., Araújo, T., Kwok, S., Chennamsetty, S. S., Safwan, M., Alex, V., Marami, B., Prastawa, M., Chan, M., Donovan, M., et al. (2019) · Medical Image Analysis · doi:10.1016/j.media.2019.05.010 · Cited by 703
A methodological approach to the classification of dermoscopy images
Celebi, M. E., Kingravi, H. A., Uddin, B., Iyatomi, H., Aslandogan, Y. A., Stoecker, W. V., & Moss, R. H. (2007) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2007.01.003 · Cited by 693
Automatic skin lesion segmentation using deep fully convolutional networks with jaccard distance.
Yuan, Y., Chao, M., & Lo, Y.-C. (2017) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2017.2695227 · Cited by 687
Convolutional Learning of Spatio-temporal Features
Taylor, G. W., Fergus, R., LeCun, Y., & Bregler, C. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15567-3_11 · Cited by 629
Transfusion: Understanding Transfer Learning for Medical Imaging
Raghu, M., Zhang, C., Kleinberg, J. M., & Bengio, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.07208 · Cited by 613
Cancer Diagnosis Using Deep Learning: A Bibliographic Review
Munir, K., Elahi, H., Ayub, A., Frezza, F., & Rizzi, A. (2019) · Cancers · doi:10.3390/cancers11091235 · Cited by 422
CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison
Irvin, J., Rajpurkar, P., Ko, M., Yu, Y., Ciurea-Ilcus, S., Chute, C., Marklund, H., Haghgoo, B., Ball, R., Shpanskaya, K., et al. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.3301590 · Cited by 392
MED-NODE: A computer-assisted melanoma diagnosis system using non-dermoscopic images
Giotis, I., Molders, N., Land, S., Biehl, M., Jonkman, M. F., & Petkov, N. (2015) · Expert Systems with Applications · doi:10.1016/j.eswa.2015.04.034 · Cited by 364
Deep Learning applications for COVID-19
Shorten, C., Khoshgoftaar, T. M., & Furht, B. (2021) · Journal of Big Data · doi:10.1186/s40537-020-00392-9 · Cited by 350 · cited 1×
“It is still in continuous inspirational development regarding the novel performance of several machine learning tasks [31–34], and it has simplified the improvement of many learning fields [35,36].”
BackgroundMelanoma Classification on Dermoscopy Images Using a Neural Network Ensemble Model
Xie, F., Fan, H., Li, Y., Jiang, Z., Meng, R., & Bovik, A. (2017) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2633551 · Cited by 327
Med3D: Transfer Learning for 3D Medical Image Analysis
Chen, S., Ma, K., & Zheng, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.00625 · Cited by 317
BCN20000: Dermoscopic Lesions in the Wild
Combalia, M., Codella, N., Rotemberg, V. M., Helba, B., Vilaplana, V., Reiter, O., Carrera, C., Barreiro, A., Halpern, A. C., Puig, S., et al. (2019) · LA Referencia (Red Federada de Repositorios Institucionales de Publicaciones Científicas) · doi:10.48550/arxiv.1908.02288 · Cited by 284
Ballerini, L., Fisher, R. B., Aldridge, B., & Rees, J. (2013) · Lecture Notes in Computational Vision and Biomechanics · doi:10.1007/978-94-007-5389-1_4 · Cited by 273
DFUNet: Convolutional Neural Networks for Diabetic Foot Ulcer Classification
Goyal, M., Reeves, N. D., Davison, A. K., Rajbhandari, S., Spragg, J., & Yap, M. H. (2020) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2018.2866254 · Cited by 273
Goebel, R., Chander, A., Holzinger, K., Lecue, F., Akata, Z., Stumpf, S., Kieseberg, P., & Holzinger, A. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-99740-7_21 · Cited by 259
Towards a Better Understanding of Transfer Learning for Medical Imaging: A Case Study
Alzubaidi, L., Fadhel, M. A., Al-Shamma, O., Zhang, J., Santamaría, J., Duan, Y., & R. Oleiwi, S. (2020) · Applied Sciences · doi:10.3390/app10134523 · Cited by 258 · cited 1×
“The authors empirically showed that in-domain TL can improve the performance against the pretrained ImageNet models [12–14].”
BackgroundSelf-citationDFU_QUTNet: diabetic foot ulcer classification using novel deep convolutional neural network
Alzubaidi, L., Fadhel, M. A., Oleiwi, S. R., Al-Shamma, O., & Zhang, J. (2020) · Multimedia Tools and Applications · doi:10.1007/s11042-019-07820-w · Cited by 220
Self-citationBCN20000: Dermoscopic Lesions in the Wild
Hernández-Pérez, C., Combalia, M., Podlipnik, S., Codella, N. C. F., Rotemberg, V., Halpern, A. C., Reiter, O., Carrera, C., Barreiro, A., Helba, B., et al. (2024) · Scientific Data · doi:10.1038/s41597-024-03387-w · Cited by 214
Hamamoto, R., Suvarna, K., Yamada, M., Kobayashi, K., Shinkai, N., Miyake, M., Takahashi, M., Jinnai, S., Shimoyama, R., Sakai, A., et al. (2020) · Cancers · doi:10.3390/cancers12123532 · Cited by 207
Alzubaidi, L., Fadhel, M. A., Al-Shamma, O., Zhang, J., & Duan, Y. (2020) · Electronics · doi:10.3390/electronics9030427 · Cited by 197
Self-citationRetinal Image Analysis for Diabetes-Based Eye Disease Detection Using Deep Learning
Nazir, T., Irtaza, A., Javed, A., Malik, H. M. A., Hussain, D., & Naqvi, R. A. (2020) · Applied Sciences · doi:10.3390/app10186185 · Cited by 173
Ferreira, C. A., Melo, T., de Sousa, P., Meyer, M. I., Shakibapour, E., Costa, P., & Campilho, A. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-93000-8_86 · Cited by 149
Alzubaidi, L., Al-Shamma, O., Fadhel, M. A., Farhan, L., Zhang, J., & Duan, Y. (2020) · Electronics · doi:10.3390/electronics9030445 · Cited by 143 · cited 3×
“The authors empirically showed that in-domain TL can improve the performance against the pretrained ImageNet models [12–14].”
BackgroundInfluentialSelf-citationDeep Learning Framework for Multi-class Breast Cancer Histology Image Classification
Vang, Y. S., Chen, Z., & Xie, X. S. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-93000-8_104 · Cited by 135
BreCaHAD: a dataset for breast cancer histopathological annotation and diagnosis
Aksac, A., Demetrick, D. J., Ozyer, T., & Alhajj, R. (2019) · BMC Research Notes · doi:10.1186/s13104-019-4121-7 · Cited by 123
Woven Fabric Pattern Recognition and Classification Based on Deep Convolutional Neural Networks
Hussain, M. A. I., Khan, B., Wang, Z., & Ding, S. (2020) · Electronics · doi:10.3390/electronics9061048 · Cited by 117
Bi, L., Feng, D. D., Fulham, M., & Kim, J. (2020) · Pattern Recognition · doi:10.1016/j.patcog.2020.107502 · Cited by 109
Breast Cancer Diagnosis with Transfer Learning and Global Pooling
Kassani, S. H., Kassani, P. H., Wesolowski, M. J., Schneider, K. A., & Deters, R. (2019) · 2019 International Conference on Information and Communication Technology Convergence (ICTC) · doi:10.1109/ictc46691.2019.8939878 · Cited by 106
A histopathological image dataset for grading breast invasive ductal carcinomas
Bolhasani, H., Amjadi, E., Tabatabaeian, M. S., & Jassbi, S. J. (2020) · Informatics in Medicine Unlocked · doi:10.1016/j.imu.2020.100341 · Cited by 106
Wang, J., Liu, Q., Xie, H., Yang, Z., & Zhou, H. (2021) · Cancers · doi:10.3390/cancers13040661 · Cited by 106
Accelerating flash calculation through deep learning methods
Li, Y., Zhang, T., Sun, S., & Gao, X. (2019) · Journal of Computational Physics · doi:10.1016/j.jcp.2019.05.028 · Cited by 67
Valieris, R., Amaro, L., Osório, C. A. B. D. T., Bueno, A. P., Rosales Mitrowsky, R. A., Carraro, D. M., Nunes, D. N., Dias-Neto, E., & Silva, I. T. D. (2020) · Cancers · doi:10.3390/cancers12123687 · Cited by 64
Breast Cancer Histology Image Classification Based on Deep Neural Networks
Guo, Y., Dong, H., Song, F., Zhu, C., & Liu, J. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-93000-8_94 · Cited by 56
Wang, Z., Dong, N., Dai, W., Rosario, S. D., & Xing, E. P. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-93000-8_84 · Cited by 50
Dayal, A., Paluru, N., Cenkeramaddi, L. R., Soumya, J., & Yalavarthy, P. K. (2021) · Electronics · doi:10.3390/electronics10020182 · Cited by 46
Lin, C.-H., Lin, C., Li, Y.-C., & Wang, S.-H. (2021) · Applied Sciences · doi:10.3390/app11020480 · Cited by 33
Mallio, C. A., Napolitano, A., Castiello, G., Giordano, F. M., D'Alessio, P., Iozzino, M., Sun, Y., Angeletti, S., Russano, M., Santini, D., et al. (2021) · Cancers · doi:10.3390/cancers13040652 · Cited by 30
Generalizability of Deep Learning System for the Pathologic Diagnosis of Various Cancers
Jang, H., Song, I. H., & Lee, S. H. (2021) · Applied Sciences · doi:10.3390/app11020808 · Cited by 20
Hierarchical Fine-Tuning for joint Liver Lesion Segmentation and Lesion Classification in CT
Heker, M., Ben-Cohen, A., & Greenspan, H. K. (2019) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2019.8857127 · Cited by 16
Alzubaidi, L., Hasan, R. I., Awad, F. H., Fadhel, M. A., Al-Shamma, O., & Zhang, J. (2019) · International Conference on Developments in eSystems Engineering (DeSE) · doi:10.1109/dese.2019.00057 · Cited by 11
Self-citationШаповалова, С. І., & Moskalenko, Y. (2020) · ScienceRise · doi:10.21303/2313-8416.2020.001550 · Cited by 3
Research works printed but not linked: 7
- Jeon, H.K., Kim, S., Edwin, J., and Yang, C.S. (2020). Sea Fog Identification From GOCI Images Using CNN Transfer Learning Models. Electronics, 9. · doi:10.3390/electronics9020311
- Xie, 2019, Dual network architecture for few-view CT-trained on ImageNet data and transferred for medical imaging, Developments in X-ray Tomography XII, Volume 11113, 111130V
- Neyshabur, B., Sedghi, H., and Zhang, C. (2020). What is being transferred in transfer learning?. arXiv.
- Heker, M., and Greenspan, H. (2020). Joint liver lesion segmentation and classification via transfer learning. arXiv.
- Gutman, D., Codella, N.C., Celebi, E., Helba, B., Marchetti, M., Mishra, N., and Halpern, A. (2016). Skin lesion analysis toward melanoma detection: A challenge at the international symposium on biomedical imaging (ISBI) 2016, hosted by the international skin imaging collaboration (ISIC). arXiv.
- Karki, S., Kulkarni, P., and Stranieri, A. (2021, January 1). Melanoma classification using EfficientNets and Ensemble of models with different input resolution. Proceedings of the 2021 Australasian Computer Science Week Multiconference, Dunedin, New Zealand. · doi:10.1145/3437378.3437396
- Sarker, M.I., Kim, H., Tarasov, D., and Akhmetzanov, D. (2019). Inception Architecture and Residual Connections in Classification of Breast Cancer Histology Images. arXiv.
Cited by
Transfer learning for medical image classification: a literature review
Kim, H. E., Cosa-Linan, A., Santhanam, N., Jannesari, M., Maros, M. E., & Ganslandt, T. (2022) · BMC Medical Imaging · doi:10.1186/s12880-022-00793-7 · Cited by 965
Alzubaidi, L., Bai, J., Al-Sabaawi, A., Santamaría, J., Albahri, A. S., Al-dabbagh, B. S. N., Fadhel, M. A., Manoufali, M., Zhang, J., Al-Timemy, A. H., et al. (2023) · Journal of Big Data · doi:10.1186/s40537-023-00727-2 · Cited by 907
Self-citationQiu, S., Zhao, H., Jiang, N., Wang, Z., Liu, L., An, Y., Zhao, H., Miao, X., Liu, R., & Fortino, G. (2022) · Information Fusion · doi:10.1016/j.inffus.2021.11.006 · Cited by 671
Zhao, Z., Alzubaidi, L., Zhang, J., Duan, Y., & Gu, Y. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.122807 · Cited by 410
Self-citationUniverSeg: Universal Medical Image Segmentation
Butoi, V. I., Gonzalez Ortiz, J. J., Ma, T., Sabuncu, M. R., Guttag, J., & Dalca, A. V. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01960 · Cited by 195
Metadata
- Authors
- Alzubaidi, L., Al-Amidie, M., Al-Asadi, A., Humaidi, A. J., Al-Shamma, O., Fadhel, M. A., Zhang, J., Santamaría, J., & Duan, Y.
- Venue
- Cancers
- Publisher
- MDPI AG
- Published
- 2021-03-30
- Volume
- 13
- Issue
- 7
- Pages
- 1590
- ISSN
- 2072-6694 (ISSN-L)
- OpenAlex
- W3148908072
- Cited by
- 292
- References
- 62
- FWCI
- 17.38
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.