The DOI matches its OpenAlex record; 36 of the 36 references it cites were found in a registry or an index.
Transfusion: Understanding Transfer Learning for Medical Imaging
Open accessAbstract
Transfer learning from natural image datasets, particularly ImageNet, using standard large models and corresponding pretrained weights has become a de-facto method for deep learning applications to medical imaging. However, there are fundamental differences in data sizes, features and task specifications between natural image classification and the target medical tasks, and there is little understanding of the effects of transfer. In this paper, we explore properties of transfer learning for medical imaging. A performance evaluation on two large scale medical imaging tasks shows that surprisingly, transfer offers little benefit to performance, and simple, lightweight models can perform comparably to ImageNet architectures. Investigating the learned representations and features, we find that some of the differences from transfer learning are due to the over-parametrization of standard models rather than sophisticated feature reuse. We isolate where useful feature reuse occurs, and outline the implications for more efficient model exploration. We also explore feature independent benefits of transfer arising from weight scalings.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,395
Dermatologist-level classification of skin cancer with deep neural networks
Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017) · Nature · doi:10.1038/nature21056 · Cited by 14,362
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S. (2024) · arXiv (Cornell University) · doi:10.57702/o9raffed · Cited by 14,214
Gulshan, V., Peng, L., Coram, M., Stumpe, M. C., Wu, D., Narayanaswamy, A., Venugopalan, S., Widner, K., Madams, T., Cuadros, J., et al. (2016) · JAMA · doi:10.1001/jama.2016.17216 · Cited by 7,697
How transferable are features in deep neural networks?
Yosinski, J., Clune, J., Bengio, Y., & Lipson, H. (2014) · arXiv · doi:10.48550/arxiv.1411.1792 · Cited by 6,061 · cited 2×
“Interestingly, there is no performance drop due to the coadaptation issue [32], with partial ImageNet initialization performing equally to initialzing with all of the ImageNet weights. results on with various architectures.”
BackgroundWang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., & Summers, R. M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.369 · Cited by 4,616
Clinically applicable deep learning for diagnosis and referral in retinal disease
De Fauw, J., Ledsam, J. R., Romera-Paredes, B., Nikolov, S., Tomasev, N., Blackwell, S., Askham, H., Glorot, X., O’Donoghue, B., Visentin, D., et al. (2018) · Nature Medicine · doi:10.1038/s41591-018-0107-6 · Cited by 2,639
Lempitsky, V., Vedaldi, A., & Ulyanov, D. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00984 · Cited by 1,916
Understanding deep image representations by inverting them
Mahendran, A., & Vedaldi, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299155 · Cited by 1,827
Geirhos, R., Rubisch, P., Michaelis, C., Bethge, M., Wichmann, F. A., & Brendel, W. (2018) · arXiv · doi:10.48550/arxiv.1811.12231 · Cited by 1,490 · cited 1×
“Despite the immense popularity of transfer learning in medical imaging, there has been little work studying its precise effects, even as recent work on transfer learning in the natural image setting [11, 16, 20, 12, 7] has challenged many commonly held beliefs.”
BackgroundDo Better ImageNet Models Transfer Better?
Kornblith, S., Shlens, J., & Le, Q. V. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00277 · Cited by 1,325
CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning
Rajpurkar, P., Irvin, J., Zhu, K., Yang, B., Mehta, H., Duan, T., Ding, D. Y., Bagul, A., Langlotz, C. P., Shpanskaya, K. S., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.05225 · Cited by 1,219
Taskonomy: Disentangling Task Transfer Learning
Zamir, A. R., Sax, A., Shen, W., Guibas, L., Malik, J., & Savarese, S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00391 · Cited by 1,128
Rethinking ImageNet Pre-Training
He, K., Girshick, R., & Dollar, P. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00502 · Cited by 1,049
Deep Learning for Identifying Metastatic Breast Cancer
Wang, D., Khosla, A., Gargeya, R., Irshad, H., & Beck, A. H. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.05718 · Cited by 756
Similarity of Neural Network Representations Revisited
Kornblith, S., Norouzi, M., Lee, H., & Hinton, G. (2019) · arXiv · doi:10.48550/arxiv.1905.00414 · Cited by 724
A Deep Learning Model to Predict a Diagnosis of Alzheimer Disease by Using 18 F-FDG PET of the Brain
Ding, Y., Sohn, J. H., Kawczynski, M. G., Trivedi, H., Harnish, R., Jenkins, N. W., Lituiev, D., Copeland, T. P., Aboian, M. S., Mari Aparici, C., et al. (2019) · Radiology · doi:10.1148/radiol.2018180958 · Cited by 577
Detecting Cancer Metastases on Gigapixel Pathology Images
Liu, Y., Gadepalli, K., Norouzi, M., Dahl, G. E., Kohlberger, T., Boyko, A., Venugopalan, S., Timofeev, A., Nelson, P. Q., Corrado, G. S., et al. (2017) · arXiv · doi:10.48550/arxiv.1703.02442 · Cited by 506
Learning Bounds for Domain Adaptation
Blitzer, J. C., Crammer, K., Kulesza, A., Pereira, F. M. B., & Wortman, J. R. (2007) · Cited by 393
CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison
Irvin, J., Rajpurkar, P., Ko, M., Yu, Y., Ciurea-Ilcus, S., Chute, C., Marklund, H., Haghgoo, B., Ball, R., Shpanskaya, K., et al. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.3301590 · Cited by 392
Efficient Deep Network Architectures for Fast Chest X-Ray Tuberculosis Screening and Visualization
Pasa, F., Golkov, V., Pfeiffer, F., Cremers, D., & Pfeiffer, D. (2019) · Scientific Reports · doi:10.1038/s41598-019-42557-4 · Cited by 370
Raghu, M., Gilmer, J., Yosinski, J., & Sohl-Dickstein, J. (2017) · arXiv · doi:10.48550/arxiv.1706.05806 · Cited by 335
Self-citationvan der Heijden, A. A., Abramoff, M. D., Verbraak, F., van Hecke, M. V., Liem, A., & Nijpels, G. (2018) · Acta Ophthalmologica · doi:10.1111/aos.13613 · Cited by 296 · cited 1×
“The research on ophthalmology has also culminated in FDA approval [28], and full clinical deployment [29].”
BackgroundA Closer Look at Deep Learning Heuristics: Learning rate restarts, Warmup and DistillationPreprint
Gotmare, A. D., Keskar, N. S., Xiong, C., & Socher, R. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.13243 · Cited by 268
What makes ImageNet good for transfer learning?
Huh, M., Agrawal, P., & Efros, A. A. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1608.08614 · Cited by 266
Are All Layers Created Equal?Preprint
Zhang, C., Bengio, S., & Singer, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.01996 · Cited by 166
Self-citationInvestigating Multilingual NMT Representations at Scale
Kudugunta, S., Bapna, A., Caswell, I. R., & Fırat, O. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1167 · Cited by 145
Domain Adaptive Transfer Learning with Specialist ModelsPreprint
Ngiam, J., Peng, D., Vasudevan, V., Kornblith, S., Le, Q. V., & Pang, R. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.07056 · Cited by 138
Diabetic retinopathy – Biomolecules and multiple pathophysiology
Ahsan, H. (2014) · Diabetes & Metabolic Syndrome Clinical Research & Reviews · doi:10.1016/j.dsx.2014.09.011 · Cited by 122
Shin, H.-C., Lü, L., Kim, L., Seff, A., Yao, J., & Summers, R. M. (2016) · arXiv (Cornell University) · doi:10.5555/2946645.3007060 · Cited by 46
Understanding Learning Dynamics Of Language Models with SVCCA
Saphra, N., & Lopez, A. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1329 · Cited by 31
A theoretical framework for deep transfer learning
Galanti, T., Wolf, L., & Hazan, T. (2016) · Information and Inference · doi:10.1093/imaiai/iaw008 · Cited by 29
Voita, E., Sennrich, R., & Titov, I. (2019) · arXiv · doi:10.48550/arxiv.1909.01380 · Cited by 17
Cited by
Review of deep learning: concepts, CNN architectures, challenges, applications, future directions
Alzubaidi, L., Zhang, J., Humaidi, A. J., Al-Dujaili, A., Duan, Y., Al-Shamma, O., Santamaría, J., Fadhel, M. A., Al-Amidie, M., & Farhan, L. (2021) · Journal of Big Data · doi:10.1186/s40537-021-00444-8 · Cited by 7,906
Domain adaptation for medical image analysis: A survey
Guan, H., & Liu, M. (2022) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2021.3117407 · Cited by 1,057
Big Transfer (BiT): General Visual Representation Learning
Kolesnikov, A., Beyer, L., Zhai, X., Puigcerver, J., Yung, J., Gelly, S., & Houlsby, N. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58558-7_29 · Cited by 1,039
Big Transfer (BiT): General Visual Representation Learning
Kolesnikov, A., Beyer, L., Zhai, X., Puigcerver, J., Yung, J., Gelly, S., & Houlsby, N. (2019) · arXiv · doi:10.48550/arxiv.1912.11370 · Cited by 1,039
Deep semantic segmentation of natural and medical images: a review
Asgari Taghanaki, S., Abhishek, K., Cohen, J. P., Cohen-Adad, J., & Hamarneh, G. (2021) · Artificial Intelligence Review · doi:10.1007/s10462-020-09854-1 · Cited by 870
Metadata
- Authors
- Raghu, M., Zhang, C., Kleinberg, J. M., & Bengio, S.
- Venue
- arXiv (Cornell University)
- Publisher
- arXiv
- Published
- 2019
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1902.07208
- OpenAlex
- W2947706148
- Cited by
- 613
- References
- 36
- Type
- posted content
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.