Breast mass classification in sonography with transfer learning using a deep convolutional neural network and color conversion
Open accessAbstract
Purpose: We propose a deep learning‐based approach to breast mass classification in sonography and compare it with the assessment of four experienced radiologists employing breast imaging reporting and data system 4th edition lexicon and assessment protocol. Methods: Several transfer learning techniques are employed to develop classifiers based on a set of 882 ultrasound images of breast masses. Additionally, we introduce the concept of a matching layer. The aim of this layer is to rescale pixel intensities of the grayscale ultrasound images and convert those images to red, green, blue (RGB) to more efficiently utilize the discriminative power of the convolutional neural network pretrained on the ImageNet dataset. We present how this conversion can be determined during fine‐tuning using back‐propagation. Next, we compare the performance of the transfer learning techniques with and without the color conversion. To show the usefulness of our approach, we additionally evaluate it using two publicly available datasets. Results: Color conversion increased the areas under the receiver operating curve for each transfer learning method. For the better‐performing approach utilizing the fine‐tuning and the matching layer, the area under the curve was equal to 0.936 on a test set of 150 cases. The areas under the curves for the radiologists reading the same set of cases ranged from 0.806 to 0.882. In the case of the two separate datasets, utilizing the proposed approach we achieved areas under the curve of around 0.890. Conclusions: The concept of the matching layer is generalizable and can be used to improve the overall performance of the transfer learning techniques using deep convolutional neural networks. When fully developed as a clinical tool, the methods proposed in this paper have the potential to help radiologists with breast mass classification in ultrasound.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Sixteen breast studies compared algorithm performance against healthcare professionals.
“Sixteen studies62,91,92,94,97–107 compared algorithm performance against healthcare professionals.”
From Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis (2021) · cites this paper as [99]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Bray, F., Ferlay, J., Soerjomataram, I., Siegel, R. L., Torre, L. A., & Jemal, A. (2018) · CA: A Cancer Journal for Clinicians · doi:10.3322/caac.21492 · Cited by 84,109 · see citing works
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 74,248 · see citing works
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · see citing works
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562 · see citing works
LIBSVM: A library for support vector machines
Chang, C.-C., & Lin, C.-J. (2011) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/1961189.1961199 · Cited by 40,918 · see citing works
An introduction to ROC analysis
Fawcett, T. (2006) · Pattern Recognition Letters · doi:10.1016/j.patrec.2005.10.010 · Cited by 21,879 · see citing works
A survey on deep learning in medical image analysis
Litjens, G., Kooi, T., Bejnordi, B. E., Setio, A. A. A., Ciompi, F., Ghafoorian, M., van der Laak, J. A. W. M., van Ginneken, B., & Sánchez, C. I. (2017) · Medical Image Analysis · doi:10.1016/j.media.2017.07.005 · Cited by 15,332 · see citing works
Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning
Szegedy, C., Ioffe, S., Vanhoucke, V., & Alemi, A. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.11231 · Cited by 4,636 · see citing works
Democracy Barometer. Methodology. Version 4.
Merkel, W., Bochsler, D., Bousbah, K., Bühlmann, M., Giebler, H., Hänni, M., Heyne, L., Müller, L., Ruth, S., & Weßels, B. (2004) · Intellectum (Universidad de La Sabana) · doi:10.1007/s13398-014-0173-7.2 · Cited by 2,331 · see citing works
Berg, W. A. (2008) · JAMA · doi:10.1001/jama.299.18.2151 · Cited by 1,527 · see citing works
Automated breast ultrasound lesions detection using convolutional neural networks
Yap, M. H., Pons, G., Martí, J., Ganau, S., Sentís, M., Zwiggelaar, R., Davison, A. K., & Martí, R. (2018) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2017.2731873 · Cited by 1,086 · see citing works
Cheng, J.-Z., Ni, D., Chou, Y.-H., Qin, J., Tiu, C.-M., Chang, Y.-C., Huang, C.-S., Shen, D., & Chen, C.-M. (2016) · Scientific Reports · doi:10.1038/srep24454 · Cited by 755 · see citing works
Automated breast cancer detection and classification using ultrasound images: A survey
Cheng, H.-D., Shan, J., Ju, W., Guo, Y., & Zhang, L. (2009) · Pattern Recognition · doi:10.1016/j.patcog.2009.05.012 · Cited by 729 · see citing works
Cheng, Z., Yang, Q., & Sheng, B. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.55 · Cited by 606 · see citing works
Controlling Perceptual Factors in Neural Style Transfer
Gatys, L. A., Ecker, A. S., Bethge, M., Hertzmann, A., & Shechtman, E. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.397 · Cited by 495 · see citing works
Hooley, R. J., Greenberg, K. L., Stackhouse, R. M., et al. (2012) · Radiology · doi:10.1148/radiol.12120621 · Cited by 407 · see citing works
Antropova, N., Huynh, B. Q., & Giger, M. L. (2017) · Medical Physics · doi:10.1002/mp.12453 · Cited by 389 · cited 2× · see citing works
“First, we used the grayscale images to perform classification of the B-mode images from the main dataset following the standard approach.(19,20,22,25) For each US image, the gray level intensities were copied to RGB channels and the VGG19 CNN was tuned.”
A deep learning framework for supporting the classification of breast lesions in ultrasound images
Han, S., Kang, H.-K., Jeong, J.-Y., Park, M.-H., Kim, W., Bang, W.-C., & Seong, Y.-K. (2017) · Physics in Medicine & Biology · doi:10.1088/1361-6560/aa82ec · Cited by 358 · see citing works
Medical breast ultrasound image segmentation by machine learning
Xu, Y. X., Wang, Y., Yuan, J., Cheng, Q., Wang, X., & Carson, P. L. (2018) · Ultrasonics · doi:10.1016/j.ultras.2018.07.006 · Cited by 280 · see citing works
Chen, C., Chou, Y., Han, K.-C., Hung, G.-S., Tiu, C., Chiou, H., & Chiou, S. (2003) · Radiology · doi:10.1148/radiol.2262011843 · Cited by 257 · see citing works
Joo, S., Yang, Y., Moon, W. K., & Kim, H. C. (2004) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2004.834617 · Cited by 255 · see citing works
Deep learning based classification of breast tumors with shear-wave elastography
Zhang, Q., Xiao, Y., Dai, W., Suo, J., Wang, C., Shi, J., & Zheng, H. (2016) · Ultrasonics · doi:10.1016/j.ultras.2016.08.004 · Cited by 240 · see citing works
Breast Image Analysis for Risk Assessment, Detection, Diagnosis, and Treatment of Cancer
Giger, M. L., Karssemeijer, N., & Schnabel, J. A. (2013) · Annual Review of Biomedical Engineering · doi:10.1146/annurev-bioeng-071812-152416 · Cited by 211 · see citing works
Open access database of raw ultrasonic signals acquired from malignant and benign breast lesions
Piotrzkowska‐Wróblewska, H., Dobruch‐Sobczak, K., Byra, M., & Nowicki, A. (2017) · Medical Physics · doi:10.1002/mp.12538 · Cited by 125 · see citing works
Self-citationBreast Ultrasound Computer-Aided Diagnosis Using BI-RADS Features
Shen, W.-C., Chang, R., Moon, W. K., Chou, Y., & Huang, C. (2007) · Academic Radiology · doi:10.1016/j.acra.2007.04.016 · Cited by 123 · see citing works
Improving classification performance of breast lesions on ultrasonography
Gómez‐Flores, W., Pereira, W. C. D. A., & Infantosi, A. F. C. (2014) · Pattern Recognition · doi:10.1016/j.patcog.2014.06.006 · Cited by 105 · see citing works
Huynh, B. Q., Drukker, K., & Giger, M. L. (2016) · Medical Physics · doi:10.1118/1.4957255 · Cited by 91 · see citing works
Color Image Segmentation Based on Different Color Space Models Using Automatic GrabCut
Khattab, D., Ebied, H. M., Hussein, A. S., et al. (2014) · The Scientific World JOURNAL · doi:10.1155/2014/126025 · Cited by 75 · see citing works
Colour Mapping: A Review of Recent Methods, Extensions and Applications
Faridul, H. S., Pouli, T., Chamaret, C., Stauder, J., Reinhard, E., Kuzovkin, D., & Trémeau, A. (2015) · Computer Graphics Forum · doi:10.1111/cgf.12671 · Cited by 75 · see citing works
Byra, M., Nowicki, A., Piotrzkowska‐Wróblewska, H., et al. (2016) · Medical Physics · doi:10.1118/1.4962928 · Cited by 56 · see citing works
Self-citationDrukker, K., Gruszauskas, N. P., Sennett, C. A., & Giger, M. L. (2008) · Radiology · doi:10.1148/radiol.2482071778 · Cited by 52 · see citing works
Breast Ultrasound Image Classification and Segmentation Using Convolutional Neural Networks
Xie, X., Shi, F., Niu, J., & Tang, X. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00764-5_19 · Cited by 51 · see citing works
Evaluation of the attenuation coefficients in normal and pathological breast tissue
Landini, L., & Sarnelli, R. (1986) · Medical & Biological Engineering & Computing · doi:10.1007/bf02441619 · Cited by 50 · see citing works
Discriminant analysis of neural style representations for breast lesion classification in ultrasound
Byra, M. (2018) · Journal of Applied Biomedicine · doi:10.1016/j.bbe.2018.05.003 · Cited by 50 · see citing works
Self-citationComputer-aided US Diagnosis of Breast Lesions by Using Cell-based Contour Grouping
Cheng, J., Chou, Y., Huang, C., Chang, Y., Tiu, C., Chen, K., & Chen, C. (2010) · Radiology · doi:10.1148/radiol.09090001 · Cited by 47 · see citing works
Bian, C., Lee, R., Chou, Y., & Cheng, J. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-66179-7_30 · Cited by 35 · see citing works
A Comparison of Color Models for Color Face Segmentation
Sanchez-Cuevas, M. C., Aguilar‐Ponce, R. M., & Tecpanecatl-Xihuitl, J. L. (2013) · Procedia Technology · doi:10.1016/j.protcy.2013.04.017 · Cited by 34 · see citing works
Nemat, H., Fehri, H., Ahmadinejad, N., et al. (2018) · Medical Physics · doi:10.1002/mp.13082 · Cited by 32 · see citing works
Dobruch‐Sobczak, K., Piotrzkowska‐Wróblewska, H., Roszkowska‐Purska, K., et al. (2016) · Clinical Radiology · doi:10.1016/j.crad.2016.11.009 · Cited by 28 · see citing works
Hu, Y., Qiao, M., Guo, Y., et al. (2017) · Medical Physics · doi:10.1002/mp.12275 · Cited by 17 · see citing works
Rodríguez-Cristerna, A., Guerrero-Cedillo, C. P., Donati-Olvera, G. A., et al. (2017) · International Conference on Electrical Engineering, Computing Science and Automatic Control (CCE) · doi:10.1109/iceee.2017.8108826 · Cited by 16 · see citing works
André, M. P., Galperin, M. Y., Berry, A., et al. (2011) · Acoustical imaging · doi:10.1007/978-90-481-3255-3_1 · Cited by 2 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 3
- [22]Yap MH, Goyal M, Osman F, End-to-end breast ultrasound lesions recognition with a deep learning approach. In: Medical Imaging 2018: Biomedical Applications in Molecular, Structural, and Functional Imaging. Vol 10578.; 2018: 1057819.
- [35]Abadi M, Barham P, Chen J, TensorFlow: A System for Large-Scale Machine Learning. In: OSDI. Vol 16.; 2016: 265– 283.
- [37]Zheng L, Zhao Y, Wang S, Wang J, Tian Q. Good practice in CNN feature transfer. arXiv Prepr arXiv 160400133. 2016.
Cited by
Sorted by citations
Liu, X., Faes, L., Kale, A. U., Wagner, S. K., Fu, D. J., Bruynseels, A., Mahendiran, T., Moraes, G., Shamdas, M., Kern, C., et al. (2019) · The Lancet Digital Health · doi:10.1016/s2589-7500(19)30123-2 · Cited by 1,883 · see citing works
Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis
Aggarwal, R., Sounderajah, V., Martin, G., Ting, D. S. W., Karthikesalingam, A., King, D., Ashrafian, H., & Darzi, A. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00438-z · Cited by 946 · see citing works
A scoping review of transfer learning research on medical image analysis using ImageNet
Morid, M. A., Borjali, A., & Del Fiol, G. (2021) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2020.104115 · Cited by 508 · see citing works
Malik, H., Farooq, M. S., Khelifi, A., Abid, A., Nasir Qureshi, J., & Hussain, M. (2020) · IEEE Access · doi:10.1109/access.2020.3004766 · Cited by 104 · see citing works
Pfob, A., Sidey-Gibbons, C., Barr, R. G., Duda, V., Alwafai, Z., Balleyguier, C., Clevert, D.-A., Fastner, S., Gomez, C., Goncalo, M., et al. (2022) · European Radiology · doi:10.1007/s00330-021-08519-z · Cited by 29 · see citing works
- Authors
- Byra, M., Galperin, M., Ojeda‐Fournier, H., Olson, L., O'Boyle, M., Comstock, C., & Andre, M.
- Affiliations
- Department of Radiology University of California, San Diego 9500 Gilman Drive La Jolla CA 92093 USA
- Almen Laboratories (United States) (US)
- University of California San Diego (US)
- Memorial Sloan Kettering Cancer Center (US)
- Venue
- Medical Physics
- Publisher
- Wiley
- Published
- 2019-01-16
- Volume
- 46
- Issue
- 2
- Pages
- 746-755
- ISSN
- 0094-2405 (ISSN-L)
- 2473-4209
- PMC
- PMC8544811
- OpenAlex
- W2906785117
- Cited by
- 253
- References
- 42
- Citations by year
- 2026: 10 · 2025: 25 · 2024: 35 · 2023: 46 · 2022: 51 · 2021: 40 · 2020: 33 · 2019: 13
- FWCI
- 12.12
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Medical Physics (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- eScholarship (California Digital Library) (submittedVersion)
- Funders
- National Institutes of Health
- National Cancer Institute
- Gustavus and Louise Pfeiffer Research Foundation
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- breast mass classification, breast ultrasound, transfer learning, color conversion, matching layer, fine-tuning, BI-RADS lexicon, radiologist comparison, computer-aided diagnosis
- MeSH terms
- Deep Learning, Adolescent, Adult, Breast Neoplasms, Color, Female, Humans, Image Processing, Computer-Assisted, ROC Curve, Ultrasonography, Young Adult
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
