The DOI matches its Crossref record; 24 of the 24 references it cites were found in a registry or an index.
Lymph Node Metastasis Prediction from Primary Breast Cancer US Images Using Deep Learning
Open accessAbstract
Background Deep learning (DL) algorithms are gaining extensive attention for their excellent performance in image recognition tasks. DL models can automatically make a quantitative assessment of complex medical image characteristics and achieve increased accuracy in diagnosis with higher efficiency. Purpose To determine the feasibility of using a DL approach to predict clinically negative axillary lymph node metastasis from US images in patients with primary breast cancer. Materials and Methods A data set of US images in patients with primary breast cancer with clinically negative axillary lymph nodes from Tongji Hospital (974 imaging studies from 2016 to 2018, 756 patients) and an independent test set from Hubei Cancer Hospital (81 imaging studies from 2018 to 2019, 78 patients) were collected. Axillary lymph node status was confirmed with pathologic examination. Three different convolutional neural networks (CNNs) of Inception V3, Inception-ResNet V2, and ResNet-101 architectures were trained on 90% of the Tongji Hospital data set and tested on the remaining 10%, as well as on the independent test set. The performance of the models was compared with that of five radiologists. The models’ performance was analyzed in terms of accuracy, sensitivity, specificity, receiver operating characteristic curves, areas under the receiver operating characteristic curve (AUCs), and heat maps. Results The best-performing CNN model, Inception V3, achieved an AUC of 0.89 (95% confidence interval [CI]: 0.83, 0.95) in the prediction of the final clinical diagnosis of axillary lymph node metastasis in the independent test set. The model achieved 85% sensitivity (35 of 41 images; 95% CI: 70%, 94%) and 73% specificity (29 of 40 images; 95% CI: 56%, 85%), and the radiologists achieved 73% sensitivity (30 of 41 images; 95% CI: 57%, 85%; P = .17) and 63% specificity (25 of 40 images; 95% CI: 46%, 77%; P = .34). Conclusion Using US images from patients with primary breast cancer, deep learning models can effectively predict clinically negative axillary lymph node metastasis. Artificial intelligence may provide an early diagnostic strategy for lymph node metastasis in patients with breast cancer with clinically negative lymph nodes. Published under a CC BY 4.0 license. Online supplemental material is available for this article. See also the editorial by Bae in this issue.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The data augmentation strategy has proved to help prevent network overfitting and memorization of the exact training image details [21].”
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
DeLong, E. R., DeLong, D. M., & Clarke-Pearson, D. L. (1988) · Biometrics · doi:10.2307/2531595 · Cited by 22,649
Proceedings of IEEE Conference on Computer Vision and Pattern Recognition
(1993) · doi:10.1109/cvpr.1993 · Cited by 17,989
Siegel, R. L., Miller, K. D., & Jemal, A. (2018) · CA A Cancer Journal for Clinicians · doi:10.3322/caac.21442 · Cited by 16,632
A survey on deep learning in medical image analysis
Litjens, G., Kooi, T., Bejnordi, B. E., Setio, A. A. A., Ciompi, F., Ghafoorian, M., van der Laak, J. A. W. M., van Ginneken, B., & Sánchez, C. I. (2017) · Medical Image Analysis · doi:10.1016/j.media.2017.07.005 · Cited by 15,332
Artificial intelligence in radiology
Hosny, A., Parmar, C., Quackenbush, J., Schwartz, L. H., & Aerts, H. J. W. L. (2018) · Nature Reviews Cancer · doi:10.1038/s41568-018-0016-5 · Cited by 3,803
Giuliano, A. E., Ballman, K. V., McCall, L. M., Beitsch, P. D., Brennan, M. B., Kelemen, P. R., Ollila, D. W., Hansen, N. M., Whitworth, P. W., Blumencranz, P. W., et al. (2017) · JAMA · doi:10.1001/jama.2017.11470 · Cited by 1,922
Primary breast cancer: ESMO Clinical Practice Guidelines for diagnosis, treatment and follow-up
Senkus, E., Kyriakides, S., Ohno, S., Penault-Llorca, F. M., Poortmans, P. M., Rutgers, E. J. T., Zackrisson, S., & Cardoso, F. (2015) · Annals of Oncology · doi:10.1093/annonc/mdv298 · Cited by 1,660
Improving Computer-Aided Detection UsingConvolutional Neural Networks and Random View Aggregation
Roth, H. R., Lu, L., Liu, J., Yao, J., Seff, A., Cherry, K., Kim, L., & Summers, R. M. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2015.2482920 · Cited by 608
A Deep Learning Model to Predict a Diagnosis of Alzheimer Disease by Using 18 F-FDG PET of the Brain
Ding, Y., Sohn, J. H., Kawczynski, M. G., Trivedi, H., Harnish, R., Jenkins, N. W., Lituiev, D., Copeland, T. P., Aboian, M. S., Mari Aparici, C., et al. (2019) · Radiology · doi:10.1148/radiol.2018180958 · Cited by 577
Self-citationMicrometastases or Isolated Tumor Cells and the Outcome of Breast Cancer
de Boer, M., van Deurzen, C. H. M., van Dijck, J. A. A. M., Borm, G. F., van Diest, P. J., Adang, E., Nortier, J. W. R., Rutgers, E. J. T., Seynaeve, C., Menke‐Pluymers, M. B. E., et al. (2009) · New England Journal of Medicine · doi:10.1056/nejmoa0904832 · Cited by 524
Reverse Engineering the Neural Networks for Rule Extraction in Classification Problems
Augasta, M. G., & Kathirvalavakumar, T. (2011) · Neural Processing Letters · doi:10.1007/s11063-011-9207-8 · Cited by 183
Cho, N., Moon, W. K., Han, W., Park, I. A., Cho, J., & Noh, D. (2009) · American Journal of Roentgenology · doi:10.2214/ajr.09.3122 · Cited by 149
Ha, R. S., Chin, C., Karcich, J., Liu, M. Z., Chang, P., Mutasa, S., Sant, E. P. V., Wynn, R. T., Connolly, E. P., & Jambawalikar, S. (2018) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-018-0144-1 · Cited by 135
AIUM Practice Guideline for the Performance of a Breast Ultrasound Examination
(2009) · Journal of Ultrasound in Medicine · doi:10.7863/jum.2009.28.1.105 · Cited by 53
Molecular and Functional Imaging of Breast Cancer
Tafreshi, N. K., Kumar, V., Morse, D. L., & Gatenby, R. A. (2010) · Cancer Control · doi:10.1177/107327481001700302 · Cited by 50
Bae, M. S., Shin, S. U., Song, S. E., Ryu, H. S., Han, W., & Moon, W. K. (2017) · Acta Radiologica · doi:10.1177/0284185117723039 · Cited by 49
Distance of Breast Cancer From the Skin and Nipple Impacts Axillary Nodal Metastases
Ansari, B., Morton, M. J., Adamczyk, D. L., Jones, K. N., Brodt, J. K., Degnim, A. C., Jakub, J. W., Lohse, C. M., & Boughey, J. C. (2011) · Annals of Surgical Oncology · doi:10.1245/s10434-011-1957-z · Cited by 45
Stachs, A., Thi, A. T.-H., Dieterich, M., Stubert, J., Hartmann, S., Glass, Ä., Reimer, T., & Gerber, B. (2015) · Ultrasound International Open · doi:10.1055/s-0035-1555872 · Cited by 38
Yenıdünya, S., Bayrak, R., & Haltaş, H. (2011) · Diagnostic Pathology · doi:10.1186/1746-1596-6-18 · Cited by 35
Management of the Node-Positive Axilla in Breast Cancer in 2017
Morrow, M. (2017) · JAMA Oncology · doi:10.1001/jamaoncol.2017.3625 · Cited by 32
The Impact of Lymphovascular Invasion on Lymph Node Status in Patients with Breast Cancer
Cornwell, L. B., McMasters, K. M., & Chagpar, A. B. (2011) · The American Surgeon · doi:10.1177/000313481107700722 · Cited by 24
Torstenson, T., Shah-Khan, M. G., Hoskin, T. L., Morton, M. J., Adamczyk, D. L., Jones, K. N., Case, J. K., Chartier, S. R., & Boughey, J. C. (2013) · Annals of Surgical Oncology · doi:10.1245/s10434-013-3110-7 · Cited by 24
Yi, A., Moon, W. K., Cho, N., Chang, J. M., Bae, M. S., Kim, S. J., Han, W., & Park, I. A. (2013) · European Radiology · doi:10.1007/s00330-013-2930-y · Cited by 23
Flagged first, then by citations
Research works printed but not linked: 5
- Fujii T, 2015, Anticancer Res, 35, 3581
- Krizhevsky A, 2012, Advances in Neural Information Processing Systems, Lake Tahoe, Nevada, 1097
- Kayalibay B, 2017, CoRR
- Srivastava N, 2014, J Mach Learn Res, 15, 1929
- Ford RA, 2016, Social Science Electronic Publishing, 23, 1
Cited by
Sorted by citations
Explainable artificial intelligence (XAI) in deep learning-based medical image analysis
van der Velden, B. H. M., Kuijf, H. J., Gilhuijs, K. G. A., & Viergever, M. A. (2022) · Medical Image Analysis · doi:10.1016/j.media.2022.102470 · Cited by 1,328
External Validation of Deep Learning Algorithms for Radiologic Diagnosis: A Systematic Review
Yu, A. C., Mohajer, B., & Eng, J. (2022) · Radiology Artificial Intelligence · doi:10.1148/ryai.210064 · Cited by 477
Axillary Nodal Evaluation in Breast Cancer: State of the Art
Chang, J. M., Leung, J. W. T., Moy, L. P., Ha, S. M., & Moon, W. K. (2020) · Radiology · doi:10.1148/radiol.2020192534 · Cited by 389
Saba, T. (2020) · Journal of Infection and Public Health · doi:10.1016/j.jiph.2020.06.033 · Cited by 298
Diagnosis of breast cancer based on modern mammography using hybrid transfer learning
Khamparia, A., Bharati, S., Podder, P., Gupta, D., Khanna, A., Phung, T. K., & Thanh, D. N. H. (2021) · Multidimensional Systems and Signal Processing · doi:10.1007/s11045-020-00756-7 · Cited by 244
Metadata
- Authors
- Zhou, L.-Q., Wu, X.-L., Huang, S.-Y., Wu, G.-G., Ye, H.-R., Wei, Q., Bao, L.-Y., Deng, Y.-B., Li, X.-R., Cui, X.-W., & Dietrich, C. F.
- Affiliations
- Wuhan Textile University (CN)
- China Resources (China) (CN)
- ORCID
- Venue
- Radiology
- Published
- 2020-01
- Volume
- 294
- Issue
- 1
- Pages
- 19-28
- ISSN
- 0033-8419 (ISSN-L)
- 1527-1315
- OpenAlex
- W2989693433
- Cited by
- 384
- References
- 24
- Citations by year
- 2026: 36 · 2025: 61 · 2024: 66 · 2023: 88 · 2022: 69 · 2021: 41 · 2020: 20
- FWCI
- 15.06
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Radiology (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
- Health Sciences › Medicine › Pathology and Forensic Medicine › Breast Lesions and Carcinomas
- Keywords
- breast cancer, axillary lymph node metastasis, ultrasonography, convolutional neural networks, InceptionV3, lymph node metastasis prediction, radiologist comparison, breast cancer early detection, lymph node echogenicity
- MeSH terms
- Deep Learning, Adult, Aged, Aged, 80 and over, Algorithms, Breast Neoplasms, Feasibility Studies, Female, Humans, Image Interpretation, Computer-Assisted, Lymph Nodes, Lymphatic Metastasis, Middle Aged, Predictive Value of Tests, Retrospective Studies, Sensitivity and Specificity, Cohort Studies, Ultrasonography, Mammary, Neural Networks, Computer, Young Adult
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review