The DOI matches its Crossref record; 51 of the 51 references it cites were found in a registry or an index.
Robust breast cancer detection in mammography and digital breast tomosynthesis using an annotation-efficient deep learning approach
Open accessAbstract
Breast cancer remains a global challenge, causing over 600,000 deaths in 2018 (ref. 1). To achieve earlier cancer detection, health organizations worldwide recommend screening mammography, which is estimated to decrease breast cancer mortality by 20-40% (refs. 2,3). Despite the clear value of screening mammography, significant false positive and false negative rates along with non-uniformities in expert reader availability leave opportunities for improving quality and access 4,5. To address these limitations, there has been much recent interest in applying deep learning to mammography 6-18, and these efforts have highlighted two key difficulties: obtaining large amounts of annotated training data and ensuring generalization across populations, acquisition equipment and modalities. Here we present an annotation-efficient deep learning approach that (1) achieves state-of-the-art performance in mammogram classification, (2) successfully extends to digital breast tomosynthesis (DBT; '3D mammography'), (3) detects cancers in clinically negative prior mammograms of patients with cancer, (4) generalizes well to a population with low screening rates and (5) outperforms five out of five full-time breast-imaging specialists with an average increase in sensitivity of 14%. By creating new 'maximum suspicion projection' (MSP) images from DBT data, our progressively trained, multiple-instance learning approach effectively trains on DBT exams using only breast-level labels while maintaining localization-based interpretability. Altogether, our results demonstrate promise towards software that can improve the accuracy of and access to screening mammography worldwide.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Conversely, there are efforts to generate more diverse public data sets, such as EMBED (180), and instances where AI performance has generalized well across different patient populations (9,10,12).”
From Artificial Intelligence in Oncology: Current Landscape, Challenges, and Future Directions (2024) · cites this paper as [10]
In the citing paper
“A subsequent study presented an AI approach that encompasses both 2D and 3D mammography, with evidence of detecting breast cancer 1 to 2 years earlier than standard radiographic review for some cases (10).”
From Artificial Intelligence in Oncology: Current Landscape, Challenges, and Future Directions (2024) · cites this paper as [10]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Bray, F., Ferlay, J., Soerjomataram, I., Siegel, R. L., Torre, L. A., & Jemal, A. (2018) · CA: A Cancer Journal for Clinicians · doi:10.3322/caac.21492 · Cited by 84,109
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
A Threshold Selection Method from Gray-Level Histograms
Otsu, N. (1979) · IEEE Transactions on Systems, Man and Cybernetics · doi:10.1109/tsmc.1979.4310076 · Cited by 44,708
Focal Loss for Dense Object Detection
Lin, T.-Y., Goyal, P., Girshick, R., He, K., & Dollar, P. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.324 · Cited by 30,581
DeLong, E. R., DeLong, D. M., & Clarke-Pearson, D. L. (1988) · Biometrics · doi:10.2307/2531595 · Cited by 22,649
Intriguing properties of neural networks
Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., & Fergus, R. (2013) · arXiv · doi:10.48550/arxiv.1312.6199 · Cited by 8,143
(2018) · Cited by 3,510
International evaluation of an AI system for breast cancer screening
McKinney, S. M., Sieniek, M., Godbole, V., Godwin, J., Antropova, N., Ashrafian, H., Back, T., Chesus, M., Corrado, G. S., Darzi, A., et al. (2020) · Nature · doi:10.1038/s41586-019-1799-6 · Cited by 3,287
Effect of Screening and Adjuvant Therapy on Mortality from Breast Cancer
Berry, D. A., Cronin, K. A., Plevritis, S. K., Fryback, D. G., Clarke, L. D., Zelen, M., Mandelblatt, J. S., Yakovlev, A. Y., Habbema, J. D. F., & Feuer, E. J. (2005) · New England Journal of Medicine · doi:10.1056/nejmoa050518 · Cited by 2,456
Screening for breast cancer: U.S. Preventive Services Task Force recommendation statement
Siu, A. L., & on behalf of the U.S. Preventive Services Task Force (2016) · Annals of Internal Medicine · doi:10.7326/m15-2886 · Cited by 2,094
Zech, J. R., Badgeley, M. A., Liu, M., Costa, A. B., Titano, J. J., & Oermann, E. K. (2018) · PLOS Medicine · doi:10.1371/journal.pmed.1002683 · Cited by 1,719
Breast Cancer Screening for Women at Average Risk
Oeffinger, K. C., Fontham, E. T. H., Etzioni, R., Herzig, A. H., Michaelson, J. S., Shih, Y. T., Walter, L. C., Church, T. R., Flowers, C. R., LaMonte, S. J., et al. (2015) · JAMA · doi:10.1001/jama.2015.12783 · Cited by 1,706
Fan, L., Strasser-Weippl, K., Li, J.-J., Louis, J. S., Finkelstein, D., Yu, K.-D., Chen, W.-Q., Shao, Z., & Goss, P. (2014) · The Lancet Oncology · doi:10.1016/s1470-2045(13)70567-9 · Cited by 1,391
THE DIGITAL DATABASE FOR SCREENING MAMMOGRAPHY
Heath, M. D., Bowyer, K. W., Kopans, D. B., & Moore, R. M. (2007) · Cited by 1,017
Large scale deep learning for computer aided detection of mammographic lesions
Kooi, T., Litjens, G., van Ginneken, B., Gubern-Mérida, A., Sánchez, C. I., Mann, R., den Heeten, A., & Karssemeijer, N. (2017) · Medical Image Analysis · doi:10.1016/j.media.2016.07.007 · Cited by 965
A Deep Learning Mammography-based Model for Improved Breast Cancer Risk Prediction
Yala, A., Lehman, C., Schuster, T., Portnoi, T., & Barzilay, R. (2019) · Radiology · doi:10.1148/radiol.2019182716 · Cited by 758
Freer, T. W., & Ulissey, M. J. (2001) · Radiology · doi:10.1148/radiol.2203001282 · Cited by 748
Deep Neural Networks Improve Radiologists’ Performance in Breast Cancer Screening
Wu, N., Phang, J., Park, J., Shen, Y., Huang, Z., Zorin, M., Jastrzȩbski, S., Févry, T., Katsnelson, J., Kim, E. S., et al. (2019) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2019.2945514 · Cited by 736
Detecting and classifying lesions in mammograms with Deep Learning
Ribli, D., Horváth, A., Unger, Z., Pollner, P., & Csabai, I. (2018) · Scientific Reports · doi:10.1038/s41598-018-22437-z · Cited by 696
Rodriguez-Ruiz, A., Lång, K., Gubern-Merida, A., Broeders, M., Gennaro, G., Clauser, P., Helbich, T. H., Chevalier, M., Tan, T., Mertelmeier, T., et al. (2019) · JNCI: Journal of the National Cancer Institute · doi:10.1093/jnci/djy222 · Cited by 692
Diagnostic Accuracy of Digital Screening Mammography With and Without Computer-Aided Detection
Lehman, C. D., Wellman, R. D., Buist, D. S. M., Kerlikowske, K., Tosteson, A. N. A., & Miglioretti, D. L. (2015) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2015.5231 · Cited by 678
Lehman, C. D., Arao, R. F., Sprague, B. L., Lee, J. M., Buist, D. S. M., Kerlikowske, K. M., Henderson, L. M., Onega, T., Tosteson, A. N. A., Rauscher, G. H., et al. (2016) · Radiology · doi:10.1148/radiol.2016161174 · Cited by 667
Influence of Computer-Aided Detection on Performance of Screening Mammography
Fenton, J. J., Taplin, S. H., Carney, P. A., Abraham, L., Sickles, E. A., D’Orsi, C. J., Berns, E. A., Cutter, G., Hendrick, R. E., Barlow, W. E., et al. (2007) · New England Journal of Medicine · doi:10.1056/nejmoa066099 · Cited by 619
Detection of Breast Cancer with Mammography: Effect of an Artificial Intelligence Support System
Rodríguez-Ruiz, A., Krupinski, E., Mordang, J.-J., Schilling, K., Heywang-Köbrunner, S. H., Sechopoulos, I., & Mann, R. M. (2019) · Radiology · doi:10.1148/radiol.2018181371 · Cited by 602
Kim, H.-E., Kim, H. H., Han, B., Kim, K., Han, K., Nam, H., Lee, E. H., & Kim, E. (2020) · The Lancet Digital Health · doi:10.1016/s2589-7500(20)30003-0 · Cited by 556
Kim, D. W., Jang, H. Y., Kim, K. W., Shin, Y., & Park, S. H. (2019) · Korean Journal of Radiology · doi:10.3348/kjr.2019.0025 · Cited by 459
Schaffter, T., Buist, D. S. M., Lee, C. I., Nikulin, Y., Ribli, D., Guan, Y., Lotter, W., Jie, Z., Du, H., Wang, S., et al. (2020) · JAMA Network Open · doi:10.1001/jamanetworkopen.2020.0265 · Cited by 409
Self-citationPerformance Benchmarks for Screening Mammography
Rosenberg, R. D., Yankaskas, B. C., Abraham, L., Sickles, E. A., Lehman, C. D., Geller, B. M., Carney, P. A., Kerlikowske, K. M., Buist, D. S. M., Weaver, D. L., et al. (2006) · Radiology · doi:10.1148/radiol.2411051504 · Cited by 336
Missed Breast Carcinoma: Pitfalls and Pearls
Majid, A. S., de Paredes, E. S., Doherty, R. D., Sharma, N. R., & Salvador, X. (2003) · Radiographics · doi:10.1148/rg.234025083 · Cited by 274
Screening for Breast Cancer in 2018—What Should We be Doing Today?
Seely, J. M., & Alhassan, T. (2018) · Current Oncology · doi:10.3747/co.25.3770 · Cited by 259
Conant, E. F., Toledano, A. Y., Periaswamy, S., Fotin, S. V., Go, J., Boatsman, J. E., & Hoffmeister, J. W. (2019) · Radiology Artificial Intelligence · doi:10.1148/ryai.2019180096 · Cited by 243
Evans, K. K., Birdwell, R. L., & Wolfe, J. M. (2013) · PLoS ONE · doi:10.1371/journal.pone.0064366 · Cited by 238
A Deep Learning Model to Triage Screening Mammograms: A Simulation Study
Yala, A., Schuster, T., Miles, R., Barzilay, R., & Lehman, C. (2019) · Radiology · doi:10.1148/radiol.2019182908 · Cited by 210
OPTIMAM Mammography Image Database: A Large-Scale Resource of Mammography Images and Clinical Data
Halling‐Brown, M., Warren, L. M., Ward, D., Lewis, E. B., Mackenzie, A., Wallis, M., Wilkinson, L., Given-Wilson, R., McAvinchey, R., & Young, K. C. (2020) · Radiology Artificial Intelligence · doi:10.1148/ryai.2020200103 · Cited by 199
Breast cancer tumor growth estimated through mammography screening data
Weedon‐Fekjær, H., Lindqvist, B. H., VATTEN, L. J., Aalen, O. O., & Tretli, S. (2008) · Breast Cancer Research · doi:10.1186/bcr2092 · Cited by 197
High-Resolution Breast Cancer Screening with Multi-View Deep Convolutional Neural NetworksPreprint
Geras, K. J., Wolfson, S., Shen, Y., Wu, N., Kim, S.-G., Kim, E. S., Heacock, L. C., Parikh, U., Moy, L. P., & Cho, K. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.07047 · Cited by 189
The growth law of primary breast cancer as inferred from mammography screening trials data
Hart, D. R., Shochat, E., & Agur, Z. (1998) · British Journal of Cancer · doi:10.1038/bjc.1998.503 · Cited by 176
A Multi-scale CNN and Curriculum Learning Strategy for Mammogram Classification
Lotter, W. E., Sorensen, G., & Cox, D. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-67558-9_20 · Cited by 155
Henriksen, E., Carlsen, J. F., Vejborg, I., Nielsen, M. B., & Lauridsen, C. A. (2018) · Acta Radiologica · doi:10.1177/0284185118770917 · Cited by 126
Digital Breast Tomosynthesis From Concept to Clinical Care
Kopans, D. B. (2014) · American Journal of Roentgenology · doi:10.2214/ajr.13.11520 · Cited by 115
Breast Density and Risk of Breast Cancer in Asian Women: A Meta-analysis of Observational Studies
Bae, J., & Kim, E. H. (2016) · Journal of Preventive Medicine and Public Health · doi:10.3961/jpmph.16.054 · Cited by 96
Ikeda, D. M., Birdwell, R. L., O′Shaughnessy, K. F., Brenner, R. J., & Sickles, E. A. (2003) · Radiology · doi:10.1148/radiol.2262011634 · Cited by 72
Park, S. H. (2018) · Radiology · doi:10.1148/radiol.2018182294 · Cited by 57
H, D. B. M. P., Marqusee, H., Koussa, M. E., & Atun, R. A. (2017) · Public Health · doi:10.1016/j.puhe.2017.07.020 · Cited by 51
Interpretation Time of Computer-aided Detection at Screening Mammography
Tchou, P. M., Haygood, T. M., Atkinson, E. N., Stephens, T. W., Davis, P., Arribas, E. M., Geiser, W. R., & Whitman, G. J. (2010) · Radiology · doi:10.1148/radiol.10092170 · Cited by 49
Missed and True Interval and Screen-detected Breast Cancers in a Population Based Screening Program
Hoff, S. R., Samset, J. H., Abrahamsen, A.-L., Vigeland, E., Klepp, O., & Hofvind, S. S. H. (2011) · Academic Radiology · doi:10.1016/j.acra.2010.11.014 · Cited by 47
The visibility of cancer on earlier mammograms in a population-based screening programme
Saarenmaa, I. E., Salminen, T. M., Geiger, U., Holli, K. A., Isola, J. J., Kärkkäinen, A., Pakkanen, J., Piironen, A., Salo, A., & Hakama, M. (1999) · European Journal of Cancer · doi:10.1016/s0959-8049(99)00103-3 · Cited by 38
Impact of prevalence and case distribution in lab-based diagnostic imaging studies
Gallas, B. D., Chen, W., Cole, E. B., Ochs, R. A., Petrick, N., Pisano, E. D., Sahiner, B., Samuelson, F. W., & Myers, K. J. (2019) · Journal of medical imaging · doi:10.1117/1.jmi.6.1.015501 · Cited by 15
Validation of a deep learning mammography model in a population with low screening ratesPreprint
Wu, K., Wu, E., Wu, Y., Tan, H., Sorensen, G., Wang, M., & Lotter, B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.00364 · Cited by 6
Cited by
Sorted by citations
Artificial intelligence and machine learning for medical imaging: A technology review
Montero, A. M. B., Javaid, U., Valdés, G., Nguyen, D. T. D., Desbordes, P., Macq, B. M. M., Willems, S., Vandewinckele, L., Holmström, M., Löfman, F., et al. (2021) · Physica Medica · doi:10.1016/j.ejmp.2021.04.016 · Cited by 510
Saber, A., Sakr, M., Abo-Seida, O. M., Keshk, A. E.-S., & Chen, H. (2021) · IEEE Access · doi:10.1109/access.2021.3079204 · Cited by 424
Dembrower, K., Crippa, A., Colón, E., Eklund, M., & Strand, F. (2023) · The Lancet Digital Health · doi:10.1016/s2589-7500(23)00153-x · Cited by 397
Large-scale pancreatic cancer detection via non-contrast CT and deep learning
Cao, K., Xia, Y., Yao, J., Han, X., Lambert, L., Zhang, T., Tang, W., Jin, G., Jiang, H., Fang, X., et al. (2023) · Nature Medicine · doi:10.1038/s41591-023-02640-w · Cited by 358
Freeman, K., Geppert, J., Stinton, C., Todkill, D., Johnson, S. A., Clarke, A., & Taylor‐Phillips, S. (2021) · BMJ · doi:10.1136/bmj.n1872 · Cited by 365
Metadata
- Authors
- Lotter, W., Diab, A. R., Haslam, B., Kim, J. G., Grisot, G., Wu, E., Wu, K., Onieva, J. O., Boyer, Y., Boxerman, J. L., Wang, M., Bandler, M., Vijayaraghavan, G. R., & Gregory Sorensen, A.
- Affiliations
- Stanford University (US)
- Brown University (US)
- Henan Provincial People's Hospital (CN)
- Medford Radiology Group (US)
- University of Massachusetts Chan Medical School (US)
- Corresponding author
- Lotter, William
- ORCID
- Venue
- Nature Medicine
- Published
- 2021-01-11
- Volume
- 27
- Issue
- 2
- Pages
- 244-249
- ISSN
- 1078-8956 (ISSN-L)
- 1546-170X
- PMC
- PMC9426656
- OpenAlex
- W2994949497
- Cited by
- 423
- References
- 50
- Citations by year
- 2026: 59 · 2025: 81 · 2024: 91 · 2023: 93 · 2022: 59 · 2021: 39
- FWCI
- 24.82
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Nature Medicine (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- arXiv (Cornell University) (submittedVersion)
- The Journal of the American Medical Association (JAMA) Network (American Medical Association) (submittedVersion)
- arXiv (Cornell University)
- Funders
- U.S. Department of Health & Human Services | NIH | National Cancer Institute (1R44CA240022, 1R37CA240403-01A1)
- National Science Foundation (1938387)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Digital Radiography and Breast Imaging
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
- Keywords
- breast cancer detection, mammography, digital breast tomosynthesis, annotation efficiency, multiple instance learning, interpretability, mammography screening, computer-aided detection
- MeSH terms
- Deep Learning, Adult, Breast, Breast Neoplasms, Female, Humans, Mammography, Middle Aged, Early Detection of Cancer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
