The DOI matches its Crossref record; 60 of the 61 references it cites were found in a registry or an index.
International evaluation of an AI system for breast cancer screening
Open accessAbstract
Screening mammography aims to identify breast cancer at earlier stages of the disease, when treatment can be more successful 1. Despite the existence of screening programmes worldwide, the interpretation of mammograms is affected by high rates of false positives and false negatives 2. Here we present an artificial intelligence (AI) system that is capable of surpassing human experts in breast cancer prediction. To assess its performance in the clinical setting, we curated a large representative dataset from the UK and a large enriched dataset from the USA. We show an absolute reduction of 5.7% and 1.2% (USA and UK) in false positives and 9.4% and 2.7% in false negatives. We provide evidence of the ability of the system to generalize from the UK to the USA. In an independent study of six radiologists, the AI system outperformed all of the human readers: the area under the receiver operating characteristic curve (AUC-ROC) for the AI system was greater than the AUC-ROC for the average radiologist by an absolute margin of 11.5%. We ran a simulation in which the AI system participated in the double-reading process that is used in the UK, and found that the AI system maintained non-inferior performance and reduced the workload of the second reader by 88%. This robust assessment of the AI system paves the way for clinical trials to improve the accuracy and efficiency of breast cancer screening.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Its modular and adaptable design supports broad generalizability across diverse clinical domains beyond cardiovascular diagnoses, such as oncology, neurology, and infectious diseases [37,38,39].”
From Enhancing Clinician Trust in AI Diagnostics: A Dynamic Framework for Confidence Calibration and Transparency (2025) · cites this paper as [38]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“CNNs-trained to classify disease states-have been extensively tested across diseases, and benchmarked against physicians.”
From Deep learning-enabled medical computer vision (2021) · cites this paper as [156]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Their performance is typically on par with experts when both are tested on the same image classification task.”
From Deep learning-enabled medical computer vision (2021) · cites this paper as [156]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In dermatology, ophthalmology, and radiology, convolutional neural networks have reported performance comparable to specialists in selected controlled tasks, such as skin cancer classification, diabetic retinopathy detection, breast cancer screening, and lung nodule identification [9,12].”
From Artificial Intelligence in Clinical Decision-Making: A Comprehensive Review of Diagnostic, Prognostic, and Therapeutic Applications, Validation Gaps, Bias, and Deployment Challenges (2026) · cites this paper as [12]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Artificial intelligence (AI) is rapidly reshaping healthcare and the competencies expected of graduating medical students”
From What the AI era doctor should know: a scoping review of proposed artificial intelligence competencies for medical education (2026) · cites this paper as [6]
Experimental — accuracy not yet measured
11 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Bray, F., Ferlay, J., Soerjomataram, I., Siegel, R. L., Torre, L. A., & Jemal, A. (2018) · CA: A Cancer Journal for Clinicians · doi:10.3322/caac.21492 · Cited by 84,109
DeLong, E. R., DeLong, D. M., & Clarke-Pearson, D. L. (1988) · Biometrics · doi:10.2307/2531595 · Cited by 22,649
The meaning and use of the area under a receiver operating characteristic (ROC) curve.
Hanley, J. A., & McNeil, B. J. (1982) · Radiology · doi:10.1148/radiology.143.1.7063747 · Cited by 21,856
An Introduction to the Bootstrap
Efron, B., & Tibshirani, R. J. (1993) · doi:10.1007/978-1-4899-4541-9 · Cited by 21,230
Dermatologist-level classification of skin cancer with deep neural networks
Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017) · Nature · doi:10.1038/nature21056 · Cited by 14,362
High-performance medicine: the convergence of human and artificial intelligence
Topol, E. J. (2019) · Nature Medicine · doi:10.1038/s41591-018-0300-7 · Cited by 10,450
Gulshan, V., Peng, L., Coram, M., Stumpe, M. C., Wu, D., Narayanaswamy, A., Venugopalan, S., Widner, K., Madams, T., Cuadros, J., et al. (2016) · JAMA · doi:10.1001/jama.2016.17216 · Cited by 7,697
Self-citationClinically applicable deep learning for diagnosis and referral in retinal disease
De Fauw, J., Ledsam, J. R., Romera-Paredes, B., Nikolov, S., Tomasev, N., Blackwell, S., Askham, H., Glorot, X., O’Donoghue, B., Visentin, D., et al. (2018) · Nature Medicine · doi:10.1038/s41591-018-0107-6 · Cited by 2,639
Self-citationArdila, D., Kiraly, A. P., Bharadwaj, S., Choi, B., Reicher, J. J., Peng, L., Tse, D., Etemadi, M., Ye, W., Corrado, G., et al. (2019) · Nature Medicine · doi:10.1038/s41591-019-0447-x · Cited by 2,167
Self-citationScreening for breast cancer: U.S. Preventive Services Task Force recommendation statement
Siu, A. L., & on behalf of the U.S. Preventive Services Task Force (2016) · Annals of Internal Medicine · doi:10.7326/m15-2886 · Cited by 2,094
Research works printed but not linked: 1
- Chihara, L. M., Hesterberg, T. C. & Dobrow, R. P. Mathematical Statistics with Resampling and R & Probability with Applications and R Set (Wiley, 2014).
Cited by
Rajpurkar, P., Chen, E., Banerjee, O., & Topol, E. J. (2022) · Nature Medicine · doi:10.1038/s41591-021-01614-0 · Cited by 3,212
“AI systems have achieved considerable improvements in accuracy for radiology tasks, including mammography interpretation 21,22 , cardiac function assessment 23,24 and lung cancer screening 25 , tackling not only diagnosis but also risk prediction and treatment 26 .”
BackgroundRevolutionizing healthcare: the role of artificial intelligence in clinical practice
Alowais, S. A., Alghamdi, S. S., Alsuhebany, N., Alqahtani, T., Alshaya, A. I., Almohareb, S. N., Aldairem, A., Alrashed, M., Bin Saleh, K., Badreldin, H. A., et al. (2023) · BMC Medical Education · doi:10.1186/s12909-023-04698-z · Cited by 3,146
“This study showed that utilizing an AI system to interpret mammograms had an absolute reduction in false positives and false negatives by 5.7% and 9.4%, respectively [11].”
MethodInfluentialData-efficient and weakly supervised computational pathology on whole-slide images
Lu, M. Y., Williamson, D. F. K., Chen, T. Y., Chen, R. J., Barbieri, M., & Mahmood, F. (2021) · Nature Biomedical Engineering · doi:10.1038/s41551-020-00682-w · Cited by 2,282
Secure, privacy-preserving and federated machine learning in medical imaging
Kaissis, G. A., Makowski, M. R., Rückert, D., & Braren, R. F. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-0186-1 · Cited by 1,676
Liu, X., Cruz Rivera, S., Moher, D., Calvert, M. J., Denniston, A. K., The SPIRIT-AI and CONSORT-AI Working Group, SPIRIT-AI and CONSORT-AI Steering Group, Chan, A.-W., Darzi, A., Holmes, C., et al. (2020) · Nature Medicine · doi:10.1038/s41591-020-1034-x · Cited by 1,421
Self-citation
Metadata
- Authors
- McKinney, S. M., Sieniek, M., Godbole, V., Godwin, J., Antropova, N., Ashrafian, H., Back, T., Chesus, M., Corrado, G. S., Darzi, A., Etemadi, M., Garcia-Vicente, F., Gilbert, F. J., Halling-Brown, M., Hassabis, D., Jansen, S., Karthikesalingam, A., Kelly, C. J., King, D., Ledsam, J. R., Melnick, D., Mostofi, H., Peng, L., Reicher, J. J., Romera-Paredes, B., Sidebottom, R., Suleyman, M., Tse, D., Young, K. C., De Fauw, J., & Shetty, S.
- Affiliations
- Google (United States) (US)
- Google DeepMind (United Kingdom) (GB)
- Imperial College London (GB)
- Cancer Research UK (GB)
- Northwestern Medicine (US)
- University of Cambridge (GB)
- Verily Life Sciences LLC (United States) (US)
- Google (United Kingdom) (GB)
- Stanford Health Care (US)
- Royal Marsden Hospital (GB)
- Royal Surrey County Hospital (GB)
- Corresponding author
- McKinney, Scott Mayer
- Tse, Daniel
- Shetty, Shravya
- ORCID
- McKinney, Scott Mayer
- Sieniek, Marcin
- Godwin, Jonathan
- Ashrafian, Hutan
- Back, Trevor
- Corrado, Greg S.
- Darzi, Ara
- Etemadi, Mozziyar
- Garcia-Vicente, Florencia
- Gilbert, Fiona J.
- Hassabis, Demis
- Jansen, Sunny
- Karthikesalingam, Alan
- Kelly, Christopher J.
- King, Dominic
- Ledsam, Joseph R.
- Melnick, David
- Peng, Lily
- Reicher, Joshua Jay
- Sidebottom, Richard
- Suleyman, Mustafa
- Tse, Daniel
- Young, Kenneth C.
- Shetty, Shravya
- Venue
- Nature
- Published
- 2020-01-01
- Volume
- 577
- Issue
- 7788
- Pages
- 89-94
- ISSN
- 0028-0836 (ISSN-L)
- 1476-4687
- OpenAlex
- W2998175747
- Semantic Scholar
- acb0b04bf282a5a04bd2ca050d8be3dec78a4668
- Cited by
- 3,287
- References
- 61
- Citations by year
- 2026: 499 · 2025: 617 · 2024: 521 · 2023: 450 · 2022: 449 · 2021: 464 · 2020: 281
- FWCI
- 161.55
- Type
- journal article
- Language
- en
- Update notices
- addendum (2020-10-14)
- Open access
- green
- doi.org
- Licence
- www.springernature.com/gp/researchers/text-and-data-mining (tdm)
- www.springernature.com/gp/researchers/text-and-data-mining (vor)
- Locations
- Nature (publishedVersion)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Spiral (Imperial College London) (submittedVersion)
- Apollo (University of Cambridge) (submittedVersion)
- Apollo
- Funders
- Cancer Research UK (28396, 26884, 17321)
- National Institute for Health and Care Research (NF-SI-0515-10067, NF-SI-0510-10186, NF-SI-0617-10038, 09/22/182)
- Engineering and Physical Sciences Research Council (EP/N014588/1)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Digital Radiography and Breast Imaging
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- breast cancer screening, mammography screening, artificial intelligence, computer-aided detection, false positives, false-negative results, radiologist performance comparison, double reading, radiologist workload reduction, breast cancer detection
- MeSH terms
- Artificial Intelligence, Breast Neoplasms, Female, United Kingdom, Humans, Mammography, United States, Reproducibility of Results, Early Detection of Cancer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: International evaluation of an AI system for breast cancer screening. https://app.cochat.ai/papers/10.1038/s41586-019-1799-6
- The paper
- McKinney, S. M., Sieniek, M., Godbole, V., Godwin, J., Antropova, N., Ashrafian, H., Back, T., Chesus, M., Corrado, G. S., Darzi, A., Etemadi, M., Garcia-Vicente, F., Gilbert, F. J., Halling-Brown, M., Hassabis, D., Jansen, S., Karthikesalingam, A., Kelly, C. J., King, D., . . . Shetty, S. (2020). International evaluation of an AI system for breast cancer screening. Nature. https://doi.org/10.1038/s41586-019-1799-6
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.