Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis
Open accessThe DOI matches its Crossref record; 143 of the 143 references it cites were found in a registry or an index; 36 of the 128 sources its claims cite were checked for support.
Abstract
Deep learning (DL) has the potential to transform medical diagnostics. However, the diagnostic accuracy of DL is uncertain. Our aim was to evaluate the diagnostic accuracy of DL algorithms to identify pathology in medical imaging. Searches were conducted in Medline and EMBASE up to January 2020. We identified 11,921 studies, of which 503 were included in the systematic review. Eighty-two studies in ophthalmology, 82 in breast disease and 115 in respiratory disease were included for meta-analysis. Two hundred twenty-four studies in other specialities were included for qualitative review. Peer-reviewed studies that reported on the diagnostic accuracy of DL algorithms to identify pathology using medical imaging were included. Primary outcomes were measures of diagnostic accuracy, study design and reporting standards in the literature. Estimates were pooled using random-effects meta-analysis. In ophthalmology, AUC’s ranged between 0.933 and 1 for diagnosing diabetic retinopathy, age-related macular degeneration and glaucoma on retinal fundus photographs and optical coherence tomography. In respiratory imaging, AUC’s ranged between 0.864 and 0.937 for diagnosing lung nodules or lung cancer on chest X-ray or CT scan. For breast imaging, AUC’s ranged between 0.868 and 0.909 for diagnosing breast cancer on mammogram, ultrasound, MRI and digital breast tomosynthesis. Heterogeneity was high between studies and extensive variation in methodology, terminology and outcome measures was noted. This can lead to an overestimation of the diagnostic accuracy of DL algorithms on medical imaging. There is an immediate need for the development of artificial intelligence-specific EQUATOR guidelines, particularly STARD, in order to provide guidance around key issues in this field.
Overview
CoChat Summary
TL;DR
This systematic review and meta-analysis found that deep learning algorithms generally achieve high diagnostic accuracy in medical imaging, with pooled AUCs ranging from 0.864 to 1 across ophthalmology, respiratory, and breast imaging. However, high heterogeneity and widespread methodological/reporting deficiencies mean these estimates are uncertain and may overestimate true accuracy.
Key findings
- Across 503 included studies, pooled AUCs were high: ophthalmology 0.933–1 for diabetic retinopathy, age-related macular degeneration, and glaucoma on retinal fundus photographs or OCT; respiratory 0.864–0.937 for lung nodules or lung cancer on chest X-ray or CT; breast 0.868–0.909 for breast cancer on mammogram, ultrasound, MRI, or digital breast tomosynthesis.
- Heterogeneity was high between studies, with many I² values above 75% and often near 99% (e.g., I² 99.9% for diabetic retinopathy on retinal fundus photographs; 99.6% for tuberculosis on X-ray).
- QUADAS-2 assessment classified a majority as high or unclear risk: patient selection 59/82 (72%) ophthalmic, 89/115 (77%) respiratory, 62/82 (76%) breast; flow/timing 66/82 (80%), 93/115 (81%), 70/82 (85%); reference standard applicability concerns 60/82 (73%), 104/115 (90%), 78/82 (95%).
- Only 8 ophthalmology, 10 respiratory, and 6 breast studies mentioned STARD-2015 adherence or included a STARD flow diagram; no studies provided a prespecified sample size calculation, and few used prospective data (8 ophthalmology, 2 respiratory, 0 breast) or external validation (29 ophthalmology, 13 respiratory, 8 breast).
Methods
- Systematic review and meta-analysis conducted per PRISMA-DTA; Medline and EMBASE searched up to 3 January 2020; peer-reviewed English studies reporting diagnostic accuracy of deep learning algorithms on medical imaging in human populations were included.
- Searches identified 11,921 abstracts; 503 studies fulfilled inclusion criteria for the systematic review, with meta-analysis in ophthalmology (82 studies), respiratory medicine (115), and breast cancer (82), plus qualitative synthesis of 224 studies in other specialities.
- Data were extracted independently by two investigators; risk of bias/applicability assessed with QUADAS-2 by three investigators; bivariate models and DerSimonian and Laird random-effects meta-analysis were used, with I² for heterogeneity and SROC where raw data were available; analyses used Stata version 15; PROSPERO CRD42020167503.
Limitations
- The paper states that many included studies had methodological deficiencies or poor reporting, so the pooled diagnostic performance estimates are uncertain and may represent an overestimation of true accuracy; it also states that no quality assessment for transparency of reporting was conducted because STARD-2015 was not designed for deep learning studies.
- The paper states that classical statistical comparison between imaging modalities was not possible, that each modality could not be separated into subsets to break down heterogeneity, and that findings may not generalise to other forms of medical imaging or AI studies.
- Evident from the text: heterogeneity was high across studies (I² often >75%, frequently >99%), most studies were at high or unclear risk of bias on QUADAS-2, and few studies were prospective, externally validated, or had prespecified sample size calculations.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures


Verification
Claims the paper makes
11 Valid · 3 Not valid · 32 Not checked · 165 Own results and interpretations
None of the sources cited for this claim have been checked yet.
In the paper
“With increasing demands upon existing radiologists (especially in low-to-middle-income countries)6–8, there is a growing need for diagnosis automation.”
§ Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis > Introduction
What the sources say
- Could not verifycites [6] Wahl, B., Cossy-Gantner, A., Germann, S. & Schwalbe, N. R. Artificial intelligence (AI) and global health: how can AI contribute to health in resource-poor settings? BMJ Glob. Health 3, e000798–e000798 (2018). sourceExperimental — accuracy not yet measured
The citation is attached to the claim about increasing demands on radiologists, especially in low- and middle-income countries. The passages in “Unanswered questions and remaining challenges” discuss AI-assisted diagnosis in resource-poor settings, but do not address radiologists’ workloads. Searches for radiologists and healthcare-worker shortages found no passage establishing that point. (LLM, conf=0.72)
- Could not verifycites [7] Zhang, L., Wang, H., Li, Q., Zhao, M.-H. & Zhan, Q.-M. Big data and medical research in China. BMJ 360, j5910 (2018). sourceExperimental — accuracy not yet measured
The citation is attached to the claim about increasing demands on radiologists, especially in low-to-middle-income countries. The “Medical Research in China” passage discusses machine learning replacing some radiologists’ work in China, but does not address radiologist workload or demand, or the specified country group (page not available). Searches for radiologist demand, workload, and shortages found no passage addressing that point. (LLM, conf=0.76)
- Not checked yetcites [8] Nakajima, Y., Yamada, K., Imamura, K. & Kobayashi, K. Radiologist supply and workload: international comparison. Radiat. Med. 26, 455–465 (2008). source
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Twenty-five studies17,28,29,35,37,39,40,44–61 compared algorithm performance against healthcare professionals.”
§ Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis > Results > Ophthalmology imaging
What the sources say
- Supportedcites [17] Gulshan, V. et al. Performance of a deep-learning algorithm vs manual grading for detecting diabetic retinopathy in India. JAMA Ophthalmol 137, 987–993 (2019). source
The Discussion reports that this study compared an automated algorithm with retinal specialists and trained graders in a prospective clinical setting. It supports citing this paper as one of the studies making that comparison; the count of 25 describes the citing paper’s group of studies, not a finding attributed to this individual source. (LLM, conf=0.93)
- Not checked yetcites [28] Gulshan, V. et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA 316, 2402–2410 (2016). source
- Supportedcites [29] Hwang, D. K. et al. Artificial intelligence-based decision-making for age-related macular degeneration. Theranostics 9, 232–245 (2019). source
The source is one of the studies cited for the comparison, not the source of the citing authors’ count of 25. In “Verification of final AI models and comparison between reviewers and AI,” it describes comparing its AI models with retinal specialists and medical students; the comparison section reports their relative performance. (LLM, conf=0.93)
- Not checked yetcites [35] Liu, S. et al. A deep learning-based algorithm identifies glaucomatous discs using monoscopic fundus photographs. Ophthalmol. Glaucoma 1, 15–22 (2018). source
- Supportedcites [37] Phene, S. et al. Deep learning and glaucoma specialists: the relative importance of optic disc features to predict glaucoma referral in fundus photographs. Ophthalmology 126, 1627–1639 (2019). source
The Results report that 10 eye-care providers graded the same images used to evaluate the algorithm, and their performance was plotted for comparison. This paper therefore qualifies as one of the studies identified in the citing sentence; the total of 25 is the citing paper’s count, not a finding attributed to this individual study. (LLM, conf=0.93)
- Supportedcites [39] Raumviboonsuk, P. et al. Deep learning versus human graders for classifying diabetic retinopathy severity in a nationwide screening program. npj Digital Med. 2, 25 (2019). source
The Abstract describes a diabetic-retinopathy study that compared a deep learning algorithm with human graders, so this paper qualifies as one of the cited ophthalmology studies. The count of 25 is the citing document’s count across its listed studies, not a count attributed to this individual source. (LLM, conf=0.92)
- Not checked yetcites [40] Sayres, R. et al. Using a deep learning algorithm and integrated gradients explanation to assist grading for diabetic retinopathy. Ophthalmology 126, 552–564 (2019). source
- Not checked yetcites [44] Van Grinsven, M. J., van Ginneken, B., Hoyng, C. B., Theelen, T. & Sánchez, C. I. Fast convolutional neural network training using selective data sampling: application to hemorrhage detection in color fundus images. IEEE Trans. Med. Imaging 35, 1273–1284 (2016). source
- Supportedcites [45] Rogers, T. W. et al. Evaluation of an AI system for the automated detection of glaucoma from stereoscopic optic disc photographs: the European Optic Disc Assessment Study. Eye 33, 1791–1797 (2019). source
The Methods state that this study compared the Pegasus AI system’s performance with that of ophthalmologists and optometrists. The Results report that comparison, so this paper is one of the studies identified by the citing sentence; the sentence does not attribute the total of 25 studies to this paper alone. (LLM, conf=0.94)
- Not checked yetcites [46] Al-Aswad, L. A. et al. Evaluation of a deep learning system for identifying glaucomatous optic neuropathy based on color fundus photographs. J. Glaucoma 28, 1029–1034 (2019). source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Only eight studies14–21 used prospectively collected data”
§ Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis > Results > Ophthalmology imaging
What the sources say
- Supportedcites [14] Abràmoff, M. D., Lavin, P. T., Birch, M., Shah, N. & Folk, J. C. Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices. npj Digital Med. 1, 39 (2018). source
The Methods (Study design; page not available) state that 900 participants were prospectively enrolled in this ophthalmology study. That supports citing this trial as an example of a study using prospectively collected data; this source alone does not establish the total of eight studies. (LLM, conf=0.91)
- Not checked yetcites [15] Bellemo, V. et al. Artificial intelligence using deep learning to screen for referable and vision-threatening diabetic retinopathy in Africa: a clinical validation study. Lancet Digital Health 1, e35–e44 (2019). source
- Supportedcites [16] Christopher, M. et al. Performance of deep learning architectures and transfer learning for detecting glaucomatous optic neuropathy in fundus photographs. Sci. Rep. 8, 16685 (2018). source
The Methods > Data Collection passage says participants came from two prospective longitudinal studies that periodically collected fundus images, supporting this paper's inclusion among studies using prospectively collected data. The count of eight is the citing paper's count across citations 14–21, not a finding attributed to this individual source. (LLM, conf=0.90)
- Supportedcites [17] Gulshan, V. et al. Performance of a deep-learning algorithm vs manual grading for detecting diabetic retinopathy in India. JAMA Ophthalmol 137, 987–993 (2019). source
The Study Population section describes this as a prospective study and gives the enrollment and data-collection periods at two eye care centers. As one of the studies cited in the sentence, it supports the claim that it used prospectively collected data; the count of eight is the citing paper’s own tally. (LLM, conf=0.92)
- Could not verifycites [18] Keel, S., Wu, J., Lee, P. Y., Scheetz, J. & He, M. Visualizing deep learning models for the detection of referable diabetic retinopathy and glaucoma. JAMA Ophthalmol. 137, 288–292 (2019). sourceExperimental — accuracy not yet measured
The Methods describe selecting images from existing validation data sets and call the study’s use of those images retrospective. They do not say whether the images were originally collected prospectively. This paper also cannot establish the citing sentence’s count of eight studies. (LLM, conf=0.76)
- Supportedcites [19] Sandhu, H. S. et al. Automated diagnosis and grading of diabetic retinopathy using optical coherence tomography. Investig. Ophthalmol. Vis. Sci. 59, 3155–3160 (2018). source
The Results section states that OCT scans were collected prospectively, confirming this paper as an example among the studies cited. This source alone does not establish the citing document’s total of eight studies. (LLM, conf=0.93)
- Not checked yetcites [20] Zheng, C. et al. Detecting glaucoma based on spectral domain optical coherence tomography imaging of peripapillary retinal nerve fiber layer: a comparison study between hand-crafted features and deep learning model. Graefes Arch. Clin. Exp. Ophthalmol. 258, 577–585 (2020). source
- Supportedcites [21] Kanagasingam, Y. et al. Evaluation of artificial intelligence-based grading of diabetic retinopathy in primary care. JAMA Netw. Open 1, e182665–e182665 (2018). source
The Methods section (‘Deployment in a Primary Care Practice’; page unavailable) describes inviting patients during a six-month deployment and acquiring their retinal images as they participated. This supports citing the paper as an example of a study using prospectively collected data; the citation does not ask this individual study to establish the total of eight. (LLM, conf=0.88)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Only eight studies in ophthalmology imaging14,21,32,33,43,55,108,109, ten studies in respiratory imaging64,66,70,72,75,79,82,87,89,110 and six studies in breast imaging62,91,97,104,106,111 mentioned adherence to the STARD-2015 guidelines or had a STARD flow diagram in the manuscript.”
§ Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis > Results > Variation of reporting
What the sources say
- Partly supportedcites [14] Abràmoff, M. D., Lavin, P. T., Birch, M., Shah, N. & Folk, J. C. Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices. npj Digital Med. 1, 39 (2018). sourceExperimental — accuracy not yet measured
The Results (Study population; page unavailable) identify this as an ophthalmology imaging study and refer to Fig. 1 when accounting for enrolled, completed, and analyzable participants. Our copy does not identify that figure as a STARD flow diagram or mention adherence to STARD-2015. This single study also cannot establish the citing paper’s count of eight studies. (LLM, conf=0.72)
- Supportedcites [21] Kanagasingam, Y. et al. Evaluation of artificial intelligence-based grading of diabetic retinopathy in primary care. JAMA Netw. Open 1, e182665–e182665 (2018). source
The Methods section states that the trial was conducted according to the Standards for Reporting of Diagnostic Accuracy (STARD) reporting guideline. The paper is a diabetic-retinopathy imaging study, so it supports its inclusion as an example among the ophthalmology studies; it is not cited as the source of the total count of eight. (LLM, conf=0.91)
- Could not verifycites [32] Li, F. et al. Automatic detection of diabetic retinopathy in retinal fundus photographs based on deep learning algorithm. Transl. Vis. Sci. Technol. 8, 4 (2019). sourceExperimental — accuracy not yet measured
The Methods section describes a flowchart of the algorithm’s image-processing steps (Figure 1), but does not identify it as a STARD flow diagram. Searches for STARD-2015, diagnostic-accuracy reporting standards, and a patient-flow diagram found no passage in our copy establishing that this study met either condition. This paper alone also cannot establish the count of eight studies. (LLM, conf=0.75)
- Not checked yetcites [33] Li, Z. et al. An automated grading system for detection of vision-threatening referable diabetic retinopathy on the basis of color fundus photographs. Diabetes Care 41, 2509–2516 (2018). source
- Supportedcites [43] Verbraak, F. D. et al. Diagnostic accuracy of a device for the automated detection of diabetic retinopathy in a primary care setting. Diabetes Care 42, 651 (2019). source
The Results section refers to “STARD diagrams [Figs. 1 and 2],” and the Conclusions label both figures as STARD diagrams. This confirms that the cited ophthalmology study had a STARD flow diagram; the count of eight is the citing paper’s tally across studies. (LLM, conf=0.93)
- Partly supportedcites [55] Li, F. et al. Deep learning-based automated detection of retinal diseases using optical coherence tomography images. Biomed. Opt. Express 10, 6204–6226 (2019). sourceExperimental — accuracy not yet measured
Section 2.2 (p. 5) identifies this as an ophthalmology imaging study and says Fig. 2 displays its dataset selection and stratification process. Our copy does not identify that figure as a STARD flow diagram or mention adherence to STARD-2015. This individual paper also cannot establish the citing document’s count of eight studies. (LLM, conf=0.75)
- Not checked yetcites [108] Li, Z. et al. Efficacy of a deep learning system for detecting glaucomatous optic neuropathy based on color fundus photographs. Ophthalmology 125, 1199–1206 (2018). source
- Could not verifycites [109] Lin, H. et al. Diagnostic efficacy and therapeutic decision-making capacity of an artificial intelligence platform for childhood cataracts in eye clinics: a multicentre randomized controlled trial. EClinicalMedicine 9, 52–59 (2019). sourceExperimental — accuracy not yet measured
The Methods (Study Design and Participants; page unavailable) state that the trial followed CONSORT guidelines, while the Results label Fig. 1 a “Trial profile.” Searches for STARD-2015 and a STARD flow diagram found no passage establishing either criterion for this paper. This source also cannot establish the citing document’s count of eight studies. (LLM, conf=0.72)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“eight91–98 studies validated algorithms on external data”
§ Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis > Results > Breast imaging
What the sources say
- Not checked yetcites [91] Becker, A. S. et al. Deep learning in mammography: diagnostic accuracy of a multipurpose image analysis software in the detection of breast cancer. Investig. Radio. 52, 434–440 (2017). source
- Not checked yetcites [92] Ciritsis, A. et al. Automatic classification of ultrasound breast lesions using a deep convolutional neural network mimicking human decision-making. Eur. Radio. 29, 5458–5468 (2019). source
- Not checked yetcites [93] Cogan, T., Cogan, M. & Tamil, L. RAMS: remote and automatic mammogram screening. Comput. Biol. Med. 107, 18–29 (2019). source
- Supportedcites [94] McKinney, S. M. et al. International evaluation of an AI system for breast cancer screening. Nature 577, 89–94 (2020). source
The Generalisation across populations section (p. 4) says the authors trained the AI system only on UK data and evaluated it on a US test set, supporting this paper as one example of external-data validation. The eight-study count refers to the group of cited studies, not to eight studies conducted in this paper. (LLM, conf=0.92)
- Not checked yetcites [95] Peng, W., Mayorga, R. V. & Hussein, E. M. A. An automated confirmatory system for analysis of mammograms. Comput. Methods Prog. Biomed. 125, 134–144 (2016). source
- Supportedcites [96] Ribli, D., Horváth, A., Unger, Z., Pollner, P. & Csabai, I. Detecting and classifying lesions in mammograms with deep learning. Sci. Rep. 8, 4165 (2018). source
The Data section states that the model was trained on DDSM and Semmelweis University data and tested on the separate public INbreast dataset. That supports citing this paper as one study that validated an algorithm on external data; the paper does not itself establish the citing sentence’s total of eight studies. (LLM, conf=0.88)
- Not checked yetcites [97] Rodríguez-Ruiz, A. et al. Detection of Breast cancer with mammography: effect of an artificial intelligence support system. Radiology 290, 305–314 (2018). source
- Supportedcites [98] Rodriguez-Ruiz, A. et al. Stand-alone artificial intelligence for breast cancer detection in mammography: comparison with 101 radiologists. J. Natl Cancer Inst. 111, 916–922 (2019). source
The Artificial Intelligence System section (p. 3) says the mammograms used in this study had never been used to train, validate, or test the algorithms. This supports citing the paper as one of the studies that evaluated an algorithm on external data; the count of eight refers to the citing document’s group of studies, not to this paper alone. (LLM, conf=0.92)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
131 more claims · 3 with their cited sources checked
Create a free account to see all 211 claims and to verify the 92 references still open.
- Probably not a substantial claim. 38
- It may state two separate claims as one. 46
- The quote only partly states this claim. 1
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Without sufficient and diverse datasets, deep models trained on histopathology images from one hospital may fail to generalize well on data from a different hospital (out-ofdistribution) 2 .”
From Federated Learning and Differential Privacy for Medical Image Analysis (2021) · cites this paper as [2]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Use of STARD-AI can aid the comprehensive reporting of research that assesses AI diagnostic accuracy using either single or combined test data and can be applied across a broad range of diagnostic modalities.”
From The STARD-AI reporting guideline for diagnostic accuracy studies using artificial intelligence (2025) · cites this paper as [32]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“This review built upon the findings of a prior systematic review conducted by members of the STARD-AI team, which evaluated the diagnostic accuracy of deep learning in medical imaging and highlighted widespread variability in study design, methodological quality and reporting practices 32 .”
From The STARD-AI reporting guideline for diagnostic accuracy studies using artificial intelligence (2025) · cites this paper as [32]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
“Two hundred seventy-three studies were included for meta-analysis, 82 in ophthalmology, 115 in respiratory medicine and 82 in breast cancer (see Fig. 1).”
§ Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis > Results > Search and study selection
Reported 273 · recomputed 279
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis > Results > Ophthalmology imaging
“specificity was 0.907 (95% CI 0.907–1.066).”
§ Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis > Results > Search and study selection
“| RFP | ROP | | | | 0.96 | 0.913–1.008 | 99.5 | 0.907 | 0.749–1.066 | 99.8 |”
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Meta-analysis in clinical trials
DerSimonian, R., & Laird, N. (1986) · Controlled Clinical Trials · doi:10.1016/0197-2456(86)90046-2 · Cited by 39,552
A survey on deep learning in medical image analysis
Litjens, G., Kooi, T., Bejnordi, B. E., Setio, A. A. A., Ciompi, F., Ghafoorian, M., van der Laak, J. A. W. M., van Ginneken, B., & Sánchez, C. I. (2017) · Medical Image Analysis · doi:10.1016/j.media.2017.07.005 · Cited by 15,332
QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies
Whiting, P. F., Rutjes, A. W. S., Westwood, M. E., Mallett, S., Deeks, J. J., Reitsma, J. B., Leeflang, M. M. G., Sterne, J. A. C., Bossuyt, P. M. M., & the QUADAS-2 Group* (2011) · Annals of Internal Medicine · doi:10.7326/0003-4819-155-8-201110180-00009 · Cited by 14,199
CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials
the CONSORT Group, Schulz, K. F., Altman, D. G., & Moher, D. (2010) · BMC Medicine · doi:10.1186/1741-7015-8-18 · Cited by 13,371
High-performance medicine: the convergence of human and artificial intelligence
Topol, E. J. (2019) · Nature Medicine · doi:10.1038/s41591-018-0300-7 · Cited by 10,450
CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials
Schulz, K. F., Altman, D. G., Moher, D., & for the CONSORT Group (2010) · BMJ · doi:10.1136/bmj.c332 · Cited by 9,329
Gulshan, V., Peng, L., Coram, M., Stumpe, M. C., Wu, D., Narayanaswamy, A., Venugopalan, S., Widner, K., Madams, T., Cuadros, J., et al. (2016) · JAMA · doi:10.1001/jama.2016.17216 · Cited by 7,697
SPIRIT 2013 Statement: Defining Standard Protocol Items for Clinical Trials
Chan, A.-W., Tetzlaff, J. M., Altman, D. G., Laupacis, A., Gøtzsche, P. C., Krleža-Jerić, K., Hróbjartsson, A., Mann, H., Dickersin, K., Berlin, J. A., et al. (2013) · Annals of Internal Medicine · doi:10.7326/0003-4819-158-3-201302050-00583 · Cited by 7,363
A guide to deep learning in healthcare
Esteva, A., Robicquet, A., Ramsundar, B., Kuleshov, V., DePristo, M., Chou, K., Cui, C., Corrado, G., Thrun, S., & Dean, J. (2019) · Nature Medicine · doi:10.1038/s41591-018-0316-z · Cited by 5,312
Cited by
Large language models in medicine
Thirunavukarasu, A. J., Ting, D. S. J., Elangovan, K., Gutierrez, L., Tan, T. F., & Ting, D. S. W. (2023) · Nature Medicine · doi:10.1038/s41591-023-02448-8 · Cited by 3,944
Self-citationAntimicrobial resistance: a concise update
Ho, C. S., Wong, C. T. H., Aung, T. T., Lakshminarayanan, R., Mehta, J. S., Rauz, S., McNally, A., Kintses, B., Peacock, S. J., de la Fuente-Nunez, C., et al. (2025) · The Lancet Microbe · doi:10.1016/j.lanmic.2024.07.010 · Cited by 634
Self-citationThe STARD-AI reporting guideline for diagnostic accuracy studies using artificial intelligence
Sounderajah, V., Guni, A., Liu, X., Collins, G. S., Karthikesalingam, A., Markar, S. R., Golub, R. M., Denniston, A. K., Shetty, S., Moher, D., et al. (2025) · Nature Medicine · doi:10.1038/s41591-025-03953-8 · Cited by 592
Self-citationOpening the black box: the promise and limitations of explainable machine learning in cardiology
Petch, J., Di, S., & Nelson, W. (2022) · Canadian Journal of Cardiology · doi:10.1016/j.cjca.2021.09.004 · Cited by 552
Federated Learning and Differential Privacy for Medical Image Analysis
Adnan, M., Kalra, S., Cresswell, J. C., Taylor, G. W., & Tizhoosh, H. (2021) · doi:10.21203/rs.3.rs-1005694/v1 · Cited by 464
Metadata
- Authors
- Aggarwal, R., Sounderajah, V., Martin, G., Ting, D. S. W., Karthikesalingam, A., King, D., Ashrafian, H., & Darzi, A.
- Affiliations
- Imperial College London (GB)
- Singapore National Eye Center (SG)
- Corresponding author
- Ashrafian, Hutan
- ORCID
- Venue
- npj Digital Medicine
- Published
- 2021-04-07
- Volume
- 4
- Issue
- 1
- Pages
- 65-65
- ISSN
- 2398-6352 (ISSN-L)
- PMC
- PMC8027892
- OpenAlex
- W3150212014
- Cited by
- 946
- References
- 143
- Citations by year
- 2026: 188 · 2025: 259 · 2024: 196 · 2023: 166 · 2022: 104 · 2021: 31
- FWCI
- 10.26
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- npj Digital Medicine (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Spiral (Imperial College London) (submittedVersion, cc-by)
- Funders
- National Institute for Health and Care Research (NF-SI-0510-10186, NF-SI-0617-10038, CL-2019-21-503, NIHR-INF-0738)
- NIHR Imperial Biomedical Research Centre
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Retinal Imaging and Analysis
- Keywords
- deep learning, diagnostic accuracy, diagnostic imaging, artificial intelligence in healthcare, diabetic retinopathy, age-related macular degeneration, glaucoma, retinal fundus photographs, optical coherence tomography, pulmonary nodules, lung cancer, breast cancer, mammography, digital breast tomosynthesis, STARD, reporting guidelines
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis. https://app.cochat.ai/papers/10.1038/s41746-021-00438-z
- The paper
- Aggarwal, R., Sounderajah, V., Martin, G., Ting, D. S. W., Karthikesalingam, A., King, D., Ashrafian, H., & Darzi, A. (2021). Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis. npj Digital Medicine. https://doi.org/10.1038/s41746-021-00438-z
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.