Improved Automated Detection of Diabetic Retinopathy on a Publicly Available Dataset Through Integration of Deep Learning
Open accessThe DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.
Abstract
PURPOSE: To compare performance of a deep-learning enhanced algorithm for automated detection of diabetic retinopathy (DR), to the previously published performance of that algorithm, the Iowa Detection Program (IDP)-without deep learning components-on the same publicly available set of fundus images and previously reported consensus reference standard set, by three US Board certified retinal specialists. METHODS: We used the previously reported consensus reference standard of referable DR (rDR), defined as International Clinical Classification of Diabetic Retinopathy moderate, severe nonproliferative (NPDR), proliferative DR, and/or macular edema (ME). Neither Messidor-2 images, nor the three retinal specialists setting the Messidor-2 reference standard were used for training IDx-DR version X2.1. Sensitivity, specificity, negative predictive value, area under the curve (AUC), and their confidence intervals (CIs) were calculated. RESULTS: Sensitivity was 96.8% (95% CI: 93.3%-98.8%), specificity was 87.0% (95% CI: 84.2%-89.4%), with 6/874 false negatives, resulting in a negative predictive value of 99.0% (95% CI: 97.8%-99.6%). No cases of severe NPDR, PDR, or ME were missed. The AUC was 0.980 (95% CI: 0.968-0.992). Sensitivity was not statistically different from published IDP sensitivity, which had a CI of 94.4% to 99.3%, but specificity was significantly better than the published IDP specificity CI of 55.7% to 63.0%. CONCLUSIONS: A deep-learning enhanced algorithm for the automated detection of DR, achieves significantly better performance than a previously reported, otherwise essentially identical, algorithm that does not employ deep learning. Deep learning enhanced algorithms have the potential to improve the efficiency of DR screening, and thereby to prevent visual loss and blindness from this devastating disease.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The first well-established device was IDx-DR, approved by the FDA in 2018 for its high diagnostic performance in clinical trials.104,142”
From Artificial intelligence in diabetes management: Advancements, opportunities, and challenges (2023) · cites this paper as [104]
In the citing paper
“In the screening and evaluation of diabetic complications, the screening of DR using retinal images was studied well in multi-ethnic cohorts, with comparable accuracy compared with professional graders (discussed in detail in the conclusion and perspective).103,104,105,106,107,108,109”
From Artificial intelligence in diabetes management: Advancements, opportunities, and challenges (2023) · cites this paper as [104]
In the citing paper
“the AI system exceeded all pre-specified superiority endpoints at sensitivity of 87.2% (>85%), specificity of 90.7% (>82.5%), and imageability rate of 96.1% for detecting the presence of more than mild DR”
From Artificial intelligence in diabetes management: Advancements, opportunities, and challenges (2023) · cites this paper as Abramoff et al.104
In the citing paper
“CNNs consistently demonstrate physician-level grading from fundus photographs [104][105][106][107] , which has led to a recent US FDA-cleared system 108 .”
From Deep learning-enabled medical computer vision (2021) · cites this paper as [106]
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
Fukushima, K. (1980) · Biological Cybernetics · doi:10.1007/bf00344251 · Cited by 5,759
Wilkinson, C. P., Ferris, F. L., Klein, R. E., Lee, P. P., Agardh, C. D., Davis, M., Dills, D., Kampik, A., Pararajasegaram, R., & Verdaguer, J. T. (2003) · Ophthalmology · doi:10.1016/s0161-6420(03)00475-5 · Cited by 3,743
Patz, A., Fine, S., Finkelstein, D., Prout, T., Aiello, L., Bradley, R., Briones, J. C., Myers, F., Bresnick, G., de Venecia, G., et al. (1978) · Ophthalmology · doi:10.1016/s0161-6420(78)35693-1 · Cited by 652
Statistics Notes: Diagnostic tests 3: receiver operating characteristic plots
Altman, D. G., & Bland, J. M. (1994) · BMJ · doi:10.1136/bmj.309.6948.188 · Cited by 599
Automated Analysis of Retinal Images for Detection of Referable Diabetic Retinopathy
Abràmoff, M. D., Folk, J. C., Han, D. P., Walker, J. D., Williams, D. F., Russell, S. R., Massin, P., Cochener, B., Gain, P., Tang, L., et al. (2013) · JAMA Ophthalmology · doi:10.1001/jamaophthalmol.2013.1743 · Cited by 463
Self-citationDREAM: Diabetic Retinopathy Analysis Using Machine Learning
Roychowdhury, S., Koozekanani, D. D., & Parhi, K. K. (2014) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2013.2294635 · Cited by 413
Automated Early Detection of Diabetic Retinopathy
Abràmoff, M. D., Reinhardt, J. M., Russell, S. R., Folk, J. C., Mahajan, V. B., Niemeijer, M., & Quellec, G. (2010) · Ophthalmology · doi:10.1016/j.ophtha.2010.03.046 · Cited by 287
Self-citationAn economic analysis of interventions for diabetes.
Klonoff, D. C., & Schwartz, D. M. (2000) · Diabetes Care · doi:10.2337/diacare.23.3.390 · Cited by 267
Ege, B. M., Hejlesen, O. K., Larsen, O. V., Møller, K., Jennings, B., Kerr, D., & Cavan, D. A. (2000) · Computer Methods and Programs in Biomedicine · doi:10.1016/s0169-2607(00)00065-1 · Cited by 214
Cited by
Items 1,001–1,001 of 1,001
ADRGS: an automatic diabetic retinopathy grading system through machine learning
Devi, Y. A. S., & Manjunathachari, K. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-024-20241-8 · Cited by 1
Metadata
- Authors
- Abràmoff, M. D., Lou, Y., Erginay, A., Clarida, W., Amelon, R., Folk, J. C., & Niemeijer, M.
- Affiliations
- University of Iowa Hospitals and Clinics (US)
- University of Iowa (US)
- Hôpital Lariboisière (FR)
- IDx LLC, Iowa City, Iowa, United States
- Corresponding author
- Abràmoff, Michael David
- Venue
- Investigative Opthalmology & Visual Science
- Published
- 2016-10-04
- Volume
- 57
- Issue
- 13
- Pages
- 5200
- ISSN
- 0146-0404 (ISSN-L)
- 1552-5783
- OpenAlex
- W2529153069
- Cited by
- 1,101
- References
- 31
- Citations by year
- 2026: 67 · 2025: 117 · 2024: 129 · 2023: 143 · 2022: 151 · 2021: 151 · 2020: 158 · 2019: 91 · 2018: 57 · 2017: 32 · 2016: 1
- FWCI
- 52.38
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Investigative Ophthalmology & Visual Science (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Retinal Imaging and Analysis
- Health Sciences › Medicine › Ophthalmology › Retinal Diseases and Treatments
- Health Sciences › Medicine › Ophthalmology › Ophthalmology and Visual Health Research
- Keywords
- diabetic retinopathy, automated detection, deep learning, fundus images, referable diabetic retinopathy, diabetic macular edema, proliferative diabetic retinopathy, non-proliferative diabetic retinopathy, diagnostic accuracy, retinal screening, Messidor-2
- MeSH terms
- Ophthalmologists, Algorithms, Automation, Diabetic Retinopathy, Diagnosis, Computer-Assisted, Diagnostic Techniques, Ophthalmological, Female, Follow-Up Studies, Humans, Male, Middle Aged, Retina, Retrospective Studies, ROC Curve, Neural Networks, Computer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.