Cited by 304Retinal Imaging and AnalysisRetinal Diseases and TreatmentsOphthalmology and Visual Health Research

Performance of a Deep-Learning Algorithm vs Manual Grading for Detecting Diabetic Retinopathy in India

Open access
Gulshan, Varun, Rajan, Renu P., Widner, Kasumi, Wu, Derek, Wubbels, Peter, Rhodes, Tyler, Whitehouse, Kira, Coram, Marc +5 more·2019·JAMA Ophthalmology

Abstract

IMPORTANCE: More than 60 million people in India have diabetes and are at risk for diabetic retinopathy (DR), a vision-threatening disease. Automated interpretation of retinal fundus photographs can help support and scale a robust screening program to detect DR. OBJECTIVE: To prospectively validate the performance of an automated DR system across 2 sites in India. DESIGN, SETTING, AND PARTICIPANTS: This prospective observational study was conducted at 2 eye care centers in India (Aravind Eye Hospital and Sankara Nethralaya) and included 3049 patients with diabetes. Data collection and patient enrollment took place between April 2016 and July 2016 at Aravind and May 2016 and April 2017 at Sankara Nethralaya. The model was trained and fixed in March 2016. INTERVENTIONS: Automated DR grading system compared with manual grading by 1 trained grader and 1 retina specialist from each site. Adjudication by a panel of 3 retinal specialists served as the reference standard in the cases of disagreement. MAIN OUTCOMES AND MEASURES: Sensitivity and specificity for moderate or worse DR or referable diabetic macula edema. RESULTS: Of 3049 patients, 1091 (35.8%) were women and the mean (SD) age for patients at Aravind and Sankara Nethralaya was 56.6 (9.0) years and 56.0 (10.0) years, respectively. For moderate or worse DR, the sensitivity and specificity for manual grading by individual nonadjudicator graders ranged from 73.4% to 89.8% and from 83.5% to 98.7%, respectively. The automated DR system's performance was equal to or exceeded manual grading, with an 88.9% sensitivity (95% CI, 85.8-91.5), 92.2% specificity (95% CI, 90.3-93.8), and an area under the curve of 0.963 on the data set from Aravind Eye Hospital and 92.1% sensitivity (95% CI, 90.1-93.8), 95.2% specificity (95% CI, 94.2-96.1), and an area under the curve of 0.980 on the data set from Sankara Nethralaya. CONCLUSIONS AND RELEVANCE: This study shows that the automated DR system generalizes to this population of Indian patients in a prospective setting and demonstrates the feasibility of using an automated DR grading system to expand screening programs.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1001/jamaophthalmol.2019.2004

Cites

23 of 23 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 1

  1. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z. Rethinking the inception architecture for computer vision. Paper presented at the 2016 IEEE Conference on Computer Vision and Pattern Recognition; June 27-30, 2016; Las Vegas, NV. https://ieeexplore.ieee.org/document/7780677. Accessed May 12, 2019.

Cited by

Sorted by citations

Authors
Gulshan, V., Rajan, R. P., Widner, K., Wu, D., Wubbels, P., Rhodes, T., Whitehouse, K., Coram, M., Corrado, G., Ramasamy, K., Raman, R., Peng, L., & Webster, D. R.
Affiliations
  • Google (United States) (US)
  • Aravind Eye Hospital (IN)
  • Verily Life Sciences LLC (United States) (US)
  • Sankara Nethralaya (IN)
Corresponding author
Peng, Lily
Venue
JAMA Ophthalmology
Published
2019-09-01
Volume
137
Issue
9
Pages
987
ISSN
2168-6165 (ISSN-L)
OpenAlex
W2949074827
Cited by
303
References
21
Citations by year
2026: 33 · 2025: 56 · 2024: 43 · 2023: 50 · 2022: 39 · 2021: 38 · 2020: 36 · 2019: 7 · 2018: 1
FWCI
19.80
Type
journal article
Language
en
Open access
Locations
  • JAMA Ophthalmology (publishedVersion)
  • PubMed Central (publishedVersion)
  • PubMed (publishedVersion)
Funders
Google
Topics
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Retinal Imaging and Analysis
  • Health Sciences › Medicine › Ophthalmology › Retinal Diseases and Treatments
  • Health Sciences › Medicine › Ophthalmology › Ophthalmology and Visual Health Research
Keywords
diabetic retinopathy, deep learning, retinal fundus photographs, diabetic macular edema, manual grading, sensitivity and specificity, retinal screening, diabetes mellitus, India, model validation, resource-limited settings, Aravind Eye Hospital

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review