An Automated Grading System for Detection of Vision-Threatening Referable Diabetic Retinopathy on the Basis of Color Fundus Photographs
The DOI matches its Crossref record; 40 of the 40 references it cites were found in a registry or an index.
Abstract
OBJECTIVE: The goal of this study was to describe the development and validation of an artificial intelligence–based, deep learning algorithm (DLA) for the detection of referable diabetic retinopathy (DR). RESEARCH DESIGN AND METHODS: A DLA using a convolutional neural network was developed for automated detection of vision-threatening referable DR (preproliferative DR or worse, diabetic macular edema, or both). The DLA was tested by using a set of 106,244 nonstereoscopic retinal images. A panel of ophthalmologists graded DR severity in retinal photographs included in the development and internal validation data sets (n = 71,043); a reference standard grading was assigned once three graders achieved consistent grading outcomes. For external validation, we tested our DLA using 35,201 images of 14,520 eyes (904 eyes with any DR; 401 eyes with vision-threatening referable DR) from population-based cohorts of Malays, Caucasian Australians, and Indigenous Australians. RESULTS: Among the 71,043 retinal images in the training and validation data sets, 12,329 showed vision-threatening referable DR. In the internal validation data set, the area under the curve (AUC), sensitivity, and specificity of the DLA for vision-threatening referable DR were 0.989, 97.0%, and 91.4%, respectively. Testing against the independent, multiethnic data set achieved an AUC, sensitivity, and specificity of 0.955, 92.5%, and 98.5%, respectively. Among false-positive cases, 85.6% were due to a misclassification of mild or moderate DR. Undetected intraretinal microvascular abnormalities accounted for 77.3% of all false-negative cases. CONCLUSIONS: This artificial intelligence–based DLA can be used with high accuracy in the detection of vision-threatening referable DR in retinal images. This technology offers potential to increase the efficiency and accessibility of DR screening programs.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The classification based on fundus images for referable DR followed the guidelines of the National Health Service (NHS) diabetic eye screening (Supplementary Table 5)40.”
From Economic evaluation for medical artificial intelligence: accuracy vs. cost-effectiveness in a diabetic retinopathy screening case (2024) · cites this paper as [40]
In the citing paper
“We employed a validated AI model for automatic DR screening using fundus images from the Lifeline Express Program and evaluated the AI’s performance against manual grading results40.”
From Economic evaluation for medical artificial intelligence: accuracy vs. cost-effectiveness in a diabetic retinopathy screening case (2024) · cites this paper as [40]
In the citing paper
“It should be noted that the specificity of the AI model in this real-world dataset was lower than the reported performance in its previous validation as well as other reported AI models40.”
From Economic evaluation for medical artificial intelligence: accuracy vs. cost-effectiveness in a diabetic retinopathy screening case (2024) · cites this paper as [40]
In the citing paper
“Li et al. [43] developed a deep CNN-based DR screening system that achieved performance comparable to or exceeding that of professional ophthalmologists across real-world datasets from multiple countries.”
From Artificial intelligence-powered prediction of diabetic complications: from clinical data to molecular omics (2026) · cites this paper as [43]
In the citing paper
“Only eight studies in ophthalmology imaging14,21,32,33,43,55,108,109, ten studies in respiratory imaging64,66,70,72,75,79,82,87,89,110 and six studies in breast imaging62,91,97,104,106,111 mentioned adherence to the STARD-2015 guidelines or had a STARD flow diagram in the manuscript.”
From Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis (2021) · cites this paper as [33]
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Rethinking the Inception Architecture for Computer Vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.308 · Cited by 32,138
Dermatologist-level classification of skin cancer with deep neural networks
Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017) · Nature · doi:10.1038/nature21056 · Cited by 14,362 · cited 1×
“These deep learning techniques have been applied in highly image-driven medical specialties, including dermatology and radiology, with promising results: the techniques have identified diseases as accurately as or more accurately than board-certified specialists (15,16).”
BackgroundInternational Diabetes Federation 2017
Carracher, A. M., Marathe, P. H., & Close, K. L. (2018) · Journal of Diabetes · doi:10.1111/1753-0407.12644 · Cited by 13,418
Gulshan, V., Peng, L., Coram, M., Stumpe, M. C., Wu, D., Narayanaswamy, A., Venugopalan, S., Widner, K., Madams, T., Cuadros, J., et al. (2016) · JAMA · doi:10.1001/jama.2016.17216 · Cited by 7,697
(2007) · Developments in Ophthalmology · doi:10.1159/isbn.978-3-318-01439-6 · Cited by 5,583
Global prevalence and major risk factors of diabetic retinopathy
Yau, J. W. Y., Rogers, S. L., Kawasaki, R., Lamoureux, E. L., Kowalski, J. W., Bek, T., Chen, S.-J., Dekker, J. M., Fletcher, A., Grauslund, J., et al. (2012) · Diabetes Care · doi:10.2337/dc11-1909 · Cited by 4,985 · cited 2×
“6%, corresponding to nearly 100 million people worldwide (1).”
BackgroundSelf-citationZhou, B., Lu, Y., Hajifathalian, K., Bentham, J., Di Cesare, M., Danaei, G., Bixby, H., Cowan, M., Ali, M., Taddei, C., et al. (2016) · The Lancet · doi:10.1016/s0140-6736(16)00618-8 · Cited by 4,311
Wilkinson, C. P., Ferris, F. L., Klein, R. E., Lee, P. P., Agardh, C. D., Davis, M., Dills, D., Kampik, A., Pararajasegaram, R., & Verdaguer, J. T. (2003) · Ophthalmology · doi:10.1016/s0161-6420(03)00475-5 · Cited by 3,743
Self-citationCheung, N., Mitchell, P., & Wong, T. Y. (2010) · The Lancet · doi:10.1016/s0140-6736(09)62124-3 · Cited by 3,288
Self-citation
Cited by
Artificial Intelligence and Deep Learning in Ophthalmology
Wang, Z., Keane, P. A., Chiang, M., Cheung, C. Y., Wong, T. Y., & Ting, D. S. W. (2022) · Artificial Intelligence in Medicine · doi:10.1007/978-3-030-64573-1_200 · Cited by 1,524
Self-citationArtificial intelligence and deep learning in ophthalmology
Ting, D. S. W., Pasquale, L. R., Peng, L., Campbell, J. P., Lee, A. Y., Raman, R., Tan, G. S. W., Schmetterer, L., Keane, P. A., & Wong, T. Y. (2019) · British Journal of Ophthalmology · doi:10.1136/bjophthalmol-2018-313173 · Cited by 1,554
Self-citationDiagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis
Aggarwal, R., Sounderajah, V., Martin, G., Ting, D. S. W., Karthikesalingam, A., King, D., Ashrafian, H., & Darzi, A. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00438-z · Cited by 946
Self-citationDigital technology, tele-medicine and artificial intelligence in ophthalmology: A global perspective
Li, J.-P. O., Liu, H., Ting, D. S. J., Jeon, S., Chan, R. V. P., Kim, J. E., Sim, D. A., Thomas, P. B. M., Lin, H., Chen, Y., et al. (2021) · Progress in Retinal and Eye Research · doi:10.1016/j.preteyeres.2020.100900 · Cited by 626
Self-citationA deep learning system for detecting diabetic retinopathy across the disease spectrum
Dai, L., Wu, L., Li, H., Cai, C., Wu, Q., Kong, H., Liu, R., Wang, X., Hou, X., Liu, Y., et al. (2021) · Nature Communications · doi:10.1038/s41467-021-23458-5 · Cited by 620
“Current deep learning systems for DR screening have been predominantly focused on the identi fi cation of patients with referable DR (moderate NPDR or worse) or vision-threatening DR, which means the patients should be referred to ophthalmologists for treatment or closer follow-up 21,22,24 .”
BackgroundInfluential
Metadata
- Authors
- Li, Z., Keel, S., Liu, C., He, Y., Meng, W., Scheetz, J., Lee, P. Y., Shaw, J., Ting, D., Wong, T. Y., Taylor, H., Chang, R., & He, M.
- Affiliations
- Sun Yat-sen University (CN)
- The University of Melbourne (AU)
- Guangzhou Healgoo Interactive Medical Technology Co. Ltd., Guangzhou, China
- Baker Heart and Diabetes Institute (AU)
- National University of Singapore (SG)
- Stanford University (US)
- Corresponding author
- He, Mingguang
- ORCID
- Venue
- Diabetes Care
- Publisher
- American Diabetes Association
- Published
- 2018-10-01
- Volume
- 41
- Issue
- 12
- Pages
- 2509-2516
- ISSN
- 0149-5992 (ISSN-L)
- 1935-5548
- OpenAlex
- W2893365278
- Cited by
- 255
- References
- 39
- Citations by year
- 2026: 23 · 2025: 31 · 2024: 37 · 2023: 43 · 2022: 29 · 2021: 38 · 2020: 31 · 2019: 19 · 2018: 4
- FWCI
- 15.89
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Diabetes Care (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Natural Science Foundation of China (81420108008)
- Science and Technology Planning Project (2013B20400003)
- Bupa Health Foundation
- Topics
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Retinal Imaging and Analysis
- Health Sciences › Medicine › Ophthalmology › Retinal Diseases and Treatments
- Health Sciences › Medicine › Ophthalmology › Ophthalmology and Visual Health Research
- Keywords
- referable diabetic retinopathy, automated grading, color fundus images, deep learning, convolutional neural networks, diabetic macular edema, preproliferative diabetic retinopathy, retinal image analysis, diabetic retinopathy screening, intraretinal microvascular abnormalities
- MeSH terms
- Adult, Aged, Aged, 80 and over, Algorithms, Artificial Intelligence, Australia, Automation, Diabetic Retinopathy, Diagnosis, Computer-Assisted, Diagnostic Techniques, Ophthalmological, Female, Humans, Male, Mass Screening, Middle Aged, Photography, Sensitivity and Specificity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: An Automated Grading System for Detection of Vision-Threatening Referable Diabetic Retinopathy on the Basis of Color Fundus Photographs. https://app.cochat.ai/papers/10.2337/dc18-0147
- The paper
- Li, Z., Keel, S., Liu, C., He, Y., Meng, W., Scheetz, J., Lee, P. Y., Shaw, J., Ting, D., Wong, T. Y., Taylor, H., Chang, R., & He, M. (2018). An Automated Grading System for Detection of Vision-Threatening Referable Diabetic Retinopathy on the Basis of Color Fundus Photographs. Diabetes Care. https://doi.org/10.2337/dc18-0147
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.