The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Artificial intelligence using deep learning to screen for referable and vision-threatening diabetic retinopathy in Africa: a clinical validation study
Open accessAbstract
BACKGROUND: Radical measures are required to identify and reduce blindness due to diabetes to achieve the Sustainable Development Goals by 2030. Therefore, we evaluated the accuracy of an artificial intelligence (AI) model using deep learning in a population-based diabetic retinopathy screening programme in Zambia, a lower-middle-income country. METHODS: We adopted an ensemble AI model consisting of a combination of two convolutional neural networks (an adapted VGGNet architecture and a residual neural network architecture) for classifying retinal colour fundus images. We trained our model on 76 370 retinal fundus images from 13 099 patients with diabetes who had participated in the Singapore Integrated Diabetic Retinopathy Program, between 2010 and 2013, which has been published previously. In this clinical validation study, we included all patients with a diagnosis of diabetes that attended a mobile screening unit in five urban centres in the Copperbelt province of Zambia from Feb 1 to June 31, 2012. In our model, referable diabetic retinopathy was defined as moderate non-proliferative diabetic retinopathy or worse, diabetic macular oedema, and ungradable images. Vision-threatening diabetic retinopathy comprised severe non-proliferative and proliferative diabetic retinopathy. We calculated the area under the curve (AUC), sensitivity, and specificity for referable diabetic retinopathy, and sensitivities of vision-threatening diabetic retinopathy and diabetic macular oedema compared with the grading by retinal specialists. We did a multivariate analysis for systemic risk factors and referable diabetic retinopathy between AI and human graders. FINDINGS: A total of 4504 retinal fundus images from 3093 eyes of 1574 Zambians with diabetes were prospectively recruited. Referable diabetic retinopathy was found in 697 (22·5%) eyes, vision-threatening diabetic retinopathy in 171 (5·5%) eyes, and diabetic macular oedema in 249 (8·1%) eyes. The AUC of the AI system for referable diabetic retinopathy was 0·973 (95% CI 0·969-0·978), with corresponding sensitivity of 92·25% (90·10-94·12) and specificity of 89·04% (87·85-90·28). Vision-threatening diabetic retinopathy sensitivity was 99·42% (99·15-99·68) and diabetic macular oedema sensitivity was 97·19% (96·61-97·77). The AI model and human graders showed similar outcomes in referable diabetic retinopathy prevalence detection and systemic risk factors associations. Both the AI model and human graders identified longer duration of diabetes, higher level of glycated haemoglobin, and increased systolic blood pressure as risk factors associated with referable diabetic retinopathy. INTERPRETATION: An AI system shows clinically acceptable performance in detecting referable diabetic retinopathy, vision-threatening diabetic retinopathy, and diabetic macular oedema in population-based diabetic retinopathy screening. This shows the potential application and adoption of such AI technology in an under-resourced African population to reduce the incidence of preventable blindness, even when the model is trained in a different population. FUNDING: National Medical Research Council Health Service Research Grant, Large Collaborative Grant, Ministry of Health, Singapore; the SingHealth Foundation; and the Tanoto Foundation.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Remote grading of retinal images is widely utilized, with 18 out of 30 studies relying on this approach [10, 15, 16, 18, 21–27, 29–35, 37]”
From Diabetic retinopathy screening model in low and middle-income countries: a scoping review (2025) · cites this paper as [23]
In the citing paper
“Various severity scales are used to classify diabetic retinopathy (DR), with the International Clinical Diabetic Retinopathy (ICDR) severity scale being the most commonly utilized in 15 out of 30 studies [9, 11–13, 15, 18, 21–23, 32, 34–36].”
From Diabetic retinopathy screening model in low and middle-income countries: a scoping review (2025) · cites this paper as [23]
In the citing paper
“The predominant method for managing these remotely graded images is ‘store and forward’ (14 out of 30 studies) [15, 16, 18, 22–24, 26–30, 33, 35, 37], with seven studies adopting the ‘real-time’ processing method [8, 10, 11, 25, 31, 32, 34]”
From Diabetic retinopathy screening model in low and middle-income countries: a scoping review (2025) · cites this paper as [23]
In the citing paper
“Recent advancements include the use of CNN architectures like VGGNet [23] and ResNet [32], achieving high AUCs for referable DR up to 0.97 [21, 23].”
From Diabetic retinopathy screening model in low and middle-income countries: a scoping review (2025) · cites this paper as [23]
In the citing paper
“Real-time national deployment in Thailand and African clinical validation studies further show that AI performance depends on workflow, image acquisition, and the surrounding screening pathway (9,11).”
From From validation to public health impact: implementing equitable AI-enabled diabetic retinopathy screening (2026) · cites this paper as [9]
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Gulshan, V., Peng, L., Coram, M., Stumpe, M. C., Wu, D., Narayanaswamy, A., Venugopalan, S., Widner, K., Madams, T., Cuadros, J., et al. (2016) · JAMA · doi:10.1001/jama.2016.17216 · Cited by 7,697
Global prevalence and major risk factors of diabetic retinopathy
Yau, J. W. Y., Rogers, S. L., Kawasaki, R., Lamoureux, E. L., Kowalski, J. W., Bek, T., Chen, S.-J., Dekker, J. M., Fletcher, A., Grauslund, J., et al. (2012) · Diabetes Care · doi:10.2337/dc11-1909 · Cited by 4,985
Self-citationAxiomatic attribution for deep networks
Sundararajan, M., Taly, A., & Yan, Q. (2017) · arXiv · doi:10.48550/arxiv.1703.01365 · Cited by 4,087
Flaxman, S. R., Bourne, R. R. A., Resnikoff, S., Ackland, P., Braithwaite, T., Cicinelli, M. V., Das, A., Jonas, J. B., Keeffe, J., Kempen, J. H., et al. (2017) · The Lancet Global Health · doi:10.1016/s2214-109x(17)30393-5 · Cited by 3,477
Self-citationEhteshami Bejnordi, B., Veta, M., Johannes van Diest, P., van Ginneken, B., Karssemeijer, N., Litjens, G., van der Laak, J. A. W. M., and the CAMELYON16 Consortium, Hermsen, M., Manson, Q. F., et al. (2017) · JAMA · doi:10.1001/jama.2017.14585 · Cited by 3,293
Ting, D. S. W., Cheung, C. Y.-L., Lim, G., Tan, G. S. W., Quang, N. D., Gan, A., Hamzah, H., Garcia-Franco, R., San Yeo, I. Y., Lee, S. Y., et al. (2017) · JAMA · doi:10.1001/jama.2017.18152 · Cited by 2,402
Self-citationPrediction of cardiovascular risk factors from retinal fundus photographs via deep learning
Poplin, R., Varadarajan, A. V., Blumer, K., Liu, Y., McConnell, M. V., Corrado, G. S., Peng, L., & Webster, D. R. (2018) · Nature Biomedical Engineering · doi:10.1038/s41551-018-0195-0 · Cited by 1,865
Abràmoff, M. D., Lavin, P. T., Birch, M., Shah, N., & Folk, J. C. (2018) · npj Digital Medicine · doi:10.1038/s41746-018-0040-6 · Cited by 1,638
Automated Identification of Diabetic Retinopathy Using Deep Learning
Gargeya, R., & Leng, T. (2017) · Ophthalmology · doi:10.1016/j.ophtha.2017.02.008 · Cited by 1,362
Abràmoff, M. D., Lou, Y., Erginay, A., Clarida, W., Amelon, R., Folk, J. C., & Niemeijer, M. (2016) · Investigative Opthalmology & Visual Science · doi:10.1167/iovs.16-19964 · Cited by 1,101
Cited by
Key challenges for delivering clinical impact with artificial intelligence
Kelly, C. J., Karthikesalingam, A., Suleyman, M., Corrado, G., & King, D. (2019) · BMC Medicine · doi:10.1186/s12916-019-1426-2 · Cited by 3,136
Artificial intelligence in healthcare: transforming the practice of medicine
Bajwa, J., Munir, U., Nori, A., & Williams, B. (2021) · Future Healthcare Journal · doi:10.7861/fhj.2021-0095 · Cited by 1,856
Artificial intelligence and deep learning in ophthalmology
Ting, D. S. W., Pasquale, L. R., Peng, L., Campbell, J. P., Lee, A. Y., Raman, R., Tan, G. S. W., Schmetterer, L., Keane, P. A., & Wong, T. Y. (2019) · British Journal of Ophthalmology · doi:10.1136/bjophthalmol-2018-313173 · Cited by 1,554
Self-citationArtificial Intelligence and Deep Learning in Ophthalmology
Wang, Z., Keane, P. A., Chiang, M., Cheung, C. Y., Wong, T. Y., & Ting, D. S. W. (2022) · Artificial Intelligence in Medicine · doi:10.1007/978-3-030-64573-1_200 · Cited by 1,524
Self-citationDiagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis
Aggarwal, R., Sounderajah, V., Martin, G., Ting, D. S. W., Karthikesalingam, A., King, D., Ashrafian, H., & Darzi, A. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00438-z · Cited by 946
Self-citation
Metadata
- Authors
- Bellemo, V., Lim, Z. W., Lim, G., Nguyen, Q. D., Xie, Y., Yip, M. Y. T., Hamzah, H., Ho, J., Lee, X. Q., Hsu, W., Lee, M. L., Musonda, L., Chandran, M., Chipalo-Mutati, G., Muma, M., Tan, G. S. W., Sivaprasad, S., Menon, G., Wong, T. Y., & Ting, D. S. W.
- Affiliations
- Singapore National Eye Center (SG)
- National University of Singapore (SG)
- Duke-NUS Medical School (SG)
- Frimley Park Hospital (GB)
- Ministry of Health (ZM)
- NIHR Moorfields Biomedical Research Centre (GB)
- SingHealth (SG)
- Corresponding author
- Ting, Daniel S W
- ORCID
- Venue
- The Lancet Digital Health
- Publisher
- Elsevier BV
- Published
- 2019-05
- Volume
- 1
- Issue
- 1
- Pages
- e35-e44
- ISSN
- 2589-7500 (ISSN-L)
- OpenAlex
- W2942586440
- Cited by
- 419
- References
- 32
- Citations by year
- 2026: 67 · 2025: 80 · 2024: 65 · 2023: 59 · 2022: 44 · 2021: 54 · 2020: 40 · 2019: 8
- FWCI
- 24.07
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by-nc-nd
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
- www.elsevier.com/legal/tdmrep-license (tdm)
- creativecommons.org/licenses/by-nc-nd/4.0/ (vor)
- Locations
- The Lancet Digital Health (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- National University of Singapore (submittedVersion, cc-by-nc-nd)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- National Health Innovation Center, Innovation to Develop (NHIC-I2D-1409022)
- SingHealth Foundation Research Grant (SHF/FG648S/2015)
- The Singapore Diabetic Retinopathy Program received funding from the Singapore Ministry of Health (AIC/RPDD/SIDRP/SERI/FY2013/0018 and AIC/HPD/FY2016/0912)
- Topics
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Retinal Imaging and Analysis
- Health Sciences › Medicine › Ophthalmology › Retinal Diseases and Treatments
- Health Sciences › Medicine › Ophthalmology › Ophthalmology and Visual Health Research
- Keywords
- diabetic macular edema, artificial intelligence, deep learning, fundus images, vision-threatening diabetic retinopathy, referable diabetic retinopathy, mobile screening, Zambia, Copperbelt, lower-middle-income countries, clinical validation, diagnostic accuracy, blindness prevention, Sustainable Development Goals
- MeSH terms
- Deep Learning, Adult, Artificial Intelligence, Diabetic Retinopathy, Female, Humans, Male, Mass Screening, Photography, Prospective Studies, Retina, Sensitivity and Specificity, Zambia, Neural Networks, Computer, Area Under Curve
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Artificial intelligence using deep learning to screen for referable and vision-threatening diabetic retinopathy in Africa: a clinical validation study. https://app.cochat.ai/papers/10.1016/s2589-7500(19)30004-4
- The paper
- Bellemo, V., Lim, Z. W., Lim, G., Nguyen, Q. D., Xie, Y., Yip, M. Y. T., Hamzah, H., Ho, J., Lee, X. Q., Hsu, W., Lee, M. L., Musonda, L., Chandran, M., Chipalo-Mutati, G., Muma, M., Tan, G. S. W., Sivaprasad, S., Menon, G., Wong, T. Y., & Ting, D. S. W. (2019). Artificial intelligence using deep learning to screen for referable and vision-threatening diabetic retinopathy in Africa: a clinical validation study. The Lancet Digital Health. https://doi.org/10.1016/s2589-7500(19)30004-4
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.