The DOI matches its Crossref record; 20 of the 21 references it cites were found in a registry or an index.
Deep learning versus human graders for classifying diabetic retinopathy severity in a nationwide screening program
Open accessAbstract
Deep learning algorithms have been used to detect diabetic retinopathy (DR) with specialist-level accuracy. This study aims to validate one such algorithm on a large-scale clinical population, and compare the algorithm performance with that of human graders. A total of 25,326 gradable retinal images of patients with diabetes from the community-based, nationwide screening program of DR in Thailand were analyzed for DR severity and referable diabetic macular edema (DME). Grades adjudicated by a panel of international retinal specialists served as the reference standard. Relative to human graders, for detecting referable DR (moderate NPDR or worse), the deep learning algorithm had significantly higher sensitivity (0.97 vs. 0.74, p < 0.001), and a slightly lower specificity (0.96 vs. 0.98, p < 0.001). Higher sensitivity of the algorithm was also observed for each of the categories of severe or worse NPDR, PDR, and DME ( p < 0.001 for all comparisons). The quadratic-weighted kappa for determination of DR severity levels by the algorithm and human graders was 0.85 and 0.78 respectively ( p < 0.001 for the difference). Across different severity levels of DR for determining referable disease, deep learning significantly reduced the false negative rate (by 23%) at the cost of slightly higher false positive rates (2%). Deep learning algorithms may serve as a valuable tool for DR screening.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures



Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Twenty-five studies17,28,29,35,37,39,40,44–61 compared algorithm performance against healthcare professionals.”
From Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis (2021) · cites this paper as [39]
In the citing paper
“CNNs consistently demonstrate physician-level grading from fundus photographs [104][105][106][107] , which has led to a recent US FDA-cleared system 108 .”
From Deep learning-enabled medical computer vision (2021) · cites this paper as [105]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Gulshan, V., Peng, L., Coram, M., Stumpe, M. C., Wu, D., Narayanaswamy, A., Venugopalan, S., Widner, K., Madams, T., Cuadros, J., et al. (2016) · JAMA · doi:10.1001/jama.2016.17216 · Cited by 7,697
Self-citationInception-v4, Inception-ResNet and the Impact of Residual Connections on Learning
Szegedy, C., Ioffe, S., Vanhoucke, V., & Alemi, A. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.11231 · Cited by 4,636
Ting, D. S. W., Cheung, C. Y.-L., Lim, G., Tan, G. S. W., Quang, N. D., Gan, A., Hamzah, H., Garcia-Franco, R., San Yeo, I. Y., Lee, S. Y., et al. (2017) · JAMA · doi:10.1001/jama.2017.18152 · Cited by 2,402
Guidelines on Diabetic Eye Care
Wong, T. Y., Sun, J., Kawasaki, R., Ruamviboonsuk, P., Gupta, N., Lansingh, V. C., Maia, M., Mathenge, W., Moreker, S., Muqit, M. M. K., et al. (2018) · Ophthalmology · doi:10.1016/j.ophtha.2018.04.007 · Cited by 839
Self-citationKrause, J., Gulshan, V., Rahimy, E., Karth, P., Widner, K., Corrado, G. S., Peng, L., & Webster, D. R. (2018) · Ophthalmology · doi:10.1016/j.ophtha.2018.01.034 · Cited by 564
Self-citationVirgili, G., Menchini, F., Casazza, G., Hogg, R., Das, R. R., Wang, X., & Michelessi, M. (2015) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd008081.pub3 · Cited by 265
Cost-effectiveness of a National Telemedicine Diabetic Retinopathy Screening Program in Singapore
Nguyen, H. V., Tan, G. S. W., Tapp, R. J., Mital, S., Ting, D. S. W., Wong, H. T., Tan, C. S., Laude, A., Tai, E. S., Tan, N. C., et al. (2016) · Ophthalmology · doi:10.1016/j.ophtha.2016.08.021 · Cited by 218
Bresnick, G. H., Mukamel, D. B., Dickinson, J. C., & Cole, D. R. (2000) · Ophthalmology · doi:10.1016/s0161-6420(99)00010-x · Cited by 163
Telehealth Practice Recommendations for Diabetic Retinopathy, Second Edition
Li, H. K., Horton, M. B., Bursell, S., Cavallerano, J. D., Zimmer-Galler, I. E., Tennant, M., Abràmoff, M. D., Chaum, E., DeBuc, D. C., Leonard-Martin, T., et al. (2011) · Telemedicine Journal and e-Health · doi:10.1089/tmj.2011.0075 · Cited by 128
Diabetic retinopathy management guidelines
Chakrabarti, R., Harper, C. A., & Keeffe, J. E. (2012) · Expert Review of Ophthalmology · doi:10.1586/eop.12.52 · Cited by 112
Research works printed but not linked: 1
- Jenchitr, W. et al. The national survey of blindness low vision and visual impairment in thailand 2006–2007. Thai. J. Pub. Hlth. Ophthalmol. 21, 10 (2007).
Cited by
Liu, X., Faes, L., Kale, A. U., Wagner, S. K., Fu, D. J., Bruynseels, A., Mahendiran, T., Moraes, G., Shamdas, M., Kern, C., et al. (2019) · The Lancet Digital Health · doi:10.1016/s2589-7500(19)30123-2 · Cited by 1,883
Artificial intelligence in healthcare: transforming the practice of medicine
Bajwa, J., Munir, U., Nori, A., & Williams, B. (2021) · Future Healthcare Journal · doi:10.7861/fhj.2021-0095 · Cited by 1,856
Artificial intelligence and deep learning in ophthalmology
Ting, D. S. W., Pasquale, L. R., Peng, L., Campbell, J. P., Lee, A. Y., Raman, R., Tan, G. S. W., Schmetterer, L., Keane, P. A., & Wong, T. Y. (2019) · British Journal of Ophthalmology · doi:10.1136/bjophthalmol-2018-313173 · Cited by 1,554
Self-citationArtificial Intelligence and Deep Learning in Ophthalmology
Wang, Z., Keane, P. A., Chiang, M., Cheung, C. Y., Wong, T. Y., & Ting, D. S. W. (2022) · Artificial Intelligence in Medicine · doi:10.1007/978-3-030-64573-1_200 · Cited by 1,524
Self-citationDeep learning-enabled medical computer vision
Esteva, A., Chou, K., Yeung, S., Naik, N., Madani, A., Mottaghi, A., Liu, Y., Topol, E., Dean, J., & Socher, R. (2021) · npj Digital Medicine · doi:10.1038/s41746-020-00376-2 · Cited by 1,406
Metadata
- Authors
- Ruamviboonsuk, P., Krause, J., Chotcomwongse, P., Sayres, R., Raman, R., Widner, K., Campana, B. J. L., Phene, S., Hemarat, K., Tadarati, M., Silpa-Archa, S., Limwattanayingyong, J., Rao, C., Kuruvilla, O., Jung, J., Tan, J., Orprayoon, S., Kangwanwongpaisan, C., Sukumalpaiboon, R., Luengchaichawang, C., Fuangkaew, J., Kongsap, P., Chualinpha, L., Saree, S., Kawinpanitan, S., Mitvongsa, K., Lawanasakol, S., Thepchatri, C., Wongpichedchai, L., Corrado, G. S., Peng, L., & Webster, D. R.
- Affiliations
- Rajavithi Hospital (TH)
- Google (United States) (US)
- Sankara Nethralaya (IN)
- Vajira Hospital (TH)
- University of California, San Francisco (US)
- University of Southern California (US)
- Lamphun Hospital (TH)
- Somdejphrajaotaksin Maharaj Hospital (TH)
- Sawanpracharak Hospital (TH)
- Photharam Hospital (TH)
- Prapokklao Hospital (TH)
- Mahasarakham Hospital (TH)
- Suratthani Hospital (TH)
- Bangkok Metropolitan Administration (TH)
- Corresponding author
- Peng, Lily
- ORCID
- Venue
- npj Digital Medicine
- Published
- 2019-04-10
- Volume
- 2
- Issue
- 1
- Pages
- 25-25
- ISSN
- 2398-6352 (ISSN-L)
- PMC
- PMC6550283
- OpenAlex
- W2935900679
- Cited by
- 233
- References
- 21
- Citations by year
- 2026: 25 · 2025: 25 · 2024: 32 · 2023: 36 · 2022: 29 · 2021: 46 · 2020: 30 · 2019: 9
- FWCI
- 18.81
- Type
- journal article
- Language
- en
- Update notices
- correction (2019-07-23)
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- npj Digital Medicine (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Greater South Information System
- Topics
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Retinal Imaging and Analysis
- Health Sciences › Medicine › Ophthalmology › Retinal Diseases and Treatments
- Health Sciences › Medicine › Ophthalmology › Ophthalmology and Visual Health Research
- Keywords
- diabetic retinopathy, diabetic macular edema, deep learning, diabetic retinopathy grading, referable diabetic retinopathy, diabetic retinopathy screening, fundus images, sensitivity and specificity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Deep learning versus human graders for classifying diabetic retinopathy severity in a nationwide screening program. https://app.cochat.ai/papers/10.1038/s41746-019-0099-8
- The paper
- Ruamviboonsuk, P., Krause, J., Chotcomwongse, P., Sayres, R., Raman, R., Widner, K., Campana, B. J. L., Phene, S., Hemarat, K., Tadarati, M., Silpa-Archa, S., Limwattanayingyong, J., Rao, C., Kuruvilla, O., Jung, J., Tan, J., Orprayoon, S., Kangwanwongpaisan, C., Sukumalpaiboon, R., . . . Webster, D. R. (2019). Deep learning versus human graders for classifying diabetic retinopathy severity in a nationwide screening program. npj Digital Medicine. https://doi.org/10.1038/s41746-019-0099-8
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.