The DOI matches its Crossref record; 16 of the 17 references it cites were found in a registry or an index.
Evaluation of a Deep Learning System For Identifying Glaucomatous Optic Neuropathy Based on Color Fundus Photographs
Open accessAbstract
Precis: Pegasus outperformed 5 of the 6 ophthalmologists in terms of diagnostic performance, and there was no statistically significant difference between the deep learning system and the “best case” consensus between the ophthalmologists. The agreement between Pegasus and gold standard was 0.715, whereas the highest ophthalmologist agreement with the gold standard was 0.613. Furthermore, the high sensitivity of Pegasus makes it a valuable tool for screening patients with glaucomatous optic neuropathy. Purpose: The purpose of this study was to evaluate the performance of a deep learning system for the identification of glaucomatous optic neuropathy. Materials and Methods: Six ophthalmologists and the deep learning system, Pegasus, graded 110 color fundus photographs in this retrospective single-center study. Patient images were randomly sampled from the Singapore Malay Eye Study. Ophthalmologists and Pegasus were compared with each other and to the original clinical diagnosis given by the Singapore Malay Eye Study, which was defined as the gold standard. Pegasus’ performance was compared with the “best case” consensus scenario, which was the combination of ophthalmologists whose consensus opinion most closely matched the gold standard. The performance of the ophthalmologists and Pegasus, at the binary classification of nonglaucoma versus glaucoma from fundus photographs, was assessed in terms of sensitivity, specificity and the area under the receiver operating characteristic curve (AUROC), and the intraobserver and interobserver agreements were determined. Results: Pegasus achieved an AUROC of 92.6% compared with ophthalmologist AUROCs that ranged from 69.6% to 84.9% and the “best case” consensus scenario AUROC of 89.1%. Pegasus had a sensitivity of 83.7% and a specificity of 88.2%, whereas the ophthalmologists’ sensitivity ranged from 61.3% to 81.6% and specificity ranged from 80.0% to 94.1%. The agreement between Pegasus and gold standard was 0.715, whereas the highest ophthalmologist agreement with the gold standard was 0.613. Intraobserver agreement ranged from 0.62 to 0.97 for ophthalmologists and was perfect (1.00) for Pegasus. The deep learning system took ∼10% of the time of the ophthalmologists in determining classification. Conclusions: Pegasus outperformed 5 of the 6 ophthalmologists in terms of diagnostic performance, and there was no statistically significant difference between the deep learning system and the “best case” consensus between the ophthalmologists. The high sensitivity of Pegasus makes it a valuable tool for screening patients with glaucomatous optic neuropathy. Future work will extend this study to a larger sample of patients.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Twenty-five studies17,28,29,35,37,39,40,44–61 compared algorithm performance against healthcare professionals.”
From Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis (2021) · cites this paper as [46]
Citations
Cites
A guideline of selecting and reporting intraclass correlation coefficients for reliability research
Koo, T. K., & Li, M. Y. (2016) · Journal of Chiropractic Medicine · doi:10.1016/j.jcm.2016.02.012 · Cited by 31,709 · cited 2×
“11 Similarly, the agreement varied between the 6 ophthalmologists and between them and the gold standard based on disc photography.”
BackgroundDeLong, E. R., DeLong, D. M., & Clarke-Pearson, D. L. (1988) · Biometrics · doi:10.2307/2531595 · Cited by 22,649
Global Prevalence of Glaucoma and Projections of Glaucoma Burden through 2040
Tham, Y.-C., Li, X., Wong, T. Y., Quigley, H. A., Aung, T., & Cheng, C.-Y. (2014) · Ophthalmology · doi:10.1016/j.ophtha.2014.05.013 · Cited by 7,159
Li, Z., He, Y., Keel, S., Meng, W., Chang, R. T., & He, M. (2018) · Ophthalmology · doi:10.1016/j.ophtha.2018.01.023 · Cited by 830 · cited 1×
“19 In addition to disease detection, AI systems can be used to establish normal examinations in ophthalmology.”
BackgroundThe coming of age of artificial intelligence in medicine
Patel, V. L., Shortliffe, E. H., Stefanelli, M., Szolovits, P., Berthold, M. R., Bellazzi, R., & Abu-Hanna, A. (2009) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2008.07.017 · Cited by 625
Self-citationFoong, A. W. P., Saw, S.-M., Loo, J.-L., Shen, S., Loon, S.-C., Rosman, M., Aung, T., Tan, D. T. H., Tai, E. S., & Wong, T. Y. (2007) · Ophthalmic Epidemiology · doi:10.1080/09286580600878844 · Cited by 432
Christopher, M., Belghith, A., Bowd, C., Proudfoot, J. A., Goldbaum, M. H., Weinreb, R. N., Girkin, C. A., Liebmann, J. M., & Zangwill, L. M. (2018) · Scientific Reports · doi:10.1038/s41598-018-35044-9 · Cited by 328
Development of machine learning models for diagnosis of glaucoma
Kim, S. J., Cho, K. J., & Oh, S. (2017) · PLOS ONE · doi:10.1371/journal.pone.0177726 · Cited by 266
Development of a deep residual learning algorithm to screen for glaucoma from fundus photography
Shibata, N., Tanito, M., Mitsuhashi, K., Fujino, Y., Matsuura, M., Murata, H., & Asaoka, R. (2018) · Scientific Reports · doi:10.1038/s41598-018-33013-w · Cited by 249 · cited 1×
“16 However, these studies have not previously compared the performance of a deployed deep learning system to the performance of ophthalmologists for the same patients, nor compared the speed at which a diagnosis outcome could be determined from the photograph.”
ResultThe Current State of Teleophthalmology in the United States
Rathi, S., Tsui, E., Mehta, N., Zahid, S., & Schuman, J. S. (2017) · Ophthalmology · doi:10.1016/j.ophtha.2017.05.026 · Cited by 248 · cited 2×
“20 Remote screening allows increased access for underserved populations and earlier detection of ophthalmic disease.”
Background
Cited by
Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis
Aggarwal, R., Sounderajah, V., Martin, G., Ting, D. S. W., Karthikesalingam, A., King, D., Ashrafian, H., & Darzi, A. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00438-z · Cited by 946
Systematic reviews of machine learning in healthcare: a literature review
Kolasa, K., Admassu, B. Y., Hołownia-Voloskova, M. E., Kędzior, K. J., Poirrier, J., & Perni, S. (2023) · Expert Review of Pharmacoeconomics & Outcomes Research · doi:10.1080/14737167.2023.2279107 · Cited by 116
Explainable Machine Learning Model for Glaucoma Diagnosis and Its Interpretation
Oh, S., Park, Y., Cho, K. J., & Kim, S.-J. (2021) · Diagnostics · doi:10.3390/diagnostics11030510 · Cited by 103
“Among them, there are many papers that diagnose glaucoma by detecting increased CDR or loss of neuro-retinal rim in the disc using fundus photography [28].”
BackgroundA deep learning approach in diagnosing fungal keratitis based on corneal photographs
Kuo, M., Hsu, B. W., Yin, Y.-K., Fang, P., Lai, H., Chen, A., Yu, M.-S., & Tseng, V. S. M. (2020) · Scientific Reports · doi:10.1038/s41598-020-71425-9 · Cited by 81
Artificial intelligence in glaucoma: opportunities, challenges, and future directions
Huang, X., Islam, M. R., Prome, S. A., Ahmed, F., Kazami, E., Serhan, H. A. A., Abd-Alrazaq, A. A., & Yousefi, S. (2023) · BioMedical Engineering OnLine · doi:10.1186/s12938-023-01187-8 · Cited by 81
“Al-Aswad et al. [289] evaluated the performance of Pegasus (an AI system based on deep learning) in glaucoma screening based on color fundus photographs by comparing with six ophthalmologists.”
Background
Metadata
- Authors
- Al-Aswad, L. A., Kapoor, R., Chu, C. K., Walters, S., Gong, D., Garg, A., Gopal, K., Patel, V., Sameer, T., Rogers, T. W., Nicolas, J., De Moraes, G. C., & Moazami, G.
- Affiliations
- Columbia University Medical Center, Harkness Eye Institute, New York, NY
- Columbia University Irving Medical Center (US)
- Visulytix Ltd
- Corresponding author
- Kapoor, Rahul
- Chu, Chia Kai
- Walters, Stephen
- Gong, Dan
- Gopal, Kalashree
- Patel, Vipul
- Moazami, Golnaz
- Venue
- Journal of Glaucoma
- Published
- 2019-06-21
- Volume
- 28
- Issue
- 12
- Pages
- 1029-1034
- ISSN
- 1057-0829 (ISSN-L)
- OpenAlex
- W2955420986
- Cited by
- 59
- References
- 17
- Citations by year
- 2026: 8 · 2025: 10 · 2024: 7 · 2023: 14 · 2022: 6 · 2021: 10 · 2020: 3 · 2019: 1
- FWCI
- 3.31
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Journal of Glaucoma (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Funders
- National Eye Institute (R21EY029605)
- Topics
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Retinal Imaging and Analysis
- Health Sciences › Medicine › Ophthalmology › Glaucoma and retinal disorders
- Health Sciences › Medicine › Ophthalmology › Ophthalmology and Visual Health Research
- Keywords
- glaucomatous optic neuropathy, glaucoma diagnosis, color fundus images, deep learning, Pegasus, ophthalmologist comparison, interobserver agreement, intraobserver agreement, glaucoma screening, Singapore Malay Eye Study, diagnostic accuracy, automated diagnosis, intraocular pressure, optic disc margin, cup-to-disc ratio
- MeSH terms
- Ophthalmologists, Deep Learning, Adult, Aged, Diagnosis, Computer-Assisted, Diagnostic Techniques, Ophthalmological, Female, Glaucoma, Open-Angle, Humans, Intraocular Pressure, Male, Middle Aged, Optic Disk, Optic Nerve Diseases, Photography, Retrospective Studies, ROC Curve, Sensitivity and Specificity, Observer Variation, Area Under Curve
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Evaluation of a Deep Learning System For Identifying Glaucomatous Optic Neuropathy Based on Color Fundus Photographs. https://app.cochat.ai/papers/10.1097/ijg.0000000000001319
- The paper
- Al-Aswad, L. A., Kapoor, R., Chu, C. K., Walters, S., Gong, D., Garg, A., Gopal, K., Patel, V., Sameer, T., Rogers, T. W., Nicolas, J., De Moraes, G. C., & Moazami, G. (2019). Evaluation of a Deep Learning System For Identifying Glaucomatous Optic Neuropathy Based on Color Fundus Photographs. Journal of Glaucoma. https://doi.org/10.1097/ijg.0000000000001319
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.