Evaluation of an AI system for the automated detection of glaucoma from stereoscopic optic disc photographs: the European Optic Disc Assessment Study
Open accessAbstract
Objectives To evaluate the performance of a deep learning based Artificial Intelligence (AI) software for detection of glaucoma from stereoscopic optic disc photographs, and to compare this performance to the performance of a large cohort of ophthalmologists and optometrists.Methods A retrospective study evaluating the diagnostic performance of an AI software (Pegasus v1.0, Visulytix Ltd., London UK) and comparing it with that of 243 European ophthalmologists and 208 British optometrists, as determined in previous studies, for the detection of glaucomatous optic neuropathy from 94 scanned stereoscopic photographic slides scanned into digital format.Results Pegasus was able to detect glaucomatous optic neuropathy with an accuracy of 83.4% (95% CI: 77.5-89.2). This is comparable to an average ophthalmologist accuracy of 80.5% (95% CI: 67.2-93.8) and average optometrist accuracy of 80% (95% CI: 67-88) on the same images. In addition, the AI system had an intra-observer agreement (Cohen's Kappa, κ) of 0.74 (95% CI: 0.63-0.85), compared with 0.70 (range: -0.13-1.00; 95% CI: 0.67-0.73) and 0.71 (range: 0.08-1.00) for ophthalmologists and optometrists, respectively. There was no statistically significant difference between the performance of the deep learning system and ophthalmologists or optometrists.Conclusion The AI system obtained a diagnostic performance and repeatability comparable to that of the ophthalmologists and optometrists. We conclude that deep learning based AI systems, such as Pegasus, demonstrate significant promise in the assisted detection of glaucomatous optic neuropathy.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Twenty-five ophthalmology studies compared algorithm performance against healthcare professionals.
“Twenty-five studies17,28,29,35,37,39,40,44–61 compared algorithm performance against healthcare professionals.”
From Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis (2021) · cites this paper as [45]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126 · see citing works
Gulshan, V., Peng, L., Coram, M., Stumpe, M. C., Wu, D., Narayanaswamy, A., Venugopalan, S., Widner, K., Madams, T., Cuadros, J., et al. (2016) · JAMA · doi:10.1001/jama.2016.17216 · Cited by 7,697 · see citing works
The number of people with glaucoma worldwide in 2010 and 2020
Quigley, H. A. (2006) · British Journal of Ophthalmology · doi:10.1136/bjo.2005.081224 · Cited by 7,417 · see citing works
Clinically applicable deep learning for diagnosis and referral in retinal disease
De Fauw, J., Ledsam, J. R., Romera-Paredes, B., Nikolov, S., Tomasev, N., Blackwell, S., Askham, H., Glorot, X., O’Donoghue, B., Visentin, D., et al. (2018) · Nature Medicine · doi:10.1038/s41591-018-0107-6 · Cited by 2,639 · see citing works
Ting, D. S. W., Cheung, C. Y.-L., Lim, G., Tan, G. S. W., Quang, N. D., Gan, A., Hamzah, H., Garcia-Franco, R., San Yeo, I. Y., Lee, S. Y., et al. (2017) · JAMA · doi:10.1001/jama.2017.18152 · Cited by 2,402 · see citing works
Glaucoma detection based on deep convolutional neural network
Chen, X., Xu, Y., Wong, D. W. K., Wong, T. Y., & Liu, J. (2015) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2015.7318462 · Cited by 429 · see citing works
Disc-Aware Ensemble Network for Glaucoma Screening From Fundus Image
Fu, H., Cheng, J., Xu, Y., Zhang, C., Wong, D. W. K., Liu, J., & Cao, X. (2018) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2018.2837012 · Cited by 429 · see citing works
Reus, N. J., Lemij, H. G., Garway‐Heath, D. F., et al. (2010) · Ophthalmology · doi:10.1016/j.ophtha.2009.09.026 · Cited by 151 · see citing works
Self-citationThe Portsmouth-based glaucoma refinement scheme: a role for virtual clinics in the future?
Trikha, S., MacGregor, C., Jeffery, M., et al. (2012) · Eye · doi:10.1038/eye.2012.120 · Cited by 69 · see citing works
Self-citationChan, H. H. L., Ong, D. N., Kong, Y. X. G., et al. (2014) · American Journal of Ophthalmology · doi:10.1016/j.ajo.2014.01.024 · Cited by 63 · see citing works
Optometrists referrals for glaucoma assessment: a prospective survey of clinical data and outcomes
Lockwood, A., Kirwan, J., & Ashleigh, Z. (2010) · Eye · doi:10.1038/eye.2010.77 · Cited by 40 · see citing works
Harper, R. A. (2001) · Graefe s Archive for Clinical and Experimental Ophthalmology · doi:10.1007/s004170100272 · Cited by 39 · see citing works
Hadwin, S. E., Redmond, T., Garway‐Heath, D. F., et al. (2013) · Ophthalmic and Physiological Optics · doi:10.1111/opo.12066 · Cited by 29 · see citing works
Topical Ripasudil Suppresses Retinal Ganglion Cell Death in a Mouse Model of Normal Tension Glaucoma
Akaiwa, K., Namekata, K., Azuchi, Y., Sano, H., Guo, X., Kimura, A. K., Harada, C., Mitamura, Y., & Harada, T. (2018) · Investigative Ophthalmology & Visual Science · doi:10.1167/iovs.17-23276 · Cited by 25 · see citing works
Grigera, D. E., Mello, P. A. A., Barbosa, W. L., et al. (2013) · Arquivos Brasileiros de Oftalmologia · doi:10.1590/s0004-27492013000300007 · Cited by 6 · see citing works
TEITELBAUM, B. A., Haefs, R., & Connor, D. (2001) · PubMed · Cited by 6 · see citing works
Seo, E., Jaccard, N., Trikha, S., et al. (2018) · Investigative Ophthalmology & Visual Science · Cited by 5 · see citing works
Self-citationRatnarajan, G. (2016) · UCL Discovery (University College London) · Cited by 1 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- Yang GG, Minasyan A, Lacey M, Gordon J, Knoll T. HMGA2 is a reliable immunohistochemical marker for separating melanoma from Nevi. Nature. 2014;94:145A.
Cited by
Sorted by citations
Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis
Aggarwal, R., Sounderajah, V., Martin, G., Ting, D. S. W., Karthikesalingam, A., King, D., Ashrafian, H., & Darzi, A. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00438-z · Cited by 946 · see citing works
- Authors
- Rogers, T. W., Jaccard, N., Carbonaro, F., Lemij, H. G., Vermeer, K. A., Reus, N. J., & Trikha, S.
- Affiliations
- Mater Dei Hospital (MT)
- Rotterdam Eye Hospital (NL)
- Amphia Ziekenhuis (NL)
- King's College Hospital NHS Foundation Trust (GB)
- Corresponding author
- Rogers, Thomas W.
- ORCID
- Venue
- Eye
- Published
- 2019-07-02
- Volume
- 33
- Issue
- 11
- Pages
- 1791-1797
- ISSN
- 0078-5334 (ISSN-L)
- 0950-222X
- 1476-5454
- PMC
- PMC7002599
- OpenAlex
- W2948320234
- Cited by
- 47
- References
- 19
- Citations by year
- 2026: 7 · 2025: 4 · 2024: 4 · 2023: 9 · 2022: 7 · 2021: 11 · 2020: 5
- FWCI
- 3.80
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Eye (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Retinal Imaging and Analysis
- Health Sciences › Medicine › Ophthalmology › Glaucoma and retinal disorders
- Health Sciences › Medicine › Ophthalmology › Ophthalmology and Visual Health Research
- Keywords
- glaucoma detection, glaucomatous optic neuropathy, deep learning, artificial intelligence, automated diagnosis, intraobserver agreement
- MeSH terms
- Optometrists, Ophthalmologists, Deep Learning, Artificial Intelligence, Clinical Competence, Diagnosis, Computer-Assisted, Europe, False Positive Reactions, Glaucoma, Open-Angle, Humans, Optic Disk, Optic Nerve Diseases, Photography, Predictive Value of Tests, Retrospective Studies, ROC Curve, Sensitivity and Specificity, Reproducibility of Results, Observer Variation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
