CoChat
Sign Up
Cited by 88Retinal Imaging and AnalysisOphthalmology and Visual Health ResearchRetinal Diseases and Treatments

Real-world artificial intelligence-based opportunistic screening for diabetic retinopathy in endocrinology and indigenous healthcare settings in Australia

Open access
Scheetz, Jane, Koca, Dilara, McGuinness, Myra, Holloway, Edith, Tan, Zachary, Zhu, Zhuoting, O’Day, Rod, Sandhu, Sukhpal +5 more·2021·Scientific Reports
DOI 10.1038/s41598-021-94178-5PMC PMC8339059
Open this paper in CoChat — free accountOpen PDF

Abstract

This study investigated the diagnostic performance, feasibility, and end-user experiences of an artificial intelligence (AI)-assisted diabetic retinopathy (DR) screening model in real-world Australian healthcare settings. The study consisted of two components: (1) DR screening of patients using an AI-assisted system and (2) in-depth interviews with health professionals involved in implementing screening. Participants with type 1 or type 2 diabetes mellitus attending two endocrinology outpatient and three Aboriginal Medical Services clinics between March 2018 and May 2019 were invited to a prospective observational study. A single 45-degree (macula centred), non-stereoscopic, colour retinal image was taken of each eye from participants and were instantly screened for referable DR using a custom offline automated AI system. A total of 236 participants, including 174 from endocrinology and 62 from Aboriginal Medical Services clinics, provided informed consent and 203 (86.0%) were included in the analysis. A total of 33 consenting participants (14%) were excluded from the primary analysis due to ungradable or missing images from small pupils (n = 21, 63.6%), cataract (n = 7, 21.2%), poor fixation (n = 2, 6.1%), technical issues (n = 2, 6.1%), and corneal scarring (n = 1, 3%). The area under the curve, sensitivity, and specificity of the AI system for referable DR were 0.92, 96.9% and 87.7%, respectively. There were 51 disagreements between the reference standard and index test diagnoses, including 29 which were manually graded as ungradable, 21 false positives, and one false negative. A total of 28 participants (11.9%) were referred for follow-up based on new ocular findings, among whom, 15 (53.6%) were able to be contacted and 9 (60%) adhered to referral. Of 207 participants who completed a satisfaction questionnaire, 93.7% specified they were either satisfied or extremely satisfied, and 93.2% specified they would be likely or extremely likely to use this service again. Clinical staff involved in screening most frequently noted that the AI system was easy to use, and the real-time diagnostic report was useful. Our study indicates that AI-assisted DR screening model is accurate and well-accepted by patients and clinicians in endocrinology and indigenous healthcare settings. Future deployments of AI-assisted screening models would require consideration of downstream referral pathways.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s41598-021-94178-5

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Real-world implementation of diabetic retinopathy screening with artificial intelligence-assisted care pathways has accelerated globally since 2020.

    “Real-world implementation of DR screening with AI-assisted care pathways has accelerated globally since 20208,13–22, yet evidence on clinical and health system outcomes such as referral uptake remains limited.”

    From Referral uptake after diabetic retinopathy screening with artificial intelligence-assisted care pathways: a systematic review and meta-analysis (2026) · cites this paper as [21]

    The Introduction (page not supplied) describes international real-world use of AI-based diabetic retinopathy screening as “early and emerging,” and the study documents implementation in Australian clinics. It does not establish that implementation accelerated globally or identify 2020 as the start of that acceleration. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 38 · Cited by 88

Research works printed but not linked: 32

  1. [3]Resnikoff S Felch C Gauthier T.-M. Spivey BJ The number of ophthalmologists in practice and training worldwide: a growing gap despite more than 200,000 practitioners Br. J. Ophthalmol. 2012 96 6 783 787 10.1136/bjophthalmol-2011-301378 22452836 · DOI · Paper page
  2. [4]Crossland L Diabetic retinopathy screening and monitoring of early stage disease in Australian general practice: tackling preventable blindness within a chronic care model J. Diabetes Res. 2016 2016 8405395 10.1155/2016/8405395 26798655 PMC4698989 · DOI
  3. [5]Larizza MF Feasibility of screening for diabetic retinopathy at an Australian pathology collection service: a pilot study Med. J. Aust. 2013 198 97 99 10.5694/mja12.11121 23373500 · DOI
  4. [6]Scanlon PH The English national screening programme for diabetic retinopathy 2003–2016 Acta Diabetol. 2017 54 515 525 10.1007/s00592-017-0974-1 28224275 PMC5429356 · DOI
  5. [7]Foreman J Adherence to diabetic eye examination guidelines in Australia: the National Eye Health Survey Med. J. Aust. 2017 206 402 406 10.5694/mja16.00989 28490306 · DOI
  6. [8]Keel S The prevalence of diabetic retinopathy in Australian adults with self-reported diabetes: the National Eye Health Survey Ophthalmology 2017 124 977 984 10.1016/j.ophtha.2017.02.004 28318640 · DOI · Paper page
  7. [9]Ting D Diabetic retinopathy–screening and management by Australian GPs Aust. Fam. Phys. 2011 40 233 238 21597536
  8. [10]Ting DSW Development and validation of a deep learning system for diabetic retinopathy and related eye diseases using retinal images from multiethnic populations with diabetes machine learning screen for diabetic retinopathy and other eye diseases machine learning screen for diabetic retinopathy and ther eye diseases JAMA 2017 318 2211 2223 10.1001/jama.2017.18152 29234807 PMC5820739 · DOI · Paper page
  9. [11]Li Z An automated grading system for detection of vision-threatening referable diabetic retinopathy on the basis of color fundus photographs Diabetes Care 2018 41 2509 2516 10.2337/dc18-0147 30275284 · DOI · Paper page
  10. [12]Abràmoff, M. D., Lavin, P. T., Birch, M., Shah, N. & Folk, J. C. Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices. NPJ Digital Med.1, 39, doi:10.1038/s41746-018-0040-6 (2018). PMC6550188 31304320 · DOI · Paper page
  11. [13]Gargeya R Leng T Automated identification of diabetic retinopathy using deep learning Ophthalmology 2017 124 962 969 10.1016/j.ophtha.2017.02.008 28359545 · DOI
  12. [14]Topol EJ High-performance medicine: the convergence of human and artificial intelligence Nat. Med. 2019 25 44 56 10.1038/s41591-018-0300-7 30617339 · DOI · Paper page
  13. [15]Kanagasingam Y Evaluation of artificial intelligence–based grading of diabetic retinopathy in primary care JAMA Netw. Open 2018 1 e182665 e182665 10.1001/jamanetworkopen.2018.2665 30646178 PMC6324474 · DOI · Paper page
  14. [16]Heydon P Prospective evaluation of an artificial intelligence-enabled algorithm for automated diabetic retinopathy screening of 30,000 patients Br. J. Ophthalmol. 2021 105 723 728 10.1136/bjophthalmol-2020-316594 32606081 PMC8077216 · DOI
  15. [17]Zhang Y Artificial intelligence-enabled screening for diabetic retinopathy: a real-world, multicenter and prospective study BMJ Open Diabetes Res Care 2021 10.1136/bmjdrc-2020-001596 33087340 PMC7580048 · DOI
  16. [20]Keel S Feasibility and patient acceptability of a novel artificial intelligence-based screening model for diabetic retinopathy at endocrinology outpatient services: a pilot study Sci. Rep. 2018 8 4330 10.1038/s41598-018-22612-2 29531299 PMC5847544 · DOI · Paper page
  17. [21]Keel S Development and validation of a deep-learning algorithm for the detection of neovascular age-related macular degeneration from colour fundus photographs Clin. Exp. Ophthalmol. 2019 47 1009 1018 10.1111/ceo.13575 31215760 · DOI
  18. [22]Li Z Efficacy of a deep learning system for detecting glaucomatous optic neuropathy based on color fundus photographs Ophthalmology 2018 125 1199 1206 10.1016/j.ophtha.2018.01.023 29506863 · DOI · Paper page
  19. [23]Peto T Tadros C Screening for diabetic retinopathy and diabetic macular edema in the United Kingdom Curr. Diab.Rep. 2012 12 338 345 10.1007/s11892-012-0285-4 22729994 · DOI
  20. [26]Naaktgeboren CA Anticipating missing reference standard data when planning diagnostic accuracy studies BMJ 2016 352 i402 10.1136/bmj.i402 26861453 PMC4772780 · DOI
  21. [27]Cohen JF STARD 2015 guidelines for reporting diagnostic accuracy studies: explanation and elaboration BMJ Open 2016 6 e012799 10.1136/bmjopen-2016-012799 PMC5128957 28137831 · DOI · Paper page
  22. [30]Gulshan V Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs accuracy of a deep learning algorithm for detection of diabetic retinopathy accuracy of a deep learning algorithm for detection of diabetic retinopathy JAMA 2016 316 2402 2410 10.1001/jama.2016.17216 27898976 · DOI · Paper page
  23. [31]Tan Z Scheetz J He M Artificial intelligence in phthalmology: accuracy, challenges, and clinical application Asia-Pac. J. Ophthalmol. 2019 8 197 199 10.22608/apo.2019122 31179666 · DOI
  24. [32]He M Li Z Liu C Shi D Tan Z Deployment of artificial intelligence in real-world practice: opportunity and challenge Asia-Pac. J. Ophthalmol. 2020 9 299 307 10.1097/apo.0000000000000301 32694344 · DOI
  25. [33]Natarajan S Jain A Krishnan R Rogye A Sivaprasad S Diagnostic accuracy of community-based diabetic retinopathy screening with an offline artificial intelligence system on a smartphone JAMA Ophthalmol. 2019 137 1182 1188 10.1001/jamaophthalmol.2019.2923 31393538 PMC6692680 · DOI
  26. [34]Bellemo V Artificial intelligence using deep learning to screen for referable and vision-threatening diabetic retinopathy in Africa: a clinical validation study Lancet Digital Health 2019 1 e35 e44 10.1016/S2589-7500(19)30004-4 33323239 · DOI · Paper page
  27. [35]Kanski, J. J. & Bowling, B. Clinical ophthalmology: a systematic approach (Elsevier Health Sciences, 2011).
  28. [36]Nuti L The impact of interventions on appointment and clinical outcomes for individuals with diabetes: a systematic review BMC Health Serv. Res. 2015 15 355 10.1186/s12913-015-0938-5 26330299 PMC4557865 · DOI
  29. [37]Jyothi S Compliance with the quality standards of National Diabetic Retinopathy Screening Committee Prim. Care Diabetes 2009 3 67 72 10.1016/j.pcd.2009.05.005 19540187 · DOI
  30. [38]Waymel Q Badr S Demondion X Cotten A Jacques T Impact of the rise of artificial intelligence in radiology: What do radiologists think? Diagn. Interv. Imaging 2019 100 327 336 10.1016/j.diii.2019.03.015 31072803 · DOI
  31. [39]Sarwar S Physician perspectives on integration of artificial intelligence into diagnostic pathology NPJ Digital Med. 2019 2 28 10.1038/s41746-019-0106-0 PMC6550202 31304375 · DOI · Paper page
  32. [41]Scheetz J He M van Wijngaarden P Ophthalmology and the emergence of artificial intelligence Med. J. Aust. 2021 214 155 157.e151 10.5694/mja2.50932 33583047 · DOI

Authors
Scheetz, J., Koca, D., McGuinness, M., Holloway, E., Tan, Z., Zhu, Z., O’Day, R., Sandhu, S., MacIsaac, R. J., Gilfillan, C., Turner, A., Keel, S., & He, M.
Affiliations
  • The Royal Victorian Eye & Ear Hospital (AU)
  • The University of Melbourne (AU)
  • Deakin University (AU)
  • Guangdong Academy of Medical Sciences (CN)
  • St Vincent's Hospital Melbourne (AU)
  • Lions Eye Institute (AU)
Corresponding author
He, Mingguang
ORCID
Venue
Scientific Reports
Publisher
Springer Science and Business Media LLC
Published
2021-08-04
Volume
11
Issue
1
Pages
15808-15808
ISSN
2045-2322 (ISSN-L)
DOI
10.1038/s41598-021-94178-5
PMC
PMC8339059
OpenAlex
W3188146589
Cited by
88
References
38
Citations by year
2026: 16 · 2025: 29 · 2024: 17 · 2023: 18 · 2022: 7 · 2021: 1
FWCI
5.68
Type
journal article
Language
en
Open access
Licence
Locations
  • Scientific Reports (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • Figshare (submittedVersion, cc-by)
  • Minerva Access (University of Melbourne) (submittedVersion, cc-by)
Funders
Melbourne Academic Centre for Health Translational Research Fellowship
Topics
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Retinal Imaging and Analysis
  • Health Sciences › Medicine › Ophthalmology › Ophthalmology and Visual Health Research
  • Health Sciences › Medicine › Ophthalmology › Retinal Diseases and Treatments
Keywords
diabetic retinopathy, artificial intelligence, AI-assisted screening, retinal image analysis, referable diabetic retinopathy, type 1 diabetes, type 2 diabetes, endocrinology outpatient care, Aboriginal Medical Services, Indigenous health, diagnostic accuracy, user satisfaction, health professional perspectives, referral adherence, real-world implementation, Australia
MeSH terms
Adult, Aged, Artificial Intelligence, Australia, Delivery of Health Care, Diabetes Mellitus, Type 2, Diabetic Retinopathy, Endocrinology, Female, Humans, Male, Mass Screening, Medicine, Traditional, Middle Aged, Prospective Studies

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review