CoChat
Sign Up
Cited by 256Retinal Imaging and AnalysisRetinal Diseases and TreatmentsOphthalmology and Visual Health Research

Pivotal Evaluation of an Artificial Intelligence System for Autonomous Detection of Referrable and Vision-Threatening Diabetic Retinopathy

Open access
Ipp, Eli, Liljenquist, David, Bode, Bruce, Shah, Viral N., Silverstein, Steven, Regillo, Carl D., Lim, Jennifer I., Sadda, SriniVas +31 more·2021·JAMA Network Open
DOI 10.1001/jamanetworkopen.2021.34254PMC PMC8593763
Open this paper in CoChat — free accountOpen PDF

Abstract

Importance: Diabetic retinopathy (DR) is a leading cause of blindness in adults worldwide. Early detection and intervention can prevent blindness; however, many patients do not receive their recommended annual diabetic eye examinations, primarily owing to limited access. Objective: To evaluate the safety and accuracy of an artificial intelligence (AI) system (the EyeArt Automated DR Detection System, version 2.1.0) in detecting both more-than-mild diabetic retinopathy (mtmDR) and vision-threatening diabetic retinopathy (vtDR). Design, Setting, and Participants: A prospective multicenter cross-sectional diagnostic study was preregistered (NCT03112005) and conducted from April 17, 2017, to May 30, 2018. A total of 942 individuals aged 18 years or older who had diabetes gave consent to participate at 15 primary care and eye care facilities. Data analysis was performed from February 14 to July 10, 2019. Interventions: Retinal imaging for the autonomous AI system and Early Treatment Diabetic Retinopathy Study (ETDRS) reference standard determination. Main Outcomes and Measures: Primary outcome measures included the sensitivity and specificity of the AI system in identifying participants' eyes with mtmDR and/or vtDR by 2-field undilated fundus photography vs a rigorous clinical reference standard comprising reading center grading of 4 wide-field dilated images using the ETDRS severity scale. Secondary outcome measures included the evaluation of imageability, dilated-if-needed analysis, enrichment correction analysis, worst-case imputation, and safety outcomes. Results: Of 942 consenting individuals, 893 patients (1786 eyes) met the inclusion criteria and completed the study protocol. The population included 449 men (50.3%). Mean (SD) participant age was 53.9 (15.2) years (median, 56; range, 18-88 years), 655 were White (73.3%), and 206 had type 1 diabetes (23.1%). Sensitivity and specificity of the AI system were high in detecting mtmDR (sensitivity: 95.5%; 95% CI, 92.4%-98.5% and specificity: 85.0%; 95% CI, 82.6%-87.4%) and vtDR (sensitivity: 95.1%; 95% CI, 90.1%-100% and specificity: 89.0%; 95% CI, 87.0%-91.1%) without dilation. Imageability was high without dilation, with the AI system able to grade 87.4% (95% CI, 85.2%-89.6%) of the eyes with reading center grades. When eyes with ungradable results were dilated per the protocol, the imageability improved to 97.4% (95% CI, 96.4%-98.5%), with the sensitivity and specificity being similar. After correcting for enrichment, the mtmDR specificity increased to 87.8% (95% CI, 86.3%-89.5%) and the sensitivity remained similar; for vtDR, both sensitivity (97.0%; 95% CI, 91.2%-100%) and specificity (90.1%; 95% CI, 89.4%-91.5%) improved. Conclusions and Relevance: This prospective multicenter cross-sectional diagnostic study noted safety and accuracy with use of the EyeArt Automated DR Detection System in detecting both mtmDR and, for the first time, vtDR, without physician assistance. These findings suggest that improved access to accurate, reliable diabetic eye examinations may increase adherence to recommended annual screenings and allow for accelerated referral of patients identified as having vtDR.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1001/jamanetworkopen.2021.34254

Cites

28 of 28 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The autonomous AI platform used (EyeArt by Eyenuk) was Health Canada-approved.

    “Images were uploaded to a Health Canada-approved autonomous AI platform, EyeArt by Eyenuk. (Woodland Hills, California, USA),18”

    From Screening for diabetic retinopathy with artificial intelligence in a primary care setting: a comparative cost analysis (2026) · cites this paper as [18]

    The Introduction identifies EyeArt as an Eyenuk system, and the Methods describe it as an autonomous AI system. The Introduction says EyeArt is FDA-cleared, but our copy does not state that it was approved by Health Canada; the approval claim is unsupported. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

  • The effectiveness of autonomous AI to increase adherence and follow-up compared to traditional referral has not been evaluated in a rigorously designed randomized trial.

    “the effectiveness of autonomous AI to increase adherence and follow-up compared to traditional referral has not been evaluated in a rigorously designed randomized trial.”

    From Autonomous artificial intelligence increases screening and follow-up for diabetic retinopathy in youth: the ACCESS randomized control trial (2024) · cites this paper as [25]

    The Methods and Outcome Measures sections (pages not provided) describe this paper as a cross-sectional diagnostic study measuring detection accuracy, not a randomized trial of adherence or follow-up. The Discussion suggests AI screening may improve adherence and enable prompt referrals, but our copy does not say whether any randomized trial has compared those outcomes with traditional referral. Searched for randomized trials, adherence, follow-up, and traditional referral. (LLM, conf=0.74)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

1 of the 29 references couldn’t be matched to a known paper.

Cited by

Sorted by citations

Related

Authors
Ipp, E., Liljenquist, D., Bode, B., Shah, V. N., Silverstein, S., Regillo, C. D., Lim, J. I., Sadda, S., Domalpally, A., Gray, G., Bhaskaranand, M., Ramachandra, C., Solanki, K., EyeArt Study Group, DuBiner, H. B., Genter, P., Graham, J., Johnson, A., Levy-Clarke, G., Pesavento, R. D., Sherman, M. D., Kim, B. T., Walman, G. B., Akturk, H. K., Joseph, H., Joshee, P., Trippe, B., Gilbert, J. M., Blodi, B. A., Reed, S., Reimers, J., Lang, K., Cohn, H., Shaw, R., Watson, S., Ewen, A., Barrett, N., Swift, M., & Gornbein, J.
Affiliations
  • University of California, Los Angeles (US)
  • Rocky Mountain Diabetes and Osteoporosis Center (US)
  • Atlanta Diabetes Associates (US)
  • University of Colorado Anschutz (US)
  • Kansas City Public Schools (US)
  • Thomas Jefferson University (US)
  • University of Illinois Chicago (US)
  • Doheny Eye Center, Arcadia, California
  • University of Wisconsin–Madison (US)
  • Data-Fi, LLC, Silver Spring, Maryland
  • Eyenuk (United States) (US)
  • for the EyeArt Study Group
Corresponding author
Ipp, Eli
ORCID
Venue
JAMA Network Open
Publisher
American Medical Association (AMA)
Published
2021-11-15
Volume
4
Issue
11
Pages
e2134254
ISSN
2574-3805 (ISSN-L)
DOI
10.1001/jamanetworkopen.2021.34254
PMC
PMC8593763
OpenAlex
W3213686666
Cited by
256
References
29
Citations by year
2026: 62 · 2025: 64 · 2024: 60 · 2023: 49 · 2022: 19 · 2021: 2
FWCI
17.17
Type
journal article
Language
en
Open access
Licence
cc-by-nc-nd
Locations
  • JAMA Network Open (publishedVersion, cc-by-nc-nd)
  • PubMed Central (publishedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
Funders
University of Wisconsin-Madison
Topics
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Retinal Imaging and Analysis
  • Health Sciences › Medicine › Ophthalmology › Retinal Diseases and Treatments
  • Health Sciences › Medicine › Ophthalmology › Ophthalmology and Visual Health Research
Keywords
diabetic retinopathy, artificial intelligence, automated retinal analysis, vision-threatening diabetic retinopathy, retinal fundus photography, Early Treatment Diabetic Retinopathy Study, diagnostic accuracy, sensitivity and specificity, imageability, blindness prevention, primary care
MeSH terms
Adolescent, Adult, Aged, Aged, 80 and over, Artificial Intelligence, Cross-Sectional Studies, Diabetic Retinopathy, Female, Humans, Male, Middle Aged, Prospective Studies, Reference Standards, Referral and Consultation, Sensitivity and Specificity, Vision Disorders, Vision Screening, Young Adult

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review