Cited by 116Retinal Imaging and AnalysisRetinal Diseases and TreatmentsOphthalmology and Visual Health Research

Diagnostic Accuracy of a Device for the Automated Detection of Diabetic Retinopathy in a Primary Care Setting

Open access
Verbraak, Frank D., Abramoff, Michael D., Bausch, Gonny C.F., Klaver, Caroline, Nijpels, Giel, Schlingemann, Reinier O., van der Heijden, Amber A.·2019·Diabetes Care

Abstract

OBJECTIVE: To determine the diagnostic accuracy in a real-world primary care setting of a deep learning–enhanced device for automated detection of diabetic retinopathy (DR). RESEARCH DESIGN AND METHODS: Retinal images of people with type 2 diabetes visiting a primary care screening program were graded by a hybrid deep learning–enhanced device (IDx-DR-EU-2.1; IDx, Amsterdam, the Netherlands), and its classification of retinopathy (vision-threatening [vt]DR, more than mild [mtm]DR, and mild or more [mom]DR) was compared with a reference standard. This reference standard consisted of grading according to the International Clinical Classification of DR by the Rotterdam Study reading center. We determined the diagnostic accuracy of the hybrid deep learning–enhanced device (IDx-DR-EU-2.1) against the reference standard. RESULTS: A total of 1,616 people with type 2 diabetes were imaged. The hybrid deep learning–enhanced device’s sensitivity/specificity against the reference standard was, respectively, for vtDR 100% (95% CI 77.1–100)/97.8% (95% CI 96.8–98.5) and for mtmDR 79.4% (95% CI 66.5–87.9)/93.8% (95% CI 92.1–94.9). CONCLUSIONS: The hybrid deep learning–enhanced device had high diagnostic accuracy for the detection of both vtDR (although the number of vtDR cases was low) and mtmDR in a primary care setting against an independent reading center. This allows its’ safe use in a primary care setting.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.2337/dc18-0148

Cites

35 of 35 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The IDx-DR system diagnoses referable DED, defined as ETDRS level 35 or higher, clinically significant macular edema, and/or center-involved macular edema, which requires further management or treatment by an ophthalmologist or retina specialist.

    “The system diagnoses specific levels of diabetic retinopathy and diabetic macular edema (Early Treatment of Diabetic Retinopathy Study level 35 and higher, clinically significant macular edema, and or center-involved macular edema)32,33, referred to as “referable DED”34, that requires further management or treatment by an ophthalmologist or retina specialist.”

    From Autonomous artificial intelligence increases screening and follow-up for diabetic retinopathy in youth: the ACCESS randomized control trial (2024) · cites this paper as [32]

    Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

    Experimental — accuracy not yet measured
  • Artificial intelligence algorithms have demonstrated a high sensitivity of 95% to 100% in clinical implementation, whereas specificity is relatively lower and with greater variation at 60% to 90%.

    “Artificial intelligence algorithms have demonstrated a high sensitivity of 95% to 100% in clinical implementation, whereas specificity is relatively lower and with greater variation at 60% to 90%.38,40,41,46”

    From Autonomous Artificial Intelligence in Diabetic Retinopathy Testing—Lessons Learned on Successful Health System Adoption (2026) · cites this paper as [38]

    The Results report 100% sensitivity for vision-threatening diabetic retinopathy, supporting one endpoint of the claimed sensitivity range. But sensitivity for more-than-mild disease was 79.4%, and specificity was 97.8% and 93.8% for the two outcomes—neither within the claimed 60%–90% range. The paper evaluates one device, not algorithms generally. (LLM, conf=0.90)

    Experimental — accuracy not yet measured
  • The effectiveness of autonomous AI to increase adherence and follow-up compared to traditional referral has not been evaluated in a rigorously designed randomized trial.

    “the effectiveness of autonomous AI to increase adherence and follow-up compared to traditional referral has not been evaluated in a rigorously designed randomized trial.”

    From Autonomous artificial intelligence increases screening and follow-up for diabetic retinopathy in youth: the ACCESS randomized control trial (2024) · cites this paper as [32]

    The Study Design section describes a retrospective diagnostic-accuracy study, and the Conclusions suggest the device could improve screening rates. Neither section addresses whether a randomized trial has evaluated adherence or follow-up against traditional referral; searches for those outcomes and a randomized comparison found no such passage. The fact that this study was not randomized does not establish that no such trial existed. (LLM, conf=0.75)

    Experimental — accuracy not yet measured
  • Only eight ophthalmology studies mentioned adherence to STARD-2015 guidelines or had a STARD flow diagram in the manuscript.

    “Only eight studies in ophthalmology imaging14,21,32,33,43,55,108,109, ten studies in respiratory imaging64,66,70,72,75,79,82,87,89,110 and six studies in breast imaging62,91,97,104,106,111 mentioned adherence to the STARD-2015 guidelines or had a STARD flow diagram in the manuscript.”

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 3

  1. Scanlon PH. The English national screening programme for sight-threatening diabetic reti- nopathy. J Med Screen 2008;15:1-4
  2. Lynch S, Abramoff MD. Catastrophic failure in image-based convolutional neural network algorithms for detecting diabetic retinopathy (Abstract). IOVS 2017;58:A3776
  3. Owens DR, Gibbins RL, Lewis PA, Wall S, Allen JC, Morton R. Screening for diabetic ret- inopathy by general practitioners: ophthalmos- copy or retinal photography as 35 mm colour transparencies? Diabet Med 1998;15: 170-175

Cited by

Sorted by citations

Authors
Verbraak, F. D., Abramoff, M. D., Bausch, G. C. F., Klaver, C., Nijpels, G., Schlingemann, R. O., & van der Heijden, A. A.
Affiliations
  • Amsterdam UMC Location Vrije Universiteit Amsterdam (NL)
  • Department of Ophthalmology and Visual Sciences, University of Iowa Hospital & Clinics, Iowa City, IA
  • Star-SHL, Rotterdam, the Netherlands
  • Erasmus MC (NL)
  • Amsterdam Public Health (NL)
  • Amsterdam University Medical Centers (NL)
  • Department of General Practice and Elderly Care Medicine, Amsterdam Public Health Research Institute, VU University Medical Center, Amsterdam, the Netherlands
Corresponding author
Verbraak, Frank D.
Venue
Diabetes Care
Published
2019-02-14
Volume
42
Issue
4
Pages
651-656
ISSN
  • 0149-5992 (ISSN-L)
  • 1935-5548
OpenAlex
W2912100147
Cited by
115
References
35
Citations by year
2026: 10 · 2025: 22 · 2024: 9 · 2023: 14 · 2022: 14 · 2021: 21 · 2020: 19 · 2019: 6
FWCI
9.98
Type
journal article
Language
en
Open access
Locations
  • Diabetes Care (publishedVersion)
  • PubMed (publishedVersion)
  • Pure Amsterdam UMC (acceptedVersion, other-oa)
  • Data Archiving and Networked Services (DANS) (submittedVersion)
  • EUR Research Repository (Erasmus University Rotterdam) (submittedVersion)
  • Amsterdam UMC (VU Amsterdam) - Institutional Repository (submittedVersion)
  • Radboud Repository (Radboud University) (submittedVersion)
Funders
  • U.S. Department of Veterans Affairs
  • Research to Prevent Blindness
  • European Foundation for the Study of Diabetes
  • Iowa City Veterans Affairs Medical Center
Topics
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Retinal Imaging and Analysis
  • Health Sciences › Medicine › Ophthalmology › Retinal Diseases and Treatments
  • Health Sciences › Medicine › Ophthalmology › Ophthalmology and Visual Health Research
Keywords
diabetic retinopathy, automated retinopathy detection, deep learning, type 2 diabetes, primary care screening, diagnostic accuracy, vision-threatening diabetic retinopathy
MeSH terms
Diabetes Mellitus, Type 2, Diabetic Retinopathy, Female, Humans, Male, Mass Screening, Middle Aged, Netherlands, Primary Health Care

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review