Cited by 16Retinal Imaging and AnalysisArtificial Intelligence in Healthcare and EducationOphthalmology and Visual Health Research

Determinants for scalable adoption of autonomous AI in the detection of diabetic eye disease in diverse practice types: key best practices learned through collection of real-world data

Open access
Goldstein, Juli, Weitzman, Dena, Lemerond, Meghan, Jones, Andrew·2023·Frontiers in Digital Health

Abstract

Autonomous Artificial Intelligence (AI) has the potential to reduce disparities, improve quality of care, and reduce cost by improving access to specialty diagnoses at the point-of-care. Diabetes and related complications represent a significant source of health disparities. Vision loss is a complication of diabetes, and there is extensive evidence supporting annual eye exams for prevention. Prior to the use of autonomous AI, store-and-forward imaging approaches using remote reading centers (asynchronous telemedicine) attempted to increase diabetes related eye exams with limited success. In 2018, after rigorous clinical validation, the first fully autonomous AI system [LumineticsCore™ (formerly IDx-DR), Digital Diagnostics Inc., Coralville, IA, United States] received U.S. Food and Drug Administration (FDA) De Novo authorization. The system diagnoses diabetic retinopathy (including macular edema) without specialist physician overread at the point-of-care. In addition to regulatory clearance, reimbursement, and quality measure updates, successful adoption requires local optimization of the clinical workflow. The general challenges of frontline care clinical workflow have been well documented in the literature. Because healthcare AI is so new, there remains a gap in the literature about challenges and opportunities to embed diagnostic AI into the clinical workflow. The goal of this review is to identify common workflow themes leading to successful adoption, measured as attainment number of exams per month using the autonomous AI system against targets set for each health center. We characterized the workflow in four different US health centers over a 12-month period. Health centers were geographically dispersed across the Midwest, Southwest, Northeast, and West Coast and varied distinctly in terms of size, staffing, resources, financing and demographics of patient populations. After 1 year, the aggregated number of diabetes-related exams per month increased from 89 after the first month of initial deployment to 174 across all sites. Across the diverse practice types, three primary determinants underscored sustainable adoption: (1) Inclusion of Executive and Clinical Champions; (2) Underlining Health Center Resources; and (3) Clinical workflows that contemplate patient identification (pre-visit), LumineticsCore Exam Capture and Provider Consult (patient visit), and Timely Referral Triage (post-visit). In addition to regulatory clearance, reimbursement and quality measures, our review shows that addressing the core determinants for workflow optimization is an essential part of large-scale adoption of innovation. These best practices can be generalizable to other autonomous AI systems in front-line care settings, thereby increasing patient access, improving quality of care, and addressing health disparities.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3389/fdgth.2023.1004130

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Liu et al found that among 180 study participants who underwent nonmydriatic fundus photography followed by automated retinal image analysis with human supervision, those referred for follow-up ophthalmic evaluation had an adherence rate of 55.4% at 1 year, compared with the historical adherence rate of 18.7%.

    “Liu et al39 found that among 180 study participants who underwent nonmydriatic fundus photography followed by automated retinal image analysis with human supervision, those referred for follow-up ophthalmic evaluation had an adherence rate of 55.4% at 1 year, compared with the historical adherence rate of 18.7%.”

    From Autonomous Artificial Intelligence in Diabetic Retinopathy Testing—Lessons Learned on Successful Health System Adoption (2026) · cites this paper as [39]

    Split into 2 atoms — 0 confirmed, 0 partially supported, 0 disputed, 0 misattributed, 2 unverifiable. Expand the claim to see per-atom verdicts.

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 19

  1. [2]Char DS Abràmoff MD Feudtner C. Identifying ethical considerations for machine learning healthcare applications. Am J Bioeth. (2020) 20(11): 7– 17. 10.1080/15265161.2020.1819469 33103967 PMC7737650 · DOI · Paper page
  2. [3]Abramoff MD Cunningham B Patel B Eydelman MB Leng T Sakamoto T Foundational considerations for artificial intelligence using ophthalmic images. Ophthalmology. (2022) 129(2): e14– 32. 10.1016/j.ophtha.2021.08.023 34478784 PMC9175066 · DOI · Paper page
  3. [4]Rhyu J Lambrechts S Han MA Freeby MJ. Utilizing point-of-care A1c to impact outcomes - can we make it happen in primary care? Curr Opin Endocrinol Diabetes Obes. (2022) 29(1): 29– 33. 10.1097/MED.0000000000000700 34889878 · DOI
  4. [6]Parikh RB Teeple S Navathe AS. Addressing bias in artificial intelligence in health care. J Am Med Assoc. (2019) 322(24): 2377– 8. 10.1001/jama.2019.18058 31755905 · DOI · Paper page
  5. [7]Camacho J Zanoletti-Mannello M Landis-Lewis Z Kane-Gill SL Boyce RD. A conceptual framework to study the implementation of clinical decision support systems (BEAR): literature review and concept mapping. J Med Internet Res. (2020) 22(8): e18388. 10.2196/18388 32759098 PMC7441385 · DOI
  6. [8]Holman GT Beasley JW Karsh BT Stone JA Smith PD Wetterneck TB. The myth of standardized workflow in primary care. J Am Med Inform Assoc. (2016) 23(1): 29– 37. 10.1093/jamia/ocv107 26335987 PMC5009941 · DOI
  7. [9]Centers for Medicare and Medicaid Services, Department of Health and Human Services. Medicare program; CY 2022 payment policies under the physician fee schedule and other changes to part B payment policies; medicare shared savings program requirements; provider enrollment regulation updates; and provider and supplier prepayment and post-payment medical review requirements. Fed Regist. (2021) 86(221): 64996– 6031.
  8. [10]Abramoff MD Roehrenbeck C Trujillo S Goldstein J Graves AS Repka MX A reimbursement framework for artificial intelligence in healthcare. Nat Digit Med. (2022) 5(1):72. 10.1038/s41746-022-00621-w. PMC9184542 35681002 · DOI · Paper page
  9. [12]Centers for Medicare and Medicaid Services, Department of Health and Human Services. Medicare program; CY 2022 payment policies under the physician fee schedule and other changes to part B payment policies; medicare shared savings program requirements; provider enrollment regulation updates; and provider and supplier prepayment and post-payment medical review requirements. BILLING CODE 4120–01–C. e. Establishment of values for remote retinal imaging (CPT code 92229), comment solicitation for fractional flow reserve derived from computed tomography (CPT code 0503T), and comment solicitation for codes involving innovative technology. Fed Regist. (2021) 86(221): 64996– 6031.
  10. [15]American Diabetes Association. 11. Microvascular complications and foot care: standards of medical care in diabetes-2020. Diabetes Care. (2020) 43(Suppl 1): S135– 51. 10.2337/dc20-S011 31862754 · DOI · Paper page
  11. [17]Lundeen EA Wittenborn J Benoit SR Saaddine J. Disparities in receipt of eye exams among medicare part B fee-for-service beneficiaries with diabetes - United States, 2017. Morb Mortal Wkly Rep. (2019) 68(45): 1020– 3. 10.15585/mmwr.mm6845a3 PMC6855512 31725705 · DOI
  12. [18]Bodenheimer T Laing BY. The teamlet model of primary care. Ann Fam Med. (2007) 5(5): 457– 61. 10.1370/afm.731 17893389 PMC2000308 · DOI · Paper page
  13. [20]Lorenz A Oza-Frank R May S Conrey EJ Panchal B Brill SB A quality improvement collaborative increased preventive education and screening rates for women at high-risk for type 2 diabetes mellitus in primary care settings. Prim Care Diabetes. (2020) 14(4): 335– 42. 10.1016/j.pcd.2019.09.010 31706949 · DOI
  14. [21]Hans PK Gray CS Gill A Tiessen J. The provider perspective: investigating the effect of the electronic patient-reported outcome (ePRO) mobile application and portal on primary care provider workflow. Prim Health Care Res Dev. (2018) 19(2): 151– 64. 10.1017/S1463423617000573 28899449 PMC6452954 · DOI
  15. [22]Robinson J Porter M Montalvo Y Peden CJ. Losing the wait: improving patient cycle time in primary care. BMJ Open Qual. (2020) 9(2): e000910. 10.1136/bmjoq-2019-000910 32381596 PMC7223280 · DOI
  16. [23]Hung DY Truong QA Liang SY. Implementing lean quality improvement in primary care: impact on efficiency in performing common clinical tasks. J Gen Intern Med. (2021) 36(2): 274– 9. 10.1007/s11606-020-06317-9 33236228 PMC7878610 · DOI
  17. [24]Bhyat R Hagens S Bryski K Kohlmaier JF. Digital health value realization through active change efforts. Front Public Health. (2021) 9: 741424. 10.3389/fpubh.2021.741424 34712641 PMC8547540 · DOI
  18. [25]Konttila J Siira H Kyngäs H Lahtinen M Elo S Kääriäinen M Healthcare professionals’ competence in digitalisation: a systematic review. J Clin Nurs. (2019) 28(5-6): 745– 61. 10.1111/jocn.14710 30376199 · DOI · Paper page
  19. [26]Kruse CS Goetz K. Summary and frequency of barriers to adoption of CPOE in the U.S. J Med Syst. (2015) 39(2): 15. 10.1007/s10916-015-0198-2 25638719 PMC4312581 · DOI

Cited by

Sorted by citations

Authors
Goldstein, J., Weitzman, D., Lemerond, M., & Jones, A.
Corresponding author
Goldstein, Juli
Venue
Frontiers in Digital Health
Published
2023-05-18
Volume
5
Pages
1004130-1004130
ISSN
2673-253X (ISSN-L)
OpenAlex
W4377107951
Cited by
16
References
2
Citations by year
2026: 5 · 2025: 8 · 2024: 3
FWCI
2.65
Type
journal article
Language
en
Open access
Locations
  • Frontiers in Digital Health (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Retinal Imaging and Analysis
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Health Sciences › Medicine › Ophthalmology › Ophthalmology and Visual Health Research
Keywords
autonomous AI systems, diabetic retinopathy, diabetic eye disease, clinical workflow optimization, artificial intelligence adoption, health centers, health disparities, point-of-care diagnosis, referral triage, patient identification, quality metrics, real-world data, diabetes complications, blindness prevention, primary care settings

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review