Integrated image-based deep learning and language models for primary diabetes care

The included studies had two distinct status quo care pathways: routine referrals for people with diabetes without prior screening and manual grading workflows including asynchronous teleophthalmology.

“It should be noted there were two distinct, status quo care pathways: studies providing routine referrals for people with diabetes without prior screening13,17 and manual grading workflows including asynchronous teleophthalmology14–16,18.”

Studies comparing AI to routine referral approaches with lower baseline rates (18.7-22.0%) showed the largest effect sizes (+36.7% to +42.0%), while studies comparing AI to existing manual screening workflows with higher baseline rates (39.6-77.3%) showed more modest improvements (+11.8% to +19.4%).

“Studies comparing AI to routine referral approaches with typically lower baseline rates of uptake (18.7–22.0%) showed the largest effect sizes (+36.7% to +42.0%)13,17, while studies comparing AI to existing manual screening workflows with higher baseline rates of uptake (39.6–77.3%) showed more modest improvements (+11.8% to +19.4%)14,16,18.”

Findings from the review suggest the status quo for DR screening involves asynchronous teleophthalmology with delayed feedback of results or no prior screening as was the case in two North American studies by Wolf et al. and Liu et al.

“However, findings from our review suggest the status quo for DR screening involves asynchronous teleophthalmology with delayed feedback of results14–16,18 or no prior screening as was the case in two North American studies by Wolf et al. and Liu et al.13,17.”

Real-world implementation of diabetic retinopathy screening with artificial intelligence-assisted care pathways has accelerated globally since 2020.

“Real-world implementation of DR screening with AI-assisted care pathways has accelerated globally since 20208,13–22, yet evidence on clinical and health system outcomes such as referral uptake remains limited.”

Several studies included co-interventions alongside AI-assisted diabetic retinopathy screening with variable effectiveness.

“Several studies included co-interventions alongside AI-assisted DR screening with variable effectiveness13,16–18.”

Li et al.'s sequential cohort study (n=298) found 77.8% versus 58.4% referral uptake within two weeks for AI-assisted screening with treatment advice based on multimodal patient data and retinal images versus unassisted manual grading (+19.4%, p=0.001).

“Li et al.‘s sequential cohort study (n = 298) compared AI-assisted screening with treatment advice based upon multimodal patient data (medical history, physical examinations, laboratory tests) and retinal images, to unassisted manual grading, achieving 77.8% versus 58.4% referral uptake within two weeks (+19.4%, p = 0.001)18.”

Li et al. provided patients with an AI generated screening report providing treatment advice which the review classed as education.

“Li et al. provided patients with an AI generated screening report providing treatment advice which we classed as education18.”

Studies comparing AI-assisted screening to existing manual grading workflows showed modest gains ranging from +11.8 to +23.5%.

“Conversely, studies comparing AI-assisted screening to existing manual grading workflows showed modest gains ranging from +11.8 to +23.5%14–16,18.”

In higher-performing systems (baseline referral uptake 39.6-77.3%), the gains from AI-assisted screening compared to manual grading were modest.

“This was most pronounced in in higher-performing systems (baseline referral uptake 39.6–77.3%)14,16,18 which indicates diminishing returns in settings with already functional screening infrastructure.”

Provision of personalised reports supports reflective motivation.

“provision of personalised reports supports reflective motivation18”

Two studies that replaced routine referrals with targeted, AI-assisted referrals showed greater effect sizes than studies with a manual grading comparator.

“Firstly, comparator types whereby two studies replaced routine referrals with targeted, AI-assisted referrals13,17 showed greater effect sizes than those studies with a manual grading comparator14–16,18.”

Two studies excluded ungradable images from analysis entirely, prior to allocation to AI or comparator arms.

“Two studies excluded ungradable images from analysis entirely, prior to allocation to AI or comparator arms14,18.”

Referral appointment timeframes varied from two weeks to twelve months and definitions of referable DR differed.

“Referral appointment timeframes varied from two weeks18 to twelve months17 and definitions of referable DR differed.”

LLMs are transforming patient care by enabling more efficient and accurate clinical decision-making.

“Additionally, LLMs are transforming patient care by enabling more efficient and accurate clinical decision-making (89).”

There was marked variation in timeframes for referral among included studies, ranging from 2 weeks for Li et al. in China to 12 months for Liu et al. in the USA.

“There was marked variation for timeframes among included studies ranging from 2 weeks for Li et al. in China18 up to 12 months for Liu et al. in the USA17.”

All studies consistently demonstrate improved referral uptake after AI-assisted DR screening.

“While all studies consistently demonstrate improved referral uptake after AI-assisted DR screening13–18, magnitude of benefit varies substantially from moderate improvements ( + 11.8%)16 to large effects ( + 42.0%)13.”

Li et al., despite allowing just a fortnight for appointments, reported referral uptake of 58.4% and 77.8% for comparator and intervention groups, respectively.

“For example, Li et al., despite allowing just a fortnight for appointments reported referral uptake of 58.4% and 77.8% for comparator and intervention groups, respectively18.”

Co-intervention usage varied across studies, with some utilising automated reminders, scheduling calls, or providing patient education alongside AI screening.

“Secondly, co-intervention usage varied across studies, with some utilising automated reminders16, scheduling calls17, or providing patient education alongside AI screening13,18.”

Li et al. developed DeepDR-LLM, a multimodal vision-language system for DR diagnosis and management, enabling multimodal generation of diagnostic and treatment recommendations and providing support for personalized diabetes care.

“Li et al. [135] developed DeepDR-LLM, a multimodal vision-language system for DR diagnosis and management, enabling multimodal generation of diagnostic and treatment recommendations and providing support for personalized diabetes care.”

Li et al. showed the most robust quality with low-to-moderate risk across domains, limited primarily by lack of sensitivity analyses.

“Li et al.18 showed the most robust quality with low-to-moderate risk across domains, limited primarily by lack of sensitivity analyses.”

Baseline health system performance influenced the marginal gains, with settings with higher baseline rates of uptake experiencing relatively lower effect size, in contrast to lower-performing systems with greater margins for improvement.

“Thirdly, baseline health system performance appeared to influence the marginal gains, following a law of diminishing returns whereby settings with higher baseline rates of uptake experienced relatively lower effect size14,16,18, in contrast to lower-performing systems with greater margins for improvement13,15,17.”