Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 56 of the 64 references it cites were found in a registry or an index; 30 of the 67 sources its claims cite were checked for support.

Economic evaluation for medical artificial intelligence: accuracy vs. cost-effectiveness in a diabetic retinopathy screening case

Open access

Abstract

Artificial intelligence (AI) models have shown great accuracy in health screening. However, for real-world implementation, high accuracy may not guarantee cost-effectiveness. Improving AI’s sensitivity finds more high-risk patients but may raise medical costs while increasing specificity reduces unnecessary referrals but may weaken detection capability. To evaluate the trade-off between AI model performance and the long-running cost-effectiveness, we conducted a cost-effectiveness analysis in a nationwide diabetic retinopathy (DR) screening program in China, comprising 251,535 participants with diabetes over 30 years. We tested a validated AI model in 1100 different diagnostic performances (presented as sensitivity/specificity pairs) and modeled annual screening scenarios. The status quo was defined as the scenario with the most accurate AI performance. The incremental cost-effectiveness ratio (ICER) was calculated for other scenarios against the status quo as cost-effectiveness metrics. Compared to the status quo (sensitivity/specificity: 93.3%/87.7%), six scenarios were cost-saving and seven were cost-effective. To achieve cost-saving or cost-effective, the AI model should reach a minimum sensitivity of 88.2% and specificity of 80.4%. The most cost-effective AI model exhibited higher sensitivity (96.3%) and lower specificity (80.4%) than the status quo. In settings with higher DR prevalence and willingness-to-pay levels, the AI needed higher sensitivity for optimal cost-effectiveness. Urban regions and younger patient groups also required higher sensitivity in AI-based screening. In real-world DR screening, the most accurate AI model may not be the most cost-effective. Cost-effectiveness should be independently evaluated, which is most likely to be affected by the AI’s sensitivity.

Overview

CoChat Summary

TL;DR

In a modeled cost-effectiveness analysis of AI-based diabetic retinopathy screening using 251,535 Chinese participants over 30 years, the most accurate AI (sensitivity/specificity 93.3%/87.7%) was not the most cost-effective. Cost-saving or cost-effective screening required minimum sensitivity of 88.2% and specificity of 80.4%, and the best cost-effective scenario had higher sensitivity (96.3%) and lower specificity (80.4%) than the status quo.

Key findings

  • Among 1100 AI performance scenarios, 6 were cost-saving and 7 were cost-effective compared with the status quo; the minimum thresholds were 88.2% sensitivity (paired specificity 90.3%) and 80.4% specificity (paired sensitivity 96.3%).
  • The most cost-saving scenario (88.2%/90.3%) saved US$5.54 million but compromised 1,490 QALYs versus the status quo; the best cost-effective scenario (96.3%/80.4%) cost an additional US$14.8 million and gained 839 QALYs in the 251,535 population over 30 years.
  • Higher referable DR prevalence and higher willingness-to-pay required higher AI sensitivity. At 4%, 8%, and 12% prevalence, the best cost-effective scenario had sensitivity/specificity of 95.8%/82.4%, 96.3%/80.4%, and 96.6%/79.2%, with additional costs of 11–16 million and extra 662–974 QALYs.
  • In subgroup analyses, urban settings required sensitivity 96.9% (specificity 77.5%) and rural settings required sensitivity 94.7% (specificity 85.6%); at the 3-time per-capita GDP WTP level (US$30,828), the best cost-effective AI had a 55.43% probability of being the dominant choice.

Methods

  • Simulation using data from the Lifeline Express DR Screening Program, a nationwide program in China that enrolled 251,535 participants with diabetes from 2014 to 2019 and collected 865,152 color fundus images; referable DR was detected in 18,709 (7.44%) participants by human graders.
  • A hybrid decision tree/Markov model from a societal perspective modeled annual AI-based DR screening over 30 1-year cycles in a population with mean age 60; 1100 sensitivity/specificity pairs were generated by threshold adjustment on a validated AI model, with the status quo defined as the maximum-AUC scenario.
  • ICERs were calculated against the status quo and reference scenarios, with WTP set at three times per-capita GDP (US$30,828); analyses included net monetary benefit, univariate and probabilistic sensitivity analyses (10,000 Monte Carlo simulations), and subgroup analyses by rural/urban setting, age group, and referable DR prevalence (4%, 8%, 12%).

Limitations

  • The paper states that only one AI model was used to classify referable DR, although 1100 sensitivity/specificity pairs were simulated; cost-effectiveness was based on China’s health economic characteristics and may not be directly generalizable; and the study did not consider diabetic macular edema or impaired visual acuity, modeling only a simple natural DR progression.
  • Text-evident limitations include reliance on a cross-sectional screening program without follow-up, so transition probabilities were derived from prior studies and assumptions; referral and treatment compliance were assumed at 50% and 70%, respectively; and ungradable images were reclassified as positive, which lowered the AI’s specificity relative to prior validation.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1038/s41746-024-01032-9

Cites

56 of 56 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Fig. 1 Cost-effective AI-based DR screening scenarios. AI artificial intelligence, DR diabetic retinopathy, BCES best cost-effective scenario, GDP gross domestic product. Circles represent different screening scenarios, while stars represent the status quo and rhombus represent BCES. a Under a 7.44% prevalence of referable DR and a willingness-to-pay level of 3 times per capita GDP (US$ 30,828), the status quo, 6 cost-saving and 7 cost-effective screening scenarios were identified and ranked according to ascending costs. b A higher rank stands for a higher effectiveness. The best cost-effective scenario (rank 14) was identified with a sensitivity of 96.3% and a specificity of 80.4%. Performance of the AI with a large decrease in specificity but a minor increase in sensitivity can still benefit cost-effectiveness.
Fig. 1 Cost-effective AI-based DR screening scenarios. AI artificial intelligence, DR diabetic retinopathy, BCES best cost-effective scenario, GDP gross domestic product. Circles represent different screening scenarios, while stars represent the status quo and rhombus represent BCES. a Under a 7.44% prevalence of referable DR and a willingness-to-pay level of 3 times per capita GDP (US$ 30,828), the status quo, 6 cost-saving and 7 cost-effective screening scenarios were identified and ranked according to ascending costs. b A higher rank stands for a higher effectiveness. The best cost-effective scenario (rank 14) was identified with a sensitivity of 96.3% and a specificity of 80.4%. Performance of the AI with a large decrease in specificity but a minor increase in sensitivity can still benefit cost-effectiveness.CC BY · PubMed Central
Fig. 2 Performance of the AI in best cost-effective scenarios across WTP levels and prevalence of referable DR. AI artificial intelligence, WTP willingness-to-pay, DR diabetic retinopathy, Prev. prevalence, Sen. sensitivity, Spe. specificity, ICER incremental cost-effectiveness ratio. The sensitivity and specificity of the best cost-effective AI were modeled across WTP levels from US$ 0 to 30,828, and at different prevalences of referable DR (4%, 8%, and 12%). In each scenario, the sensitivity and specificity with the best cost-effective performance (ICER lower than WTP level and providing the highest effect) were used in this figure. Red-yellow lines indicate sensitivity and blue-green lines represent corresponding specificity. A darker color represents a higher prevalence of referable DR.
Fig. 2 Performance of the AI in best cost-effective scenarios across WTP levels and prevalence of referable DR. AI artificial intelligence, WTP willingness-to-pay, DR diabetic retinopathy, Prev. prevalence, Sen. sensitivity, Spe. specificity, ICER incremental cost-effectiveness ratio. The sensitivity and specificity of the best cost-effective AI were modeled across WTP levels from US$ 0 to 30,828, and at different prevalences of referable DR (4%, 8%, and 12%). In each scenario, the sensitivity and specificity with the best cost-effective performance (ICER lower than WTP level and providing the highest effect) were used in this figure. Red-yellow lines indicate sensitivity and blue-green lines represent corresponding specificity. A darker color represents a higher prevalence of referable DR.CC BY · PubMed Central
Fig. 3 Workflow of annual AI-based DR screening. AI artificial intelligence, DR diabetic retinopathy, FP false positive, TN true negative, TP true positive, FN false negative. The cost-effectiveness analysis was based on a nationwide DR screening program (the Lifeline Express Program) comprising 251,535 individuals with diabetes. In this cohort, participants accepted annual DR screening using a previously validated AI. Participants classified as referable DR or ungradable were suggested referral to ophthalmologists, and those with referable DR diagnosed by ophthalmologists would be suggested for appropriate treatment. Participants can choose to accept or refuse these suggestions. Participants with no DR, non-referable DR and untreated referable DR will follow the natural DR progression process without treatment, while those who accepted treatment were estimated to continue treatment in subsequent years without entering the following cycles of DR screening. Those who refused DR treatment, and those already blind would also not enter DR screening in the subsequent years. Individuals were estimated to have a risk of death in any state.
Fig. 3 Workflow of annual AI-based DR screening. AI artificial intelligence, DR diabetic retinopathy, FP false positive, TN true negative, TP true positive, FN false negative. The cost-effectiveness analysis was based on a nationwide DR screening program (the Lifeline Express Program) comprising 251,535 individuals with diabetes. In this cohort, participants accepted annual DR screening using a previously validated AI. Participants classified as referable DR or ungradable were suggested referral to ophthalmologists, and those with referable DR diagnosed by ophthalmologists would be suggested for appropriate treatment. Participants can choose to accept or refuse these suggestions. Participants with no DR, non-referable DR and untreated referable DR will follow the natural DR progression process without treatment, while those who accepted treatment were estimated to continue treatment in subsequent years without entering the following cycles of DR screening. Those who refused DR treatment, and those already blind would also not enter DR screening in the subsequent years. Individuals were estimated to have a risk of death in any state.CC BY · PubMed Central

Verification

Claims the paper makes

15 Valid · 4 Not valid · 7 Not checked · 57 Own results and interpretations

45 more claims · 5 with their cited sources checked

Create a free account to see all 83 claims and every source verdict.

See all claims — free account
  • It may state two separate claims as one. 22
  • Probably not a substantial claim. 14

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

Show all 64 references

Research works printed but not linked: 2

  1. Zhang, L. & Qiu, Z. Current situation for patients with diabetic retinopathy of visiting hospital and analysis on influencing factors. Chin. J. Prat. Ophthalmol. 29, 1170–1172 (2011).
  2. Zhang, W. & Wei, M. The evaluation of the mortality and life expectancy of Chinese population. Popul. J. 38, 18–28 (2016).

Cited by

See all 76

Metadata

Authors
Wang, Y., Liu, C., Hu, W., Luo, L., Shi, D., Zhang, J., Yin, Q., Zhang, L., Han, X., & He, M.
Affiliations
  • Sun Yat-sen University (CN)
  • City University of Macau (MO)
  • The Royal Victorian Eye & Ear Hospital (AU)
  • Hong Kong Polytechnic University (HK)
  • The Alfred Hospital (AU)
Corresponding author
  • Zhang, Lei
  • Han, Xiaotong
  • He, Mingguang
Venue
npj Digital Medicine
Published
2024-02-21
Volume
7
Issue
1
Pages
43-43
ISSN
2398-6352 (ISSN-L)
OpenAlex
W4391991419
Cited by
76
References
64
Citations by year
2026: 32 · 2025: 39 · 2024: 5
FWCI
19.29
Type
journal article
Language
en
Open access
Licence
Locations
  • npj Digital Medicine (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • PolyU Institutional Research Archive (Hong Kong Polytechnic University) (submittedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • Minerva Access (University of Melbourne) (submittedVersion, cc-by)
Funders
  • National Natural Science Foundation of China (82101171, 82000901, P0046113)
  • State Key Laboratory of Ophthalmology (303060202400201209)
Topics
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Retinal Imaging and Analysis
  • Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Health Systems, Economic Evaluations, Quality of Life
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Keywords
diabetic retinopathy screening, artificial intelligence, cost-effectiveness, sensitivity and specificity trade-off, incremental cost-effectiveness ratio, diabetes mellitus, screening programs, willingness to pay, prevalence

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-13, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-13 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Economic evaluation for medical artificial intelligence: accuracy vs. cost-effectiveness in a diabetic retinopathy screening case. https://app.cochat.ai/papers/10.1038/s41746-024-01032-9
The paper
Wang, Y., Liu, C., Hu, W., Luo, L., Shi, D., Zhang, J., Yin, Q., Zhang, L., Han, X., & He, M. (2024). Economic evaluation for medical artificial intelligence: accuracy vs. cost-effectiveness in a diabetic retinopathy screening case. npj Digital Medicine. https://doi.org/10.1038/s41746-024-01032-9

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.