CoChat
Sign Up
Cited by 65Health Systems, Economic Evaluations, Quality of LifeArtificial Intelligence in Healthcare and EducationArtificial Intelligence in Healthcare

Economic evaluations of artificial intelligence-based healthcare interventions: a systematic literature review of best practices in their conduct and reporting

Open access
Vithlani, Jai, Hawksworth, Claire, Elvidge, Jamie, Ayiku, Lynda, Dawoud, Dalia·2023·Frontiers in Pharmacology
DOI 10.3389/fphar.2023.1220950PMC PMC10486896
Open this paper in CoChat — free account

Abstract

Objectives: Health economic evaluations (HEEs) help healthcare decision makers understand the value of new technologies. Artificial intelligence (AI) is increasingly being used in healthcare interventions. We sought to review the conduct and reporting of published HEEs for AI-based health interventions. Methods: We conducted a systematic literature review with a 15-month search window (April 2021 to June 2022) on 17 th June 2022 to identify HEEs of AI health interventions and update a previous review. Records were identified from 3 databases (Medline, Embase, and Cochrane Central). Two reviewers screened papers against predefined study selection criteria. Data were extracted from included studies using prespecified data extraction tables. Included studies were quality assessed using the National Institute for Health and Care Excellence (NICE) checklist. Results were synthesized narratively. Results: A total of 21 studies were included. The most common type of AI intervention was automated image analysis (9/21, 43%) mainly used for screening or diagnosis in general medicine and oncology. Nearly all were cost-utility (10/21, 48%) or cost-effectiveness analyses (8/21, 38%) that took a healthcare system or payer perspective. Decision-analytic models were used in 16/21 (76%) studies, mostly Markov models and decision trees. Three (3/16, 19%) used a short-term decision tree followed by a longer-term Markov component. Thirteen studies (13/21, 62%) reported the AI intervention to be cost effective or dominant. Limitations tended to result from the input data, authorship conflicts of interest, and a lack of transparent reporting, especially regarding the AI nature of the intervention. Conclusion: Published HEEs of AI-based health interventions are rapidly increasing in number. Despite the potentially innovative nature of AI, most have used traditional methods like Markov models or decision trees. Most attempted to assess the impact on quality of life to present the cost per QALY gained. However, studies have not been comprehensively reported. Specific reporting standards for the economic evaluation of AI interventions would help improve transparency and promote their usefulness for decision making. This is fundamental for reimbursement decisions, which in turn will generate the necessary data to develop flexible models better suited to capturing the potentially dynamic nature of AI interventions.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3389/fphar.2023.1220950

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Our study broadened the scope to encompass clinical AI interventions across multiple healthcare settings and specialties compared to Voets et al. and Vithlani et al.

    “Our study builds upon and extends the work of Voets et al.18 and Vithlani et al.19 by broadening the scope to encompass clinical AI interventions across multiple healthcare settings and specialties.”

    From Systematic review of cost effectiveness and budget impact of artificial intelligence in healthcare (2025) · cites this paper as [19]

    The Limitations section says Vithlani et al. focused specifically on economic evaluations, so a study of clinical AI interventions more generally could have a broader scope. But Results §3.2 reports interventions across numerous medical specialties and stages of care; encompassing multiple specialties is not, by itself, an extension beyond this review. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 42 · Cited by 65

Research works printed but not linked: 36

  1. Adams S. J. Mondal P. Penz E. Tyan C. C. Lim H. Babyn P. (2021). Development and cost analysis of a lung nodule management strategy combining artificial intelligence and lung-RADS for baseline lung cancer screening. J. Am. Coll. Radiology 18 (5), 741– 751. 10.1016/j.jacr.2020.11.014 33482120 · DOI
  2. Adarkwah C. C. van Gils P. F. Hiligsmann M. Evers S. M. A. A. (2016). Risk of bias in model-based economic evaluations: The ECOBIAS checklist. Expert Rev. Pharmacoeconomics Outcomes Res. 16 (4), 513– 523. 10.1586/14737167.2015.1103185 26588001 · DOI
  3. Areia M. Mori Y. Correale L. Repici A. Bretthauer M. Sharma P. (2022). Cost-effectiveness of artificial intelligence for screening colonoscopy: A modelling study. Lancet Digital Health 4 (6), 436– 444. 10.1016/S2589-7500(22)00042-5 35430151 · DOI
  4. Dall T. M. Gallo P. D. Chakrabarti R. West T. Semilla A. P. Storm M. V. (2013). An aging population and growing disease burden will require ALarge and specialized health care workforce by 2025. Health Aff. 32 (11), 2013– 2020. 10.1377/hlthaff.2013.0714 24191094 · DOI
  5. Davis S. Stevenson M. Tappenden P. Wailoo A. (2014) Nice dsu technical support document 15: Cost-effectiveness modelling using patient-level simulation. 27466644
  6. de Vos J. Visser L. A. de Beer A. A. Fornasa M. Thoral P. J. Elbers P. W. G. (2022). The potential cost-effectiveness of a machine learning tool that can prevent untimely intensive care unit discharge. Value Health 25 (3), 359– 367. 10.1016/j.jval.2021.06.018 35227446 · DOI
  7. Drummond M. (2015). Methods for the economic evaluation of health care programmes. Fourth: Oxford University Press.
  8. Elvidge J. Summerfield A. Nicholls D. Dawoud D. (2022). Diagnostics and treatments of COVID-19: A living systematic review of economic evaluations. Value Health 25 (5), 773– 784. 10.1016/j.jval.2022.01.001 35181207 PMC8847103 · DOI
  9. Ericson O. Hjelmgren J. Sjövall F. Söderberg J. Persson I. (2022). The potential cost and cost-effectiveness impact of using a machine learning algorithm for early detection of sepsis in intensive care units in Sweden. J. Health Econ. Outcomes Res. 9 (1), 101– 110. 10.36469/jheor.2022.33951 35620451 PMC9042649 · DOI
  10. Fusfeld L. Menon S. Gupta G. Lawrence C. Masud S. F. Goss T. F. (2022). US payer budget impact of a microarray assay with machine learning to evaluate kidney transplant rejection in for-cause biopsies. J. Med. Econ. 25 (1), 515– 523. 10.1080/13696998.2022.2059221 35345966 · DOI
  11. Gunasekeran D. V. Tseng R. M. W. W. Tham Y. C. Wong T. Y. (2021). Applications of digital health for public health responses to COVID-19: A systematic scoping review of artificial intelligence, telehealth and related technologies. NPJ Digit. Med. 4 (1), 40– 41. 10.1038/s41746-021-00412-9 33637833 PMC7910557 · DOI
  12. Guo C. Ashrafian H. Ghafur S. Fontana G. Gardner C. Prime M. (2020). Challenges for the evaluation of digital health solutions—a call for innovative evidence generation approaches. NPJ Digit. Med. 3 (1), 110– 114. 10.1038/s41746-020-00314-2 32904379 PMC7453198 · DOI
  13. Hubbard W. Walsh N. Hudson T. Heath A. Dietz J. Rogers G. (2022). Development and validation of paired MEDLINE and Embase search filters for cost-utility studies. BMC Med. Res. Methodol. 22 (1), 310– 319. 10.1186/s12874-022-01796-2 36463100 PMC9719242 · DOI
  14. Husereau D. Drummond M. Augustovski F. de Bekker-Grob E. Briggs A. H. Carswell C. (2022). Consolidated health economic evaluation reporting standards 2022 (CHEERS 2022) statement: Updated reporting guidance for health economic evaluations. BMJ 376 (Cheers), e067975– e067977. 10.1136/bmj-2021-067975 35017145 PMC8749494 · DOI
  15. Kessler S. Desai M. McConnell W. Jai E. M. Mebine P. Nguyen J. (2021). Economic and utilization outcomes of medication management at a large medicaid plan with disease management pharmacists using a novel artificial intelligence platform from 2018 to 2019: A retrospective observational study using regression methods. J. Manag. Care Specialty Pharm. 27 (9), 1186– 1196. 10.18553/jmcp.2021.21036 PMC10394210 34032130 · DOI · Paper page
  16. MacPherson P. Webb E. L. Kamchedzera W. Joekes E. Mjoli G. Lalloo D. G. (2021). Computer-aided X-ray screening for tuberculosis and HIV testing among adults with cough in Malawi (the prospect study): A randomised trial and cost-effectiveness analysis. PLoS Med. 18 (9), 10037522– e1003817. 10.1371/journal.pmed.1003752 PMC8459969 34499665 · DOI · Paper page
  17. Mallow P. J. Belk K. W. (2021). Cost-utility analysis of single nucleotide polymorphism panel-based machine learning algorithm to predict risk of opioid use disorder. J. Comp. Eff. Res. 10 (18), 1349– 1361. 10.2217/cer-2021-0115 34672212 · DOI
  18. Morrison S. L. Dukhovny D. Chan R. V. P. Chiang M. F. Campbell J. P. (2022). Cost-effectiveness of artificial intelligence-based retinopathy of prematurity screening. JAMA Ophthalmol. 140 (4), 401– 409. 10.1001/jamaophthalmol.2022.0223 35297945 PMC8931675 · DOI · Paper page
  19. National Institute for Health and Care Excellence (2012). Methods for the development of NICE public health guidance. Appendix I quality appraisal checklist- economic evaluations 27905711
  20. Nsengiyumva N. P. Hussain H. Oxlade O. Majidulla A. Nazish A. Khan A. J. (2021). Triage of persons with tuberculosis symptoms using artificial intelligence-based chest radiograph interpretation: A cost-effectiveness analysis. Open Forum Infect. Dis. 8 (12), 567. 10.1093/ofid/ofab567 PMC8671604 34917694 · DOI · Paper page
  21. Panch T. Szolovits P. Atun R. (2018). Artificial intelligence, machine learning and health systems. J. Glob. Health 8 (2), 020303– 020308. 10.7189/jogh.08.020303 30405904 PMC6199467 · DOI
  22. Philips Z. Ginnelly L. Sculpher M. Claxton K. Golder S. Riemsma R. (2004). Review of guidelines for good practice in decision-analytic modelling in health technology assessment. Health Technol. Assess. 8 (36)– iv, ix-xi, 1-158. 10.3310/hta8360 15361314 · DOI
  23. Ramlal A. Ahmad S. Kumar L. Khan F. Chongtham R. (2021). “ From molecules to patients: The clinicalapplications of biological databases andelectronic health records,” in Translational bioinformatics in healthcare and medicine. Editors Raza K. Dey N. (First Edit. Academic Press), 107– 125. 10.1016/B978-0-323-89824-9.00009-4 · DOI
  24. Rivera S. C. Liu X. Chan A. W. Denniston A. K. Calvert M. J. SPIRIT-AI and CONSORT-AI Working Group (2020). Guidelines for clinical trial protocols for interventions involving artificial intelligence: The SPIRIT-AI Extension. BMJ 370, m3210– m3214. 10.1136/bmj.m3210 32907797 PMC7490785 · DOI · Paper page
  25. Rosenthal M. B. Dudley R. A. (2007). Pay-for-Performance. JAMA 297 (7), 740– 744. 10.1001/jama.297.7.740 17312294 · DOI
  26. Rudmik L. Drummond M. (2013). Health economic evaluation: Important principles and methodology. Laryngoscope 123 (6), 1341– 1347. 10.1002/lary.23943 23483522 · DOI
  27. Salcedo J. Rosales M. Kim J. S. Nuno D. Suen S. C. Chang A. H. (2021). Cost-effectiveness of artificial intelligence monitoring for active tuberculosis treatment: A modeling study. PloS one 16 (7), e0254950. 10.1371/journal.pone.0254950 34288951 PMC8294556 · DOI · Paper page
  28. Schwendicke F. Mertens S. Cantu A. G. Chaurasia A. Meyer-Lueckel H. Krois J. (2022). Cost-effectiveness of AI for caries detection: Randomized trial. J. Dent. 119, 104080. 10.1016/j.jdent.2022.104080 35245626 · DOI
  29. Szymanski T. Ashton R. Sekelj S. Petrungaro B. Pollock K. G. Sandler B. (2022). Budget impact analysis of a machine learning algorithm to predict high risk of atrial fibrillation among primary care patients. Eur. Eur. pacing, Arrhythm. cardiac Electrophysiol. J. Work. groups cardiac pacing, Arrhythm. cardiac Cell. Electrophysiol. Eur. Soc. Cardiol. 24 (8), 1240– 1247. 10.1093/europace/euac016 35226101 · DOI · Paper page
  30. Thokala P. Srivastava T. Smith R. Ren S. Whittington M. D. Elvidge J. (2023). Living health technology assessment: Issues, challenges and opportunities. PharmacoEconomics 41 (3), 227– 237. 10.1007/s40273-022-01229-4 36652184 PMC9848020 · DOI
  31. Tseng A. S. Thao V. Borah B. J. Attia I. Z. Medina Inojosa J. Kapa S. (2021). Cost effectiveness of an electrocardiographic deep learning algorithm to detect asymptomatic Left ventricular dysfunction. Mayo Clin. Proc. 96 (7), 1835– 1844. 10.1016/j.mayocp.2020.11.032 34116837 · DOI
  32. Turino C. Benítez I. D. Rafael-Palou X. Mayoral A. Lopera A. Pascual L. (2021). Management and treatment of patients with obstructive sleep apnea using an intelligent monitoring system based on machine learning aiming to improve continuous positive airway pressure treatment compliance: Randomized controlled trial. J. Med. Internet Res. 23 (10), 240722– e24112. 10.2196/24072 PMC8561405 34661550 · DOI
  33. Unsworth H. Dillon B. Collinson L. Powell H. Salmon M. Oladapo T. (2021). The NICE Evidence Standards Framework for digital health and care technologies – developing and maintaining an innovative evidence framework with global impact. Digit. Health 7, 20552076211018617– 20552076211018620. 10.1177/20552076211018617 34249371 PMC8236783 · DOI
  34. van Leeuwen K. G. Meijer F. J. A. Schalekamp S. Rutten M. J. C. M. van Dijk E. J. van Ginneken B. (2021). Cost-effectiveness of artificial intelligence aided vessel occlusion detection in acute stroke: An early health technology assessment. Insights into Imaging 12 (133), 133. 10.1186/s13244-021-01077-4 34564764 PMC8464539 · DOI · Paper page
  35. Voets M. M. Veltman J. Slump C. H. Siesling S. Koffijberg H. (2022). Systematic review of health economic evaluations focused on artificial intelligence in healthcare: The tortoise and the cheetah. Value Health 25 (3), 340– 349. 10.1016/j.jval.2021.11.1362 35227444 · DOI · Paper page
  36. Ziegelmayer S. Graf M. Makowski M. Gawlitza J. Gassert F. (2022). Cost-effectiveness of artificial intelligence support in computed tomography-based lung cancer screening. Cancers 14 (7), 1729. 10.3390/cancers14071729 35406501 PMC8997030 · DOI · Paper page

Authors
Vithlani, J., Hawksworth, C., Elvidge, J., Ayiku, L., & Dawoud, D.
Affiliations
  • National Institute for Health and Care Excellence (GB)
  • Cairo University (EG)
Corresponding author
Hawksworth, Claire
ORCID
Venue
Frontiers in Pharmacology
Publisher
Frontiers Media SA
Published
2023-08-08
Volume
14
Pages
1220950-1220950
ISSN
1663-9812 (ISSN-L)
DOI
10.3389/fphar.2023.1220950
PMC
PMC10486896
OpenAlex
W4386156219
Cited by
65
References
42
Citations by year
2026: 32 · 2025: 25 · 2024: 7 · 2023: 1
FWCI
4.02
Type
journal article
Language
en
Open access
Licence
Locations
  • Frontiers in Pharmacology (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
Horizon 2020
Topics
  • Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Health Systems, Economic Evaluations, Quality of Life
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
Keywords
artificial intelligence in healthcare, health economic evaluation, automated image analysis, cost-utility analysis, cost-effectiveness analysis, Markov model, decision tree, reporting guidelines, transparency, reimbursement decisions, quality of life

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review