Assessing the Economic Value of Clinical Artificial Intelligence: Challenges and Opportunities
Open accessThe DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
Abstract
OBJECTIVES Clinical artificial intelligence (AI) is a novel technology, and few economic evaluations have focused on it to date. Before its wider implementation, it is important to highlight the aspects of AI that challenge traditional health technology assessment methods. METHODS We used an existing broad value framework to assess potential ways AI can provide good value for money. We also developed a rubric of how economic evaluations of AI should vary depending on the case of its use. RESULTS We found that the measurement of core elements of value-health outcomes and cost-are complicated by AI because its generalizability across different populations is often unclear and because its use may necessitate reconfigured clinical processes. Clinicians' productivity may improve when AI is used. If poorly implemented though, AI may also cause clinicians' workload to increase. Some AI has been found to exacerbate health disparities. Nevertheless, AI may promote equity by expanding access to medical care and, when properly trained, providing unbiased diagnoses and prognoses. The approach to assessment of AI should vary based on its use case: AI that creates new clinical possibilities can improve outcomes, but regulation and evidence collection may be difficult; AI that extends clinical expertise can reduce disparities and lower costs but may result in overuse; and AI that automates clinicians' work can improve productivity but may reduce skills. CONCLUSIONS The potential uses of clinical AI create challenges for health technology assessment methods originally developed for pharmaceuticals and medical devices. Health economists should be prepared to examine data collection and methods used to train AI, as these may impact its future value.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
High-performance medicine: the convergence of human and artificial intelligence
Topol, E. J. (2019) · Nature Medicine · doi:10.1038/s41591-018-0300-7 · Cited by 10,450
Dissecting racial bias in an algorithm used to manage the health of populations
Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019) · Science · doi:10.1126/science.aax2342 · Cited by 7,137
Anomalies: The Endowment Effect, Loss Aversion, and Status Quo Bias
Kahneman, D., Knetsch, J. L., & Thaler, R. H. (1991) · Journal of Economic Perspectives · doi:10.1257/jep.5.1.193 · Cited by 5,019
Artificial intelligence in healthcare: past, present and future
Jiang, F., Jiang, Y., Zhi, H., Dong, Y., Li, H., Ma, S., Wang, Y., Dong, Q., Shen, H., & Wang, Y. (2017) · Stroke and Vascular Neurology · doi:10.1136/svn-2017-000101 · Cited by 4,837
A model for types and levels of human interaction with automation
Parasuraman, R., Sheridan, T. B., & Wickens, C. D. (2000) · IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans · doi:10.1109/3468.844354 · Cited by 4,195
International evaluation of an AI system for breast cancer screening
McKinney, S. M., Sieniek, M., Godbole, V., Godwin, J., Antropova, N., Ashrafian, H., Back, T., Chesus, M., Corrado, G. S., Darzi, A., et al. (2020) · Nature · doi:10.1038/s41586-019-1799-6 · Cited by 3,287
An overview of clinical decision support systems: Benefits, risks, and strategies for success
Sutton, R. T., Pincock, D., Baumgart, D. C., Sadowski, D. C., Fedorak, R. N., & Kroeker, K. I. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0221-y · Cited by 3,019
Large-Scale Assessment of a Smartwatch to Identify Atrial Fibrillation
Perez, M. V., Mahaffey, K. W., Hedlin, H., Rumsfeld, J. S., Garcia, A., Ferris, T., Balasubramanian, V., Russo, A. M., Rajmane, A., Cheung, L., et al. (2019) · New England Journal of Medicine · doi:10.1056/nejmoa1901183 · Cited by 1,917
Zech, J. R., Badgeley, M. A., Liu, M., Costa, A. B., Titano, J. J., & Oermann, E. K. (2018) · PLOS Medicine · doi:10.1371/journal.pmed.1002683 · Cited by 1,719
The role of trust in automation reliance
Dzindolet, M. T., Peterson, S. A., Pomranky, R. A., Pierce, L. G., & Beck, H. P. (2003) · International Journal of Human-Computer Studies · doi:10.1016/s1071-5819(03)00038-7 · Cited by 1,276
Nagendran, M., Chen, Y., Lovejoy, C. A., Gordon, A. C., Komorowski, M., Harvey, H., Topol, E. J., Ioannidis, J. P. A., Collins, G. S., & Maruthappu, M. (2020) · BMJ · doi:10.1136/bmj.m689 · Cited by 1,152
The ethics of AI in health care: A mapping review
Morley, J., Machado, C. C. V., Burr, C., Cowls, J., Joshi, I., Taddeo, M., & Floridi, L. (2020) · Social Science & Medicine · doi:10.1016/j.socscimed.2020.113172 · Cited by 1,105
Park, S. H., & Han, K. (2018) · Radiology · doi:10.1148/radiol.2017171920 · Cited by 894
Artificial intelligence in medical practice: the question to the answer?
Miller, D. D., & Brown, E. W. (2018) · The American Journal of Medicine · doi:10.1016/j.amjmed.2017.10.035 · Cited by 774
Addressing bias in artificial intelligence in health care
Parikh, R. B., Teeple, S., & Navathe, A. S. (2019) · JAMA · doi:10.1001/jama.2019.18058 · Cited by 764
A Deep Learning Mammography-based Model for Improved Breast Cancer Risk Prediction
Yala, A., Lehman, C., Schuster, T., Portnoi, T., & Barzilay, R. (2019) · Radiology · doi:10.1148/radiol.2019182716 · Cited by 758
Beede, E., Baylor, E., Hersch, F., Iurchenko, A., Wilcox, L., Ruamviboonsuk, P., & Vardoulakis, L. M. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376718 · Cited by 614
Potential liability for physicians using artificial intelligence
Price, W. N., Gerke, S., & Cohen, I. G. (2019) · JAMA · doi:10.1001/jama.2019.15064 · Cited by 579
Beyond prediction: using big data for policy problems
Athey, S. (2017) · Science · doi:10.1126/science.aal4321 · Cited by 575
Fan, W., Liu, J., Zhu, S., & Pardalos, P. M. (2020) · Annals of Operations Research · doi:10.1007/s10479-018-2818-y · Cited by 346
van der Heijden, A. A., Abramoff, M. D., Verbraak, F., van Hecke, M. V., Liem, A., & Nijpels, G. (2018) · Acta Ophthalmologica · doi:10.1111/aos.13613 · Cited by 296
Mammographic Breast Density Assessment Using Deep Learning: Clinical Implementation
Lehman, C. D., Yala, A., Schuster, T., Dontchos, B., Bahl, M., Swanson, K., & Barzilay, R. (2019) · Radiology · doi:10.1148/radiol.2018180694 · Cited by 294
An algorithmic approach to reducing unexplained pain disparities in underserved populations
Pierson, E., Cutler, D. M., Leskovec, J., Mullainathan, S., & Obermeyer, Z. (2021) · Nature Medicine · doi:10.1038/s41591-020-01192-7 · Cited by 268
Framing the challenges of artificial intelligence in medicine
Yu, K.-H., & Kohane, I. S. (2019) · BMJ Quality & Safety · doi:10.1136/bmjqs-2018-008551 · Cited by 233
Evolution of Insulin Delivery Devices: From Syringes, Pens, and Pumps to DIY Artificial Pancreas
Kesavadev, J., Saboo, B., Krishna, M. B., & Krishnan, G. (2020) · Diabetes Therapy · doi:10.1007/s13300-020-00831-z · Cited by 221
Wiens, J., Guttag, J., & Horvitz, E. (2014) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-002162 · Cited by 123
Jones, B. A., Dailey, A., Calvocoressi, L., Reams, K., Kasl, S. V., Lee, C., & Hsu, H. (2005) · Cancer Causes & Control · doi:10.1007/s10552-005-2905-7 · Cited by 112
Cabitza, F., & Zeitoun, J.-D. (2019) · Annals of Translational Medicine · doi:10.21037/atm.2019.04.07 · Cited by 72
An ethically mindful approach to AI for health care
Morley, J., & Floridi, L. (2020) · The Lancet · doi:10.1016/s0140-6736(19)32975-7 · Cited by 69
Research works printed but not linked: 36
- Garvey, 2018, Interview with Colin Garvey, Rensselaer Polytechnic Institute. Artificial intelligence and systems medicine convergence, Omics, 22, 130, 10.1089/omi.2017.0218 · DOI
- Miller, 2009, Computer-assisted diagnostic decision support: history, challenges, and possible paths forward, Adv Health Sci Educ Theory Pract, 14, 89, 10.1007/s10459-009-9186-y · DOI
- Stinton C, Jenkinson D, Adekanmbi V, Clarke A, Taylor-Phillips S. Does time of day influence cancer detection and recall rates in mammography? In: Proceeding from the SPIE 10136, Medical Imaging 2017: Image Perception, Observer Performance, and Technology Assessment; March 10, 2017; Orlando, FL. Abstract 10136B. · doi:10.1117/12.2254280
- Berrouiguet, 2019, Combining mobile-health (mHealth) and artificial intelligence (AI) methods to avoid suicide attempts: the Smartcrises study protocol, BMC Psychiatry, 19, 277, 10.1186/s12888-019-2260-y · DOI · Paper page
- Beam, 2016, Translating artificial intelligence into clinical care, JAMA, 316, 2368, 10.1001/jama.2016.17217 · DOI · Paper page
- Zakhem, 2018, How should artificial intelligence screen for skin cancer and deliver diagnostic predictions to patients?, JAMA Dermatol, 154, 1383, 10.1001/jamadermatol.2018.2714 · DOI
- Hekler, 2019, Superior skin cancer classification by the combination of human and artificial intelligence, Eur J Cancer, 120, 114, 10.1016/j.ejca.2019.07.019 · DOI · Paper page
- Lin, 2018, Reimagining clinical documentation with artificial intelligence, Mayo Clin Proc, 93, 563, 10.1016/j.mayocp.2018.02.016 · DOI
- Chaudhari, 2021, Prospective deployment of deep learning in MRI: a framework for important considerations, challenges, and recommendations for best practices, J Magn Reson Imaging, 54, 357, 10.1002/jmri.27331 · DOI
- Dixon, 2007, Development of a taxonomy for health information technology, Stud Health Technol Inform, 129, 616
- Combi, 2019, Clinical information systems and artificial intelligence: recent research trends, Yearb Med Inform, 28, 83, 10.1055/s-0039-1677915 · DOI
- Habbema, 2015, Models in the development of clinical practice guidelines, Ann Intern Med, 162, 530, 10.7326/L15-5075-2 · DOI
- Ostherr K. Artificial intelligence and medical humanities [published online July 11, 2020]. J Med Humanit. https://doi.org/10.1007/s10912-020-09636-4. · DOI · Paper page
- Lakdawalla, 2018, Defining elements of value in health care—a health economics approach: an ISPOR Special Task Force Report:[3], Value Health, 21, 131, 10.1016/j.jval.2017.12.007 · DOI · Paper page
- 2016, General wellness: policy for low risk devices; guidance for Industry and Food and Drug Administration staff; availability, Fed Regist, 81, 49993
- Hwang, 2019, Lifecycle regulation of artificial intelligence--and machine learning--based software devices in medicine, JAMA, 322, 2285, 10.1001/jama.2019.16842 · DOI
- Babic, 2021, Direct-to-consumer medical machine learning and artificial intelligence applications, Nat Mach Intell, 3, 283, 10.1038/s42256-021-00331-0 · DOI
- Wu, 2021, How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals, Nat Med, 27, 582, 10.1038/s41591-021-01312-x · DOI · Paper page
- Obermeyer, 2020, Adoption of artificial intelligence and machine learning is increasing, but irrational exuberance remains, NEJM Catal, 1, 2
- Schwalbe, 2020, Artificial intelligence and the future of global health, Lancet, 395, 1579, 10.1016/S0140-6736(20)30226-9 · DOI · Paper page
- Topol, 2020, Welcoming new guidelines for AI clinical research, Nat Med, 26, 1318, 10.1038/s41591-020-1042-x · DOI
- Park, 2021, Key principles of clinical validation, device approval, and insurance coverage decisions of artificial intelligence, Korean J Radiol, 22, 442, 10.3348/kjr.2021.0048 · DOI
- Rose, 2018, Machine learning for prediction in electronic health data, JAMA Netw Open, 1, 10.1001/jamanetworkopen.2018.1404 · DOI · Paper page
- Salim, 2020, External evaluation of 3 commercial artificial intelligence algorithms for independent assessment of screening mammograms, JAMA Oncol, 6, 1581, 10.1001/jamaoncol.2020.3321 · DOI · Paper page
- Schaffter, 2020, Evaluation of combined artificial intelligence and radiologist assessment to interpret screening mammograms [published correction appears in JAMA Netw Open. 2020;3(3):e204429], JAMA Netw Open, 3
- Wu, 2020, Deep neural networks improve radiologists’ performance in breast cancer screening, IEEE Trans Med Imaging, 39, 1184, 10.1109/TMI.2019.2945514 · DOI · Paper page
- Sondermann, 2019, Prediction of melanoma evolution in melanocytic nevi via artificial intelligence: a call for prospective data [published correction appears in Eur J Cancer. 2019;123:171], Eur J Cancer, 119, 30, 10.1016/j.ejca.2019.07.009 · DOI · Paper page
- van Erkel, 1998, Receiver operating characteristic (ROC) analysis: basic principles and applications in radiology, Eur J Radiol, 27, 88, 10.1016/S0720-048X(97)00157-5 · DOI · Paper page
- Levy, 2019, Deskilling in ophthalmology is the inevitable controllable?, Eye (Lond), 33, 347, 10.1038/s41433-018-0252-7 · DOI
- Schapira, 2018, Communication practices of mammography facilities and timely follow-up of a screening mammogram with a BI-RADS 0 assessment, Acad Radiol, 25, 1118, 10.1016/j.acra.2017.12.028 · DOI
- Lake, 2020, Black patients referred to a lung cancer screening program experience lower rates of screening and longer time to follow-up, BMC Cancer, 20, 561, 10.1186/s12885-020-06923-0 · DOI
- Gupta, 2020, Transfer learning for clinical time series analysis using deep neural networks, J Healthc Inform Res, 4, 112, 10.1007/s41666-019-00062-3 · DOI · Paper page
- Mukherjee, 2003, Estimating dataset size requirements for classifying DNA microarray data, J Comput Biol, 10, 119, 10.1089/106652703321825928 · DOI · Paper page
- Figueroa, 2012, Predicting sample size required for classification performance, BMC Med Inform Decis Mak, 12, 8, 10.1186/1472-6947-12-8 · DOI · Paper page
- Morey JR, Fiano E, Yaeger KA, Zhang X, Fifi JT. Impact of viz LVO on time-to-treatment and clinical outcomes in large vessel occlusion stroke patients presenting to primary stroke centers. Preprint. Posted online July 5, 2020. medRxiv 2020.07.02.20143834. https://doi.org/10.1101/2020.07.02.20143834. · DOI
- Maliha, 2021, Artificial intelligence and liability in medicine: balancing safety and innovation, Milbank Q, 99, 629, 10.1111/1468-0009.12504 · DOI · Paper page
Cited by
Systematic review of cost effectiveness and budget impact of artificial intelligence in healthcare
El Arab, R. A., & Al Moosa, O. A. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01722-y · Cited by 149
Characterizing the clinical adoption of medical AI devices through U.S. insurance claims
Wu, K., Wu, E., Theodorou, B., Liang, W., Mack, C., Glass, L., Sun, J., & Zou, J. (2024) · NEJM AI · doi:10.1056/aioa2300030 · Cited by 103
“69 For instance, researchers have argued that clinics that use diabetic retinopathy algorithms may operate at a de fi cit for every patient evaluated and propose modi fi cations to the existing payment structure to encourage adoption.”
BackgroundMust-have Qualities of Clinical Research on Artificial Intelligence and Machine Learning
Koçak, B., Cuocolo, R., Santos, D. P. D., Stanzione, A., & Ugga, L. (2023) · Balkan Medical Journal · doi:10.4274/balkanmedj.galenos.2022.2022-11-51 · Cited by 64
Yousefi, F., Dehnavieh, R., Laberge, M., Gagnon, M.-P., Ghaemi, M. M., Nadali, M., & Azizi, N. (2025) · BMC Primary Care · doi:10.1186/s12875-025-02785-2 · Cited by 70
Khan, S. D., Hoodbhoy, Z., Raja, M. H. R., Kim, J. Y., Hogg, H. D. J., Manji, A. A. A., Gulamali, F., Hasan, A., Shaikh, A., Tajuddin, S., et al. (2024) · PLOS Digital Health · doi:10.1371/journal.pdig.0000514 · Cited by 36
Metadata
- Authors
- Hendrix, N., Veenstra, D. L., Cheng, M., Anderson, N. C., & Verguet, S.
- Venue
- Value in Health
- Publisher
- Elsevier BV
- Published
- 2022-03
- Volume
- 25
- Issue
- 3
- Pages
- 331-339
- ISSN
- 1098-3015 (ISSN-L)
- OpenAlex
- W3206003576
- Cited by
- 83
- References
- 29
- FWCI
- 7.03
- Type
- journal article
- Language
- en
- Open access
- bronze
- Licence
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
- www.elsevier.com/legal/tdmrep-license (tdm)
- www.elsevier.com/open-access/userlicense/1.0/ (vor, 365 d)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.