Challenges in synthesising cost-effectiveness estimates
Open accessThe DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
Abstract
Economic evaluations help decision-makers faced with tough decisions on how to allocate resources. Systematic reviews of economic evaluations are useful as they allow readers to assess whether interventions have been demonstrated to be cost effective, the uncertainty in the evidence base, and key limitations or gaps in the evidence base. The synthesis of systematic reviews of economic evaluations commonly takes a narrative approach whereas a meta-analysis is common step for reviews of clinical evidence (e.g. effectiveness or adverse event outcomes). As they are common objectives in other reviews, readers may query why a synthesis has not been attempted for economic outcomes. However, a meta-analysis of incremental cost-effectiveness ratios, costs, or health benefits (including quality-adjusted life years) is fraught with issues largely due to heterogeneity across study designs and methods and further practical challenges. Therefore, meta-analysis is rarely feasible or robust. This commentary outlines these issues, supported by examples from the literature, to support researchers and reviewers considering systematic review of economic evidence.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Cochrane Handbook for Systematic Reviews of Interventions
(2019) · doi:10.1002/9781119536604 · Cited by 14,828
Decision Modelling For Health Economic Evaluation
Briggs, A., Claxton, K., & Sculpher, M. (2006) · doi:10.1093/oso/9780198526629.001.0001 · Cited by 3,153
Ramsey, S. D., Willke, R. J., Glick, H., Reed, S. D., Augustovski, F., Jonsson, B., Briggs, A., & Sullivan, S. D. (2015) · Value in Health · doi:10.1016/j.jval.2015.02.001 · Cited by 811
A comparison of the EQ‐5D and SF‐6D across seven patient groups
Brazier, J., Roberts, J., Tsuchiya, A., & Busschbach, J. (2004) · Health Economics · doi:10.1002/hec.866 · Cited by 797
Caro, J. J., Briggs, A. H., Siebert, U., & Kuntz, K. M. (2012) · Medical Decision Making · doi:10.1177/0272989x12454577 · Cited by 520
Systematic reviews of economic evaluations: utility or futility?
Anderson, R. (2010) · Health Economics · doi:10.1002/hec.1486 · Cited by 173
Research works printed but not linked: 38
- Luhnen M, Prediger B, Neugebauer EAM, et al. Systematic reviews of health economic evaluations: a structured analysis of characteristics and methods applied. Res Synth Methods. 2019;10:195–206. https://doi.org/10.1002/jrsm.1342. · DOI
- Shields GE, Buck D, Elvidge J, et al. Cost-effectiveness evaluations of psychological therapies for schizophrenia and bipolar disorder: a systematic review. Int J Technol Assess Health Care. 2019;35:317–26. https://doi.org/10.1017/S0266462319000448. · DOI
- Javanbakht M, Moloney E, Brazzelli M, et al. Surgical treatments for women with stress urinary incontinence: a systematic review of economic evidence. Syst Rev. 2020;9:85. https://doi.org/10.1186/s13643-020-01352-3. · doi:10.1186/s13643-020-01352-3
- Haidich AB. Meta-analysis in medical research. Hippokratia. 2010 Published Online First: 2010. https://doi.org/10.5005/jp/books/10519. · DOI
- Briggs AH, Weinstein MC, Fenwick EAL, et al. Model parameter estimation and uncertainty analysis: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force Working Group-6. Med Decis Making. 2012;32:722–32. https://doi.org/10.1177/0272989X12458348. · DOI · Paper page
- Williams C, Lewsey JD, Mackay DF, et al. Estimation of survival probabilities for use in cost-effectiveness analyses: a comparison of a multi-state modeling survival analysis approach with partitioned survival and markov decision-analytic modeling. Med Decis Making. 2017;37:427–39. https://doi.org/10.1177/0272989X16670617. · DOI · Paper page
- Degeling K, Franken MD, May AM, et al. Matching the model with the evidence: comparing discrete event simulation and state-transition modeling for time-to-event predictions in a cost-effectiveness analysis of treatment in metastatic colorectal cancer patients. Cancer Epidemiol. 2018;57:60–7. https://doi.org/10.1016/j.canep.2018.09.008. · DOI
- Gibson EJ, Begum N, Koblbauer I, et al. Modeling the economic outcomes of immuno-oncology drugs: alternative model frameworks to capture clinical outcomes. Clin Outcomes Res. 2018;10:139–54. https://doi.org/10.2147/CEOR.S144208. · DOI
- Gibson EJ, Begum N, Koblbauer I, et al. Cohort versus patient level simulation for the economic evaluation of single versus combination immuno-oncology therapies in metastatic melanoma. J Med Econ. 2019:1–14. https://doi.org/10.1080/13696998.2019.1569446. · DOI
- Smare C, Lakhdari K, Doan J, et al. Evaluating partitioned survival and Markov decision-analytic modeling approaches for use in cost-effectiveness analysis: estimating and comparing survival outcomes. Pharmacoeconomics. 2020;38:97–108. https://doi.org/10.1007/s40273-019-00845-x. · doi:10.1007/s40273-019-00845-x
- Bullement A, Cranmer HL, Shields GE. A review of recent decision-analytic models used to evaluate the economic value of cancer treatments. Appl Health Econ Health Policy. 2019;17:771–80. https://doi.org/10.1007/s40258-019-00513-3. · DOI
- Petrou S. Rationale and methodology for trial-based economic evaluation. Clin Investig (Lond). 2012;2:1191–200. https://doi.org/10.4155/cli.12.121. · DOI
- Shields GE, Wells A, Doherty P, et al. Cost-effectiveness of cardiac rehabilitation: a systematic review. Heart. 2018;104:1403–10. https://doi.org/10.1136/heartjnl-2017-312809. · DOI · Paper page
- Canadian Agency for Drugs and Technologies in Health. Guidelines for the economic evaluation of health technologies: Canada 4th edition CADTH METHODS AND GUIDELINES Service Line: CADTH Methods and Guidelines. CADTH Methods Guidel. 2017.https://www.cadth.ca/about-cadth/how-we-do-it/methods-and-guidelines/guidelines-for-the-economic-evaluation-of-health-technologies-canada (Accessed 13 Aug 2020).
- Kwok MQT, Kareem MA, Cash MJ, et al. Adherence to discounting guidelines: evidence from over 2000 published cost-effectiveness analyses. Pharmacoeconomics. 2020. https://doi.org/10.1007/s40273-020-00916-4 Published Online First: 28 April 2020. · doi:10.1007/s40273-020-00916-4
- Franklin M, Thorn J. Self-reported and routinely collected electronic healthcare resource-use data for trial-based economic evaluations: the current state of play in England and considerations for the future. BMC Med Res Methodol. 2019;19:8. https://doi.org/10.1186/s12874-018-0649-9. · DOI
- Shields GE, Elvidge J, Davies LM. A systematic review of economic evaluations of seasonal influenza vaccination for the elderly population in the European Union. BMJ Open. 2017;7. https://doi.org/10.1136/bmjopen-2016-014847. · DOI
- Senn SJ. Overstating the evidence - double counting in meta-analysis and related problems. BMC Med Res Methodol. 2009;9:10. https://doi.org/10.1186/1471-2288-9-10. · DOI · Paper page
- Neumann PJ, Cohen JT, et al. JAMA. 2018;319:2473–4. https://doi.org/10.1001/jama.2018.6072. · DOI
- Brazier J, Deverill M, Green C. A review of the use of health status measures in economic evaluation. J Health Serv Res Policy. 1999. https://doi.org/10.1177/135581969900400310. · DOI · Paper page
- Noel CW, Lee DJ, Kong Q, et al. Comparison of health state utility measures in patients with head and neck cancer. JAMA Otolaryngol - Head Neck Surg. 2015;141:696–703. https://doi.org/10.1001/jamaoto.2015.1314. · DOI
- Blieden Betts M, Gandra SR, Cheng LI, et al. Differences in utility elicitation methods in cardiovascular disease: a systematic review. J Med Econ. 2018;21:74–84. https://doi.org/10.1080/13696998.2017.1379410. · DOI
- Arnold D, Girling A, Stevens A, et al. Comparison of direct and indirect methods of estimating health state utilities for resource allocation: review and empirical analysis. BMJ. 2009;339:b2688. https://doi.org/10.1136/bmj.b2688. · DOI · Paper page
- Kontodimopoulos N, Stamatopoulou E, Brinia A, et al. Are condition-specific utilities more valid than generic preference-based ones in asthma? Evidence from a study comparing EQ-5D-3L and SF-6D with AQL-5D. Expert Rev Pharmacoecon Outcomes Res. 2018;18:667–75. https://doi.org/10.1080/14737167.2018.1505506. · DOI
- Abel H, Kephart G, Packer T, et al. Discordance in utility measurement in persons with neurological conditions: a comparison of the SF-6D and the HUI3. Value Health. 2017;20:1157–65. https://doi.org/10.1016/j.jval.2017.04.008. · DOI
- Byford S, Raftery J. Perspectives in economic evaluation. BMJ. 1998;316:1529. https://doi.org/10.1136/bmj.316.7143.1529. · DOI · Paper page
- National Health Care Institute. Guideline for economic evaluations in healthcare. 2016. https://english.zorginstituutnederland.nl/publications/reports/2016/06/16/guideline-for-economic-evaluations-in-healthcare (Accessed 13 Aug 2020).
- Neumann PJ. Costing and perspective in published cost-effectiveness analysis. Med Care. 2009;47. https://doi.org/10.1097/mlr.0b013e31819bc09d. · DOI · Paper page
- Camacho EM, Shields GE. Cost-effectiveness of interventions for perinatal anxiety and/or depression: a systematic review. BMJ Open. 2018;8:e022022. https://doi.org/10.1136/bmjopen-2018-022022. · DOI
- Claxton K. Exploring uncertainty in cost-effectiveness analysis. Pharmacoeconomics. 2008;26:781–98. https://doi.org/10.2165/00019053-200826090-00008. · DOI · Paper page
- Hatswell AJ, Bullement A, Briggs A, et al. Probabilistic sensitivity analysis in cost-effectiveness models: determining model convergence in cohort models. Pharmacoeconomics. 2018;36:1421–6. https://doi.org/10.1007/s40273-018-0697-3. · DOI
- Stinnett AA, Mullahy J. Net health benefits: a new framework for the analysis of uncertainty in cost-effectiveness analysis. Med Decis Making. 1998;18. https://doi.org/10.1177/0272989x98018002s09. · DOI · Paper page
- Cohen DJ, Reynolds MR. Interpreting the results of cost-effectiveness studies. J Am Coll Cardiol. 2008;52:2119–26. https://doi.org/10.1016/j.jacc.2008.09.018. · DOI · Paper page
- Thompson SG, Brown LC, Sweeting MJ, et al. Systematic review and meta-analysis of the growth and rupture rates of small abdominal aortic aneurysms: implications for surveillance intervals and their cost-effectiveness. Health Technol Assess. 2013;17:1–118. https://doi.org/10.3310/hta17410. · DOI · Paper page
- Connelly JB, Hill GB, Millar WJ. The detection and management of abdominal aortic aneurysm: a cost-effectiveness analysis. Clin Invest Med. 2002;25:127–33 http://www.ncbi.nlm.nih.gov/pubmed/12220039 (Accessed 13 Aug 2020).
- Briggs AH, O’Brien BJ. The death of cost-minimization analysis? Health Econ. 2001;10:179–84. https://doi.org/10.1002/hec.584. · DOI · Paper page
- Crespo C, Monleon A, Díaz W, et al. Comparative efficiency research (COMER): meta-analysis of cost-effectiveness studies. BMC Med Res Methodol. 2014;14:139. https://doi.org/10.1186/1471-2288-14-139. · DOI
- Haider S, Chaikledkaew U, Thavorncharoensap M, et al. Systematic review and meta-analysis of cost-effectiveness of rotavirus vaccine in low-income and lower-middle-income countries. Open Forum Infect Dis. 2019;6:ofz117. https://doi.org/10.1093/ofid/ofz117. · DOI
Cited by
Eisman, A. B., Quanbeck, A., Bounthavong, M., Panattoni, L., & Glasgow, R. E. (2021) · Implementation Science · doi:10.1186/s13012-021-01143-x · Cited by 126
Barbosa, A., Whiting, S., Ding, D., Brito, J., & Mendes, R. (2022) · European Journal of Public Health · doi:10.1093/eurpub/ckac074 · Cited by 23
Shields, G. E., Buck, D., Varese, F., Yung, A. R., Thompson, A., Husain, N., Broome, M. R., Upthegrove, R., Byrne, R., & Davies, L. M. (2022) · BMC Psychiatry · doi:10.1186/s12888-022-03769-7 · Cited by 16
“, meta-analysis) that could be attempted would likely be uncertain and precarious [27].”
BackgroundSelf-citationFaller, J., Le, L. K.-D., Chatterton, M. L., Perez, J. K., Chiotelis, O., Tran, H. N. Q., Sultana, M., Hall, N., Lee, Y. Y., Chapman, C., et al. (2023) · BJPsych Open · doi:10.1192/bjo.2023.515 · Cited by 6
Osstyn, S., Rauwenhoff, J., Handels, R., de Vugt, M. E., Evers, S., van Mastrigt, G. A. P. G., & van Heugten, C. M. (2025) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462324004811
Metadata
- Authors
- Shields, G. E., & Elvidge, J.
- Venue
- Systematic Reviews
- Published
- 2020-12
- Volume
- 9
- Issue
- 1
- Pages
- 289-289
- ISSN
- 2046-4053 (ISSN-L)
- OpenAlex
- W3085985318
- Cited by
- 101
- References
- 6
- FWCI
- 10.24
- Type
- journal article
- Language
- en
- Open access
- gold
- Licence
- creativecommons.org/licenses/by/4.0/ (tdm)
- creativecommons.org/licenses/by/4.0/ (vor, 8 d)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Challenges in synthesising cost-effectiveness estimates. https://app.cochat.ai/papers/10.1186/s13643-020-01536-x
- The paper
- Shields, G. E., & Elvidge, J. (2020). Challenges in synthesising cost-effectiveness estimates. Systematic Reviews. https://doi.org/10.1186/s13643-020-01536-x
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.