CoChat

Challenges in synthesising cost-effectiveness estimates

Open access
Shields, Gemma E., Elvidge, Jamie · 2020 · Systematic Reviews
Cited by 101

The DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.

Abstract

Economic evaluations help decision-makers faced with tough decisions on how to allocate resources. Systematic reviews of economic evaluations are useful as they allow readers to assess whether interventions have been demonstrated to be cost effective, the uncertainty in the evidence base, and key limitations or gaps in the evidence base. The synthesis of systematic reviews of economic evaluations commonly takes a narrative approach whereas a meta-analysis is common step for reviews of clinical evidence (e.g. effectiveness or adverse event outcomes). As they are common objectives in other reviews, readers may query why a synthesis has not been attempted for economic outcomes. However, a meta-analysis of incremental cost-effectiveness ratios, costs, or health benefits (including quality-adjusted life years) is fraught with issues largely due to heterogeneity across study designs and methods and further practical challenges. Therefore, meta-analysis is rarely feasible or robust. This commentary outlines these issues, supported by examples from the literature, to support researchers and reviewers considering systematic review of economic evidence.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1186/s13643-020-01536-x

Cites

6 of 6 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 6 references

Research works printed but not linked: 38

  1. Luhnen M, Prediger B, Neugebauer EAM, et al. Systematic reviews of health economic evaluations: a structured analysis of characteristics and methods applied. Res Synth Methods. 2019;10:195–206. https://doi.org/10.1002/jrsm.1342. · DOI
  2. Shields GE, Buck D, Elvidge J, et al. Cost-effectiveness evaluations of psychological therapies for schizophrenia and bipolar disorder: a systematic review. Int J Technol Assess Health Care. 2019;35:317–26. https://doi.org/10.1017/S0266462319000448. · DOI
  3. Javanbakht M, Moloney E, Brazzelli M, et al. Surgical treatments for women with stress urinary incontinence: a systematic review of economic evidence. Syst Rev. 2020;9:85. https://doi.org/10.1186/s13643-020-01352-3. · doi:10.1186/s13643-020-01352-3
  4. Haidich AB. Meta-analysis in medical research. Hippokratia. 2010 Published Online First: 2010. https://doi.org/10.5005/jp/books/10519. · DOI
  5. Briggs AH, Weinstein MC, Fenwick EAL, et al. Model parameter estimation and uncertainty analysis: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force Working Group-6. Med Decis Making. 2012;32:722–32. https://doi.org/10.1177/0272989X12458348. · DOI · Paper page
  6. Williams C, Lewsey JD, Mackay DF, et al. Estimation of survival probabilities for use in cost-effectiveness analyses: a comparison of a multi-state modeling survival analysis approach with partitioned survival and markov decision-analytic modeling. Med Decis Making. 2017;37:427–39. https://doi.org/10.1177/0272989X16670617. · DOI · Paper page
  7. Degeling K, Franken MD, May AM, et al. Matching the model with the evidence: comparing discrete event simulation and state-transition modeling for time-to-event predictions in a cost-effectiveness analysis of treatment in metastatic colorectal cancer patients. Cancer Epidemiol. 2018;57:60–7. https://doi.org/10.1016/j.canep.2018.09.008. · DOI
  8. Gibson EJ, Begum N, Koblbauer I, et al. Modeling the economic outcomes of immuno-oncology drugs: alternative model frameworks to capture clinical outcomes. Clin Outcomes Res. 2018;10:139–54. https://doi.org/10.2147/CEOR.S144208. · DOI
  9. Gibson EJ, Begum N, Koblbauer I, et al. Cohort versus patient level simulation for the economic evaluation of single versus combination immuno-oncology therapies in metastatic melanoma. J Med Econ. 2019:1–14. https://doi.org/10.1080/13696998.2019.1569446. · DOI
  10. Smare C, Lakhdari K, Doan J, et al. Evaluating partitioned survival and Markov decision-analytic modeling approaches for use in cost-effectiveness analysis: estimating and comparing survival outcomes. Pharmacoeconomics. 2020;38:97–108. https://doi.org/10.1007/s40273-019-00845-x. · doi:10.1007/s40273-019-00845-x
  11. Bullement A, Cranmer HL, Shields GE. A review of recent decision-analytic models used to evaluate the economic value of cancer treatments. Appl Health Econ Health Policy. 2019;17:771–80. https://doi.org/10.1007/s40258-019-00513-3. · DOI
  12. Petrou S. Rationale and methodology for trial-based economic evaluation. Clin Investig (Lond). 2012;2:1191–200. https://doi.org/10.4155/cli.12.121. · DOI
  13. Shields GE, Wells A, Doherty P, et al. Cost-effectiveness of cardiac rehabilitation: a systematic review. Heart. 2018;104:1403–10. https://doi.org/10.1136/heartjnl-2017-312809. · DOI · Paper page
  14. Canadian Agency for Drugs and Technologies in Health. Guidelines for the economic evaluation of health technologies: Canada 4th edition CADTH METHODS AND GUIDELINES Service Line: CADTH Methods and Guidelines. CADTH Methods Guidel. 2017.https://www.cadth.ca/about-cadth/how-we-do-it/methods-and-guidelines/guidelines-for-the-economic-evaluation-of-health-technologies-canada (Accessed 13 Aug 2020).
  15. Kwok MQT, Kareem MA, Cash MJ, et al. Adherence to discounting guidelines: evidence from over 2000 published cost-effectiveness analyses. Pharmacoeconomics. 2020. https://doi.org/10.1007/s40273-020-00916-4 Published Online First: 28 April 2020. · doi:10.1007/s40273-020-00916-4
  16. Franklin M, Thorn J. Self-reported and routinely collected electronic healthcare resource-use data for trial-based economic evaluations: the current state of play in England and considerations for the future. BMC Med Res Methodol. 2019;19:8. https://doi.org/10.1186/s12874-018-0649-9. · DOI
  17. Shields GE, Elvidge J, Davies LM. A systematic review of economic evaluations of seasonal influenza vaccination for the elderly population in the European Union. BMJ Open. 2017;7. https://doi.org/10.1136/bmjopen-2016-014847. · DOI
  18. Senn SJ. Overstating the evidence - double counting in meta-analysis and related problems. BMC Med Res Methodol. 2009;9:10. https://doi.org/10.1186/1471-2288-9-10. · DOI · Paper page
  19. Neumann PJ, Cohen JT, et al. JAMA. 2018;319:2473–4. https://doi.org/10.1001/jama.2018.6072. · DOI
  20. Brazier J, Deverill M, Green C. A review of the use of health status measures in economic evaluation. J Health Serv Res Policy. 1999. https://doi.org/10.1177/135581969900400310. · DOI · Paper page
  21. Noel CW, Lee DJ, Kong Q, et al. Comparison of health state utility measures in patients with head and neck cancer. JAMA Otolaryngol - Head Neck Surg. 2015;141:696–703. https://doi.org/10.1001/jamaoto.2015.1314. · DOI
  22. Blieden Betts M, Gandra SR, Cheng LI, et al. Differences in utility elicitation methods in cardiovascular disease: a systematic review. J Med Econ. 2018;21:74–84. https://doi.org/10.1080/13696998.2017.1379410. · DOI
  23. Arnold D, Girling A, Stevens A, et al. Comparison of direct and indirect methods of estimating health state utilities for resource allocation: review and empirical analysis. BMJ. 2009;339:b2688. https://doi.org/10.1136/bmj.b2688. · DOI · Paper page
  24. Kontodimopoulos N, Stamatopoulou E, Brinia A, et al. Are condition-specific utilities more valid than generic preference-based ones in asthma? Evidence from a study comparing EQ-5D-3L and SF-6D with AQL-5D. Expert Rev Pharmacoecon Outcomes Res. 2018;18:667–75. https://doi.org/10.1080/14737167.2018.1505506. · DOI
  25. Abel H, Kephart G, Packer T, et al. Discordance in utility measurement in persons with neurological conditions: a comparison of the SF-6D and the HUI3. Value Health. 2017;20:1157–65. https://doi.org/10.1016/j.jval.2017.04.008. · DOI
  26. Byford S, Raftery J. Perspectives in economic evaluation. BMJ. 1998;316:1529. https://doi.org/10.1136/bmj.316.7143.1529. · DOI · Paper page
  27. National Health Care Institute. Guideline for economic evaluations in healthcare. 2016. https://english.zorginstituutnederland.nl/publications/reports/2016/06/16/guideline-for-economic-evaluations-in-healthcare (Accessed 13 Aug 2020).
  28. Neumann PJ. Costing and perspective in published cost-effectiveness analysis. Med Care. 2009;47. https://doi.org/10.1097/mlr.0b013e31819bc09d. · DOI · Paper page
  29. Camacho EM, Shields GE. Cost-effectiveness of interventions for perinatal anxiety and/or depression: a systematic review. BMJ Open. 2018;8:e022022. https://doi.org/10.1136/bmjopen-2018-022022. · DOI
  30. Claxton K. Exploring uncertainty in cost-effectiveness analysis. Pharmacoeconomics. 2008;26:781–98. https://doi.org/10.2165/00019053-200826090-00008. · DOI · Paper page
  31. Hatswell AJ, Bullement A, Briggs A, et al. Probabilistic sensitivity analysis in cost-effectiveness models: determining model convergence in cohort models. Pharmacoeconomics. 2018;36:1421–6. https://doi.org/10.1007/s40273-018-0697-3. · DOI
  32. Stinnett AA, Mullahy J. Net health benefits: a new framework for the analysis of uncertainty in cost-effectiveness analysis. Med Decis Making. 1998;18. https://doi.org/10.1177/0272989x98018002s09. · DOI · Paper page
  33. Cohen DJ, Reynolds MR. Interpreting the results of cost-effectiveness studies. J Am Coll Cardiol. 2008;52:2119–26. https://doi.org/10.1016/j.jacc.2008.09.018. · DOI · Paper page
  34. Thompson SG, Brown LC, Sweeting MJ, et al. Systematic review and meta-analysis of the growth and rupture rates of small abdominal aortic aneurysms: implications for surveillance intervals and their cost-effectiveness. Health Technol Assess. 2013;17:1–118. https://doi.org/10.3310/hta17410. · DOI · Paper page
  35. Connelly JB, Hill GB, Millar WJ. The detection and management of abdominal aortic aneurysm: a cost-effectiveness analysis. Clin Invest Med. 2002;25:127–33 http://www.ncbi.nlm.nih.gov/pubmed/12220039 (Accessed 13 Aug 2020).
  36. Briggs AH, O’Brien BJ. The death of cost-minimization analysis? Health Econ. 2001;10:179–84. https://doi.org/10.1002/hec.584. · DOI · Paper page
  37. Crespo C, Monleon A, Díaz W, et al. Comparative efficiency research (COMER): meta-analysis of cost-effectiveness studies. BMC Med Res Methodol. 2014;14:139. https://doi.org/10.1186/1471-2288-14-139. · DOI
  38. Haider S, Chaikledkaew U, Thavorncharoensap M, et al. Systematic review and meta-analysis of cost-effectiveness of rotavirus vaccine in low-income and lower-middle-income countries. Open Forum Infect Dis. 2019;6:ofz117. https://doi.org/10.1093/ofid/ofz117. · DOI

Cited by

See all 101

Metadata

Authors
Shields, G. E., & Elvidge, J.
Venue
Systematic Reviews
Published
2020-12
Volume
9
Issue
1
Pages
289-289
ISSN
2046-4053 (ISSN-L)
OpenAlex
W3085985318
Cited by
101
References
6
FWCI
10.24
Type
journal article
Language
en
Open access
gold
Licence

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Challenges in synthesising cost-effectiveness estimates. https://app.cochat.ai/papers/10.1186/s13643-020-01536-x
The paper
Shields, G. E., & Elvidge, J. (2020). Challenges in synthesising cost-effectiveness estimates. Systematic Reviews. https://doi.org/10.1186/s13643-020-01536-x

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.