CoChat
Sign Up
Cited by 887Health Systems, Economic Evaluations, Quality of LifeDelphi Technique in ResearchStatistical Methods in Epidemiology

Model Parameter Estimation and Uncertainty Analysis: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force Working Group–6

Open access
Briggs, Andrew H., Weinstein, Milton C., Fenwick, Elisabeth A. L., Karnon, Jonathan, Sculpher, Mark J., Paltiel, A. David·2012·Medical Decision Making
DOI 10.1177/0272989x12458348
Open this paper in CoChat — free accountOpen PDF

Abstract

A model’s purpose is to inform medical decisions and health care resource allocation. Modelers employ quantitative methods to structure the clinical, epidemiological, and economic evidence base and gain qualitative insight to assist decision makers in making better decisions. From a policy perspective, the value of a model-based analysis lies not simply in its ability to generate a precise point estimate for a specific outcome but also in the systematic examination and responsible reporting of uncertainty surrounding this outcome and the ultimate decision being addressed. Different concepts relating to uncertainty in decision modeling are explored. Stochastic (first-order) uncertainty is distinguished from both parameter (second-order) uncertainty and from heterogeneity, with structural uncertainty relating to the model itself forming another level of uncertainty to consider. The article argues that the estimation of point estimates and uncertainty in parameters is part of a single process and explores the link between parameter uncertainty through to decision uncertainty and the relationship to value-of-information analysis. The article also makes extensive recommendations around the reporting of uncertainty, both in terms of deterministic sensitivity analysis techniques and probabilistic methods. Expected value of perfect information is argued to be the most appropriate presentational technique, alongside cost-effectiveness acceptability curves, for representing decision uncertainty from probabilistic analysis.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1177/0272989x12458348

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The probabilistic sensitivity analysis randomly drew parameter sets from distributions that reflect the current uncertainty in model parameter estimates.

    “To quantify the effect of model parameter uncertainty on cost-effectiveness outcomes, we conducted a PSA by randomly drawing K parameter sets (n_sim) from distributions that reflect the current uncertainty in model parameter estimates.26”

    From An Introductory Tutorial on Cohort State-Transition Models in R Using a Cost-Effectiveness Analysis Example (2023) · cites this paper as [26]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 28

  1. Caro JJ, Briggs AH, Siebert U, Kuntz KM. Modeling good research practices-overview: a report of the ISPOR-SMDM Mod- eling Good Research Practices Task Force-1. Med Decis Making. 2012;32(5):667-677.
  2. Roberts M, Russell LB, Paltiel AD, Chambers M, McEwan P, Krahn M. Conceptualizing a model: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-2. Med Decis Mak- ing. 2012;32(5):678-689.
  3. Karnon J, Stahl J, Brennan A, Caro JJ, Mar J, Mo ¨ller J. Modeling using discrete event simulation: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-4. Med Decis Mak- ing. 2012;32(5):701-711.
  4. Pitman R, Fisman D, Zaric GS, et al. Dynamic transmission modeling: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force Working Group-5. Med Decis Making. 2012;32(5):712-721.
  5. Eddy DM, Hollingworth W, Caro JJ, Tsevat J, McDonald KM, Wong JB. Model transparency and validation: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-7. Med Decis Making. 2012;32(5):733-743.
  6. Claxton K. Exploring uncertainty in cost-effectiveness analysis. Pharmacoeconomics. 2008;26:781-98.
  7. Briggs A, Sculpher M, Buxton MJ. Uncertainty in the economic evaluation of health care technologies: the role of sensitivity anal- ysis. Health Econ. 1994;3:95-104.
  8. Gold MR, Siegel JE, Russell LB, Weinstein MC. Cost-effectiveness in Health and Medicine. New York: Oxford University Press; 2007.
  9. Brisson M, Edmunds WJ. Impact of model, methodological, and parameter uncertainty in the economic analysis of vaccination programs. Med Decis Making. 2006;26:434-46.
  10. Everitt BS. The Cambridge Dictionary of Statistics. Cambridge, UK: Cambridge University Press; 1998.
  11. Briggs AH, Sculpher MJ, Claxton K. Decision Modelling for Health Economic Evaluation. Oxford, UK: Oxford University Press; 2006.
  12. O'Hagan A, Buck CE, Daneshkhah A, et al. Uncertain Judge- ments: Eliciting Expert Probabilities. Chichester, UK: John Wiley and Sons; 2006.
  13. Lunn D, Spiegelhalter D, Thomas A, Best N. The BUGS project: evolution, critique and future directions. Stat Med. 2009;28: 3049-67.
  14. Ades AE, Cliffe S. Markov chain Monte Carlo estimation of a multiparameter decision model: consistency of evidence and the accurate assessment of uncertainty. Med Decis Making. 2002; 22:359-71.
  15. Karnon J, Czoski Murray C, Smith KJ, Brand C. A hybrid cohort individual sampling natural history model of age-related macular degeneration: assessing the cost-effectiveness of screening using probabilistic calibration, Med Decis Making. 2009;29:304-16.
  16. Karnon J, Vanni T. Calibrating models in economic evaluation: a comparison of alternative measures of goodness-of-fit, parameter search strategies, and convergence criteria. Pharmacoeconomics. 2011;29:51-62.
  17. Kim JJ, Kuntz KM, Stout NK, et al. Multiparameter calibration of a natural history model of cervical cancer. Am J Epidemiol. 2007; 166:137-50.
  18. Taylor DCA, Pawar V, Kruzikas DT, et al. Incorporating cali- brated model parameters into sensitivity analyses deterministic and probabilistic approaches. Pharmacoeconomics. 2012;30:119-26.
  19. Vanni T, Karnon J, Madan J, et al. Calibrating models in economic evaluation: a seven step approach. Pharmacoeconomics. 2011;29:35-49.
  20. Bojke L, Claxton K, Sculpher M, Palmer S. Characterizing structural uncertainty in decision analytic models: a review and application of methods. Value Health. 2009;12:739-49.
  21. Chapman CH, Thompson SG, Sharples LD. Accounting for uncertainty in health economic decision models by using model averaging. J R Stat Soc Ser A Stat Soc. 2009;172(2):383-404.
  22. Jackson CH, Bojke L, Thompson SG, et al. A framework for addressing structural uncertainty in decision models. Med Decis Making. 2011;31:662-74.
  23. Briggs A, Fenn P. Confidence intervals or surfaces? Uncer- tainty on the cost-effectiveness plane. Health Econ. 1998;7:723-40.
  24. Stinnett AA, Mullahy J. The negative side of cost-effectiveness analysis (letter). JAMA. 1997;277:1931-2.
  25. Claxton K, Sculpher M, Drummond M. A rational framework for decision making by the National Institute for Clinical Excel- lence (NICE). Lancet. 2002;360:711-15.
  26. Brennan A, Kharroubi S, O'Hagan A, et al. Calculating partial expected value of perfect information via Monte Carlo sampling algorithms. Med Decis Making. 2007;27:448-70.
  27. Groot KoerkampB, Hunink MG, Stijnen T, Weinstein MC. Iden- tifying key parameters in cost-effectiveness analysis using value of information: a comparison of methods. Health Econ. 2006;4:383-92.
  28. Ades AE, Lu G, Claxton K. Expected value of sample informa- tion calculations in medical decision modeling. Med Decis Mak- ing. 2004;24:207-27.

Cited by

Sorted by citations

Authors
Briggs, A. H., Weinstein, M. C., Fenwick, E. A. L., Karnon, J., Sculpher, M. J., & Paltiel, A. D.
Affiliations
Harvard University (US)
ORCID
Venue
Medical Decision Making
Publisher
SAGE Publications
Published
2012-09
Volume
32
Issue
5
Pages
722-732
ISSN
  • 0272-989X (ISSN-L)
  • 1552-681X
DOI
10.1177/0272989x12458348
OpenAlex
W2013329800
Cited by
887
References
2
Citations by year
2026: 67 · 2025: 90 · 2024: 82 · 2023: 73 · 2022: 72 · 2021: 73 · 2020: 64 · 2019: 64 · 2018: 50 · 2017: 54 · 2016: 53 · 2015: 55 · 2014: 46 · 2013: 28 · 2012: 13
FWCI
107.63
Type
journal article
Language
en
Open access
Licence
journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
Locations
  • Medical Decision Making (publishedVersion)
  • PubMed (publishedVersion)
  • ENLIGHTEN (Jurnal Bimbingan dan Konseling Islam) (acceptedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
  • Adelaide Research & Scholarship (AR&S) (University of Adelaide) (submittedVersion)
Topics
  • Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Health Systems, Economic Evaluations, Quality of Life
  • Social Sciences › Sociology and Political Science › Delphi Technique in Research
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
Keywords
model parameter estimation, uncertainty analysis, aleatoric uncertainty, parameter uncertainty, structural uncertainty, heterogeneity, value of information, expected value of perfect information, cost-effectiveness acceptability curves, sensitivity analysis, decision models, healthcare resource allocation
MeSH terms
Decision Making, Models, Theoretical, Stochastic Processes, Uncertainty

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review