Calibrating Models in Economic Evaluation: A Comparison of Alternative Measures of Goodness of Fit, Parameter Search Strategies and Convergence Criteria

Abstract

Background The importance of assessing the accuracy of health economic decision models is widely recognized. Many applied decision models (implicitly) assume that the process of identifying relevant values for a model's input parameters is sufficient to prove the model's accuracy. The selection of infeasible combinations of input parameter values is most likely in the context of probabilistic sensitivity analysis (PSA), where parameter values are drawn from independently specified probability distributions for each model parameter. Model calibration involves the identification of input parameter values that produce model output parameters that best predict observed data. Methods An empirical comparison of three key calibration issues is presented: the applied measure of goodness of fit (GOF); the search strategy for selecting sets of input parameter values; and the convergence criteria for determining acceptable GOF. The comparisons are presented in the context of probabilistic calibration, a widely applicable approach to calibration that can be easily integrated with PSA. The appendix provides a user's guide to probabilistic calibration, with the reader invited to download the Microsoft® Excel-based model reported in this article. Results The calibrated models consistently provided higher mean estimates of the models' output parameter, illustrating the potential gain in accuracy derived from calibrating decision models. Model uncertainty was also reduced. The chi-squared GOF…