The DOI matches its Crossref record; 67 of the 67 references it cites were found in a registry or an index.
Review of guidelines for good practice in decision-analytic modelling in health technology assessment
Open accessAbstract
OBJECTIVES: To identify existing guidelines and develop a synthesised guideline plus accompanying checklist. In addition to provide guidance on key theoretical, methodological and practical issues and consider the implications of this research for what might be expected of future decision-analytic models. DATA SOURCES: Electronic databases. REVIEW METHODS: A systematic review of existing good practice guidelines was undertaken to identify and summarise guidelines currently available for assessing the quality of decision-analytic models that have been undertaken for health technology assessment. A synthesised good practice guidance and accompanying checklist was developed. Two specific methods areas in decision modelling were considered. The first method's topic is the identification of parameter estimates from published literature. Parameter searches were developed and piloted using a case-study model. The second topic relates to bias in parameter estimates; that is, how to adjust estimates of treatment effect from observational studies where there are risks of selection bias. A systematic literature review was conducted to identify those studies looking at quantification of bias in parameter estimates and the implication of this bias. RESULTS: Fifteen studies met the inclusion criteria and were reviewed and consolidated into a single set of brief statements of good practice. From this, a checklist was developed and applied to three independent decision-analytic models. Although the checklist provided excellent guidance on some key issues for model evaluation, it was too general to pick up on the specific nuances of each model. The searches that were developed helped to identify important data for inclusion in the model. However, the quality of life searches proved to be problematic: the published search filters did not focus on those measures specific to cost-effectiveness analysis and although the strategies developed as part of this project were more successful few data were found. Of the 11 studies meeting the criteria on the effect of selection bias, five concluded that a non-randomised trial design is associated with bias and six studies found 'similar' estimates of treatment effects from observational studies or non-randomised clinical trials and randomised controlled trials (RCTs). One purpose of developing the synthesised guideline and checklist was to provide a framework for critical appraisal by the various parties involved in the health technology assessment process. First, the guideline and checklist can be used by groups that are reviewing other analysts' models and, secondly, the guideline and checklist could be used by the various analysts as they develop their models (to use it as a check on how they are developing and reporting their analyses). The Expert Advisory Group (EAG) that was convened to discuss the potential role of the guidance and checklist felt that, in general, the guidance and checklist would be a useful tool, although the checklist is not meant to be used exclusively to determine a model's quality, and so should not be used as a substitute for critical appraisal. CONCLUSIONS: The review of current guidelines showed that although authors may provide a consistent message regarding some aspects of modelling, in other areas conflicting attributes are presented in different guidelines. In general, the checklist appears to perform well, in terms of identifying those aspects of the model that should be of particular concern to the reader. The checklist cannot, however, provide answers to the appropriateness of the model structure and structural assumptions, as these may be seen as a general problem with generic checklists and do not reflect any shortcoming with the synthesised guidance and checklist developed here. The assessment of the checklist, as well as feedback from the EAG, indicated the importance of its use in conjunction with a more general checklist or guidelines on economic evaluation. Further methods research into the following areas would be valuable: the quantification of selection bias in non-controlled studies and in controlled observational studies; the level of bias in the different non-RCT study designs; a comparison of results from RCTs with those from other non-randomised studies; assessment of the strengths and weaknesses of alternative ways to adjust for bias in a decision model; and how to prioritise searching for parameter estimates.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Schulz (1995) · doi:10.1001/jama.1995.03520290060030 · Cited by 5,634
The National Institute for Clinical Excellence
(2001) · Nursing Management · doi:10.7748/nm.8.2.39.s15 · Cited by 3,876
Randomized, Controlled Trials, Observational Studies, and the Hierarchy of Research Designs
Concato, J., Shah, N., & Horwitz, R. I. (2000) · New England Journal of Medicine · doi:10.1056/nejm200006223422507 · Cited by 3,563
Evaluating non-randomised intervention studies
Deeks, J., Dinnes, J., D'Amico, R., Sowden, A., Sakarovitch, C., Song, F., Petticrew, M., & Altman, D. (2003) · Health Technology Assessment · doi:10.3310/hta7270 · Cited by 2,907
Markov Models in Medical Decision Making: A Practical Guide
Sonnenberg, F. A., & Beck, J. R. (1993) · Medical Decision Making · doi:10.1177/0272989x9301300409 · Cited by 2,419
A comparison of observational studies and randomized, controlled trials.
Benson, K., & Hartz, A. J. (2000) · New England Journal of Medicine · doi:10.1056/nejm200006223422506 · Cited by 2,347
Guidelines for authors and peer reviewers of economic submissions to the BMJ
Drummond, M. F., & Jefferson, T. O. (1996) · BMJ · doi:10.1136/bmj.313.7052.275 · Cited by 1,827
Bayesian Approaches to Clinical Trials and Health‐Care Evaluation
Spiegelhalter, D. J., Abrams, K. R., & Myles, J. P. (2003) · Wiley eBooks · doi:10.1002/0470092602 · Cited by 1,673
Comparison of Evidence of Treatment Effects in Randomized and Nonrandomized Studies
Ioannidis, J. P. A. (2001) · JAMA · doi:10.1001/jama.286.7.821 · Cited by 919
Claxton, K. (1999) · Journal of Health Economics · doi:10.1016/s0167-6296(98)00039-3 · Cited by 889
Self-citationProbabilistic Sensitivity Analysis Using Monte Carlo Simulation: A Practical Approach
Doubilet, P., Begg, C. B., Weinstein, M. C., Braun, P., & McNeil, B. J. (1985) · Medical Decision Making · doi:10.1177/0272989x8500500205 · Cited by 860
Randomized versus historical controls for clinical trials
Sacks, H., Chalmers, T. C., & Smith, H. (1982) · The American Journal of Medicine · doi:10.1016/0002-9343(82)90815-4 · Cited by 834
Handling Uncertainty in Cost-Effectiveness Models
Briggs, A. H. (2000) · PharmacoEconomics · doi:10.2165/00019053-200017050-00006 · Cited by 807
Kunz, R., & Oxman, A. D. (1998) · BMJ · doi:10.1136/bmj.317.7167.1185 · Cited by 770
Economic Evaluation in Health Care: Merging Theory with Practice
McGuire, A. J., & Drummond, M. (2002) · London School of Economics and Political Science Research Online (London School of Economics and Political Science) · Cited by 723
Bayesian methods in meta-analysis and evidence synthesis
Sutton, A. J., & Abrams, K. R. (2001) · Statistical Methods in Medical Research · doi:10.1177/096228020101000404 · Cited by 710
Balk, E. M. (2002) · JAMA · doi:10.1001/jama.287.22.2973 · Cited by 478
A convenient approximation of life expectancy (the “DEALE”)
Beck, J. R., Pauker, S. G., Gottlieb, J. E., Klein, K., & Kassirer, J. P. (1982) · The American Journal of Medicine · doi:10.1016/0002-9343(82)90787-2 · Cited by 469
A comparison of observational studies and randomized, controlled trials
Benson, K., & Hartz, A. J. (2000) · American Journal of Ophthalmology · doi:10.1016/s0002-9394(00)00754-6 · Cited by 457
Briggs, A. H., Goeree, R., Blackhouse, G., & O'Brien, B. J. (2002) · Medical Decision Making · doi:10.1177/027298902400448867 · Cited by 414
A convenient approximation of life expectancy (the “DEALE”)
Beck, J. R., Kassirer, J. P., & Pauker, S. G. (1982) · The American Journal of Medicine · doi:10.1016/0002-9343(82)90786-0 · Cited by 413
An economic approach to clinical trial design and research priority-setting
Claxton, K., & Posnett, J. (1996) · Health Economics · doi:10.1002/(sici)1099-1050(199611)5:6<513::aid-hec237>3.0.co;2-9 · Cited by 395
Self-citationHow to Calculate Indirect Costs in Economic Evaluations
Liljas, B. (1998) · PharmacoEconomics · doi:10.2165/00019053-199813010-00001 · Cited by 344
Decision Making in Health and Medicine: Integrating Evidence and Values
Ulahannan, T. J. (2002) · Journal of the Royal Society of Medicine · doi:10.1177/014107680209500220 · Cited by 338
Health Economic Guidelines—Similarities, Differences and Some Implications
Hjelmgren, J., Berggren, F., & Andersson, F. (2001) · Value in Health · doi:10.1046/j.1524-4733.2001.43040.x · Cited by 322
MacLehose, Reeves, Harvey, Sheldon, Russell, & Black (2000) · Health Technology Assessment · doi:10.3310/hta4340 · Cited by 311
Methods for the analysis of quality-of-life and survival data in health technology assessment.
Billingham, Abrams, & Jones (1999) · Health Technology Assessment · doi:10.3310/hta3100 · Cited by 307
Bayesian Approaches to Clinical Trials and Health-Care Evaluation
Best, M. A. (2005) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2004.00347_17.x · Cited by 303
Modelling in the economic evaluation of health care: selecting the appropriate approach
Barton, P., Bryan, S., & Robinson, S. (2004) · Journal of Health Services Research & Policy · doi:10.1258/135581904322987535 · Cited by 291
Claxton, K., Ginnelly, L., Sculpher, M., Philips, Z., & Palmer, S. (2004) · Health Technology Assessment · doi:10.3310/hta8310 · Cited by 258
Self-citationSystematic reviews of trials and other studies.
Sutton, Abrams, Jones, Sheldon, & Song (1998) · Health Technology Assessment · doi:10.3310/hta2190 · Cited by 246
Economics notes: Costing in economic evaluation
Raftery, J. (2000) · BMJ · doi:10.1136/bmj.320.7249.1597 · Cited by 212
Probabilistic Analysis of Decision Trees Using Monte Carlo Simulation
Critchfield, G. C., & Willard, K. E. (1986) · Medical Decision Making · doi:10.1177/0272989x8600600205 · Cited by 198
Determining Transition Probabilities: Confusion and Suggestions
Miller, D. K., & Homan, S. M. (1994) · Medical Decision Making · doi:10.1177/0272989x9401400107 · Cited by 197
Royle, P., & Waugh, N. (2003) · Health Technology Assessment · doi:10.3310/hta7340 · Cited by 194
Briggs, A. H., Ades, A. E., & Price, M. J. (2003) · Medical Decision Making · doi:10.1177/0272989x03255922 · Cited by 193
Modeling for Health Care and Other Policy Decisions: Uses, Roles, and Validity
Weinstein, M. C., Toy, E. L., Sandberg, E. A., Neumann, P. J., Evans, J. S., Kuntz, K. M., Graham, J. D., & Hammitt, J. K. (2001) · Value in Health · doi:10.1046/j.1524-4733.2001.45061.x · Cited by 190
Discounting in the Economic Evaluation of Health Care Interventions
KRAHN, M., & GAFNI, A. (1993) · Medical Care · doi:10.1097/00005650-199305000-00003 · Cited by 189
BAYESIAN VALUE-OF-INFORMATION ANALYSIS: An Application to a Policy Model of Alzheimer's Disease
Claxton, K., Neumann, P. J., Araki, S., & Weinstein, M. C. (2001) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462301104058 · Cited by 184
Self-citationDiscounting for health effects in cost–benefit and cost‐effectiveness analysis
Gravelle, H., & Smith, D. (2001) · Health Economics · doi:10.1002/hec.618 · Cited by 177
Sackett, D. L. (1979) · The Case-Control Study Consensus and Controversy · doi:10.1016/b978-0-08-024907-0.50013-4 · Cited by 169
Claxton, K. (1999) · Health Economics · Cited by 137
Self-citationRandom versus nonrandom assignment in controlled experiments: Do you get the same answer?
Shadish, W. R., & Ragsdale, K. (1996) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.64.6.1290 · Cited by 126
Costing in economic evaluations
Brouwer, W. B. F., Rutten, F., & Koopmanschap, M. A. (2001) · Oxford University Press eBooks · doi:10.1093/oso/9780192631770.003.0004 · Cited by 113
Ades, A. E., & Cliffe, S. (2002) · Medical Decision Making · doi:10.1177/027298902400448920 · Cited by 104
Testing the Validity of Cost-Effectiveness Models
McCabe, C., & Dixon, S. (2000) · PharmacoEconomics · doi:10.2165/00019053-200017050-00007 · Cited by 99
A Bayesian Method for Synthesizing Evidence: The Confidence Profile Method
Eddy, D. M., Hasselblad, V., & Shachter, R. D. (1990) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462300008928 · Cited by 98
Huovinen, P. (1987) · Antimicrobial Agents and Chemotherapy · doi:10.1128/aac.31.10.1451 · Cited by 90
Modelling in economic evaluation
Kuntz, K. M., & Weinstein, M. C. (2001) · Economic Evaluation in Health Care · doi:10.1093/oso/9780192631770.003.0007 · Cited by 87
Soto, J. (2002) · PubMed · Cited by 83
Toward a Peer Review Process for Medical Decision Analysis Models
Sonnenberg, F. A., Roberts, M. S., Tsevat, J., Wong, J. B., Barry, M. J., & KENT, D. L. (1994) · Medical Care · doi:10.1097/00005650-199407001-00005 · Cited by 73
Health and Economic Outcomes Modeling Practices: A Suggested Framework
Halpern, M. T., Luce, B. R., Brown, R. E., & Genesté, B. (1998) · Value in Health · doi:10.1046/j.1524-4733.1998.120131.x · Cited by 70
Etext on Health Technology Assessment (HTA) Information Resources
Topfer, L.-A., & Auston, I. (2000) · Cited by 59
Decision analytic modelling in the economic evaluation of health technologies: a consensus statement
Severens, J. L. (2000) · Radboud Repository (Radboud University) · Cited by 55
Searching for information on outcomes: do you need to be comprehensive?
Brettle, A., Long, A., Grant, M. J., & Greenhalgh, J. (1998) · BMJ Quality & Safety · doi:10.1136/qshc.7.3.163 · Cited by 47
How to Calculate Indirect Costs in Economic Evaluations
Brouwer, W. B. F., & Koopmanschap, M. A. (1998) · PharmacoEconomics · doi:10.2165/00019053-199813050-00008 · Cited by 47
Assessing Non-Consent Bias with Parallel Randomized and Nonrandomized Clinical Trials
Marcus, S. M. (1997) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(97)00068-1 · Cited by 46
Analysing clinical decision analyses
Habbema, J. D. F., Bossuyt, P. M., Dippel, D. W. J., Marshall, S. L., & Hilden, J. (1990) · Statistics in Medicine · doi:10.1002/sim.4780091104 · Cited by 41
Choosing between randomised and non-randomised studies: a systematic review
Campbell, M. J. (1998) · BMJ · doi:10.1136/bmj.317.7167.1258a · Cited by 40
Life expectancy biases in clinical decision modeling
Kuntz, K. M., & Weinstein, M. C. (1995) · Medical Decision Making · doi:10.1177/0272989x9501500209 · Cited by 38
Systematic validation of disease models for pharmacoeconomic evaluations
Sendi, P. P., Craig, B. Α., Pfluger, D. H., Gafni, A., & Bucher, H. C. (1999) · Journal of Evaluation in Clinical Practice · doi:10.1046/j.1365-2753.1999.00174.x · Cited by 35
Panel 2: Methodological Issues in Conducting Pharmacoeconomic Evaluations—Modeling Studies
Hay, J. W., Jackson, J., Luce, B. R., Avorn, J., & Ashraf, T. (1999) · Value in Health · doi:10.1046/j.1524-4733.1999.02203.x · Cited by 33
MacLehose, R., Reeves, B., Harvey, I. M., National Coordinating Centre for Health Technology Assessment, S. (. K., & NHS R and D HTA Programme, S. (. K. (2000) · OpenGrey (Institut de l'Information Scientifique et Technique) · Cited by 32
Moyer, A. E., & Finney, J. W. (2002) · Journal of Studies on Alcohol · doi:10.15288/jsa.2002.63.542 · Cited by 24
A convenient approximation of life expectancy. (The “DEALE”). I. Validation of the method
(1982) · The American Journal of Medicine · doi:10.1016/0002-9343(82)90791-4 · Cited by 20
Holland, R. R., Ellis, C. A., Geller, B. M., Plante, D. A., & Walker, R. H. S. (1999) · Medical Decision Making · doi:10.1177/0272989x9901900406 · Cited by 11
Palmer, S. C., & Strippoli, G. F. (2022) · Wiley eBooks · doi:10.1002/9781119105954.ch20 · Cited by 5
Cited by
Systematic reviews: CRD's guidance for undertaking reviews in health care
Tacconelli, E. (2010) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(10)70065-7 · Cited by 3,245
Picot, J., Jones, J., Colquitt, J., Gospodarevskaya, E., Loveman, E., Baxter, L., & Clegg, A. (2009) · Health Technology Assessment · doi:10.3310/hta13410 · Cited by 1,066
Eddy, D. M., Hollingworth, W., Caro, J. J., Tsevat, J., McDonald, K. M., & Wong, J. B. (2012) · Medical Decision Making · doi:10.1177/0272989x12454579 · Cited by 799
Latthe, P., Latthe, M., Say, L., Gülmezoglu, M., & Khan, K. S. (2006) · BMC Public Health · doi:10.1186/1471-2458-6-177 · Cited by 764
Connock, M., Juarez-Garcia, A., Jowett, S., Frew, E., Liu, Z., Taylor, R., Fry-Smith, A., Day, E., Lintzeris, N., Roberts, T., et al. (2007) · Health Technology Assessment · doi:10.3310/hta11090 · Cited by 585
Metadata
- Authors
- Philips, Z., Ginnelly, L., Sculpher, M., Claxton, K., Golder, S., Riemsma, R., Woolacoot, N., & Glanville, J.
- Venue
- Health Technology Assessment
- Published
- 2004-09
- Volume
- 8
- Issue
- 36
- Pages
- iii-iv, ix
- ISSN
- 1366-5278 (ISSN-L)
- 2046-4924
- OpenAlex
- W2142704172
- Cited by
- 697
- References
- 67
- FWCI
- 48.20
- Type
- journal article
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.