The DOI matches its Crossref record; 57 of the 57 references it cites were found in a registry or an index.
Model Transparency and Validation: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force–7
Open accessAbstract
Trust and confidence are critical to the success of health care models. There are two main methods for achieving this: transparency (people can see how the model is built) and validation (how well it reproduces reality). This report describes recommendations for achieving transparency and validation, developed by a task force appointed by the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) and the Society for Medical Decision Making (SMDM). Recommendations were developed iteratively by the authors. A nontechnical description should be made available to anyone—including model type and intended applications; funding sources; structure; inputs, outputs, other components that determine function, and their relationships; data sources; validation methods and results; and limitations. Technical documentation, written in sufficient detail to enable a reader with necessary expertise to evaluate the model and potentially reproduce it, should be made available openly or under agreements that protect intellectual property, at the discretion of the modelers. Validation involves face validity (wherein experts evaluate model structure, data sources, assumptions, and results), verification or internal validity (check accuracy of coding), cross validity (comparison of results with other models analyzing same problem), external validity (comparing model results to real-world results), and predictive validity (comparing model results with prospectively observed events). The last two are the strongest form of validation. Each section of this paper contains a number of recommendations that were iterated among the authors, as well as the wider modeling task force jointly set up by the International Society for Pharmacoeconomics and Outcomes Research and the Society for Medical Decision Making.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Simulation Modeling and Analysis.
Schervish, M. J., Law, A. M., & Kelton, W. D. (1983) · Journal of the American Statistical Association · doi:10.2307/2288169 · Cited by 8,697
Effects of torcetrapib in patients at high risk for coronary events
Barter, P. J., Caulfield, M., Eriksson, M., Grundy, S. M., Kastelein, J. J. P., Komajda, M., Lopez-Sendon, J., Mosca, L., Tardif, J.-C., Waters, D. D., et al. (2007) · New England Journal of Medicine · doi:10.1056/nejmoa0706628 · Cited by 3,139
Effect of Screening and Adjuvant Therapy on Mortality from Breast Cancer
Berry, D. A., Cronin, K. A., Plevritis, S. K., Fryback, D. G., Clarke, L., Zelen, M., Mandelblatt, J. S., Yakovlev, A. Y., Habbema, J. D. F., & Feuer, E. J. (2005) · New England Journal of Medicine · doi:10.1056/nejmoa050518 · Cited by 2,460
Bratley, P., Fox, B. L., & Schrage, L. E. (1987) · Springer eBooks · doi:10.1007/978-1-4419-8724-2 · Cited by 1,468
Reproducible research in computational science
Peng, R. D. (2011) · Science · doi:10.1126/science.1213847 · Cited by 1,449
Weinstein, M. C., O'Brien, B., Hornberger, J., Jackson, J., Johannesson, M., McCabe, C., & Luce, B. R. (2003) · Value in Health · doi:10.1046/j.1524-4733.2003.00234.x · Cited by 1,172
Briggs, A. H., Weinstein, M. C., Fenwick, E. A. L., Karnon, J., Sculpher, M. J., & Paltiel, A. D. (2012) · Medical Decision Making · doi:10.1177/0272989x12458348 · Cited by 887
Briggs, A. H., Weinstein, M. C., Fenwick, E. A. L., Karnon, J., Sculpher, M. J., & Paltiel, A. D. (2012) · Value in Health · doi:10.1016/j.jval.2012.04.014 · Cited by 823
Philips, Z., Ginnelly, L., Sculpher, M., Claxton, K., Golder, S., Riemsma, R., Woolacoot, N., & Glanville, J. (2004) · Health Technology Assessment · doi:10.3310/hta8360 · Cited by 697
Zauber, A. G., Lansdorp-Vogelaar, I., Knudsen, A. B., Wilschut, J., van Ballegooijen, M., & Kuntz, K. M. (2008) · Annals of Internal Medicine · doi:10.7326/0003-4819-149-9-200811040-00244 · Cited by 581
Expanded Screening for HIV in the United States — An Analysis of Cost-Effectiveness
Paltiel, A. D., Weinstein, M. C., Kimmel, A. D., Seage, G. R., Losina, E., Zhang, H., Freedberg, K. A., & Walensky, R. P. (2005) · New England Journal of Medicine · doi:10.1056/nejmsa042088 · Cited by 553
Cost-Effectiveness of Implantable Cardioverter–Defibrillators
Sanders, G. D., Hlatky, M. A., & Owens, D. K. (2005) · New England Journal of Medicine · doi:10.1056/nejmsa051989 · Cited by 547
Haynes, R. B., & Wilczynski, N. L. (2004) · BMJ · doi:10.1136/bmj.38068.557998.ee · Cited by 547
Caro, J. J., Briggs, A. H., Siebert, U., & Kuntz, K. M. (2012) · Medical Decision Making · doi:10.1177/0272989x12454577 · Cited by 520
Self-citationThe Cost Effectiveness of Combination Antiretroviral Therapy for HIV Disease
Freedberg, K. A., Losina, E., Weinstein, M. C., Paltiel, A. D., Cohen, C. J., Seage, G. R., Craven, D. E., Zhang, H., Kimmel, A. D., & Goldie, S. J. (2001) · New England Journal of Medicine · doi:10.1056/nejm200103153441108 · Cited by 508
Philips, Z., Bojke, L., Sculpher, M., Claxton, K., & Golder, S. (2006) · PharmacoEconomics · doi:10.2165/00019053-200624040-00006 · Cited by 502
A guide to simulation (2nd ed.)
Bratley, P., Fox, B. L., & Schrage, L. E. (1986) · Springer eBooks · Cited by 500
Haynes, R. B., McKibbon, K. A., Wilczynski, N. L., Walter, S. D., & Werre, S. R. (2005) · BMJ · doi:10.1136/bmj.38446.498542.8f · Cited by 439
Schruben, L. W., Bratley, P., Fox, B. L., & Schrage, L. E. (1985) · Journal of the American Statistical Association · doi:10.2307/2287950 · Cited by 426
Optimal search strategies for retrieving systematic reviews from Medline: analytical survey
Montori, V. M., Wilczynski, N. L., Morgan, D., & Haynes, R. B. (2005) · BMJ · doi:10.1136/bmj.38336.804167.47 · Cited by 422
Conceptualizing a Model: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force–2
Roberts, M., Russell, L. B., Paltiel, A. D., Chambers, M., McEwan, P., & Krahn, M. (2012) · Medical Decision Making · doi:10.1177/0272989x12454941 · Cited by 389
Weinstein, M. C., Coxson, P. G., Williams, L. W., Pass, T. M., Stason, W. B., & Goldman, L. (1987) · American Journal of Public Health · doi:10.2105/ajph.77.11.1417 · Cited by 363
State-Transition Modeling: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force–3
Siebert, U., Alagoz, O., Bayoumi, A. M., Jahn, B., Owens, D. K., Cohen, D. J., & Kuntz, K. M. (2012) · Medical Decision Making · doi:10.1177/0272989x12455463 · Cited by 348
Karnon, J., Stahl, J., Brennan, A., Caro, J. J., Mar, J., & Möller, J. (2012) · Medical Decision Making · doi:10.1177/0272989x12455462 · Cited by 336
Self-citationReproducible Epidemiologic Research
Peng, R. D., Dominici, F., & Zeger, S. L. (2006) · American Journal of Epidemiology · doi:10.1093/aje/kwj093 · Cited by 292
Kahn, R., Alperin, P., Eddy, D., Borch-Johnsen, K., Buse, J., Feigelman, J., Gregg, E., Holman, R. R., Kirkman, M. S., Stern, M., et al. (2010) · The Lancet · doi:10.1016/s0140-6736(09)62162-0 · Cited by 287
Self-citationCost-Effectiveness of HIV Treatment in Resource-Poor Settings — The Case of Côte d'Ivoire
Goldie, S. J., Yazdanpanah, Y., Losina, E., Weinstein, M. C., Anglaret, X., Walensky, R. P., Hsu, H. E., Kimmel, A., Holmes, C., Kaplan, J. E., et al. (2006) · New England Journal of Medicine · doi:10.1056/nejmsa060247 · Cited by 272
Reproducible research and Biostatistics
Peng, R. D. (2009) · Biostatistics · doi:10.1093/biostatistics/kxp014 · Cited by 237
Use of Genotypic Resistance Testing To Guide HIV Therapy: Clinical Impact and Cost-Effectiveness
Weinstein, M. C., Goldie, S. J., Losina, E., Cohen, C. J., Baxter, J. D., Zhang, H., Kimmel, A. D., & Freedberg, K. A. (2001) · Annals of Internal Medicine · doi:10.7326/0003-4819-134-6-200103200-00008 · Cited by 235
Drummond, M., Brandt, A., Luce, B., & Rovira, J. (1993) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462300003007 · Cited by 230
Pitman, R., Fisman, D., Zaric, G. S., Postma, M., Kretzschmar, M., Edmunds, J., & Brisson, M. (2012) · Value in Health · doi:10.1016/j.jval.2012.06.011 · Cited by 221
Reproducible Research: Moving toward Research the Public Can Really Trust
Laine, C., Goodman, S. N., Griswold, M. E., & Sox, H. C. (2007) · Annals of Internal Medicine · doi:10.7326/0003-4819-146-6-200703200-00154 · Cited by 213
Sculpher, M., Fenwick, E., & Claxton, K. (2000) · PharmacoEconomics · doi:10.2165/00019053-200017050-00005 · Cited by 212
Computer Modeling of Diabetes and Its Complications
The Mount Hood 4 Modeling Group (2007) · Diabetes Care · doi:10.2337/dc07-9919 · Cited by 199
Validation of the Archimedes Diabetes Model
Eddy, D. M., & Schlessinger, L. (2003) · Diabetes Care · doi:10.2337/diacare.26.11.3102 · Cited by 196
Self-citationRoyle, P., & Waugh, N. (2003) · Health Technology Assessment · doi:10.3310/hta7340 · Cited by 194
Modeling for Health Care and Other Policy Decisions: Uses, Roles, and Validity
Weinstein, M. C., Toy, E. L., Sandberg, E. A., Neumann, P. J., Evans, J. S., Kuntz, K. M., Graham, J. D., & Hammitt, J. K. (2001) · Value in Health · doi:10.1046/j.1524-4733.2001.45061.x · Cited by 190
Shining Light into Black Boxes
Morin, A., Urban, J., Adams, P. D., Foster, I., Sali, A., Baker, D., & Sliz, P. (2012) · Science · doi:10.1126/science.1218263 · Cited by 189
A review and critique of modelling in prioritising and designing screening programmes
Karnon, J., Goyder, E., Tappenden, P., McPhie, S., Towers, I., Brazier, J., & Madan, J. (2007) · Health Technology Assessment · doi:10.3310/hta11520 · Cited by 184
Pitman, R., Fisman, D. N., Zaric, G. S., Postma, M. J., Kretzschmar, M., Edmunds, J., & Brisson, M. (2012) · Medical Decision Making · doi:10.1177/0272989x12454578 · Cited by 170
Archimedes: a new model for simulating health care systems—the mathematical formulation
Schlessinger, L., & Eddy, D. M. (2002) · Journal of Biomedical Informatics · doi:10.1016/s1532-0464(02)00006-0 · Cited by 160
Self-citationValidation of population-based disease simulation models: a review of concepts and methods
Kopec, J. A., Finès, P., Manuel, D. G., Buckeridge, D. L., Flanagan, W. M., Oderkirk, J., Abrahamowicz, M., Harper, S. B., Sharif, B., Okhmatovskaia, A., et al. (2010) · BMC Public Health · doi:10.1186/1471-2458-10-710 · Cited by 153
Calibration Methods Used in Cancer Simulation Models and Suggested Reporting Guidelines
Stout, N. K., Knudsen, A. B., Kong, C. Y., McMahon, P. M., & Gazelle, G. S. (2009) · PharmacoEconomics · doi:10.2165/11314830-000000000-00000 · Cited by 142
When to Start Antiretroviral Therapy in Resource-Limited Settings
Walensky, R. P., Wolf, L. L., Wood, R., Fofana, M. O., Freedberg, K. A., Martinson, N. A., Paltiel, A. D., Anglaret, X., Weinstein, M. C., Losina, E., et al. (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-3-200908040-00138 · Cited by 129
Computer Simulation: A Practical Perspective
McHaney, R. (1991) · Cited by 105
The frequency of cervical cancer screening. Comparison of a mathematical model with empirical data
Eddy, D. M. (1987) · Cancer · Cited by 83
Self-citationDisclose all data in publications
Baggerly, K. (2010) · Nature · doi:10.1038/467401b · Cited by 69
Simulation for Decision Making
Thesen, A., Travis, L. E., & Gordon, R. (1992) · Cited by 59
Analytic Choices in Economic Models of Treatments for Rheumatoid Arthritis: What Makes a Difference?
Drummond, M., Barbieri, M., & Wong, J. B. (2005) · Medical Decision Making · doi:10.1177/0272989x05280561 · Cited by 51
Self-citationD, M. P. S. M., Williams, K., Eddy, D. M., & Kahn, R. (2008) · Diabetes Care · doi:10.2337/dc08-0521 · Cited by 50
Self-citationScreening Surgeons for HIV Infection: A Cost-effectiveness Analysis
Owens, D. K., Harris, R. A., Scott, P. M., & Nease, R. F. (1995) · Annals of Internal Medicine · doi:10.7326/0003-4819-122-9-199505010-00001 · Cited by 44
Empirically Evaluating Decision-Analytic Models
Goldhaber‐Fiebert, J. D., Stout, N. K., & Goldie, S. J. (2010) · Value in Health · doi:10.1111/j.1524-4733.2010.00698.x · Cited by 28
Reproducible research and the substantive context
Keiding, N. (2010) · Biostatistics · doi:10.1093/biostatistics/kxq033 · Cited by 28
Analytic Tools for Public Health Decision Making
Owens, D. K. (2002) · Medical Decision Making · doi:10.1177/027298902237969 · Cited by 26
The importance of independent academic statistical analysis
DeAngelis, C. D., & Fontanarosa, P. (2010) · Biostatistics · doi:10.1093/biostatistics/kxq027 · Cited by 14
Principles of good modeling practice in healthcare cost-effectiveness studies
Goeree, R. A., OʼBrien, B. J., & Blackhouse, G. (2004) · Expert Review of Pharmacoeconomics & Outcomes Research · doi:10.1586/14737167.4.2.189 · Cited by 13
The wider concept of data sharing: view from the BMJ
Groves, T. (2010) · Biostatistics · doi:10.1093/biostatistics/kxq031 · Cited by 13
Research works printed but not linked: 3
- BMJ. Health economics. Accessed from: http://www.bmj.com/about-bmj/resources-authors/article-types/research/health-economics
- JAMA. Instructions for authors. Accessed from: http://jama.ama-assn.org/site/misc/ifora.xhtml
- 10.1002/1097-0142(19870901)60:5<1117::AID-CNCR2820600533>3.0.CO;2-H · doi:10.1002/1097-0142(19870901)60:5<1117::aid-cncr2820600533>3.0.co;2-h
Cited by
Guidelines for accurate and transparent health estimates reporting: the GATHER statement
Stevens, G. A., Alkema, L., Black, R. E., Boerma, J. T., Collins, G. S., Ezzati, M., Grove, J. T., Hogan, D. R., Hogan, M. C., Horton, R., et al. (2016) · The Lancet · doi:10.1016/s0140-6736(16)30388-9 · Cited by 1,798
Sullivan, S. D., Mauskopf, J. A., Augustovski, F., Jaime Caro, J., Lee, K. M., Minchin, M., Orlewska, E., Penna, P., Rodriguez Barrios, J.-M., & Shau, W.-Y. (2014) · Value in Health · doi:10.1016/j.jval.2013.08.2291 · Cited by 1,172
Briggs, A. H., Weinstein, M. C., Fenwick, E. A. L., Karnon, J., Sculpher, M. J., & Paltiel, A. D. (2012) · Medical Decision Making · doi:10.1177/0272989x12458348 · Cited by 887
Ramsey, S. D., Willke, R. J., Glick, H., Reed, S. D., Augustovski, F., Jonsson, B., Briggs, A., & Sullivan, S. D. (2015) · Value in Health · doi:10.1016/j.jval.2015.02.001 · Cited by 811
Husereau, D., Drummond, M., Augustovski, F., de Bekker-Grob, E., Briggs, A. H., Carswell, C., Caulley, L., Chaiyakunapruk, N., Greenberg, D., Loder, E., et al. (2022) · Value in Health · doi:10.1016/j.jval.2021.10.008 · Cited by 847
Metadata
- Authors
- Eddy, D. M., Hollingworth, W., Caro, J. J., Tsevat, J., McDonald, K. M., & Wong, J. B.
- Venue
- Medical Decision Making
- Publisher
- SAGE Publications
- Published
- 2012-09
- Volume
- 32
- Issue
- 5
- Pages
- 733-743
- ISSN
- 0272-989X (ISSN-L)
- 1552-681X
- OpenAlex
- W2134023402
- Cited by
- 799
- References
- 57
- FWCI
- 97.72
- Type
- journal article
- Language
- en
- Open access
- hybrid
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.