The DOI matches its Crossref record; 75 of the 77 references it cites were found in a registry or an index.
Good Research Practices for Comparative Effectiveness Research: Analytic Methods to Improve Causal Inference from Nonrandomized Studies of Treatment Effects Using Secondary Data Sources: The ISPOR Good Research Practices for Retrospective Database Analysis Task Force Report—Part III
Open accessAbstract
Objectives Most contemporary epidemiologic studies require complex analytical methods to adjust for bias and confounding. New methods are constantly being developed, and older more established methods are yet appropriate. Careful application of statistical analysis techniques can improve causal inference of comparative treatment effects from nonrandomized studies using secondary databases. A Task Force was formed to offer a review of the more recent developments in statistical control of confounding. Methods The Task Force was commissioned and a chair was selected by the ISPOR Board of Directors in October 2007. This Report, the third in this issue of the journal, addressed methods to improve causal inference of treatment effects for nonrandomized studies. Results The Task Force Report recommends general analytic techniques and specific best practices where consensus is reached including: use of stratification analysis before multivariable modeling, multivariable regression including model performance and diagnostic testing, propensity scoring, instrumental variable, and structural modeling techniques including marginal structural models, where appropriate for secondary data. Sensitivity analyses and discussion of extent of residual confounding are discussed. Conclusions Valid findings of causal therapeutic benefits can be produced from nonrandomized studies using an array of state-of-the-art analytic techniques. Improving the quality and uniformity of these studies will impr…
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Adjustment for time-varying confounding (i.e., confounders that simultaneously act as intermediate steps in the pathway between intervention and outcome) require special methods such as marginal structural analysis or g-estimation [38,54].”
From State-Transition Modeling: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force-3 (2012) · cites this paper as [54]
Citations
Cites
(1990) · Choice Reviews Online · doi:10.5860/choice.27-3338 · Cited by 35,305
The central role of the propensity score in observational studies for causal effects
ROSENBAUM, P. R., & RUBIN, D. B. (1983) · Biometrika · doi:10.1093/biomet/70.1.41 · Cited by 31,241
Sample Selection Bias as a Specification Error
Heckman, J. J. (1979) · Econometrica · doi:10.2307/1912352 · Cited by 28,779
(2001) · RePEc: Research Papers in Economics · Cited by 27,577
Multiple Imputation for Nonresponse in Surveys
Rubin, D. B. (1987) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316696 · Cited by 20,701
Hosmer, D. W., & Lemeshow, S. (2000) · doi:10.1002/0471722146 · Cited by 20,302
Regression Modeling Strategies
Harrell, F. E. (2001) · Springer series in statistics · doi:10.1007/978-1-4757-3462-1 · Cited by 9,270
Applied Regression Analysis and Other Multivariable Methods
Walker, E., Kleinbaum, D. G., Kupper, L. L., & Muller, K. E. (1989) · Technometrics · doi:10.2307/1270375 · Cited by 8,538
Instrumental Variables Regression with Weak Instruments
Staiger, D., & Stock, J. H. (1997) · Econometrica · doi:10.2307/2171753 · Cited by 7,081
Everitt, B. S. (2001) · Psychology Press eBooks · doi:10.4324/9781410605542-11 · Cited by 6,877
Research works printed but not linked: 2
- Berger, 2009, Good research practices for comparative effectiveness research: defining, reporting and interpreting non-randomized studies of treatment effects using secondary data sources. ISPOR TF Report 2009—Part I, Value Health
- Montaquila JM., Ponikowski CH. An evaluation of alternative imputation methods. Proceedings of the Section on Survey Research Methods, American Statistical Association, 1995.
Cited by
Sorted by citations
Items 201–281 of 281
Where Are We on “Risk-Sharing” Agreements?
Neumann, P. J. (2013) · Value in Health · doi:10.1016/j.jval.2013.04.012 · Cited by 4
van den Akker, O. R., Thibault, R. T., Ioannidis, J. P. A., Schorr, S. G., & Strech, D. (2025) · Royal Society Open Science · doi:10.1098/rsos.241364 · Cited by 5
Measurement in Comparative Effectiveness Research
Chubak, J., Rutter, C. M., Kamineni, A., Johnson, E. A., Stout, N. K., Weiss, N. S., Doria-Rose, V. P., Doubeni, C. A., & Buist, D. S. M. (2013) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2013.01.006 · Cited by 3
Conrads‐Frank, A., Schnell‐Inderst, P., Neusser, S., Hallsson, L. R., Stojkov, I., Siebert, S., Kühne, F., Jahn, B., Siebert, U., & Sroczynski, G. (2022) · PubMed Central · doi:10.3205/000313 · Cited by 3
Self-citationDing, Q., Spatz, E. S., Bena, J., Morrison, S. L., Levay, M., Lin, H., Grey, M. J., Edwards, N. E., Isaacs, D. M., West, L. M., et al. (2023) · Journal of the American Heart Association · doi:10.1161/jaha.122.029058 · Cited by 3
Петрова, А., Reddy, S., & Mehta, R. (2019) · PLoS ONE · doi:10.1371/journal.pone.0214683 · Cited by 3
Non-Experimental Comparative Effectiveness Research: How to Plan and Conduct a Good Study
Ehrenstein, V., Christiansen, C. F., Schmidt, M., & Sørensen, H. T. (2014) · Current Epidemiology Reports · doi:10.1007/s40471-014-0021-5 · Cited by 3
Specialty pharmaceuticals and the quest for better outcomes
Brook, R. A. (2015) · Journal of Medical Economics · doi:10.3111/13696998.2015.1091351 · Cited by 3
Bhojani, N., Morris, K., White, J., Rojanasarot, S., Tran, E. D., & Monga, M. (2025) · Expert Review of Medical Devices · doi:10.1080/17434440.2025.2557403 · Cited by 2
The use of real-world evidence among healthcare payers: a scoping review
Masucci, L., Lewis, D., Zhao, J., Carter, C., Chan, K., & Wong, W. W. L. (2025) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462325100445 · Cited by 2
Béjar-Prado, L., Gili-Ortiz, E., & López-Méndez, J. (2015) · Studies in big data · doi:10.1007/978-3-319-11056-1_9 · Cited by 2
Lopez, M. J., & Gutman, R. (2014) · Statistical Methods in Medical Research · doi:10.1177/0962280214560046 · Cited by 2
Willke, R. J. (2017) · Springer eBooks · doi:10.1007/978-981-10-3262-2_9 · Cited by 2
Principles of Evidence-Based Medicine
Knežević, N. N., Manchikanti, L., & Hirsch, J. A. (2024) · Springer eBooks · doi:10.1007/978-3-031-46217-7_8 · Cited by 2
Henk, H. J., Li, X., Becker, L. K., Xu, H., Gong, Q., Deeter, R. G., & Barron, R. L. (2015) · Journal of Comparative Effectiveness Research · doi:10.2217/cer.14.62 · Cited by 2
Владиславовна, Х. Ю., Osina, G. S., Новодережкина, Е. А., & Goldina, T. A. (2021) · Medical Technologies Assessment and Choice · doi:10.17116/medtech20214304116 · Cited by 2
Comparative Effectiveness Research: What It Is and How to Assess Its Quality
Takeshita, J., & Gelfand, J. M. (2014) · Wiley eBooks · doi:10.1002/9781118357606.ch15 · Cited by 2
Statistical methods for assessing treatment effects for observational studies.
Gardner, K. (2014) · doi:10.18297/etd/479 · Cited by 2
Comparative- and cost-effectiveness of continuous glucose monitoring for Type 1 diabetes
McQueen, R. B. (2013) · doi:10.25677/5jzc-na10 · Cited by 2
In Reply to Fodor and Di Muzio
Lionel Perrier, A., Morelle, M., Dussart, S., & Giraud, P. (2016) · International Journal of Radiation Oncology*Biology*Physics · doi:10.1016/j.ijrobp.2016.09.003 · Cited by 2
Comparative Effectiveness Research
Setoguchi, S., & Wong, I. C. K. (2019) · Pharmacoepidemiology · doi:10.1002/9781119413431.ch26 · Cited by 2
Real-world evidence: a practical toolbox for collecting health state utilities
Lambert-Obry, V., Lafrance, J., Savoie, M., & Lachaîne, J. (2021) · Journal of Comparative Effectiveness Research · doi:10.2217/cer-2021-0121 · Cited by 1
Szamreta, E. A., Modi, M., Periyasamy, R., Yamsani, B., Machiraju, P., Menghani, N., & Monberg, M. (2025) · PLoS ONE · doi:10.1371/journal.pone.0321654 · Cited by 1
Martins-Branco, D. A., Lobo-Martins, S., Aftimos, P. G., Pereira, B. A., de Matos, L. V., Fernandes, L., Campôa, E., Marta, G. N., Moreau, M., Taylor, D., et al. (2026) · Breast Cancer Research and Treatment · doi:10.1007/s10549-026-08012-5 · Cited by 1
Ali, A. A., Xiao, H., Campbell, E. S., & Diaby, V. (2015) · Pharmaceutical Medicine · doi:10.1007/s40290-015-0113-7 · Cited by 1
Bellows, B. K., & BISKUPIAK, J. E. (2011) · Journal of Pain & Palliative Care Pharmacotherapy · doi:10.3109/15360288.2011.599482 · Cited by 1
Data-Driven Approaches for Developing Clinical Practice Guidelines
Zhang, Y., & Padman, R. (2016) · Advances in healthcare information systems and administration book series · doi:10.4018/978-1-5225-0920-2.ch003 · Cited by 1
Kuehne, F., Hallsson, L. R., Arvandi, M., Puntscher, S., Jahn, B., Sroczynski, G., & Siebert, U. (2023) · Prävention und Gesundheitsförderung · doi:10.1007/s11553-023-01033-8 · Cited by 1
Self-citationDecision making in drug reimbursement
Franken, M. G. (2014) · Cited by 1
A Perspective on Health Services Research: Comparative Effectiveness Research
Horner, R. D. (2016) · Medical Care · doi:10.1097/mlr.0000000000000604 · Cited by 1
Challenges in Developing and Assessing Comparative Effectiveness Evidence for Medical Technology
Price, R., & Long, G. (2017) · Springer eBooks · doi:10.1007/978-981-10-3262-2_18 · Cited by 1
Hess, L. M., Grabner, M., Wang, L., Liepa, A. M., Li, X. I., Cui, Z. L., Bowman, L., & Schelman, W. R. (2020) · Pragmatic and Observational Research · doi:10.2147/por.s241427 · Cited by 1
Baglio, G., Zanette, M., Cesare, M. D., Tella, S. D., Clerici, M., Baglio, F., & Blasi, V. (2021) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2020.609819 · Cited by 1
Henk, H. J., Shih, Y. T., & Borah, B. J. (2022) · JNCI Monographs · doi:10.1093/jncimonographs/lgac013 · Cited by 1
Research Methods: Epidemiologic Research in Geriatric Oncology
Bastiaannet, E. (2020) · Springer eBooks · doi:10.1007/978-3-319-57415-8_8 · Cited by 1
Public Infrastructure and Its Impacts on Social Inclusion and Foster Economic Growth in South Africa
Rukema, J. R. (2022) · Anthropology and Ethnology Open Access Journal · doi:10.23880/aeoaj-16000166 · Cited by 1
Fukasawa, T., Iwagami, M., Hara, A., NONAKA, T., & Urushihara, H. (2023) · Japanese Journal of Pharmacoepidemiology/Yakuzai ekigaku · doi:10.3820/jjpe.28.17 · Cited by 1
Lionel Perrier, A., Morelle, M., Huguet, M., Dussart, S., & Giraud, P. (2016) · International Journal of Radiation Oncology*Biology*Physics · doi:10.1016/j.ijrobp.2016.09.032 · Cited by 1
Health economic aspects of minimally invasive surgical techniques
Gehrman, J. (2018) · Gothenburg University Publications Electronic Archive (Gothenburg University) · Cited by 1
Effectiveness Research Using Electronic Health Records ( EHRs )
Li, X., Shen, C., & Li, L. (2016) · Wiley StatsRef: Statistics Reference Online · doi:10.1002/9781118445112.stat07901 · Cited by 1
Garabedian, L. F. (2013) · Digital Access to Scholarship at Harvard (DASH) (Harvard University)
Mosley, D. G. (2013) · UA Campus Repository (The University of Arizona)
Percutaneous coronary intervention in patients with chronic kidney disease: where’s the evidence?
Tsai, T. T., & Nallamothu, B. K. (2010) · Interventional Cardiology · doi:10.2217/ica.10.29
Stepping Up Asthma Therapy in the “Real World”
Nyenhuis, S. M., & Krishnan, J. A. (2015) · Annals of the American Thoracic Society · doi:10.1513/annalsats.201504-225ed
Pharmacoepidemiologic study of acute exacerbations of COPD: challenges and opportunities
Tsai, C. (2010) · Expert Review of Anti-infective Therapy · doi:10.1586/eri.10.112
Koshy, M., & Sher, D. J. (2015) · International Journal of Radiation Oncology*Biology*Physics · doi:10.1016/j.ijrobp.2015.03.031
Get to Know Comparative Effectiveness Research: It's Here to Stay
Leslie, R. S. (2009)
Incorporac Ión de eva Luac Iones económ Icas
Icos, L. (2012)
Schink, T. (2014) · Deutsches Ärzteblatt international · doi:10.3238/arztebl.2014.0680b
Huguet, M., Lionel Perrier, A., Bally, O., Joutard, X., Havet, N., Farsi, F., Benayoun, D., Hilaire, P. D. S., Beal-Ardisson, D., Morelle, M., et al. (2016) · RePEc: Research Papers in Economics · doi:10.13140/rg.2.1.5133.9763
Vergütungshöhe und Preissetzung
Walzer, S., Gerber-Grote, A., John, J., & Rogowski, W. H. (2016) · Springer eBooks · doi:10.1007/978-3-658-08186-7_9
Método de extração de coortes em bases de dados assistenciais para estudos da doença cardiovascular
Abrahão, M. T. F. (2016) · Digital Library of Theses and Dissertations (Universidade de São Paulo) · doi:10.11606/t.5.2016.tde-04082016-160129
Os Desafios da Reavaliação de Tecnologias de Saúde em Portugal
Laires, P. A., Inês, M. S., Gouveia, M. R. D. A., Mateus, C., Miguel, L. S., & Pinto, C. G. (2016) · DOAJ (DOAJ: Directory of Open Access Journals) · doi:10.25756/rpf.v8i3.125
The Challenges of Health Technology Assessment in Portugal
Laires, P. A., Inês, M. S., Gouveia, M. R. D. A., Mateus, C., Miguel, L. S., & Pinto, C. G. (2016) · DOAJ (DOAJ: Directory of Open Access Journals) · doi:10.25756/rpf.v8i3.128
Research Methods: Epidemiologic Research in Geriatric Oncology
Bastiaannet, E. (2017) · Springer eBooks · doi:10.1007/978-3-319-44870-1_8-1
Kobalava, Z. D., Вадимовна, В. С., & Анатольевич, Ш. А. (2018) · Rational Pharmacotherapy in Cardiology · doi:10.20996/1819-6446-2018-14-2-298-305
Shah, P. S., & Acharya, G. (2018) · Acta Obstetricia Et Gynecologica Scandinavica · doi:10.1111/aogs.13465
Methodological challenges to evaluating the health economic impact of nutritional interventions
Schmier, J. K., & Chang, E. T. (2019) · World Nutrition · doi:10.26596/wn.201910138-53
Outcomes Research Using Claims Databases: A Critical Review and Case Study
Wittbrodt, E. T. (2019) · Carolina Digital Repository (University of North Carolina at Chapel Hill) · doi:10.17615/w76q-x065
Case Study 2: Discovering Qualitative Research
Weldon, S. M. (2019) · Springer eBooks · doi:10.1007/978-3-030-26837-4_46
LETTURA CRITICA DI UN ARTICOLO SCIENTIFICO
Cantarutti, A., & Corrao, G. (2019) · il Diabete · doi:10.30682/ildia1903b
Interpreting evidence on long-acting injectable antipsychotics
Morrato, E. H. (2021) · The Lancet Psychiatry · doi:10.1016/s2215-0366(21)00072-9
Perera, K. P. N. (2013) · UA Campus Repository (The University of Arizona)
Data-Driven Approaches for Developing Clinical Practice Guidelines
Zhang, Y., & Padman, R. (2017) · IGI Global eBooks · doi:10.4018/978-1-5225-0978-3.ch059
Koerber, F., Karrasch, S., Rogowski, W. H., & Schellhorn, H. (2016) · Springer eBooks · doi:10.1007/978-3-658-08186-7_2
Lapi, F., Marconi, E., Lombardo, F. P., Micheletto, C., & Cricelli, C. A. (2022) · Current Medical Research and Opinion · doi:10.1080/03007995.2022.2129229
Matrix of Evidence Tool: Assessing the quality of real-world study designs (Preprint)Preprint
Hess, L. M., GOETZ, I. A., Muehlenkord, A., & Siebert, U. (2022) · doi:10.2196/preprints.42876
Self-citationUnderstanding the landscape and promoting the use of guides for real-world study: a scoping review
Qi, Y., Zhao, K., Chen, N., Xue, X., Yu, J., & Sun, X. (2024) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2024.111551
Doshi, R., Divino, V. M., Greenwalt, L., & DeKoven, M. (2025) · Expert Review of Pharmacoeconomics & Outcomes Research · doi:10.1080/14737167.2025.2453503
Deng, Q., Wong, H. M., & Peng, S. (2025) · Oral Diseases · doi:10.1111/odi.15261
Real-World Evidence and Outcomes Associated with Pharmacotherapy in Schizophrenia
Dickson, M. C., Lambert, T. L., & Skrepnek, G. H. (2024) · Springer eBooks · doi:10.1007/978-3-031-40858-8_191-1
Effectiveness and costs of new medical technologies : register-based research in psoriasis
Norlin, J. M. (2013) · KTH Publication Database DiVA (KTH Royal Institute of Technology)
Liu, M., Wang, W., Wang, M., He, Q., Li, L., Li, G., He, L., Zou, K., & Sun, X. (2022) · Figshare · doi:10.6084/m9.figshare.18071026
Liu, M., Wang, W., Wang, M., He, Q., Li, L., Li, G., He, L., Zou, K., & Sun, X. (2022) · Figshare · doi:10.6084/m9.figshare.18071029
Evaluating treatment effectiveness: Complementing RCTs with real-world data
Rivera, C. G., Mozaffari, E., Read, S. H., & Kalil, A. C. (2026) · American Journal of Health-System Pharmacy · doi:10.1093/ajhp/zxag035
Metadata
- Authors
- Johnson, M. L., Crown, W., Martin, B. C., Dormuth, C. R., & Siebert, U.
- Affiliations
- University of Houston (US)
- University of Arkansas for Medical Sciences (US)
- University of British Columbia (CA)
- Harvard University (US)
- Corresponding author
- Johnson, Michael L.
- Venue
- Value in Health
- Publisher
- Elsevier BV
- Published
- 2009-11
- Volume
- 12
- Issue
- 8
- Pages
- 1062-1073
- ISSN
- 1098-3015 (ISSN-L)
- OpenAlex
- W2153012086
- Cited by
- 285
- References
- 77
- Citations by year
- 2026: 2 · 2025: 6 · 2024: 10 · 2023: 10 · 2022: 18 · 2021: 14 · 2020: 25 · 2019: 19 · 2018: 11 · 2017: 30 · 2016: 30 · 2015: 17 · 2014: 19 · 2013: 24 · 2012: 16
- FWCI
- 13.87
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- Value in Health (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- comparative effectiveness research, causal inference, non-randomized studies, secondary data sources, confounding adjustment, propensity scoring, instrumental variables, marginal structural models, structural equation modeling, stratified analysis, multivariable regression, residual confounding, sensitivity analysis, good scientific practice
- MeSH terms
- Data Interpretation, Statistical, Decision Making, Epidemiologic Methods, Health Policy, Humans, Regression Analysis, Retrospective Studies, ROC Curve, Selection Bias, Causality, Confounding Factors, Epidemiologic, Multivariate Analysis, Databases, Factual, Outcome Assessment, Health Care, Economics, Pharmaceutical, Internationality, Comparative Effectiveness Research, Drug-Related Side Effects and Adverse Reactions
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review