Target Validity and the Hierarchy of Study Designs
Open accessAbstract
In recent years, increasing attention has been paid to problems of external validity, specifically to methodological approaches for both quantitative generalizability and transportability of study results. However, most approaches to these issues have considered external validity separately from internal validity. Here we argue that considering either internal or external validity in isolation may be problematic. Further, we argue that a joint measure of the validity of an effect estimate with respect to a specific population of interest may be more useful: We call this proposed measure target validity. In this work, we introduce and formally define target bias as the total difference between the true causal effect in the target population and the estimated causal effect in the study sample, and target validity as target bias = 0. We illustrate this measure with a series of examples and show how this measure may help us to think more clearly about comparisons between experimental and nonexperimental research results. Specifically, we show that even perfect internal validity does not ensure that a causal effect will be unbiased in a specific target population.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
After discovering a causal mechanism, researchers still need to identify conditions that modify its effects, which requires testing for differences in effects between people, calling for more diverse samples than those initially tested.
“after discovering a causal mechanism, researchers still need to identify conditions that modify its effects. This requires testing for differences in effects between people, calling for more diverse samples than those initially tested [3,7].”
From Hasty generalizations and generics in medical research: A systematic review (2024) · cites this paper as [7]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
(2002) · Cited by 13,211 · see citing works
Causal Diagrams for Epidemiologic Research
Greenland, S., Pearl, J., & Robins, J. M. (1999) · Epidemiology · doi:10.1097/00001648-199901000-00008 · Cited by 3,728 · see citing works
Randomized, Controlled Trials, Observational Studies, and the Hierarchy of Research Designs
Concato, J., Shah, N., & Horwitz, R. I. (2000) · New England Journal of Medicine · doi:10.1056/nejm200006223422507 · Cited by 3,563 · see citing works
Air Pollution and Hospital Admissions for Heart Disease in Eight U.S. Counties
Schwartz, J. D. (1999) · Epidemiology · doi:10.1097/00001648-199901000-00005 · Cited by 3,044 · see citing works
Understanding and misunderstanding randomized controlled trials
Deaton, A., & Cartwright, N. (2018) · Social Science & Medicine · doi:10.1016/j.socscimed.2017.12.005 · Cited by 1,841 · see citing works
Maternal Pesticide Exposure from Multiple Sources and Selected Congenital Anomalies
Shaw, G. M., Wasserman, C. R., O’Malley, C. D., Nelson, V. E., & Jackson, R. J. (1999) · Epidemiology · doi:10.1097/00001648-199901000-00011 · Cited by 1,783 · see citing works
8. Misunderstandings Between Experimentalists and Observationalists About Causal Inference
Imai, K., King, G., & Stuart, E. A. (2008) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/j.1467-985x.2007.00527.x · Cited by 895 · see citing works
Self-citationWhy Representativeness Should Be Avoided
Rothman, K. J., Gallacher, J. E., & Hatch, E. E. (2013) · International Journal of Epidemiology · doi:10.1093/ije/dys223 · Cited by 861 · see citing works
Holm, J., Nowicka, P., Farpour‐Lambert, N. J., O’Malley, G., Hassapidou, M. N., Weiss, R., & Baker, J. L. (2014) · Obesity Facts · doi:10.1159/000365773 · Cited by 596 · see citing works
Generalizing evidence from randomized clinical trials to target populations: The ACTG 320 trial.
Cole, S., & Stuart, E. (2010) · American Journal of Epidemiology · doi:10.1093/aje/kwq084 · Cited by 535 · see citing works
Self-citationExternal Validity: From Do-Calculus to Transportability Across Populations
Pearl, J., & Bareinboim, E. (2014) · Statistical Science · doi:10.1214/14-sts486 · Cited by 455 · see citing works
External validity: From do-calculus to transportability across populations
Pearl, J., & Bareinboim, E. (2013) · Cited by 398 · see citing works
Maldonado, G., & Greenland, S. (2002) · International Journal of Epidemiology · doi:10.1093/intjepid/31.2.422 · Cited by 396 · see citing works
Transportability of Trial Results Using Inverse Odds of Sampling Weights
Westreich, D. J., Edwards, J. K., Lesko, C. R., Stuart, E. A., & Cole, S. R. (2017) · American Journal of Epidemiology · doi:10.1093/aje/kwx164 · Cited by 324 · see citing works
Self-citationGeneralizing Study Results: A Potential Outcomes Perspective
Lesko, C. R., Buchanan, A. L., Westreich, D., Edwards, J. K., Hudgens, M. G., & Cole, S. R. (2017) · Epidemiology · doi:10.1097/ede.0000000000000664 · Cited by 269 · see citing works
Self-citationCompound Treatments and Transportability of Causal Inference
Hernán, M. A., & VanderWeele, T. J. (2011) · Epidemiology · doi:10.1097/ede.0b013e3182109296 · Cited by 256 · see citing works
Commentary: Representativeness is usually not necessary and often should be avoided
Richiardi, L., Pizzi, C., & Pearce, N. (2013) · International Journal of Epidemiology · doi:10.1093/ije/dyt103 · Cited by 214 · see citing works
A General Algorithm for Deciding Transportability of Experimental Results
Bareinboim, E., & Pearl, J. (2013) · Journal of Causal Inference · doi:10.1515/jci-2012-0004 · Cited by 209 · see citing works
Cigarette Smoking and Dementia
Hernán, M. Á., Alonso, Á., & Logroscino, G. (2008) · Epidemiology · doi:10.1097/ede.0b013e31816bbe14 · Cited by 161 · see citing works
Using causal diagrams to guide analysis in missing data problems
Daniel, R. M., Kenward, M. G., Cousens, S. N., & De Stavola, B. L. (2012) · Statistical Methods in Medical Research · doi:10.1177/0962280210394469 · Cited by 155 · cited 2× · see citing works
“study, which is seen in the open path from S = 1 (which is boxed, to indicate that we analyze only those who are in our study sample) to the outcome Y, [S = 1] ← Z → Y (17, 18).”
BackgroundExternal Validity in Policy Evaluations That Choose Sites Purposively
Olsen, R. B., Orr, L. L., Bell, S. H., & Stuart, E. A. (2012) · Journal of Policy Analysis and Management · doi:10.1002/pam.21660 · Cited by 132 · see citing works
Self-citationInvited Commentary: Selection Bias Without Colliders
Hernán, M. A. (2017) · American Journal of Epidemiology · doi:10.1093/aje/kwx077 · Cited by 116 · see citing works
Nguyen, T. Q., Ebnesajjad, C. F., Cole, S. R., & Stuart, E. A. (2017) · The Annals of Applied Statistics · doi:10.1214/16-aoas1001 · Cited by 78 · see citing works
Self-citationNohr, E. A., & Olsen, J. (2013) · International Journal of Epidemiology · doi:10.1093/ije/dyt102 · Cited by 75 · see citing works
Commentary: On representativeness
Elwood, M. (2013) · International Journal of Epidemiology · doi:10.1093/ije/dyt101 · Cited by 72 · see citing works
Greenhouse, J. B., Kaizar, E. E., Kelleher, K. J., Seltman, H. J., & Gardner, W. P. (2008) · Statistics in Medicine · doi:10.1002/sim.3218 · Cited by 64 · see citing works
Compound Treatments, Transportability, and the Structural Causal Model
Petersen, M. L. (2011) · Epidemiology · doi:10.1097/ede.0b013e3182126127 · Cited by 45 · see citing works
Challenging the hegemony of randomized controlled trials: A commentary on Deaton and Cartwright
Pearl, J. (2018) · Social Science & Medicine · doi:10.1016/j.socscimed.2018.04.024 · Cited by 23 · see citing works
Imbens, G. W. (2018) · Social Science & Medicine · doi:10.1016/j.socscimed.2018.04.028 · Cited by 18 · see citing works
An argument for renewed focus on epidemiology for public health
McQuade, E. T. R., Gray, C. L., & Poole, C. (2016) · Annals of Epidemiology · doi:10.1016/j.annepidem.2016.08.008 · Cited by 17 · see citing works
Is representativeness the right question?
Schooling, C., & Jones, H. (2014) · International Journal of Epidemiology · doi:10.1093/ije/dyt264 · Cited by 11 · see citing works
Hoggatt, K. J., & Greenland, S. (2013) · Epidemiology · doi:10.1097/ede.0000000000000023 · Cited by 11 · see citing works
Cole, S. R. (2016) · Annals of Epidemiology · doi:10.1016/j.annepidem.2016.02.005 · Cited by 2 · see citing works
Self-citation
Flagged first, then by citations
1 of the 34 references couldn’t be matched to a known paper.
Research works printed but not linked: 5
- Bareinboim E, Lee S, Honavar V, et al. Transportability from multiple environments with limited experiments. In: Burges CJC, Bottou L, Welling M, et al., eds. Advances in Neural Information Processing Systems 26. Red Hook, NY: Curran Associates, Inc.; 2014:136-144.
- Bareinboim E, Pearl J. Transportability from multiple environments with limited experiments: completeness results. In: Ghahramani Z, Welling M, Cortes C, et al., eds. Advances Am J Epidemiol. 2019;188(2):438-443 in Neural Information Processing Systems 27. Red Hook, NY: Curran Associates, Inc.; 2015:280-288.
- Pearl J, Bareinboim E. External validity and transportability: a formal approach. In: 2011 JSM Proceedings: Papers Presented at the Joint Statistical Meetings, Miami Beach, Florida, July 30-August 4, 2011, and Other ASA-Sponsored Conferences. Alexandria, VA: American Statistical Association; 2011:157-171.
- Bareinboim E, Tian J, Pearl J. Recovering from selection bias in causal and statistical inference. In: Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence and the Twenty-Sixth Innovative Applications of Artificial Intelligence Conference. Palo Alto, CA: AAAI Press; 2014: 2410-2416.
- Hoggatt KJ, Greenland S. Commentary: extending organizational schema for causal effects. Epidemiology. 2014; 25(1):98-102.
Cited by
Sorted by citations
Findley, M. G., Kikuta, K., & Denly, M. (2021) · Annual Review of Political Science · doi:10.1146/annurev-polisci-041719-102556 · Cited by 230 · see citing works
Toward a Clearer Definition of Selection Bias When Estimating Causal Effects
Lu, H., Cole, S. R., Howe, C. J., & Westreich, D. J. (2022) · Epidemiology · doi:10.1097/ede.0000000000001516 · Cited by 185 · see citing works
Self-citationUK Biobank, big data, and the consequences of non-representativeness
Keyes, K. M., & Westreich, D. (2019) · The Lancet · doi:10.1016/s0140-6736(18)33067-8 · Cited by 187 · see citing works
Self-citationOn the Need to Revitalize Descriptive Epidemiology
Fox, M. P., Murray, E. J., Lesko, C. R., & Sealy-Jefferson, S. M. (2022) · American Journal of Epidemiology · doi:10.1093/aje/kwac056 · Cited by 146 · see citing works
Self-citationRoss, R. K., Breskin, A., & Westreich, D. J. (2020) · American Journal of Epidemiology · doi:10.1093/aje/kwaa124 · Cited by 134 · see citing works
Self-citationCausal Inference Methods for Combining Randomized Trials and Observational Studies: A Review
Colnet, B., Mayer, I., Chen, G., Dieng, A., Li, R., Varoquaux, G. P., Vert, J., Josse, J., & Yang, S. (2024) · Statistical Science · doi:10.1214/23-sts889 · Cited by 141 · see citing works
Elements of External Validity: Framework, Design, and Analysis
Egami, N., & Hartman, E. (2022) · American Political Science Review · doi:10.1017/s0003055422000880 · Cited by 126 · see citing works
Stürmer, T., Webster‐Clark, M., Lund, J. L., Wyss, R., Ellis, A. R., Lunt, M., Rothman, K. J., & Glynn, R. J. (2021) · American Journal of Epidemiology · doi:10.1093/aje/kwab041 · Cited by 124 · see citing works
Targeted validation: validating clinical prediction models in their intended population and setting
Sperrin, M., Riley, R. D., Collins, P. G. S., & Martin, G. P. (2022) · Diagnostic and Prognostic Research · doi:10.1186/s41512-022-00136-8 · Cited by 115 · see citing works
Defining representativeness of study samples in medical and population health research
Rudolph, J. E., Zhong, Y., Duggal, P., Mehta, S. H., & Lau, B. M. (2023) · BMJ Medicine · doi:10.1136/bmjmed-2022-000399 · Cited by 114 · see citing works
Recent Developments in Causal Inference and Machine Learning
Brand, J. E., Zhou, X., & Xie, Y. F. (2023) · Annual Review of Sociology · doi:10.1146/annurev-soc-030420-015345 · Cited by 124 · see citing works
Low Evidence for Tinnitus Risk Factors: A Systematic Review and Meta-analysis
Biswas, R., Genitsaridi, E., Trpchevska, N., Lugo, A., Schlee, W., Cederroth, C. R., Gallus, S., & Hall, D. A. (2022) · Journal of the Association for Research in Otolaryngology · doi:10.1007/s10162-022-00874-y · Cited by 97 · see citing works
Selection Mechanisms and Their Consequences: Understanding and Addressing Selection Bias
Smith, L. H. (2020) · Current Epidemiology Reports · doi:10.1007/s40471-020-00241-6 · Cited by 90 · see citing works
Matthay, E. C., Hagan, E., Gottlieb, L. M., Tan, M. L., Vlahov, D., Adler, N. E., & Glymour, M. M. (2019) · SSM - Population Health · doi:10.1016/j.ssmph.2019.100526 · Cited by 83 · see citing works
Real-world evidence: the devil is in the detail
Gokhale, M., Stürmer, T., & Buse, J. B. (2020) · Diabetologia · doi:10.1007/s00125-020-05217-1 · Cited by 74 · see citing works
Improving quantitative synthesis to achieve generality in ecology
Spake, R., O’Dea, R., Nakagawa, S., Doncaster, C. P., Ryo, M., Callaghan, C., & Bullock, J. (2022) · Nature Ecology & Evolution · doi:10.1038/s41559-022-01891-z · Cited by 62 · see citing works
A directed acyclic graph for interactions
Nilsson, A., Bonander, C., Strömberg, U., & Björk, J. (2020) · International Journal of Epidemiology · doi:10.1093/ije/dyaa211 · Cited by 60 · see citing works
Improving Trial Generalizability Using Observational Studies
Lee, D., Yang, S., Dong, L., Wang, X., Zeng, D., & Cai, J. (2021) · Biometrics · doi:10.1111/biom.13609 · Cited by 60 · see citing works
Duchesneau, E. D., Jackson, B. E., Webster‐Clark, M., Lund, J. L., Reeder-Hayes, K. E., Nápoles, A. M., & Strassle, P. D. (2022) · Cancer Epidemiology Biomarkers & Prevention · doi:10.1158/1055-9965.epi-22-0495 · Cited by 61 · see citing works
Monte Carlo Simulation Approaches for Quantitative Bias Analysis: A Tutorial
Banack, H. R., Hayes‐Larson, E., & Mayeda, E. R. (2021) · Epidemiologic Reviews · doi:10.1093/epirev/mxab012 · Cited by 54 · see citing works
Evans, R., Moore, G. F., Movsisyan, A., & Rehfuess, E. A. (2020) · Journal of Epidemiology & Community Health · doi:10.1136/jech-2020-214468 · Cited by 51 · see citing works
Remiro-Azócar, A., Heath, A., & Baio, G. (2021) · Research Synthesis Methods · doi:10.1002/jrsm.1565 · Cited by 47 · see citing works
Simoes, J. P., Daoud, E., Shabbir, M., Amanat, S., Assouly, K. K. S., Biswas, R., Casolani, C., Dode, A., Enzler, F., Jacquemin, L., et al. (2021) · Frontiers in Aging Neuroscience · doi:10.3389/fnagi.2021.647285 · Cited by 42 · see citing works
G-computation for policy-relevant effects of interventions on time-to-event outcomes
Breskin, A., Edmonds, A., Cole, S. R., Westreich, D. J., Cocohoba, J. M., Cohen, M. H., Kassaye, S. G., Metsch, L. R., Sharma, A., Williams, M. S., et al. (2020) · International Journal of Epidemiology · doi:10.1093/ije/dyaa156 · Cited by 38 · see citing works
Self-citationSelection Bias in Health Research: Quantifying, Eliminating, or Exacerbating Health Disparities?
Rojas‐Saunero, L. P., Glymour, M. M., & Mayeda, E. R. (2023) · Current Epidemiology Reports · doi:10.1007/s40471-023-00325-z · Cited by 38 · see citing works
Transmission of SARS-CoV-2 by Children
Merckx, J., Labrecque, J. A., & Kaufman, J. S. (2020) · Deutsches Ärzteblatt international · doi:10.3238/arztebl.2020.0553 · Cited by 34 · see citing works
Target Validity: Bringing Treatment of External Validity in Line with Internal Validity
Lesko, C. R., Ackerman, B., Webster‐Clark, M., & Edwards, J. K. (2020) · Current Epidemiology Reports · doi:10.1007/s40471-020-00239-0 · Cited by 32 · see citing works
Self-citationBonander, C., Nilsson, A., Björk, J., Blomberg, A., Engström, G., Jernberg, T., Sundström, J., Östgren, C. J., Bergström, G. M. L., & Strömberg, U. (2022) · PLoS ONE · doi:10.1371/journal.pone.0265088 · Cited by 32 · see citing works
Replicability and generalizability in population psychiatric neuroimaging
Marek, S. A., & Laumann, T. O. (2024) · Neuropsychopharmacology · doi:10.1038/s41386-024-01960-w · Cited by 31 · see citing works
Martı́nez, H., Pachón, H., Kancherla, V., & Oakley, G. P. (2021) · American Journal of Epidemiology · doi:10.1093/aje/kwab061 · Cited by 31 · see citing works
Ling, A. Y., Montez‐Rath, M. E., Carita, P., Chandross, K. J., Lucats, L., Meng, Z., Sébastien, B., Kapphahn, K., & Desai, M. (2023) · Epidemiology · doi:10.1097/ede.0000000000001633 · Cited by 31 · see citing works
Geng, E., Powell, B. J., Goss, C. W., Lewis, C. C., Sales, A. E., & Kim, B. (2025) · Implementation Science · doi:10.1186/s13012-025-01427-6 · Cited by 31 · see citing works
Athletic Injury Research: Frameworks, Models and the Need for Causal Knowledge
Kalkhoven, J. T. (2024) · Sports Medicine · doi:10.1007/s40279-024-02008-1 · Cited by 27 · see citing works
Turner, A. J., Sammon, C., Latimer, N. R., Adamson, B. J. S., Beal, B., Subbiah, V., Abrams, K. R., & Ray, J. A. (2023) · PharmacoEconomics · doi:10.1007/s40273-023-01323-1 · Cited by 26 · see citing works
Mathur, M. B., & Shpitser, I. (2024) · American Journal of Epidemiology · doi:10.1093/aje/kwae145 · Cited by 25 · see citing works
Terefe, B., Jembere, M. M., Asgedom, D. K., & Lakew, A. M. (2024) · BMC Public Health · doi:10.1186/s12889-024-18717-1 · Cited by 24 · see citing works
Benbow, J. H., Rivera, D. R. R., Lund, J. L., Feldman, J., & Kim, E. S. (2022) · American Society of Clinical Oncology Educational Book · doi:10.1200/edbk_350574 · Cited by 23 · see citing works
DeRouen, M. C., Thompson, C. A., Canchola, A. J., Jin, A., Nie, S., Wong, C., Jain, J. P., Lichtensztajn, D. Y., Li, Y., Allen, L., et al. (2021) · Cancer Epidemiology Biomarkers & Prevention · doi:10.1158/1055-9965.epi-21-0019 · Cited by 22 · see citing works
Effect heterogeneity and variable selection for standardizing causal effects to a target population
Huitfeldt, A., Swanson, S. A., Stensrud, M. J., & Suzuki, E. (2019) · European Journal of Epidemiology · doi:10.1007/s10654-019-00571-w · Cited by 21 · see citing works
Goertz, C., Hurwitz, E. L., Murphy, B. A., & Coulter, I. D. (2021) · Journal of Manipulative and Physiological Therapeutics · doi:10.1016/j.jmpt.2021.02.003 · Cited by 20 · see citing works
Systematic review of applied transportability and generalizability analyses: A landscape analysis
Vuong, Q., Metcalfe, R., Ling, A. Y., Ackerman, B., Inoue, K., & Park, J. (2025) · Annals of Epidemiology · doi:10.1016/j.annepidem.2025.03.001 · Cited by 20 · see citing works
Mehrotra, M., Westreich, D. J., Glymour, M. M., Geng, E. H., & Glidden, D. V. (2021) · American Journal of Epidemiology · doi:10.1093/aje/kwab045 · Cited by 19 · see citing works
Self-citationTarget estimands for population‐adjusted indirect comparisons
Remiro‐Azócar, A. (2022) · Statistics in Medicine · doi:10.1002/sim.9413 · Cited by 20 · see citing works
Petersen, G. L., Jørgensen, T. S. H., Mathisen, J., Osler, M., Mortensen, E. L., Molbo, D., Hougaard, C. Ø., Lange, T., & Lund, R. (2024) · International Journal of Epidemiology · doi:10.1093/ije/dyae097 · Cited by 18 · see citing works
The Epidemiologic Toolbox: Identifying, Honing, and Using the Right Tools for the Job
Lesko, C. R., Keil, A. P., & Edwards, J. K. (2020) · American Journal of Epidemiology · doi:10.1093/aje/kwaa030 · Cited by 17 · see citing works
Self-citationNilsson, A., Bonander, C., Strömberg, U., Canivet, C. A., Östergren, P., & Björk, J. (2021) · PLoS ONE · doi:10.1371/journal.pone.0253969 · Cited by 17 · see citing works
Mehrotra, M., Westreich, D. J., McMahan, V., Glymour, M. M., Geng, E. H., Grant, R. M., & Glidden, D. V. (2019) · JAIDS Journal of Acquired Immune Deficiency Syndromes · doi:10.1097/qai.0000000000002037 · Cited by 18 · see citing works
Self-citationGomes, M., Turner, A. J., Sammon, C., Dawoud, D., Ramagopalan, S. V., Simpson, A., & Siebert, U. (2024) · Value in Health · doi:10.1016/j.jval.2024.01.020 · Cited by 16 · see citing works
Alves, E. D. S., dos Santos, R. V. T., Lira, F. S., de Almeida, A. A., Edwards, K. M., Benvenutti, M. J., Tufik, S. B., & de Mello, M. T. (2020) · Journal of Spinal Cord Medicine · doi:10.1080/10790268.2020.1752976 · Cited by 16 · see citing works
Cultural adaptation of two school-based smoking prevention programs in Bogotá, Colombia
Sánchez-Franco, S., Arias, L. F., Jaramillo, J., Murray, J., Hunter, R. F., Llorente, B. A., Bauld, L., Good, S., West, J., Kee, F., et al. (2021) · Translational Behavioral Medicine · doi:10.1093/tbm/ibab019 · Cited by 16 · see citing works
Saltzman, R., Zucker, I. J., Campbell, K., Gandhi, D. A., Otiono, K., Weber, A., Masterson, T. A., & Ramasamy, R. (2022) · Contemporary Clinical Trials Communications · doi:10.1016/j.conctc.2022.100986 · Cited by 15 · see citing works
How Choice of Effect Measure Influences Minimally Sufficient Adjustment Sets for External Validity
Webster‐Clark, M., & Keil, A. P. (2023) · American Journal of Epidemiology · doi:10.1093/aje/kwad041 · Cited by 14 · see citing works
Breskin, A., Westreich, D. J., Cole, S. R., & Edwards, J. K. (2019) · American Journal of Epidemiology · doi:10.1093/aje/kwz060 · Cited by 14 · see citing works
Self-citationChioléro, A., Tancredi, S., & Ioannidis, J. P. A. (2023) · European Journal of Epidemiology · doi:10.1007/s10654-023-01049-6 · Cited by 14 · see citing works
Four targets: an enhanced framework for guiding causal inference from observational data
Lu, H., Li, F., Lesko, C. R., Fink, D. S., Rudolph, K. E., Harhay, M. O., Rentsch, C. T., Fiellin, D. A., & Gonsalves, G. (2024) · International Journal of Epidemiology · doi:10.1093/ije/dyaf003 · Cited by 12 · see citing works
Self-citationFraming Causal Questions in Life Course Epidemiology
Stavola, B. L. D., Herle, M., & Pickles, A. R. (2021) · Annual Review of Statistics and Its Application · doi:10.1146/annurev-statistics-040120-024748 · Cited by 12 · see citing works
Mollan, K. R., Pence, B. W., Xu, S. G., Edwards, J. K., Mathews, W. C., O’Cleirigh, C. M., Crane, H. M., Eaton, E., Collier, A. C., Weideman, A. M., et al. (2021) · American Journal of Epidemiology · doi:10.1093/aje/kwab136 · Cited by 12 · see citing works
Self-citationHasty generalizations and generics in medical research: A systematic review
Peters, U., Sherling, H. R., & Chin-Yee, B. (2024) · PLOS ONE · doi:10.1371/journal.pone.0306749 · Cited by 11 · see citing works
Hayes‐Larson, E., Zhou, Y., Wu, Y., Rojas‐Saunero, L. P., Seamans, M. J., Gee, G. C., Brookmeyer, R., Gilsanz, P., Whitmer, R. A., & Mayeda, E. R. (2024) · American Journal of Epidemiology · doi:10.1093/aje/kwae182 · Cited by 11 · see citing works
Real-world evidence: state-of-the-art and future perspectives
Fitzke, H., Fayzan, T., Watkins, J. D., Galimov, E. R., & Pierce, B. F. (2025) · Journal of Comparative Effectiveness Research · doi:10.57264/cer-2024-0130 · Cited by 11 · see citing works
Manke-Reimers, F., Brugger, V., Bärnighausen, T., & Kohler, S. (2025) · European Journal of Epidemiology · doi:10.1007/s10654-025-01217-w · Cited by 11 · see citing works
Introduction to the Symposium: Causal Inference and Public Health
Aiello, A. E., & Green, L. W. (2019) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-111918-103312 · Cited by 11 · see citing works
Bradburn, M. J., Lee, E. C., White, D. A., Hind, D., Waugh, N. R., Cooke, D. D., Hopkins, D., Mansell, P., & Heller, S. R. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.05.001 · Cited by 12 · see citing works
Østergaard, S., Lastein, D. B., Emanuelson, U., Rustas, B.-O., Krogh, M. A., Kudahl, A. B., Munksgaard, L., & Kristensen, T. (2020) · Livestock Science · doi:10.1016/j.livsci.2020.104004 · Cited by 11 · see citing works
Liset, R., Grønli, J., Henriksen, R. E., Henriksen, T. E. G., Nilsen, R. M., & Pallesen, S. (2022) · PLoS ONE · doi:10.1371/journal.pone.0262799 · Cited by 11 · see citing works
Proxy Variables and the Generalizability of Study Results
Nilsson, A., Björk, J., & Bonander, C. (2022) · American Journal of Epidemiology · doi:10.1093/aje/kwac200 · Cited by 10 · see citing works
Rao, A., Mhlophe, H., Pretorius, A., Mcingana, M., Mcloughlin, J., Shipp, L., Baral, S., Hausler, H., Schwartz, S., & Lesko, C. R. (2023) · The Lancet HIV · doi:10.1016/s2352-3018(23)00262-x · Cited by 10 · see citing works
Self-citationDijkzeul, A., Tiemeier, H. W., Muetzel, R. L., & Labrecque, J. A. (2024) · Human Brain Mapping · doi:10.1002/hbm.26562 · Cited by 9 · see citing works
Kennedy‐Shaffer, L. (2024) · American Journal of Epidemiology · doi:10.1093/aje/kwae019 · Cited by 11 · see citing works
Nilsson, A., Bonander, C., Strömberg, U., & Björk, J. (2020) · BMC Public Health · doi:10.1186/s12889-020-10004-z · Cited by 9 · see citing works
Montez‐Rath, M. E., LUBWAMA, R. N., Kapphahn, K., Ling, A. Y., LoCasale, R. J., Robinson, L. B., Chandross, K. J., & Desai, M. (2022) · Current Medical Research and Opinion · doi:10.1080/03007995.2022.2088715 · Cited by 9 · see citing works
Lee, I.-M. M., Moore, C. C., & Evenson, K. R. (2023) · Journal for the Measurement of Physical Behaviour · doi:10.1123/jmpb.2022-0035 · Cited by 9 · see citing works
Hayes‐Larson, E., Zhou, Y., Rojas‐Saunero, L. P., Shaw, C. L., Seamans, M. J., Glymour, M. M., Murchland, A. R., Westreich, D. J., & Mayeda, E. R. (2024) · Epidemiology · doi:10.1097/ede.0000000000001780 · Cited by 8 · see citing works
Self-citationLópez, D. S., Huang, D., Tsilidis, K. K., Canfield, S. E., Khera, M., Baillargeon, J., Kuo, Y., Peek, M. K., Platz, E. A., & Markides, K. P. (2021) · Cancer Causes & Control · doi:10.1007/s10552-021-01450-0 · Cited by 8 · see citing works
Choice of Outcome in COVID-19 Studies and Implications for Policy: Mortality and Fatality
Westreich, D. J., Edwards, J. K., Tennant, P. W. G., Murray, E. J., & van Smeden, M. (2021) · American Journal of Epidemiology · doi:10.1093/aje/kwab244 · Cited by 8 · see citing works
Self-citationTurkiewicz, A., Nilsson, P. M., & Kiadaliri, A. (2020) · American Journal of Epidemiology · doi:10.1093/aje/kwaa134 · Cited by 8 · see citing works
ITEM QUALITY ANALYSIS FOR MEASURING MATHEMATICAL PROBLEM-SOLVING SKILLS
Wijayanti, P. S. (2020) · AKSIOMA Jurnal Program Studi Pendidikan Matematika · doi:10.24127/ajpm.v9i4.3036 · Cited by 8 · see citing works
Malmivaara, A. V. (2021) · Journal of Rehabilitation Medicine · doi:10.2340/16501977-2843 · Cited by 7 · see citing works
Lesko, C. R., & Fox, M. P. (2024) · American Journal of Epidemiology · doi:10.1093/aje/kwae367 · Cited by 7 · see citing works
Self-citationVariable selection when estimating effects in external target populations
Webster‐Clark, M., Ross, R. K., Keil, A. P., & Platt, R. W. (2024) · American Journal of Epidemiology · doi:10.1093/aje/kwae048 · Cited by 7 · see citing works
Lund, J. L., Webster‐Clark, M., Hinton, S. P., Shmuel, S., Stürmer, T., & Sanoff, H. K. (2020) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.5148 · Cited by 7 · see citing works
How subgroup analyses can miss the trees for the forest plots: A simulation study
Webster‐Clark, M., Baron, J. A., Funk, M. J., & Westreich, D. J. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.06.020 · Cited by 7 · see citing works
Self-citationScélo, G., Zugna, D., Popovic, M. B., Strandberg‐Larsen, K., & Richiardi, L. (2024) · Frontiers in Epidemiology · doi:10.3389/fepid.2024.1335241 · Cited by 6 · see citing works
Engeda, J. C., Lhachimi, S. K., Rosamond, W. D., Lund, J. L., Keyserling, T. C., Safford, M. M., Colantonio, L. D., Muntner, P., & Avery, C. L. (2020) · PLoS Medicine · doi:10.1371/journal.pmed.1003280 · Cited by 6 · see citing works
Social determinants of health and selection bias in genome‐wide association studies
Riehm, K. E., Keyes, K. M., & Susser, E. (2023) · World Psychiatry · doi:10.1002/wps.21047 · Cited by 6 · see citing works
Say where you sample: Increasing site selection transparency in urban ecology
Dyson, K., Dawwas, E. B., Poulton‐Kamakura, R., Alberti, M., & Fuentes, T. L. (2023) · Ecosphere · doi:10.1002/ecs2.4466 · Cited by 6 · see citing works
Lu, H., Zivich, P. N., Rudolph, J. E., Liew, Z., Mukherjee, B., & Li, F. (2025) · International Journal of Epidemiology · doi:10.1093/ije/dyaf109 · Cited by 5 · see citing works
External validity in distributed data networks
Webster‐Clark, M., Toh, S., Arnold, J., McTigue, K. M., Carton, T. W., & Platt, R. W. (2023) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.5666 · Cited by 5 · see citing works
Lund, J. L., Webster‐Clark, M., Westreich, D. J., Sanoff, H. K., Robert, N., Frytak, J. R., Boyd, M., Shmuel, S., Stürmer, T., & Keil, A. P. (2023) · Epidemiology · doi:10.1097/ede.0000000000001694 · Cited by 5 · see citing works
Self-citationWebster‐Clark, M., Filion, K. B., & Platt, R. W. (2024) · American Journal of Epidemiology · doi:10.1093/aje/kwae015 · Cited by 5 · see citing works
Levy, N. S., & Keyes, K. M. (2021) · American Journal of Epidemiology · doi:10.1093/aje/kwab274 · Cited by 5 · see citing works
Velloza, J., Roche, S. D., Concepcion, T. L., & Ortblad, K. F. (2022) · Current Opinion in HIV and AIDS · doi:10.1097/coh.0000000000000768 · Cited by 6 · see citing works
Dempsey, W. (2023) · The Annals of Applied Statistics · doi:10.1214/23-aoas1744 · Cited by 5 · see citing works
Lund, J. L., Rivera, M. P., Su, I., Long, J. M., Chen, X., Pak, J., Hudgens, M. G., Stürmer, T., Reuland, D. S., & Henderson, L. M. (2024) · Cancer Epidemiology Biomarkers & Prevention · doi:10.1158/1055-9965.epi-23-1491 · Cited by 4 · see citing works
Kondowe, F. J. M., Gittins, M., Clayton, P., Brison, D. R., & Roberts, S. A. (2025) · Human Reproduction · doi:10.1093/humrep/deaf045 · Cited by 4 · see citing works
Daniel Capurro, N., Barbé, M., Daza, C., María, J. S., & Trincado, J. (2020) · JMIR Medical Informatics · doi:10.2196/medinform.6924 · Cited by 4 · see citing works
Rysavy, M. A., Eggleston, B. S., Dahabreh, I. J., Tyson, J. E., Patel, R. M., Watterberg, K. L., Greenberg, R. G., Pedroza, C., Trotta, M., Stevenson, D. K., et al. (2023) · The Journal of Pediatrics · doi:10.1016/j.jpeds.2023.113453 · Cited by 4 · see citing works
Keyes, K. M., Pakserian, D., Rudolph, K. E., Salum, G. A., & Stuart, E. A. (2024) · Current topics in behavioral neurosciences · doi:10.1007/7854_2024_465 · Cited by 4 · see citing works
Self-citationLesko et al. Respond to “The Importance of Descriptive Epidemiology”
Lesko, C. R., Fox, M. P., & Edwards, J. K. (2022) · American Journal of Epidemiology · doi:10.1093/aje/kwac152 · Cited by 4 · see citing works
Self-citationSimpson, A., & Ramagopalan, S. V. (2022) · Journal of Comparative Effectiveness Research · doi:10.2217/cer-2022-0194 · Cited by 4 · see citing works
- Authors
- Westreich, D., Edwards, J. K., Lesko, C. R., Cole, S. R., & Stuart, E. A.
- Affiliations
- University of North Carolina at Chapel Hill (US)
- Johns Hopkins University (US)
- Corresponding author
- Westreich, Daniel
- Venue
- American Journal of Epidemiology
- Publisher
- Oxford University Press (OUP)
- Published
- 2018-10-09
- Volume
- 188
- Issue
- 2
- Pages
- 438-443
- ISSN
- 0002-9262 (ISSN-L)
- 1476-6256
- PMC
- PMC6357801
- OpenAlex
- W2900846215
- Cited by
- 165
- References
- 34
- Citations by year
- 2026: 15 · 2025: 15 · 2024: 28 · 2023: 26 · 2022: 24 · 2021: 21 · 2020: 23 · 2019: 12
- FWCI
- 11.62
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model (vor)
- Locations
- American Journal of Epidemiology (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Carolina Digital Repository (University of North Carolina at Chapel Hill) (submittedVersion)
- UNC Libraries
- Funders
- Eunice Kennedy Shriver National Institute of Child Health and Human Development
- National Institutes of Health (DP2-HD084070)
- National Institute of Allergy and Infectious Diseases (R01 AI100654)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- target bias, generalizability, internal validity, causal effect estimation, target population, transportability, causal inference
- MeSH terms
- Epidemiologic Methods, Humans, Reproducibility of Results, Bias, Causality
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
