The DOI matches its Crossref record; 33 of the 34 references it cites were found in a registry or an index.
Target Validity and the Hierarchy of Study Designs
Open accessAbstract
In recent years, increasing attention has been paid to problems of external validity, specifically to methodological approaches for both quantitative generalizability and transportability of study results. However, most approaches to these issues have considered external validity separately from internal validity. Here we argue that considering either internal or external validity in isolation may be problematic. Further, we argue that a joint measure of the validity of an effect estimate with respect to a specific population of interest may be more useful: We call this proposed measure target validity. In this work, we introduce and formally define target bias as the total difference between the true causal effect in the target population and the estimated causal effect in the study sample, and target validity as target bias = 0. We illustrate this measure with a series of examples and show how this measure may help us to think more clearly about comparisons between experimental and nonexperimental research results. Specifically, we show that even perfect internal validity does not ensure that a causal effect will be unbiased in a specific target population.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“after discovering a causal mechanism, researchers still need to identify conditions that modify its effects. This requires testing for differences in effects between people, calling for more diverse samples than those initially tested [3,7].”
From Hasty generalizations and generics in medical research: A systematic review (2024) · cites this paper as [7]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(2002) · Cited by 13,211
Causal Diagrams for Epidemiologic Research
Greenland, S., Pearl, J., & Robins, J. M. (1999) · Epidemiology · doi:10.1097/00001648-199901000-00008 · Cited by 3,728
Randomized, Controlled Trials, Observational Studies, and the Hierarchy of Research Designs
Concato, J., Shah, N., & Horwitz, R. I. (2000) · New England Journal of Medicine · doi:10.1056/nejm200006223422507 · Cited by 3,563
Air Pollution and Hospital Admissions for Heart Disease in Eight U.S. Counties
Schwartz, J. D. (1999) · Epidemiology · doi:10.1097/00001648-199901000-00005 · Cited by 3,045
Understanding and misunderstanding randomized controlled trials
Deaton, A., & Cartwright, N. (2018) · Social Science & Medicine · doi:10.1016/j.socscimed.2017.12.005 · Cited by 1,841
Maternal Pesticide Exposure from Multiple Sources and Selected Congenital Anomalies
Shaw, G. M., Wasserman, C. R., O’Malley, C. D., Nelson, V. E., & Jackson, R. J. (1999) · Epidemiology · doi:10.1097/00001648-199901000-00011 · Cited by 1,783
8. Misunderstandings Between Experimentalists and Observationalists About Causal Inference
Imai, K., King, G., & Stuart, E. A. (2008) · Yale University Press eBooks · doi:10.1111/j.1467-985x.2007.00527.x · Cited by 925
Self-citationWhy Representativeness Should Be Avoided
Rothman, K. J., Gallacher, J. E., & Hatch, E. E. (2013) · International Journal of Epidemiology · doi:10.1093/ije/dys223 · Cited by 861
Holm, J., Nowicka, P., Farpour‐Lambert, N. J., O’Malley, G., Hassapidou, M. N., Weiss, R., & Baker, J. L. (2014) · Obesity Facts · doi:10.1159/000365773 · Cited by 596
Generalizing evidence from randomized clinical trials to target populations: The ACTG 320 trial.
Cole, S., & Stuart, E. (2010) · American Journal of Epidemiology · doi:10.1093/aje/kwq084 · Cited by 535
Self-citation
Research works printed but not linked: 4
- Bareinboim, 2014, Advances in Neural Information Processing Systems 26., 136
- Bareinboim, 2015, Advances in Neural Information Processing Systems 27., 280
- Pearl, 2011, 2011 JSM Proceedings: Papers Presented at the Joint Statistical Meetings, Miami Beach, Florida, July 30–August 4, 2011, and Other ASA-Sponsored Conferences, 157
- Bareinboim, 2014, Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence and the Twenty-Sixth Innovative Applications of Artificial Intelligence Conference., 2410
Cited by
Sorted by citations
Findley, M. G., Kikuta, K., & Denly, M. (2021) · Annual Review of Political Science · doi:10.1146/annurev-polisci-041719-102556 · Cited by 312
Toward a Clearer Definition of Selection Bias When Estimating Causal Effects
Lu, H., Cole, S. R., Howe, C. J., & Westreich, D. J. (2022) · Epidemiology · doi:10.1097/ede.0000000000001516 · Cited by 201
Self-citationUK Biobank, big data, and the consequences of non-representativeness
Keyes, K. M., & Westreich, D. (2019) · The Lancet · doi:10.1016/s0140-6736(18)33067-8 · Cited by 187
Self-citationOn the Need to Revitalize Descriptive Epidemiology
Fox, M. P., Murray, E. J., Lesko, C. R., & Sealy-Jefferson, S. M. (2022) · American Journal of Epidemiology · doi:10.1093/aje/kwac056 · Cited by 155
Self-citationRoss, R. K., Breskin, A., & Westreich, D. J. (2020) · American Journal of Epidemiology · doi:10.1093/aje/kwaa124 · Cited by 145
Self-citation
Metadata
- Authors
- Westreich, D., Edwards, J. K., Lesko, C. R., Cole, S. R., & Stuart, E. A.
- Affiliations
- University of North Carolina at Chapel Hill (US)
- Johns Hopkins University (US)
- Corresponding author
- Westreich, Daniel
- Venue
- American Journal of Epidemiology
- Publisher
- Oxford University Press (OUP)
- Published
- 2018-10-09
- Volume
- 188
- Issue
- 2
- Pages
- 438-443
- ISSN
- 0002-9262 (ISSN-L)
- 1476-6256
- PMC
- PMC6357801
- OpenAlex
- W2900846215
- Cited by
- 165
- References
- 34
- Citations by year
- 2026: 15 · 2025: 15 · 2024: 28 · 2023: 26 · 2022: 24 · 2021: 21 · 2020: 23 · 2019: 12
- FWCI
- 11.62
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model (vor)
- Locations
- American Journal of Epidemiology (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Carolina Digital Repository (University of North Carolina at Chapel Hill) (submittedVersion)
- UNC Libraries
- Funders
- Eunice Kennedy Shriver National Institute of Child Health and Human Development
- National Institutes of Health (DP2-HD084070)
- National Institute of Allergy and Infectious Diseases (R01 AI100654)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- target bias, generalizability, internal validity, causal effect estimation, target population, transportability, causal inference
- MeSH terms
- Epidemiologic Methods, Humans, Reproducibility of Results, Bias, Causality
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
