How to assess the external validity of therapeutic trials: a conceptual approach
Open accessThe DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
Abstract
BACKGROUND: External validity of study results is an important issue from a clinical point of view. From a methodological point of view, however, the concept of external validity is more complex than it seems to be at first glance. METHODS: Methodological review to address the concept of external validity. RESULTS: External validity refers to the question whether results are generalizable to persons other than the population in the original study. The only formal way to establish the external validity would be to repeat the study for that specific target population. We propose a three-way approach for assessing the external validity for specified target populations. (i) The study population might not be representative for the eligibility criteria that were intended. It should be addressed whether the study population differs from the intended source population with respect to characteristics that influence outcome. (ii) The target population will, by definition, differ from the study population with respect to geographical, temporal and ethnical conditions. Pondering external validity means asking the question whether these differences may influence study results. (iii) It should be assessed whether the study's conclusions can be generalized to target populations that do not meet all the eligibility criteria. CONCLUSION: Judging the external validity of study results cannot be done by applying given eligibility criteria to a single target population. Rather, it is a complex reflection in which prior knowledge, statistical considerations, biological plausibility and eligibility criteria all have place.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“It can reflect a high generalizability, supporting the external validity of this study.16”
From Small Interfering RNA for LDL-Cholesterol Reduction (2026) · cites this paper as [16]
In the citing paper
“it is important to note that confounding and selection bias refer to biases that are internal to the study (‘internal validity’) and not to issues of generalisability or applicability (‘external validity’) [20].”
From COSMOS-E: Guidance on conducting systematic reviews and meta-analyses of observational studies of etiology (2019) · cites this paper as [20]
Citations
Cites
The Effect of Spironolactone on Morbidity and Mortality in Patients with Severe Heart Failure
Pitt, B., Zannad, F., Remme, W. J., Cody, R., Castaigne, A., Perez, A., Palensky, J., & Wittes, J. (1999) · New England Journal of Medicine · doi:10.1056/nejm199909023411001 · Cited by 9,214
Moher, D., Schulz, K. F., & Altman, D. G. (2001) · The Lancet · doi:10.1016/s0140-6736(00)04337-3 · Cited by 5,373
The Effect of Carvedilol on Morbidity and Mortality in Patients with Chronic Heart Failure
Packer, M., Bristow, M. R., Cohn, J. N., Colucci, W. S., Fowler, M. B., Gilbert, E. M., & Shusterman, N. H. (1996) · New England Journal of Medicine · doi:10.1056/nejm199605233342101 · Cited by 4,649
External validity of randomised controlled trials: “To whom do the results of this trial apply?”
Rothwell, P. M. (2005) · The Lancet · doi:10.1016/s0140-6736(04)17670-8 · Cited by 2,544
(2005) · The Lancet · doi:10.1016/s0140-6736(05)67660-x · Cited by 1,858
Rates of hyperkalemia after publication of the Randomized Aldactone Evaluation Study
Juurlink, D. N., Mamdani, M. M., Lee, D. S., Kopp, A., Austin, P. C., Laupacis, A., & Redelmeier, D. A. (2004) · New England Journal of Medicine · doi:10.1056/nejmoa040135 · Cited by 1,761
Chen, Z.-M. (1997) · The Lancet · doi:10.1016/s0140-6736(97)04010-5 · Cited by 1,232
Materson, B. J., Reda, D. J., Cushman, W. C., Massie, B. M., Freis, E. D., Kochar, M. S., Hamburger, R. J., Fye, C., Lakshman, R., Gottdiener, J., et al. (1993) · New England Journal of Medicine · doi:10.1056/nejm199304013281303 · Cited by 1,199
Lung Cancer and Other Causes of Death in Relation to Smoking
Doll, R., & Hill, A. B. (1956) · BMJ · doi:10.1136/bmj.2.5001.1071 · Cited by 795
How to use the evidence: assessment and application of scientific evidence
Glasziou, P., Irwig, L., Bain, C., & Colditz, G. A. (2000) · Cited by 395
Travers, J., Marsh, S., Williams, M., Weatherall, M., Caldwell, B., Shirtcliffe, P., Aldington, S., & Beasley, R. (2007) · Thorax · doi:10.1136/thx.2006.066837 · Cited by 310
External Validity of Clinical Trials in Acute Myocardial Infarction
Steg, P. G. (2007) · Archives of Internal Medicine · doi:10.1001/archinte.167.1.68 · Cited by 269
Single-Drug Therapy for Hypertension in Men
Pettinger, W. A., & Lee, H. C. (1993) · New England Journal of Medicine · doi:10.1056/nejm199309303291417 · Cited by 259
Single-drug therapy for hypertension in men.
Pettinger, W. A., & Lee, H. C. (1993) · PubMed · Cited by 257
External validity: We need to do more
Glasgow, R. E., Green, L. W., Klesges, L. M., Abrams, D. B., Fisher, E. B., Goldstein, M. G., Hayman, L. L., Ockene, J. K., & Tracy Orleans, C. (2006) · Annals of Behavioral Medicine · doi:10.1207/s15324796abm3102_1 · Cited by 223
Effects of Ergocalciferol Added to Calcium on the Risk of Falls in Elderly High-Risk Women
Prince, R. L. (2008) · Archives of Internal Medicine · doi:10.1001/archinternmed.2007.31 · Cited by 215
Meier, C., Woitge, H. W., Witte, K., Lemmer, B., & Seibel, M. J. (2004) · Journal of Bone and Mineral Research · doi:10.1359/jbmr.040511 · Cited by 200
Pablos-Méndez, A., Barr, R. G., & Shea, S. (1998) · JAMA · doi:10.1001/jama.279.3.222 · Cited by 164
External validity of randomized controlled trials in COPD
Travers, J., Marsh, S., Caldwell, B. O., Williams, M., Aldington, S., Weatherall, M., Shirtcliffe, P., & Beasley, R. W. (2006) · Respiratory Medicine · doi:10.1016/j.rmed.2006.10.011 · Cited by 147
Salman, R. A., Vousden, C., & Warlow, C. P. (2005) · BMJ · doi:10.1136/bmj.38624.397569.68 · Cited by 105
Generalizability of the Results of Randomized Trials
Weiss, N. S., Koepsell, T. D., & Psaty, B. M. (2008) · Archives of Internal Medicine · doi:10.1001/archinternmed.2007.30 · Cited by 84
Reporting the study populations of clinical trials
Shapiro, S. H., Weijer, C., & Freedman, B. (2000) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(00)00227-4 · Cited by 51
Brunner, M., Cooper‐DeHoff, R. M., Gong, Y., Karnes, J. H., Langaee, T. Y., Pepine, C. J., & Johnson, J. A. (2007) · The American Journal of Cardiology · doi:10.1016/j.amjcard.2007.01.029 · Cited by 26
Yuasa, H., Kurita, K., & Westesson, P. L. (2003) · British Journal of Oral and Maxillofacial Surgery · doi:10.1016/s0266-4356(03)00037-8 · Cited by 14
Cited by
The PRECIS-2 tool: designing trials that are fit for purpose
Loudon, K., Treweek, S., Sullivan, F., Donnan, P., Thorpe, K. E., & Zwarenstein, M. (2015) · BMJ · doi:10.1136/bmj.h2147 · Cited by 1,846
Dekkers, O. M., Vandenbroucke, J. P., Cevallos, M., Renehan, A. G., Altman, D. G., & Egger, M. (2019) · PLOS Medicine · doi:10.1371/journal.pmed.1002742 · Cited by 525
Self-citationMultisystem morbidity and mortality in Cushing’s syndrome: a cohort study
Dekkers, O. M., Horváth-Puhó, E., Jørgensen, J. O. L., Cannegieter, S. C., Ehrenstein, V., Vandenbroucke, J. P., Pereira, A. M., & Sørensen, H. T. (2013) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/jc.2012-3582 · Cited by 419
Self-citationBig Data in Organ Transplantation: Registries and Administrative Claims
Massie, A. B., Kuricka, L. M., & Segev, D. L. (2014) · American Journal of Transplantation · doi:10.1111/ajt.12777 · Cited by 382
Distinguishing Case Series From Cohort Studies
Dekkers, O. M., Egger, M., Altman, D. G., & Vandenbroucke, J. P. (2012) · Annals of Internal Medicine · doi:10.7326/0003-4819-156-1-201201030-00006 · Cited by 308
Self-citation
Metadata
- Authors
- Dekkers, O. M., Elm, E. V., Algra, A., Romijn, J. A., & Vandenbroucke, J. P.
- Affiliations
- Leiden University Medical Center (NL)
- Corresponding author
- Dekkers, O M
- Venue
- International Journal of Epidemiology
- Publisher
- Oxford University Press (OUP)
- Published
- 2009-04-17
- Volume
- 39
- Issue
- 1
- Pages
- 89-94
- ISSN
- 0300-5771 (ISSN-L)
- 1464-3685
- OpenAlex
- W2145975717
- Cited by
- 219
- References
- 24
- Citations by year
- 2026: 3 · 2025: 8 · 2024: 18 · 2023: 17 · 2022: 20 · 2021: 14 · 2020: 11 · 2019: 13 · 2018: 7 · 2017: 12 · 2016: 11 · 2015: 15 · 2014: 14 · 2013: 14 · 2012: 15
- FWCI
- 11.62
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- International Journal of Epidemiology (publishedVersion)
- PubMed (publishedVersion)
- Pure Amsterdam UMC (publishedVersion)
- reroDoc Digital Library (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- Leiden Repository (Leiden University) (submittedVersion)
- BORIS (University Library Bern) (submittedVersion, other-oa)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- generalizability, target population, study population representativeness, eligibility criteria, research design, biological plausibility, prior knowledge
- MeSH terms
- Age Factors, Clinical Trials as Topic, Humans, Research Design, Socioeconomic Factors, Time Factors, Reproducibility of Results, Epidemiologic Factors, Selection Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.