How to assess the external validity of therapeutic trials: a conceptual approach
Open accessThe DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
Abstract
BACKGROUND: External validity of study results is an important issue from a clinical point of view. From a methodological point of view, however, the concept of external validity is more complex than it seems to be at first glance. METHODS: Methodological review to address the concept of external validity. RESULTS: External validity refers to the question whether results are generalizable to persons other than the population in the original study. The only formal way to establish the external validity would be to repeat the study for that specific target population. We propose a three-way approach for assessing the external validity for specified target populations. (i) The study population might not be representative for the eligibility criteria that were intended. It should be addressed whether the study population differs from the intended source population with respect to characteristics that influence outcome. (ii) The target population will, by definition, differ from the study population with respect to geographical, temporal and ethnical conditions. Pondering external validity means asking the question whether these differences may influence study results. (iii) It should be assessed whether the study's conclusions can be generalized to target populations that do not meet all the eligibility criteria. CONCLUSION: Judging the external validity of study results cannot be done by applying given eligibility criteria to a single target population. Rather, it is a complex reflection in which prior knowledge, statistical considerations, biological plausibility and eligibility criteria all have place.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“It can reflect a high generalizability, supporting the external validity of this study.16”
From Small Interfering RNA for LDL-Cholesterol Reduction (2026) · cites this paper as [16]
In the citing paper
“it is important to note that confounding and selection bias refer to biases that are internal to the study (‘internal validity’) and not to issues of generalisability or applicability (‘external validity’) [20].”
From COSMOS-E: Guidance on conducting systematic reviews and meta-analyses of observational studies of etiology (2019) · cites this paper as [20]
Citations
Cites
The Effect of Spironolactone on Morbidity and Mortality in Patients with Severe Heart Failure
Pitt, B., Zannad, F., Remme, W. J., Cody, R., Castaigne, A., Perez, A., Palensky, J., & Wittes, J. (1999) · New England Journal of Medicine · doi:10.1056/nejm199909023411001 · Cited by 9,214
Moher, D., Schulz, K. F., & Altman, D. G. (2001) · The Lancet · doi:10.1016/s0140-6736(00)04337-3 · Cited by 5,373
The Effect of Carvedilol on Morbidity and Mortality in Patients with Chronic Heart Failure
Packer, M., Bristow, M. R., Cohn, J. N., Colucci, W. S., Fowler, M. B., Gilbert, E. M., & Shusterman, N. H. (1996) · New England Journal of Medicine · doi:10.1056/nejm199605233342101 · Cited by 4,649
External validity of randomised controlled trials: “To whom do the results of this trial apply?”
Rothwell, P. M. (2005) · The Lancet · doi:10.1016/s0140-6736(04)17670-8 · Cited by 2,544
(2005) · The Lancet · doi:10.1016/s0140-6736(05)67660-x · Cited by 1,858
Rates of hyperkalemia after publication of the Randomized Aldactone Evaluation Study
Juurlink, D. N., Mamdani, M. M., Lee, D. S., Kopp, A., Austin, P. C., Laupacis, A., & Redelmeier, D. A. (2004) · New England Journal of Medicine · doi:10.1056/nejmoa040135 · Cited by 1,761
Chen, Z.-M. (1997) · The Lancet · doi:10.1016/s0140-6736(97)04010-5 · Cited by 1,232
Materson, B. J., Reda, D. J., Cushman, W. C., Massie, B. M., Freis, E. D., Kochar, M. S., Hamburger, R. J., Fye, C., Lakshman, R., Gottdiener, J., et al. (1993) · New England Journal of Medicine · doi:10.1056/nejm199304013281303 · Cited by 1,199
Lung Cancer and Other Causes of Death in Relation to Smoking
Doll, R., & Hill, A. B. (1956) · BMJ · doi:10.1136/bmj.2.5001.1071 · Cited by 795
Cited by
Items 201–270 of 270
Mwaniki, D. N. (2022) · SU+ Digital Repository (Strathmore University) · Cited by 2
Hardy, A., Beetz, G., Chemali, K., Courgeon, M., Sandman, É., Williamson, D. R., Goetti, P., Leduc, S., Beaumont, L. D., & Rouleau, D. M. (2022) · International Orthopaedics · doi:10.1007/s00264-022-05642-4 · Cited by 1
What phases of the drug development framework can epidemiological studies emulate?
Burgess, S., Woolf, B. A. R., Bassett, E., Robertson, D. S., Villar, S. S., & Labrecque, J. A. (2026) · American Journal of Epidemiology · doi:10.1093/aje/kwag194 · Cited by 1
ASSIST Applicability Scoring of Surgical trials. An Investigator-reported aSsessment Tool
Tourabaly, I., Boutron, I., Nizard, R., & Ravaud, P. (2012) · PLoS ONE · doi:10.1371/journal.pone.0042258 · Cited by 1
Machine Learning Methods for Causal Inference with Observational Biomedical Data
Averitt, A. (2020) · doi:10.7916/d8-je06-eh12 · Cited by 1
Martínez‐García, M. Á., Oscullo, G., & Gómez-Olivas, J. D. (2022) · SLEEP · doi:10.1093/sleep/zsac021 · Cited by 1
Azevedo, M. A., & Baetu, T. M. (2023) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.13931 · Cited by 1
Adjustment of selection bias for clinical trials: a simulation study
Lu, Y., Chen, H., Wang, W., Huang, Y., Cheng, F., & Daley, E. M. (2024) · Communications in Statistics - Simulation and Computation · doi:10.1080/03610918.2024.2316281 · Cited by 1
Johnston, J. D. (2025) · Statistics in Biopharmaceutical Research · doi:10.1080/19466315.2024.2441857 · Cited by 1
Introduction: The Economic Approach to the Analysis of Terrorism
Basuchoudhary, A., & Schulze, G. G. (2026) · Cambridge University Press eBooks · doi:10.1017/9781108989046.001 · Cited by 1
Model-based drug development: the case of delayed-onset muscle soreness
Katz, N. P. (2015) · Pain · doi:10.1097/j.pain.0000000000000168 · Cited by 1
International joint venture success in the automotive industry
Staley, L. (2015) · Loughborough University Institutional Repository (Loughborough University) · Cited by 1
Singleton, K. (2016) · eScholarship (California Digital Library) · Cited by 1
Multimorbidity challenges physical activity: Implications for targeting patient treatment
Dörenkamp, S. (2016) · Research Publications (Maastricht University) · doi:10.26481/dis.20160915sd · Cited by 1
Raise the roof, build the floor
Pecker, L. H., & Ellsworth, P. (2022) · Blood · doi:10.1182/blood.2022017213 · Cited by 1
Small Interfering RNA for LDL-Cholesterol Reduction
Batista, F., Mamede, I., & Dacoregio, M. (2026) · JACC: Advances · doi:10.1016/j.jacadv.2026.102932
Analysis and modeling of quality improvement on clinical fitness landscapes
Manukyan, N. (2014) · ScholarWorks -A service of University of Vermont Libraries (University of Vermont)
Determining Evidence-based Practices in Asthma Management
Smiley, E. (2011) · CORE Scholar (Wright State University)
Is critical care service relevant to Iran's hospital care?
Jeddian, A. (2014) · University of Birmingham Institutional Research Archive (University of Birmingham)
Critique of African RCTs into Male Circumcision and HIV Sexual Transmission
Boyle, G. J. (2013) · Springer eBooks · doi:10.1007/978-94-007-6407-1_15
Mid-urethral slings for treatment of stress urinary incontinence : long- and mid-term follow-up
Palva, K. (2011) · Helda - University of Helsinki Institutional Repository
Insights from a User-Centered Approach to Computerized Guidelines for Chronic Disease.
Shaten, B. J. (2011) · University of Minnesota Digital Conservancy (University of Minnesota)
Kessler, M. C. M. (2015) · RIUnB Institutional Repository (University of Brasília) · doi:10.26512/2015.12.t.19524
Calibrating care in midwifery: weighing the evidence on weight and weight gain for pregnant women
Daemers (2017) · Research Publications (Maastricht University) · doi:10.26481/dis.20170531dd
Mijan, M. C. (2017) · Radboud Repository (Radboud University)
Overcoming Barriers to HIV/STD Partner Services in Kenya and the United States
Goyette, M. S. (2017) · ResearchWorks at the University of Washington (University of Washington)
Statistics in Endocrinology: Meta-Analysis Advantages and Pitfalls
Dekkers, O. M. (2018) · Elsevier eBooks · doi:10.1016/b978-0-12-801238-3.65799-9
Self-citationGimenez, P. (2017) · Dipòsit Digital de la Universitat de Barcelona (Universitat de Barcelona)
What indeed can be tested with an instrumental variable?
Burgess, S. (2018) · European Journal of Epidemiology · doi:10.1007/s10654-018-0423-7
Harrison, S., Buchan, V., Orsi, R., Gandy, J., & Canetto, S. (2017) · Digital Collections of Colorado (Colorado State University) · doi:10.25675/3.020146
Baron, R. (2015) · Figshare · doi:10.17613/pt1bb-60202
Concepts of Evidence-Based Practice: Analysis of Evidence-Based Practice and Its Debate
Reiersgaard, A. (2020) · NORA - Norwegian Open Research Archives
Corbett, M. S. (2019) · White Rose eTheses Online (University of Leeds, The University of Sheffield, University of York)
Generalizability of Randomized Controlled Trials in Heart Failure
Lim, Y. M. F. (2023) · Utrecht University Repository (Utrecht University) · doi:10.33540/1879
Exploring Differences in Screening and Enrollment Metrics in Orthopaedic Clinical Trials
Stiegel, L., Visperas, A., Piuzzi, N. S., & Klika, A. K. (2023) · The Journal of Knee Surgery · doi:10.1055/a-2179-8281
Ndounga‐Diakou, L. A. (2017) · theses.fr (ABES) · doi:10.70675/1c91978ez4946z40a4z8c9bz3b057c6e9dd8
Gauthier, V. (2023) · theses.fr (ABES) · doi:10.70675/b7e544cfz12e5z4f6fz9bc3z85140ebb539c
What Do RCTs Tell Us, and Could They Tell Us More? Looking Within and Beyond the Study Sample
Sim, J., Lancaster, G., & Lewis, M. (2024) · Springer eBooks · doi:10.1007/978-3-031-25859-6_5
Top Ten Tips Palliative Care Clinicians Should Know About Interpreting a Clinical Trial
Gurgenci, T., O’Leary, C., Hui, D., Yennurajalingam, S., Bruera, E. D., Davis, M. P., Agar, M. R., Zimmermann, C., Philip, J. A. M., Mercadante, S., et al. (2024) · Journal of Palliative Medicine · doi:10.1089/jpm.2024.0258
Dammann, O. (2025) · Springer eBooks · doi:10.1007/978-3-031-82271-1_5
Embracing complexity in transfusion research
Fustolo-Gunnink, S. F. (2026) · Vox Sanguinis · doi:10.1111/vox.70264
Dammann, O. (2025) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780197625835.013.0023
How to Measure the Generalizability of Clinical Trials
Lu, Y., Chen, H., Wang, W., Huang, Y., & Valente, M. J. (2025) · Medical Research Archives · doi:10.18103/mra.v13i9.6896
Economic Evaluation Methods for HIV Testing of Children and Adolescents in Zimbabwe
Vasantharoopan, A. (2025) · LSHTM Research Online (London School of Hygiene and Tropical Medicine) · doi:10.17037/pubs.04675163
Almazam, S. A. (2025) · Enlighten: Theses (The University of Glasgow) · doi:10.5525/gla.thesis.84908
Arena, L., Zanamwe, M., Halleux, C. M., Carrara, V. I., Angus, B. J., Ariana, P., Humphreys, G. S., Richmond, C. L., Stepniewska, K., Guerin, P. J., et al. (2024) · Figshare · doi:10.6084/m9.figshare.26567753
Arena, L., Zanamwe, M., Halleux, C. M., Carrara, V. I., Angus, B. J., Ariana, P., Humphreys, G. S., Richmond, C. L., Stepniewska, K., Guerin, P. J., et al. (2024) · Figshare · doi:10.6084/m9.figshare.26567732
Arena, L., Zanamwe, M., Halleux, C. M., Carrara, V., Angus, B. J., Ariana, P., Humphreys, G. S., Richmond, C. L., Stepniewska, K., Guerin, P. J., et al. (2024) · Figshare · doi:10.6084/m9.figshare.26567756
Arena, L., Zanamwe, M., Halleux, C. M., Carrara, V. I., Angus, B. J., Ariana, P., Humphreys, G. S., Richmond, C. L., Stepniewska, K., Guerin, P. J., et al. (2024) · Figshare · doi:10.6084/m9.figshare.26567735
Arena, L., Zanamwe, M., Halleux, C. M., Carrara, V., Angus, B. J., Ariana, P., Humphreys, G. S., Richmond, C. L., Stepniewska, K., Guerin, P. J., et al. (2024) · Figshare · doi:10.6084/m9.figshare.26567747
Alsallumi, K. (2025) · Enlighten: Theses (The University of Glasgow) · doi:10.5525/gla.thesis.84981
Arena, L., Zanamwe, M., Halleux, C. M., Carrara, V., Angus, B. J., Ariana, P., Humphreys, G. S., Richmond, C. L., Stepniewska, K., Guerin, P. J., et al. (2024) · Figshare · doi:10.6084/m9.figshare.26567741
Arena, L., Zanamwe, M., Halleux, C. M., Carrara, V. I., Angus, B. J., Ariana, P., Humphreys, G. S., Richmond, C. L., Stepniewska, K., Guerin, P. J., et al. (2024) · Figshare · doi:10.6084/m9.figshare.26567744
Arena, L., Zanamwe, M., Halleux, C. M., Carrara, V., Angus, B. J., Ariana, P., Humphreys, G. S., Richmond, C. L., Stepniewska, K., Guerin, P. J., et al. (2024) · Figshare · doi:10.6084/m9.figshare.26567759
Arena, L., Zanamwe, M., Halleux, C. M., Carrara, V., Angus, B. J., Ariana, P., Humphreys, G. S., Richmond, C. L., Stepniewska, K., Guerin, P. J., et al. (2024) · Figshare · doi:10.6084/m9.figshare.26567750
Arena, L., Zanamwe, M., Halleux, C. M., Carrara, V., Angus, B. J., Ariana, P., Humphreys, G. S., Richmond, C. L., Stepniewska, K., Guerin, P. J., et al. (2024) · Figshare · doi:10.6084/m9.figshare.26567738
The Value of Cybersecurity Investment in Electronic Health Records (EHRs
Katta, A. (2024) · ScholarWorks (Walden University)
Borgersen, E. (2018) · NORA - Norwegian Open Research Archives
Validez externa de los ensayos clínicos en pacientes con mieloma múltiple.
Vega González de Viñaspre, A. (2017) · Communities in ADDI (Universidad del Pais Vasco)
Wanneer leugens de waarheid veranderen : het effect van liegen op het geheugen in de rechtszaal
Riesthuis, P. (2023) · Lirias (KU Leuven)
Working with Sexually Violent Persons: Grit, the Supervisory Working Alliance, and Burnout
Harris, S. (2021) · ScholarWorks (Walden University)
Battle-Bradley, C. (2024) · ScholarWorks (Walden University)
Orenti, A., Adamoli, A., Zolin, A., & Ambrogi, F. (2023) · Archivio Istituzionale della Ricerca (Universita Degli Studi Di Milano)
Statistics in Endocrinology: Meta-Analysis Advantages and Pitfalls
Dekkers, O. M. (2019) · Elsevier eBooks · doi:10.1016/b978-0-443-13825-6.00394-0
Self-citationJevtić, S. (2026) · National Repository of Dissertations in Serbia
Comparing randomized trials: a formal approachPreprint
Chiu, Y., & Dahabreh, I. J. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.14895
Keathley, J., Garneau, V., Zavala-Mora, D., Heister, R. R., Gauthier, E., Morin-Bernier, J., Green, R., & Vohl, M.-C. (2021) · doi:10.3389/fnut.2021.789215.s001
Wiranata, J. A., Hutajulu, S. H., Astari, Y. K., Leo, B., Bintoro, B. S., Hardianti, M. S., Taroeno-Hariadi, K. W., Kurnianda, J., & Purwanto, I. (2024) · doi:10.1371/journal.pone.0298928.r004
Gheorghe, A. (2014) · University of Birmingham Institutional Research Archive (University of Birmingham)
Metadata
- Authors
- Dekkers, O. M., Elm, E. V., Algra, A., Romijn, J. A., & Vandenbroucke, J. P.
- Affiliations
- Leiden University Medical Center (NL)
- Corresponding author
- Dekkers, O M
- Venue
- International Journal of Epidemiology
- Publisher
- Oxford University Press (OUP)
- Published
- 2009-04-17
- Volume
- 39
- Issue
- 1
- Pages
- 89-94
- ISSN
- 0300-5771 (ISSN-L)
- 1464-3685
- OpenAlex
- W2145975717
- Cited by
- 219
- References
- 24
- Citations by year
- 2026: 3 · 2025: 8 · 2024: 18 · 2023: 17 · 2022: 20 · 2021: 14 · 2020: 11 · 2019: 13 · 2018: 7 · 2017: 12 · 2016: 11 · 2015: 15 · 2014: 14 · 2013: 14 · 2012: 15
- FWCI
- 11.62
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- International Journal of Epidemiology (publishedVersion)
- PubMed (publishedVersion)
- Pure Amsterdam UMC (publishedVersion)
- reroDoc Digital Library (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- Leiden Repository (Leiden University) (submittedVersion)
- BORIS (University Library Bern) (submittedVersion, other-oa)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- generalizability, target population, study population representativeness, eligibility criteria, research design, biological plausibility, prior knowledge
- MeSH terms
- Age Factors, Clinical Trials as Topic, Humans, Research Design, Socioeconomic Factors, Time Factors, Reproducibility of Results, Epidemiologic Factors, Selection Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.