External validity of randomised controlled trials: “To whom do the results of this trial apply?”
The DOI matches its Crossref record; 152 of the 157 references it cites were found in a registry or an index.
Abstract
In making treatment decisions, doctors and patients must take into account relevant randomised controlled trials (RCTs) and systematic reviews. Relevance depends on external validity (or generalisability)--ie, whether the results can be reasonably applied to a definable group of patients in a particular clinical setting in routine practice. There is concern among clinicians that external validity is often poor, particularly for some pharmaceutical industry trials, a perception that has led to underuse of treatments that are effective. Yet researchers, funding agencies, ethics committees, the pharmaceutical industry, medical journals, and governmental regulators alike all neglect external validity, leaving clinicians to make judgments. However, reporting of the determinants of external validity in trial publications and systematic reviews is usually inadequate. This review discusses those determinants, presents a checklist for clinicians, and makes recommendations for greater consideration of external validity in the design and reporting of RCTs.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Such specification is required for replication of trial results, comparison between DHI, synthesising data across trials in systematic reviews and meta-analyses 55 and may help with determining the criteria for ‘substantial equivalence’ of digital interventions.”
From Evaluating Digital Health Interventions (2016) · cites this paper as [55]
In the citing paper
“RCTs maximize the internal validity, to ensure that the effect found can be attributed to the investigated intervention [19,20].”
In the citing paper
“However, the findings of typical economic evaluations (e.g., based on clinical trials) are rarely sufficient to inform decisions about how health service organizations and policymakers should finance investments in EBPs.”
From Translating economic evaluations into financing strategies for implementing evidence-based practices (2021) · cites this paper as [15]
Citations
Cites
Tissue Plasminogen Activator for Acute Ischemic Stroke
(1995) · New England Journal of Medicine · doi:10.1056/nejm199512143332401 · Cited by 11,654
Downs, S. H., & Black, N. (1998) · Journal of Epidemiology and Community Health · doi:10.1136/jech.52.6.377 · Cited by 8,252
Why Don't Physicians Follow Clinical Practice Guidelines?: A Framework for Improvement
Cabana, M. D., Rand, C. S., Powe, N. R., Wu, A. W., Wilson, M. H., Abboud, P.-A. C., & Rubin, H. R. (1999) · JAMA · doi:10.1001/jama.282.15.1458 · Cited by 6,677
Incidence of Adverse Drug Reactions in Hospitalized Patients: A Meta-analysis of Prospective Studies
Lazarou, J., Pomeranz, B. H., & Corey, P. N. (1998) · JAMA · doi:10.1001/jama.279.15.1200 · Cited by 4,807
The Effect of Carvedilol on Morbidity and Mortality in Patients with Chronic Heart Failure
Packer, M., Bristow, M. R., Cohn, J. N., Colucci, W. S., Fowler, M. B., Gilbert, E. M., & Shusterman, N. H. (1996) · New England Journal of Medicine · doi:10.1056/nejm199605233342101 · Cited by 4,649
Endarterectomy for Asymptomatic Carotid Artery Stenosis
Walker, M. D. (1995) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.1995.03520420037035 · Cited by 4,403
The Revised CONSORT Statement for Reporting Randomized Trials: Explanation and Elaboration
Altman, D. G., Schulz, K. F., Moher, D., Egger, M., Davidoff, F., Elbourne, D., Gøtzsche, P. C., Lang, T., & for the CONSORT Group (2001) · Annals of Internal Medicine · doi:10.7326/0003-4819-134-8-200104170-00012 · Cited by 3,835
Randomized, Controlled Trials, Observational Studies, and the Hierarchy of Research Designs
Concato, J., Shah, N., & Horwitz, R. I. (2000) · New England Journal of Medicine · doi:10.1056/nejm200006223422507 · Cited by 3,563
Research works printed but not linked: 6
- Horton, 2000, Common sense and figures: the rhetoric of validity in medicine (Bradford Hill Memorial Lecture 1999), Stat Med, 19, 3149, 10.1002/1097-0258(20001215)19:23<3149::aid-sim617>3.0.co;2-e · DOI
- Ricci, 1996, Between country variations in the use of medical treatments for acute stroke: An update, Cerebrovasc Dis, 6, 133
- Roberts, 1999, The implications of variation in outcome between health professionals for the design and analysis of randomized controlled trials, Stat Med, 18, 2605, 10.1002/(sici)1097-0258(19991015)18:19<2605::aid-sim237>3.0.co;2-n · DOI
- Bond, 2003, High morbidity due to endarterectomy for asymptomatic carotid stenosis, Cerebrovasc Dis, 16, 65
- van Gijn, 1992, Down with stroke scales, Cerebrovasc Dis, 2, 244
- Lip, 1997, A survey of atrial fibrillation in general practice, Br J Gen Pract, 47, 285
Cited by
Moher, D., Hopewell, S., Schulz, K. F., Montori, V., Gotzsche, P. C., Devereaux, P. J., Elbourne, D., Egger, M., & Altman, D. G. (2010) · BMJ · doi:10.1136/bmj.c869 · Cited by 8,883
Moher, D., Hopewell, S., Schulz, K. F., Montori, V., Gøtzsche, P. C., Devereaux, P. J., Elbourne, D., Egger, M., & Altman, D. G. (2012) · International Journal of Surgery · doi:10.1016/j.ijsu.2011.10.001 · Cited by 2,345
“In particular, a clear understanding of these criteria is one of several elements required to judge to whom the results of a trial apply d that is, the trial’s generalisability (applicability) and relevance to clinical or public health practice (see item 21) [94].”
BackgroundGlanz, K., & Bishop, D. B. (2010) · Annual Review of Public Health · doi:10.1146/annurev.publhealth.012809.103604 · Cited by 2,336
Boutron, I., Moher, D., Altman, D. G., Schulz, K. F., Ravaud, P., & for the CONSORT Group* (2008) · Annals of Internal Medicine · doi:10.7326/0003-4819-148-4-200802190-00008 · Cited by 2,292
“These differences were not mentioned in any of the publications for the European Carotid Surgery Trial, yet they probably had an important impact on the outcomes (111).”
BackgroundInfluentialMoher, D., Hopewell, S., Schulz, K. F., Montori, V., Gøtzsche, P. C., Devereaux, P. J., Elbourne, D., Egger, M., & Altman, D. G. (2010) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.03.004 · Cited by 1,858
Metadata
- Authors
- Rothwell, P. M.
- Affiliations
- University of Oxford (GB)
- Corresponding author
- Rothwell, Peter M
- ORCID
- Rothwell, Peter M
- Venue
- The Lancet
- Publisher
- Elsevier BV
- Published
- 2005-01
- Volume
- 365
- Issue
- 9453
- Pages
- 82-93
- ISSN
- 0099-5355 (ISSN-L)
- 0140-6736
- OpenAlex
- W2015712245
- Cited by
- 2,544
- References
- 157
- Citations by year
- 2026: 158 · 2025: 169 · 2024: 105 · 2023: 101 · 2022: 125 · 2021: 175 · 2020: 151 · 2019: 161 · 2018: 123 · 2017: 142 · 2016: 148 · 2015: 147 · 2014: 137 · 2013: 112 · 2012: 97
- FWCI
- 87.78
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
- Locations
- The Lancet (publishedVersion)
- PubMed (publishedVersion)
- DigitalGeorgetown (Georgetown University Library) (submittedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Health Sciences › Nursing › Issues, ethics and legal aspects › Pharmaceutical industry and healthcare
- Keywords
- generalizability, randomized controlled trials, trial reporting, systematic reviews, trial design, clinical decision-making
- MeSH terms
- Data Interpretation, Statistical, Decision Making, Humans, Publishing, Review Literature as Topic, Reproducibility of Results, Randomized Controlled Trials as Topic, Treatment Outcome, Outcome Assessment, Health Care, Practice Guidelines as Topic, Patient Selection, Evidence-Based Medicine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: External validity of randomised controlled trials: “To whom do the results of this trial apply?”. https://app.cochat.ai/papers/10.1016/s0140-6736(04)17670-8
- The paper
- Rothwell, P. M. (2005). External validity of randomised controlled trials: “To whom do the results of this trial apply?”. The Lancet. https://doi.org/10.1016/s0140-6736(04)17670-8
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.