Per-Protocol Analyses of Pragmatic Trials
The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Abstract
Pragmatic trials are designed to address real-world questions about care options. This article addresses issues that may arise from per-protocol and intention-to-treat analyses of such trials, outlines alternative analytic approaches, and provides guidance on how to choose among them.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“We developed a revised risk-of-bias assessment tool to address these issues, incorporate advances in assessment of risk of bias used in other recently developed tools, 9 10 and integrate recent developments in estimation of intervention effects from randomised trials. 11”
From RoB 2: a revised tool for assessing risk of bias in randomised trials (2019) · cites this paper as [11]
In the citing paper
“For trials comparing interventions that are sustained over time, appropriate methods also require measurement of and adjustment for the values of prognostic factors, both before and after randomisation, that predict deviations from intervention. 11”
From RoB 2: a revised tool for assessing risk of bias in randomised trials (2019) · cites this paper as [11]
In the citing paper
“Each effect might be of interest. 11”
From RoB 2: a revised tool for assessing risk of bias in randomised trials (2019) · cites this paper as [11]
In the citing paper
“RoB 2 also incorporates recent developments in estimation of intervention effects from randomised trials: we distinguish bias in the effect of assignment to interventions from bias in the effect of adhering to intervention as specified in the trial protocol.”
From RoB 2: a revised tool for assessing risk of bias in randomised trials (2019) · cites this paper as [11]
Citations
Cites
Marginal Structural Models and Causal Inference in Epidemiology
Robins, J. M., Hernán, M. Á., & Brumback, B. (2000) · Epidemiology · doi:10.1097/00001648-200009000-00011 · Cited by 5,581
Self-citationRobins, J. (1986) · Mathematical Modelling · doi:10.1016/0270-0255(86)90088-6 · Cited by 2,451
Self-citationRosenbaum, P. R. (2002) · Springer Series in Statistics · doi:10.1007/978-1-4757-3692-2 · Cited by 2,394
Doubly robust estimation in missing data and causal inference models
Bang, H., & Robins, J. M. (2005) · Biometrics · doi:10.1111/j.1541-0420.2005.00377.x · Cited by 1,998
Self-citationIntention-to-treat concept: A review
Gupta, S. (2011) · Perspectives in Clinical Research · doi:10.4103/2229-3485.83221 · Cited by 1,921
The prevention and treatment of missing data in clinical trials
Little, R. J., D'Agostino, R., Cohen, M. L., Dickersin, K., Emerson, S. S., Farrar, J. T., Frangakis, C., Hogan, J. W., Molenberghs, G., Murphy, S. A., et al. (2012) · New England Journal of Medicine · doi:10.1056/nejmsr1203730 · Cited by 1,884
Why Most Discovered True Associations Are Inflated
Ioannidis, J. P. A. (2008) · Epidemiology · doi:10.1097/ede.0b013e31818131e7 · Cited by 1,495
An introduction to instrumental variables for epidemiologists
Greenland, S. (2000) · International Journal of Epidemiology · doi:10.1093/ije/29.4.722 · Cited by 1,156
Instruments for Causal Inference: An Epidemiologist's Dream?
Hernán, M. A., & Robins, J. M. (2006) · Epidemiology · doi:10.1097/01.ede.0000222409.00878.37 · Cited by 961
Self-citationRobins, J. M., & Finkelstein, D. M. (2000) · Biometrics · doi:10.1111/j.0006-341x.2000.00779.x · Cited by 877
Self-citationTargeted Learning: Causal Inference for Observational and Experimental Data
van der Laan, M. J., & Rose, S. (2011) · Springer Series in Statistics · doi:10.1007/978-1-4419-9782-1 · Cited by 864
Rothman, K. J. (1988) · Cited by 690
The Coronary Drug Project Research Group (1980) · New England Journal of Medicine · doi:10.1056/nejm198010303031804 · Cited by 659
Sensitivity Analysis Without Assumptions
Ding, P., & VanderWeele, T. J. (2016) · Epidemiology · doi:10.1097/ede.0000000000000457 · Cited by 570
Beyond the intention-to-treat in comparative effectiveness research
Hernán, M. A., & Hernández-Díaz, S. (2012) · Clinical Trials · doi:10.1177/1740774511420743 · Cited by 528
Self-citationHolme, Ø., Løberg, M., Kalager, M., Bretthauer, M., Hernán, M. A., Aas, E., Eide, T. J., Skovlund, E., Schneede, J., Tveit, K. M., et al. (2014) · JAMA · doi:10.1001/jama.2014.8266 · Cited by 425
Self-citationBlaschke, T. F., Osterberg, L., Vrijens, B., & Urquhart, J. (2012) · Annual Review of Pharmacology and Toxicology · doi:10.1146/annurev-pharmtox-011711-113247 · Cited by 419
Clinical Trials: A Methodologic Perspective
Piantadosi, S. (2005) · Wiley Series in Probability and Statistics · doi:10.1002/0471740136 · Cited by 366
Intention-to-treat analysis and the goals of clinical trials*
Sheiner, L. B., & Rubin, D. B. (1995) · Clinical Pharmacology & Therapeutics · doi:10.1016/0009-9236(95)90260-0 · Cited by 314
Estimation of the causal effects of time-varying exposures
Robins, J., & Hernan, M. (2008) · Chapman & Hall/CRC Handbooks of Modern Statistical Methods · doi:10.1201/9781420011579.ch23 · Cited by 312
Self-citationCorrection for non-compliance in equivalence trials
Robins, J. M. (1998) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19980215)17:3<269::aid-sim763>3.0.co;2-j · Cited by 173
Self-citationHernán, M. A., Cole, S. R., Margolick, J., Cohen, M., & Robins, J. M. (2005) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.1064 · Cited by 169
Self-citationCorrection for non-compliance in equivalence trials
Robins, J. M. (1998) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19980215)17:3<371::aid-sim768>3.0.co;2-o · Cited by 167
Self-citationHave, T. R. T., Normand, S. T., Marcus, S. M., Brown, C. H., Lavori, P. W., & Duan, N. (2008) · Psychiatric Annals · doi:10.3928/00485713-20081201-10 · Cited by 146
Nonadherence to treatment protocol in published randomised controlled trials: a review
Dodd, S. R., White, I. R., & Williamson, P. R. (2012) · Trials · doi:10.1186/1745-6215-13-84 · Cited by 108
Toh, S., Hernández–Díaz, S., Logan, R. W., Robins, J. M., & Hernán, M. Á. (2010) · Epidemiology · doi:10.1097/ede.0b013e3181df1b69 · Cited by 70
Self-citationMurray, E. J., & Hernán, M. A. (2016) · Clinical Trials · doi:10.1177/1740774516634335 · Cited by 57 · cited 1×
“covariates, and used g-methods to adjust for postrandomization covariates.(10) In the updated analysis, the differ-”
MethodSelf-citationGoetghebeur, E. J. T., & Loeys, T. (2002) · Epidemiologic Reviews · doi:10.1093/epirev/24.1.85 · Cited by 47
Bounding the per-protocol effect in randomized trials: an application to colorectal cancer screening
Swanson, S. A., Holme, Ø., Løberg, M., Kalager, M., Bretthauer, M., Hoff, G. S., Aas, E., & Hernán, M. Á. (2015) · Trials · doi:10.1186/s13063-015-1056-8 · Cited by 29
Self-citationEffect of acyclovir on herpetic ocular recurrence using a structural nested model
Cole, S. R., & Chu, H. (2005) · Contemporary Clinical Trials · doi:10.1016/j.cct.2005.01.009 · Cited by 28
The per-protocol effect of immediate versus deferred antiretroviral therapy initiation
Lodi, S., Sharma, S., Lundgren, J. D., Phillips, A. N., Cole, S. R., Logan, R., Agan, B. K., Babiker, A., Klinker, H., Chu, H., et al. (2016) · AIDS · doi:10.1097/qad.0000000000001243 · Cited by 27
Self-citationDunn, G., & Goetghebeur, E. J. T. (2005) · Statistical Methods in Medical Research · doi:10.1191/0962280205sm402ed · Cited by 9
Cited by
RoB 2: a revised tool for assessing risk of bias in randomised trials
Sterne, J. A. C., Savović, J., Page, M. J., Elbers, R. G., Blencowe, N. S., Boutron, I., Cates, C. J., Cheng, H.-Y., Corbett, M. S., Eldridge, S. M., et al. (2019) · BMJ · doi:10.1136/bmj.l4898 · Cited by 34,787
“For trials comparing interventions that are sustained over time, appropriate methods also require measurement of and adjustment for both the pre- and post-randomisation prognostic factors that predict deviations from intervention.(11) For these reasons, most systematic reviews are likely to address the effect of assig…”
BackgroundMethodInfluentialSelf-citationEstruch, R., Ros, E., Salas-Salvadó, J., Covas, M.-I., Corella, D., Arós, F., Gómez-Gracia, E., Ruiz-Gutiérrez, V., Fiol, M., Lapetra, J., et al. (2018) · New England Journal of Medicine · doi:10.1056/nejmoa1800389 · Cited by 3,616
“It assumes that loss to follow-up, data collection, and adherence can be treated as sequentially randomized at each time point, given the measured prognostic factors before and after randomization.(22) Both Mediterranean-diet groups were combined for”
MethodSelf-citationAssessing risk of bias in a randomized trial
Higgins, J. P., Savović, J., Page, M. J., Elbers, R. G., & Sterne, J. A. (2019) · Cochrane Handbook for Systematic Reviews of Interventions · doi:10.1002/9781119536604.ch8 · Cited by 1,765
Effect of Colonoscopy Screening on Risks of Colorectal Cancer and Related Death
Bretthauer, M., Løberg, M., Wieszczy, P., Kalager, M., Emilsson, L., Garborg, K., Rupinski, M., Dekker, E., Spaander, M., Bugajski, M., et al. (2022) · New England Journal of Medicine · doi:10.1056/nejmoa2208375 · Cited by 817
“We estimated the per-protocol effect of screening, which was defined as the effect of screening if all the participants who were randomly assigned to the invited group had undergone screening.(14) The risk of colorectal cancer may have differed between the participants who underwent screening and those who were invite…”
BackgroundMethodInfluentialSelf-citationSpinal Anesthesia or General Anesthesia for Hip Surgery in Older Adults
Neuman, M. D., Feng, R., Carson, J. L., Gaskins, L. J., Dillane, D., Sessler, D. I., Sieber, F., Magaziner, J., Marcantonio, E. R., Mehta, S., et al. (2021) · New England Journal of Medicine · doi:10.1056/nejmoa2113514 · Cited by 478
Metadata
- Authors
- Hernán, M. A., & Robins, J. M.
- Affiliations
- Harvard University (US)
- ORCID
- Robins, James M.
- Venue
- New England Journal of Medicine
- Publisher
- Massachusetts Medical Society
- Published
- 2017-10-05
- Volume
- 377
- Issue
- 14
- Pages
- 1391-1398
- ISSN
- 0028-4793 (ISSN-L)
- 1533-4406
- OpenAlex
- W2764336450
- Cited by
- 660
- References
- 32
- Citations by year
- 2026: 98 · 2025: 89 · 2024: 95 · 2023: 74 · 2022: 82 · 2021: 79 · 2020: 73 · 2019: 39 · 2018: 26 · 2017: 3
- FWCI
- 26.81
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- New England Journal of Medicine (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- pragmatic trials, per-protocol analysis, intention-to-treat analysis, real-world evidence, treatment effect estimation
- MeSH terms
- Clinical Protocols, Data Interpretation, Statistical, Humans, Patient Compliance, Research Design, Intention to Treat Analysis, Pragmatic Clinical Trials as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.