Per-Protocol Analyses of Pragmatic Trials
The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Abstract
Pragmatic trials are designed to address real-world questions about care options. This article addresses issues that may arise from per-protocol and intention-to-treat analyses of such trials, outlines alternative analytic approaches, and provides guidance on how to choose among them.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“We developed a revised risk-of-bias assessment tool to address these issues, incorporate advances in assessment of risk of bias used in other recently developed tools, 9 10 and integrate recent developments in estimation of intervention effects from randomised trials. 11”
From RoB 2: a revised tool for assessing risk of bias in randomised trials (2019) · cites this paper as [11]
In the citing paper
“For trials comparing interventions that are sustained over time, appropriate methods also require measurement of and adjustment for the values of prognostic factors, both before and after randomisation, that predict deviations from intervention. 11”
From RoB 2: a revised tool for assessing risk of bias in randomised trials (2019) · cites this paper as [11]
In the citing paper
“Each effect might be of interest. 11”
From RoB 2: a revised tool for assessing risk of bias in randomised trials (2019) · cites this paper as [11]
In the citing paper
“RoB 2 also incorporates recent developments in estimation of intervention effects from randomised trials: we distinguish bias in the effect of assignment to interventions from bias in the effect of adhering to intervention as specified in the trial protocol.”
From RoB 2: a revised tool for assessing risk of bias in randomised trials (2019) · cites this paper as [11]
Citations
Cites
Marginal Structural Models and Causal Inference in Epidemiology
Robins, J. M., Hernán, M. Á., & Brumback, B. (2000) · Epidemiology · doi:10.1097/00001648-200009000-00011 · Cited by 5,581
Self-citationRobins, J. (1986) · Mathematical Modelling · doi:10.1016/0270-0255(86)90088-6 · Cited by 2,451
Self-citationRosenbaum, P. R. (2002) · Springer Series in Statistics · doi:10.1007/978-1-4757-3692-2 · Cited by 2,394
Doubly robust estimation in missing data and causal inference models
Bang, H., & Robins, J. M. (2005) · Biometrics · doi:10.1111/j.1541-0420.2005.00377.x · Cited by 1,998
Self-citationIntention-to-treat concept: A review
Gupta, S. (2011) · Perspectives in Clinical Research · doi:10.4103/2229-3485.83221 · Cited by 1,921
The prevention and treatment of missing data in clinical trials
Little, R. J., D'Agostino, R., Cohen, M. L., Dickersin, K., Emerson, S. S., Farrar, J. T., Frangakis, C., Hogan, J. W., Molenberghs, G., Murphy, S. A., et al. (2012) · New England Journal of Medicine · doi:10.1056/nejmsr1203730 · Cited by 1,884
Why Most Discovered True Associations Are Inflated
Ioannidis, J. P. A. (2008) · Epidemiology · doi:10.1097/ede.0b013e31818131e7 · Cited by 1,495
An introduction to instrumental variables for epidemiologists
Greenland, S. (2000) · International Journal of Epidemiology · doi:10.1093/ije/29.4.722 · Cited by 1,156
Instruments for Causal Inference: An Epidemiologist's Dream?
Hernán, M. A., & Robins, J. M. (2006) · Epidemiology · doi:10.1097/01.ede.0000222409.00878.37 · Cited by 961
Self-citationRobins, J. M., & Finkelstein, D. M. (2000) · Biometrics · doi:10.1111/j.0006-341x.2000.00779.x · Cited by 877
Self-citation
Cited by
RoB 2: a revised tool for assessing risk of bias in randomised trials
Sterne, J. A. C., Savović, J., Page, M. J., Elbers, R. G., Blencowe, N. S., Boutron, I., Cates, C. J., Cheng, H.-Y., Corbett, M. S., Eldridge, S. M., et al. (2019) · BMJ · doi:10.1136/bmj.l4898 · Cited by 34,787
“For trials comparing interventions that are sustained over time, appropriate methods also require measurement of and adjustment for both the pre- and post-randomisation prognostic factors that predict deviations from intervention.(11) For these reasons, most systematic reviews are likely to address the effect of assig…”
BackgroundMethodInfluentialSelf-citationEstruch, R., Ros, E., Salas-Salvadó, J., Covas, M.-I., Corella, D., Arós, F., Gómez-Gracia, E., Ruiz-Gutiérrez, V., Fiol, M., Lapetra, J., et al. (2018) · New England Journal of Medicine · doi:10.1056/nejmoa1800389 · Cited by 3,616
“It assumes that loss to follow-up, data collection, and adherence can be treated as sequentially randomized at each time point, given the measured prognostic factors before and after randomization.(22) Both Mediterranean-diet groups were combined for”
MethodSelf-citationAssessing risk of bias in a randomized trial
Higgins, J. P., Savović, J., Page, M. J., Elbers, R. G., & Sterne, J. A. (2019) · Cochrane Handbook for Systematic Reviews of Interventions · doi:10.1002/9781119536604.ch8 · Cited by 1,765
Effect of Colonoscopy Screening on Risks of Colorectal Cancer and Related Death
Bretthauer, M., Løberg, M., Wieszczy, P., Kalager, M., Emilsson, L., Garborg, K., Rupinski, M., Dekker, E., Spaander, M., Bugajski, M., et al. (2022) · New England Journal of Medicine · doi:10.1056/nejmoa2208375 · Cited by 817
“We estimated the per-protocol effect of screening, which was defined as the effect of screening if all the participants who were randomly assigned to the invited group had undergone screening.(14) The risk of colorectal cancer may have differed between the participants who underwent screening and those who were invite…”
BackgroundMethodInfluentialSelf-citationSpinal Anesthesia or General Anesthesia for Hip Surgery in Older Adults
Neuman, M. D., Feng, R., Carson, J. L., Gaskins, L. J., Dillane, D., Sessler, D. I., Sieber, F., Magaziner, J., Marcantonio, E. R., Mehta, S., et al. (2021) · New England Journal of Medicine · doi:10.1056/nejmoa2113514 · Cited by 478
Real-world data: A brief review of the methods, applications, challenges and opportunities
Liu, F., & Panagiotakos, D. (2022) · BMC Medical Research Methodology · doi:10.1186/s12874-022-01768-6 · Cited by 496
“Due to the data characteristics of RWD, new guidelines and methodologies are developed to mitigate bias in RWE generated by RWD for decision making and causal inference, especially for per-protocol analysis [61, 62].”
MethodWang, S. V., Schneeweiss, S., Initiative, R.-D., Franklin, J. M., Desai, R. J., Feldman, W., Garry, E. M., Glynn, R. J., Lin, K. J., Paik, J., et al. (2023) · JAMA · doi:10.1001/jama.2023.4221 · Cited by 428
MRI-Targeted or Standard Biopsy in Prostate Cancer Screening
Eklund, M., Jäderling, F., Discacciati, A., Bergman, M., Annerstedt, M., Aly, M., Glaessgen, A., Carlsson, S., Grönberg, H., & Nordström, T. (2021) · New England Journal of Medicine · doi:10.1056/nejmoa2100852 · Cited by 400
Axillary Surgery in Breast Cancer — Primary Results of the INSEMA Trial
Reimer, T., Stachs, A., Veselinovic, K., Kühn, T., Heil, J., Polata, S., Marmé, F., Müller, T., Hildebrandt, G., Krug, D., et al. (2025) · New England Journal of Medicine · doi:10.1056/nejmoa2412063 · Cited by 313
Young, J. G., Stensrud, M. J., Tchetgen Tchetgen, E. J., & Hernán, M. A. (2020) · Statistics in Medicine · doi:10.1002/sim.8471 · Cited by 338
Self-citationOmitting Axillary Dissection in Breast Cancer with Sentinel-Node Metastases
de Boniface, J., Filtenborg Tvedskov, T., Rydén, L., Szulkin, R., Reimer, T., Kühn, T., Kontos, M., Gentilini, O. D., Olofsson Bagge, R., Sund, M., et al. (2024) · New England Journal of Medicine · doi:10.1056/nejmoa2313487 · Cited by 307
Balanced Multielectrolyte Solution versus Saline in Critically Ill Adults
Finfer, S., Micallef, S., Hammond, N., Navarra, L., Bellomo, R., Billot, L., Delaney, A., Gallagher, M., Gattas, D., Li, Q., et al. (2022) · New England Journal of Medicine · doi:10.1056/nejmoa2114464 · Cited by 290
Transparent Reporting of Observational Studies Emulating a Target Trial—The TARGET Statement
Cashin, A. G., Hansford, H. J., Hernán, M. A., Swanson, S. A., Lee, H., Jones, M. D., Dahabreh, I. J., Dickerman, B. A., Egger, M., Garcia-Albeniz, X., et al. (2025) · JAMA · doi:10.1001/jama.2025.13350 · Cited by 260
Self-citationCohort Profile: Design and methods of the PREDIMED-Plus randomized trial
Martínez-González, M. A., Buil-Cosiales, P., Corella, D., Bulló, M., Fitó, M., Vioque, J., Romaguera, D., Martínez, J. A., Wärnberg, J., López-Miranda, J., et al. (2019) · International Journal of Epidemiology · doi:10.1093/ije/dyy225 · Cited by 309
Directed acyclic graphs: a tool for causal studies in paediatrics
Williams, T. C., Bach, C. C., Matthiesen, N. B., Henriksen, T. B., & Gagliardi, L. (2018) · Pediatric Research · doi:10.1038/s41390-018-0071-3 · Cited by 261
“Of course, the effect of treatment actually received may be of interest, and a per-protocol analysis, carefully controlled for confounders, may be justified to extract the maximum of information from clinical trials (52).”
Causal Inference About the Effects of Interventions From Observational Studies in Medical Journals
Dahabreh, I. J., & Bibbins-Domingo, K. (2024) · JAMA · doi:10.1001/jama.2024.7741 · Cited by 242
Automated insulin delivery in women with pregnancy complicated by type 1 diabetes
Lee, T. T. M., Collett, C., Bergford, S., Hartnell, S., Scott, E. M., Lindsay, R. S., Hunt, K. F., McCance, D. R., Barnard-Kelly, K., Rankin, D., et al. (2023) · New England Journal of Medicine · doi:10.1056/nejmoa2303911 · Cited by 241
Blum, M. R., Sallevelt, B. T. G. M., Spinewine, A., O’Mahony, D., Moutzouri, E., Feller, M., Baumgartner, C., Roumet, M., Jungo, K. T., Schwab, N., et al. (2021) · BMJ · doi:10.1136/bmj.n1585 · Cited by 218
The Target Trial Framework for Causal Inference From Observational Data: Why and When Is It Helpful?
Hernán, M. A., Dahabreh, I. J., Dickerman, B. A., & Swanson, S. A. (2025) · Annals of Internal Medicine · doi:10.7326/annals-24-01871 · Cited by 202
“Valid estimation of the per protocol effect may require that baseline and time-varying confounders be measured and appropriately adjusted for (12).”
Self-citationGLUCOCOVID investigators, Corral-Gudino, L., Bahamonde, A., Arnaiz-Revillas, F., Gómez-Barquero, J., Abadía-Otero, J., García-Ibarbia, C., Mora, V., Cerezo-Hernández, A., Hernández, J. L., et al. (2021) · Wiener klinische Wochenschrift · doi:10.1007/s00508-020-01805-8 · Cited by 214
Olanzapine versus placebo in adult outpatients with anorexia nervosa: a randomized clinical trial
Attia, E., Steinglass, J. E., Walsh, B. T., Wang, Y., Wu, P., Schreyer, C., Wildes, J., Yilmaz, Z., Guarda, A. S., Kaplan, A. S., et al. (2019) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2018.18101125 · Cited by 211
How to estimate the effect of treatment duration on survival outcomes using observational data
Hernán, M. A. (2018) · BMJ · doi:10.1136/bmj.k182 · Cited by 206
Self-citationProne position for acute respiratory failure in adults
Bloomfield, R., Noble, D. W., & Sudlow, A. (2015) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd008095.pub2 · Cited by 193
Toward a Clearer Definition of Selection Bias When Estimating Causal Effects
Lu, H., Cole, S. R., Howe, C. J., & Westreich, D. (2022) · Epidemiology · doi:10.1097/ede.0000000000001516 · Cited by 190
The estimands framework: a primer on the ICH E9(R1) addendum
Kahan, B. C., Hindley, J., Edwards, M., Cro, S., & Morris, T. P. (2024) · BMJ · doi:10.1136/bmj-2023-076316 · Cited by 180
Kent, D. M., van Klaveren, D., Paulus, J. K., D'Agostino, R., Goodman, S., Hayward, R., Ioannidis, J. P. A., Patrick-Lake, B., Morton, S., Pencina, M., et al. (2020) · Annals of Internal Medicine · doi:10.7326/m18-3668 · Cited by 179
Smith, V. A., Coffman, C. J., & Hudgens, M. G. (2021) · JAMA · doi:10.1001/jama.2021.2825 · Cited by 173
Xie, Y., Bowe, B. C., Gibson, A. K., McGill, J. B., Maddukuri, G. S., Yan, Y., & Al‐Aly, Z. (2020) · Diabetes Care · doi:10.2337/dc20-1890 · Cited by 166
Estimated Lifetime Gained With Cancer Screening Tests: A Meta-Analysis of Randomized Clinical Trials
Bretthauer, M., Wieszczy, P., Løberg, M., Kaminski, M. F., Werner, T. F., Helsingen, L. M., Mori, Y., Holme, Ø., Adami, H., & Kalager, M. (2023) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2023.3798 · Cited by 160
Naoum, G. E., Roberts, S. A., Brunelle, C. L., Shui, A. M., Salama, L., Daniell, K. M., Gillespie, T. C., Bucci, L. K., Smith, B. L., Ho, A. Y., et al. (2020) · Journal of Clinical Oncology · doi:10.1200/jco.20.00459 · Cited by 146
“4 positive LNs are present in an ITT analysis led to biased BCRL rates because ITT analyses estimate the effect of treatment assignment rather than treatment effect.(27) Other trials, such as the IBCSG23-01, reported physician assessment for BCRL with no usage of objective arm measurement, leading to subjectivity in B…”
MethodHydrochlorothiazide and Prevention of Kidney-Stone Recurrence
Dhayat, N. A., Bonny, O., Roth, B., Christe, A., Ritter, A., Mohebbi, N., Faller, N., Pellegrini, L., Bedino, G., Venzin, R. M., et al. (2023) · New England Journal of Medicine · doi:10.1056/nejmoa2209275 · Cited by 140
Heated tobacco products for smoking cessation and reducing smoking prevalence
Tattan‐Birch, H., Hartmann‐Boyce, J., Kock, L. S., Simonavicius, E., Brose, L. S., Jackson, S. E., Shahab, L., & Brown, J. (2022) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013790.pub2 · Cited by 178
Electronic medical records can be used to emulate target trials of sustained treatment strategies
Danaei, G., Rodríguez, L. A. G., Cantero, O. F., Logan, R. W., & Hernán, M. Á. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.11.021 · Cited by 152
“(22) Unfortunately, as we did not have data on the reasons to change treatment, our per-protocol effect estimates considered all deviations from the baseline treatment as non-adherence.”
BackgroundSelf-citationExtending inferences from a randomized trial to a target population
Dahabreh, I. J., & Hernán, M. Á. (2019) · European Journal of Epidemiology · doi:10.1007/s10654-019-00533-2 · Cited by 140
Self-citationArbillaga-Etxarri, A., Gimeno-Santos, E., Barberan-Garcia, A., Balcells, E., Benet, M., Borrell, E., Celorrio, N., Delgado, A., Jané, C., Marin, A., et al. (2018) · European Respiratory Journal · doi:10.1183/13993003.00063-2018 · Cited by 124
Greer, J. A., Temel, J. S., El-Jawahri, A. R., Rinaldi, S. P., Kamdar, M. M., Park, E. R., Horick, N., Pintro, K., Rabideau, D. J., Schwamm, L. H., et al. (2024) · JAMA · doi:10.1001/jama.2024.13964 · Cited by 134
Effect Estimates in Randomized Trials and Observational Studies: Comparing Apples With Apples
Lodi, S., Phillips, A. N., Lundgren, J. D., Logan, R. W., Sharma, S., Cole, S. R., Babiker, A. G., Law, M. G., Chu, H., Byrne, D. D., et al. (2019) · American Journal of Epidemiology · doi:10.1093/aje/kwz100 · Cited by 129
“As another example, randomized trials tend to use intention-to-treat estimates that quantify the effect of being assigned to treatment, regardless of whether treatment is actually received, whereas many observational studies quantify the effect of the treatment that was actually received (3).”
BackgroundSelf-citationDelayed Denosumab Injections and Fracture Risk Among Patients With Osteoporosis
Lyu, H., Yoshida, K., Zhao, S. S., Wei, J., Zeng, C., Tedeschi, S. K., Leder, B. Z., Lei, G., Tang, P., & Solomon, D. H. (2020) · Annals of Internal Medicine · doi:10.7326/m20-0882 · Cited by 115
Spence, J. D., Viscoli, C. M., Inzucchi, S. E., Dearborn-Tomazos, J., Ford, G. A., Gorman, M., Furie, K. L., Lovejoy, A. M., Young, L. H., Kernan, W. N., et al. (2019) · JAMA Neurology · doi:10.1001/jamaneurol.2019.0079 · Cited by 109
Transparent reporting of observational studies emulating a target trial: the TARGET Statement
Cashin, A. G., Hansford, H. J., Hernán, M. Á., Swanson, S. A., Lee, H., Jones, M. D., Dahabreh, I. J., Dickerman, B. A., Egger, M., García‐Albéniz, X., et al. (2025) · BMJ · doi:10.1136/bmj-2025-087179 · Cited by 122
Self-citationConsiderations for Pharmacoepidemiological Studies of Drug–Cancer Associations
Pottegård, A., Friis, S., Stürmer, T., Hallas, J., & Bahmanyar, S. (2017) · Basic & Clinical Pharmacology & Toxicology · doi:10.1111/bcpt.12946 · Cited by 110
Methodological considerations when analysing and interpreting real-world data
Stürmer, T., Wang, T., Golightly, Y. M., Keil, A. P., Lund, J. L., & Funk, M. J. (2019) · Rheumatology · doi:10.1093/rheumatology/kez320 · Cited by 116
Randomized Trial of Nocturnal Oxygen in Chronic Obstructive Pulmonary Disease
Lacasse, Y., Sériès, F., Corbeil, F., Baltzan, M., Paradis, B., Simão, P., Fernández, A. A., Esteban, C., Guimarães, M., Bourbeau, J., et al. (2020) · New England Journal of Medicine · doi:10.1056/nejmoa2013219 · Cited by 103
Castells, A., Quintero, E., Bujanda, L., Cameo, S. C., Cubiella, J., Díaz-Tasende, J., Lanas, Á., Ono, A., Serra‐Burriel, M., Frías-Arrocha, E., et al. (2025) · The Lancet · doi:10.1016/s0140-6736(25)00145-x · Cited by 95
Using propensity scores to estimate effects of treatment initiation decisions: State of the science
Webster‐Clark, M., Stürmer, T., Wang, T., Man, K. K. C., Marinac‐Dabic, D., Rothman, K. J., Ellis, A. R., Gokhale, M., Lunt, M., Girman, C. J., et al. (2020) · Statistics in Medicine · doi:10.1002/sim.8866 · Cited by 109
Petito, L. C., García‐Albéniz, X., Logan, R. W., Howlader, N., Mariotto, A. B., Dahabreh, I. J., & Hernán, M. Á. (2020) · JAMA Network Open · doi:10.1001/jamanetworkopen.2020.0452 · Cited by 104
Self-citationIntention to treat and per protocol analyses: differences and similarities
Calafell, J. M., Burón, A., Castells, X., & Porta, M. S. (2024) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2024.111457 · Cited by 103
Murray, E. J., Caniglia, E. C., & Petito, L. C. (2020) · Research Methods in Medicine & Health Sciences · doi:10.1177/2632084320961043 · Cited by 110
“Adapted from Hernan and Robins, New England Journal of Medicine, 2017.(12) Murray et al.”
BackgroundMethodMitchell, S. L., Volandes, A. E., Gutman, R., Gozalo, P. L., Ogarek, J. A., Loomer, L., McCreedy, E. M., Zhai, R., & Mor, V. (2020) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2020.2366 · Cited by 90
Wester, A., Shang, Y., Grip, E. T., Matthews, A., & Hagström, H. (2024) · Gut · doi:10.1136/gutjnl-2023-330962 · Cited by 91
“The IPCW models included age, sex, education, diabetes duration and liver disease aetiology at baseline, and the following time-varying covariates: compensated cirrhosis and the same range of comorbidities and medications as described above.”
Hoogland, J., IntHout, J., Belias, M., Rovers, M. M., Riley, R. D., Harrell, F. E., Moons, K. G. M., Debray, T. P. A., & Reitsma, J. B. (2021) · Statistics in Medicine · doi:10.1002/sim.9154 · Cited by 99
“Any conditioning on post-randomization information is no longer protected by randomization and the exchangeability assumption will no longer be guaranteed to hold [14, 15].”
BackgroundIntention-to-treat versus as-treated versus per-protocol approaches to analysis
Ahn, E. J., & Kang, H. (2023) · Korean Journal of Anesthesiology · doi:10.4097/kja.23278 · Cited by 84
“…only the randomized groups without excluding any subjects, and the AT approach, which considers only the actual treatments received without excluding subjects for noncompliance, nonadherence, or with missing data, the PP approach aims to confirm treatment effects under optimal conditions [13].”
MethodGarcía‐Albéniz, X., Hernán, M. Á., Logan, R. W., Price, M., Armstrong, K., & Hsu, J. (2020) · Annals of Internal Medicine · doi:10.7326/m18-1199 · Cited by 85
Self-citationHealth Center–Based Community-Supported Agriculture: An RCT
Berkowitz, S. A., O’Neill, J., Sayer, E., Shahid, N. N., Petrie, M., Schouboe, S., Saraceno, M., & Bellin, R. (2019) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2019.07.015 · Cited by 84
Azithromycin or Doxycycline for Asymptomatic Rectal Chlamydia trachomatis
Lau, A., Kong, F. Y. S., Fairley, C. K., Templeton, D. J., Amin, J., Phillips, S., Law, M. G., Chen, M. Y., Bradshaw, C. S., Donovan, B. J., et al. (2021) · New England Journal of Medicine · doi:10.1056/nejmoa2031631 · Cited by 71
Liberal or Restrictive Transfusion Strategy in Aneurysmal Subarachnoid Hemorrhage
English, S. W., Delaney, A. P., Fergusson, D. A., Chassé, M., Turgeon, A. F., Lauzier, F., Tuttle, A., Sadan, O., Griesdale, D. E. G., Redekop, G., et al. (2024) · New England Journal of Medicine · doi:10.1056/nejmoa2410962 · Cited by 84
Margolis, K. L., Bergdall, A. R., Crain, A. L., JaKa, M. M., Anderson, J. P., Solberg, L. I., Sperl-Hillen, J., Beran, M., Green, B. B., Haugen, P., et al. (2022) · Hypertension · doi:10.1161/hypertensionaha.122.19816 · Cited by 67
“37,38 In telehealth care, patients were considered adherent to protocol if they: (1) attended an intake visit with an MTM pharmacist within 6 weeks of the index date, (2) submitted ≥ 1 home BP measurement, and (3) completed ≥ 1 follow-up visit with the pharmacist.”
MethodChiu, Y., Chavarro, J. E., Dickerman, B. A., Manson, J. E., Mukamal, K. J., Rexrode, K. M., Rimm, E. B., & Hernán, M. Á. (2021) · American Journal of Clinical Nutrition · doi:10.1093/ajcn/nqab100 · Cited by 79
Self-citationRufibach, K. (2018) · Pharmaceutical Statistics · doi:10.1002/pst.1917 · Cited by 74
“ned treatment policy”. Related, and also not specific to T2E endpoints, Hernan and Robins address issues that may arise from uncritical reliance on the intention-to-treat principle in pragmatic trials [23]. They provide guidance on how per-protocol analyses can be used to overcome these issues. Finally, if applicable,…”
BackgroundMethodKutcher, S., Brophy, J. M., Banack, H. R., Kaufman, J. S., & Samuel, M. (2021) · Canadian Journal of Cardiology · doi:10.1016/j.cjca.2021.05.012 · Cited by 87
Jung, B., Jabaudon, M., JONG, A. D., Bitker, L., Audard, J., Klouche, K., Sarton, B., Guitton, C., Lasocki, S., Rieu, B., et al. (2025) · JAMA · doi:10.1001/jama.2025.20231 · Cited by 70
Watson, J., Dreibelbis, R., Aunger, R. V., Deola, C., King, K., Long, S., Chase, R. P., & Cumming, O. (2018) · International Journal of Hygiene and Environmental Health · doi:10.1016/j.ijheh.2018.09.002 · Cited by 69
“As this was a proof-of-concept study, we conducted a perprotocol analysis to estimate the effect of the intervention on handwashing (Hernan and Robins, 2017).”
MethodPrevention of pressure injury in the operating room: Heels operating room pressure injury trial
Eberhardt, T. D., de Lima, S. B. S., Soares, R. S. D. Á., Silveira, L. B. T. D., Pozzebon, B. R., Reis, C. R., dos Santos, K. P. P., & Alves, P. J. P. (2020) · International Wound Journal · doi:10.1111/iwj.13538 · Cited by 60
“A protocol analysis was used to assess the effects of interventions under ideal or experimental conditions.(31) This study was approved by the Ethics Committee at the Federal University of Santa Maria with Certificate of Presentation for Ethical Evaluation (CAAE) 77103617.”
MethodMurray, E. J., Caniglia, E. C., Swanson, S. A., Hernández–Díaz, S., & Hernán, M. Á. (2018) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2018.06.009 · Cited by 62
“Estimating per-protocol effects requires adjusting for post-randomization adherence and, therefore can introduce bias.(29) Adjusting for this bias is possible but requires controlling for post-randomization confounders of adherence and the outcome, using a method such as inverse probability weighting.”
MethodSelf-citationTang, B., Sjölander, A., Wastesson, J. W., Maura, G., Blotiere, P.-O., Szilcz, M., Mak, J. K. L., Qin, C., Alvarsson, M., Religa, D., et al. (2024) · eClinicalMedicine · doi:10.1016/j.eclinm.2024.102689 · Cited by 57
Bjerre, E. D., Petersen, T. H., Jørgensen, A. B., Johansen, C., Krustrup, P., Langdahl, B. L., Poulsen, M. H., Madsen, S. S., Østergren, P. B., Borre, M., et al. (2019) · PLoS Medicine · doi:10.1371/journal.pmed.1002936 · Cited by 57
“The analyses on the PP population were performed with the use of principles proposed by Hernán and colleagues [22,23].”
MethodLogan, K., Bahnson, H. T., Ylescupidez, A., Beyer, K., Bellach, J., Campbell, D. E., Craven, J., Toit, G. D., Mills, E. N. C., Perkin, M. R., et al. (2022) · Allergy · doi:10.1111/all.15597 · Cited by 61
“11 In addition to ITT and PP analyses of the pooled data from both cohorts, we have implemented causal inference analysis approaches to estimate the effect of early introduction of peanut among the different risk strata and adherence populations and to adjust PP analyses for biases often present…”
Total Hip Replacement or Resistance Training for Severe Hip Osteoarthritis
Frydendal, T., Christensen, R., Mechlenburg, I., Mikkelsen, L. R., Varnum, C., Graversen, A. E., Kjærsgaard‐Andersen, P., Revald, P., Hofbauer, C., Bieder, M. J., et al. (2024) · New England Journal of Medicine · doi:10.1056/nejmoa2400141 · Cited by 55
Limiting Dynamic Driving Pressure in Patients Requiring Mechanical Ventilation*
Urner, M., Jüni, P., Rojas‐Saunero, L. P., Hansen, B. E., Brochard, L. J., Ferguson, N. D., & Fan, E. (2023) · Critical Care Medicine · doi:10.1097/ccm.0000000000005844 · Cited by 54
“KEY WORDS: acute respiratory failure; adult; artificial; driving pressure; respiratory distress syndrome; ventilation The mortality of patients requiring mechanical ventilation in the ICU remains high (1).”
BackgroundInfluentialLauffenburger, J. C., Ghazinouri, R., Jan, S., Makanji, S., Ferro, C. A., Lewey, J., Wittbrodt, E., Lee, J., Haff, N., Fontanet, C. P., et al. (2019) · PLOS ONE · doi:10.1371/journal.pone.0214754 · Cited by 50
Frei, A. L., Radtke, T., Lana, K. D., Brun, P., Sigrist, T., Spielmanns, M., Beyer, S., Riegler, T. F., Büsching, G., Spielmanns, S., et al. (2022) · CHEST Journal · doi:10.1016/j.chest.2022.07.026 · Cited by 55
Xie, Y., Bowe, B. C., Gibson, A. K., McGill, J. B., Maddukuri, G. S., & Al‐Aly, Z. (2021) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2021.2488 · Cited by 54
Levenson, M. S., He, W., Chen, J., Fang, Y., Faries, D. E., Goldstein, B. A., Ho, M., Lee, K., Mishra-Kalyani, P. S., Rockhold, F. W., et al. (2021) · Statistics in Biopharmaceutical Research · doi:10.1080/19466315.2021.1883473 · Cited by 50
“…to each approach, with proponents of intent-to-treat analyses suggesting that it minimizes post treatment assignment bias and measures treatment strategy effectiveness, and proponents of per protocol analyses suggesting that it measures the true efficacy of the treatment (Hernán and Robins 2017).”
BackgroundNaimi, A. I., Perkins, N. J., Sjaarda, L. A., Mumford, S. L., Platt, R. W., Silver, R. M., & Schisterman, E. F. (2021) · Annals of Internal Medicine · doi:10.7326/m20-0469 · Cited by 50
“Indeed, such postrandomization confounding represents an important challenge in quantifying per protocol effects in randomized trials, accounted for with g-computation (10).”
BackgroundInfluentialHernández–Díaz, S., Huybrechts, K. F., Chiu, Y., Yland, J. J., Bateman, B. T., & Hernán, M. Á. (2022) · Epidemiology · doi:10.1097/ede.0000000000001562 · Cited by 56
Self-citationGaber, C. E., Ghazarian, A. A., Strassle, P. D., Ribeiro, T. B., Salas, M., Maringe, C., García‐Albéniz, X., Wyss, R., Du, W., & Lund, J. L. (2024) · Cancer Medicine · doi:10.1002/cam4.70461 · Cited by 59
Fu, E. L., Harhay, M. O., Schneeweiss, S., Desai, R. J., & Hernán, M. A. (2026) · BMJ · doi:10.1136/bmj-2025-084909 · Cited by 42
“The per protocol effect can be estimated via a per protocol analysis 35 that censors individuals (ie, stops their follow-up) when they stop adhering to the protocol.”
Self-citationDíaz‐Cambronero, Ó., Mazzinari, G., Lorente, B. F., García-Gregorio, N., Robles, D., Arnal, L. E. O., de Pablos, Á. M., Schultz, M. J., Errando, C. L., & Navarro, M. P. A. (2020) · British journal of surgery · doi:10.1002/bjs.11736 · Cited by 41
Matthews, A., Dahabreh, I. J., Fröbert, O., Lindahl, B., James, S. K., Feychting, M., Jernberg, T., Berglund, A., & Hernán, M. Á. (2022) · American Journal of Epidemiology · doi:10.1093/aje/kwac098 · Cited by 45
“Then, for point interventions like thrombus aspiration, the observational analysis can only estimate the per-protocol effect—that is, the effect of receiving treatment (21).”
Self-citationXie, Y., Bowe, B. C., Gibson, A. K., McGill, J. B., Yan, Y., Maddukuri, G. S., & Al‐Aly, Z. (2020) · Diabetes Care · doi:10.2337/dc20-1231 · Cited by 41
“per-protocol effect, which accounts for nonadherence and offers estimates of effectiveness that may be more generalizable across different settings (40).”
BackgroundWalker, A. S., White, I. R., Turner, R., Hsu, L. Y., Yeo, T. W., White, N. J., Sharland, M., & Thwaites, G. E. (2021) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(20)30791-x · Cited by 42
“Changes from randomised treatment could also be adjusted for using inverse probability weighting methods.(30)”
MethodA Trial Emulation Approach for Policy Evaluations with Group-level Longitudinal Data
Ben‐Michael, E., Feller, A., & Stuart, E. A. (2021) · Epidemiology · doi:10.1097/ede.0000000000001369 · Cited by 48
Cautions as Regulators Move to End Exclusive Reliance on Intention to Treat
Hernán, M. Á., & Scharfstein, D. O. (2018) · Annals of Internal Medicine · doi:10.7326/m17-3354 · Cited by 43
“One example is the per protocol effect, the effect under full adherence to the treatment strategies defined in the protocol (3).”
Self-citationEvaluation of the impact of calorie labeling on McDonald’s restaurant menus: a natural experiment
Petimar, J., Ramirez, M., Rifas‐Shiman, S. L., Linakis, S. K., Mullen, J., Roberto, C. A., & Block, J. P. (2019) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/s12966-019-0865-7 · Cited by 41
Pellegrini, V. D., Eikelboom, J. W., Evarts, C. M., Franklin, P. D., Garvin, K. L., Goldhaber, S. Z., Iorio, R. R., Lambourne, C. A., Magaziner, J. S., & Magder, L. S. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-060000 · Cited by 42
“46 47 Therefore, the primary analysis of non-inferiority will be a per protocol analysis of patients receiving the medication to which they were randomised.”
BackgroundYin, Y., Zhang, L., Marshall, I. J., Wolfe, C. D. A., & Wang, Y. (2022) · Neuroepidemiology · doi:10.1159/000525672 · Cited by 41
Heterogeneity in pragmatic randomised trials: sources and management
Giraudeau, B., Caille, A., Eldridge, S. M., Weijer, C., Zwarenstein, M., & Taljaard, M. (2022) · BMC Medicine · doi:10.1186/s12916-022-02569-w · Cited by 44
“Some argue that per-protocol analyses are of interest if the intervention is expected to be scaled up in settings where adherence to treatment is expected to be better than in the conducted trial [40].”
BackgroundBikdeli, B., Welsh, J. W., Akram, Y., Punnanithinont, N., Lee, I., Desai, N. R., Kaul, S., Stone, G. W., Ross, J. S., & Krumholz, H. M. (2019) · Circulation · doi:10.1161/circulationaha.119.040214 · Cited by 45
Li, P., Lin, H., Ni, Z., Zhan, Y., He, Y., Yang, H., Fang, J., Wang, N., Li, W., Cai, G., et al. (2020) · Phytomedicine · doi:10.1016/j.phymed.2020.153231 · Cited by 47
García‐Albéniz, X., del Amo, J., Polo, R., Morales‐Asencio, J. M., & Hernán, M. Á. (2022) · European Journal of Epidemiology · doi:10.1007/s10654-022-00891-4 · Cited by 38
Self-citationHevey, D., O’Raghallaigh, J. W., O’Doherty, V., Lonergan, K., & Heffernan, M. (2018) · Chronic Illness · doi:10.1177/1742395318792063 · Cited by 45
“ITT analysis results are affected by the trial-specific pattern of adherence to the treatment under study and therefore may not be directly relevant for guiding decisions in clinical settings.(28) Bootstrapped hierarchal multi-level modeling (MLM) for continuous data and Poisson models for the health care utilization…”
MethodAbuogi, L., Onono, M. A., Odeny, T., Owuor, K. O., Helová, A., Hampanda, K., Odwar, T., Onyango, D., McClure, L. A., Bukusi, E. A., et al. (2022) · Journal of the International AIDS Society · doi:10.1002/jia2.25852 · Cited by 39
“Thus, although our PP analyses had promising results for women who received higher exposure to the interventions, the potential biases and limitations of PP analyses are acknowledged [34].”
ResultKhera, R., Schuemie, M. J., Lu, Y., Ostropolets, A., Chen, R., Hripcsak, G., Ryan, P. B., Krumholz, H. M., & Suchard, M. A. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-057977 · Cited by 37
Piske, M., Thomson, T. J., Krebs, E., Hongdilokkul, N., Bruneau, J., Greenland, S., Gustafson, P., Karim, M. E., McCandless, L. C., Maclure, M., et al. (2020) · BMJ Open · doi:10.1136/bmjopen-2019-036102 · Cited by 38
“While the assumption of no uncontrolled confounding cannot be verified in observational settings, we adjust for all potential confounders available within our linked database.”
BackgroundMacDonald, C., Nielsen, S. M., Bjørner, J. B., Johansen, M. Y., Christensen, R., Vaag, A. A., Lieberman, D. E., Pedersen, B. K., Langberg, H., Ried‐Larsen, M., et al. (2021) · BMJ Open Diabetes Research & Care · doi:10.1136/bmjdrc-2020-001840 · Cited by 41
“The sensitivity analyses using the per-protocol population (table 3) and the single-imputation non-responder technique of missing data confirmed the robustness of these findings (online supplemental table 1).”
Rein, S. M., Lodi, S., Logan, R. W., Touloumi, G., Antoniadou, A., Wittkop, L., Bonnet, F., van Sighem, A., van der Valk, M., Reiss, P., et al. (2023) · The Lancet HIV · doi:10.1016/s2352-3018(23)00233-3 · Cited by 41
Self-citationGuideline-Based Physical Activity and Survival Among US Men With Nonmetastatic Prostate Cancer
Dickerman, B. A., Giovannucci, E. L., Pernar, C. H., Mucci, L. A., & Hernán, M. Á. (2018) · American Journal of Epidemiology · doi:10.1093/aje/kwy261 · Cited by 37
Self-citationBell, M. L., Floden, L. L., Rabe, B. A., Hudgens, S. A., Dhillon, H. M., Bray, V., & Vardy, J. L. (2019) · Patient Related Outcome Measures · doi:10.2147/prom.s178963 · Cited by 40
“For example, the trial’s objective may be to determine efficacy (causal effect of the intervention), rather than effectiveness (treatment policy), which the ITT set may not be appropriate for.(15,18,20) Thoughtful ITT analyses have been proposed.”
Boyne, D. J., Cuthbert, C. A., O'Sullivan, D. E., Sajobi, T. T., Hilsden, R. J., Friedenreich, C. M., Cheung, W. Y., & Brenner, D. R. (2019) · JAMA Network Open · doi:10.1001/jamanetworkopen.2019.4154 · Cited by 36
Yang, Y., Dixon‐Suen, S. C., Dugué, P., Hodge, A. M., Lynch, B. M., & English, D. R. (2021) · International Journal of Epidemiology · doi:10.1093/ije/dyab181 · Cited by 37
Metadata
- Authors
- Hernán, M. A., & Robins, J. M.
- Affiliations
- Harvard University (US)
- ORCID
- Robins, James M.
- Venue
- New England Journal of Medicine
- Publisher
- Massachusetts Medical Society
- Published
- 2017-10-05
- Volume
- 377
- Issue
- 14
- Pages
- 1391-1398
- ISSN
- 0028-4793 (ISSN-L)
- 1533-4406
- OpenAlex
- W2764336450
- Cited by
- 660
- References
- 32
- Citations by year
- 2026: 98 · 2025: 89 · 2024: 95 · 2023: 74 · 2022: 82 · 2021: 79 · 2020: 73 · 2019: 39 · 2018: 26 · 2017: 3
- FWCI
- 26.81
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- New England Journal of Medicine (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- pragmatic trials, per-protocol analysis, intention-to-treat analysis, real-world evidence, treatment effect estimation
- MeSH terms
- Clinical Protocols, Data Interpretation, Statistical, Humans, Patient Compliance, Research Design, Intention to Treat Analysis, Pragmatic Clinical Trials as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.