Per-Protocol Analyses of Pragmatic Trials
The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Abstract
Pragmatic trials are designed to address real-world questions about care options. This article addresses issues that may arise from per-protocol and intention-to-treat analyses of such trials, outlines alternative analytic approaches, and provides guidance on how to choose among them.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“We developed a revised risk-of-bias assessment tool to address these issues, incorporate advances in assessment of risk of bias used in other recently developed tools, 9 10 and integrate recent developments in estimation of intervention effects from randomised trials. 11”
From RoB 2: a revised tool for assessing risk of bias in randomised trials (2019) · cites this paper as [11]
In the citing paper
“For trials comparing interventions that are sustained over time, appropriate methods also require measurement of and adjustment for the values of prognostic factors, both before and after randomisation, that predict deviations from intervention. 11”
From RoB 2: a revised tool for assessing risk of bias in randomised trials (2019) · cites this paper as [11]
In the citing paper
“Each effect might be of interest. 11”
From RoB 2: a revised tool for assessing risk of bias in randomised trials (2019) · cites this paper as [11]
In the citing paper
“RoB 2 also incorporates recent developments in estimation of intervention effects from randomised trials: we distinguish bias in the effect of assignment to interventions from bias in the effect of adhering to intervention as specified in the trial protocol.”
From RoB 2: a revised tool for assessing risk of bias in randomised trials (2019) · cites this paper as [11]
Citations
Cites
Marginal Structural Models and Causal Inference in Epidemiology
Robins, J. M., Hernán, M. Á., & Brumback, B. (2000) · Epidemiology · doi:10.1097/00001648-200009000-00011 · Cited by 5,581
Self-citationRobins, J. (1986) · Mathematical Modelling · doi:10.1016/0270-0255(86)90088-6 · Cited by 2,451
Self-citationRosenbaum, P. R. (2002) · Springer Series in Statistics · doi:10.1007/978-1-4757-3692-2 · Cited by 2,394
Doubly robust estimation in missing data and causal inference models
Bang, H., & Robins, J. M. (2005) · Biometrics · doi:10.1111/j.1541-0420.2005.00377.x · Cited by 1,998
Self-citationIntention-to-treat concept: A review
Gupta, S. (2011) · Perspectives in Clinical Research · doi:10.4103/2229-3485.83221 · Cited by 1,921
The prevention and treatment of missing data in clinical trials
Little, R. J., D'Agostino, R., Cohen, M. L., Dickersin, K., Emerson, S. S., Farrar, J. T., Frangakis, C., Hogan, J. W., Molenberghs, G., Murphy, S. A., et al. (2012) · New England Journal of Medicine · doi:10.1056/nejmsr1203730 · Cited by 1,884
Why Most Discovered True Associations Are Inflated
Ioannidis, J. P. A. (2008) · Epidemiology · doi:10.1097/ede.0b013e31818131e7 · Cited by 1,495
An introduction to instrumental variables for epidemiologists
Greenland, S. (2000) · International Journal of Epidemiology · doi:10.1093/ije/29.4.722 · Cited by 1,156
Instruments for Causal Inference: An Epidemiologist's Dream?
Hernán, M. A., & Robins, J. M. (2006) · Epidemiology · doi:10.1097/01.ede.0000222409.00878.37 · Cited by 961
Self-citationRobins, J. M., & Finkelstein, D. M. (2000) · Biometrics · doi:10.1111/j.0006-341x.2000.00779.x · Cited by 877
Self-citation
Cited by
Items 301–400 of 661
Gothe, H., Ihle, P., & Swart, E. (2021) · Das Gesundheitswesen · doi:10.1055/a-1686-8936 · Cited by 7
Szmulewicz, A., Madenci, A. L., Ferguson, R. E., Liang, M. H., Lew, R., Katz, I. R., & Hernán, M. Á. (2023) · Journal of Psychopharmacology · doi:10.1177/02698811231166460 · Cited by 9
“Decision making by clinicians and patients can be improved by complementing the estimate of the ITT effect of lithium with an estimate of the effect of lithium when used as specified in the protocol, that is, the perprotocol effect (Hernán and Robins, 2017).”
BackgroundMethodSelf-citationYang, Y., Hodge, A. M., Lynch, B. M., Dugué, P., Williamson, E., Jayasekara, H., Mishra, G. D., & English, D. R. (2023) · American Journal of Epidemiology · doi:10.1093/aje/kwad164 · Cited by 7
“All-cause mortality and cause of death data were complete to December 31, 2018, and December 31, 2016, respectively. on 20-year all-cause and 18-year cancer mortality among Australian women aged 47–52 years (13, 18, 19).”
Nudging a Nation — The Danish NUDGE Trial Concept
Johansen, N. D., Vaduganathan, M., Bhatt, A. S., & Biering‐Sørensen, T. (2023) · NEJM Evidence · doi:10.1056/evidctw2300024 · Cited by 7
Favaretti, C., Pillai, V. S., Murthy, S., Chandrasekar, A., Yan, S. D., Sulaiman, H., Gautam, A., Kaur, B., Ali, M. K., McConnell, M. A., et al. (2024) · BMC Public Health · doi:10.1186/s12889-024-19894-9 · Cited by 9
“Finally, we estimated treatment on treated (ToT) effects using an instrumental variables approach [26].”
MethodShi, J., Løberg, M., Kalager, M., Wieszczy, P., Pilonis, N., Adami, H., Kaminski, M. F., Bretthauer, M., Hernán, M. Á., The NordICC Study Group, et al. (2025) · European Journal of Epidemiology · doi:10.1007/s10654-025-01209-w · Cited by 8
“Intention-to-treat analyses underestimate the true benefits of colonoscopy screening because they include all randomized individuals regardless of receipt of the intervention, and may be less relevant in decision making for individuals who consider screening [3].”
Self-citationMei, H., Wang, J., & Ma, S. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.07.001 · Cited by 7
Evans, R., Harris, J. M., Rogers, C. A., & MacGowan, A. P. (2022) · Journal of Antimicrobial Chemotherapy · doi:10.1093/jac/dkac374 · Cited by 7
Kalia, S., Saarela, O., O’Neill, B. G., Meaney, C. A., Moineddin, R., Sullivan, F., & Greiver, M. (2023) · American Journal of Epidemiology · doi:10.1093/aje/kwad011 · Cited by 9
Liu, J. X., Treleaven, E., Whidden, C., Doumbia, S., Koné, N., Cissé, A., Diop, A., Berthé, M., Guindo, M., Koné, B. M., et al. (2024) · Bulletin of the World Health Organization · doi:10.2471/blt.23.290975 · Cited by 11
“25,26 We then estimated the IRR of under-five mortality using the time updated Poisson regression models described above with stabilized inverse probability weighting.”
MethodWebster‐Clark, M., Stürmer, T., Edwards, J. K., Poole, C., Simpson, R. J., & Lund, J. L. (2020) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.5069 · Cited by 8
Holt, J. M., Cusatis, R. N., Winn, A. N., Asan, O., Spanbauer, C., Williams, J. S., Flynn, K. E., Somai, M. M., Laud, P. W., & Crotty, B. H. (2020) · JMIR Research Protocols · doi:10.2196/20309 · Cited by 8
“In this study, the TOT analysis will adjust for participants’ nonadherence to the group assignment, a common occurrence in pragmatic trials [56].”
BackgroundMethodInfluentialGrøvle, L., Hasvik, E., & Haugen, A. J. (2021) · Pain · doi:10.1097/j.pain.0000000000002380 · Cited by 7
Lerche, A. F., Mathiassen, S. E., Rasmussen, C. L., Straker, L. M., Søgaard, K., & Holtermann, A. (2022) · BMC Public Health · doi:10.1186/s12889-022-12643-w · Cited by 7
“We will use both intention-to-treat effect estimates and perprotocol effect estimates to assess the effectiveness of the intervention, as recommended when evaluating trials conducted in real life settings [51].”
MethodThe RESOLVE Trial for people with chronic low back pain: statistical analysis plan
Bagg, M. K., Lo, S., Cashin, A. G., Herbert, R., O'Connell, N. E., Lee, H., Hübscher, M., Wand, B. M., O’Hagan, E. T., Rizzo, R. R. N., et al. (2020) · Brazilian Journal of Physical Therapy · doi:10.1016/j.bjpt.2020.06.002 · Cited by 8
“We will also estimate the average treatment effect in the treated (ATET) using propensity score weighting.(61,62)”
MethodFotheringham, J., Latimer, N. R., Froissart, M., Kronenberg, F., Stenvinkel, P., Floege, J., Eckardt, K., & Wheeler, D. C. (2020) · Clinical Kidney Journal · doi:10.1093/ckj/sfaa250 · Cited by 9
“We present a target trial observational data analysis where a large dataset was used to emulate the desired clinical trial with inclusion and exclusion criteria, follow-up time, adherence and endpoints reflecting benefits and harms [15, 16].”
MethodInfluentialRofagha, S. (2022) · Ophthalmology Retina · doi:10.1016/j.oret.2021.12.018 · Cited by 7
Housing Instability and Type 2 Diabetes Outcomes
Berkowitz, S. A., Ochoa, A., Kuhn, M. L., Dankovchik, J., Donovan, J., LaPoint, M., Gao, M., Basu, S., Hudgens, M. G., & Gold, R. B. (2025) · JAMA Network Open · doi:10.1001/jamanetworkopen.2025.4852 · Cited by 6
de Rojas, J. P., Toledo, E., Estruch, R., Guasch‐Ferré, M., Salas‐Salvadó, J., Gómez‐Gracia, E., Ros, E., Fitó, M., Arós, F., Fiol, M., et al. (2025) · American Heart Journal · doi:10.1016/j.ahj.2025.08.021 · Cited by 9
Suri, P., Heagerty, P. J., Korpak, A. M., Jensen, M. P., Gold, L. S., Chan, K. C. G., Timmons, A. K. I., Friedly, J. L., Jarvik, J. G., & Baraff, A. J. (2022) · Journal of Pain · doi:10.1016/j.jpain.2022.09.017 · Cited by 7
Bengtsen, M. B., Heide‐Jørgensen, U., Borre, M., & Nørgaard, M. (2023) · The Prostate · doi:10.1002/pros.24540 · Cited by 8
“We reported results from both ITT and PP analyses due to the limitations inherited in each approach.(41) A limitation to the ITT approach is that nonadherence can result in a disparity between the treatment that participants were assigned to receive and the treatment they actually received.”
MethodAdherence, per‐protocol effects, and the estimands framework
Keene, O. N. (2023) · Pharmaceutical Statistics · doi:10.1002/pst.2326 · Cited by 6
“In their excellent and highly influential book on causal inference, Hern (cid:1) an and Robins 1 define the latter as the per-protocol effect “ … the causal effect of treatment that would have been observed if all individuals had adhered to their assigned treatment as specified in the protocol of the experiment.”
BackgroundMethodInfluentialDebertin, J. G., Vélez, J. A. J., Corlin, L., Hidalgo, B., & Murray, E. J. (2024) · Epidemiology · doi:10.1097/ede.0000000000001758 · Cited by 6
“22 In 2016, a reanalysis of the CDP data using g-methods to account for postrandomization adherence confounding and selection bias demonstrated that these methods could successfully eliminate healthy adherer bias, at least in the placebo arm.”
MethodEstimating the impact of addressing food needs on diabetes outcomes
Berkowitz, S. A., Ochoa, A., Donovan, J., Dankovchik, J., LaPoint, M., Kuhn, M. L., Morrissey, S., Gao, M., Hudgens, M. G., Basu, S., et al. (2024) · SSM - Population Health · doi:10.1016/j.ssmph.2024.101709 · Cited by 7
“…protocol is necessary to prevent bias that could otherwise be introduced by excluding those individuals from the analysis entirely or not accounting for the fact that their data are no longer consistent with their ‘assigned ’ treatment strategy (Hern ´ an et al., 2016; Hern ´ an & Robins, 2017).”
BackgroundBelviso, N., Zhang, Y., Aronow, H. D., Wyss, R., Barbour, M. M., Kogut, S. J., Lawal, O. D., Zhan, S. Y., Don, P. K., & Wen, X. (2021) · American Journal of Epidemiology · doi:10.1093/aje/kwab242 · Cited by 7
One Drop Improves Productivity for Workers With Type 2 Diabetes
Lavaysse, L. M., Imrisek, S. D., Lee, M., Osborn, C. Y., Hirsch, A., Hoy‐Rosas, J., Nagra, H., Goldner, D., Dachis, J., & Sears, L. E. (2022) · Journal of Occupational and Environmental Medicine · doi:10.1097/jom.0000000000002577 · Cited by 7
“Therefore, to balance those limitations, we report both, which benefits the scientific and practical/clinical interpretation of findings.(35) Complete data sets for use in the ITT analyses were developed using multiple imputation to correct for missing data(36) on outcomes, specifically, follow-up absenteeism (n = 4).”
MethodBaptiste, P. J., Wong, A. Y. S., Schultze, A., Clase, C. M., Leyrat, C., Williamson, E., Powell, E. M., Mann, J. F. E., Cunnington, M. C., Teo, K. K., et al. (2024) · American Journal of Epidemiology · doi:10.1093/aje/kwae137 · Cited by 8
“11,33,34 However, since we did not have access to the outcome data from ONTARGET, we were unable to estimate this in the reference trial.”
A translational framework of genoproteomic studies for cardiovascular drug discovery
Yang, Z., Zhao, J. V., Qi, Y., Deng, X., Ji, Z., & Liu, J. (2024) · npj Cardiovascular Health · doi:10.1038/s44325-024-00015-9 · Cited by 6
“This stage involves implementing a hypothetically randomized controlled trial (i.e., target trial emulation 68,69 or pragmatic trials 70,71 ) to assess the impact of an intervention, providing a rigorous evaluation of the anticipated bene fi ts and potential risks associated with the drug-targeting…”
Methods in causal inference. Part 4: confounding in experiments
Bulbulia, J. A. (2024) · Evolutionary Human Sciences · doi:10.1017/ehs.2024.34 · Cited by 6
“Table Table 2 G 6 – 8 are adapted from Hernán and Robins (2017).”
InfluentialBernasconi, D. P., Antolini, L., Rossi, E., Blanco, J., Galimberti, S., Andersen, P. K., & Valsecchi, M. G. (2021) · International Journal of Epidemiology · doi:10.1093/ije/dyab150 · Cited by 6
Palin, V., van Staa, T. P., Steels, S., Troxel, A. B., Groenwold, R. H. H., MacDonald, T. M., Torgerson, D. J., Faries, D. E., Mancini, P., Ouwens, M., et al. (2022) · British Journal of Clinical Pharmacology · doi:10.1111/bcp.15441 · Cited by 8
Pragmatischere randomisierte Studien mit Fokus auf Registerbasierung
Lange, S., & Lauterberg, J. (2022) · Prävention und Gesundheitsförderung · doi:10.1007/s11553-022-00974-w · Cited by 6
“Dies ist insbesondere bei Nichtunterlegenheitsstudien problematisch, da hier eine Nivellierung das Studienziel begünstigen kann [6, 13].”
BackgroundInfluentialLow-dose aspirin in reproductive health: effects on menstrual cycle characteristics
Evans, M. B., Nobles, C. J., Kim, K., Hill, M. J., DeCherney, A. H., Silver, R. M., Mumford, S. L., Sjaarda, L. A., Perkins, N. J., & Schisterman, E. F. (2020) · Fertility and Sterility · doi:10.1016/j.fertnstert.2020.06.022 · Cited by 6
Psioda, M. A., Jones, S. B., Xenakis, J. G., & D’Agostino, R. B. (2021) · Medical Care · doi:10.1097/mlr.0000000000001580 · Cited by 6
Guo, F. R., Richardson, T. S., & Robins, J. M. (2021) · Biometrika · doi:10.1093/biomet/asab029 · Cited by 6
“The methods presented have close relationships to others in the literature, in particular to a series of papers by Robins, Hernán and collaborators on analyzing observational studies as a series of randomized trials (Hernán et al., 2005, 2008; Hernán and Robins, 2017), also described as emulating a desired target rand…”
MethodSelf-citationEliza, D., Syafhan, N. F., Andrajati, R., & Fitriani, S. W. (2024) · Research Journal of Pharmacy and Technology · doi:10.52711/0974-360x.2024.00662 · Cited by 5
Strategies to manage auxiliary pain medications in chronic pain trials: a topical review
Hasvik, E., Gran, J. M., Haugen, A. J., & Grøvle, L. (2022) · European Journal of Clinical Pharmacology · doi:10.1007/s00228-022-03355-6 · Cited by 6
“If the event implies discontinuation of study medication or dropout, then data obtained after the event can be considered as missing, and methods for dealing with missing data become applicable [9, 25].”
BackgroundMethodInfluentialPérez, T. V., Øvergaard, K. R., Frigessi, A., & Biele, G. (2025) · International Journal of Epidemiology · doi:10.1093/ije/dyaf010 · Cited by 6
“However, discontinuation of treatment is not necessarily a treatment protocol violation [43], so ideally we would have data on the reason for discontinuation.”
Oba, Y., Maduke, T., Fakhouri, E. W., & Goite, Y. (2025) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd015824.pub2 · Cited by 7
So, R., Kariyama, K., Oyamada, S., Matsushita, S., Nishimura, H., Tezuka, Y., Sunami, T., Furukawa, T. A., Sahker, E., Kawaguchi, M., et al. (2025) · BMJ · doi:10.1136/bmj-2024-083985 · Cited by 6
“38 39 40 We used an adjusted cluster level approach with two stage least squares estimation, 41 where randomisation assignment served as the instrumental variable and participant reported recognition of receiving advice as the endogenous variable.”
Long-Term Effect of Cocoa Extract Supplementation on Incident Hypertension
Hamaya, R., Li, S., Lau, J., Allison, M., Haring, B., Shadyab, A. H., Nudy, M. S., Martin, L. W., Rist, P. M., Manson, J. E., et al. (2025) · Hypertension · doi:10.1161/hypertensionaha.125.25209 · Cited by 6
Gupta, A., Duffield, S. J., Shephard, C. S., Ting, E., Popat, S., Cheung, W. Y., & Arora, P. (2025) · Journal of Comparative Effectiveness Research · doi:10.57264/cer-2025-0041 · Cited by 6
“Emerging approaches, such as the local average treatment effect [9] or per-protocol effects adjusted for nonadherence [10] can improve comparability across jurisdictions by aligning estimates with realistic adherence patterns, despite some challenges in interpretability and estimation relative to…”
Elucidating some common biases in randomized controlled trials using directed acyclic graphs
Gabriel, E. E., Ocampo, A., & Sjölander, A. (2025) · European Journal of Epidemiology · doi:10.1007/s10654-025-01298-7 · Cited by 8
The Role of Assignment in Defining and Identifying Causal Effects in Randomized Trials
Dahabreh, I. J., Ung, L., Hernán, M. Á., & Chiu, Y. (2026) · Epidemiology · doi:10.1097/ede.0000000000002034 · Cited by 7
Self-citationStrauss, J. A., Ranasinghe, C., Cowley, E. S., Schwingshackl, L., Shepherd, S. O., Chaplin, M., & Garner, P. (2020) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013617 · Cited by 5
Keller, S., Tschan, F., Semmer, N. K., Trelle, S., Manser, T., & Beldi, G. (2022) · Trials · doi:10.1186/s13063-022-06775-y · Cited by 6
“However, it is now well known that a naïve per-protocol analysis is problematic as it does not properly take confounding/selection bias into account [42].”
BackgroundLi, C. X., Cole, S. R., Seligman, H. K., & Berkowitz, S. A. (2023) · American Journal of Epidemiology · doi:10.1093/aje/kwad156 · Cited by 5
Deshpande, A., Walker, R. E., Schulte, R. J., Pallotta, A. M., Tereshchenko, L. G., Hu, B., Kadri, S. S., Klompas, M., & Rothberg, M. B. (2023) · Trials · doi:10.1186/s13063-023-07615-3 · Cited by 5
“We will use an inverse probability weighting approach to minimize selection bias [19].”
MethodWebster‐Clark, M., Filion, K. B., & Platt, R. W. (2024) · American Journal of Epidemiology · doi:10.1093/aje/kwae015 · Cited by 7
“32 This study was approved by the CPRD’s Independent Scientific Advisory Committee and the Jewish General Hospital Research Ethics Board.”
MethodTerasawa, T., Tadano, T., Abe, K., Sasaki, S., Hosono, S., Katayama, T., Hoshi, K., Nakayama, T., & Hamashima, C. (2025) · BMC Medicine · doi:10.1186/s12916-025-03948-9 · Cited by 8
“Sophisticated analytical approaches addressing the bias due to deviations from the intended interventions [94] and advanced data imputation methods for missing outcome data [95] may have merit in addressing the issues in analyzing pragmatic RCTs.”
Practical use of electronic health records among patients with diabetes in scientific research
Shen, Y., Zhou, J., & Hu, G. (2020) · Chinese Medical Journal · doi:10.1097/cm9.0000000000000784 · Cited by 5
“In general, like a proposal of a clinical trial, several components should be fully considered before we can precede our analysis, including eligibility criteria, treatment strategies, treatment assignment, start and end of follow-up, outcomes, causal contrasts, and statistical analysis plan.([55]) Firstly, to identif…”
BackgroundPer-protocol analysis of the ZINC trial for HIV disease among alcohol users
Lodi, S., Freiberg, M. S., Gnatienko, N., Blokhina, E. A., Yaroslavtseva, T. V., Krupitsky, E. M., Murray, E. J., Samet, J. H., & Cheng, D. M. (2021) · Trials · doi:10.1186/s13063-021-05178-9 · Cited by 7
Elahy, V., Jiang, L., Lee, S., & Odegaard, A. (2023) · Obesity · doi:10.1002/oby.23688 · Cited by 6
“The robustness of the estimates and causal inference is achieved by minimizing unmeasured confounding (selection of confounders through a thorough review of existing studies), selection bias (incomplete follow-up expanded to include questionnaire nonresponses and incomplete responses to dietary questions), measurement…”
BackgroundScheim, D. E., Aldous, C. M., Osimani, B., Fordham, E. J., & Hoy, W. E. (2023) · Journal of Clinical Medicine · doi:10.3390/jcm12113625 · Cited by 6
“efficacy of a novel treatment when protocol compliance is low [1,3,4].”
BackgroundResultInfluentialContextual assessment: evaluating a novel self-guided online therapeutic assessment
Levis, M., & Levis, A. J. (2020) · International Journal of Psychiatry in Clinical Practice · doi:10.1080/13651501.2020.1794010 · Cited by 6
“Study used the ‘per-protocol analysis’ approach (Hern an and Robins 2017), only including participants that finished all study time points.”
MethodNeuman, M. D., Kappelman, M. D., Israel, E., Ellenberg, S. S., Girman, C., Robb, J., Rabinowitz, A., & Trontell, A. E. (2020) · Contemporary Clinical Trials · doi:10.1016/j.cct.2020.106171 · Cited by 5
Statistical Considerations for Trials in Adjuvant Treatment of Colorectal Cancer
Saad, E. D., & Buyse, M. E. (2020) · Cancers · doi:10.3390/cancers12113442 · Cited by 5
“Although the ITT analysis remains the most important and the one able to exploit the advantages of randomization to the greatest extent possible, there are increasing concerns about whether estimating an effect in accordance with the ITT principle always represents the treatment effect of greatest relevance in all sit…”
BackgroundIsHak, W. W., Korouri, S., Darwish, T., Vanle, B., Dang, J., Edwards, G. G., Black, J. T., Aronow, H. U., Kimchi, A., Spiegel, B. M. R., et al. (2021) · PLoS ONE · doi:10.1371/journal.pone.0244453 · Cited by 6
“In the non-naïve per-protocol analysis, propensity scores for adherence will be estimated as a function of percentage of completed BA sessions (BA arm) and the brief medication questionnaire score (MEDS arm) at each time point and incorporated in the final models as inverse-probability weights [55].”
MethodBaber, U., Blaha, M. J., & Mehran, R. (2022) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2022.06.010 · Cited by 5
Margolis, K. L., Crain, A. L., Green, B. B., O’Connor, P. J., Solberg, L. I., Beran, M., Bergdall, A. R., Pawloski, P. A., Ziegenfuss, J. Y., JaKa, M. M., et al. (2022) · Trials · doi:10.1186/s13063-022-06611-3 · Cited by 7
“The analysis may be supplemented by a secondary per-protocol analysis that compares results in patients who adhered to the intervention in both groups, using propensity weights to take into account the post-randomization differences between groups [32].”
MethodA Practical Guide to Causal Inference in Three-Wave Panel StudiesPreprint
Bulbulia, J. A. (2024) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/uyg3d · Cited by 5
Heinrich, M., Zagorscak, P., Bohn, J., Knaevelsrud, C., & Schulze, L. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01936-0 · Cited by 4
“It is an ongoing discussion about which approach is the best e.g., 10,61,62 .”
The efficacy-effectiveness gap
Groenwold, R. H. H. (2021) · Elsevier eBooks · doi:10.1016/b978-0-12-817663-4.00024-6 · Cited by 6
Comparative safety of tramadol and other opioids following total hip and knee arthroplasty
Bosco, E., Riester, M. R., Beaudoin, F. L., Schoenfeld, A. J., Gravenstein, S., Mor, V., & Zullo, A. R. (2024) · BMC Geriatrics · doi:10.1186/s12877-024-04933-2 · Cited by 4
“We were interested in both the ITT estimand (the effect of individuals being prescribed a treatment regardless of future adherence or treatment switching) and the PP esti-mand (the effect of individuals adhering to and remaining on the prescribed treatment) [36].”
BackgroundTransportation Barriers and Diabetes Outcomes: A Longitudinal Analysis
Berkowitz, S. A., Ochoa, A., LaPoint, M., Kuhn, M. L., Dankovchik, J., Donovan, J., Gao, M., Basu, S., Hudgens, M. G., & Gold, R. B. (2025) · Journal of Primary Care & Community Health · doi:10.1177/21501319251320709 · Cited by 4
“17,28 In these cases, data were censored from the point an individual stopped following their “assigned” exposure.”
Zhao, Y., Bai, W., Ding, R., You, N., Zheng, L., Li, L., Wu, J., Zhang, P., Huang, W., Zhang, H., et al. (2025) · Liver Cancer · doi:10.1159/000546530 · Cited by 4
“Even when one treatment is clearly superior, the ITT analysis may lead to incorrect conclusions [29].”
Zhang, Z., Liu, J., Kang, Y., Liu, Y., & Fu, P. (2025) · Kidney Diseases · doi:10.1159/000548711 · Cited by 4
“It is likely that residual confounding due to patient and prescriber behaviors could have contributed to the observed findings, especially the increased risk of MACE outcome, in ways that would not occur in an RCT [47, 48].”
Wang, K., Rosman, L., & Lu, H. (2026) · European Journal of Epidemiology · doi:10.1007/s10654-025-01341-7 · Cited by 5
Merlo, C., Essig, S., Bădărău, D. O., Leuppi, J. D., Speich, B., Erlanger, T. E., Hemkens, L. G., & Zeller, A. (2020) · Trials · doi:10.1186/s13063-020-04848-4 · Cited by 4
Zhang, Y.-Y., Zhang, Q., Li, J., Wang, Y., Zhuang, Z., & Zhuang, J.-Y. (2023) · Risk Management and Healthcare Policy · doi:10.2147/rmhp.s433068 · Cited by 5
Deep brain stimulation for Tourette's syndrome
Wang, S., Zhang, Y., Wang, M., Meng, F., Liu, Y., & Zhang, J. (2024) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd015924 · Cited by 5
Patient-focused drug development and real-world study
Hu, H., Wang, H., Ang, L., Shi, M., Wu, X., Zhang, C., Han, M., Liu, S., Li, K., Zhang, J., et al. (2024) · Integrative Medicine Research · doi:10.1016/j.imr.2024.101119 · Cited by 4
Surgical interventions for presbyopia
Alvarado-Villacorta, R., Yim, T. W., Hernández-Quíntela, E., la Torre-González, E. D., Munarriz, C. A. L., & Martínez‐Zapata, M. J. (2025) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd015711.pub2 · Cited by 8
Adan, M. M., Mao, X., Ding, L., Abdullahi, K., & Li, G. (2025) · Oncology Letters · doi:10.3892/ol.2025.15094 · Cited by 4
“…adequate methods (31); deviation from intended interventions, with some trials showing discrepancies in adherence to treatment protocols (32); missing outcome data, where only a small number of studies had incomplete data, which did not significantly affect the results (33); measurement…”
Gothe, H., Gothe, R. M., Arvandi, M., Hintringer, K., Toell, T., Oberaigner, W., Rajšić, S., Kügler, J., Kiechl, S. J., Willeit, J., et al. (2020) · Das Gesundheitswesen · doi:10.1055/a-1101-8949 · Cited by 4
Rogers, A., Craig, G., Flynn, A., Mackenzie, I. S., MacDonald, T. M., & Doney, A. S. F. (2020) · Trials · doi:10.1186/s13063-020-04357-4 · Cited by 4
Karaboyas, A., Morgenstern, H., Fleischer, N. L., Schaubel, D. E., & Robinson, B. M. (2020) · Clinical Epidemiology · doi:10.2147/clep.s283321 · Cited by 4
Quantitative and Qualitative Strategies to Strengthen Internal Validity in Randomized Trials
Sidani, S., & O’Rourke, H. M. (2020) · Canadian Journal of Nursing Research · doi:10.1177/0844562120974197 · Cited by 6
“This will strengthen the confidence in attributing the outcomes to the intervention (Hernán & Robins, 2017).”
BackgroundHossain, M. B., Mosquera, L., & Karim, M. E. (2022) · BMC Medical Research Methodology · doi:10.1186/s12874-022-01518-8 · Cited by 4
“Inability to adjust for the unmeasured confounding could bias the treatment effect, depending on the method of estimation [4, 5].”
BackgroundMethodInfluentialKarim, M. E., & Hossain, M. B. (2022) · Research Methods in Medicine & Health Sciences · doi:10.1177/26320843221085649 · Cited by 4
“Tutorials, editorials, and guideline documents are being written to raise awareness, and to fill the gap in the renewed interest about this method in recent years.(1,2,22) The strength of this method lies in utilizing high-quality pre and post-randomization factors in the analysis that can predict the adherence patter…”
MethodJanvin, M., Young, J. G., & Stensrud, M. J. (2023) · Statistics in Biopharmaceutical Research · doi:10.1080/19466315.2022.2108138 · Cited by 5
Hull, K. L., Bramham, K., Brookes, C. L., Cluley, V., Conefrey, C., Cooper, N. J., Eborall, H., Fotheringham, J., Graham-Brown, M. P. M., Gray, L. J., et al. (2023) · Trials · doi:10.1186/s13063-023-07565-w · Cited by 4
“{20b} There is increasing demand by clinicians and patients for ‘per-protocol’ analyses which quantify the effect of being randomised to, receiving and continuing a treatment [44].”
BackgroundWeir, I. R., Dufault, S. M., & Phillips, P. P. J. (2024) · Trials · doi:10.1186/s13063-024-07999-w · Cited by 4
“A s i m i l a r t a r g e t t o p e r - p r o t o c o l a n a l y s e s i n p r e v i o u s t r i a l s , b u t a d i ff e r e n t e s t i m a - t i o n m e t h o d m o r e c o rr e c t t o t h e “ p e r - p r o t o c o l ” l a b e l [ 12 ] .”
Hagiwara, Y., Shinozaki, T., Mukai, H., & Matsuyama, Y. (2020) · Biometrics · doi:10.1111/biom.13296 · Cited by 5
“In response to the recent support to the per-protocol effect (Hernán and Hernández-Díaz, 2012; Hernán and Robins, 2017), adjustment for subsequent treatments not recommended in the trial protocol may be important to complement the ITT analysis result.”
BackgroundRandomized Controlled Trials 1: Design
Curtis, B. M., Barrett, B. J., & Parfrey, P. S. (2021) · Methods in molecular biology · doi:10.1007/978-1-0716-1138-8_11 · Cited by 7
Yan, J. T., Boyne, D. J., Lo, E., Farah, E., O'Sullivan, D. E., & Cheung, W. Y. (2023) · Journal of Comparative Effectiveness Research · doi:10.57264/cer-2023-0061 · Cited by 4
“…estimation of the per-protocol effect of adhering to different PRO monitoring strategies, including the assessment of whether or not there was an appropriate referral once an issue has been reported or the specific frequency and duration of PRO monitoring needed to observe a meaningful benefit [20] .”
BackgroundMethodGarcía‐Albéniz, X., Hsu, J., Etzioni, R., Chan, J. M., Shi, J., Dickerman, B. A., & Hernán, M. Á. (2024) · JCO Clinical Cancer Informatics · doi:10.1200/cci.24.00094 · Cited by 7
Self-citationKim, M., Wang, C., & Xue, X. (2018) · Statistics in Medicine · doi:10.1002/sim.7999 · Cited by 5
Sample Size Calculation When Planning Clinical Trials with Intercurrent Events
Fang, Y., & Jin, M. (2021) · Therapeutic Innovation & Regulatory Science · doi:10.1007/s43441-021-00284-x · Cited by 4
“In the definition of ATE , we use the hypothetical strategy to envisage a scenario in which the subjects would take their initially assigned treatment from t = 1 to tmax [3].”
MethodRegression and CausalityPreprint
Schomaker, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.11754 · Cited by 9
“As discussed above, re-defining the estimand may be an option in some cases; though accounting for non-adherence by measuring appropriate data and using methods other than regression is the most viable option in most cases [64, 65].”
MethodSupport for parents and families after stillbirth and neonatal death
Loughnan, S., Bakhbakhi, D., Ellwood, D. A., Boyle, F., Middleton, P., Burden, C., Ludski, K., Saunders, R., & Flenady, V. (2024) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd015798 · Cited by 3
Park, C., Song, H., Kim, H., Kim, E., Jo, H.-J., Kim, J., Lee, J. H., & Kim, J. (2025) · Journal of Medical Internet Research · doi:10.2196/69981 · Cited by 4
Dan, A. S. M., Linoby, A., Kasim, S. S., Zaki, S., Sazali, R., Yusoff, Y., Nasir, Z., & Abidin, A. H. (2026) · Journal of Technology in Behavioral Science · doi:10.1007/s41347-026-00618-2 · Cited by 3
Madley‐Dowd, P., Horne, E. M. F., Hulme, W. J., Palmer, T., Walker, V. M., Denholm, R., & Sterne, J. A. C. (2026) · Vaccine · doi:10.1016/j.vaccine.2026.128276 · Cited by 3
Hossain, M. B., & Karim, M. E. (2022) · Pharmaceutical Statistics · doi:10.1002/pst.2258 · Cited by 3
“When non-adherence only depends on baseline covariates, the baseline-adjusted PP and AT can produce unbiased estimates.(18) However, these baseline-adjusted method as well as the traditional methods to adjust for time-varying covariates should be avoided when there is treatment-confounder feedback.”
BackgroundMethodInfluentialChen, C., Huang, B., Kouril, M., Liu, J., Kim, H., Sivaganisan, S., Welge, J. A., & DelBello, M. P. (2023) · Frontiers in Computer Science · doi:10.3389/fcomp.2023.1183380 · Cited by 3
Casey, M., Pannu, S., Bajwa, S., Duarte‐García, A., & Putman, M. (2024) · Arthritis Care & Research · doi:10.1002/acr.25340 · Cited by 3
“23,24 The unadjusted incidence of neu-roin fl ammatory lesions was calculated using person years for the overall cohort, for TNF vs non-TNF, for each disease, and with respect to each class of medications.”
Mazzeo, L., Corso, F., Baili, P., Scotti, F., Torri, V., Ganzinelli, M., Miskovic, V., Leporati, R., Provenzano, L., Spagnoletti, A., et al. (2025) · ESMO Real World Data and Digital Oncology · doi:10.1016/j.esmorw.2024.100109 · Cited by 3
Oba, Y., Pathak, M., Maduke, T., Fakhouri, E. W., & Dias, S. (2025) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd015997 · Cited by 6
Rodríguez, A., Esposito, F. M., Oliveres, H., Torres, F., & Maurel, J. (2021) · Journal of Clinical Medicine · doi:10.3390/jcm10040746 · Cited by 3
“It is important to acknowledge that methodology for causal inference from real-world data has evolved substantially in the last years [6].”
BackgroundMethod
Metadata
- Authors
- Hernán, M. A., & Robins, J. M.
- Affiliations
- Harvard University (US)
- ORCID
- Robins, James M.
- Venue
- New England Journal of Medicine
- Publisher
- Massachusetts Medical Society
- Published
- 2017-10-05
- Volume
- 377
- Issue
- 14
- Pages
- 1391-1398
- ISSN
- 0028-4793 (ISSN-L)
- 1533-4406
- OpenAlex
- W2764336450
- Cited by
- 660
- References
- 32
- Citations by year
- 2026: 98 · 2025: 89 · 2024: 95 · 2023: 74 · 2022: 82 · 2021: 79 · 2020: 73 · 2019: 39 · 2018: 26 · 2017: 3
- FWCI
- 26.81
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- New England Journal of Medicine (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- pragmatic trials, per-protocol analysis, intention-to-treat analysis, real-world evidence, treatment effect estimation
- MeSH terms
- Clinical Protocols, Data Interpretation, Statistical, Humans, Patient Compliance, Research Design, Intention to Treat Analysis, Pragmatic Clinical Trials as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.