Beware of on-treatment safety analyses
Open accessAbstract
Introduction: Assessing safety is important to evaluating new medications. In many randomized clinical trials, assessment of safety relies on so-called on-treatment analysis, where data on adverse events are collected only while the participant is taking study medication and perhaps for a few (7, 14, or 30) days after stopping. This article discusses the consequence of such failure to use intent-to-treat analyses in assessing safety. Methods: This article discusses two approaches to analysis of safety data: intention-to-treat and on-treatment analysis with reference to principles of the design of randomized clinical trial. Results: On-treatment analysis violates randomization and is often not well defined. Moreover, because the typical on-treatment analysis ignores the reason participants in clinical trials stop treatment, on-treatment analyses can lead to biased estimates of risk. Examples show biases that can result from failure to count all adverse events. An example from a study of rofecoxib shows an on-treatment analysis that led to likely underestimation of harm; an example from a study of saxagliptin shows an on-treatment analysis that led to a likely overestimate of harms. Conclusion: For major safety outcomes in long-term clinical trials, intention-to-treat analysis should be performed in the framework of benefit–risk evaluation. More generally, analyses of safety should be tailored to the specific question being asked with the specific study design under consideration. On-treatment analyses are subject to bias; however, the direction of that bias is not necessarily clear.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Exclusion of data from participants who did not adhere to the protocol can compromise the randomisation and introduce bias, particularly if the frequency of and the reasons for non-adherence vary between the trial groups.
“Exclusion of data from participants who did not adhere to the protocol can compromise the randomisation and introduce bias, particularly if the frequency of and the reasons for non-adherence vary between the trial groups (eg, healthy adherers bias).423”
From SPIRIT 2025 explanation and elaboration: updated guideline for protocols of randomised trials (2025) · cites this paper as [423]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Cardiovascular Events Associated with Rofecoxib in a Colorectal Adenoma Chemoprevention Trial
Bresalier, R. S., Sandler, R. S., Quan, H., Bolognese, J. A., Oxenius, B., Horgan, K., Lines, C., Riddell, R., Morton, D., Lanas, A., et al. (2005) · New England Journal of Medicine · doi:10.1056/nejmoa050493 · Cited by 2,519 · see citing works
The Coronary Drug Project Research Group (1980) · New England Journal of Medicine · doi:10.1056/nejm198010303031804 · Cited by 659 · see citing works
Through the looking glass: understanding non-inferiority
Schumi, J., & Wittes, J. T. (2011) · Trials · doi:10.1186/1745-6215-12-106 · Cited by 446 · see citing works
Self-citationA clinical review of GLP-1 receptor agonists: efficacy and safety in diabetes and beyond
Prasad-Reddy, L., & Isaacs, D. (2015) · Drugs in Context · doi:10.7573/dic.212283 · Cited by 365 · see citing works
An Additional Basic Science for Clinical Medicine: II. The Limitations of Randomized Trials
Feinstein, A. R. (1983) · Annals of Internal Medicine · doi:10.7326/0003-4819-99-4-544 · Cited by 254 · see citing works
Cardiovascular events associated with rofecoxib: final analysis of the APPROVe trial
Baron, J. A., Sandler, R. S., Bresalier, R. S., Lanas, Á., Morton, D. G., Riddell, R. H., Iverson, E. R., & DeMets, D. L. (2008) · The Lancet · doi:10.1016/s0140-6736(08)61490-7 · Cited by 232 · see citing works
Randomized Trials Analyzed as Observational Studies
Hernán, M. Á., Hernández–Dı́az, S., & Robins, J. M. (2013) · Annals of Internal Medicine · doi:10.7326/0003-4819-159-8-201310150-00709 · Cited by 181 · see citing works
Chin, M. P., Wrolstad, D., Bakris, G. L., Chertow, G. M., de Zeeuw, D., Goldsberry, A., Linde, P. G., McCullough, P. A., McMurray, J. J. V., Wittes, J. T., et al. (2014) · Journal of Cardiac Failure · doi:10.1016/j.cardfail.2014.10.001 · Cited by 164 · see citing works
Self-citationThe FDA's Critique of the Anturane Reinfarction Trial
Temple, R., & Pledger, G. W. (1980) · New England Journal of Medicine · doi:10.1056/nejm198012183032534 · Cited by 150 · see citing works
The Multiple Risk Factor Intervention Trial (MRFIT)—Importance Then and Now
Stamler, J. (2008) · JAMA · doi:10.1001/jama.300.11.1343 · Cited by 85 · see citing works
Intent-to-treat analysis and the problem of crossovers
Peduzzi, P. N., Detre, K. M., Wittes, J. T., & Holford, T. R. (1991) · Journal of Thoracic and Cardiovascular Surgery · doi:10.1016/s0022-5223(19)36731-5 · Cited by 47 · see citing works
Self-citationSeeking harmony: estimands and sensitivity analyses for confirmatory clinical trials
Mehrotra, D. V., Hemmings, R. J., Russek‐Cohen, E., & on behalf of the ICH E9/R1 Expert Working Group (2016) · Clinical Trials · doi:10.1177/1740774516633115 · Cited by 41 · see citing works
Geiger, M. J., Mehta, C. R., Turner, J. R., Arbet‐Engels, C., Hantel, S., Hirshberg, B., Koglin, J., Mendzelevski, B., Sager, P. T., Shapiro, D., et al. (2014) · Therapeutic Innovation & Regulatory Science · doi:10.1177/2168479014549860 · Cited by 24 · see citing works
Mak, G. J., Murphy, N. F., Ali, A. S., Walsh, A., O’Loughlin, C., Conlon, C. M., McCaffrey, D. J., Ledwidge, M. T., & McDonald, K. (2008) · Journal of Cardiac Failure · doi:10.1016/j.cardfail.2008.03.003 · Cited by 19 · see citing works
Yang, F., DeMets, D. L., Stewart, M. W., & Ye, J. (2015) · Diabetes Metabolic Syndrome and Obesity · doi:10.2147/dmso.s84005 · Cited by 8 · see citing works
Self-citationTesting Noninferiority of Response Rates for Regulatory Filings Using Transformations
Wiens, B. L., & Iglewicz, B. (2001) · Drug Information Journal · doi:10.1177/009286150103500413 · Cited by 6 · see citing works
The Multiple Risk Factor Intervention Trial
Burch, P. R. J. (1983) · JAMA · doi:10.1001/jama.1983.03330350017009 · Cited by 2 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- Feinstein A. An additional basic science for clinical medi- cine: II. Ann Intern Med 1983; 99(4): 544-550.
- Spinar J and Smahelova A. Saxagliptin and cardiovascu- lar outcomes in patients with type 2 diabetes mellitus. Vnitr Lek 2013; 59(11): 1003-1007.
Cited by
Sorted by citations
SPIRIT 2025 explanation and elaboration: updated guideline for protocols of randomised trials
Hróbjartsson, A., Boutron, I., Hopewell, S., Moher, D., Schulz, K. F., Collins, G. S., Tunn, R., Aggarwal, R., Berkwits, M., Berlin, J. A., et al. (2025) · BMJ · doi:10.1136/bmj-2024-081660 · Cited by 171 · see citing works
Citicoline for treating people with acute ischemic stroke
Martí-Carvajal, A. J., Valli, C., Martí-Amarista, C. E., Solà, I., Martí‐Fábregas, J., & Bonfill, X. (2020) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013066.pub2 · Cited by 56 · see citing works
Minutolo, R., Liberti, M. E., Simeon, V., Sasso, F. C., Borrelli, S., Nicola, L. D., & Garofalo, C. (2023) · Clinical Kidney Journal · doi:10.1093/ckj/sfad143 · Cited by 47 · see citing works
Presenting Risks and Benefits: Helping the Data Monitoring Committee Do Its Job
Evans, S. R., Bigelow, R., Chuang‐Stein, C., Ellenberg, S. S., Gallo, P. P., He, W., Jiang, Q., & Rockhold, F. W. (2019) · Annals of Internal Medicine · doi:10.7326/m19-1491 · Cited by 34 · see citing works
Lack of Statin Therapy and Outcomes After Ischemic Stroke: A Population-Based Study
Åivo, J., Ruuskanen, J. O., Tornio, A., Rautava, P. T., & Kytö, V. (2023) · Stroke · doi:10.1161/strokeaha.122.040536 · Cited by 32 · see citing works
Stegherr, R., Beyersmann, J., Jehl, V., Rufibach, K., Leverkus, F., Schmoor, C., & Friede, T. (2020) · Biometrical Journal · doi:10.1002/bimj.201900347 · Cited by 24 · see citing works
A Framework for Safety Evaluation Throughout the Product Development Life-Cycle
Ball, G., Reblin, T., Buchanan, J. R., Hendrickson, B. A., Lewis, E., Schnell, P., & Rockhold, F. W. (2019) · Therapeutic Innovation & Regulatory Science · doi:10.1007/s43441-019-00021-5 · Cited by 17 · see citing works
Fleming, T. R., Carroll, K. J., Wittes, J. T., Emerson, S. S., Rothmann, M. D., Collins, S. H., & Levin, G. (2025) · Statistics in Medicine · doi:10.1002/sim.70104 · Cited by 16 · see citing works
Self-citationThe Data and Safety Monitoring Board: The Toughest Job in Clinical Trials
Evans, S. R., Zeng, L., & Dai, W. (2023) · NEJM Evidence · doi:10.1056/evidctw2200220 · Cited by 15 · see citing works
Hypoxia-Inducible Factor Prolyl Hydroxylase Inhibitors in Kidney Disease
Ha, J. T. K., Hiremath, S. S., Jun, M., Green, S. C., Wheeler, D. C., Coyne, D. W., Perkovic, V., & Badve, S. V. (2024) · NEJM Evidence · doi:10.1056/evidoa2300189 · Cited by 14 · see citing works
Hashimoto, H., Takeuchi, M., & Kawakami, K. (2023) · Clinical Rheumatology · doi:10.1007/s10067-023-06710-9 · Cited by 14 · see citing works
‘Improving smart medication management’: an online expert discussion
Bates, D. W., Cheng, H.-Y., Cheung, N., Jew, R. K., Mir, F. A., Tamblyn, R. M., & Li, Y. (2022) · BMJ Health & Care Informatics · doi:10.1136/bmjhci-2021-100540 · Cited by 13 · see citing works
Intention-to-diagnose and distinct research foci in diagnostic accuracy studies
Evans, S. R., Pennello, G. A., Zhang, S., Li, Y., Wang, Y., Cao, Q., Komarow, L., Hamasaki, T., Petrides, V., Meier, K., et al. (2025) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(25)00070-2 · Cited by 11 · see citing works
Application of estimand framework in ICH E9 (R1) to safety evaluation
Wang, X., Yang, P., Yuan, S. S., & Wang, W. W. B. (2023) · Journal of Biopharmaceutical Statistics · doi:10.1080/10543406.2023.2189452 · Cited by 11 · see citing works
Application of estimand framework in ICH E9 (R1) to vaccine trials
Fu, R., Li, H., Wang, X., Shou, Q., & Wang, W. W. B. (2023) · Journal of Biopharmaceutical Statistics · doi:10.1080/10543406.2023.2197040 · Cited by 10 · see citing works
Heinrich, M., Zagorscak, P., Bohn, J., Knaevelsrud, C., & Schulze, L. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01936-0 · Cited by 4 · see citing works
Data Monitoring Committee Reports: Telling the Data’s Story
Zeng, L., Hamasaki, T., & Evans, S. R. (2024) · Therapeutic Innovation & Regulatory Science · doi:10.1007/s43441-024-00727-1 · Cited by 4 · see citing works
Ezetimibe use and mortality after myocardial infarction: A nationwide cohort study
Kytö, V., & Tornio, A. (2024) · American Journal of Preventive Cardiology · doi:10.1016/j.ajpc.2024.100702 · Cited by 4 · see citing works
Pannoi, T., Promchai, C., Apiromruck, P., Wongpraphairot, S., Yang, C., & Pan, W. (2022) · International Journal of Nephrology and Renovascular Disease · doi:10.2147/ijnrd.s389238 · Cited by 4 · see citing works
Debate: Are HIF Stabilizers a Viable Alternative to ESAs in the Management of Anemia in CKD? PRO
Singh, A. K. (2022) · American Journal of Nephrology · doi:10.1159/000523948 · Cited by 3 · see citing works
A safety estimand for late phase clinical trials where the analysis period varies over the subjects
Hedman, K., Lisovskaja, V., & Nyström, P. O. (2024) · Clinical Trials · doi:10.1177/17407745241230933 · Cited by 1 · see citing works
Inside the Mind of the DMC: A Review of Principles and Issues with Case Studies
Ge, L., Hamasaki, T., & Evans, S. R. (2024) · Therapeutic Innovation & Regulatory Science · doi:10.1007/s43441-024-00720-8 · Cited by 1 · see citing works
Patient-Reported Tolerability in Oncology Drug Development
Roydhouse, J. K., Basch, E., Breslin, M. C., Calvert, M., Cella, D. F., Dueck, A. C., Reeve, B. B., Smith, M. L., Thanarajasingam, G., Wagner, L. I., et al. (2024) · ICSA book series in statistics · doi:10.1007/978-3-031-67110-4_10 · Cited by 1 · see citing works
Carroll, K. J., Wittes, J. T., Snapinn, S. M., Solomon, S. D., Causland, F. R. M., Claggett, B. L., Blackorby, A., Cobitz, A. R., van Adelsberg, J., Khavandi, K., et al. (2025) · Statistics in Biopharmaceutical Research · doi:10.1080/19466315.2025.2526440
Self-citationHendrickson, B. A., McShea, C., Hammad, T. A., Meer, J., Zhang, W., Whalen, E., Talbot, S. M., Sinvhal, R., Panico, M. B., Lin, L., et al. (2025) · Pharmaceutical Medicine · doi:10.1007/s40290-025-00596-5
Leveraging the Estimand Framework for Integrated Evidence Planning
Guo, W., Hampson, L. V., lzem, R., Wang, A., Yang, N., Du, Q., Lu, C., & Siegel, J. (2025) · Springer series in pharmaceutical statistics · doi:10.1007/978-3-032-02192-2_21
Select Contemporary Statistical Concepts in Heart Failure Clinical Trials
Blumer, V. L., Wittes, J. T., Pomeroy, J. E., Fleming, T. R., Solomon, S. D., Lindenfeld, J. M., Saville, B. R., Fiuzat, M., O'Connor, C., & Psotka, M. A. (2026) · JACC Heart Failure · doi:10.1016/j.jchf.2026.103180
Self-citationLópez‐Romero, P., Crowe, B. J., He, P., Kan‐Dobrosky, N., Sashegyi, A. I., & Siegel, J. (2026) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2026-114587
Wang, M. K., Conen, D., Devereaux, P. J., Baskaran, G., Razeghi, G., Bell, C., Healey, J. S., Kamel, H., Lensing, A. W. A., McIntyre, W. F., et al. (2026) · Thrombosis Research · doi:10.1016/j.thromres.2026.109859
- Authors
- Yang, F., Wittes, J., & Pitt, B.
- Affiliations
- KBP Biosciences (Singapore) (SG)
- Statistics Collaborative (US)
- University of Michigan (US)
- Corresponding author
- Wittes, Janet
- Venue
- Clinical Trials
- Publisher
- SAGE Publications
- Published
- 2018-11-16
- Volume
- 16
- Issue
- 1
- Pages
- 63-70
- ISSN
- 1740-7745 (ISSN-L)
- 1740-7753
- OpenAlex
- W2901034360
- Cited by
- 29
- References
- 17
- Citations by year
- 2026: 3 · 2025: 7 · 2024: 6 · 2023: 6 · 2022: 3 · 2020: 2 · 2019: 2
- FWCI
- 0.83
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Clinical Trials (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Life Sciences › Pharmacology, Toxicology and Pharmaceutics › Toxicology › Pharmacovigilance and Adverse Drug Reactions
- Keywords
- intention-to-treat analysis, drug safety assessment, randomization, risk-benefit assessment, rofecoxib, saxagliptin, treatment discontinuation
- MeSH terms
- Data Interpretation, Statistical, Humans, Research Design, Observer Variation, Risk Assessment, Intention to Treat Analysis, Drug-Related Side Effects and Adverse Reactions
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
