Measurement in clinical trials: A neglected issue for statisticians?
Abstract
Biostatisticians have frequently uncritically accepted the measurements provided by their medical colleagues engaged in clinical research. Such measures often involve considerable loss of information. Particularly, unfortunate is the widespread use of the so-called 'responder analysis', which may involve not only a loss of information through dichotomization, but also extravagant and unjustified causal inference regarding individual treatment effects at the patient level, and, increasingly, the use of the so-called number needed to treat scale of measurement. Other problems involve inefficient use of baseline measurements, the use of covariates measured after the start of treatment, the interpretation of titrations and composite response measures. Many of these bad practices are becoming enshrined in the regulatory guidance to the pharmaceutical industry. We consider the losses involved in inappropriate measures and suggest that statisticians should pay more attention to this aspect of their work.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Current tools for the homogeneity assumption might not help with treatment-effect heterogeneity in randomized clinical trials, where several factors can drive variation in treatment response.
“For example, our current tools to tackle the homogeneity assumption may not help address the treatment-effect het- erogeneity problem (67–69) in research clinical trials (RCTs), where multiple factors may drive treatment-response variation.”
From Methods and challenges for assessing heterogeneity (2020) · cites this paper as [68]
Cites
Regression Modeling Strategies
Harrell, F. E. (2001) · Springer series in statistics · doi:10.1007/978-1-4757-3462-1 · Cited by 9,270 · see citing works
Harrell (2015) · Springer Series in Statistics · doi:10.1007/978-3-319-19425-7 · Cited by 6,878 · see citing works
Sacks, F. M., Svetkey, L. P., Vollmer, W. M., Appel, L. J., Bray, G. A., Harsha, D., Obarzanek, E., Conlin, P. R., Miller, E. R., Simons-Morton, D. G., et al. (2001) · New England Journal of Medicine · doi:10.1056/nejm200101043440101 · Cited by 5,638 · see citing works
Regression Models for Ordinal Data
McCullagh, P. (1980) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1980.tb01109.x · Cited by 4,453 · see citing works
Bombardier, C., Laine, L., Reicin, A. S., Shapiro, D., Burgos‐Vargas, R., Davis, B. R., Day, R. O., Ferraz, M. B., Hawkey, C. J., Hochberg, M. C., et al. (2000) · New England Journal of Medicine · doi:10.1056/nejm200011233432103 · Cited by 4,014 · see citing works
Effect of carvedilol on survival in severe chronic heart failure
Packer, M., Coats, A. J. S., Fowler, M. B., Katus, H. A., Krum, H., Mohacsi, P., Rouleau, J. L., Tendera, M., Castaigne, A., Roecker, E. B., et al. (2001) · New England Journal of Medicine · doi:10.1056/nejm200105313442201 · Cited by 3,093 · see citing works
Dichotomizing continuous predictors in multiple regression: a bad idea
Royston, P., Altman, D. G., & Sauerbrei, W. (2006) · Statistics in Medicine · doi:10.1002/sim.2331 · Cited by 2,312 · see citing works
The number needed to treat: a clinically useful measure of treatment effect
Cook, R. J., & Sackett, D. L. (1995) · BMJ · doi:10.1136/bmj.310.6977.452 · Cited by 1,885 · see citing works
Regression Using Fractional Polynomials of Continuous Covariates: Parsimonious Parametric Modelling
Royston, P. J., & Altman, D. G. (1994) · Journal of the Royal Statistical Society Series C (Applied Statistics) · doi:10.2307/2986270 · Cited by 1,843 · see citing works
An Assessment of Clinically Useful Measures of the Consequences of Treatment
Laupacis, A., Sackett, D. L., & Roberts, R. S. (1988) · New England Journal of Medicine · doi:10.1056/nejm198806303182605 · Cited by 1,681 · see citing works
Effect of carvedilol on survival in severe chronic heart failure
Packer, M., Coats, A. J. S., & Fowler, M. B. (2001) · ACC Current Journal Review · doi:10.1016/s1062-1458(01)00477-9 · Cited by 1,651 · see citing works
The two‐period cross‐over clinical trial.
Hills, M., & Armitage, P. (1979) · British Journal of Clinical Pharmacology · doi:10.1111/j.1365-2125.1979.tb05903.x · Cited by 1,403 · see citing works
Dangers of Using "Optimal" Cutpoints in the Evaluation of Prognostic Factors
Altman, D. G., Lausen, B., Sauerbrei, W., & Schumacher, M. (1994) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/86.11.829 · Cited by 1,143 · see citing works
Die Systolendauer im Elektrokardiogramm bei normalen Menschen und bei Herzkranken
Fridericia, L. S. (1920) · Acta Medica Scandinavica · doi:10.1111/j.0954-6820.1920.tb18266.x · Cited by 1,011 · see citing works
de Haes, J., van Knippenberg, F. C. E., & Neijt, J. P. (1990) · British Journal of Cancer · doi:10.1038/bjc.1990.434 · Cited by 907 · see citing works
The hospital anxiety and depression scale
Snaith, R. P., & Zigmond, A. S. (1986) · BMJ · doi:10.1136/bmj.292.6516.344 · Cited by 879 · see citing works
AN ANALYSIS OF THE TIME‐RELATIONS OF ELECTROCARDIOGRAMS.
Bazett, H. C. (1997) · Annals of Noninvasive Electrocardiology · doi:10.1111/j.1542-474x.1997.tb00325.x · Cited by 817 · see citing works
Sample sizes for clinical trials with normal data.
Julious, S. A. (2004) · Statistics in Medicine · doi:10.1002/sim.1783 · Cited by 605 · see citing works
Self-citationSpurious Correlation and the Fallacy of the Ratio Standard Revisited
Kronmal, R. A. (1993) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.2307/2983064 · Cited by 593 · see citing works
Frison, L., & Pocock, S. J. (1992) · Statistics in Medicine · doi:10.1002/sim.4780111304 · Cited by 593 · see citing works
Senn, S. J., Frison, L., & Pocock, S. (1994) · Statistics in Medicine · doi:10.1002/sim.4780130210 · Cited by 529 · see citing works
Self-citationInterpreting treatment effects in randomised trials
Guyatt, G. H., Juniper, E. F., Walter, S. D., Griffith, L. E., & Goldstein, R. S. (1998) · BMJ · doi:10.1136/bmj.316.7132.690 · Cited by 516 · see citing works
Campbell, M. J., Julious, S. A., & Altman, D. G. (1995) · BMJ · doi:10.1136/bmj.311.7013.1145 · Cited by 476 · see citing works
Self-citationAn evidence based approach to individualising treatment
Glasziou, P., & Irwig, L. (1995) · BMJ · doi:10.1136/bmj.311.7016.1356 · Cited by 389 · see citing works
Sager, P. T., Nebout, T., & Darpö, B. (2005) · Drug Information Journal · doi:10.1177/009286150503900407 · Cited by 385 · see citing works
Statistical Issues in Drug Development
Senn, S. J. (2021) · Wiley eBooks · doi:10.1002/9781119238614 · Cited by 384 · see citing works
Self-citationQuality-of-Life Assessment: Can We Keep It Simple?
Cox, D. R., Fitzpatrick, R. M., Fletcher, A., Gore, S. M., Spiegelhalter, D. J., & Jones, D. R. H. (1992) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.2307/2982889 · Cited by 348 · see citing works
A Method for Comparing Profiles of Repeated Measurements
Kenward, M. G. (1987) · Journal of the Royal Statistical Society Series C (Applied Statistics) · doi:10.2307/2347788 · Cited by 303 · see citing works
Numbers needed to treat derived from meta-analyses---sometimes informative, usually misleading
Smeeth, L. L., Haines, A., & Ebrahim, S. B. J. (1999) · BMJ · doi:10.1136/bmj.318.7197.1548 · Cited by 301 · see citing works
Sample size calculations for ordered categorical data
Whitehead, J. (1993) · Statistics in Medicine · doi:10.1002/sim.4780122404 · Cited by 297 · see citing works
Combining different phases in the development of medical treatments within a single trial
Bauer, P., & Kieser, M. (1999) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19990730)18:14 · Cited by 289 · see citing works
Statistics and the Theory of Measurement
Hand, D. J. (1996) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.2307/2983326 · Cited by 264 · see citing works
Ombardier, C. L. B., Aine, L. O. L., Eicin, A. L. R., Hapiro, D. E. S., Urgos, R. U. B., Avis, B. A. D., Ay, R., Arcos, M. J., Erraz, B. O. F., Awkey, C. H. J. H., et al. (2010) · Cited by 252 · see citing works
Statistical Issues in Drug Development
Ziegel, E. R., & Senn, S. J. (1999) · Technometrics · doi:10.2307/1271380 · Cited by 224 · see citing works
Self-citationIndividual response to treatment: is it a valid assumption?
Senn, S. J. (2004) · BMJ · doi:10.1136/bmj.329.7472.966 · Cited by 177 · see citing works
Self-citationFurther Comparative Analyses of Pretest-Posttest Research Designs
Laird, N. M. (1983) · The American Statistician · doi:10.1080/00031305.1983.10483133 · Cited by 133 · see citing works
WHY ARE PHARMACOKINETIC DATA SUMMARIZED BY ARITHMETIC MEANS?
Julious, S. A., & Debarnot, C. A. M. (2000) · Journal of Biopharmaceutical Statistics · doi:10.1081/bip-100101013 · Cited by 113 · see citing works
Self-citationNumber Needed to Treat: Properties and Problems
Hutton, J. L. (2000) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/1467-985x.00175 · Cited by 112 · see citing works
Differences in proximal femur bone density over two centuries
Lees, B., Stevenson, J. C., Molleson, T., et al. (1993) · The Lancet · doi:10.1016/0140-6736(93)90433-h · Cited by 110 · see citing works
Some Classification Problems with Multivariate Qualitative Data
Cochran, W. G., & Hopkins, C. E. (1961) · Biometrics · doi:10.2307/2527493 · Cited by 107 · see citing works
Individual Therapy: New Dawn or False Dawn?
Senn, S. J. (2001) · Drug Information Journal · doi:10.1177/009286150103500443 · Cited by 100 · see citing works
Self-citationInference and Estimation in a Changepoint Regression Problem
Julious, S. A. (2001) · Journal of the Royal Statistical Society Series D (The Statistician) · doi:10.1111/1467-9884.00260 · Cited by 96 · see citing works
Self-citationMalik, M. (2002) · Pacing and Clinical Electrophysiology · doi:10.1046/j.1460-9592.2002.00209.x · Cited by 93 · see citing works
Statistical Issues in Drug Development
(1999) · Technometrics · doi:10.1080/00401706.1999.10485963 · Cited by 77 · see citing works
Further Comparative Analyses of Pretest-Posttest Research Designs
Laird, N. M. (1983) · The American Statistician · doi:10.2307/2682785 · Cited by 57 · see citing works
Tutorial in biostatistics - Sample sizes for clinical trials with Normal data
Julious, S. A. (2004) · White Rose Research Online (University of Leeds, The University of Sheffield, University of York) · Cited by 55 · see citing works
Self-citationThe use of baselines in clinical trials of bronchodilators
Senn, S. J. (1989) · Statistics in Medicine · doi:10.1002/sim.4780081106 · Cited by 52 · see citing works
Self-citationModeling the effects of continuous risk factors
Royston, P. J. (2000) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(99)00163-8 · Cited by 42 · see citing works
Odds ratios for a continuous outcome variable without dichotomizing
Moser, B. K., & Coombs, L. P. (2004) · Statistics in Medicine · doi:10.1002/sim.1776 · Cited by 37 · see citing works
Lewis, J. A. (2004) · Pharmaceutical Statistics · doi:10.1002/pst.107 · Cited by 37 · see citing works
Applying results of randomised trials to patients: Authors' reply
Senn, S. J. (1998) · BMJ · doi:10.1136/bmj.317.7157.537 · Cited by 35 · see citing works
Self-citationSample sizes for studies using the short form 36 (SF-36)
Julious, S. A., George, S., & Campbell, M. J. (1995) · Journal of Epidemiology & Community Health · doi:10.1136/jech.49.6.642 · Cited by 35 · see citing works
Self-citationThe number needed to treat: a useful clinical measure or a case of the Emperor's new clothes?
Grieve, A. P. (2003) · Pharmaceutical Statistics · doi:10.1002/pst.33 · Cited by 33 · see citing works
On Sequential Treatment Allocations in Clinical Trials
Burman, C. (1996) · Chalmers Publication Library (Chalmers University of Technology) · Cited by 30 · see citing works
Kieser, M., Röhmel, J., & Friede, T. (2004) · Statistics in Medicine · doi:10.1002/sim.1910 · Cited by 28 · see citing works
Issues with number needed to treat
Julious, S. A. (2005) · Statistics in Medicine · doi:10.1002/sim.2150 · Cited by 27 · see citing works
Self-citationSample sizes for randomized trials measuring quality of life in cancer patients
Julious, S. A., George, S., Machin, D., & Stephens, R. J. (1997) · Quality of Life Research · doi:10.1023/a:1026481815304 · Cited by 25 · see citing works
Self-citationAuthor’s Reply to Walter and Guyatt
Senn, S. J. (2003) · Drug Information Journal · doi:10.1177/009286150303700103 · Cited by 18 · see citing works
Self-citationThe marriage of clinical trials and clinical decision science
Hilden, J., & Habbema, J. D. F. (1990) · Statistics in Medicine · doi:10.1002/sim.4780091105 · Cited by 18 · see citing works
Dose equivalence and bronchoprotective effects of salmeterol and salbutamol in asthma
Higham, M., Sharara, A. M., Wilson, P. M. A., et al. (1997) · Thorax · doi:10.1136/thx.52.11.975 · Cited by 15 · see citing works
Statistical Issues in Short-Term Trials in Asthma
Senn, S. J. (1993) · Drug Information Journal · doi:10.1177/009286159302700321 · Cited by 13 · see citing works
Self-citationThe ABC of pharmaceutical trial design: some basic principles
Julious, S. A., & Zariffa, N. M.-D. (2002) · Pharmaceutical Statistics · doi:10.1002/pst.6 · Cited by 12 · see citing works
Self-citationDouble-Blind Studies of Calcium Antagonists in the Treatment of Hypertension in Japan
Kuramoto, K. (1989) · Journal of Cardiovascular Pharmacology · doi:10.1097/00005344-198900131-00007 · Cited by 12 · see citing works
Some Fundamental Statistical Concepts in Clinical Trials and their Application in Herpes Zoster
Kay, R. (1995) · Antiviral chemistry & chemotherapy · doi:10.1177/09563202950060s607 · Cited by 7 · see citing works
CPMP note for guidance: clinical investigation of anti-anginal drugs in stable angina pectoris.
Mato, J. M., & Ortiz, P. (1996) · PubMed · Cited by 6 · see citing works
Harmonizing multiple choice question marks with essay marks
Ashby, D., & Baron, D. N. (1986) · Medical Education · doi:10.1111/j.1365-2923.1986.tb01373.x · Cited by 3 · see citing works
Shah, R. (1996) · PubMed · Cited by 3 · see citing works
Harrell, F., Jr. (1996) · doi:10.1002/(sici)1097-0258(19960229)15:4
Flagged first, then by citations
Cited by
Sorted by citations
White, P., Goldsmith, K., Johnson, A., Potts, L., Walwyn, R., DeCesare, J., Baber, H., Burgess, M., Clark, L., Cox, D., et al. (2011) · The Lancet · doi:10.1016/s0140-6736(11)60096-2 · Cited by 897 · see citing works
Mastering variation: Variance components and personalised medicine
Senn, S. (2016) · Statistics in Medicine · doi:10.1002/sim.6739 · Cited by 205 · see citing works
Self-citationMethods and challenges for assessing heterogeneity
Feczko, E., & Fair, D. A. (2020) · Biological Psychiatry · doi:10.1016/j.biopsych.2020.02.015 · Cited by 96 · see citing works
Tidwell, J. W., Dougherty, M. R., Chrabaszcz, J. R., Thomas, R. P., & Mendoza, J. L. (2014) · Psychonomic Bulletin & Review · doi:10.3758/s13423-013-0560-7 · Cited by 45 · see citing works
- Authors
- Senn, S., & Julious, S.
- Affiliations
- University of Glasgow (GB)
- University of Sheffield (GB)
- Corresponding author
- Senn, Stephen
- Venue
- Statistics in Medicine
- Publisher
- Wiley
- Published
- 2009-05-19
- Volume
- 28
- Issue
- 26
- Pages
- 3189-3209
- ISSN
- 0277-6715 (ISSN-L)
- 1097-0258
- OpenAlex
- W1971440802
- Cited by
- 136
- References
- 69
- Citations by year
- 2026: 4 · 2025: 6 · 2024: 3 · 2023: 5 · 2022: 7 · 2021: 15 · 2020: 8 · 2019: 8 · 2018: 11 · 2017: 3 · 2016: 7 · 2015: 5 · 2014: 7 · 2013: 16 · 2012: 10
- FWCI
- 8.32
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- Statistics in Medicine (publishedVersion)
- PubMed (publishedVersion)
- White Rose Research Online (University of Leeds, The University of Sheffield, University of York) (submittedVersion)
- Funders
- Engineering and Physical Research Council
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Keywords
- responder analysis, dichotomization, information loss, number needed to treat, baseline measurement, titration, causal inference
- MeSH terms
- Clinical Trials as Topic, Data Interpretation, Statistical, Electrocardiography, Humans, Randomized Controlled Trials as Topic, Treatment Outcome, Outcome Assessment, Health Care, Cross-Over Studies, Biostatistics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
