Trends in study design and the statistical methods employed in a leading general medicine journal
Abstract
WHAT IS KNOWN AND OBJECTIVE: Study design and statistical methods have become core components of medical research, and the methodology has become more multifaceted and complicated over time. The study of the comprehensive details and current trends of study design and statistical methods is required to support the future implementation of well-planned clinical studies providing information about evidence-based medicine. Our purpose was to illustrate study design and statistical methods employed in recent medical literature. METHODS: This was an extension study of Sato et al. (N Engl J Med 2017; 376: 1086-1087), which reviewed 238 articles published in 2015 in the New England Journal of Medicine (NEJM) and briefly summarized the statistical methods employed in NEJM. Using the same database, we performed a new investigation of the detailed trends in study design and individual statistical methods that were not reported in the Sato study. RESULTS AND DISCUSSION: Due to the CONSORT statement, prespecification and justification of sample size are obligatory in planning intervention studies. Although standard survival methods (eg Kaplan-Meier estimator and Cox regression model) were most frequently applied, the Gray test and Fine-Gray proportional hazard model for considering competing risks were sometimes used for a more valid statistical inference. With respect to handling missing data, model-based methods, which are valid for missing-at-random data, were more frequently used than single imputation methods. These methods are not recommended as a primary analysis, but they have been applied in many clinical trials. Group sequential design with interim analyses was one of the standard designs, and novel design, such as adaptive dose selection and sample size re-estimation, was sometimes employed in NEJM. WHAT IS NEW AND CONCLUSION: Model-based approaches for handling missing data should replace single imputation methods for primary analysis in the light of the information found in some publications. Use of adaptive design with interim analyses is increasing after the presentation of the FDA guidance for adaptive design.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In RCTs, many statistical analyses are based on logistic and Cox regression models, which use the multiplicative scale.
“In RCTs, many statistical analyses are based on logistic and Cox regression models (ie binary outcomes are often analysed) which utilize the multiplicative scale.11”
From Understanding of interaction (subgroup) analysis in clinical trials (2019) · cites this paper as [11]
Cites
A Proportional Hazards Model for the Subdistribution of a Competing Risk
Fine, J. P., & Gray, R. J. (1999) · Journal of the American Statistical Association · doi:10.1080/01621459.1999.10474144 · Cited by 14,290 · cited 2× · see citing works
“5099 14 (6) 12 (7) 2 (3) 100299 32 (13) 27 (16) 5 (7) 300999 63 (26) 56 (33) 7 (10)”
The ASA Statement on <i>p</i> -Values: Context, Process, and Purpose
Wasserstein, R. L., & Lazar, N. A. (2016) · The American Statistician · doi:10.1080/00031305.2016.1154108 · Cited by 6,306 · see citing works
A Class of $K$-Sample Tests for Comparing the Cumulative Incidence of a Competing Risk
Gray, R. J. (1988) · The Annals of Statistics · doi:10.1214/aos/1176350951 · Cited by 4,957 · see citing works
CONSORT 2010 statement: updated guidelines for reporting parallel group randomized trials
Schulz, K. F., Altman, D. G., Moher, D., & for the CONSORT Group* (2010) · Annals of Internal Medicine · doi:10.7326/0003-4819-152-11-201006010-00232 · Cited by 3,824 · see citing works
A Multiple Testing Procedure for Clinical Trials
O'Brien, P. C., & Fleming, T. R. (1979) · Biometrics · doi:10.2307/2530245 · Cited by 3,311 · see citing works
Discrete sequential boundaries for clinical trials
GORDON LAN, K. K., & DEMETS, D. L. (1983) · Biometrika · doi:10.1093/biomet/70.3.659 · Cited by 2,310 · see citing works
Discrete Sequential Boundaries for Clinical Trials
Lan, K. K. G., & DeMets, D. L. (1983) · Biometrika · doi:10.2307/2336502 · Cited by 2,310 · see citing works
The prevention and treatment of missing data in clinical trials
Little, R. J., D'Agostino, R., Cohen, M. L., Dickersin, K., Emerson, S. S., Farrar, J. T., Frangakis, C., Hogan, J. W., Molenberghs, G., Murphy, S. A., et al. (2012) · New England Journal of Medicine · doi:10.1056/nejmsr1203730 · Cited by 1,884 · see citing works
Horton, N., & Kleinman, K. (2007) · American Statistician · doi:10.1198/000313007x172556 · Cited by 476 · see citing works
The Prevention and Treatment of Missing Data in Clinical Trials
National Research Council (2010) · National Academies Press eBooks · doi:10.17226/12955 · Cited by 439 · see citing works
Guidelines for Statistical Reporting in Articles for Medical Journals
Bailar, J. C., & MOSTELLER, F. (1988) · Annals of Internal Medicine · doi:10.7326/0003-4819-108-2-266 · Cited by 387 · see citing works
Long-Term Efficacy of a Hepatitis E Vaccine
Zhang, J., Zhang, X.-F., Huang, S., et al. (2015) · New England Journal of Medicine · doi:10.1056/nejmoa1406011 · Cited by 384 · see citing works
Evolution of Reporting <i>P</i> Values in the Biomedical Literature, 1990-2015
Chavalarias, D., Wallach, J. D., Li, A. H. T., & Ioannidis, J. P. A. (2016) · JAMA · doi:10.1001/jama.2016.1952 · Cited by 370 · see citing works
Handbook of Missing Data Methodology
Molenberghs, G., Fitzmaurice, G. M., Kenward, M. G., Tsiatis, A. A., & Verbeke, G. N. (2014) · Chapman & Hall/CRC eBooks · doi:10.1201/b17622 · Cited by 354 · see citing works
Use of Statistical Analysis inThe New England Journal of Medicine
Emerson, J. D., & Colditz, G. A. (1983) · New England Journal of Medicine · doi:10.1056/nejm198309223091206 · Cited by 226 · see citing works
Burton, A., & Altman, D. G. (2004) · British Journal of Cancer · doi:10.1038/sj.bjc.6601907 · Cited by 219 · see citing works
Brannath, W., Burger, H. U., Glimm, E., Stallard, N., Vandemeulebroecke, M., & Wassmer, G. (2010) · Journal of Biopharmaceutical Statistics · doi:10.1080/10543406.2010.514453 · Cited by 193 · see citing works
Statistical Methods in theJournal
Horton, N. J., & Switzer, S. S. (2005) · New England Journal of Medicine · doi:10.1056/nejm200511033531823 · Cited by 141 · see citing works
Review of the Use of Statistics in Infection andImmunity
Olsen, C. H. (2003) · Infection and Immunity · doi:10.1128/iai.71.12.6689-6692.2003 · Cited by 126 · see citing works
Statistical Methods in the Surgical Literature
Kurichi, J. E., & Sonnad, S. S. (2006) · Journal of the American College of Surgeons · doi:10.1016/j.jamcollsurg.2005.11.018 · Cited by 66 · see citing works
Statistical Methods in the Journal — An Update
Sato, Y., Gosho, M., Nagashima, K., et al. (2017) · New England Journal of Medicine · doi:10.1056/nejmc1616211 · Cited by 37 · see citing works
Self-citationGOLDIN, J. G., Zhu, W., & Sayre, J. W. (1996) · Clinical Radiology · doi:10.1016/s0009-9260(96)80219-4 · Cited by 36 · see citing works
Statistical methods in rehabilitation literature: A survey of recent publications
Schwartz, S. J., Sturr, M., & Goldberg, G. (1996) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/s0003-9993(96)90040-4 · Cited by 28 · see citing works
Optimization of Multistage Testing Times and Critical Values in Clinical Trials
Brittain, E., & Bailey, K. R. (1993) · Biometrics · doi:10.2307/2532197 · Cited by 22 · see citing works
Biostatistical Trends in Pediatrics: Implications for the Future
Hayden, G. F. (1983) · PEDIATRICS · doi:10.1542/peds.72.1.84 · Cited by 21 · see citing works
A survey of statistics in three UK general practice journal
Rigby, A. S., Armstrong, G., Campbell, M. J., & Summerton, N. (2004) · BMC Medical Research Methodology · doi:10.1186/1471-2288-4-28 · Cited by 17 · see citing works
Hartung, J. G. (2006) · Biometrical Journal · doi:10.1002/bimj.200510212 · Cited by 16 · see citing works
Rao, M. H., & Khan, N. B. (2010) · PubMed · Cited by 6 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Understanding of interaction (subgroup) analysis in clinical trials
Brankovic, M., Kardys, I., Steyerberg, E. W., Lemeshow, S., Markovic, M., Rizopoulos, D., & Boersma, E. (2019) · European Journal of Clinical Investigation · doi:10.1111/eci.13145 · Cited by 130 · see citing works
Dimairo, M., Pallmann, P., Wason, J., Todd, S., Jaki, T., Julious, S. A., Mander, A. P., Weir, C. J., Koenig, F., Walton, M. K., et al. (2020) · Trials · doi:10.1186/s13063-020-04334-x · Cited by 58 · see citing works
- Authors
- Gosho, M., Sato, Y., Nagashima, K., & Takahashi, S.
- Affiliations
- University of Tsukuba (JP)
- Chiba University (JP)
- Chiba University Hospital (JP)
- Corresponding author
- Gosho, M.
- Venue
- Journal of Clinical Pharmacy and Therapeutics
- Publisher
- Wiley
- Published
- 2017-07-27
- Volume
- 43
- Issue
- 1
- Pages
- 36-44
- ISSN
- 0269-4727 (ISSN-L)
- OpenAlex
- W2742162420
- Cited by
- 26
- References
- 28
- Citations by year
- 2025: 3 · 2023: 1 · 2022: 2 · 2021: 1 · 2020: 13 · 2019: 3 · 2018: 2 · 2017: 1
- FWCI
- 3.61
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- Journal of Clinical Pharmacy and Therapeutics (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- statistical methods in clinical trials, New England Journal of Medicine, CONSORT, sample size justification, competing risks, missing data handling, model-based approach, single imputation, group sequential designs, interim analysis, sample size re-estimation, adaptive trial design, Kaplan-Meier estimator, Cox proportional hazards model
- MeSH terms
- Data Interpretation, Statistical, Humans, Publications, Research Design, Models, Statistical, Evidence-Based Medicine, Biomedical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review