CoChat

Evaluating the Evidence: Statistical Methods in Randomized Controlled Trials in the Urological Literature

Scales, Charles D., Norris, Regina D., Preminger, Glenn M., Vieweg, Johannes, Peterson, Bercedis L., Dahm, Philipp · 2008 · Journal of Urology
Cited by 32

The DOI matches its Crossref record; 7 of the 7 references it cites were found in a registry or an index.

Abstract

PURPOSE: Randomized controlled trials potentially provide the highest level of evidence to inform clinical decision making. Appropriate use of statistical methods is a critical aspect of all clinical research, including randomized controlled trials. We report the first formal evaluation to our knowledge of the statistical methods of randomized controlled trials published in the urological literature in 1996 and 2004. MATERIALS AND METHODS: All human subjects randomized controlled trials published in 4 leading urology journals in 1996 and 2004 were identified for formal review. A standardized evaluation form was developed based on the Consolidated Standards of Reporting Trials statement. Each article was evaluated by 2 independent reviewers with formal training in research design and biostatistics who were blinded to study authors and institution. Discrepancies were settled by consensus. RESULTS: A total of 152 randomized controlled trials were reviewed (65 in 1996, 87 in 2004). The median sample size (IQR) per arm of parallel design randomized controlled trials published in 1996 and 2004 was 36 (11, 96) and 50 (26, 134) study subjects, respectively (p = 0.157). Sample size justifications were provided by 19% of studies in 1996 and 47% of studies in 2004 (p = 0.001). Of randomized controlled trials 16 (25%) vs 32 (37%) identified a single primary outcome variable (p = 0.110). Effect size estimates for primary or secondary outcome variables were provided by 5% vs 13% (p = 0.090) and the precision of the effect was detailed by 5% vs 10% of randomized controlled trials (p = 0.195). CONCLUSIONS: This formal review suggests that statistical analysis in urological randomized controlled trials has improved. However, considerable deficiencies remain. Ongoing education in applied statistics may further improve urological randomized controlled trial reporting.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.juro.2008.06.026

Cites

7 of 7 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 13

  1. Sackett, 1996, Evidence based medicine: what it is and what it isn't, BMJ, 312, 71, 10.1136/bmj.312.7023.71 · DOI · Paper page
  2. Scales, 2007, Evidence based clinical practice: a primer for urologists, J Urol, 178, 775, 10.1016/j.juro.2007.05.020 · DOI
  3. Begg, 1996, Improving the quality of reporting of randomized controlled trials, JAMA, 276, 637, 10.1001/jama.276.8.637 · doi:10.1001/jama.276.8.637 · Paper page
  4. Moher, 2001, The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials, Lancet, 357, 1191, 10.1016/S0140-6736(00)04337-3 · DOI · Paper page
  5. Schulz, 1995, Empirical evidence of bias, JAMA, 273, 408, 10.1001/jama.273.5.408 · doi:10.1001/jama.273.5.408
  6. Hutton, 2000, Bias in meta-analysis due to outcome variable selection within studies, Appl Stat, 49, 359
  7. Ruiz-Canela, 2000, Intention to treat analysis is related to methodological quality, BMJ, 320, 1007, 10.1136/bmj.320.7240.1007 · DOI · Paper page
  8. Hall, 1996, Methodologic standards in surgical trials, Surgery, 119, 466, 10.1016/S0039-6060(96)80149-8 · DOI
  9. Assmann, 2000, Subgroup analysis and other (mis)uses of baseline data in clinical trials, Lancet, 355, 1064, 10.1016/S0140-6736(00)02039-0 · DOI · Paper page
  10. Breau, 2006, Inadequate statistical power of negative clinical trials in urological literature, J Urol, 176, 263, 10.1016/S0022-5347(06)00505-2 · DOI · Paper page
  11. Welk, 2006, Randomized controlled trials in pediatric urology: room for improvement, J Urol, 176, 306, 10.1016/S0022-5347(06)00560-X · DOI
  12. Scales, 2007, A critical assessment of the quality of reporting of randomized, controlled trials in the urology literature, J Urol, 177, 1090, 10.1016/j.juro.2006.10.027 · DOI
  13. Altman, 2001, The revised CONSORT statement for reporting randomized trials: explanation and elaboration, Ann Intern Med, 134, 663, 10.7326/0003-4819-134-8-200104170-00012 · DOI · Paper page

Cited by

See all 32

Metadata

Authors
Scales, C. D., Norris, R. D., Preminger, G. M., Vieweg, J., Peterson, B. L., & Dahm, P.
Venue
Journal of Urology
Published
2008-10
Volume
180
Issue
4
Pages
1463-1467
ISSN
  • 0022-5347 (ISSN-L)
  • 1527-3792
OpenAlex
W1977319231
Cited by
32
References
7
FWCI
4.50
Type
journal article
Language
en
Open access
closed
Licence
www.elsevier.com/tdm/userlicense/1.0/ (tdm)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.