CoChat

The Use of Statistics in Medical Research: A Comparison of The New England Journal of Medicine and Nature Medicine

Strasak, Alexander M, Zaman, Qamruz, Marinell, Gerhard, Pfeiffer, Karl P, Ulmer, Hanno · 2007 · The American Statistician
Cited by 103

The DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.

Abstract

There is widespread evidence of the extensive use of statistical methods in medical research. Just the same, standards are generally low and a growing body of literature points to statistical errors in most medical journals. However, there is no comprehensive study contrasting the top medical journals of basic and clinical science for recent practice in their use of statistics.All original research articles in Volume 10, Numbers 1–6 of Nature Medicine (Nat Med) and Volume 350, Numbers 1–26 of The New England Journal of Medicine (NEJM) were screened for their statistical content. Types, frequencies, and complexity of applied statistical methods were systematically recorded. A 46-item checklist was used to evaluate statistical quality for a subgroup of papers.94.5 percent (95% CI 87.6–98.2) of NEJM articles and 82.4 percent (95% CI 65.5–93.2) of Nat Med articles contained inferential statistics. NEJM papers were significantly more likely to use advanced statistical methods (p < 0.0001). Statistical errors were identified in a considerable proportion of articles, although not always serious in nature. Documentation of applied statistical methods was generally poor and insufficient, particularly in Nat Med.Compared to 1983, a vast increase in usage and complexity of statistical methods could be observed for NEJM papers. This does not necessarily hold true for Nat Med papers, as the results of the study indicate that basic science sticks with basic analysis. As statistical errors seem to remain common in medical literature, closer attention to statistical methodology should be seriously considered to raise standards.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1198/000313007x170242

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 2 references

Cited by

See all 103

Metadata

Authors
Strasak, A. M., Zaman, Q., Marinell, G., Pfeiffer, K. P., & Ulmer, H.
Venue
The American Statistician
Published
2007-02
Volume
61
Issue
1
Pages
47-55
ISSN
  • 0003-1305 (ISSN-L)
  • 1537-2731
OpenAlex
W1575656708
Cited by
103
References
2
FWCI
2.70
Type
journal article
Language
en
Open access
closed

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: The Use of Statistics in Medical Research: A Comparison of The New England Journal of Medicine and Nature Medicine. https://app.cochat.ai/papers/10.1198/000313007x170242
The paper
Strasak, A. M., Zaman, Q., Marinell, G., Pfeiffer, K. P., & Ulmer, H. (2007). The Use of Statistics in Medical Research: A Comparison of The New England Journal of Medicine and Nature Medicine. The American Statistician. https://doi.org/10.1198/000313007x170242

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.