The DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
t-tests, non-parametric tests, and large studies—a paradox of statistical practice?
Open accessAbstract
BACKGROUND: During the last 30 years, the median sample size of research studies published in high-impact medical journals has increased manyfold, while the use of non-parametric tests has increased at the expense of t-tests. This paper explores this paradoxical practice and illustrates its consequences. METHODS: A simulation study is used to compare the rejection rates of the Wilcoxon-Mann-Whitney (WMW) test and the two-sample t-test for increasing sample size. Samples are drawn from skewed distributions with equal means and medians but with a small difference in spread. A hypothetical case study is used for illustration and motivation. RESULTS: The WMW test produces, on average, smaller p-values than the t-test. This discrepancy increases with increasing sample size, skewness, and difference in spread. For heavily skewed data, the proportion of p<0.05 with the WMW test can be greater than 90% if the standard deviations differ by 10% and the number of observations is 1000 in each group. The high rejection rates of the WMW test should be interpreted as the power to detect that the probability that a random sample from one of the distributions is less than a random sample from the other distribution is greater than 50%. CONCLUSIONS: Non-parametric tests are most useful for small studies. Using non-parametric tests in large studies may provide answers to the wrong question, thus confusing readers. For studies with a large sample size, t-tests and their corresponding confidence intervals can and should be used even for heavily skewed data.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Practical Statistics for Medical Research.
Walter, S. D., & Altman, D. G. (1992) · Biometrics · doi:10.2307/2532320 · Cited by 11,668
Practical Statistics for Medical Research
Altman, D. G. (1990) · doi:10.1201/9780429258589 · Cited by 10,668
Practical Statistics for Medical Research.
Bartko, J. J., & Altman, D. G. (1992) · Journal of the American Statistical Association · doi:10.2307/2290246 · Cited by 4,797
Kirkwood, B. R., & Sterne, J. A. C. (2014) · Bristol Research (University of Bristol) · Cited by 2,820
An Introduction to Medical Statistics.
(1931) · Journal of the American Medical Association · doi:10.1001/jama.1931.02730150055037 · Cited by 2,084
Smithson, M. (2000) · SAGE Publications Ltd eBooks · doi:10.4135/9781446218525 · Cited by 1,871
Integration of Antiretroviral Therapy with Tuberculosis Treatment
Abdool Karim, S. S., Naidoo, K., Grobler, A., Padayatchi, N., Baxter, C., Gray, A. L., Gengiah, T., Gengiah, S., Naidoo, A., Jithoo, N., et al. (2011) · New England Journal of Medicine · doi:10.1056/nejmoa1014181 · Cited by 487
Mann-Whitney test is not just a test of medians: differences in spread can be important
Hart, A. (2001) · BMJ · doi:10.1136/bmj.323.7309.391 · Cited by 433
Parodi, G., Marcucci, R., Valenti, R., Gori, A. M., Migliorini, A., Giusti, B., Buonamici, P., Gensini, G. F., Abbate, R., & Antoniucci, D. (2011) · JAMA · doi:10.1001/jama.2011.1332 · Cited by 404
Abciximab and Heparin versus Bivalirudin for Non–ST-Elevation Myocardial Infarction
Kastrati, A., Neumann, F.-J., Schulz, S., Massberg, S., Byrne, R. A., Ferenc, M., Laugwitz, K.-L., Pache, J., Ott, I., Hausleiter, J., et al. (2011) · New England Journal of Medicine · doi:10.1056/nejmoa1109596 · Cited by 314
Cited by
Single-cell spatial immune landscapes of primary and metastatic brain tumours
Karimi, E., Yu, M. W., Maritan, S. M., Perus, L. J. M., Rezanejad, M., Sorin, M., Dankner, M., Fallah, P., Doré, S., Zuo, D., et al. (2023) · Nature · doi:10.1038/s41586-022-05680-3 · Cited by 401
Li, Y., Ge, X., Peng, F., Li, W., & Li, J. J. (2022) · Genome Biology · doi:10.1186/s13059-022-02648-4 · Cited by 303
Zeifman, R. J., Wagner, A. C., Watts, R., Kettner, H., Mertens, L. J., & Carhart-Harris, R. L. (2020) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2020.00782 · Cited by 106
“, > 30; (85, 86)], with Bonferroni corrections for multiple comparisons.”
Residual Cdk1/2 activity after DNA damage promotes senescence
Müllers, E., Silva Cascales, H., Burdova, K., Macurek, L., & Lindqvist, A. (2017) · Aging Cell · doi:10.1111/acel.12588 · Cited by 56
Reduced telomere length in neurodegenerative disorders may suggest shared biology
Kota, L. N., Bharath, S., Purushottam, M., Moily, N. S., Sivakumar, P. T., Varghese, M., Pal, P. K., & Jain, S. (2015) · The Journal of Neuropsychiatry and Clinical Neurosciences · doi:10.1176/appi.neuropsych.13100240 · Cited by 53
Metadata
- Authors
- Fagerland, M. W.
- Venue
- BMC Medical Research Methodology
- Published
- 2012-06-14
- Volume
- 12
- Issue
- 1
- Pages
- 78-78
- ISSN
- 1471-2288 (ISSN-L)
- OpenAlex
- W2144683894
- Cited by
- 547
- References
- 19
- FWCI
- 10.79
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: t-tests, non-parametric tests, and large studies—a paradox of statistical practice? https://app.cochat.ai/papers/10.1186/1471-2288-12-78
- The paper
- Fagerland, M. W. (2012). t-tests, non-parametric tests, and large studies—a paradox of statistical practice? BMC Medical Research Methodology. https://doi.org/10.1186/1471-2288-12-78
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.