CoChat

Normality and Sample Size Do Not Matter for the Selection of an Appropriate Statistical Test for Two-Group Comparisons

Open access

The DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.

Abstract

Many applied researchers are taught to use the t-test when distributions appear normal and/or sample sizes are large and non-parametric tests otherwise, and fear inflated error rates if the “wrong” test is used. In a simulation study (four tests: t-test, Mann-Whitney test, Robust t-test, Permutation test; seven sample sizes between 2 × 10 and 2 × 500; four distributions: normal, uniform, log-normal, bimodal; under the null and alternate hypotheses), we show that type 1 errors are well controlled in all conditions. The t-test is most powerful under the normal and the uniform distributions, the Mann-Whitney test under the lognormal distribution, and the robust t-test under the bimodal distribution. Importantly, even the t-test was more powerful under asymmetric distributions than under the normal distribution for the same effect size. It appears that normality and sample size do not matter for the selection of a test to compare two groups of same size and variance. The researcher can opt for the test that fits the scientific hypothesis the best, without fear of poor test performance.

Overview

CoChat Summary

TL;DR

This paper argues that, for two-group comparisons, the choice of statistical test should not be driven by normality or sample size alone. The key idea is that the appropriate test depends on the research question, the type of data, and the inferential target rather than on automated rules.

Key findings

  • The paper’s central conclusion is stated in the title: “Normality and Sample Size Do Not Matter for the Selection of an Appropriate Statistical Test for Two-Group Comparisons.”
  • It was published as a methodological article in *Methodology*, volume 12(2), pages 61–71, in 2016.
  • Not stated in the paper.

Methods

  • The paper is a methodological article focused on selecting statistical tests for two-group comparisons.
  • The provided text does not state a study sample, dataset, or empirical data source.
  • The provided text does not state the specific analyses performed.

Limitations

  • Not stated in the paper.
  • Evident from the provided text: no results, numerical findings, or study design details are included here beyond bibliographic information and the article title.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1027/1614-2241/a000110

Cites

31 of 31 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

No claims could be anchored to a verbatim quote in the text.

Citations

Cites

Show all 31 references

Cited by

See all 49

Metadata

Authors
Poncet, A., Courvoisier, D. S., Combescure, C., & Perneger, T. V.
Affiliations
University of Geneva (CH)
Venue
Methodology
Published
2016-04
Volume
12
Issue
2
Pages
61-71
ISSN
  • 1614-1881 (ISSN-L)
  • 1614-2241
OpenAlex
W2467068711
Cited by
49
References
30
Citations by year
2026: 2 · 2025: 5 · 2024: 6 · 2023: 6 · 2022: 9 · 2020: 8 · 2019: 5 · 2018: 5 · 2017: 3
FWCI
4.39
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • Methodology (publishedVersion)
  • Archive ouverte UNIGE (University of Geneva) (submittedVersion, cc-by)
Topics
  • Physical Sciences › Mathematics › Statistics and Probability › Probability and Statistical Research
  • Physical Sciences › Mathematics › Statistics and Probability › Advanced Statistical Methods and Models
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Distribution Estimation and Applications
Keywords
statistical test selection, two-group comparison, t-test, Mann-Whitney test, permutation tests, type I error rate, statistical power, non-normal distributions, sample size, bimodal distribution, lognormal distribution, uniform distribution, effect size

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-10-05 (model deepseek-v4.1-flash, prompt extract-v3).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Normality and Sample Size Do Not Matter for the Selection of an Appropriate Statistical Test for Two-Group Comparisons. https://app.cochat.ai/papers/10.1027/1614-2241/a000110
The paper
Poncet, A., Courvoisier, D. S., Combescure, C., & Perneger, T. V. (2016). Normality and Sample Size Do Not Matter for the Selection of an Appropriate Statistical Test for Two-Group Comparisons. Methodology. https://doi.org/10.1027/1614-2241/a000110

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.