CoChat
Sign Up
Cited by 296Probability and Statistical ResearchStatistical Distribution Estimation and ApplicationsStatistical Methods in Clinical Trials

To test or not to test: Preliminary assessment of normality when comparing two independent samples

Open access
Rochon, Justine, Gondan, Matthias, Kieser, Meinhard·2012·BMC Medical Research Methodology
DOI 10.1186/1471-2288-12-81PMC PMC3444333
Open this paper in CoChat — free accountOpen PDF

Abstract

BACKGROUND: Student's two-sample t test is generally used for comparing the means of two independent samples, for example, two treatment arms. Under the null hypothesis, the t test assumes that the two samples arise from the same normally distributed population with unknown variance. Adequate control of the Type I error requires that the normality assumption holds, which is often examined by means of a preliminary Shapiro-Wilk test. The following two-stage procedure is widely accepted: If the preliminary test for normality is not significant, the t test is used; if the preliminary test rejects the null hypothesis of normality, a nonparametric test is applied in the main analysis. METHODS: Equally sized samples were drawn from exponential, uniform, and normal distributions. The two-sample t test was conducted if either both samples (Strategy I) or the collapsed set of residuals from both samples (Strategy II) had passed the preliminary Shapiro-Wilk test for normality; otherwise, Mann-Whitney's U test was conducted. By simulation, we separately estimated the conditional Type I error probabilities for the parametric and nonparametric part of the two-stage procedure. Finally, we assessed the overall Type I error rate and the power of the two-stage procedure as a whole. RESULTS: Preliminary testing for normality seriously altered the conditional Type I error rates of the subsequent main analysis for both parametric and nonparametric tests. We discuss possible explanations for the observed results, the most important one being the selection mechanism due to the preliminary test. Interestingly, the overall Type I error rate and power of the entire two-stage procedure remained within acceptable limits. CONCLUSION: The two-stage procedure might be considered incorrect from a formal perspective; nevertheless, in the investigated examples, this procedure seemed to satisfactorily maintain the nominal significance level and had acceptable power properties.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1186/1471-2288-12-81

Cites

62 of 62 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The sample of each group was large enough, and comparisons were not affected by the shape of the error distribution and no transformation was applied.

    “The sample of each group was large enough, and comparisons were not affected by the shape of the error distribution and no transformation was applied [27].”

    From Effects of short-term breathing exercises on respiratory recovery in patients with COVID-19: a quasi-experimental study (2022) · cites this paper as [27]

    The Results (Strategy I; page unavailable) report that an unconditional two-sample t test maintained its nominal Type I error rate for the simulated normal, uniform, and exponential distributions. The paper does not establish that the citing study’s groups were large enough or that its comparisons were unaffected by their error distribution; it also does not address the citing authors’ decision not to transform their data. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

1 of the 63 references couldn’t be matched to a known paper.

Research works printed but not linked: 3

  1. New England Journal of Medicine: Guidelines for manuscript submission. (Retrieved from http://www.nejm.org/page/author-center/manuscript-submission); 2011
  2. Pappas PA DePuy V An overview of non-parametric tests in SAS: When, why and how Proceeding of the. SouthEast SAS Users Group Conference (SESUG 2004): Paper TU04 2004 Miami, FL, SouthEast SAS Users Group 1 5
  3. ICH E3 Structure and content of clinical study reports 1995 International Conference on Harmonisation, London, UK

Cited by

Sorted by citations

Related

Authors
Rochon, J., Gondan, M., & Kieser, M.
Affiliations
Heidelberg University (DE)
Corresponding author
Rochon, Justine
ORCID
Venue
BMC Medical Research Methodology
Publisher
Springer Science and Business Media LLC
Published
2012-06-19
Volume
12
Issue
1
Pages
81-81
ISSN
1471-2288 (ISSN-L)
DOI
10.1186/1471-2288-12-81
PMC
PMC3444333
OpenAlex
W2162286641
Cited by
296
References
63
Citations by year
2026: 28 · 2025: 44 · 2024: 35 · 2023: 25 · 2022: 33 · 2021: 37 · 2020: 24 · 2019: 15 · 2018: 13 · 2017: 10 · 2016: 9 · 2015: 13 · 2014: 6 · 2013: 3 · 2012: 1
FWCI
7.03
Type
journal article
Language
en
Open access
Licence
Locations
  • BMC Medical Research Methodology (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Research at the University of Copenhagen (University of Copenhagen) (publishedVersion, other-oa)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Physical Sciences › Mathematics › Statistics and Probability › Probability and Statistical Research
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Distribution Estimation and Applications
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
Keywords
two-sample t-test, normality testing, Shapiro-Wilk test, two-stage procedure, selection bias, Mann-Whitney test, nonparametric tests, statistical power, independent samples, exponential distribution, uniform distribution
MeSH terms
Computer Simulation, Data Interpretation, Statistical, Humans, Models, Statistical, Selection Bias, Normal Distribution, Statistics, Nonparametric

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review