CoChat

Do Baseline P-Values Follow a Uniform Distribution in Randomised Trials?

Open access
Bland, Martin · 2013 · PLoS ONE

The DOI matches its Crossref record; 11 of the 11 references it cites were found in a registry or an index.

Abstract

BACKGROUND: The theory has been put forward that if a null hypothesis is true, P-values should follow a Uniform distribution. This can be used to check the validity of randomisation. METHOD: The theory was tested by simulation for two sample t tests for data from a Normal distribution and a Lognormal distribution, for two sample t tests which are not independent, and for chi-squared and Fisher's exact test using small and using large samples. RESULTS: For the two sample t test with Normal data the distribution of P-values was very close to the Uniform. When using Lognormal data this was no longer true, and the distribution had a pronounced mode. For correlated tests, even using data from a Normal distribution, the distribution of P-values varied from simulation run to simulation run, but did not look close to Uniform in any realisation. For binary data in a small sample, only a few probabilities were possible and distribution was very uneven. With a sample of two groups of 1,000 observations, there was great unevenness in the histogram and a poor fit to the Uniform. CONCLUSIONS: The notion that P-values for comparisons of groups using baseline data in randomised clinical trials should follow a Uniform distribution if the randomisation is valid has been found to be true only in the context of independent variables which follow a Normal distribution, not for Lognormal data, correlated variables, or binary data using either chi-squared or Fisher's exact tests. This should not be used as a check for valid randomisation.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1371/journal.pone.0076010

Cites

11 of 11 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 Distribution of P-values for 10,000 two sample t tests for Normal data. Means were compared between two groups of 10 observations from a Standard Normal distribution.
Figure 1 Distribution of P-values for 10,000 two sample t tests for Normal data. Means were compared between two groups of 10 observations from a Standard Normal distribution.CC BY · PubMed Central
Figure 2 Distribution of P-values for 10,000 two sample t tests for Lognormal data. Means were compared between two groups of 10 observations from a Lognormal distribution.
Figure 2 Distribution of P-values for 10,000 two sample t tests for Lognormal data. Means were compared between two groups of 10 observations from a Lognormal distribution.CC BY · PubMed Central
Figure 3 P-values from four realisations of 10,000 correlated t tests for Normal data. Means were compared between two groups of 10 observations from a Standard Normal distribution where each test used variables with correlation 0.5 with the other variables.
Figure 3 P-values from four realisations of 10,000 correlated t tests for Normal data. Means were compared between two groups of 10 observations from a Standard Normal distribution where each test used variables with correlation 0.5 with the other variables.CC BY · PubMed Central
Figure 4 Distribution of P-values for chi-squared tests and Fisher’s exact tests for two by two tables. Chi-squared and Fisher’s exact test, both two-sided and one-sided, were calculated for the comparison of two samples of size 10 with a binary outcome variable with probability 0.5 of being 0 and 0.5 of being 1.
Figure 4 Distribution of P-values for chi-squared tests and Fisher’s exact tests for two by two tables. Chi-squared and Fisher’s exact test, both two-sided and one-sided, were calculated for the comparison of two samples of size 10 with a binary outcome variable with probability 0.5 of being 0 and 0.5 of being 1.CC BY · PubMed Central
Figure 5 P-values for chi-squared and Fisher’s exact tests for large samples with fixed size groups. Chi-squared and Fisher’s exact test, both two-sided and one-sided, were calculated for10,000 comparisons of two samples of size 1,000 with a binary outcome variable with probability 0.5 of being 0 and 0.5 of being 1.
Figure 5 P-values for chi-squared and Fisher’s exact tests for large samples with fixed size groups. Chi-squared and Fisher’s exact test, both two-sided and one-sided, were calculated for10,000 comparisons of two samples of size 1,000 with a binary outcome variable with probability 0.5 of being 0 and 0.5 of being 1.CC BY · PubMed Central
Figure 6 P-values for chi-squared and Fisher’s exact tests for large groups of varying size. Chi-squared and Fisher’s exact test, both two-sided and one-sided, were calculated for10,000 comparisons in samples of size 2,000, both group and outcome having probability 0.5 of being 0 and 0.5 of being 1.
Figure 6 P-values for chi-squared and Fisher’s exact tests for large groups of varying size. Chi-squared and Fisher’s exact test, both two-sided and one-sided, were calculated for10,000 comparisons in samples of size 2,000, both group and outcome having probability 0.5 of being 0 and 0.5 of being 1.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Cited by

See all 47

Metadata

Authors
Bland, M.
Affiliations
University of York (GB)
Corresponding author
Bland, Martin
Venue
PLoS ONE
Published
2013-10-01
Volume
8
Issue
10
Pages
e76010
ISSN
1932-6203 (ISSN-L)
OpenAlex
W2090163739
Cited by
47
References
11
Citations by year
2025: 1 · 2024: 4 · 2023: 3 · 2022: 3 · 2021: 1 · 2020: 10 · 2019: 3 · 2018: 7 · 2017: 5 · 2016: 5 · 2015: 2 · 2014: 3
FWCI
2.93
Type
journal article
Language
en
Open access
Locations
  • PLoS ONE (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
National Institute for Health and Care Research (NF-SI-0507-10133)
Topics
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
  • Physical Sciences › Mathematics › Statistics and Probability › Probability and Statistical Research
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Keywords
p-value distribution, uniform distribution, baseline data, randomized controlled trials, two-sample t-test, lognormal distribution, correlated data, binary data, chi-square test, Fisher's exact test, null hypothesis testing
MeSH terms
Humans, Statistics as Topic, Reproducibility of Results, Randomized Controlled Trials as Topic

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.