Do Baseline P-Values Follow a Uniform Distribution in Randomised Trials?
Open accessThe DOI matches its Crossref record; 11 of the 11 references it cites were found in a registry or an index.
Abstract
BACKGROUND: The theory has been put forward that if a null hypothesis is true, P-values should follow a Uniform distribution. This can be used to check the validity of randomisation. METHOD: The theory was tested by simulation for two sample t tests for data from a Normal distribution and a Lognormal distribution, for two sample t tests which are not independent, and for chi-squared and Fisher's exact test using small and using large samples. RESULTS: For the two sample t test with Normal data the distribution of P-values was very close to the Uniform. When using Lognormal data this was no longer true, and the distribution had a pronounced mode. For correlated tests, even using data from a Normal distribution, the distribution of P-values varied from simulation run to simulation run, but did not look close to Uniform in any realisation. For binary data in a small sample, only a few probabilities were possible and distribution was very uneven. With a sample of two groups of 1,000 observations, there was great unevenness in the histogram and a poor fit to the Uniform. CONCLUSIONS: The notion that P-values for comparisons of groups using baseline data in randomised clinical trials should follow a Uniform distribution if the randomisation is valid has been found to be true only in the context of independent variables which follow a Normal distribution, not for Lognormal data, correlated variables, or binary data using either chi-squared or Fisher's exact tests. This should not be used as a check for valid randomisation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures






Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“However, it is possible to get a non-uniform distribution of p-values when the two groups were randomised, for example, for skewed data. 15”
From Automated detection of over- and under-dispersion in baseline tables in randomised controlled trials. (2022) · cites this paper as [15]
The cited source says what the claim attributes to it.
In the citing paper
“I simulated data using two scenarios described by Bland 15 with no concern about randomisation but where the p-value distribution would be non-uniform, meaning the uniform test could return a high percentage of false positives”
From Automated detection of over- and under-dispersion in baseline tables in randomised controlled trials. (2022) · cites this paper as [15]
In the citing paper
“The distribution of baseline p-values from simulations of variables with highly positively skewed lognormal distributions, and simulations in which all variables had fixed or high levels of correlation was not uniform [8, 9].”
From Rounding, but not randomization method, non-normality, or correlation, affected baseline p-value distributions in randomized trials. (2019) · cites this paper as [8]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Confidence intervals rather than P values: estimation rather than hypothesis testing.
Gardner, M. J., & Altman, D. G. (1986) · BMJ · doi:10.1136/bmj.292.6522.746 · Cited by 2,010 · cited 1×
“For many years, researchers reporting clinical trials have been urged to replace P values with estimates and confidence intervals [10,11].”
BackgroundPocock, S. J., Assmann, S. E., Enos, L. E., & Kasten, L. E. (2002) · Statistics in Medicine · doi:10.1002/sim.1296 · Cited by 1,077
Cha, M., Eastwaad, S., Hartan, R., Maher, D., Olkin, I., Pitkin, R., Rennie, D., Schulz, K. F., Simel, D. L., & Straup, D. F. (1996) · Cited by 737
Schulz, K. F. (1994) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.1994.03520020051014 · Cited by 512
Randomisation and baseline comparisons in clinical trials
Altman, D. G., & Doré, C. J. (1990) · The Lancet · doi:10.1016/0140-6736(90)90014-v · Cited by 508 · cited 1×
“There is a large literature criticising the use of significance tests of differences in baseline variables in randomised clinical trials [1,2,3,4,5,6].”
BackgroundTesting for baseline balance in clinical trials
Senn, S. (1994) · Statistics in Medicine · doi:10.1002/sim.4780131703 · Cited by 503 · cited 1×
“There is a large literature criticising the use of significance tests of differences in baseline variables in randomised clinical trials [1,2,3,4,5,6].”
BackgroundComparability of Randomised Groups
Altman, D. G. (1985) · The Statistician · doi:10.2307/2987510 · Cited by 328 · cited 1×
“There is a large literature criticising the use of significance tests of differences in baseline variables in randomised clinical trials [1,2,3,4,5,6].”
BackgroundUnderstanding controlled trials: Baseline imbalance in randomised controlled trials
Roberts, C., & Torgerson, D. J. (1999) · BMJ · doi:10.1136/bmj.319.7203.185 · Cited by 286 · cited 1×
“There is a large literature criticising the use of significance tests of differences in baseline variables in randomised clinical trials [1,2,3,4,5,6].”
BackgroundDetecting Selection Bias in Randomized Clinical Trials
Berger, V. W., & Exner, D. V. (1999) · Controlled Clinical Trials · doi:10.1016/s0197-2456(99)00014-8 · Cited by 163 · cited 1×
“Berger and Exner [9] present a test for imbalance which can be more powerful than baseline comparisons, being based on response data rather than baseline.”
Background59 Subversion of allocation in a randomised controlled trial
Kennedy, A., & Grant, A. (1997) · Controlled Clinical Trials · doi:10.1016/s0197-2456(97)91044-8 · Cited by 16
Cited by
Kuiper, J. S., Zuidersma, M., Zuidema, S. U., Burgerhof, J. G. M., Stolk, R. P., Oude Voshaar, R. C., & Smidt, N. (2016) · International Journal of Epidemiology · doi:10.1093/ije/dyw089 · Cited by 301
Toolkit for Weighting and Analysis of Nonequivalent Groups: A tutorial for the twang package
Ridgeway, G. K., McCarey, D., Morral, A. R., Burgette, L. F., & Grin, B. A. (2014) · Cited by 285
Why Welch's test is Type I error robust
Derrick, B., & White, P. (2016) · The Quantitative Methods for Psychology · doi:10.20982/tqmp.12.1.p030 · Cited by 258
“If H0 is true and if underlying assumptions hold, then the p-values from a valid test procedure are expected to be uniformly distributed (Bland, 2013).”
BackgroundHaverkamp, B. F., Wiersma, R., Vertessen, K., van Ewijk, H., Oosterlaan, J., & Hartman, E. (2020) · Journal of Sports Sciences · doi:10.1080/02640414.2020.1794763 · Cited by 238
“If p values were reported as p > 0.05, we assumed a p-value of 0.53 (i.e. the average of 0.05 and 1.00) in order to calculate the standard error and 95% CI (Bland, 2013).”
MethodInfluentialvan Aert, R. C. M., Wicherts, J. M., & van Assen, M. A. L. M. (2016) · Perspectives on Psychological Science · doi:10.1177/1745691616650874 · Cited by 219
“The distribution of (conditional) p values based on the true effect size is only uniform when the assumptions of the underlying statistical model (e.g., independence or effect distribution are valid; Bland, 2013]).”
BackgroundNo evidence for a bilingual executive function advantage in the ABCD study
Dick, A. S., Garcia, N. L., Pruden, S. M., Thompson, W. K., Hawes, S. W., Sutherland, M. T., Riedel, M. C., Laird, A. R., & Gonzalez, R. (2019) · Nature Human Behaviour · doi:10.1038/s41562-019-0609-3 · Cited by 201
The Statistical Value of Raw Fluorescence Signal in Luminex xMAP Based Multiplex Immunoassays
Breen, E. J., Tan, W., & Khan, A. (2016) · Scientific Reports · doi:10.1038/srep26996 · Cited by 134
“Value Chisq Pr(>Chisq) Sig April (42) Normal-COPD − 0.908 14.060 0.040 * − 0.656 4.678 1.000 April (42) Normal-Mono 1.383 69.060 0.000 * * * 1.484 54.024 0.000 * * * Baff (37) Normal-Mono − 0.907 29.659 0.000 * * * − 0.319 2.492 1.000 CD163 (46) Normal-Mono − 0.775 21.684 0.001 * * * − 0.760 14.180 0.032 * CD30 (53) N…”
MethodInfluentialReproducibility of Methods to Detect Differentially Expressed Genes from Single-Cell RNA Sequencing
Mou, T., Deng, W., Gu, F., Pawitan, Y., & Vu, T. N. (2020) · Frontiers in Genetics · doi:10.3389/fgene.2019.01331 · Cited by 76
“Theoretically, the p-values should follow a uniform distribution if the null hypothesis is true (Murdoch et al., 2008; Bland, 2013).”
BackgroundMethods to assess research misconduct in health-related research: A scoping review
Bordewijk, E. M., Li, W., van Eekelen, R., Wang, R., Showell, M., Mol, B. W., & van Wely, M. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.05.012 · Cited by 68
Perneger, T., & Combescure, C. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.04.003 · Cited by 51
Machine learning methods for vehicle predictive maintenance using off-board and on-board data
Prytz, R. (2014) · DiVA (Digitala Vetenskapliga Arkivet) · Cited by 71
Clark, L., Fairhurst, C., Hewitt, C. E., Birks, Y., Brabyn, S., Cockayne, S., Rodgers, S., Hicks, K., Hodgson, R., Littlewood, E., et al. (2014) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2014.04.007 · Cited by 36
“We suggest that this method can be used to assess the reliability of randomization; however, caution must be taken, because the P-values for baseline tests will follow a uniform distribution only if the tests used to compute them are valid and independent [13].”
MethodNetwork‐Based Approaches for Pathway Level Analysis
Nguyen, T., Mitrea, C., & Drăghici, S. (2018) · Current Protocols in Bioinformatics · doi:10.1002/cpbi.42 · Cited by 41
“In principle, the p -values produced by a sound statistical test must be uniformly distributed under the null hypothesis (Barton, Crozier, Lillycrop, Godfrey, & Inskip, 2013; Bland, 2013; Fodor, Tickle, & Richard-son, 2007; Storey & Tibshirani, 2003).”
BackgroundMethodBolland, M. J., Gamble, G. D., Avenell, A., & Grey, A. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.03.001 · Cited by 31
“In addition, concerns have been raised that non-normal, or fixed and/or h ighly correlated baseline variables may lead to non-uniformly distributed baseline p-values [7-9].”
BackgroundMethodBolland, M. J., Gamble, G. D., Avenell, A., Grey, A., & Lumley, T. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.05.006 · Cited by 27
Nguyen, T., Mitrea, C., Tagett, R., & Drăghici, S. (2016) · Proceedings of the IEEE · doi:10.1109/jproc.2015.2507119 · Cited by 28
“The p-values produced by a sound statistical test must be uniformly distributed in the interval [0, 1] when the null hypothesis is true [33]–[35], [46].”
BackgroundMethodOn two-stage Monte Carlo tests of composite hypotheses
Baddeley, A. J., Hardegen, A., Lawrence, T., Milne, R. K., Nair, G. M., & Rakshit, S. (2017) · Computational Statistics & Data Analysis · doi:10.1016/j.csda.2017.04.003 · Cited by 28
“…tests are currently used beyond their known range23 of validity (Diggle, 2003, p. 89; Baddeley et al., 2014; Myllymäki et al., 2017).24 Similar concerns apply to p-values in randomization tests (Bland, 2013).25 One effective remedy is to perform two nested stages of sampling or simula-26 tion.”
Statistical approach for selection of biologically informative genes
Das, S., Rai, A. K., Mishra, D. C., & Rai, S. N. (2018) · Gene · doi:10.1016/j.gene.2018.02.044 · Cited by 18
“It may be noted that these P i 's ( i = 1, 2, … , p ) are iid's and each one follows a uniform distribution with parameter [0, 1] (Bland, 2013) ( i.e.”
The distribution of probability values in medical abstracts: an observational study
Ginsel, B. L., Aggarwal, A., Xuan, W., & Harris, I. A. (2015) · BMC Research Notes · doi:10.1186/s13104-015-1691-x · Cited by 15
Parasites make hosts more profitable but less available to predators
Prosnier, L., Loeuille, N., Hulot, F. D., Renault, D., Piscart, C., Bicocchi, B., Deparis, M., Lam, M., & Médoc, V. (2024) · Oikos · doi:10.1111/oik.10469 · Cited by 14
Shetty, S., Cappellari, M., McDermid, R. M., Krajnović, D., de Zeeuw, P. T., Davies, R. L., & Kobayashi, C. (2020) · Monthly Notices of the Royal Astronomical Society · doi:10.1093/mnras/staa1043 · Cited by 14
“Hence a high value of p-value is only indicative of the acceptance of the hypothesis but not a quantification of it (Bland 2013).”
BackgroundStatistical Approach for Gene Set Analysis with Trait Specific Quantitative Trait Loci
Das, S., Rai, A. K., Mishra, D. C., & Rai, S. N. (2018) · Scientific Reports · doi:10.1038/s41598-018-19736-w · Cited by 11
The equilibrium phase in heterogeneous Hertzian chains
Przedborski, M., Sen, S., & Harroun, T. A. (2017) · Journal of Statistical Mechanics Theory and Experiment · doi:10.1088/1742-5468/aa9a62 · Cited by 10
Correlation among baseline variables yields non-uniformity of p-values
Betensky, R. A., & Chiou, S. H. (2017) · PLOS ONE · doi:10.1371/journal.pone.0184531 · Cited by 8
“These latter two points were studied by Bland [3], who showed that even under the null of equality of randomized groups, Table 3.”
BackgroundMethodResultInfluentialSteimer, A., Zubler, F., & Schindler, K. A. (2015) · NeuroImage · doi:10.1016/j.neuroimage.2015.05.089 · Cited by 9
No evidence for a bilingual executive function advantage in the ABCD study.Preprint
Dick, A. S. (2018) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/gjh95 · Cited by 12
Barnett, A. (2022) · F1000Research · doi:10.12688/f1000research.123002.1 · Cited by 7
“I simulated data using two scenarios described by Bland 15 with no concern about randomisation but where the p-value distribution would be non-uniform, meaning the uniform test could return a high percentage of false positives: (cid:129) Small trials with a sample size of 10 and summary statistics using binary data (e…”
InfluentialRiedle, B. N., Neath, A. A., & Cavanaugh, J. (2020) · Journal of Applied Statistics · doi:10.1080/02664763.2020.1754360 · Cited by 6
“For instance, Bland [6] and Boos and Stefanski [7] argue the need for a more refined interpretation of p -value distribution theory.”
BackgroundVerykouki, E., & Nakas, C. T. (2023) · Stats · doi:10.3390/stats6020035 · Cited by 4
Griffin, B. A., Sanchez, R., Cefalu, M. S., Stelzner, C., Ayer, L. A., Coffman, D. L., Vegetabile, B., Pane, J., Burgette, L. F., & McCaffrey, D. F. (2020) · RAND Corporation eBooks · doi:10.7249/tla570-2 · Cited by 2
Dissertation R.C.M. van AertPreprint
van Aert, R. C. M. (2018) · MetArXiv (OSF Preprints) · doi:10.31222/osf.io/eqhjd · Cited by 3
Automated detection of over- and under-dispersion in baseline tables in randomised controlled trialsPreprint
Barnett, A. G. (2022) · Research Square · doi:10.21203/rs.3.rs-1496802/v1 · Cited by 2
CHIEN, P. F. W. (2023) · International Journal of Gynecology & Obstetrics · doi:10.1002/ijgo.15185 · Cited by 2
A Hypothesis Testing Approach to Sharing Logs with Confidence
Long, Y., Xu, L., & Gunter, C. A. (2020) · ACM Conference on Data and Application Security and Privacy (CODASPY) · doi:10.1145/3374664.3375743 · Cited by 2
“, the log-generating process is γ -log indistinguishable), the p-values should be uniformly distributed [7].”
BackgroundStack overflow's hidden nuances: How does zip code define user contribution?
Zolduoarrati, E., Licorish, S. A., & Stanger, N. (2025) · Journal of Systems and Software · doi:10.1016/j.jss.2025.112374 · Cited by 2
RNA-Seq Analysis: Methods, Applications and Challenges
vari, A., Geraci, F., Saha, I., & Bianchini, M. (2020) · Frontiers research topics · doi:10.3389/978-2-88963-705-8 · Cited by 4
Martínez, O., Arce-Rodríguez, M. L., Hernández-Godínez, F., Escoto-Sandoval, C., Cervantes‐Hernández, F., Hayano‐Kanashiro, C., Ordaz-Ortíz, J. J., Reyes‐Valdés, M. H., Razo-Mendivil, F. G., Claver, A. G., et al. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.10.05.326470 · Cited by 1
Nonlinear dynamics of granular assemblies
Przedborski, M. (2017) · Brock University Digital Repository (Brock University)
Deviations from Expectations: A Commentary on Aliev et al.
van der Sluis, S., Vroom, C.-R., & Dolan, C. V. (2018) · Behavior Genetics · doi:10.1007/s10519-018-9891-5
“2008; Bland 2013).”
A Brief Critique of the TATES Procedure
Alıev, F., Salvatore, J. E., Arpana Agrawal, A., Almasy, L. A., Chan, G., Edenberg, H. J., HESSELBROCK, V. M., Samuel Kuperman, C., Meyers, J. L., & Dick, D. M. (2018) · Behavior Genetics · doi:10.1007/s10519-018-9890-6
“In the ideal case the distribution will be uniform (Bland 2013; Murdoch et al. 2008).”
BackgroundDiagnosing fraudulent baseline data in clinical trials
Proschan, M. A., & Shaw, P. A. (2020) · PLoS ONE · doi:10.1371/journal.pone.0239121 · Cited by 2
“3 Number of significant z-scores We have focused on L(2) as a test statistic for detecting cheating, but other goodness of fit statistics such as those considered by Betensky and Chiou [3] and Bland [4] have similar behavior.”
BackgroundInfluentialDas, S. (2020) · ThinkIR: The University of Louisville's Institutional Repository (University of Louisville) · doi:10.18297/etd/3537
A statistical perspective of gene set analysis with trait-specific QTL in molecular crop breeding
Das, S., & Rai, S. N. (2022) · Elsevier eBooks · doi:10.1016/b978-0-323-85243-2.00005-2
Intersections of Statistical and Substantive Significance Under a True and False Null HypothesisPreprint
Komaroff, E. (2024) · Qeios · doi:10.32388/3qqsnc · Cited by 1
A new theoretical base for intracranial brain volume normalization in neuroimaging studies
Marco, M., Alessandro, E., Caproni, S., Carlo, P., Massimo, P., Di, R. A., Pitzianti, M. B., Michelina, C., Stefania, F., Marco, I., et al. (2024) · Journal of Neurology Neurological Science and Disorders · doi:10.17352/jnnsd.000062
A Redemption Song for Statistical SignificancePreprint
Komaroff, E. (2024) · Qeios · doi:10.32388/3qqsnc.3
Metadata
- Authors
- Bland, M.
- Affiliations
- University of York (GB)
- Corresponding author
- Bland, Martin
- ORCID
- Bland, Martin
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2013-10-01
- Volume
- 8
- Issue
- 10
- Pages
- e76010
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC3788030
- OpenAlex
- W2090163739
- Cited by
- 47
- References
- 11
- Citations by year
- 2025: 1 · 2024: 4 · 2023: 3 · 2022: 3 · 2021: 1 · 2020: 10 · 2019: 3 · 2018: 7 · 2017: 5 · 2016: 5 · 2015: 2 · 2014: 3
- FWCI
- 2.93
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- National Institute for Health and Care Research (NF-SI-0507-10133)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Physical Sciences › Mathematics › Statistics and Probability › Probability and Statistical Research
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- p-value distribution, uniform distribution, baseline data, randomized controlled trials, two-sample t-test, lognormal distribution, correlated data, binary data, chi-square test, Fisher's exact test, null hypothesis testing
- MeSH terms
- Humans, Statistics as Topic, Reproducibility of Results, Randomized Controlled Trials as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.