Combining independent p values: Extensions of the Stouffer and binomial methods.
Abstract
The Stouffer z method, and other popular methods for combining p values from independent significance tests, suffer from three problems: vulnerability to criticisms of the individual studies being pooled, difficulty in handling the "file drawer problem," and vague conclusions. These problems can be reduced or eliminated by supplementing a test of combined probability with a variety of new analyses described here. Along with other advantages, these analyses provide a way to address the file drawer problem without making limiting assumptions about the nature of studies not included in the pooled analysis. These analyses can supplement a traditional meta-analysis, yielding conclusions not provided by widely used meta-analytic procedures.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
If 10 individual results are significant with α = .05, pooled p ≤ .05 only if the total number of studies is up to 110.
“If 10 individual results are significant with α = .05,then pooled p ≤ .05only if the total number of studies is up to 110.”
From On the correspondence between dream content and target material under laboratory conditions: A meta-analysis of dream-ESP studies, 1966-2016 (2017) · cites this paper as Darlington and Hayes's (2000, p. 503, Table 2)
Darlington and Hayes (2000) regard mean(z) as the real test statistic for testing overall significance of a database.
“As Storm et al. (2010) advised, a more stringent measure than the Stouffer Z for testing the overall significance of a database is provided by Darlington and Hayes (2000), who regard”
From On the correspondence between dream content and target material under laboratory conditions: A meta-analysis of dream-ESP studies, 1966-2016 (2017) · cites this paper as Darlington and Hayes (2000)
Cited by
Sorted by citations
Finak, G., McDavid, A., Yajima, M., Deng, J., Gersuk, V., Shalek, A. K., Slichter, C. K., Miller, H. W., McElrath, M. J., Prlic, M., et al. (2015) · Genome Biology · doi:10.1186/s13059-015-0844-5 · Cited by 3,159 · see citing works
Hagedoorn, M., Sanderman, R., Bolks, H. N., Tuinstra, J., & Coyne, J. C. (2008) · Psychological Bulletin · doi:10.1037/0033-2909.134.1.1 · Cited by 777 · see citing works
Dowling, N. A., Merkouris, S. S., Greenwood, C. J., Oldenhof, E., Toumbourou, J. W., & Youssef, G. J. (2017) · Clinical Psychology Review · doi:10.1016/j.cpr.2016.10.008 · Cited by 439 · see citing works
Storm, L., Tressoldi, P. E., & Di Risio, L. (2010) · Psychological Bulletin · doi:10.1037/a0019457 · Cited by 140 · see citing works
Storm, L., Sherwood, S. J., Roe, C. A., Tressoldi, P. E., Rock, A. J., & Di Risio, L. (2017) · International Journal of Dream Research · doi:10.11588/ijodr.2017.2.34888 · Cited by 29 · see citing works
“(2010) advised, a more stringent measure than the Stouffer Z for testing the overall significance of a database is provided by Darlington and Hayes (2000), who regard “mean(z) as the real test statistic” (p. 505). We applied Darlington and Hayes’s (2000) so-called ‘Stouffer-max test’ that provides a so-called MeanZ(s,…”
BackgroundMethod
- Authors
- Darlington, R. B., & Hayes, A. F.
- Affiliations
- Cornell University (US)
- Venue
- Psychological Methods
- Published
- 2000
- Volume
- 5
- Issue
- 4
- Pages
- 496-515
- ISSN
- 1082-989X (ISSN-L)
- 1939-1463
- OpenAlex
- W2090778449
- Cited by
- 66
- References
- 0
- Citations by year
- 2025: 2 · 2024: 1 · 2023: 1 · 2022: 3 · 2021: 4 · 2020: 4 · 2019: 1 · 2018: 3 · 2017: 5 · 2016: 6 · 2015: 8 · 2014: 5 · 2013: 1 · 2012: 1
- FWCI
- 1.27
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Methods (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Physical Sciences › Mathematics › Statistics and Probability › Probability and Statistical Research
- Keywords
- combining p-values, file drawer problem, meta-analysis, publication bias, meta-analytic methods
- MeSH terms
- Humans, Probability, Psychometrics, Meta-Analysis as Topic, Binomial Distribution
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
