Managing the terror of publication bias: A comprehensive p-curve analysis of the Terror Management Theory literature
Open accessThe DOI matches its Crossref record; 81 of the 81 references it cites were found in a registry or an index; 5 of the 41 sources its claims cite were checked for support.
Abstract
We assessed the evidential value of the large literature ( k =826 studies) investigating the mortality salience (MS) hypothesis from terror management theory. We employed a multitool assessment approach and reviewed past efforts to replicate experiments testing the MS hypothesis, and conducted a p -curve, a z -curve, and a random effects meta-analysis including bias corrections of the selection model, PET-PEESE, and WAAP-WLS on the studies. Overall, the different meta-analytic tools pointed to conflicting conclusions, reflecting differences in the methodology and philosophy of these tools. Our synthesis of these findings suggests there are true effects underlying some studies of the MS hypothesis, although the effects are highly heterogeneous, and the majority of studies are underpowered. We recommend future replications to assume a smaller effect size ( r = .10 ~ .15) and to follow expert guidance in the experimental protocol. Given the conflicting findings that emerged, we suggest that future attempts to evaluate other literatures would benefit from a multitool assessment approach.
Overview
CoChat Summary
TL;DR
This preprint uses p-curve, z-curve, and several bias-corrected meta-analytic methods to assess the mortality salience literature in terror management theory (TMT), based on 826 studies. The authors conclude that some true effects likely exist, but the literature is highly heterogeneous, underpowered, and affected by publication bias, so the average effect is much smaller than published estimates suggest.
Key findings
- The p-curve main analysis included 826 studies and found significant right skew in both the half p-curve (Z = -14.08, p < .0012) and full p-curve (Z = -14.52, p < .001), which the authors interpret as evidence that nonzero effects underlie some studies.
- P-curve estimated average power at 26% with 90% CI [21.6%, 29.7%], below the 33% benchmark the authors use for adequate power.
- Z-curve on the same 826 significant effects estimated expected replication rate (ERR) = .19 (95% CI [.13, .26]) and expected discovery rate (EDR) = .08 (95% CI [.05, .16]), suggesting low overall power and many studies indistinguishable from false positives.
- Traditional random-effects meta-analysis on 687 re-coded effects estimated Hedge’s g = .66 (95% CI [.62, .69]) with τ = .40 (95% CI [.36, .43]), but the funnel plot was highly asymmetric, indicating publication bias.
- Bias-corrected methods gave smaller estimates: the selection model estimated g = .21 (95% CI [.06, .35]; roughly r = .10), PET gave g = -0.08 (SE = .04; p = .029), and WAAP-WLS used 89/687 effects with 80%+ power to estimate g = .30 (SE = .02; 95% CI [.25, .35]; roughly r = .15).
Methods
- The authors conducted a comprehensive literature search using PsycINFO, prior reviews, and a TMT website, yielding 1373 unique manuscripts; after screening and applying inclusion criteria, 826 studies were suitable for p-curve/z-curve and 687 statistical tests were used in the classical meta-analyses.
- Included studies had to test the mortality salience (MS) hypothesis on distal defenses, use a true experimental manipulation with random assignment, and have a p-curvable significant test tied to the primary hypothesis.
- They hand-coded effects and analyzed the literature with p-curve, z-curve, and traditional meta-analyses with selection model, PET-PEESE, and WAAP-WLS bias corrections, using R and JASP.
Limitations
- The paper explicitly notes that the literature is highly heterogeneous, and that p-curve results should not be interpreted as an overall average effect across all studies because of outliers, idiosyncratic methods, and diverse populations.
- The authors state that many studies are underpowered and that published effect sizes are likely inflated by publication bias and p-hacking; this is a limitation of the literature and of some methods used to summarize it.
- Evident from the text: the authors did not formally register the review protocol, though they say they used an OSF analysis plan and provide a PRISMA checklist.
Model-written summary of a shortened version of the paper’s full text (it is longer than we read). Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
5 Valid · 5 Not valid · 28 Not checked · 151 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“The results of this attempt did not provide much support for the MS hypothesis. The meta-analytic effect size was negligible (Hedges' g = 0.03), even in those studies where the replicators strictly adhered to the original authors' guidelines.”
What the sources say
- Partly supportedcites Klein et al., 2022 sourceExperimental — accuracy not yet measured
Research Question 1 reports little evidence for the predicted mortality-salience effect across sites, and the Discussion says neither the Author Advised nor In House protocols replicated the original finding. But the overall meta-analytic estimates were Hedges’ g = 0.07–0.09, not 0.03. The g = 0.03 estimate in section 6 applies to five sites whose investigators reported substantial TMT knowledge under one exclusion rule; the author-advised-only estimates were 0.08–0.19. (LLM, conf=0.90)
plainn = 2,220- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“A multisite replication effort (Klein et al., 2022) overall found little evidence that the selected study replicated”
What the sources say
- Supportedcites Klein et al., 2022 source
The Abstract reports that the 17-lab effort was unable to replicate the selected Terror Management Theory finding under any conditions. The Discussion likewise describes the result as a failure to replicate. (LLM, conf=0.94)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“in the multisite replication effort (Klein et al., 2022), the effect size estimates were nominally higher in the author-advised locations than in those that followed an in-house protocol”
What the sources say
- Supportedcites Klein et al., 2022 source
The section on expert-only protocols says effect sizes were descriptively larger at Author Advised sites. Research Question 2 reports positive but nonsignificant differences between Author Advised and In House protocols, consistent with the claim’s qualification that the estimates were only nominally higher. (LLM, conf=0.93)
hedgedstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“the multisite replication effort did not successfully replicate the targeted study (Klein et al., 2022)”
What the sources say
- Supportedcites Klein et al., 2022 source
The Discussion states that neither the Author Advised nor the In House protocols successfully replicated the original finding. This directly supports the claim about the multisite replication effort. (LLM, conf=0.95)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“we found an estimated ERR of .19 (CI 95 = [.13, .26])”
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
107 more claims
Create a free account to see all 189 claims and to verify the 1 reference still open.
- Probably not a substantial claim. 46
- It may state two separate claims as one. 39
Statistics recomputed
Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.
- Consistentz = -14.08reported p < .0012 · recomputed p < .001
“Z = -14.08, p < .0012”
- Consistentz = -14.52reported p < .001 · recomputed p < .001
“Z = -14.52, p < .001”
- Consistentz = 2.84reported p = .005 · recomputed p = .005
“(z = 2.84, p = .005”
- Consistentt(685) = -2.19reported p = .029 · recomputed p = .029
“t(685) = -2.19, p = .029”
- Consistentt(88) = 12.76reported p < .001 · recomputed p < .001
“t(88)=12.76, p < .001”
1 more recomputed test
Create a free account to see every test the Stats check recomputed.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Self-check results appear here once an independent check has confirmed them.
Citations
Cites
R: A Language and Environment for Statistical Computing
R Core Team (2025) · doi:10.32614/r.manuals · Cited by 352,181
Conducting Meta-Analyses in R with the metafor Package
Viechtbauer, W. (2010) · Journal of Statistical Software · doi:10.18637/jss.v036.i03 · Cited by 20,464
The PRISMA 2020 statement: an updated guideline for reporting systematic reviews
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., et al. (2021) · Systematic Reviews · doi:10.1186/s13643-021-01626-4 · Cited by 13,874
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
(2000) · Cited by 9,198
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068
Estimating the reproducibility of psychological science
Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
The Causes and Consequences of a Need for Self-Esteem: A Terror Management Theory
Greenberg, J., Pyszczynski, T., & Solomon, S. (1986) · Public Self and Private Self · doi:10.1007/978-1-4613-9564-5_10 · Cited by 3,032
Raise standards for preclinical cancer research
Begley, C. G., & Ellis, L. M. (2012) · Nature · doi:10.1038/483531a · Cited by 2,979
Research works printed but not linked: 6
- [4]4. Barto š , F ., & Schimmack, U. (2021). Z-curve.2.0: Estimating the replication and discovery rates. MetaPsychology, in press.
- [5]5. Baumeister, R. F., Tice, D. M., & Bushman, B. J. (in press). A review of multisite replication projects in social psychology: Is it viable to sustain any confidence in social psychology's knowledge base? Perspectives on Psychological Science.
- [16]16. Ellefson, M. R., & Oppenheimer, D. M. (in press). Is replication possible without fidelity? Psychological Methods.
- [39]39. McShane, B. B., Böckenholt, U., & Hansen, K. T. (2016). Adjusting for publication bias in metaanalysis: An evaluation of selection methods and some cautionary notes. Perspectives on Psychological Science, 11(5), 730-749.
- [56]56. Sætrevik, B., & Sjåstad, H. (2019). Failed pre-registered replication of mortality salience effects in traditional and novel measures. Retrieved from https://psyarxiv.com/dkg53/
- [70]70. Solomon, S., Greenberg, J., & Pyszczynski, T. (2015). The worm at the core. London, UK: Penguin.
Cited by
Treger, S., Benau, E. M., & Timko, C. A. (2023) · PLOS ONE · doi:10.1371/journal.pone.0285267 · Cited by 25
Riesthuis, P., & Otgaar, H. (2024) · Journal of Criminal Psychology · doi:10.1108/jcp-04-2024-0031 · Cited by 8
Untrustworthy Evidence in Dishonesty Research
Bartos̆, F. (2024) · Meta-Psychology · doi:10.15626/mp.2023.3987 · Cited by 4
Arendsen, J. L., Brugman, B. C., van Koningsbruggen, G. M., & Fransen, M. L. (2025) · International Journal of Consumer Studies · doi:10.1111/ijcs.70056 · Cited by 4
The Untrustworthy Evidence in Dishonesty ResearchPreprint
Bartos̆, F. (2023) · doi:10.31234/osf.io/dy5s2
Metadata
- Authors
- Chen, L., Benjamin, R., Guo, Y., Lai, A., & Heine, S. J.
- Affiliations
- McGill University (CA)
- University of British Columbia (CA)
- Venue
- Research Square
- Publisher
- Research Square Platform LLC
- Published
- 2023-05-15
- OpenAlex
- W4376604188
- Cited by
- 5
- References
- 67
- Citations by year
- 2024: 3 · 2023: 2
- Type
- posted content
- Language
- en
- Open access
- green
- doi.org
- Locations
- Research Square (acceptedVersion, cc-by)
- Topics
- Social Sciences › Psychology › Social Psychology › Death Anxiety and Social Exclusion
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Keywords
- terror management theory, mortality salience, publication bias, p-curve, Z-curve, replication, underpowered studies, effect size heterogeneity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Managing the terror of publication bias: A comprehensive p-curve analysis of the Terror Management Theory literature. https://app.cochat.ai/papers/10.21203/rs.3.rs-1254756/v1
- The paper
- Chen, L., Benjamin, R., Guo, Y., Lai, A., & Heine, S. J. (2023). Managing the terror of publication bias: A comprehensive p-curve analysis of the Terror Management Theory literature. Research Square. https://doi.org/10.21203/rs.3.rs-1254756/v1
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.