CoChat

Managing the terror of publication bias: A comprehensive p-curve analysis of the Terror Management Theory literature

Open access

The DOI matches its Crossref record; 81 of the 81 references it cites were found in a registry or an index; 5 of the 41 sources its claims cite were checked for support.

Abstract

We assessed the evidential value of the large literature ( k =826 studies) investigating the mortality salience (MS) hypothesis from terror management theory. We employed a multitool assessment approach and reviewed past efforts to replicate experiments testing the MS hypothesis, and conducted a p -curve, a z -curve, and a random effects meta-analysis including bias corrections of the selection model, PET-PEESE, and WAAP-WLS on the studies. Overall, the different meta-analytic tools pointed to conflicting conclusions, reflecting differences in the methodology and philosophy of these tools. Our synthesis of these findings suggests there are true effects underlying some studies of the MS hypothesis, although the effects are highly heterogeneous, and the majority of studies are underpowered. We recommend future replications to assume a smaller effect size ( r = .10 ~ .15) and to follow expert guidance in the experimental protocol. Given the conflicting findings that emerged, we suggest that future attempts to evaluate other literatures would benefit from a multitool assessment approach.

Overview

CoChat Summary

TL;DR

This preprint uses p-curve, z-curve, and several bias-corrected meta-analytic methods to assess the mortality salience literature in terror management theory (TMT), based on 826 studies. The authors conclude that some true effects likely exist, but the literature is highly heterogeneous, underpowered, and affected by publication bias, so the average effect is much smaller than published estimates suggest.

Key findings

  • The p-curve main analysis included 826 studies and found significant right skew in both the half p-curve (Z = -14.08, p < .0012) and full p-curve (Z = -14.52, p < .001), which the authors interpret as evidence that nonzero effects underlie some studies.
  • P-curve estimated average power at 26% with 90% CI [21.6%, 29.7%], below the 33% benchmark the authors use for adequate power.
  • Z-curve on the same 826 significant effects estimated expected replication rate (ERR) = .19 (95% CI [.13, .26]) and expected discovery rate (EDR) = .08 (95% CI [.05, .16]), suggesting low overall power and many studies indistinguishable from false positives.
  • Traditional random-effects meta-analysis on 687 re-coded effects estimated Hedge’s g = .66 (95% CI [.62, .69]) with τ = .40 (95% CI [.36, .43]), but the funnel plot was highly asymmetric, indicating publication bias.
  • Bias-corrected methods gave smaller estimates: the selection model estimated g = .21 (95% CI [.06, .35]; roughly r = .10), PET gave g = -0.08 (SE = .04; p = .029), and WAAP-WLS used 89/687 effects with 80%+ power to estimate g = .30 (SE = .02; 95% CI [.25, .35]; roughly r = .15).

Methods

  • The authors conducted a comprehensive literature search using PsycINFO, prior reviews, and a TMT website, yielding 1373 unique manuscripts; after screening and applying inclusion criteria, 826 studies were suitable for p-curve/z-curve and 687 statistical tests were used in the classical meta-analyses.
  • Included studies had to test the mortality salience (MS) hypothesis on distal defenses, use a true experimental manipulation with random assignment, and have a p-curvable significant test tied to the primary hypothesis.
  • They hand-coded effects and analyzed the literature with p-curve, z-curve, and traditional meta-analyses with selection model, PET-PEESE, and WAAP-WLS bias corrections, using R and JASP.

Limitations

  • The paper explicitly notes that the literature is highly heterogeneous, and that p-curve results should not be interpreted as an overall average effect across all studies because of outliers, idiosyncratic methods, and diverse populations.
  • The authors state that many studies are underpowered and that published effect sizes are likely inflated by publication bias and p-hacking; this is a limitation of the literature and of some methods used to summarize it.
  • Evident from the text: the authors did not formally register the review protocol, though they say they used an OSF analysis plan and provide a PRISMA checklist.

Model-written summary of a shortened version of the paper’s full text (it is longer than we read). Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.21203/rs.3.rs-1254756/v1

Cites

81 of 81 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

5 Valid · 5 Not valid · 28 Not checked · 151 Own results and interpretations

107 more claims

Create a free account to see all 189 claims and to verify the 1 reference still open.

See all claims — free account
  • Probably not a substantial claim. 46
  • It may state two separate claims as one. 39

Create a free account to see these claims.

Statistics recomputed

Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.

  • Consistentz = -14.08reported p < .0012 · recomputed p < .001

    “Z = -14.08, p < .0012”

  • Consistentz = -14.52reported p < .001 · recomputed p < .001

    “Z = -14.52, p < .001”

  • Consistentz = 2.84reported p = .005 · recomputed p = .005

    “(z = 2.84, p = .005”

  • Consistentt(685) = -2.19reported p = .029 · recomputed p = .029

    “t(685) = -2.19, p = .029”

  • Consistentt(88) = 12.76reported p < .001 · recomputed p < .001

    “t(88)=12.76, p < .001”

5 of 6 · the rest aren’t loaded

1 more recomputed test

Create a free account to see every test the Stats check recomputed.

See all tests — free account

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Self-check results appear here once an independent check has confirmed them.

Citations

Cites

Show all 81 references

Research works printed but not linked: 6

  1. [4]4. Barto š , F ., & Schimmack, U. (2021). Z-curve.2.0: Estimating the replication and discovery rates. MetaPsychology, in press.
  2. [5]5. Baumeister, R. F., Tice, D. M., & Bushman, B. J. (in press). A review of multisite replication projects in social psychology: Is it viable to sustain any confidence in social psychology's knowledge base? Perspectives on Psychological Science.
  3. [16]16. Ellefson, M. R., & Oppenheimer, D. M. (in press). Is replication possible without fidelity? Psychological Methods.
  4. [39]39. McShane, B. B., Böckenholt, U., & Hansen, K. T. (2016). Adjusting for publication bias in metaanalysis: An evaluation of selection methods and some cautionary notes. Perspectives on Psychological Science, 11(5), 730-749.
  5. [56]56. Sætrevik, B., & Sjåstad, H. (2019). Failed pre-registered replication of mortality salience effects in traditional and novel measures. Retrieved from https://psyarxiv.com/dkg53/
  6. [70]70. Solomon, S., Greenberg, J., & Pyszczynski, T. (2015). The worm at the core. London, UK: Penguin.

Metadata

Authors
Chen, L., Benjamin, R., Guo, Y., Lai, A., & Heine, S. J.
Affiliations
  • McGill University (CA)
  • University of British Columbia (CA)
Venue
Research Square
Published
2023-05-15
OpenAlex
W4376604188
Cited by
5
References
67
Citations by year
2024: 3 · 2023: 2
Type
posted content
Language
en
Open access
Locations
Research Square (acceptedVersion, cc-by)
Topics
  • Social Sciences › Psychology › Social Psychology › Death Anxiety and Social Exclusion
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
Keywords
terror management theory, mortality salience, publication bias, p-curve, Z-curve, replication, underpowered studies, effect size heterogeneity

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Managing the terror of publication bias: A comprehensive p-curve analysis of the Terror Management Theory literature. https://app.cochat.ai/papers/10.21203/rs.3.rs-1254756/v1
The paper
Chen, L., Benjamin, R., Guo, Y., Lai, A., & Heine, S. J. (2023). Managing the terror of publication bias: A comprehensive p-curve analysis of the Terror Management Theory literature. Research Square. https://doi.org/10.21203/rs.3.rs-1254756/v1

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.