The DOI matches its Crossref record, with a small difference in title or year; 41 of the 41 references it cites were found in a registry or an index.
p -Curve and Effect Size: Correcting for Publication Bias Using Only Significant Results
Abstract
Journals tend to publish only statistically significant evidence, creating a scientific record that markedly overstates the size of effects. We provide a new tool that corrects for this bias without requiring access to nonsignificant results. It capitalizes on the fact that the distribution of significant p values, p-curve, is a function of the true underlying effect. Researchers armed only with sample sizes and test results of the published findings can correct for publication bias. We validate the technique with simulations and by reanalyzing data from the Many-Labs Replication project. We demonstrate that p-curve can arrive at conclusions opposite that of existing tools by reanalyzing the meta-analysis of the “choice overload” literature.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“If findings cannot be published in peer-reviewed journals due to bias toward publishing only statistically significant results (Simonsohn et al., 2014), researchers should consider filing a report on sites such as PsyArXiv so that the results can be included in future reviews or analyses.”
From Acceptance and Commitment Therapy Processes and Mediation: Challenges and How to Address Them (2023) · cites this paper as Simonsohn et al., 2014
In the citing paper
“A p-curve plots the distribution of statistically significant p values (p < 0.05) in published studies, and the shape of the p-curve is a function only of the effect size and sample size, when the power level is taken into account.”
From Working Memory Training Does Not Improve Performance on Measures of Intelligence or Other Measures of “Far Transfer”: Evidence From a Meta-Analytic Review (2016) · cites this paper as Simonsohn, Nelson & Simmons, 2014
In the citing paper
“P-curve is a recently developed method that deals with the weaknesses in the funnel plot/trim- and-fill analysis (Simonsohn, Nelson & Simmons, 2014).”
From Working Memory Training Does Not Improve Performance on Measures of Intelligence or Other Measures of “Far Transfer”: Evidence From a Meta-Analytic Review (2016) · cites this paper as Simonsohn, Nelson & Simmons, 2014
In the citing paper
“However, more novel and promising methods, such as p-curve (Simonsohn, Nelson, & Simmons, 2014) and variations of Egger's regression (Egger, Smith, Schneider, & Minder, 1997;Moreno et al., 2009), have been found to perform better than the trim and fill method.”
From Working memory training revisited: A multi-level meta-analysis of n-back training studies (2017) · cites this paper as Simonsohn et al., 2014
In the citing paper
“Reporting bias is assessed by funnel plots including Egger’s test (Egger et al., 1997) and p-curves (Simonsohn et al., 2014) also using the dmetar package.”
From Mindfulness-Based and Mindfulness-Informed Interventions at the Workplace: A Systematic Review and Meta-Regression Analysis of RCTs (2023) · cites this paper as Simonsohn et al., 2014
Citations
Cites
Measuring inconsistency in meta-analyses
Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003) · BMJ · doi:10.1136/bmj.327.7414.557 · Cited by 63,139
The Framing of Decisions and the Psychology of Choice
Tversky, A., & Kahneman, D. (1981) · Science · doi:10.1126/science.7455683 · Cited by 18,497 · cited 2×
“The Asian disease problem consisted of a binary hypothetical choice problem that presented participants with the same information using either a gains or a losses frame (Tversky & Kahneman, 1981).”
Duval, S., & Tweedie, R. (2000) · Biometrics · doi:10.1111/j.0006-341x.2000.00455.x · Cited by 14,099
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068
Power failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3475 · Cited by 7,641
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
Self-citationJournal of The American Statistical Association
Fienberg, S. E. (2005) · Encyclopedia of Statistical Sciences · doi:10.1002/0471667196.ess1317 · Cited by 6,125
Toward a positive theory of consumer choice
Thaler, R. (1980) · Journal of Economic Behavior & Organization · doi:10.1016/0167-2681(80)90051-7 · Cited by 5,505
Instructional manipulation checks: Detecting satisficing to increase statistical power
Oppenheimer, D. M., Meyvis, T., & Davidenko, N. (2009) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2009.03.009 · Cited by 3,493
A Nonparametric “Trim and Fill” Method of Accounting for Publication Bias in Meta-Analysis
Duval, S., & Tweedie, R. (2000) · Journal of the American Statistical Association · doi:10.1080/01621459.2000.10473905 · Cited by 2,955
Research works printed but not linked: 2
- Simmons J. P., 2012, Dialogue: The Official Newsletter of the Society for Personality and Social Psychology, 26, 4
- Sterling T. D., 1959, Journal of the American Statistical Association, 54, 30
Cited by
The Extent and Consequences of P-Hacking in Science
Head, M. L., Holman, L., Lanfear, R., Kahn, A. T., & Jennions, M. D. (2015) · PLOS Biology · doi:10.1371/journal.pbio.1002106 · Cited by 1,388
Sisk, V. F., Burgoyne, A. P., Sun, J., Butler, J. L., & Macnamara, B. N. (2018) · Psychological Science · doi:10.1177/0956797617739704 · Cited by 1,159
Studying and modifying brain function with non-invasive brain stimulation
Polanía, R., Nitsche, M. A., & Ruff, C. C. (2018) · Nature Neuroscience · doi:10.1038/s41593-017-0054-4 · Cited by 1,028
Indices of Effect Existence and Significance in the Bayesian Framework
Makowski, D., Ben-Shachar, M. S., Chen, S. H. A., & Lüdecke, D. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.02767 · Cited by 983
“It is interesting to note that this perspective unites significance testing with the focus on effect size (involving a discrete separation between at least two categories: negligible and non-negligible), which finds an echo in recent statistical recommendations (Ellis and Steyn, 2003; Sullivan and Feinn, 2012; Simonso…”
BackgroundMuris, P., Merckelbach, H., Otgaar, H., & Meijer, E. (2017) · Perspectives on Psychological Science · doi:10.1177/1745691616666070 · Cited by 946
Metadata
- Authors
- Simonsohn, U., Nelson, L. D., & Simmons, J. P.
- Affiliations
- University of Pennsylvania (US)
- University of California, Berkeley (US)
- Venue
- Perspectives on Psychological Science
- Publisher
- SAGE Publications
- Published
- 2014-11
- Volume
- 9
- Issue
- 6
- Pages
- 666-681
- ISSN
- 1745-6916 (ISSN-L)
- 1745-6924
- OpenAlex
- W3125384312
- Cited by
- 717
- References
- 40
- Citations by year
- 2026: 30 · 2025: 39 · 2024: 53 · 2023: 52 · 2022: 86 · 2021: 90 · 2020: 107 · 2019: 83 · 2018: 54 · 2017: 60 · 2016: 37 · 2015: 22 · 2014: 4
- FWCI
- 40.12
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Locations
- Perspectives on Psychological Science (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Keywords
- p-curve, publication bias, effect size estimation, effect size overestimation, choice overload, meta-analysis reanalysis, statistical significance
- MeSH terms
- Computer Simulation, Statistics as Topic, Publication Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: p -Curve and Effect Size: Correcting for Publication Bias Using Only Significant Results. https://app.cochat.ai/papers/10.1177/1745691614553988
- The paper
- Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2014). p -Curve and Effect Size: Correcting for Publication Bias Using Only Significant Results. Perspectives on Psychological Science. https://doi.org/10.1177/1745691614553988
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.