P-curve: A key to the file-drawer.
The DOI matches its Crossref record; 49 of the 49 references it cites were found in a registry or an index.
Abstract
Because scientists tend to report only studies (publication bias) or analyses (p-hacking) that "work," readers must ask, "Are these effects true, or do they merely reflect selective reporting?" We introduce p-curve as a way to answer this question. P-curve is the distribution of statistically significant p values for a set of studies (ps < .05). Because only true effects are expected to generate right-skewed p-curves-containing more low (.01s) than high (.04s) significant p values--only right-skewed p--curves are diagnostic of evidential value. By telling us whether we can rule out selective reporting as the sole explanation for a set of findings, p-curve offers a solution to the age-old inferential problems caused by file-drawers of failed studies and analyses.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Incorrect p-values could affect the outcome of tests that analyze the distribution of p-values, such as the p-curve (Simonsohn, Nelson, & Simmons, 2014) and p-uniform (Van Assen, Van Aert, & Wicherts, 2014).”
From The prevalence of statistical reporting errors in psychology (1985–2013) (2016) · cites this paper as Simonsohn, Nelson, & Simmons, 2014
In the citing paper
“Even if the first REM episode occurs about 70 minutes after falling asleep (33), some subjects may present a REM EEG and occulomotor pattern a way farster.The REM sleep can actually occur pretty rapidly in some individuals according to the MD (sleep specialist) who co-supervised this study.Furthermore, the MSLT has specialy been built for this puporse (see32)”
From Is there a Publication Bias in Behavioural Intranasal Oxytocin Research on Humans? Opening the File Drawer of One Laboratory (2016) · cites this paper as [33]
In the citing paper
“After excluding studies which assessed maltreatment as part of broader measures of victimization (k=3) (25, 27, 29) or ACEs (k=5) (30, 34, 39, 44, 49), or focused on institutional neglect (k=3) (15, 45, 46), the meta-analytic effect size was also similar to the original estimate (Cohen’s d=0.34, 95% CI=0.26-0.42).”
From Childhood Maltreatment and Mental Health Problems: A Systematic Review and Meta-Analysis of Quasi-Experimental Studies (2023) · cites this paper as [25]
In the citing paper
“Trim‐and‐fill procedures do suffer from some well‐known limitations that could result in the underestimation or overestimation of publication bias (Rothstein, 2008; Simonsohn, Nelson, & Simmons, 2014).”
From Hot spots policing of small geographic areas effects on crime (2019) · cites this paper as Simonsohn, Nelson, & Simmons, 2014
In the citing paper
“Applying selection modeling approaches like p-uniform or 3PSM to the simple effects may therefore substantially overestimate the true effect size.”
From Do Salient Social Norms Moderate Mortality Salience Effects? A (Challenging) Meta-Analysis of Terror Management Studies (2023) · cites this paper as Simonsohn et al., 2014
Citations
Cites
Statistical power analysis for the behavioral sciences
(1990) · Computers, Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,269
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722
(1963) · Medical Entomology and Zoology · Cited by 15,085
Duval, S., & Tweedie, R. (2000) · Biometrics · doi:10.1111/j.0006-341x.2000.00455.x · Cited by 14,099
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068
The case for motivated reasoning.
Kunda, Z. (1990) · Psychological Bulletin · doi:10.1037/0033-2909.108.3.480 · Cited by 9,022
Cohen, J. (1992) · Psychological Bulletin · doi:10.1037//0033-2909.112.1.155 · Cited by 8,771
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
Self-citationTesting Statistical Hypotheses
(2021) · Wiley Series in Probability and Statistics · doi:10.1002/9781119243830.ch12 · Cited by 5,106
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 1×
“1 Trying multiple analyses to obtain statistical significance has received many names, including: bias (Ioannidis, 2005), significance chasing (Ioannidis & Trikalinos, 2007), data snooping (White, 2003), fiddling (Bross, 1971), publication bias in situ (Phillips, 2004) and specification searching (Gerber & Malhotra, 2…”
Cited by
Items 201–300 of 1,005
Cuddy, A. J. C., Schultz, S. J., & Fosse, N. E. (2018) · Psychological Science · doi:10.1177/0956797617746749 · Cited by 132
van der Pol, T. M., Hoeve, M., Noom, M. J., Stams, G. J. J. M., DORELEIJERS, T. A. H., van Domburgh, L., & Vermeiren, R. R. J. M. (2017) · Journal of Child Psychology and Psychiatry · doi:10.1111/jcpp.12685 · Cited by 129
“Third, we utilized the P-curve method, which was recently introduced by Simonsohn, Nelson, and Simmons (2014).”
MethodPaper Meets Plastic: The Perceived Environmental Friendliness of Product Packaging
Sokolova, T., Krishna, A., & Döring, T. (2023) · Journal of Consumer Research · doi:10.1093/jcr/ucad008 · Cited by 135
99% impossible: A valid, or falsifiable, internal meta-analysis.
Vosgerau, J., Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2019) · Journal of Experimental Psychology: General · doi:10.1037/xge0000663 · Cited by 98
Self-citationVadillo, M. A., Gold, N., & Osman, M. (2016) · Psychological Science · doi:10.1177/0956797616654911 · Cited by 141
Quality Uncertainty Erodes Trust in Science
Vazire, S. (2017) · Collabra: Psychology · doi:10.1525/collabra.74 · Cited by 106
Gómez‐de‐Mariscal, E., Guerrero, V., Sneider, A., Jayatilaka, H., Phillip, J. M., Wirtz, D., & Muñoz‐Barrutia, A. (2021) · Scientific Reports · doi:10.1038/s41598-021-00199-5 · Cited by 133
Booksmythe, I., Mautz, B. S., Davis, J., Nakagawa, S., & Jennions, M. D. (2015) · Biological reviews/Biological reviews of the Cambridge Philosophical Society · doi:10.1111/brv.12220 · Cited by 112
Rom, S. C., Weiß, A., & Conway, P. J. (2016) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2016.09.007 · Cited by 115
Predicting replication outcomes in the Many Labs 2 study
Forsell, E., Viganola, D., Pfeiffer, T., Almenberg, J., Wilson, B., Chen, Y., Nosek, B. A., Johannesson, M., & Dreber, A. (2018) · Journal of Economic Psychology · doi:10.1016/j.joep.2018.10.009 · Cited by 111
van Dierendonck, D., & Lam, H. (2022) · Applied Psychology: Health and Well-Being · doi:10.1111/aphw.12398 · Cited by 107
Risk and protective factors for (internet) gaming disorder: A meta-analysis of pre-COVID studies
Ropovik, I., Martončik, M., Babinčák, P., Baník, G., Vargová, L., & Adamkovic, M. (2022) · Addictive Behaviors · doi:10.1016/j.addbeh.2022.107590 · Cited by 136
Outlier Removal and the Relation with Reporting Errors and Quality of Psychological Research
Bakker, M., & Wicherts, J. M. (2014) · PLoS ONE · doi:10.1371/journal.pone.0103360 · Cited by 101
Estimating the deep replicability of scientific findings using human and artificial intelligence
Yang, Y., Youyou, W., & Uzzi, B. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1909046117 · Cited by 119
“277 between individual metrics and replicability (1, 7, 14, 15) (details are in SI Appendix, Section S1 and Table S1).”
BackgroundLife After NHST: How to Describe Your Data Without “p-ing” Everywhere
Valentine, J. C., Aloe, A. M., & Lau, T. S. (2015) · Basic and Applied Social Psychology · doi:10.1080/01973533.2015.1060240 · Cited by 111
Threats to Racial Status Promote Tea Party Support Among White AmericansPreprint
Willer, R., Feinberg, M., & Wetts, R. (2016) · SSRN Electronic Journal · doi:10.2139/ssrn.2770186 · Cited by 105
Boggero, I. A., Hostinar, C. E., Haak, E. A., Murphy, M. L. M., & Segerstrom, S. C. (2017) · Biological Psychology · doi:10.1016/j.biopsycho.2017.08.058 · Cited by 97
“These alternatives were differentiated by testing the set of findings against a null of 33% power as outlined by Simonsohn et al. (2014); if significant, findings could be assumed to lack evidential value; otherwise, evidential value could not be confirmed or denied.”
BackgroundMethodInfluentialA meta-analysis of childhood maltreatment and intimate partner violence perpetration
Li, S., Zhao, F., & Yu, G. (2019) · Aggression and Violent Behavior · doi:10.1016/j.avb.2019.101362 · Cited by 133
Martínez‐Vizcaíno, V., Sanabria‐Martínez, G., Fernández‐Rodríguez, R., Cavero‐Redondo, I., Pascual‐Morena, C., Álvarez‐Bueno, C., & Martínez‐Hortelano, J. A. (2023) · BJOG: An International Journal of Obstetrics & Gynaecology · doi:10.1111/1471-0528.17304 · Cited by 97
Nieuwenstein, M. R., Wierenga, T., Morey, R. D., Wicherts, J. M., Blom, T. N., Wagenmakers, E., & van Rijn, H. (2015) · Judgment and Decision Making · doi:10.1017/s1930297500003144 · Cited by 126
“9 Simonsohn et al. (2014) show that the trim-and-fill performs poorly in correcting for publication bias when this bias is based on selective reporting of significant effects, that is, when there is a publication bias against non-significant effects.”
Oser, J. L., Grinson, A., Boulianne, S., & Halperin, E. (2022) · Political Communication · doi:10.1080/10584609.2022.2086329 · Cited by 128
Shewach, O. R., Sackett, P. R., & Quint, S. (2019) · Journal of Applied Psychology · doi:10.1037/apl0000420 · Cited by 97
Dunning, T., Grossman, G., Humphreys, M., Hyde, S., McIntosh, C., Nellis, G., Adida, C. L., Arias, E., Bicalho, C., Boas, T. C., et al. (2019) · Science Advances · doi:10.1126/sciadv.aaw2612 · Cited by 127
Specificity of Future Thinking in Depression: A Meta-Analysis
Gamble, B., Moreau, D., Tippett, L. J., & Addis, D. R. (2019) · Perspectives on Psychological Science · doi:10.1177/1745691619851784 · Cited by 110
An Aberrant Abundance of Cronbach’s Alpha Values at .70
Hussey, I., Alsalti, T., Bosco, F. A., Elson, M., & Arslan, R. C. (2025) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459241287123 · Cited by 228
“In this sense, the distributions of α s are suspicious, much like a player in Blackjack who gets exactly 21 too often—or indeed, a series of studies with p values mostly between .025 and .049 (Simonsohn et al., 2014).”
Celebrating Organization Theory
Lounsbury, M. D., & Beckman, C. M. (2014) · Journal of Management Studies · doi:10.1111/joms.12091 · Cited by 129
“…psychology) decrying a crisis of data mining, ‘p-hacking’, and the search for ‘asterisks’ (Bettis, 2012; DeLong and Lang, 1992; Goldfarb and King, 2013; Simonsohn et al., 2013), a focus on ‘facts’ may encourage searching vast quantities of data for statistical significance rather than producing…”
BackgroundProgress toward openness, transparency, and reproducibility in cognitive neuroscience
Gilmore, R. O., Diaz, M. T., Wyble, B. A., & Yarkoni, T. (2017) · Annals of the New York Academy of Sciences · doi:10.1111/nyas.13325 · Cited by 95
Demographics as predictors of suicidal thoughts and behaviors: A meta-analysis
Huang, X., Ribeiro, J. D., Musacchio, K. M., & Franklin, J. C. (2017) · PLOS ONE · doi:10.1371/journal.pone.0180793 · Cited by 94
Cooke, S. J., Birnie‐Gauvin, K., Lennox, R. J., Taylor, J. J., Rytwinski, T., Rummer, J. L., Franklin, C. E., Bennett, J., & Haddaway, N. (2017) · Conservation Physiology · doi:10.1093/conphys/cox043 · Cited by 115
“Otherwise, bias can be introduced by serial implementation of possible statistical methods until one is identi fi ed that yields desired results (p-hacking: Simonsohn et al. , 2014; Head et al. , 2015).”
MethodPredicting the replicability of social science lab experiments
Altmejd, A., Dreber, A., Forsell, E., Huber, J., Imai, T., Johannesson, M., Kirchler, M., Nave, G., & Camerer, C. F. (2019) · PLoS ONE · doi:10.1371/journal.pone.0225826 · Cited by 118
“[4, 35, 36]) should be seen as a complement to our work and could hopefully be used to improve future versions of this predictive model.”
BackgroundPollet, T. V., & van der Meij, L. (2016) · Adaptive Human Behavior and Physiology · doi:10.1007/s40750-016-0050-z · Cited by 122
“This could be done for example via p curves (Simonsohn et al. 2014).”
Background10 years of Bayesian theories of autism: A comprehensive review
Angeletos Chrysaitis, N., & Seriès, P. (2023) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2022.105022 · Cited by 92
You Cannot Step Into the Same River Twice: When Power Analyses Are Optimistic
McShane, B. B., & Böckenholt, U. (2014) · Perspectives on Psychological Science · doi:10.1177/1745691614548513 · Cited by 101
Registered Replication Report: Dijksterhuis and van Knippenberg (1998)
O’Donnell, M., Nelson, L. D., Ackermann, E., Aczel, B., Akhtar, A., Aldrovandi, S., Alshaif, N., Andringa, R., Aveyard, M., Babincak, P., et al. (2018) · Perspectives on Psychological Science · doi:10.1177/1745691618755704 · Cited by 92
“Yet a more recent study using p -curve (Simonsohn, Nelson, & Simmons, 2014) to evaluate the “18 p -values from 16 studies reported in 8 articles” on professor priming indicated that “the studies contain evidential value” (Lakens, 2017, p. 8).”
Self-citationFocused deterrence strategies effects on crime: A systematic review
Braga, A. A., Weisburd, D., & Turchan, B. (2019) · Campbell Systematic Reviews: Better Evidence for a Better World · doi:10.1002/cl2.1051 · Cited by 92
Mesquida, C., Murphy, J., Lakens, D., & Warne, J. P. (2022) · Royal Society Open Science · doi:10.1098/rsos.220946 · Cited by 116
“Fortunately, issues relating to the power of the studies, p-hacking and publication bias can be explored via the distribution of reported p-values [43,44].”
BackgroundMethodInfluentialGildersleeve, K. A., Haselton, M. G., & Fales, M. R. (2014) · Psychological Bulletin · doi:10.1037/a0037714 · Cited by 108
Cybulski, Ł., Mayo‐Wilson, E., & Grant, S. (2016) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000115 · Cited by 95
Cunningham, M. R., & Baumeister, R. F. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2016.01639 · Cited by 98
Statistical learning and dyslexia: a systematic review
Schmalz, X., Altoè, G., & Mulatti, C. (2016) · Annals of Dyslexia · doi:10.1007/s11881-016-0136-0 · Cited by 108
Corruption Information and Vote Share: A Meta-Analysis and Lessons for Experimental Design
Incerti, T. (2020) · American Political Science Review · doi:10.1017/s000305542000012x · Cited by 93
Grebe, N. M., Giudice, M. D., Thompson, M. E., Nickels, N., Ponzi, D., Zilioli, S., Maestripieri, D., & Gangestad, S. W. (2019) · Hormones and Behavior · doi:10.1016/j.yhbeh.2019.01.006 · Cited by 97
“The p-curve analyses are equivocal as to the evidential value of the reported effects (i.e., the analyses cannot determine whether selective reporting is the sole explanation for the given effects; see Simonsohn et al., 2014 and supplementary online materials [SOM] for interpretation).”
InfluentialFoley, S., & Hughes, C. H. (2018) · Developmental Review · doi:10.1016/j.dr.2018.03.007 · Cited by 105
Human Fear Chemosignaling: Evidence from a Meta-Analysis
de Groot, J. H. B., & Smeets, M. A. M. (2017) · Chemical Senses · doi:10.1093/chemse/bjx049 · Cited by 88
“With only 5 P -values, p -curve analysis will have higher statistical power than the individual studies (Simonsohn et al. 2014a).”
InfluentialDistributions of p -values smaller than .05 in psychology: what is going on?
Hartgerink, C. H. J., van Aert, R. C. M., Nuijten, M. B., Wicherts, J. M., & van Assen, M. A. L. M. (2016) · PeerJ · doi:10.7717/peerj.1935 · Cited by 89
“Indeed, the more right-skewed 251 the p-value distribution, the higher the effect size estimate when keeping constant studies’ sample sizes 252 (Simonsohn et al., 2014; van Assen et al., 2015).”
BackgroundMethodHandling effect size dependency in meta-analysis
Gucciardi, D. F., Lines, R. L. J., & Ntoumanis, N. (2021) · International Review of Sport and Exercise Psychology · doi:10.1080/1750984x.2021.1946835 · Cited by 97
Schwens, C., Zapkau, F. B., Bierwerth, M., Isidor, R., Knight, G., & Kabst, R. (2017) · Entrepreneurship Theory and Practice · doi:10.1111/etap.12280 · Cited by 97
RetractedA Validity-Based Framework for Understanding Replication in Psychology
Fabrigar, L. R., Wegener, D. T., & Petty, R. E. (2020) · Personality and Social Psychology Review · doi:10.1177/1088868320931366 · Cited by 123
“Because of this, some researchers have focused on developing indices aimed at detecting when selective reporting might be occurring in an effort to prevent the publication of exaggerated statistical evidence (e.g., Francis, 2012; Schimmack, 2012; Simonsohn et al., 2014).”
BackgroundSocial Comparisons and Deception Across Workplace Hierarchies: Field and Experimental Evidence
Edelman, B. G., & Larkin, I. (2014) · Organization Science · doi:10.1287/orsc.2014.0938 · Cited by 103
Yilmaz, O., & Saribay, S. A. (2016) · Judgment and Decision Making · doi:10.1017/s1930297500003119 · Cited by 88
A Meta-Analytic Re-Appraisal of the Framing Effect
Steiger, A., & Kühberger, A. (2018) · Zeitschrift für Psychologie · doi:10.1027/2151-2604/a000321 · Cited by 106
Laraway, S., Snycerski, S., Pradhan, S., & Huitema, B. E. (2019) · Perspectives on Behavior Science · doi:10.1007/s40614-019-00193-3 · Cited by 103
“Simonsohn, Nelson, and Simmons (2014) gave other examples of p-hacking: “adding a covariate [post hoc], data peeking (adding new observations), data exclusions (e.g., dropping “outliers”), choosing among several correlated dependent variables, and choosing among several experimental conditions” (p.…”
BackgroundKepes, S., Bennett, A. A., & McDaniel, M. A. (2014) · Academy of Management Learning and Education · doi:10.5465/amle.2013.0193 · Cited by 110
Can infants learn phonology in the lab? A meta-analytic answer
Cristià, A. (2017) · Cognition · doi:10.1016/j.cognition.2017.09.016 · Cited by 94
Deciding what to observe: Thoughts for a post-WEIRD generation
Barrett, H. C. (2020) · Evolution and Human Behavior · doi:10.1016/j.evolhumbehav.2020.05.006 · Cited by 98
van den Akker, O. R., van Assen, M. A. L. M., van Vugt, M., & Wicherts, J. M. (2020) · Journal of Economic Psychology · doi:10.1016/j.joep.2020.102329 · Cited by 98
Reproducibility in Neuroimaging Analysis: Challenges and Solutions
Botvinik‐Nezer, R., & Wager, T. D. (2022) · Biological Psychiatry Cognitive Neuroscience and Neuroimaging · doi:10.1016/j.bpsc.2022.12.006 · Cited by 116
Improving urban energy efficiency: What role does the digital economy play?
Wu, J., Lin, K., & Sun, J. (2023) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2023.138104 · Cited by 96
Aldrich, N. J., Chen, J., & Alfieri, L. (2021) · Developmental Review · doi:10.1016/j.dr.2021.100946 · Cited by 99
Does lack of control lead to conspiracy beliefs? A meta‐analysis
Stojanov, A., & Halberstadt, J. B. (2020) · European Journal of Social Psychology · doi:10.1002/ejsp.2690 · Cited by 106
A meta-analysis of loneliness and risk of dementia using longitudinal data from >600,000 individuals
Luchetti, M., Aschwanden, D., Sesker, A. A., Zhu, X., O’Súilleabháin, P. S., Stéphan, Y., Terracciano, A., & Sutin, A. R. (2024) · Nature Mental Health · doi:10.1038/s44220-024-00328-9 · Cited by 112
“We further conducted P curve analyses 92,93 to explore the consistency and distribution of statistically significant results to assess whether significant findings are the result of selective reporting.”
The profile of abstract rule learning in infancy: Meta‐analytic and experimental evidence
Rabagliati, H., Ferguson, B., & Lew‐Williams, C. (2018) · Developmental Science · doi:10.1111/desc.12704 · Cited by 95
“These tests are performed by using Stouffer’s method to aggregate the probability of obtaining each significant p value under the uniform or low power distributions (see Simonsohn et al., 2014, 2015, for further details).”
MethodRogde, K., Hagen, Å. M. M., Melby‐Lervåg, M., & Lervåg, A. (2019) · Campbell Systematic Reviews · doi:10.1002/cl2.1059 · Cited by 120
“…publication bias in this review, we selected a p ‐ curve analysis instead. p ‐ Curve analysis is a recently developed method that addresses the weaknesses in funnel plot/trim ‐ and ‐ fill analysis (Simonsohn et al., 2014 “ no evidential value ” (i.e., no support for an appreciable effect size).”
BackgroundMethodInfluentialSchaeffer, M., & Kas, J. (2023) · International Migration Review · doi:10.1177/01979183231170809 · Cited by 102
The impact of sample size on the reproducibility of voxel-based lesion-deficit mappings
Lorca‐Puls, D. L., Gajardo‐Vidal, A., White, J., Seghier, M. L., Leff, A. P., Green, D. W., Crinion, J., Ludersdorfer, P., Hope, T. M. H., Bowman, H., et al. (2018) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2018.03.014 · Cited by 91
Investigating resting-state functional connectivity in the cervical spinal cord at 3 T
Eippert, F., Kong, Y., Winkler, A. M., Andersson, J., Finsterbusch, J., Büchel, C., Brooks, J. C. W., & Tracey, I. (2016) · NeuroImage · doi:10.1016/j.neuroimage.2016.12.072 · Cited by 84
Where Are the Self-Correcting Mechanisms in Science?
Vazire, S., & Holcombe, A. O. (2022) · Review of General Psychology · doi:10.1177/10892680211033912 · Cited by 81
“Techniques to detect publication biases and other sources of distortion in a field are evolving rapidly (Simonsohn et al., 2014; Andrews & Kasy, 2019).”
BackgroundBottema‐Beutel, K., Kim, S. Y., & Crowley, S. (2019) · Autism Research · doi:10.1002/aur.2055 · Cited by 79
“In our study, we used the p-curve app [Simonsohn et al., 2014], an online calculator for generating p-curve plots and testing the distribution of P-values against what is expected for true effects.”
BackgroundMethodA comprehensive meta-analysis of money priming.
Lodder, P., Ong, H. H., Grasman, R. P. P. P., & Wicherts, J. M. (2019) · Journal of Experimental Psychology General · doi:10.1037/xge0000570 · Cited by 92
Metaphorical framing in political discourse through words vs. concepts: a meta-analysis
Brugman, B. C., Burgers, C. F., & Vis, B. (2019) · Language and Cognition · doi:10.1017/langcog.2019.5 · Cited by 88
“The time frame was set on 2000–2017 to update the meta-analysis by Sopory and Dillard (2002), who included studies that were published between 1983 and 2000.”
MethodPreregistration Is Neither Sufficient nor Necessary for Good Science
Pham, M. T., & Oh, T. T. (2020) · Journal of Consumer Psychology · doi:10.1002/jcpy.1209 · Cited by 104
Reducing Stigma and Punitive Attitudes Toward Pedophiles Through Narrative Humanization
Harper, C. A., Bartels, R. M., & Hogue, T. E. (2016) · Sexual Abuse · doi:10.1177/1079063216681561 · Cited by 95
Traut, N., Beggiato, A., Bourgeron, T., Delorme, R., Rondi‐Reig, L., Paradis, A., & Toro, R. (2017) · Biological Psychiatry · doi:10.1016/j.biopsych.2017.09.029 · Cited by 91
“Finally, we plotted the p curve, which shows the distribution of significant p values (18).”
Do Parenting Practices and Child Disclosure Predict Parental Knowledge? A Meta-Analysis
Liu, D., Chen, D., & Brown, B. B. (2019) · Journal of Youth and Adolescence · doi:10.1007/s10964-019-01154-4 · Cited by 95
How Accurate Are Our Students? A Meta-analytic Systematic Review on Self-assessment Scoring Accuracy
León, S. P., Panadero, E., & García‐Martínez, I. (2023) · Educational Psychology Review · doi:10.1007/s10648-023-09819-0 · Cited by 123
Paying more when paying for others.
Jung, M. H., Nelson, L. D., Gneezy, A., & Gneezy, U. (2014) · Journal of Personality and Social Psychology · doi:10.1037/a0037345 · Cited by 98
Self-citationMorales Parra, T. (2021) · LA Referencia (Red Federada de Repositorios Institucionales de Publicaciones Científicas) · Cited by 82
Bishop, D. V. M., & Thompson, P. A. (2016) · PeerJ · doi:10.7717/peerj.1715 · Cited by 90
“As shown by Simonsohn et al. 101 (2014), the degree of right skew will be proportional to sample size (N), as we have more power”
BackgroundMethodInfluentialSijtsma, K. (2015) · Psychometrika · doi:10.1007/s11336-015-9446-0 · Cited by 96
Kossmeier, M., Tran, U. S., & Voracek, M. (2020) · BMC Medical Research Methodology · doi:10.1186/s12874-020-0911-9 · Cited by 89
A Primer on Bayesian Model-Averaged Meta-Analysis
Gronau, Q. F., Heck, D. W., Berkhout, S. W., Haaf, J. M., & Wagenmakers, E. (2021) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459211031256 · Cited by 99
A systematic review and meta-analysis of the evidence for unaware fear conditioning
Mertens, G., & Engelhard, I. M. (2019) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2019.11.012 · Cited by 96
Sun, L., Hu, L., & Zhou, D. (2021) · Journal of Computer Assisted Learning · doi:10.1111/jcal.12545 · Cited by 98
Blake, A., Luu, V., Petrenko, O., Gardner, W., Moergen, K. J. N., & Ezerins, M. E. (2022) · Leadership Quarterly · doi:10.1016/j.leaqua.2021.101593 · Cited by 64
Lanier, M., Waddell, T. F., Elson, M., Tamul, D. J., Ivory, J., & Przybylski, A. K. (2019) · Computers in Human Behavior · doi:10.1016/j.chb.2019.06.015 · Cited by 83
“As discussed previously, inclusion rules were created based on recommendations by (Simonsohn et al., 2014a).”
BackgroundMethodResultInfluentialEstimating Population Mean Power Under Conditions of Heterogeneity and Selection for Significance
Brunner, J., & Schimmack, U. (2020) · Meta-Psychology · doi:10.15626/mp.2018.874 · Cited by 76
“1 .484 .491 .496 .497 .499 (.175) (.139) (.102) (.073) (.046) P-uniform .473 .485 .493 .496 .499 (.170) (.133) (.097) (.070) (.044) ML-model .479 .489 .495 .497 .499 (.166) (.130) (.095) (.068) (.045) Z-curve .513 .516 .513 .508 .502 (.151) (.121) (.091) (.068) (.045) Population Mean Power = 0.”
BackgroundMethodResultInfluentialPre-registration: Weighing costs and benefits for researchers
Logg, J. M., & Dorison, C. A. (2021) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2021.05.006 · Cited by 91
Neuropsychological functioning in adult anorexia nervosa: A meta-analysis
Stedal, K., Broomfield, C., Hay, P., Touyz, S. W., & Scherer, R. (2021) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2021.08.021 · Cited by 80
“Fourth, we plotted the p-curves underlying the effects and examined their skewness (Simonsohn et al., 2014).”
MethodSupernatural and secular monitors promote human cooperation only if they remind of punishment
Yılmaz, O., & Bahçekapılı, H. G. (2015) · Evolution and Human Behavior · doi:10.1016/j.evolhumbehav.2015.09.005 · Cited by 83
Reparation or reactance? The influence of guilt on reaction to persuasive communication
Graton, A., Ric, F., & Gonzalez, E. (2015) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2015.09.016 · Cited by 96
Can Results-Free Review Reduce Publication Bias? The Results and Implications of a Pilot Study
Findley, M. G., Jensen, N. M., Malesky, E. J., & Pepinsky, T. B. (2016) · Comparative Political Studies · doi:10.1177/0010414016655539 · Cited by 85
The influence of fear on risk taking: a meta-analysis
Wake, S., Wormwood, J. B., & Satpute, A. B. (2020) · Cognition & Emotion · doi:10.1080/02699931.2020.1731428 · Cited by 94
Work Engagement and Patient Quality of Care: A Meta-Analysis and Systematic Review
Wee, K. Z., & Lai, A. Y. (2021) · Medical Care Research and Review · doi:10.1177/10775587211030388 · Cited by 97
“Recent research has also demonstrated that p-hacking (Simonsohn et al., 2014), in which researchers manipulate data to turn a nonsignificant finding into a significant one, could be a bigger source of publication bias.”
BackgroundBem, D., Tressoldi, P. E., Rabeyron, T., & Duggan, M. (2016) · F1000Research · doi:10.12688/f1000research.7177.2 · Cited by 74
“If a genuine non-zero effect exists, however, then p-curve’s expected distribution will be right-skewed: We expect to observe more low significant p values (p .01) than high significant p values (.04 p .05) (Simonsohn et al., 2014b, pp. 666–667)...”
BackgroundMethodInfluential“All that's best of dark and bright”: Day and night perceptions of Hong Kong cityscape
Huang, W., & Wang, P. (2017) · Tourism Management · doi:10.1016/j.tourman.2017.12.006 · Cited by 79
A Meta-analysis of University STEM Summer Bridge Program Effectiveness
Bradford, B. C., Beier, M. E., & Oswald, F. L. (2021) · CBE—Life Sciences Education · doi:10.1187/cbe.20-03-0046 · Cited by 73
Heterogeneity in effect size estimates
Holzmeister, F., Johannesson, M., BOHM, R. M., Dreber, A., Huber, J., & Kirchler, M. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2403490121 · Cited by 103
“While p - hacking is often thought of as marginally affecting results around the significance thresholds (7–9), the extent of observed analytical heterogeneity suggests that there is potential for much larger systematic bias in published effect sizes.”
Delay and Death-Thought Accessibility: A Meta-Analysis
Steinman, C. T., & Updegraff, J. A. (2015) · Personality and Social Psychology Bulletin · doi:10.1177/0146167215607843 · Cited by 72
Do subtle reminders of money change people’s political views?
Rohrer, D., Pashler, H., & Harris, C. R. (2015) · Journal of Experimental Psychology General · doi:10.1037/xge0000058 · Cited by 98
What Drives False Memories in Psychopathology? A Case for Associative Activation
Otgaar, H., Muris, P., Howe, M. L., & Merckelbach, H. L. G. J. (2017) · Clinical Psychological Science · doi:10.1177/2167702617724424 · Cited by 106
Metadata
- Authors
- Simonsohn, U., Nelson, L. D., & Simmons, J. P.
- Affiliations
- University of Pennsylvania (US)
- Venue
- Journal of Experimental Psychology: General
- Published
- 2014-04
- Volume
- 143
- Issue
- 2
- Pages
- 534-547
- ISSN
- 0096-3445 (ISSN-L)
- 1939-2222
- OpenAlex
- W3124690569
- Cited by
- 1,500
- References
- 49
- Citations by year
- 2026: 75 · 2025: 96 · 2024: 110 · 2023: 121 · 2022: 146 · 2021: 157 · 2020: 158 · 2019: 154 · 2018: 143 · 2017: 113 · 2016: 103 · 2015: 80 · 2014: 35
- FWCI
- 82.62
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Experimental Psychology General (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Keywords
- p-curve, publication bias, selective reporting, p-hacking, file drawer problem, statistical significance, evidentiary value
- MeSH terms
- Data Interpretation, Statistical, Humans, Psychology, Experimental, Statistics as Topic, Reproducibility of Results, Models, Statistical, Publication Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.