P-curve: A key to the file-drawer.
The DOI matches its Crossref record; 49 of the 49 references it cites were found in a registry or an index.
Abstract
Because scientists tend to report only studies (publication bias) or analyses (p-hacking) that "work," readers must ask, "Are these effects true, or do they merely reflect selective reporting?" We introduce p-curve as a way to answer this question. P-curve is the distribution of statistically significant p values for a set of studies (ps < .05). Because only true effects are expected to generate right-skewed p-curves-containing more low (.01s) than high (.04s) significant p values--only right-skewed p--curves are diagnostic of evidential value. By telling us whether we can rule out selective reporting as the sole explanation for a set of findings, p-curve offers a solution to the age-old inferential problems caused by file-drawers of failed studies and analyses.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Incorrect p-values could affect the outcome of tests that analyze the distribution of p-values, such as the p-curve (Simonsohn, Nelson, & Simmons, 2014) and p-uniform (Van Assen, Van Aert, & Wicherts, 2014).”
From The prevalence of statistical reporting errors in psychology (1985–2013) (2016) · cites this paper as Simonsohn, Nelson, & Simmons, 2014
In the citing paper
“Even if the first REM episode occurs about 70 minutes after falling asleep (33), some subjects may present a REM EEG and occulomotor pattern a way farster.The REM sleep can actually occur pretty rapidly in some individuals according to the MD (sleep specialist) who co-supervised this study.Furthermore, the MSLT has specialy been built for this puporse (see32)”
From Is there a Publication Bias in Behavioural Intranasal Oxytocin Research on Humans? Opening the File Drawer of One Laboratory (2016) · cites this paper as [33]
In the citing paper
“After excluding studies which assessed maltreatment as part of broader measures of victimization (k=3) (25, 27, 29) or ACEs (k=5) (30, 34, 39, 44, 49), or focused on institutional neglect (k=3) (15, 45, 46), the meta-analytic effect size was also similar to the original estimate (Cohen’s d=0.34, 95% CI=0.26-0.42).”
From Childhood Maltreatment and Mental Health Problems: A Systematic Review and Meta-Analysis of Quasi-Experimental Studies (2023) · cites this paper as [25]
In the citing paper
“Trim‐and‐fill procedures do suffer from some well‐known limitations that could result in the underestimation or overestimation of publication bias (Rothstein, 2008; Simonsohn, Nelson, & Simmons, 2014).”
From Hot spots policing of small geographic areas effects on crime (2019) · cites this paper as Simonsohn, Nelson, & Simmons, 2014
In the citing paper
“Applying selection modeling approaches like p-uniform or 3PSM to the simple effects may therefore substantially overestimate the true effect size.”
From Do Salient Social Norms Moderate Mortality Salience Effects? A (Challenging) Meta-Analysis of Terror Management Studies (2023) · cites this paper as Simonsohn et al., 2014
Citations
Cites
Statistical power analysis for the behavioral sciences
(1990) · Computers, Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,269
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722
(1963) · Medical Entomology and Zoology · Cited by 15,085
Duval, S., & Tweedie, R. (2000) · Biometrics · doi:10.1111/j.0006-341x.2000.00455.x · Cited by 14,099
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068
The case for motivated reasoning.
Kunda, Z. (1990) · Psychological Bulletin · doi:10.1037/0033-2909.108.3.480 · Cited by 9,022
Cohen, J. (1992) · Psychological Bulletin · doi:10.1037//0033-2909.112.1.155 · Cited by 8,771
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
Self-citationTesting Statistical Hypotheses
(2021) · Wiley Series in Probability and Statistics · doi:10.1002/9781119243830.ch12 · Cited by 5,106
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 1×
“1 Trying multiple analyses to obtain statistical significance has received many names, including: bias (Ioannidis, 2005), significance chasing (Ioannidis & Trikalinos, 2007), data snooping (White, 2003), fiddling (Bross, 1971), publication bias in situ (Phillips, 2004) and specification searching (Gerber & Malhotra, 2…”
Cited by
Items 601–700 of 1,005
A Quantitative Analysis of Latino Acculturation and Alcohol Use: Myth Versus Reality
Álvarez, M. J., Frietze, G. A., Ramos, C., Field, C. A., & Zárate, M. A. (2017) · Alcoholism Clinical and Experimental Research · doi:10.1111/acer.13420 · Cited by 34
Sweetman, J., Maio, G. R., Spears, R., Manstead, A. S. R., & Livingstone, A. (2019) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2019.01.001 · Cited by 29
Action Effects on Visual Perception of Distances: A Multilevel Bayesian Meta-Analysis
Molto, L., Nalborczyk, L., Palluel-Germain, R., & Morgado, N. (2020) · Psychological Science · doi:10.1177/0956797619900336 · Cited by 28
Community-Augmented Meta-Analyses (CAMAs) in Psychology: Potentials and Current Systems
Burgard, T., Bošnjak, M., & Studtrucker, R. (2021) · Zeitschrift für Psychologie · doi:10.1027/2151-2604/a000431 · Cited by 24
Air Pollution and Mortality in the Medicare Population
Young, S. S. (2018) · JAMA · doi:10.1001/jama.2018.3927 · Cited by 29
Psychological and Hierarchical Closeness as Opposing Factors in Whistleblowing: A Meta-Analysis
Batolas, D., Perkovic, S., & Mitkidis, P. (2022) · Journal of Business and Psychology · doi:10.1007/s10869-022-09849-5 · Cited by 35
How Does Time Pressure Influence Risk Preferences? Answers from a Meta-Analysis
Belli, A., Carrillat, F. A., Zlatevska, N., & Cowley, E. (2023) · Journal of Consumer Research · doi:10.1093/jcr/ucad053 · Cited by 30
Are many sex/gender differences really power differences?
Galinsky, A. D., Turek, A., Agarwal, G., Anicich, E. M., Rucker, D. D., Bowles, H. R., Liberman, N., Levin, C., & Magee, J. C. (2024) · PNAS Nexus · doi:10.1093/pnasnexus/pgae025 · Cited by 30
“We began by identifying outcomes significantly affected by experimental manipulations of power, consistent with a P -curve approach (50, 51).”
MethodThe mind in the object—Psychological valuation of materialized human expression.
Kreuzbauer, R., King, D., & Basu, S. (2015) · Journal of Experimental Psychology General · doi:10.1037/xge0000080 · Cited by 27
Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2019) · PLoS ONE · doi:10.1371/journal.pone.0213454 · Cited by 27
“Simulations are not needed here as one can instead rely on the noncentral t-distribution (see e.g., Supplement 1 in Simonsohn et al., 2014b), but we did our best to follow the procedures by Bruns and Ioannidis (2016).”
MethodSelf-citationLinder, C., & Farahbakhsh, S. (2020) · Business Ethics Quarterly · doi:10.1017/beq.2019.52 · Cited by 26
Calderon, S., Giolla, E. M., Ask, K., & Granhag, P. A. (2020) · Journal of Personality and Social Psychology · doi:10.1037/pspa0000214 · Cited by 26
“…other hand, is indicative of an effect with low replicability because such a distribution should only occur if significant p values are selectively reported or if there have been systematic attempts by researchers to bring p values below the .05 threshold through p-hacking (Simonsohn et al., 2014).”
BackgroundMuthard, C., & Gilbertson, R. J. (2016) · Psi Chi Journal of Psychological Research · doi:10.24839/2164-8204.jn21.1.16 · Cited by 36
Glöckner, A., Fiedler, S., & Renkewitz, F. (2018) · Psychologische Rundschau · doi:10.1026/0033-3042/a000384 · Cited by 28
Lynn, M. (2017) · International Journal of Hospitality Management · doi:10.1016/j.ijhm.2017.10.004 · Cited by 23
How to Enhance the Power to Detect Brain–Behavior Correlations With Limited Resources
de Haas, B. (2018) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2018.00421 · Cited by 26
“Effect size estimates are inflated in both cases, because they are based on significant results only [the simulation assumes a ‘ file drawer problem’ (Simonsohn et al., 2014)].”
BackgroundDo Monetary Incentives Undermine Performance on Intrinsically Enjoyable Tasks? A Field Test
Esteves‐Sorenson, C., & Broce, R. (2020) · The Review of Economics and Statistics · doi:10.1162/rest_a_00947 · Cited by 38
Changing the logic of replication: A case from infant studies
Margoni, F., & Shepperd, M. (2020) · Infant Behavior and Development · doi:10.1016/j.infbeh.2020.101483 · Cited by 22
Embracing Open Science and Transparency in Health Psychology
Hagger, M. S. (2019) · Health Psychology Review · doi:10.1080/17437199.2019.1605614 · Cited by 22
Field Experimentation and the Study of Law and Policy
GREEN, D. P., & Thorley, D. (2014) · Annual Review of Law and Social Science · doi:10.1146/annurev-lawsocsci-110413-030936 · Cited by 23
Pitfalls of significance testing and $p$-value variability: An econometrics perspective
Hirschauer, N., Grüner, S., Mußhoff, O., & Becker, C. (2018) · Statistics Surveys · doi:10.1214/18-ss122 · Cited by 23
“Instead of test scores, p-curve analysis looks directly at the distribution of p-values (Head et al. 2015; Simonsohn et al. 2014).”
BackgroundThe Replication Crisis and Open Science in Psychology: Methodological Challenges and Developments
Renkewitz, F., & Heene, M. (2019) · Zeitschrift für Psychologie · doi:10.1027/2151-2604/a000389 · Cited by 30
Zhang, Y., Chen, S., Wu, H., & Guo, C. (2021) · Mindfulness · doi:10.1007/s12671-021-01775-6 · Cited by 31
“Third, we used p-curve analysis (including the right-skewness and flatness tests) to assess publication bias and estimate the true effect size (Simonsohn et al., 2014).”
MethodA systematic review and meta-analysis of vineyard techniques used to delay ripening
Previtali, P., Giorgini, F., Mullen, R. S., Dokoozlian, N., Wilkinson, K. L., & Ford, C. M. (2022) · Horticulture Research · doi:10.1093/hr/uhac118 · Cited by 22
“The p-curve of studies on delayed ripening is reported in Figure 2A (blue line) and compared to a null effect curve (red dotted line), in which p-values are uniform, and a curve suggesting an adequate evidential value (green dotted line) as reported by the authors of this technique [25].”
BackgroundMethodResultInfluentialSolana, P., & Santiago, J. (2022) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2022.104834 · Cited by 28
P ‐hacking in meta‐analyses: A formalization and new meta‐analytic methods
Mathur, M. B. (2024) · Research Synthesis Methods · doi:10.1002/jrsm.1701 · Cited by 32
“These methods analyze only significant p -values and assess whether these p -values are left-skewed, which is taken as evidence of SWS. 17,26 These methods are special cases of selection models in which it is implicitly assumed that there is no heterogeneity across studies and that a study's…”
BackgroundMethodResultInfluentialDickes, A., Kemmis-Riggs, J., & McAloon, J. (2017) · Clinical Child and Family Psychology Review · doi:10.1007/s10567-017-0248-z · Cited by 28
Fact or fiction: reducing the proportion and impact of false positives
Stahl, D. R., & Pickles, A. R. (2017) · Psychological Medicine · doi:10.1017/s003329171700294x · Cited by 23
Peer-Review Guidelines Promoting Replicability and Transparency in Psychological Science
Davis, W. E., Giner‐Sorolla, R., Lindsay, D. S., Lougheed, J. P., Makel, M. C., Meier, M. E., Sun, J., Vaughn, L. A., & Zelenski, J. M. (2018) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245918806489 · Cited by 24
Evaluation of a meta-analysis of air quality and heart attacks, a case study
Young, S. S., & Kindzierski, W. B. (2019) · Critical Reviews in Toxicology · doi:10.1080/10408444.2019.1576587 · Cited by 27
“Analysis manipulation, intentional or unintentional, can give rise to small p-values so the contention that replication implies a correct claim is questioned (Simonsohn et al. 2014).”
BackgroundMethodResultInfluentialSanjuan, P. M., Pearson, M. R., Fokas, K. F., & Leeman, L. M. (2019) · Psychology of Addictive Behaviors · doi:10.1037/adb0000543 · Cited by 28
P-Curve and Effect Size: Correcting for Publication Bias Using Only Significant ResultsPreprint
Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2014) · SSRN Electronic Journal · doi:10.2139/ssrn.2377290 · Cited by 22
Self-citationExcess success for three related papers on racial bias
Francis, G. (2015) · Frontiers in Psychology · doi:10.3389/fpsyg.2015.00512 · Cited by 34
“…under conditions of publication bias (van Assen et al., 2014) and for identifying whether biased experiment sets contain some evidential value (Simonsohn et al., 2014), these methods do not apply to the work of Eberhardt et al. (2004) because, as described below, the experiments measured…”
MethodCheng, C., & Ying, W. (2023) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2023.1097598 · Cited by 53
“Finally, p-curve analysis was used to assess the robustness of the present meta-analytic findings by estimating the true effect size (82).”
MethodDavid, L., Biwer, F., Baars, M., Wijnia, L., PAAS, F. G. W. C., & de Bruin, A. B. H. (2024) · Educational Psychology Review · doi:10.1007/s10648-024-09903-z · Cited by 35
“We investigated this further by conducting a p -curve analysis using the p -curve app (Version 4.06; Simonsohn et al., 2014).”
InfluentialCosta, J. M., Moro, S., Miranda, G. L., & Arnold, T. (2020) · Research in Learning Technology · doi:10.25304/rlt.v28.2396 · Cited by 21
“The p-curve method provides a meta-analysis method for determining the likelihood that an effect can be explained as a result of publication bias or p-hacking (Simonsohn, Nelson, and Simmons 2014a).”
BackgroundMethodEnhancing and Accelerating Social Science Via Automation: Challenges and Opportunities
Yarkoni, T., Eckles, D., Heathers, J. A. J., Levenstein, M. C., Smaldino, P. E., & Lane, J. I. (2021) · Harvard Data Science Review · doi:10.1162/99608f92.df2262f5 · Cited by 26
Heterogeneity estimates in a biased world
Hönekopp, J., & Linden, A. H. (2022) · PLoS ONE · doi:10.1371/journal.pone.0262809 · Cited by 28
“Such practices have become known as p-hacking [22].”
BackgroundWang, Y., Wilson, A. E., & Liu, N. (2022) · Frontiers in Physiology · doi:10.3389/fphys.2022.818531 · Cited by 29
“, findings that statistics suggest are meaningful when they are not (Simonsohn et al., 2014)].”
BackgroundTybur, J. M., Fan, L., Jones, B. C., Holzleitner, I. J., Lee, A. J., & DeBruine, L. M. (2022) · Evolution and Human Behavior · doi:10.1016/j.evolhumbehav.2022.01.003 · Cited by 23
Crowe, E., Staiger, P. K., Bowe, S. J., Rehm, I. C., Moulding, R., Herrick, C. A., & Hallford, D. J. (2023) · Journal of Affective Disorders · doi:10.1016/j.jad.2023.11.010 · Cited by 34
P-Hacking, Data Type and Data-Sharing Policy
Brodeur, A., Cook, N., & Neisser, C. (2022) · The Economic Journal · doi:10.1093/ej/uead104 · Cited by 41
Debunking the Myth of Value-Neutral Virginity: Toward Truth in Scientific Advertising
Mandel, D. R., & Tetlock, P. E. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2016.00451 · Cited by 22
“…such as selectively reporting results that are likely to entice peer reviewers and editors or torturing the data until a statistically significant finding gives itself up (Simonsohn et al., 2014), we should see a rise in their prevalence, signaling a shift toward cargo-cult science (Feynman, 1974).”
BackgroundArcher, D. P., McCann, S., Walker, A. M., Premji, Z., Rogan, K. J., Hutton, M. J. H., & Gray, L. J. (2018) · British Journal of Anaesthesia · doi:10.1016/j.bja.2018.07.024 · Cited by 21
Adam‐Troian, J., Bonetto, É., & Arciszewski, T. (2021) · Social Psychological and Personality Science · doi:10.1177/1948550620987672 · Cited by 22
“Subsequent P-curve analyses (Simonsohn et al., 2014; conducted online at http://www.p-curve.com/app4/) confirmed strong evidential value for the effect of plural FPP on both protests, Z ¼ 5.4, p < .001 and participants counts Z¼ 6.3, p < .001 (Figure 4).”
BackgroundInfluentialTaylor, A. E., & Munafo, M. R. (2016) · BMC Psychology · doi:10.1186/s40359-016-0129-0 · Cited by 27
“P-curve analysis [11] uses the distribution of P-values beneath the conventional cut off for determining statistical significance (P < 0.”
BackgroundMethodResultInfluentialDistances on hills look farther than distances on flat ground: Evidence from converging measures
Tenhundfeld, N. L., & Witt, J. K. (2017) · Attention Perception & Psychophysics · doi:10.3758/s13414-017-1305-x · Cited by 21
Manipulating belief in free will and its downstream consequences: A meta-analysisPreprint
Genschow, O., Cracco, E., Schneider, J., Protzko, J., Wisniewski, D., Braß, M., & Schooler, J. W. (2021) · doi:10.31234/osf.io/quwgr · Cited by 21
Pham, M. T., & Oh, T. T. (2020) · Journal of Consumer Psychology · doi:10.1002/jcpy.1213 · Cited by 20
“We believe that, somewhat paradoxically, it is the very focus on the control of type I error with a nominal threshold of p < .05 that has contributed to the problems of p-hacking that SNS have pointed out in their in fl uential work (e.g., Simmons et al., 2011; Simonsohn et al., 2014).”
Murdoch, E. M., Lines, R. L. J., Crane, M. F., Ntoumanis, N., Brade, C. J., Quested, E., Ayers, J., & Gucciardi, D. F. (2021) · International Review of Sport and Exercise Psychology · doi:10.1080/1750984x.2021.1977974 · Cited by 22
“Finally, we conducted a p -curve analysis to assess the evidential value via a distribution of statistically signi fi cant fi ndings only; a left-skewed curve indicates possible bias and a right-skewed supports evidential value (Simonsohn et al., 2014).”
MethodNaaman, K., Grant, S., Kianersi, S., Supplee, L. H., Henschel, B., & Mayo-Wilson, E. (2023) · Royal Society Open Science · doi:10.1098/rsos.221093 · Cited by 30
Prototypes and same-gender bias in perceptions of hiring discrimination
Carlsson, R., & Sinclair, S. (2017) · The Journal of Social Psychology · doi:10.1080/00224545.2017.1341374 · Cited by 30
Unexpected Gains: Being Overweight Buffers Asian Americans From Prejudice Against Foreigners
Handron, C., Kirby, T. A., Wang, J., Matskewich, H. E., & Cheryan, S. (2017) · Psychological Science · doi:10.1177/0956797617720912 · Cited by 25
Ulrich, R., Miller, J. O., & Erdfelder, E. (2018) · Zeitschrift für Psychologie · doi:10.1027/2151-2604/a000319 · Cited by 21
“…Only Significant Results Computer simulations have revealed that effect sizes are strongly overestimated when the estimates are based exclusively on significant results (Anderson, Kelley, & Maxwell, 2017; Lane & Dunlap, 1978; Simonsohn et al., 2014b; van Assen, van Aert, Nuijten, & Wicherts, 2014).”
BackgroundInfluentialMeasuring the Issue Content of Supreme Court Opinions
Rice, D. R. (2019) · Journal of Law and Courts · doi:10.1086/701130 · Cited by 20
Fendel, J. C., Bürkle, J. J., & Göritz, A. S. (2019) · BMJ Open · doi:10.1136/bmjopen-2019-032295 · Cited by 26
Kids These Days! Increasing delay of gratification ability over the past 50 years in children
Protzko, J. (2020) · Intelligence · doi:10.1016/j.intell.2020.101451 · Cited by 27
Schmalz, X., Robidoux, S., Castles, A. E., Coltheart, M., & Marinus, E. (2017) · Collabra Psychology · doi:10.1525/collabra.72 · Cited by 24
Willis, M., Birthrong, A., King, J. S., Nelson‐Gray, R. O., & Latzman, R. D. (2017) · Personality and Individual Differences · doi:10.1016/j.paid.2017.06.015 · Cited by 24
Goubran, M., Farajzadeh, A., Lahart, I. M., Bilodeau, M., & Boisgontier, M. P. (2025) · Physical Therapy · doi:10.1093/ptj/pzaf050 · Cited by 46
“P-curve analysis 61 was conducted to assess whether the distribution of the statistically significant results was consistent with what would be expected if only true effects were present.”
The role of time perception in temporal binding: Impaired temporal resolution in causal sequences
Fereday, R., Buehner, M. J., & Rushton, S. K. (2019) · Cognition · doi:10.1016/j.cognition.2019.06.017 · Cited by 26
Inter-study and intra-study replications in leading marketing journals: a longitudinal analysis
Kwon, E. S., Yan, S., Lee, J. S., & Reid, L. N. (2017) · European Journal of Marketing · doi:10.1108/ejm-07-2015-0450 · Cited by 20
Privacy Protective Research: Facilitating Ethically Responsible Access to Administrative Data
Goroff, D. L., Polonetsky, J., & Tene, O. (2017) · The Annals of the American Academy of Political and Social Science · doi:10.1177/0002716217742605 · Cited by 20
Farrar, B. G., Altschul, D., Fischer, J., van der Mescht, J., Placì, S., Troisi, C. A., Vernouillet, A., Clayton, N. S., & Ostojić, L. (2020) · Animal Behavior and Cognition · doi:10.26451/abc.07.03.09.2020 · Cited by 17
“P-distribution analyses, for example, are most valid when used to assess a family of research testing the same or very similar hypotheses (Simonsohn et al., 2014), and publication bias may be best viewed at the level of the individual research group, or journal.”
BackgroundBrough, A. R., Donnelly, G. E., Griskevicius, V., Markowitz, E. M., Raimi, K. T., Reeck, C., Trudel, R., Waldman, K. B., Winterich, K. P., & Wolske, K. S. (2020) · Social Marketing Quarterly · doi:10.1177/1524500420970129 · Cited by 23
“…of published interventions demonstrate success while a much larger proportion of failed interventions remain in the file drawer (Rosenthal, 1979; Simonsohn et al., 2014), we hope this typology increases awareness of how interventions may fail and leads to designs that are better equipped to…”
BackgroundNavas, J. F., Verdejo‐García, A., & Vadillo, M. A. (2021) · Obesity Reviews · doi:10.1111/obr.13338 · Cited by 19
“Unlike standard meta-analytic methods, p-curve analysis does not rely on effect sizes but on the crucial statistic of each experiment.(28) These crucial statistics are often different from the information used to com-”
MethodA Meta-analysis of the relationship between growth mindset and grit
Lam, K. K. L., & Zhou, M. (2025) · Acta Psychologica · doi:10.1016/j.actpsy.2025.104872 · Cited by 34
Bestiary of Questionable Research Practices in Psychology
Nagy, T., Hergert, J., Elsherif, M. M., Wallrich, L., Schmidt, K., Waltzer, T., Payne, J. W., Gjoneska, B., Seetahul, Y., Wang, Y. A., et al. (2025) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459251348431 · Cited by 37
Substantive Importance and the Veil of Statistical Significance
McCaskey, K., & Rainey, C. (2015) · Statistics Politics and Policy · doi:10.1515/spp-2015-0001 · Cited by 21
Trail and Ultrarunning: The Impact of Distance, Nature, and Personality on Flow and Well-Being
Martinez, C. T., & Scott, C. (2016) · Psi Chi Journal of Psychological Research · doi:10.24839/2164-8204.jn21.1.6 · Cited by 29
Schütt, M. (2021) · Environmental and Resource Economics · doi:10.1007/s10640-021-00536-2 · Cited by 27
Yip, J. A., & Schweitzer, M. E. (2022) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2021.104109 · Cited by 21
Reckoning With Our Crisis: An Agenda for the Field of Social and Personality Psychology
Schiavone, S. R., & Vazire, S. (2022) · Perspectives on Psychological Science · doi:10.1177/17456916221101060 · Cited by 25
Effects of Neurofeedback training on performance in laboratory tasks: A systematic review
Chiasson, P., Boylan, M. R., Elhamiasl, M., Pruitt, J., Ranjan, S., Riels, K. M., Sahoo, A. K., Mirifar, A., & Keil, A. (2022) · International Journal of Psychophysiology · doi:10.1016/j.ijpsycho.2023.04.005 · Cited by 20
When Science Replaces Religion: Science as a Secular Authority Bolsters Moral Sensitivity
Yılmaz, O., & Bahçekapılı, H. G. (2015) · PLoS ONE · doi:10.1371/journal.pone.0137499 · Cited by 23
“It is also possible that the original findings by Ma-Kellams and Blascovich are spurious, a plausible interpretation given their small sample size and barely significant results (see [14]).”
ResultOn the significance of statistically insignificant results in consumer behavior experiments
Peterson, R. A., & Umesh, U. N. (2017) · Journal of the Academy of Marketing Science · doi:10.1007/s11747-017-0528-7 · Cited by 22
“In p-hacking researchers manipulate factors in a research study repeatedly, or repeat an experiment until significantly positive results are obtained (e.g., Simonsohn et al. 2014).”
BackgroundReactivation-Dependent Amnesia for Contextual Fear Memories: Evidence for Publication Bias
Schroyens, N., Sigwald, E. L., Noortgate, W. V. D., Beckers, T., & Luyten, L. (2020) · eNeuro · doi:10.1523/eneuro.0108-20.2020 · Cited by 21
“In addition, effect sizes are most densely plotted in the gray area at the border of statistical significance, which might suggest a biased distribution of effect sizes (Simonsohn et al., 2014).”
BackgroundDeng, J., Zhou, F., Wong, C. Y., Huang, E., & Zheng, E. (2020) · Journal of Clinical Apheresis · doi:10.1002/jca.21790 · Cited by 18
“”(44-46) We identified the presence of publication bias by observing the presence of a left-skew (indicative of “p-hacking”) in the p-curve.(46)”
BackgroundSimonsohn, U., Simmons, J. P., & Nelson, L. D. (2013) · SSRN Electronic Journal · doi:10.2139/ssrn.2351926 · Cited by 23
Self-citationIdentification of and Correction for Publication Bias
Andrews, I., & Kasy, M. (2017) · National Bureau of Economic Research · doi:10.3386/w23298 · Cited by 33
What’s in a looking time preference?Preprint
Bergmann, C., Rabagliati, H., & Tsuji, S. (2019) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/6u453 · Cited by 21
Mu, Y., Rubin, E., & Zou, E. (2021) · National Bureau of Economic Research · doi:10.3386/w28735 · Cited by 28
When cutting edge meets silver tongue: Understanding the word-of-machine effect on travel decisions
Sun, D., Wong, I. A., Xiong, X., & Li, S. (2025) · Tourism Management · doi:10.1016/j.tourman.2025.105271 · Cited by 23
Davis, J. D., Sanchez‐Ramos, L., McKinney, J. A., Lin, L., & Kaunitz, A. M. (2023) · American Journal of Obstetrics and Gynecology · doi:10.1016/j.ajog.2022.07.047 · Cited by 17
Decades of Trivers-Willard research on humans: What conclusions can be drawn?
Thouzeau, V., Bollée, J., Cristià, A., & Chevallier, C. (2022) · Evolution and Human Behavior · doi:10.1016/j.evolhumbehav.2023.03.005 · Cited by 18
Theory and methods of the multiverse: an application for panel-based models
Cantone, G. G., & Tomaselli, V. (2023) · Quality & Quantity · doi:10.1007/s11135-023-01698-5 · Cited by 21
“(Simonsohn et al. 2014).”
InfluentialGama, H. C., Amorós, M. A., de Araújo, M. A., Sha, C. M., Vieira, M. P. S., Torres, R. G. D., Souza, G. F., Junkes, J. A., Dokholyan, N. V., Gitaí, D. L. G., et al. (2024) · Non-coding RNA Research · doi:10.1016/j.ncrna.2024.02.006 · Cited by 20
“…meta-analysis, with the PyMARE 0.0.4 [57] as the regression engine and computation of Cochran ’ s Q test and the I 2 statistic, pypcurve 0.1.0 [ 54] for the p-curve method, Pandas 2.0.1 [58] to organize our data, and the myforestplot package (https:// github.com/toshiakiasakura/myforestplot)…”
MethodInfluentialEntertaining without endorsing: The case for the scientific investigation of anomalous cognition.
Schooler, J. W., Baumgart, S., & Franklin, M. S. (2018) · Psychology of Consciousness Theory Research and Practice · doi:10.1037/cns0000151 · Cited by 23
Statistical methods for replicability assessment
Hung, K. E., & Fithian, W. (2020) · The Annals of Applied Statistics · doi:10.1214/20-aoas1336 · Cited by 26
“In particular, p-hacking — the iterative tweaking of an analysis until the p-value drops below the researcher’s desired significance level α0 — is commonly suspected to produce a pileup of p-values just below the significance threshold (see e.g. Simonsohn et al., 2014b).”
BackgroundP-curving x-phi: Does experimental philosophy have evidential value?
Stuart, M. T., Colaço, D., & Machery, É. (2019) · Analysis · doi:10.1093/analys/anz007 · Cited by 24
“In this article, we examine whether the corpus of x-phi suffers from selection bias and p-hacking by employing a p-curve (Simonsohn et al. 2014a).”
MethodOlsen, J., Mosen, J., Voracek, M., & Kirchler, E. M. (2019) · Royal Society Open Science · doi:10.1098/rsos.190738 · Cited by 20
“The coding scheme closely followed strategies employed in prior relatedmeta-research [11,30].”
MethodResultDo Incidental Environmental Anchors Bias Consumers’ Price Estimations?
Shanks, D. R., Barbieri-Hermitte, P., & Vadillo, M. A. (2020) · Collabra Psychology · doi:10.1525/collabra.310 · Cited by 29
Preregistration as a way to limit questionable research practice in advertising research
Bergkvist, L. (2020) · International Journal of Advertising · doi:10.1080/02650487.2020.1753441 · Cited by 21
Modelling publication bias and p ‐hacking
Moss, J., & Bin, R. D. (2021) · Biometrics · doi:10.1111/biom.13560 · Cited by 24
“Their meta-analysis is not conventional, but a p - curve analysis (Simonsohn et al. , 2014a).”
MethodInfluentialMeta-analyzing intelligence and religiosity associations: Evidence from the multiverse
Dürlinger, F., & Pietschnig, J. (2022) · PLoS ONE · doi:10.1371/journal.pone.0262699 · Cited by 21
“Specifically, p-curve [41] relies on the observed distribution of independent significant pvalues of a set of studies.”
BackgroundMeasuring the effects of publication bias in political science
Esarey, J., & Wu, A. (2016) · Research & Politics · doi:10.1177/2053168016665856 · Cited by 21
Ethical considerations in the early detection of Alzheimer's disease using speech and AI
Petti, U., Nyrup, R., Skopek, J. M., & Korhonen, A. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594063 · Cited by 16
“Moreover, researchers themselves often tend to only report studies that “work” [69] and assume that the findings based on a small proportion and diversity of the population are generalizable to a wider group [70].”
BackgroundReevaluating excess success in psychological science
van Boxtel, J. J. A., & Koch, C. (2016) · Psychonomic Bulletin & Review · doi:10.3758/s13423-016-1010-0 · Cited by 17
“Our evaluation revealed that only one article was highlighted as exhibiting evidence of p-hacking, although many studies were highlighted as having inadequate evidential value as calculated according to Simonsohn et al. (2014).”
MethodPietschnig, J., Gittler, G., Stieger, S., Forster, M., Gadek, N., Gartus, A., Kocsis-Bogár, K., Kubicek, B., Lüftenegger, M., Olsen, J., et al. (2018) · PLoS ONE · doi:10.1371/journal.pone.0202873 · Cited by 19
Allocating moral responsibility to multiple agents
Gantman, A. P., Sternisko, A., Gollwitzer, P. M., Oettingen, G., & Bavel, J. J. V. (2020) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2020.104027 · Cited by 21
Metadata
- Authors
- Simonsohn, U., Nelson, L. D., & Simmons, J. P.
- Affiliations
- University of Pennsylvania (US)
- Venue
- Journal of Experimental Psychology: General
- Published
- 2014-04
- Volume
- 143
- Issue
- 2
- Pages
- 534-547
- ISSN
- 0096-3445 (ISSN-L)
- 1939-2222
- OpenAlex
- W3124690569
- Cited by
- 1,500
- References
- 49
- Citations by year
- 2026: 75 · 2025: 96 · 2024: 110 · 2023: 121 · 2022: 146 · 2021: 157 · 2020: 158 · 2019: 154 · 2018: 143 · 2017: 113 · 2016: 103 · 2015: 80 · 2014: 35
- FWCI
- 82.62
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Experimental Psychology General (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Keywords
- p-curve, publication bias, selective reporting, p-hacking, file drawer problem, statistical significance, evidentiary value
- MeSH terms
- Data Interpretation, Statistical, Humans, Psychology, Experimental, Statistics as Topic, Reproducibility of Results, Models, Statistical, Publication Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.