P-curve: A key to the file-drawer.
The DOI matches its Crossref record; 49 of the 49 references it cites were found in a registry or an index.
Abstract
Because scientists tend to report only studies (publication bias) or analyses (p-hacking) that "work," readers must ask, "Are these effects true, or do they merely reflect selective reporting?" We introduce p-curve as a way to answer this question. P-curve is the distribution of statistically significant p values for a set of studies (ps < .05). Because only true effects are expected to generate right-skewed p-curves-containing more low (.01s) than high (.04s) significant p values--only right-skewed p--curves are diagnostic of evidential value. By telling us whether we can rule out selective reporting as the sole explanation for a set of findings, p-curve offers a solution to the age-old inferential problems caused by file-drawers of failed studies and analyses.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Incorrect p-values could affect the outcome of tests that analyze the distribution of p-values, such as the p-curve (Simonsohn, Nelson, & Simmons, 2014) and p-uniform (Van Assen, Van Aert, & Wicherts, 2014).”
From The prevalence of statistical reporting errors in psychology (1985–2013) (2016) · cites this paper as Simonsohn, Nelson, & Simmons, 2014
In the citing paper
“Even if the first REM episode occurs about 70 minutes after falling asleep (33), some subjects may present a REM EEG and occulomotor pattern a way farster.The REM sleep can actually occur pretty rapidly in some individuals according to the MD (sleep specialist) who co-supervised this study.Furthermore, the MSLT has specialy been built for this puporse (see32)”
From Is there a Publication Bias in Behavioural Intranasal Oxytocin Research on Humans? Opening the File Drawer of One Laboratory (2016) · cites this paper as [33]
In the citing paper
“After excluding studies which assessed maltreatment as part of broader measures of victimization (k=3) (25, 27, 29) or ACEs (k=5) (30, 34, 39, 44, 49), or focused on institutional neglect (k=3) (15, 45, 46), the meta-analytic effect size was also similar to the original estimate (Cohen’s d=0.34, 95% CI=0.26-0.42).”
From Childhood Maltreatment and Mental Health Problems: A Systematic Review and Meta-Analysis of Quasi-Experimental Studies (2023) · cites this paper as [25]
In the citing paper
“Trim‐and‐fill procedures do suffer from some well‐known limitations that could result in the underestimation or overestimation of publication bias (Rothstein, 2008; Simonsohn, Nelson, & Simmons, 2014).”
From Hot spots policing of small geographic areas effects on crime (2019) · cites this paper as Simonsohn, Nelson, & Simmons, 2014
In the citing paper
“Applying selection modeling approaches like p-uniform or 3PSM to the simple effects may therefore substantially overestimate the true effect size.”
From Do Salient Social Norms Moderate Mortality Salience Effects? A (Challenging) Meta-Analysis of Terror Management Studies (2023) · cites this paper as Simonsohn et al., 2014
Citations
Cites
Statistical power analysis for the behavioral sciences
(1990) · Computers, Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,269
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722
(1963) · Medical Entomology and Zoology · Cited by 15,085
Duval, S., & Tweedie, R. (2000) · Biometrics · doi:10.1111/j.0006-341x.2000.00455.x · Cited by 14,099
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068
The case for motivated reasoning.
Kunda, Z. (1990) · Psychological Bulletin · doi:10.1037/0033-2909.108.3.480 · Cited by 9,022
Cohen, J. (1992) · Psychological Bulletin · doi:10.1037//0033-2909.112.1.155 · Cited by 8,771
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
Self-citationTesting Statistical Hypotheses
(2021) · Wiley Series in Probability and Statistics · doi:10.1002/9781119243830.ch12 · Cited by 5,106
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 1×
“1 Trying multiple analyses to obtain statistical significance has received many names, including: bias (Ioannidis, 2005), significance chasing (Ioannidis & Trikalinos, 2007), data snooping (White, 2003), fiddling (Bross, 1971), publication bias in situ (Phillips, 2004) and specification searching (Gerber & Malhotra, 2…”
Cited by
Items 401–500 of 1,005
P-curving the fusiform face area: Meta-analyses support the expertise hypothesis
Burns, E. J., Arnold, T., & Bukach, C. M. (2019) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2019.07.003 · Cited by 56
Shen, H., Xie, J., Li, J., & Cheng, Y. (2021) · Scientometrics · doi:10.1007/s11192-021-03888-0 · Cited by 64
Psychology, replication & beyond
Laws, K. R. (2016) · BMC Psychology · doi:10.1186/s40359-016-0135-2 · Cited by 59
“[52].”
Persuasion Bias in Science: Can Economics Help?
Tillio, A. D., Ottaviani, M., & Sørensen, P. N. (2017) · The Economic Journal · doi:10.1111/ecoj.12515 · Cited by 58
Less “story” and more “reliability” in cognitive neuroscience
Huber, D. E., Potter, K. W., & Huszar, L. D. (2018) · Cortex · doi:10.1016/j.cortex.2018.10.030 · Cited by 53
Shen, Y., & Shao, W. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph19159673 · Cited by 47
“[41], the inclusion criteria for the literature were determined before electronic retrieval: (1) peer-reviewed journals published in international journals; (2) empirical studies performing the implementation”
Hope that Benefits Others: A Systematic Literature Review of Hope Theory and Prosocial Outcomes
Schornick, Z. T., Ellis, N., Ray, E., Snyder, B.-J., & Thomas, K. J. (2023) · International Journal of Applied Positive Psychology · doi:10.1007/s41042-022-00084-0 · Cited by 68
Benefits of the microalgae Spirulina and Schizochytrium in fish nutrition: a meta-analysis
Trevi, S., Webster, T. M. U., Consuegra, S., & de Leániz, C. G. (2023) · Scientific Reports · doi:10.1038/s41598-023-29183-x · Cited by 60
Improving functional magnetic resonance imaging reproducibility
Pernet, C. R., & Poline, J. (2015) · GigaScience · doi:10.1186/s13742-015-0055-8 · Cited by 45
“studies [6], or be declared significant after too many variations of the analysis procedure [7,8] without controlling appropriately for the overall risk of error (p-hacking effect [6,9]).”
BackgroundSystematic Review of Polygenic Gene–Environment Interaction in Tobacco, Alcohol, and Cannabis Use
Pasman, J. A., Verweij, K. J. H., & Vink, J. M. (2019) · Behavior Genetics · doi:10.1007/s10519-019-09958-7 · Cited by 55
“As most studies did report p values for the G×E analysis, a p curve analysis could be conducted (Simonsohn et al. 2014) to give an indication of the strength of the evidence and of the probability that p hacking occurred in the included studies.”
ResultReporting errors and biases in published empirical findings: Evidence from innovation research
Bruns, S. B., Asanov, I., Bode, R., Dunger, M., Funk, C., Hassan, S. M., Hauschildt, J., Heinisch, D. P., Kempa, K., König, J., et al. (2019) · Research Policy · doi:10.1016/j.respol.2019.05.005 · Cited by 61
Above averaging in literature reviews
Simonsohn, U., Simmons, J. P., & Nelson, L. D. (2022) · Nature Reviews Psychology · doi:10.1038/s44159-022-00101-8 · Cited by 44
Self-citationZarski, A., Velten, J., Knauer, J., Berking, M., & Ebert, D. D. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00670-1 · Cited by 54
Hallford, D. J., Rusanov, D., Yeow, J., & Barry, T. J. (2022) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.2786 · Cited by 50
“A p-curve analysis was also used to assess the potential of p-hacking, that is, selective presentation or analysis of data motivated to attain a p-value below an alpha of .05 (Simonsohn et al., 2014).”
MethodGrandparental Support and Maternal Postpartum Mental Health: A Review and Meta-Analysis
Riem, M. M. E., Bakermans‐Kranenburg, M. J., Cima, M., & van IJzendoorn, M. H. (2023) · Human Nature · doi:10.1007/s12110-023-09440-8 · Cited by 67
“Lastly, p-curve analysis was performed in order to detect potential bias due to p-hacking (Simonsohn et al., 2014).”
MethodA meta-analysis of the relation between dream content and memory consolidation
Hudachek, L., & Wamsley, E. J. (2023) · SLEEP · doi:10.1093/sleep/zsad111 · Cited by 46
Chopra, F., Haaland, I., Roth, C., & Stegmann, A. (2023) · The Economic Journal · doi:10.1093/ej/uead060 · Cited by 85
Fisher, M. L., Olivares, M. A., & Staats, B. R. (2019) · Manufacturing & Service Operations Management · doi:10.1287/msom.2019.0812 · Cited by 53
“” Others have highlighted the harm such an approach can cause for research because scholars may be tempted to pursue less than ethical means to achieve their goals (Simonsohn et al. 2014).”
BackgroundOutlier exclusion procedures must be blind to the researcher’s hypothesis.
André, Q. (2021) · Journal of Experimental Psychology General · doi:10.1037/xge0001069 · Cited by 53
DeKay, M. L., Rubinchik, N., Li, Z., & de Boeck, P. A. L. (2022) · Perspectives on Psychological Science · doi:10.1177/17456916221079611 · Cited by 56
Impacts and Determinants of Health Levels in Low-Income Countries
Pascaline Dupas, Y., & Miguel, E. A. (2016) · National Bureau of Economic Research · doi:10.3386/w22235 · Cited by 46
Conservative Tests under Satisficing Models of Publication Bias
McCrary, J., Christensen, G., & Fanelli, D. (2016) · PLoS ONE · doi:10.1371/journal.pone.0149590 · Cited by 49
“In recent years the 11 issue has received renewed attention in finance [14–18], in statistics [19–22], in political 12 science [23,24], in psychology [1, 25–27], in medicine [28–30], and in other fields.”
BackgroundMesquida, C., Murphy, J., Lakens, D., & Warne, J. P. (2023) · Journal of Sports Sciences · doi:10.1080/02640414.2023.2269357 · Cited by 57
“The presence of publication bias and studies with underpow-ered designs in a body of literature can be examined using a z-curve analysis (Bartoš & Schimmack, 2022; Brunner & Schimmack, 2020; see also Simonsohn et al., 2014a, 2014b for p -curve).”
From Tools to Agents: Meta-Analytic Insights into Human Acceptance of AI
Li, B., Lai, E. Y., & Wang, X. (. (2025) · Journal of Marketing · doi:10.1177/00222429251355266 · Cited by 78
A Tutorial on Hunting Statistical Significance by Chasing N
Szűcs, D. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2016.01444 · Cited by 53
Too Good to be False: Nonsignificant Results Revisited
Hartgerink, C. H. J., Wicherts, J. M., & van Assen, M. A. L. M. (2017) · Collabra: Psychology · doi:10.1525/collabra.71 · Cited by 45
Craig, R., Cox, A., Tourish, D., & Thorpe, A. (2020) · Research Policy · doi:10.1016/j.respol.2020.103930 · Cited by 56
“” P-hacking is an umbrella term for a number of practices that, collectively, seek to increase a researcher’s chances of obtaining desired p-values, such as excluding experimental conditions or dependent measures that do not produce a statistically significant p-value (Simonsohn et al., 2014).”
BackgroundAyorinde, A., Williams, I., Mannion, R., Song, F., Skrybant, M., Lilford, R. J., & Chen, Y. (2020) · BMC Medical Research Methodology · doi:10.1186/s12874-020-01010-1 · Cited by 49
“In addition to the conventional methods discussed above, new methods such as p-curves for detecting phacking have emerged in recent years [36, 37].”
MethodTovmasyan, A., Monk, R. L., & Heim, D. (2022) · PLoS ONE · doi:10.1371/journal.pone.0262670 · Cited by 53
“com/) to assess potential p-hacking [76].”
Lakens, D. (2014) · Social Cognition · doi:10.1521/soco.2014.32.supp.168 · Cited by 51
Goals in Nutrition Science 2015–2020
Allison, D. B., Bassaganya-Riera, J., Burlingame, B., Brown, A. W., le Coutre, J., Dickson, S. L., van Eden, W., Garssen, J., Hontecillas, R., Khoo, C. S. H., et al. (2015) · Frontiers in Nutrition · doi:10.3389/fnut.2015.00026 · Cited by 45
“[called p-hacking (139)d, investigator degrees of”
What reduces prejudice in the real world? A meta‐analysis of prejudice reduction field experiments
Hsieh, W., Faulkner, N., & Wickes, R. L. (2021) · British Journal of Social Psychology · doi:10.1111/bjso.12509 · Cited by 54
Hornstra, R., Groenman, A. P., der Oord, S. V., Luman, M., Dekkers, T. J., van der Veen‐Mulders, L., Hoekstra, P. J., & van den Hoofdakker, B. J. (2022) · Child and Adolescent Mental Health · doi:10.1111/camh.12561 · Cited by 63
Vieira, V. A., Wolter, J. S., Araújo, C. F., & Frio, R. S. (2022) · International Journal of Research in Marketing · doi:10.1016/j.ijresmar.2022.09.002 · Cited by 55
Brañas‐Garza, P., Galizzi, M. M., & Nieboer, J. (2018) · International Economic Review · doi:10.1111/iere.12299 · Cited by 61
Carbine, K. A., & Larson, M. J. (2019) · Health Psychology Review · doi:10.1080/17437199.2019.1622144 · Cited by 45
Pietschnig, J., Siegel, M., Eder, J. S. N., & Gittler, G. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.02874 · Cited by 56
Lutz, M. C., Kok, R., Verveer, I., Malbec, M., Koot, S., van Lier, P. A. C., & Franken, I. H. A. (2021) · Journal of Psychiatry and Neuroscience · doi:10.1503/jpn.200031 · Cited by 44
“61 However, these assessments have their limitations, 62 so we also performed a p curve analysis 62–65 (Appendix 1, section 5) to inspect whether sig nificant p values ( p 0.05 62 ) provided proof of evidential value (for a full description and application of this assessment, see Harrer and…”
InfluentialIncreasing the Credibility of Political Science Research: A Proposal for Journal Reforms
Nyhan, B. J. (2015) · PS Political Science & Politics · doi:10.1017/s1049096515000463 · Cited by 58
“…self-deception—that is, as authors, reviewers, and editors, we are simply too practiced in rationalizing choices that produce the results we want, including “p-hacking” (Simmons, Nelson, and Simonsohn 2011; Simonsohn, Nelson, and Simmons 2013) and the use of low-power studies (Button et al. 2013).”
BackgroundRubio‐Fernández, P. (2018) · Infant Behavior and Development · doi:10.1016/j.infbeh.2018.09.009 · Cited by 50
Yeung, S. K., Yay, T., & Feldman, G. (2021) · Personality and Social Psychology Bulletin · doi:10.1177/01461672211042315 · Cited by 52
“048 P-curve (Simonsohn et al., 2014) Evidential value is present, adjusted effect: d = 0.”
BackgroundMethodInfluentialAssessing Publication Bias: a 7-Step User’s Guide with Best-Practice Recommendations
Kepes, S., Wang, W., & Cortina, J. M. (2022) · Journal of Business and Psychology · doi:10.1007/s10869-022-09840-0 · Cited by 49
Hu, J., & Cheung, C. (2024) · Frontiers in Psychology · doi:10.3389/fpsyg.2024.1361163 · Cited by 81
“We test the estimate indicator credibility through p -curve analysis (Simonsohn et al., 2014).”
MethodFear and safety learning in anxiety- and stress-related disorders: An updated meta-analysis
Kausche, F. M., Carsten, H. P., Sobania, K. M., & Riesel, A. (2024) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2024.105983 · Cited by 62
Approaching a fair deal for significance and other concerns
Giner‐Sorolla, R. (2016) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2016.01.010 · Cited by 45
A Proposal to Organize and Promote Replications
Coffman, L. C., Niederle, M., & Wilson, A. J. (2017) · American Economic Review · doi:10.1257/aer.p20171122 · Cited by 50
Responsible Research Assessment II: A specific proposal for hiring and promotion in psychologyPreprint
Gärtner, A., Leising, D., Freyer, N., Musfeld, P., Lange, J., & Schönbrodt, F. D. (2022) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/5yexm · Cited by 42
Performance implications of outsourcing: A meta-analysis
Lahiri, S., Karna, A., Kalubandi, S. C., & Edacherian, S. (2021) · Journal of Business Research · doi:10.1016/j.jbusres.2021.10.061 · Cited by 72
Neves, A. R., Montoya-Botero, P., & Polyzos, N. P. (2022) · American Journal of Obstetrics and Gynecology · doi:10.1016/j.ajog.2022.03.051 · Cited by 42
Campbell, L., Loving, T. J., & LeBel, E. P. (2014) · Personal Relationships · doi:10.1111/pere.12053 · Cited by 45
Vermeulen, I., Beukeboom, C. J., Batenburg, A., Avrămiea, A.-E., Stoyanov, D. S., van de Velde, B., & Oegema, D. (2015) · Communication Methods and Measures · doi:10.1080/19312458.2015.1096333 · Cited by 47
“However, a population of findings with a very high frequency of “barely significant” findings is not (Simonsohn et al., 2014).”
BackgroundChatard, A., Bocage-Barthélémy, Y., Selimbegović, L., & Guimond, S. (2017) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2017.05.005 · Cited by 59
On the Ethics of Neuromarketing and Sensory Marketing
Spence, C. J. (2019) · Advances in neuroethics · doi:10.1007/978-3-030-27177-0_3 · Cited by 55
Anderson, S. F. (2019) · Psychological Methods · doi:10.1037/met0000248 · Cited by 46
“These findings relate broadly to the p -curve (Simonsohn, Nelson, & Simmons, 2014b) which depicts the expected percentages of different significant p values for various levels of power and sample size.”
Dominance or prestige: A review of the effects of power poses and other body postures
Körner, R., & Schütz, A. (2020) · Social and Personality Psychology Compass · doi:10.1111/spc3.12559 · Cited by 49
Questionable research practices may have little effect on replicability
Ulrich, R., & Miller, J. O. (2020) · eLife · doi:10.7554/elife.58237 · Cited by 52
“Such practices violate not only the basic assumptions of the null hypothesis significance testing (NHST) framework but also those underlying decision making within the Bayesian framework, where researchers could analogously use QRPs to obtain large Bayes factors (Simonsohn, 2014).”
MethodMeasuring Sex Differences and Similarities
Giudice, M. D. (2022) · Focus on sexuality research · doi:10.1007/978-3-030-84273-4_1 · Cited by 47
Advantages of Starting with Theory
Shaw, J. D. (2017) · Academy of Management Journal · doi:10.5465/amj.2017.4003 · Cited by 53
Doffer, D. P. A., Dekkers, T. J., Hornstra, R., der Oord, S. V., Luman, M., Leijten, P. H. O., Hoekstra, P. J., van den Hoofdakker, B. J., & Groenman, A. P. (2023) · JCPP Advances · doi:10.1002/jcv2.12196 · Cited by 62
“P‐curves (Simonsohn et al., 2014) for all outcome domains were right‐skewed and indicated evidential value of findings (i.”
BackgroundMethodThe case against categorical risk estimates
Scurich, N. (2018) · Behavioral Sciences & the Law · doi:10.1002/bsl.2382 · Cited by 44
Chuard, P. J. C., Vrtílek, M., Head, M. L., & Jennions, M. D. (2019) · PLoS Biology · doi:10.1371/journal.pbio.3000127 · Cited by 55
Expanding Statistical Frontiers in Sexual Science: Taxometric, Invariance, and Equivalence Testing
Sakaluk, J. K. (2019) · The Journal of Sex Research · doi:10.1080/00224499.2019.1568377 · Cited by 42
Has the evidence for moral licensing been inflated by publication bias?
Kuper, N., & Bott, A. (2019) · Meta-Psychology · doi:10.15626/mp.2018.878 · Cited by 30
Defending one's worldview under mortality salience: Testing the validity of an established idea
Schindler, S., Reinhardt, N., & Reinhard, M.-A. (2021) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2020.104087 · Cited by 40
The fragility index can be used for sample size calculations in clinical trials
Baer, B. R., Gaudino, M. F. L., Fremes, S. E., Charlson, M. E., & Wells, M. T. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.08.010 · Cited by 47
Hantula, D. A. (2019) · Perspectives on Behavior Science · doi:10.1007/s40614-019-00194-2 · Cited by 45
Schönbrodt, F. D., Gärtner, A., Frank, M., Gollwitzer, M., Ihle, M., Mischkowski, D., Phan, L. V., Schmitt, M., Scheel, A. M., Schubert, A., et al. (2022) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/rgh5b · Cited by 45
Mertens, G., Lund, M., & Engelhard, I. M. (2021) · Acta Psychologica · doi:10.1016/j.actpsy.2021.103424 · Cited by 40
Single-case experimental research designs.
Kazdin, A. E. (2015) · Methodological issues and strategies in clinical research (4th ed.). · doi:10.1037/14805-029 · Cited by 39
Miller, J. O., & Ulrich, R. (2016) · Perspectives on Psychological Science · doi:10.1177/1745691616649170 · Cited by 43
Control Variables in Marketing Research
Klarmann, M., & Feurer, S. (2018) · Marketing ZFP · doi:10.15358/0344-1369-2018-2-26 · Cited by 59
“As which control variables need to be included is often unclear, researchers may play around with different sets of control variables until they receive the results they like – a practice sometimes referred to as “p-hacking” (Simonsohn et al. 2014, p. 670).”
BackgroundBruns, S. B., & Stern, D. I. (2018) · Empirical Economics · doi:10.1007/s00181-018-1446-3 · Cited by 48
“…sampling errors but not based on overfitted lag lengths: where d f i denotes the degrees of freedom of a single equation of the VAR model used in primary study i and z (cid:8) (cid:9) , where π is the p -value of study i and Φ − 1 is the inverse cumulative distribution function of the standard…”
BackgroundMethodInfluentialMeta-analysis of the reduced relative feedback frequency effect on motor learning and performance
McKay, B., Hussien, J., Vinh, M.-A., Mir-Orefice, A., Brooks, H., & Ste‐Marie, D. M. (2022) · Psychology of sport and exercise · doi:10.1016/j.psychsport.2022.102165 · Cited by 42
Advances in transparency and reproducibility in the social sciences
Freese, J., Rauf, T., & Voelkel, J. G. (2022) · Social Science Research · doi:10.1016/j.ssresearch.2022.102770 · Cited by 60
Chance, Z., Gorlin, M., & Dhar, R. (2014) · Customer Needs and Solutions · doi:10.1007/s40547-014-0025-9 · Cited by 50
“Research publications favor statistically significant results, with differences large enough—and often only large enough—to imply a high likelihood of observing a measurable difference under similar circumstances in the future [69].”
BackgroundSome properties of p-curves, with an application to gradual publication bias.
Ulrich, R., & Miller, J. O. (2017) · Psychological Methods · doi:10.1037/met0000125 · Cited by 38
Replication and Pre-Registration in Comparative Psychology
Beran, M. J. (2018) · International Journal of Comparative Psychology · doi:10.46867/ijcp.2018.31.01.09 · Cited by 42
Collapsing Categorical Variables and Measurement Invariance
Rutkowski, L. A., Svetina, D., & Liaw, Y. (2019) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705511.2018.1547640 · Cited by 55
Toward Open and Reproducible Epidemiology
Mathur, M. B., & Fox, M. P. (2023) · American Journal of Epidemiology · doi:10.1093/aje/kwad007 · Cited by 56
“Preregistration can help limit unintentional or intentional selective reporting of results, sometimes called “P-hacking” or “data dredging” (51).”
BackgroundMixed-effects design analysis for experimental phonetics
Kirby, J., & Sonderegger, M. (2018) · Journal of Phonetics · doi:10.1016/j.wocn.2018.05.005 · Cited by 42
Willard, A. K., Shariff, A., & Norenzayan, A. (2016) · Current Opinion in Psychology · doi:10.1016/j.copsyc.2016.06.003 · Cited by 42
“Effect-size analyses assess the strength and reliability of an effect, and P-curve analyses assess the degree to which questionable research practices and researcher degrees of freedom (also known as P-hacking) are a factor [19 ,20].”
BackgroundInterpreting t-Statistics Under Publication Bias: Rough Rules of Thumb
Winship, C., & Zhuo, X. (2018) · Journal of Quantitative Criminology · doi:10.1007/s10940-018-9387-8 · Cited by 46
Forging a Stronger Academic-Practitioner Partnership–The Case of Net Promoter Score (NPS)
Bendle, N. T., Bagga, C. K., & Nastasoiu, A. (2019) · The Journal of Marketing Theory and Practice · doi:10.1080/10696679.2019.1577689 · Cited by 37
Jamadar, S. D. (2020) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2020.107446 · Cited by 48
Primates, M., Altschul, D., Beran, M. J., Bohn, M., Caspar, K. R., Fichtel, C., Försterling, M., Grebe, N. M., Hernandez-Aguilar, R. A., Kwok, S. C., et al. (2019) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/8w7zd · Cited by 38
Analytic Review as a Solution to the Misreporting of Statistical Results in Psychological Science
Sakaluk, J., Williams, A., & Biernat, M. (2014) · Perspectives on Psychological Science · doi:10.1177/1745691614549257 · Cited by 32
Erasmus, A., Holman, B., & Ioannidis, J. P. A. (2021) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2020-111584 · Cited by 41
Röseler, L., Schütz, A., Blank, P. A., Dück, M., Fels, S., Kupfer, J., Scheelje, L., & Seida, C. (2020) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2020.104066 · Cited by 48
Does Sociology Need Open Science?
Breznau, N. (2021) · Societies · doi:10.3390/soc11010009 · Cited by 46
“Similar results supporting the biased red dotted line in Figure 1 were found in studies by Simonsohn, Nelson, and Simmons in psychology [29,30] and Brodeur, Cook, and Heyes in economics [31].”
ResultGiudice, M. D. (2020) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2020.11.009 · Cited by 36
“To assess the evidentiary value of the study set, I performed a p-curve analysis (Simonsohn et al., 2014) with the online application at https://p-curve.”
MethodChang, X., Gao, H., & Li, W. (2024) · Journal of Accounting Research · doi:10.1111/1475-679x.12579 · Cited by 37
Implicit and explicit influences of religious cognition on Dictator Game transfers
Billingsley, J., Gomes, C., & McCullough, M. E. (2018) · Royal Society Open Science · doi:10.1098/rsos.170238 · Cited by 36
The robustness of anchoring effects on preferential judgments
Yoon, S., Fong, N. M., & Dimoka, A. (2019) · Judgment and Decision Making · doi:10.1017/s1930297500006148 · Cited by 41
How can food choice best be trained? Approach-avoidance versus go/no-go training
Veling, H., Verpaalen, I. A. M., Liu, H., Mosannenzadeh, F., Becker, D., & Holland, R. W. (2021) · Appetite · doi:10.1016/j.appet.2021.105226 · Cited by 35
Transparent, Open, and Reproducible Prevention Science
Grant, S., Wendt, K. E., Leadbeater, B. J. R., Supplee, L. H., Mayo‐Wilson, E., Gardner, F. E. M., & Bradshaw, C. P. (2022) · Prevention Science · doi:10.1007/s11121-022-01336-w · Cited by 46
“For example, “p-hacking” refers to repeatedly searching a dataset or trying multiple alternative analyses until a statistically significant or desired finding is obtained—and then failing to fully report how this result was obtained (Simonsohn et al., 2014).”
BackgroundManyPrimates, Altschul, D., Beran, M. J., Bohn, M., Caspar, K., Fichtel, C., Försterling, M., Grebe, N. M., Hernandez-Aguilar, R. A., Kwok, S. C., et al. (2019) · PsyArXiv (OSF Preprints) · doi:10.24602/sjpr.62.3_205 · Cited by 42
Evaluating Replication ResultsPreprint
Simonsohn, U. (2013) · SSRN Electronic Journal · doi:10.2139/ssrn.2259879 · Cited by 38
Self-citationSchindler, S., Hilgard, J., Fritsche, I., Burke, B., & Pfattheicher, S. (2023) · Personality and Social Psychology Review · doi:10.1177/10888683221107267 · Cited by 43
“05), at least one simple effect is likely to be highly significant (p < .01; Simonsohn et al., 2014).”
BackgroundInfluentialMaas genannt Bermpohl, F., Hülsmann, L., & Martin, A. (2023) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2023.1160908 · Cited by 44
“Furthermore, we used the p-curve method, focussing on the distribution of statistically significant p-values (31).”
MethodDoes the conclusion follow from the evidence? Recommendations for improving research
Hales, A. H. (2016) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2015.09.011 · Cited by 54
Metadata
- Authors
- Simonsohn, U., Nelson, L. D., & Simmons, J. P.
- Affiliations
- University of Pennsylvania (US)
- Venue
- Journal of Experimental Psychology: General
- Published
- 2014-04
- Volume
- 143
- Issue
- 2
- Pages
- 534-547
- ISSN
- 0096-3445 (ISSN-L)
- 1939-2222
- OpenAlex
- W3124690569
- Cited by
- 1,500
- References
- 49
- Citations by year
- 2026: 75 · 2025: 96 · 2024: 110 · 2023: 121 · 2022: 146 · 2021: 157 · 2020: 158 · 2019: 154 · 2018: 143 · 2017: 113 · 2016: 103 · 2015: 80 · 2014: 35
- FWCI
- 82.62
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Experimental Psychology General (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Keywords
- p-curve, publication bias, selective reporting, p-hacking, file drawer problem, statistical significance, evidentiary value
- MeSH terms
- Data Interpretation, Statistical, Humans, Psychology, Experimental, Statistics as Topic, Reproducibility of Results, Models, Statistical, Publication Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.