P-curve: A key to the file-drawer.
The DOI matches its Crossref record; 49 of the 49 references it cites were found in a registry or an index.
Abstract
Because scientists tend to report only studies (publication bias) or analyses (p-hacking) that "work," readers must ask, "Are these effects true, or do they merely reflect selective reporting?" We introduce p-curve as a way to answer this question. P-curve is the distribution of statistically significant p values for a set of studies (ps < .05). Because only true effects are expected to generate right-skewed p-curves-containing more low (.01s) than high (.04s) significant p values--only right-skewed p--curves are diagnostic of evidential value. By telling us whether we can rule out selective reporting as the sole explanation for a set of findings, p-curve offers a solution to the age-old inferential problems caused by file-drawers of failed studies and analyses.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Incorrect p-values could affect the outcome of tests that analyze the distribution of p-values, such as the p-curve (Simonsohn, Nelson, & Simmons, 2014) and p-uniform (Van Assen, Van Aert, & Wicherts, 2014).”
From The prevalence of statistical reporting errors in psychology (1985–2013) (2016) · cites this paper as Simonsohn, Nelson, & Simmons, 2014
In the citing paper
“Even if the first REM episode occurs about 70 minutes after falling asleep (33), some subjects may present a REM EEG and occulomotor pattern a way farster.The REM sleep can actually occur pretty rapidly in some individuals according to the MD (sleep specialist) who co-supervised this study.Furthermore, the MSLT has specialy been built for this puporse (see32)”
From Is there a Publication Bias in Behavioural Intranasal Oxytocin Research on Humans? Opening the File Drawer of One Laboratory (2016) · cites this paper as [33]
In the citing paper
“After excluding studies which assessed maltreatment as part of broader measures of victimization (k=3) (25, 27, 29) or ACEs (k=5) (30, 34, 39, 44, 49), or focused on institutional neglect (k=3) (15, 45, 46), the meta-analytic effect size was also similar to the original estimate (Cohen’s d=0.34, 95% CI=0.26-0.42).”
From Childhood Maltreatment and Mental Health Problems: A Systematic Review and Meta-Analysis of Quasi-Experimental Studies (2023) · cites this paper as [25]
In the citing paper
“Trim‐and‐fill procedures do suffer from some well‐known limitations that could result in the underestimation or overestimation of publication bias (Rothstein, 2008; Simonsohn, Nelson, & Simmons, 2014).”
From Hot spots policing of small geographic areas effects on crime (2019) · cites this paper as Simonsohn, Nelson, & Simmons, 2014
In the citing paper
“Applying selection modeling approaches like p-uniform or 3PSM to the simple effects may therefore substantially overestimate the true effect size.”
From Do Salient Social Norms Moderate Mortality Salience Effects? A (Challenging) Meta-Analysis of Terror Management Studies (2023) · cites this paper as Simonsohn et al., 2014
Citations
Cites
Statistical power analysis for the behavioral sciences
(1990) · Computers, Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,269
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722
(1963) · Medical Entomology and Zoology · Cited by 15,085
Duval, S., & Tweedie, R. (2000) · Biometrics · doi:10.1111/j.0006-341x.2000.00455.x · Cited by 14,099
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068
The case for motivated reasoning.
Kunda, Z. (1990) · Psychological Bulletin · doi:10.1037/0033-2909.108.3.480 · Cited by 9,022
Cohen, J. (1992) · Psychological Bulletin · doi:10.1037//0033-2909.112.1.155 · Cited by 8,771
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
Self-citationTesting Statistical Hypotheses
(2021) · Wiley Series in Probability and Statistics · doi:10.1002/9781119243830.ch12 · Cited by 5,106
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 1×
“1 Trying multiple analyses to obtain statistical significance has received many names, including: bias (Ioannidis, 2005), significance chasing (Ioannidis & Trikalinos, 2007), data snooping (White, 2003), fiddling (Bross, 1971), publication bias in situ (Phillips, 2004) and specification searching (Gerber & Malhotra, 2…”
Cited by
Items 701–800 of 1,005
Qiu, K., Chen, E. H., Wan, S., & Bailey, D. H. (2021) · Journal of Experimental Psychology Learning Memory and Cognition · doi:10.1037/xlm0001087 · Cited by 23
A Reexamination of the Impact of Decision Conflict on Choice Deferral
Evangelidis, I., Levav, J., & Simonson, I. (2020) · Management Science · doi:10.1287/mnsc.2022.4484 · Cited by 17
The "P"-Value: The Primary Alphabet of Research Revisited
Das, D., & Das, T. (2023) · International Journal of Preventive Medicine · doi:10.4103/ijpvm.ijpvm_200_22 · Cited by 23
“A p‐curve can be used to assess the reliability of scientific literature, such as by detecting publication bias or p‐hacking.[6] According to the American Statistical Association (ASA), P values are often misused and misinterpreted.”
MethodPrice Partitioning of Socio-Moral Surcharges
Goenka, S., & Bagchi, R. (2024) · Journal of Consumer Research · doi:10.1093/jcr/ucae026 · Cited by 16
One Decade Into the Replication Crisis, How Have Psychological Results Changed?
Bogdan, P. C. (2025) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459251323480 · Cited by 21
Carbine, K. A., Lindsey, H. M., Rodeback, R. E., & Larson, M. J. (2019) · International Journal of Psychophysiology · doi:10.1016/j.ijpsycho.2019.06.004 · Cited by 16
Updating the p-curve analysis of Carbine and Larson with results from preregistered experiments
Veling, H., Chen, Z., Liu, H., Quandt, J., & Holland, R. W. (2019) · Health Psychology Review · doi:10.1080/17437199.2019.1669482 · Cited by 18
Rigor and reproducibility for data analysis and design in the behavioral sciences
Hildebrandt, T., & Prenoveau, J. M. (2020) · Behaviour Research and Therapy · doi:10.1016/j.brat.2020.103552 · Cited by 19
Optimizing Research Output: How Can Psychological Research Methods Be Improved?
Miller, J. O., & Ulrich, R. (2021) · Annual Review of Psychology · doi:10.1146/annurev-psych-020821-094927 · Cited by 21
The Self-Referencing task: Theoretical overview and empirical evidence
Mattavelli, S., Richetin, J., Gallucci, M., & Perugini, M. (2017) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2017.02.006 · Cited by 19
A Bayesian decision-making framework for replication
Hardwicke, T. E., Tessler, M. H., Peloquin, B. N., & Frank, M. C. (2018) · Behavioral and Brain Sciences · doi:10.1017/s0140525x18000675 · Cited by 16
Transparency, Replication, and Cumulative Learning: What Experiments Alone Cannot Achieve
Dunning, T. (2016) · Annual Review of Political Science · doi:10.1146/annurev-polisci-072516-014127 · Cited by 16
McKay, B., Yantha, Z. D., Hussien, J., Carter, M. J., & Ste‐Marie, D. M. (2021) · doi:10.31234/osf.io/8d3nb · Cited by 16
“Observing a p-curve significantly “flatter” than what would be expected with 33% power would suggest a lack of evidential value among the significant results (Simonsohn et al., 2014a, 2014b).”
BackgroundResultMaterial Scarcity and Unethical Economic Behavior: A Systematic Review and Meta-AnalysisPreprint
Elbaek, C., Mitkidis, P., Aarøe, L., & Otterbring, T. (2021) · Research Square · doi:10.21203/rs.3.rs-800481/v2 · Cited by 17
Leonard, K. S., Evans, M. B., Oravecz, Z., Smyth, J. M., & Downs, D. S. (2020) · Journal of Technology in Behavioral Science · doi:10.1007/s41347-020-00155-6 · Cited by 19
Meier, M., Lonsdorf, T. B., LUPIEN, S. J., Stalder, T., Laufer, S., Sicorello, M., Linz, R., & Puhlmann, L. M. C. (2022) · Comprehensive Psychoneuroendocrinology · doi:10.1016/j.cpnec.2022.100144 · Cited by 20
ȘTIR, M., & Zaiţ, A. (2022) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2022.2131263 · Cited by 16
“…consequences — future investigators may spend time and resources conducting research on topics that have been already studied, but not published, or unreal effects might appear as being supported by research (Hubbard & Armstrong, 1992; Rosenthal, 1979; Scargle, 2000; Simonsohn et al., 2014).”
BackgroundText Generation Benefits Learning: a Meta-Analytic Review
Schindler, J., & Richter, T. (2023) · Educational Psychology Review · doi:10.1007/s10648-023-09758-w · Cited by 20
Torka, A., Mazei, J., Bosco, F. A., Cortina, J. M., Götz, M., Kepes, S., O’Boyle, E. H., & Hüffmeier, J. (2023) · European Journal of Work and Organizational Psychology · doi:10.1080/1359432x.2023.2206571 · Cited by 20
“Further, in Bedeian et al. (2010) study, one third of researchers reported that they excluded data post hoc (29%), or selectively included control variables (33%), that is, they engaged in p-hacking (i.e., “report only [. . .] analyses [. . .] that ‘work’” Simonsohn et al., 2014, p. 534).”
BackgroundA toolbox to evaluate the trustworthiness of published findings
Adler, S. J., Röseler, L., & Schöniger, M. K. (2023) · Journal of Business Research · doi:10.1016/j.jbusres.2023.114189 · Cited by 21
Baptista, F., Nery, E. C. H. P., Cruz, E. B., Afreixo, V. M. A., & Silva, A. G. (2023) · Clinical Rehabilitation · doi:10.1177/02692155231215216 · Cited by 22
Borrower‐based macroprudential measures and credit growth: How biased is the existing literature?
Malovaná, S., Hodula, M., Gric, Z., & Bajzík, J. (2024) · Journal of Economic Surveys · doi:10.1111/joes.12608 · Cited by 27
André, Q., & Reinholtz, N. (2024) · Journal of Consumer Research · doi:10.1093/jcr/ucae028 · Cited by 19
The Problem of Biased Data and Potential Solutions for Health and Environmental Assessments
Suter, G. W., & Cormier, S. M. (2014) · Human and Ecological Risk Assessment An International Journal · doi:10.1080/10807039.2014.974499 · Cited by 17
“Evidence of these questionable practices leading to publication bias is seen in the fact that published P values cluster just below 0.05 (Simonsohn et al. 2013).”
BackgroundInfluentialThe distribution of probability values in medical abstracts: an observational study
Ginsel, B. L., Aggarwal, A., Xuan, W., & Harris, I. A. (2015) · BMC Research Notes · doi:10.1186/s13104-015-1691-x · Cited by 15
“They declare that a right skewed p-curve is evidence of biased analysis or selective reporting [4].”
BackgroundREES: A Registry of Efficacy and Effectiveness Studies in Education
Anderson, D., Spybrook, J. K., & Maynard, R. A. (2018) · Educational Researcher · doi:10.3102/0013189x18810513 · Cited by 16
Prevalence of Publication Bias Tests in Speech, Language, and Hearing Research
Chow, J. C. (2018) · Journal of Speech Language and Hearing Research · doi:10.1044/2018_jslhr-l-18-0098 · Cited by 15
Standard Errors in Quantitative Criminology: Taking Stock and Looking Forward
Sweeten, G. (2020) · Journal of Quantitative Criminology · doi:10.1007/s10940-020-09463-9 · Cited by 19
Compliance With Standards for STARD 2015 Reporting Recommendations in Pathology
Hogan, K. O., & Fraga, G. R. (2020) · American Journal of Clinical Pathology · doi:10.1093/ajcp/aqaa103 · Cited by 19
The Academic–Industrial Complexity: Failure to Launch
Levin, L. A., & Béhar‐Cohen, F. (2017) · Trends in Pharmacological Sciences · doi:10.1016/j.tips.2017.10.003 · Cited by 15
“from incorrect interpretation, “p hacking” [11], or falsification; (2) the experimental data from the industry side are not true; (3) both sets of data are true, but different, because experimental models, reagents, parameters, instruments, or environments differ between the two; or (4) the data are reproduced accurat…”
BackgroundDetecting Evidential Value and p-Hacking With the p-Curve Tool: A Word of Caution
Erdfelder, E., & Heck, D. W. (2019) · Zeitschrift für Psychologie · doi:10.1027/2151-2604/a000383 · Cited by 22
Crane, H. (2017) · SSRN Electronic Journal · doi:10.2139/ssrn.3074083 · Cited by 15
“Claim (ii) ignores the dependence of ψα on the significance cutoff α, as supported by empirical studies [9, 14, 20] and common sense (i.”
Ferguson, C. J., & Smith, S. (2024) · Aggression and Violent Behavior · doi:10.1016/j.avb.2023.101905 · Cited by 14
“Publication bias was assessed with tools including basic funnel plot analysis, Trim and Fill (Fern ´ andez-Castilla et al., 2021), PET/PEESE (Barto ˇ s et al., 2022), and p -curve (Simonsohn et al., 2014).”
MethodNo meta-analytical effect of economic inequality on well-being or mental health
Sommet, N., Fillon, A., Rudmann, O., Cunha, A. S., & Ehsan, A. (2025) · Nature · doi:10.1038/s41586-025-09797-z · Cited by 15
When Shooting for the Stars Becomes Aiming for Asterisks: P -Hacking in Family Business Research
Brinkerink, J. (2021) · Entrepreneurship Theory and Practice · doi:10.1177/10422587211050354 · Cited by 16
“10 To assess the presence of p -hacking in family business research, we will fi rst examine the so-called p -curve (Simonsohn et al., 2014): a histogram displaying the empirical distribution of the identi fi ed theory-testing p -values in our sample.”
MethodResultMichael, H. U., Youbi, E., Ohadoma, S. C., Ramlall, S., Oosthuizen, F., & Polyakova, M. (2021) · Neuropsychology Review · doi:10.1007/s11065-021-09527-y · Cited by 16
Blanch, À. (2022) · Educational Psychology Review · doi:10.1007/s10648-022-09670-9 · Cited by 23
“These plots highlight a right-skewed pattern with most values falling within the lowest range (0 to 0.01), suggesting that there was evidential value in the data as long as selective reporting could be dismissed as the sole explanation of the findings (Simonsohn et al., 2014).”
BackgroundInfluentialIs There Evidence of P-Hacking in Imaging Research?
Rooprai, P., Islam, N., Salameh, J., Ebrahimzadeh, S., Kazi, A., Frank, R., Ramsay, T., Mathur, M. B., Absi, M., Khalil, A. A., et al. (2022) · Canadian Association of Radiologists Journal · doi:10.1177/08465371221139418 · Cited by 18
“Further, P-values reported in abstracts are likely to be selected as relating to the most important findings or primary hypotheses, which are expected to be more strongly phacked than P-values related to secondary objectives.(4) Indeed, some empirical evidence suggests that P-values reported in abstracts are substanti…”
BackgroundTempest in a teacup: An analysis of p-Hacking in organizational research
Gupta, A., & Bosco, F. A. (2023) · PLoS ONE · doi:10.1371/journal.pone.0281938 · Cited by 19
“Commonly studied QRPs include hypothesizing after results are known (HARKing; [3,4]), contingent inclusion or exclusion of influential observations, and p-hacking [5].”
BackgroundInverse publication reporting bias favouring null, negative results
Ioannidis, J. P. A. (2023) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2023-112292 · Cited by 16
Akhtar, S. M. M., Ali, A., Khan, M. S., Хан, В. В., Fareed, A., Saleem, S. Z., Mumtaz, M., Ahsan, M. N., Iqbal, S., & Asghar, M. S. (2024) · International Journal of Gynecology & Obstetrics · doi:10.1002/ijgo.15467 · Cited by 16
“P - curve analysis, proposed by Simonsohn et al. 12–14 serves as a method to examine the distribution of P - values and to detect P - hacking in meta-analyses.”
ROCZNIKI PSYCHOLOGICZNE / ANNALS OF PSYCHOLOGY
Soroko, E., & Psychologii, I. (2013) · Cited by 14
A Theory of Statistical Inference for Ensuring the Robustness of Scientific Results
Coker, B., Rudin, C. D., & King, G. (2021) · Management Science · doi:10.1287/mnsc.2020.3818 · Cited by 42
“Problems with Replication of Scientific Studies and Proposed Solutions The evidence for p-hacking primarily comes from two types of meta-analyses: replication studies and the distribution of p-values for a set of independent findings (or “p-curve”) (Simonsohn et al. 2014).”
MethodA Meta-Analysis of Infants' Mispronunciation Sensitivity Development.
Holzen, K. V., & Bergmann, C. (2018) · PubMed · Cited by 14
Pitfalls in interpretation of large registry data on hernia repair
Schwáb, R., Dietz, U. A., Menzel, S., & Wiegering, A. (2018) · Hernia · doi:10.1007/s10029-018-1837-6 · Cited by 14
Making Research Data Accessible
Kapiszewski, D., & Karcher, S. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108762519.008 · Cited by 16
Baron, H., & Young, L. (2021) · Political Science Research and Methods · doi:10.1017/psrm.2021.48 · Cited by 13
“…of methodological innovation and research, just as other forms of research transparency have become in recent years as scholars have developed better methods for detecting p-hacking, preregistration, and replication (Humphreys et al., 2013; Simonsohn et al., 2014; Open Science Collaboration, 2015).”
An empirically based power primer for laboratory aggression research
Hyatt, C. S., Crowe, M. L., West, S. J., Vize, C. E., Carter, N. T., Chester, D. S., & Miller, J. D. (2021) · Aggressive Behavior · doi:10.1002/ab.21996 · Cited by 15
“Importantly, this does not represent the raw mean effect size observed across these studies, but rather the effect size that best fits the observed data after accounting for publication bias given the estimates of the p -curve (see Simonsohn et al., 2014b).”
BackgroundMethodA very simple, re-executable neuroimaging publication
Ghosh, S., Poline, J., Keator, D. B., Halchenko, Y. O., Thomas, A. G., Kessler, D. A., & Kennedy, D. N. (2017) · F1000Research · doi:10.12688/f1000research.10783.2 · Cited by 13
“…, 2013; Ioannidis, 2005); P-hacking, trying various statistical approaches in order to find analyses that reach significance (Simmons et al. , 2011; Simonsohn et al. , 2014); completeness of methods description, the written text of a publication cannot completely describe an analytic method in its…”
Wenzel, M., Lind, M., Rowland, Z., Zahn, D., & Kubiak, T. (2019) · Social Psychology · doi:10.1027/1864-9335/a000365 · Cited by 14
Wielding a gun increases judgments of others as holding guns: a randomized controlled trial
Witt, J. K., Parnes, J. E., & Tenhundfeld, N. L. (2020) · Cognitive Research Principles and Implications · doi:10.1186/s41235-020-00260-3 · Cited by 15
Ausmaß und Risikofaktoren des Publication Bias in der deutschen Soziologie
Auspurg, K., Hinz, T., & Schneck, A. (2014) · KZfSS Kölner Zeitschrift für Soziologie und Sozialpsychologie · doi:10.1007/s11577-014-0284-3 · Cited by 20
Questionable Research Practices and Open Science in Quantitative CriminologyPreprint
Chin, J. M., Pickett, J. T., Vazire, S., & Holcombe, A. O. (2021) · SocArXiv (OSF Preprints) · doi:10.31235/osf.io/bwm7s · Cited by 12
Advancing Meta‐Analytic Methods in L2 Research
Plonsky, L., Hu, Y., Sudina, E., & Oswald, F. L. (2023) · Wiley eBooks · doi:10.1002/9781394259670.ch14 · Cited by 20
Heyard, R., Pawel, S., Frese, J., Voelkl, B., Würbel, H., McCann, S., Held, L., Wever, K. E., Hartmann, H., Townsin, L., et al. (2025) · Royal Society Open Science · doi:10.1098/rsos.242076 · Cited by 20
Behavioral Labeling: Prompting Consumer Behavior Through Activity Tags
Fritze, M. P., Völckner, F., & Melnyk, V. (2023) · Journal of Marketing · doi:10.1177/00222429231213011 · Cited by 15
“We provide p -curve analyses along with a p -curve disclosure table (Simonsohn, Nelson, and Simmons 2014) and post hoc power calculations in Web Appendix I. Data and codes for all studies can be accessed at https://osf.io/4qwsu/?view_only = 1dbf66c8b35542019383cfbd2c4dbd65.”
MethodReciprocal altruism in Norway rats
Engelhardt, S. C., & Taborsky, M. (2023) · Ethology · doi:10.1111/eth.13418 · Cited by 14
Simson, J., Pfisterer, F., & Kern, C. A. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3658974 · Cited by 41
“They closely mirror practices of varying evaluation criteria to achieve significant p-values, a practice commonly referred to as “p-hacking”, which gave rise to the introduction of multiverse analysis in psychology in the first place [52].”
BackgroundAnything goes for participant, patient and public involvement in youth mental health research
Bakermans‐Kranenburg, M. J., & van IJzendoorn, M. H. (2024) · JCPP Advances · doi:10.1002/jcv2.12258 · Cited by 14
Lehmann, R., & Vogt, B. (2024) · Brain Informatics · doi:10.1186/s40708-024-00232-z · Cited by 16
Ma, M., Li, J., Song, J., & Chen, X. (2024) · Communications Earth & Environment · doi:10.1038/s43247-024-01786-w · Cited by 16
“Moreover, this paper employs the method of creating new variables for breakpoint regression to further verify the U-curve relationship (Supplementary Method 3) 34 .”
Replicability and the Psychology of Science
Clark, C. J., Honeycutt, N., & Jussim, L. J. (2022) · Springer eBooks · doi:10.1007/978-3-031-04968-2_3 · Cited by 14
Preregistration and Registered Reports in Sociology: Strengths, Weaknesses, and Other Considerations
Manago, B. N. (2023) · The American Sociologist · doi:10.1007/s12108-023-09563-6 · Cited by 18
“In fact, if the effect and statistical power are large enough, we might even expect a higher rate of papers to be published at the p 0.01 than the p 0.05 level (Simonsohn et al., 2014).”
BackgroundInfluentialPromoting replicable sexual science: A methodological review and call for metascience
Sakaluk, J. K. (2016) · The Canadian Journal of Human Sexuality · doi:10.3138/cjhs.251-co1 · Cited by 13
P-Hacking in Orthopaedic Literature: A Twist to the Tail
Razak, H. R. B. A., Ang, J.-G. E., Attal, H., Howe, T.-S., & Allen, J. C. (2016) · Journal of Bone and Joint Surgery · doi:10.2106/jbjs.16.00479 · Cited by 16
Information Aggregation and P-Hacking
Rytchkov, O., & Zhong, X. (2019) · Management Science · doi:10.1287/mnsc.2018.3259 · Cited by 14
Bruns, S. B., König, J., & Stern, D. I. (2018) · Energy Economics · doi:10.1016/j.eneco.2018.10.007 · Cited by 14
Publication bias: A “bird's-eye view” of meta-analytic practice in criminology and criminal justice
Kim, B. (2021) · Journal of Criminal Justice · doi:10.1016/j.jcrimjus.2021.101878 · Cited by 16
Waldman, I. D., & Lilienfeld, S. O. (2015) · Psychometrika · doi:10.1007/s11336-015-9447-z · Cited by 14
Toward a Radically Embodied Neuroscience of Attachment and Relationships?Preprint
Beckes, L., IJzerman, H., & Mattie Tops, A. (2014) · SSRN Electronic Journal · doi:10.2139/ssrn.2429522 · Cited by 13
Using Split Samples to Improve Inference about Causal Effects
Fafchamps, M., & Labonne, J. (2016) · National Bureau of Economic Research · doi:10.3386/w21842 · Cited by 21
p-Hacking and Publication Bias Interact to Distort Meta-Analytic Effect Size EstimatesPreprint
Friese, M., & Frankenbach, J. (2020) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/bvm58 · Cited by 15
Spurious Precision in Meta-Analysis of Observational ResearchPreprint
Iršová, Z., Bom, P. R. D., Havránek, T., & Rachinger, H. J. (2023) · MetArXiv (OSF Preprints) · doi:10.31222/osf.io/3qp2w · Cited by 21
Trusting autonomous vehicles as moral agents improves related policy support
Hurst, K. F., & Sintov, N. D. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.976023 · Cited by 14
The Reconnection Process: Mobilizing the Social Capital of Dormant Ties
Rondi, E., Levin, D. Z., & Massis, A. D. (2023) · Organization Science · doi:10.1287/orsc.2023.1685 · Cited by 12
Beyond statistical significance: Five principles for the new era of data analysis and reporting
Wedel, M., & Gal, D. (2023) · Journal of Consumer Psychology · doi:10.1002/jcpy.1379 · Cited by 13
Effectiveness of Flipped Classrooms for K–12 Students: Evidence From a Three-Level Meta-Analysis
Li, S., Fu, W., Liu, X., & Hwang, G. (2024) · Review of Educational Research · doi:10.3102/00346543241261732 · Cited by 39
“The statistically significant findings in the included studies contained evidential value (Simonsohn et al., 2014).”
Brohmer, H., Hofer, G., Bauch, S. A., Beitner, J., Berkessel, J. B., Corcoran, K., García, D., Gruber, F. M., Giuliani, F., Jauk, E., et al. (2024) · International Review of Social Psychology · doi:10.5334/irsp.522 · Cited by 21
Tan, L.-P., Gong, S., Wang, Y., Guo, X., Xu, X., & Wang, Y. (2025) · Educational Psychology Review · doi:10.1007/s10648-025-10001-x · Cited by 25
“Finally, we utilized the p -curve method (Simonsohn et al., 2014) to assess the potential for publication bias.”
Kommattam, P., Jonas, K. J., & Fischer, A. H. (2017) · Emotion · doi:10.1037/emo0000261 · Cited by 16
Wege aus der Vertrauenskrise: Individuelle Schritte hin zu verlässlicher und offener Forschung
Geukes, K., Schönbrodt, F. D., Utesch, T., Geukes, S., & Back, M. D. (2016) · Zeitschrift für Sportpsychologie · doi:10.1026/1612-5010/a000167 · Cited by 15
Cross-Study, Cross-Method Associations Between Negative Urgency and Internalizing Symptoms
King, K. M., Halvorson, M. A., Kuehn, K. S., Feil, M. C., & Lengua, L. J. (2021) · Assessment · doi:10.1177/1073191120983889 · Cited by 15
“Calls for replication, confirmation, and extension of past empirical findings have been at the forefront of psychological science in recent years (Aarts et al., 2015; Ledgerwood, 2014; Mischel, 2006; Simonsohn et al., 2014).”
Information markets and nonmarkets
Bergemann, D., & Ottaviani, M. (2021) · Handbook of Industrial Organization · doi:10.1016/bs.hesind.2021.11.008 · Cited by 24
Measurement of Sexual Interests with Pupillary Responses: A Meta-Analysis
Attard-Johnson, J., Vasilev, M., Ciardha, C. Ó., Bindemann, M., & Babchishin, K. (2021) · Archives of Sexual Behavior · doi:10.1007/s10508-021-02137-y · Cited by 11
Is menu design effective? A p-curving analysis
Chark, R., & Michael Man Him, I. P. (2022) · International Journal of Hospitality Management · doi:10.1016/j.ijhm.2022.103378 · Cited by 12
Rigor, Replication, and Reproducibility: Increasing the Relevance of Behavioral Disorders Research
Gage, N. A., & Stevens, R. N. (2018) · Education and Treatment of Children · doi:10.1353/etc.2018.0029 · Cited by 13
May I have your attention, please? Methodological and analytical flexibility in the addiction stroop
Jones, A., Worrall, S., Rudin, L., Duckworth, J. J., & Christiansen, P. (2021) · Addiction Research & Theory · doi:10.1080/16066359.2021.1876847 · Cited by 13
“However, according to (Simonsohn et al. 2014) if selective reporting or p-hacking exists in the literature (in this case due to the large number of analysis decisions) then the p-curve should be left skewed (a greater number of p-values .”
BackgroundHensel, P. G. (2021) · Accountability in Research · doi:10.1080/08989621.2021.1981870 · Cited by 17
“These standards are intended to reduce the use of questionable research practices (Banks et al. 2016; Butler, Delaney, and Spoelstra 2017; Baum and Bromiley 2019; Bruns and Kalthaus 2020; Bedeian, Taylor, and Miller 2010) such as p-hacking (Simonsohn, Nelson, and Simmons 2014).”
BackgroundInterpreting confidence intervals: A comment on Hoekstra, Morey, Rouder, and Wagenmakers (2014)
Miller, J., & Ulrich, R. (2016) · Psychonomic Bulletin & Review · doi:10.3758/s13423-015-0859-7 · Cited by 12
“4, Tübingen 72076, Germany replicability of published findings (e.g., Carpenter, 2012; Francis, 2013; Open Science Collaboration, 2012; Pashler and Harris, 2012; Simonsohn, Nelson, and Simmons 2014).”
Chen, L., Benjamin, R., Lai, A., & Heine, S. (2022) · doi:10.31234/osf.io/kuhy6 · Cited by 14
“However, we note that Simonsohn et al. (2014a), in designing the �atness test, suggested using 33% power as the cutoff to decide whether the power of a body of literature is su�cient.”
BackgroundMethodInfluentialGu, Y., & Cheng, L. (2026) · Frontiers in Psychology · doi:10.3389/fpsyg.2025.1716745 · Cited by 19
Recommendations to improve use and reporting of statistics in animal experiments
Rowe, A. (2022) · Laboratory Animals · doi:10.1177/00236772221140669 · Cited by 15
“It includes p-hacking, which refers to applying multiple statistical analyses and sub-analyses until uncovering a statistically significant result without clearly reporting how it was obtained.(42) It is also known as data dredging, snooping, fishing, significance chasing and double dipping.”
MethodAkhtar, S. M. M., Fareed, A., Afzal, U., Asghar, M. S., Mumtaz, M., Faraz, F., Naeem, H., & Vaid, R. (2024) · The American Journal of Surgery · doi:10.1016/j.amjsurg.2024.01.032 · Cited by 13
Zabat, M. A., Giakas, A. M., Hohmann, A. L., & Lonner, J. H. (2024) · The Journal of Arthroplasty · doi:10.1016/j.arth.2024.01.044 · Cited by 11
Wegener, D. T., Pek, J. C. X., & Fabrigar, L. R. (2024) · PLoS ONE · doi:10.1371/journal.pone.0307999 · Cited by 13
“Manapat and colleagues use the QRP terminology to refer to situations in which people undertake such practices as a way to enhance the significance of their results (aka p-hacking) [10] or when they use a method known to have subpar performance [9].”
MethodAcademic Cheating, Achievement Orientations, and Culture Values: A Meta-Analysis
Zhao, L., Yang, X., Yu, X., Zheng, J., Mao, H., Fu, G., Fang, F., & Lee, K. (2024) · Review of Educational Research · doi:10.3102/00346543241288240 · Cited by 23
“Because some researchers might selectively report significant results, or some researchers may keep collecting data until the effect of the result is significant (e.g., p -hacking), the p -curve analysis has been developed to address this issue (Simonsohn et al., 2014).”
Adam‐Troian, J., Çelebi, E. Y., Bonetto, É., Taşdemir, N., & Yurtbakan, T. (2020) · International Journal of Intercultural Relations · doi:10.1016/j.ijintrel.2020.05.002 · Cited by 13
Huang, Q. E., Peng, W., & Simmons, J. V. (2020) · Group Processes & Intergroup Relations · doi:10.1177/1368430220957081 · Cited by 14
“Thus, these results did not indicate that the evidential value of studies in Group A was inadequate (Simonsohn et al., 2014a; Simonsohn et al., 2015).”
BackgroundMethodResultInfluentialSelf-citationKim, J., Kim, S., Jhang, J., Lee, J., & Koo, C. (2023) · Tourism Management · doi:10.1016/j.tourman.2023.104741 · Cited by 15
Machine Learning for Experiments in the Social Sciences
Green, J., & Mark H. White, II (2023) · Cambridge University Press eBooks · doi:10.1017/9781009168236 · Cited by 19
Metadata
- Authors
- Simonsohn, U., Nelson, L. D., & Simmons, J. P.
- Affiliations
- University of Pennsylvania (US)
- Venue
- Journal of Experimental Psychology: General
- Published
- 2014-04
- Volume
- 143
- Issue
- 2
- Pages
- 534-547
- ISSN
- 0096-3445 (ISSN-L)
- 1939-2222
- OpenAlex
- W3124690569
- Cited by
- 1,500
- References
- 49
- Citations by year
- 2026: 75 · 2025: 96 · 2024: 110 · 2023: 121 · 2022: 146 · 2021: 157 · 2020: 158 · 2019: 154 · 2018: 143 · 2017: 113 · 2016: 103 · 2015: 80 · 2014: 35
- FWCI
- 82.62
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Experimental Psychology General (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Keywords
- p-curve, publication bias, selective reporting, p-hacking, file drawer problem, statistical significance, evidentiary value
- MeSH terms
- Data Interpretation, Statistical, Humans, Psychology, Experimental, Statistics as Topic, Reproducibility of Results, Models, Statistical, Publication Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.