Empirical assessment of published effect sizes and power in the recent cognitive neuroscience and psychology literature
Open accessPublished version of a preprint
The DOI matches its Crossref record; 62 of the 62 references it cites were found in a registry or an index.
Abstract
We have empirically assessed the distribution of published effect sizes and estimated power by analyzing 26,841 statistical records from 3,801 cognitive neuroscience and psychology papers published recently. The reported median effect size was D = 0.93 (interquartile range: 0.64-1.46) for nominally statistically significant results and D = 0.24 (0.11-0.42) for nonsignificant results. Median power to detect small, medium, and large effects was 0.12, 0.44, and 0.73, reflecting no improvement through the past half-century. This is so because sample sizes have remained small. Assuming similar true effect sizes in both disciplines, power was lower in cognitive neuroscience than in psychology. Journal impact factors negatively correlated with power. Assuming a realistic range of prior probabilities for null hypotheses, false report probability is likely to exceed 50% for the whole literature. In light of our findings, the recently reported low replication success in psychology is realistic, and worse performance may be expected for cognitive neuroscience.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures





Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“Despite methodological challenges10–11, many of the findings of cognitive neuroscience provide a solid basis on which to build.”
From Cognitive computational neuroscience (2018) · cites this paper as [10]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In several disciplines, the average statistical power of studies was found to be significantly below the recommended 80% level (13 -16).”
From Is science really facing a reproducibility crisis, and do we need it to? (2018) · cites this paper as [14]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“such studies showed that power varies widely between subfields or methodologies”
From Is science really facing a reproducibility crisis, and do we need it to? (2018) · cites this paper as [14]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“For most published research findings to be false in psychology and neuroscience, for example, one must assume that the hypotheses tested in these disciplines are correct much less than 50% of the time (14, 18).”
From Is science really facing a reproducibility crisis, and do we need it to? (2018) · cites this paper as [14]
The cited source says what the claim attributes to it.
In the citing paper
“and has contributed, in part, to the replication crisis in other fields (Button et al., 2013; Szucs and Ioannidis, 2017; Ioannidis et al., 2017).”
From With Little Power Comes Great Responsibility (2020) · cites this paper as Szucs and Ioannidis, 2017
2 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
Self-citationEstimating the reproducibility of psychological science
Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515
Power failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3475 · Cited by 7,641
Self-citationSimmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
Methods of Meta-Analysis: Correcting Error and Bias in Research Findings
Schmidt, F. L., & Hunter, J. E. (2015) · doi:10.4135/9781483398105 · Cited by 5,542
Effect size estimates: Current use, calculations, and interpretation.
Fritz, C. O., Morris, P. E., & Richler, J. J. (2012) · Journal of Experimental Psychology: General · doi:10.1037/a0024338 · Cited by 5,199
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 9×
“…examined p value reporting errors in the cognitive neuroscience literature as recent studies found that 12.5% to 20% of psychology papers have major p value reporting errors which affect judgments about statistical significance (Bakker and Wicherts, 2011; Nuijten et al. 2015; Veldkamp et al. 2014).”
BackgroundMethodResultInfluentialSelf-citationCluster failure: why fMRI inferences for spatial extent have inflated false-positive rates
Eklund, A., Nichols, T. E., & Knutsson, H. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1602413113 · Cited by 3,487
Promoting an open research culture
Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., Buck, S., Chambers, C. D., Chin, G., Christensen, G., et al. (2015) · Science · doi:10.1126/science.aab2374 · Cited by 2,708
Circular analysis in systems neuroscience: the dangers of double dipping
Kriegeskorte, N., Simmons, W. K., Bellgowan, P. S. F., & Baker, C. I. (2009) · Nature Neuroscience · doi:10.1038/nn.2303 · Cited by 2,696
Cited by
Items 301–400 of 787
Pinho, J., Cunliffe, V. T., Kareklas, K., Petri, G., & Oliveira, R. F. (2023) · Communications Biology · doi:10.1038/s42003-023-04999-5 · Cited by 18
Drawing causal inferences from randomized controlled trials in psychotherapy research
Kazdin, A. E. (2023) · Psychotherapy Research · doi:10.1080/10503307.2022.2130112 · Cited by 16
“Sufficient sample size is not trivial because weak power to detect small and medium effects is a rather common problem (e.g., Götz et al., 2022; Szucs & Ioannidis, 2017).”
BackgroundThe Brain Disorders Debate, Chekhov, and Mental Health Humanities
Valtonen, J., & Lewis, B. E. (2023) · Journal of Medical Humanities · doi:10.1007/s10912-023-09786-1 · Cited by 17
van Woudenberg, T. J., Rozendaal, E., & Buijzen, M. (2023) · International Journal of Social Research Methodology · doi:10.1080/13645579.2023.2222539 · Cited by 17
Chapter 4. Psychoneuroendocrinology
Sforzini, L., Weston, F. I., & Pariante, C. M. (2024) · American Psychiatric Association Publishing eBooks · doi:10.1176/appi.books.9781615375523.as04 · Cited by 16
Charting a Future for fMRI in Communication Science
Turner, B. O., Huskey, R., & Weber, R. (2018) · Communication Methods and Measures · doi:10.1080/19312458.2018.1520823 · Cited by 16
“…et al., 2008) aspects of fMRI analysis, as well as pointing out the widespread lack of power (and therefore, replicability) that plagues most fMRI studies (Button et al., 2013; Cremers, Wager, & Yarkoni, 2017; Szucs & Ioannidis, 2017; Turner, Paul, Miller, & Barbey, 2018), appearing almost monthly.”
Reito, A. R., Raittio, L., & Helminen, O. (2019) · PeerJ · doi:10.7717/peerj.6813 · Cited by 16
Carbine, K. A., Lindsey, H. M., Rodeback, R. E., & Larson, M. J. (2019) · International Journal of Psychophysiology · doi:10.1016/j.ijpsycho.2019.06.004 · Cited by 16
Sundman, M., Lim, K., That, V. T., Mizell, J.-M., Ugonna, C., Rodriguez, R., Chen, N., Fuglevand, A. J., Liu, Y., Wilson, R. C., et al. (2020) · Brain Communications · doi:10.1093/braincomms/fcaa203 · Cited by 17
Re-Analyzing Phase III Bremelanotide Trials for “Hypoactive Sexual Desire Disorder” in Women
Spielmans, G. I. (2021) · The Journal of Sex Research · doi:10.1080/00224499.2021.1885601 · Cited by 17
“…in other fields, including psychiatric genetics (Border et al., 2019), psychiatric gene x environment interaction research (Duncan & Keller, 2011), structural brain-behavior associations (Masouleh et al., 2019), cognitive neuroscience (Szucs et al., 2017), and economics (Camerer et al., 2016).”
BackgroundReplicability Crisis and Scientific Reforms: Overlooked Issues and Unmet Challenges
Andreoletti, M. (2020) · International Studies in the Philosophy of Science · doi:10.1080/02698595.2021.1943292 · Cited by 11
“Then, according to this narrative, replication failure stems from the very simple fact that many published results could possibly be false (Szucs and Ioannidis 2017).”
BackgroundCampbell, D. S., & Reiman, A. (2021) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2021.104255 · Cited by 16
Orekhova, E. V., Rostovtseva, E. N., Manyukhina, V. O., Prokofiev, A. O., Obukhova, T. S., Nikolaeva, A. Y., Schneiderman, J. F., & Stroganova, T. A. (2020) · NeuroImage · doi:10.1016/j.neuroimage.2020.116753 · Cited by 16
Internal consistency and test–retest reliability of an affective task-switching paradigm.
Eckart, C., Kraft, D., & Fiebach, C. J. (2021) · Emotion · doi:10.1037/emo0000972 · Cited by 17
Limited evidence for the effect of red color on cognitive performance: A meta-analysis
Gnambs, T. (2020) · Psychonomic Bulletin & Review · doi:10.3758/s13423-020-01772-1 · Cited by 16
“Thus, color psychology seems to suffer from similar power problems to many other psychological fields (e.g., Lamberink et al., 2018; Szucs & Ioannidis, 2017).”
BackgroundJaljuli, I., Kafkafi, N., Giladi, E., Golani, I., Gozes, I., Chesler, E. J., Bogue, M. A., & Benjamini, Y. (2023) · PLoS Biology · doi:10.1371/journal.pbio.3002082 · Cited by 15
“The first one is increas- ing the sample size, as argued by Szucs and Ioannidis [31].”
BackgroundBoot, E., Levy, A. D., Gaeta, G., Gunasekara, N., Parkkinen, E., Kontaris, E., Jacquot, M., & Tachtsidis, I. (2024) · Frontiers in Neuroscience · doi:10.3389/fnins.2024.1266664 · Cited by 17
Demidenko, M. I., Mumford, J. A., & Poldrack, R. A. (2024) · Imaging Neuroscience · doi:10.1162/imag_a_00262 · Cited by 18
“The median reported sample size in fMRI is <30 subjects ( Poldrack et al., 2017; Szucs & Ioannidis, 2017).”
BackgroundEffects of one session of theta or high alpha neurofeedback on EEG activity and working memory
Chikhi, S., Matton, N., Sanna, M., & Blanchet, S. (2024) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-024-01218-4 · Cited by 15
“This often results in an inflation of effect sizes and low reproducibility of results (Button et al., 2013; Szucs & Ioannidis, 2017; Thibault & Pedder, 2022).”
Koscelny, S. N., Sadralashrafi, S., & Neyens, D. M. (2025) · Applied Ergonomics · doi:10.1016/j.apergo.2025.104515 · Cited by 16
An umbrella review of effect size, bias, and power across meta‐analyses in emergency medicine
Parish, A. J., Yuan, D. M. K., Raggi, J. R., Omotoso, O. O., West, J. R., & Ioannidis, J. P. A. (2021) · Academic Emergency Medicine · doi:10.1111/acem.14312 · Cited by 15
Self-citationRecent advances in understanding neural correlates of anxiety disorders in children and adolescents
Zugman, A., Winkler, A. M., & Pine, D. S. (2021) · Current Opinion in Psychiatry · doi:10.1097/yco.0000000000000743 · Cited by 15
An empirically based power primer for laboratory aggression research
Hyatt, C. S., Crowe, M. L., West, S. J., Vize, C. E., Carter, N. T., Chester, D. S., & Miller, J. D. (2021) · Aggressive Behavior · doi:10.1002/ab.21996 · Cited by 15
On the Interpretation of Giving in Dictator Games When the Recipient is a Charity
Livingston, J. A., & Rasulmukhamedov, R. (2023) · Journal of Economic Behavior & Organization · doi:10.1016/j.jebo.2023.01.025 · Cited by 15
Klau, S., Schönbrodt, F. D., Patel, C. J., Ioannidis, J. P. A., Boulesteix, A., & Hoffmann, S. (2023) · Meta-Psychology · doi:10.15626/mp.2020.2556 · Cited by 18
Self-citationA biopsychological–social view of mathematical development
Szűcs, D., & Mammarella, I. C. (2023) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2023.101332 · Cited by 18
“Hence, testing a few constructs with high statistical power likely results in detecting significant correlations [101,105].”
Chiou, R. Y.-C., & Ralph, M. A. L. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2017.12.085 · Cited by 17
Current PTSD symptomatology distorts memory for past symptoms
Nahleen, S., Nixon, R. D. V., & Takarangi, M. K. T. (2019) · Psychiatry Research · doi:10.1016/j.psychres.2019.02.052 · Cited by 16
Inflated false negative rates undermine reproducibility in task-based fMRIPreprint
Lohmann, G., Stelzer, J., Müller, K., Lacosse, E., Buschmann, T., Kumar, V., Grodd, W., & Scheffler, K. (2017) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/122788 · Cited by 16
“Two major factors contribute to this problem, namely inflated false positive rates [6] and lack of statistical power, i.e. inflated false negative rates [7–10].”
BackgroundDalmaijer, E. S. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2309.00866 · Cited by 15
“While it may be impossible to estimate effect sizes within each measured variable, the distribution of effect sizes within a field of study can be established empirically: Szucs and Ioannidis (2017) extracted 26841 reported statistics from 3801 papers in cognitive neuroscience and psychology.”
BackgroundMethodResultInfluentialMurphy, S. L., Merz, R., Reimann, L., & Fernández, A. (2025) · Royal Society Open Science · doi:10.1098/rsos.242167 · Cited by 16
“This is due both to the use of one-sided statistical tests (effects in the direction opposite to that hypothesized are always nonsignificant) and research having less than perfect statistical power to find what are theorized to often be small-to-medium effects [3].”
Introduction to Power Analysis
Bartlett, J. E. (2017) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.17605/osf.io/pcfvj · Cited by 15
Demidenko, M. I., Ip, K. I., Kelly, D. P., Constante, K., Goetschius, L. G., & Keating, D. P. (2021) · Cortex · doi:10.1016/j.cortex.2021.02.032 · Cited by 14
Causal effects of psychostimulants on neural connectivity: a mechanistic, randomized clinical trial
Wang, Y., Kessel, E. M., Lee, S., Hong, S., Raffanello, E., Hulvershorn, L. A., Margolis, A. E., Peterson, B. S., & Posner, J. E. (2022) · Journal of Child Psychology and Psychiatry · doi:10.1111/jcpp.13585 · Cited by 15
Expanding the scope, reach, and impact of evidence-based psychological treatments
Kazdin, A. E. (2022) · Journal of Behavior Therapy and Experimental Psychiatry · doi:10.1016/j.jbtep.2022.101744 · Cited by 16
Makowski, D., Sperduti, M., Pelletier, J., Blondé, P., Corte, V. L., Arcangeli, M., Zalla, T., Lemaire, S., Dokic, J., Nicolas, S., et al. (2019) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-018-00681-0 · Cited by 14
Computational optimization of associative learning experiments
Melinščak, F., & Bach, D. R. (2020) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1007593 · Cited by 16
“Optimizing experimental designs could also help address the recent concerns over underpowered studies in cognitive sciences [55]: the usuallyprescribed use of larger samples should be complemented with the use of better designed experiments.”
BackgroundInconsistency of findings due to low power: A structural MRI study of bilingualism
Munson, B. A., & Hernández, A. E. (2019) · Brain and Language · doi:10.1016/j.bandl.2019.104642 · Cited by 15
Smith, P. F. (2017) · PubMed · Cited by 15
Brunyé, T. T. (2020) · Journal of Cognitive Enhancement · doi:10.1007/s41465-020-00186-0 · Cited by 16
Neuroscout, a unified platform for generalizable and reproducible fMRI research
de la Vega, A. I., Rocca, R., Blair, R. W., Markiewicz, C. J., Mentch, J., Kent, J. D., Herholz, P., Ghosh, S., Poldrack, R. A., & Yarkoni, T. (2022) · eLife · doi:10.7554/elife.79277 · Cited by 16
Small effect size leads to reproducibility failure in resting-state fMRI studiesPreprint
Jia, X., Zhao, N., Barton, B., Burciu, R. G., Carrière, N., Cerasa, A., Chen, B., Chen, J., Coombes, S. A., Defebvre, L., et al. (2018) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/285171 · Cited by 14
Schneck, A. (2023) · PLoS ONE · doi:10.1371/journal.pone.0292717 · Cited by 16
“Statistical power has hardly increased over time since the pioneering study by Cohen [10: 150f.] that examined the state of statistical power in leading psychological journals, and is still inadequate in the social sciences [11, 12] but also in other disciplines like medicine [13–15].”
BackgroundMethodResultInfluentialClarifying the reliability paradox: poor measurement reliability attenuates group differences
Karvelis, P., & Diaconescu, A. O. (2025) · Frontiers in Psychology · doi:10.3389/fpsyg.2025.1592658 · Cited by 16
Giofrè, D., Boedker, I., Cumming, G., Rivella, C., & Tressoldi, P. E. (2022) · Behavior Research Methods · doi:10.3758/s13428-022-01993-3 · Cited by 13
“…practices, statistical problems (including low statistical power), and barriers to data sharing, including a lack of encouragement and training from journals and institutions to publicly share data (Agnoli et al., 2017; Houtkoop et al., 2018; Szucs & Ioannidis, 2017; Tressoldi & Giofrè, 2015).”
BackgroundLeipold, S., Oderbolz, C., Greber, M., & Jäncke, L. (2019) · International Journal of Psychophysiology · doi:10.1016/j.ijpsycho.2018.12.016 · Cited by 13
Muñoz‐Vergara, D., Berrueta, L., Carmody, C., An, X., Wayne, P. M., Zavacki, A. M., & Langevin, H. M. (2020) · American Journal of Physical Medicine & Rehabilitation · doi:10.1097/phm.0000000000001465 · Cited by 13
Chu, S.-H., Parhi, K. K., Schreiner, M. W., Lenglet, C., Mueller, B. A., Klimes‐Dougan, B., & Cullen, K. R. (2021) · Journal of Clinical Medicine · doi:10.3390/jcm10194322 · Cited by 13
Sample size determination for Bayesian ANOVAs with informative hypotheses
Fu, Q., Moerbeek, M., & Hoijtink, H. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.947768 · Cited by 13
The Best-Case Heuristic: Relative Optimism in Relationships, Politics, and a Global Health Pandemic
Sjåstad, H., & Bavel, J. J. V. (2023) · Personality and Social Psychology Bulletin · doi:10.1177/01461672231191360 · Cited by 13
Gronemann, D. C., Koch, K., Bantel, C., & Sörös, P. (2020) · medRxiv · doi:10.1101/2020.04.16.20068197 · Cited by 13
Null-hacking, a lurking problemPreprint
Protzko, J. (2018) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/9y3mp · Cited by 15
Mathur, M. B., & VanderWeele, T. J. (2019) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/p3xyd · Cited by 13
Feng, Q., Li, W., Li, W., Wang, R., Crispin, J., Longobardi, S., D’Hooghe, T., & Mol, B. W. (2025) · Human Reproduction · doi:10.1093/humrep/deaf022 · Cited by 13
“Thirdly, we chose a sample size of 50 articles for manual validation as it was practical, achieved high sensitivity and specificity, and aligned with validation methods commonly used by researchers working with large datasets (Chavalarias et al. , 2016; Szucs and Ioannidis, 2017).”
A simple model suggesting economically rational sample-size choice drives irreproducibility
Braganza, O. (2020) · PLoS ONE · doi:10.1371/journal.pone.0229615 · Cited by 12
“5]) and an empirically matched distribution of d [11].”
BackgroundMethodResultInfluentialPhysiological comfort evaluation under different airflow directions in a heating environment
Tamura, K., Matsumoto, S., Tseng, Y. H., Kobayashi, T., Miwa, J., Miyazawa, K., Matsumoto, S., Hiramatsu, S., Otake, H., & Okamoto, T. (2022) · Journal of PHYSIOLOGICAL ANTHROPOLOGY · doi:10.1186/s40101-022-00289-x · Cited by 14
“The typical sample size of neuroscience studies involving EEG measurements tends to be low because of the need for considerable time, investment, and resources per participant, as well as more complex measurement tools than required for behavioral studies [30, 31].”
BackgroundGibson, B. C., Claus, E. D., Sanguinetti, J., Witkiewitz, K. A., & Clark, V. P. (2022) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2022.104821 · Cited by 13
Instruction effects on randomness in sequence generation
Guseva, M., Bogler, C., Allefeld, C., & Haynes, J. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1113654 · Cited by 13
“Given the pervasive problem of underpowered studies in the psychology literature (Szucs and Ioannidis, 2017) and the ease of online data collection we decided to double the sample size.”
BackgroundPsychology Faculty Overestimate the Magnitude of Cohen’s d Effect Sizes by Half a Standard Deviation
Schuetze, B. A., & Yan, V. X. (2023) · Collabra Psychology · doi:10.1525/collabra.74020 · Cited by 16
Myles, D., Bennett, D., Carter, A., Yücel, M., Albertella, L., de Lacy‐Vawdon, C., & Livingstone, C. H. (2023) · Addictive Behaviors Reports · doi:10.1016/j.abrep.2023.100500 · Cited by 13
“…in the journal International Gambling Studies (Blaszczynski & Gainsbury, 2019; LaPlante, 2019; Wohl et al., 2019), alongside a number of highly publicised concerns about current rates of reproducible research in psychology and medicine (Open Science Collaboration, 2015; Szucs & Ioannidis, 2017).”
BackgroundCampbell, H., & Gustafson, P. (2019) · The American Statistician · doi:10.1080/00031305.2018.1555101 · Cited by 14
Koski, J. E., McHaney, J. R., Rigney, A. E., & Beer, J. S. (2020) · NeuroImage · doi:10.1016/j.neuroimage.2020.116752 · Cited by 12
Comparison of Two Lighting Assessment Methods when Reading with Low Vision
Henry, R., Duquette, J., & Wittich, W. (2020) · Optometry and Vision Science · doi:10.1097/opx.0000000000001499 · Cited by 12
Niederstrasser, N. G., & Cook, S. (2020) · Journal of Pain · doi:10.1016/j.jpain.2020.07.005 · Cited by 14
Hypnotic Suggestions Can Induce Rapid Change in Implicit Attitudes
Dessel, P. V., & Houwer, J. D. (2019) · Psychological Science · doi:10.1177/0956797619865183 · Cited by 12
What Neuroscience Can and Cannot Answer.
Choi, O. (2017) · PubMed · Cited by 18
Lafit, G., Artner, R., & Ceulemans, E. (2024) · Behavior Research Methods · doi:10.3758/s13428-024-02435-y · Cited by 13
Open Science als Beitrag zur Qualität in der Bildungsforschung
Krammer, G., & Svecnik, E. (2020) · Zeitschrift für Bildungsforschung · doi:10.1007/s35834-020-00286-z · Cited by 14
Mental travel in the person domain
Hayman, M., & Arzy, S. (2021) · Journal of Neurophysiology · doi:10.1152/jn.00695.2020 · Cited by 11
Impact of COVID-19 on Journal Impact Factor
Uddin, S., Khan, A. M., & Lu, H. (2023) · Journal of Informetrics · doi:10.1016/j.joi.2023.101458 · Cited by 13
Gossmann, A., Pezeshk, A. X., & Sahiner, B. (2018) · SPIE Medical Imaging: Image Perception, Observer Performance, and Technology Assessment · doi:10.1117/12.2293818 · Cited by 13
Is the Psychopathic Brain an Artifact of Coding Bias? A Systematic Review
Jalava, J., Griffiths, S. Y., Larsen, R. R., & Alcott, B. E. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.654336 · Cited by 13
“Second, there is a well-recognized set of biases favoring positive findings, which may be eliminating true null-findings from the literature (Vul et al., 2009; Wager et al., 2009; Button et al., 2013; Nugent et al., 2013; Szucs and Ioannidis, 2017; Vul and Pashler, 2017).”
BackgroundDemidenko, M. I., Huntley, E. D., Weigard, A. S., Keating, D. P., & Beltz, A. M. (2022) · Journal of Neuroscience Research · doi:10.1002/jnr.25005 · Cited by 11
Leong, M. Q., Yu, Z., Tsikandilakis, M., & Tong, E. M. W. (2023) · Evolutionary Behavioral Sciences · doi:10.1037/ebs0000333 · Cited by 12
The Impact of P -hacking on “Redefine Statistical Significance”
Crane, H. (2018) · Basic and Applied Social Psychology · doi:10.1080/01973533.2018.1474111 · Cited by 15
“(The choice of 80% power reflects the conventional “gold standard” suggested in many introductory statistics texts, whereas 50% more closely reflects empirical analyses of power in the scientific literature, e.g., Button et al., 2013; Maxwell, 2004; Szucs & Ioannidis, 2017.)”
BackgroundProgress toward openness, transparency, and reproducibility in cognitive neurosciencePreprint
Gilmore, R. O., Diaz, M. T., Wyble, B., & Yarkoni, T. (2017) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/5veew · Cited by 11
Dunleavy, D. J. (2020) · Thesis Commons (OSF Preprints) · doi:10.31237/osf.io/f2c7b · Cited by 11
Toffalini, E., Gambarota, F., Perugini, A., Girardi, P., Tobia, V., Altoè, G., Giofrè, D., Psicostat Core Team, & Feraco, T. (2024) · International Journal of Psychology · doi:10.1002/ijop.13246 · Cited by 10
Frei, N., Willinger, D., Haller, P., González, G. F., Pamplona, G. S. P., Haugg, A., Lutz, C. G., Coraj, S., Hefti, E., & Brem, S. (2025) · Journal of Neuroscience · doi:10.1523/jneurosci.1119-24.2025 · Cited by 10
“However, we acknowledge recent fi ndings suggesting that current standards in psychological studies may be underpowered, indicating potential for improvement in future research (Szucs and Ioannidis, 2017).”
Costa, C., Pezzetta, R., Toffalini, E., Grassi, M., Cona, G., Miniussi, C., Bauer, P. J., Borgomaneri, S., Brysbaert, M., Chambers, C., et al. (2025) · Behavior Research Methods · doi:10.3758/s13428-025-02705-3 · Cited by 15
“The (lack of) statistical power of studies is often addressed as one of the major problems in contemporary research in cognitive psychology and neuropsychology (Button et al., 2013; Cohen, 1962; Sedlmeier & Gigerenzer, 1992; Szucs & Ioannidis, 2017, 2020).”
Kurnianingsih, Y. A., & Mullette-Gillman, O. A. (2016) · Frontiers in Neuroscience · doi:10.3389/fnins.2016.00507 · Cited by 12
“…a highly specific analyses and within-study replication, safeguard Frontiers in Neuroscience | www.frontiersin.org 4 November 2016 | Volume 10 | Article 507 us against growing concerns of the rate of false-positive findings in the field of cognitive neuroscience (Szucs and Ioannidis, 2016).”
van der Werff, L., O’Shea, D. E., Healy, G. F., Buckley, F., Real, C., Keane, M. P., & Lynn, T. (2022) · Applied Psychology · doi:10.1111/apps.12437 · Cited by 10
An Interactionist Approach to Cognitive Debiasing
Bland, S. (2020) · Episteme · doi:10.1017/epi.2020.9 · Cited by 13
“Worse still, the average and median statistical power of psychological studies over the last 50 years has consistently been found to be below 50% (Cohen 1962; Sedlmeier and Gigerenzer 1989; Szucs and Ioannidis 2017).”
BackgroundVai, B., Cazzetta, S., Scalisi, R., Donati, A., Bechi, M., Poletti, S., Sforzini, L., Visintini, R., Maffei, C., & Benedetti, F. (2020) · Journal of Affective Disorders · doi:10.1016/j.jad.2020.09.060 · Cited by 11
The Cultural Evolution of Science
Wu, J., O’Connor, C., & Smaldino, P. E. (2023) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780198869252.013.78 · Cited by 14
The Creative Brain: Myths and Truths
Abraham, A. (2024) · The MIT Press eBooks · doi:10.7551/mitpress/14313.001.0001 · Cited by 20
West, S. J., Hyatt, C. S., Miller, J. D., & Chester, D. S. (2020) · Aggressive Behavior · doi:10.1002/ab.21937 · Cited by 13
Kossmeier, M., Vilsmeier, J. K., Dittrich, R., Fritz, T., Kolmanz, C., Plessen, C. Y., Slowik, A., Tran, U. S., & Voracek, M. (2019) · Zeitschrift für Psychologie · doi:10.1027/2151-2604/a000384 · Cited by 10
The New Statistics for Better Science: Ask How Much, How Uncertain, and What Else is KnownPreprint
Calin-Jageman, R. J., & Cumming, G. (2018) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/3mztg · Cited by 11
The Dangers of Small Samples and Insufficient Methodological Detail
Barch, D. M. (2022) · Schizophrenia Bulletin · doi:10.1093/schbul/sbac137 · Cited by 10
Saunders, B., & Inzlicht, M. (2021) · International Journal of Psychophysiology · doi:10.1016/j.ijpsycho.2021.01.018 · Cited by 9
“statistical power is low, as is likely true for the average ERP study (Clayson et al., 2019; Clayson et al., 2020; Szucs and Ioannidis, 2017).”
BackgroundAkeman, E., White, E. J., Wolitzky-Taylor, K. B., Santiago, J., McDermott, T. J., DeVille, D. C., Stewart, J. L., Paulus, M. P., Taylor, C. T., & Aupperle, R. L. (2021) · Behavior Modification · doi:10.1177/01454455211030506 · Cited by 10
“It is important to recognize that effect sizes observed in the present study may be inflated due to the small sample size and the lack of a comparison intervention (Szucs & Ioannidis, 2017).”
BackgroundA tutorial for calculating field-specific effect size distributions
Glaser, B. D., Kang, H., Audunsdottir, K., Sartorius, A. M., & Quintana, D. S. (2026) · Behavior Research Methods · doi:10.3758/s13428-026-03003-2 · Cited by 11
A cautionary note against selective applications of the Bayes factor.
Schreiner, M. R., & Kunde, W. (2024) · Journal of Experimental Psychology General · doi:10.1037/xge0001666 · Cited by 8
Nudging effect of default options: A meta-analysis
Zhao, N., Liu, X., Li, S., & Zheng, R. (2022) · Advances in Psychological Science · doi:10.3724/sp.j.1042.2022.01230 · Cited by 12
Bayesian Model Selection Maps for Group Studies Using M/EEG Data
Harris, C. D., Rowe, E., Randeniya, R., & Garrido, M. I. (2018) · Frontiers in Neuroscience · doi:10.3389/fnins.2018.00598 · Cited by 9
“Furthermore, a replication study such as this one has the additional benefit of being responsive to the persisting replication crisis that continues to pose a significant problem for neuroscience and psychology (Hartshorne, 2012; Larson and Carbine, 2017; Szucs et al., 2017).”
BackgroundCórdova‐Palomera, A., & Priest, J. R. (2019) · Scientific Reports · doi:10.1038/s41598-019-52969-x · Cited by 9
Five Models of Science, Illustrating How Selection Shapes Methods
Smaldino, P. E. (2022) · University of Pittsburgh Press eBooks · doi:10.2307/j.ctv31djr2f.5 · Cited by 10
A Bayes Factor for Replications of ANOVA ResultsPreprint
Harms, C. (2018) · OSF Preprints (OSF Preprints) · Cited by 9
Visual gamma oscillations predict sensory sensitivity in females as they do in males
Manyukhina, V. O., Rostovtseva, E. N., Prokofyev, A. O., Obukhova, T. S., Schneiderman, J. F., Stroganova, T. A., & Orekhova, E. V. (2021) · Scientific Reports · doi:10.1038/s41598-021-91381-2 · Cited by 9
Metadata
- Authors
- Szucs, D., & Ioannidis, J. P. A.
- Affiliations
- Stanford Medicine (US)
- Venue
- PLOS Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2017-03-02
- Volume
- 15
- Issue
- 3
- Pages
- e2000797
- ISSN
- 1544-9173 (ISSN-L)
- 1545-7885
- PMC
- PMC5333800
- OpenAlex
- W2951081425
- Cited by
- 761
- References
- 62
- Citations by year
- 2026: 28 · 2025: 43 · 2024: 120 · 2023: 75 · 2022: 80 · 2021: 76 · 2020: 94 · 2019: 102 · 2018: 91 · 2017: 46
- FWCI
- 81.43
- Type
- journal article
- Language
- en
- Update notices
- correction (2021-03-05)
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS Biology (publishedVersion, cc-by)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Figshare (submittedVersion, cc-by)
- Apollo (University of Cambridge) (submittedVersion, cc-by)
- Apollo (cc-by)
- Funders
- James S. McDonnell Foundation (220020370)
- Universiteit van Tilburg
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Keywords
- effect size, statistical power, cognitive neuroscience, psychology literature, sample size, replication success, journal impact factor, null hypothesis, publication bias, statistical records
- MeSH terms
- Cognitive Neuroscience, Humans, Probability, Psychology, Publications, Surveys and Questionnaires, Empirical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.