Do studies of statistical power have an effect on the power of studies?
Open accessThe DOI matches its Crossref record; 54 of the 55 references it cites were found in a registry or an index.
Abstract
The long-term impact of studies of statistical power is investigated using J. Cohen's (1962) pioneering work as an example. We argue that the impact is nil; the power of studies in the same journal that Cohen reviewed (now the Journal of Abnormal Psychology) has not increased over the past 24 years. In 1960 the median power (i.e., the probability that a significant result will be obtained if there is a true effect) was.46 for a medium size effect, whereas in 1984 it was only.37. The decline of power is a result of alpha-adjusted procedures. Low power seems to go unnoticed: only 2 out of 64 experi-ments mentioned power, and it was never estimated. Nonsignificance was generally interpreted as confirmation of the null hypothesis (if this was the research hypothesis), although the median power was as low as.25 in these cases. We discuss reasons for the ongoing neglect of power. Since J. Cohen's (1962) classical study on the statistical power of the studies published in the 1960 volume of the Journal of Abnormal and Social Psychology, a number of power analyses have been performed. These studies exhorted researchers to pay attention to the power of their tests rather than to focus exclu-sively on the level of significance. Historically, the concept of
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“A little over two decades after Cohen’s original paper, two meta-analyses by Sedlmeier & Gigerenzer [66] and Rossi [67] examined a total of 25 reviews of statistical power in the psychological and social science literature between 1960 and 1984.”
From The natural selection of bad science (2016) · cites this paper as [66]
In the citing paper
“In fact, the average power of studies published in the 1984 volume of the Journal of Abnormal Psychology (a successor to the JASP) had dropped slightly compared with Cohen's (1962) results.”
From GPOWER: A general power analysis program (1996) · cites this paper as Sedlmeier and Gigerenzer (1989)
In the citing paper
“Sedlmeier and Gigerenzer (1989) investigated the impact of power analysis studies and textbooks on the power of recent psychological studies. Surprisingly, these authors found no significant increase in power values since 1962 when Cohen published his power study of the 1960 volume of the Journal of Abnormal & Social Psychology (JASP).”
From GPOWER: A general power analysis program (1996) · cites this paper as Sedlmeier and Gigerenzer (1989)
Citations
Cites
Statistical power analysis for the behavioral sciences
(1990) · Computers, Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,269
Statistical Power Analysis for the Behavioral Sciences (2nd ed.).
Lachenbruch, P. A., & Cohen, J. (1989) · Journal of the American Statistical Association · doi:10.2307/2290095 · Cited by 37,395
Fisher, R. A. (1936) · BMJ · doi:10.1136/bmj.1.3923.554-a · Cited by 4,980
Fundamental Statistics in Psychology and Education.
Smith, B. B., & Guilford, J. P. (1951) · Journal of the Royal Statistical Society. Series A (General) · doi:10.2307/2980908 · Cited by 3,455
Belief in the law of small numbers.
Tversky, A., & Kahneman, D. (1971) · Psychological Bulletin · doi:10.1037/h0031322 · Cited by 3,246
The statistical power of abnormal-social psychological research: A review.
Cohen, J. (1962) · The Journal of Abnormal and Social Psychology · doi:10.1037/h0045186 · Cited by 1,824
Freiman, J. A., Chalmers, T. C., Smith, H., & Kuebler, R. R. (1978) · New England Journal of Medicine · doi:10.1056/nejm197809282991304 · Cited by 1,628
Statistical Methods and Scientific Inference.
Bartlett, M. S., & Fisher, R. A. (1957) · Biometrika · doi:10.2307/2333276 · Cited by 1,278
Consequences of prejudice against the null hypothesis.
Greenwald, A. G. (1975) · Psychological Bulletin · doi:10.1037/h0076157 · Cited by 1,022
Handbook of clinical psychology
Wolman, B. B. (1965) · McGraw-Hill eBooks · Cited by 960
Cited by
Items 201–300 of 753
Maniadis, Z., Tufano, F., & List, J. A. (2017) · The Economic Journal · doi:10.1111/ecoj.12527 · Cited by 106
The case for confidence intervals in controlled clinical trials
Borenstein, M. (1994) · Controlled Clinical Trials · doi:10.1016/0197-2456(94)90036-1 · Cited by 89
Effect Sizes: Why, When, and How to Use Them
Rosnow, R. L., & Rosenthal, R. (2009) · Zeitschrift für Psychologie / Journal of Psychology · doi:10.1027/0044-3409.217.1.6 · Cited by 83
Article Commentary: On the Persistence of Low Power in Psychological Science
Vankov, I., Bowers, J., & Munafò, M. R. (2014) · Quarterly Journal of Experimental Psychology · doi:10.1080/17470218.2014.885986 · Cited by 82
“Sedlmeier and Gigerenzer (1989) argued that a lack of understanding of these distinctions partly explained the lack of consideration of statistical power: while we (nominally, at least) adhere to a 5% Type I error rate, we pay little attention to the Type II error rate, despite the need to consider both when evaluatin…”
BackgroundMethodInfluentialDetecting publication selection bias through excess statistical significance
Stanley, T. D., Doucouliagos, H., Ioannidis, J. P. A., & Carter, E. C. (2021) · Research Synthesis Methods · doi:10.1002/jrsm.1512 · Cited by 90
Onghena, P. (1992) · Lirias (KU Leuven) · Cited by 82
Subtypes and comorbidity in mathematical learning disabilities
Szűcs, D. (2016) · Progress in brain research · doi:10.1016/bs.pbr.2016.04.027 · Cited by 109
Szűcs, D., & Myers, T. (2016) · Trends in Neuroscience and Education · doi:10.1016/j.tine.2016.11.002 · Cited by 96
Effective programmes in reading and mathematics: lessons from the Best Evidence Encyclopaedia1
Slavin, R. E. (2013) · School Effectiveness and School Improvement · doi:10.1080/09243453.2013.797913 · Cited by 78
Smith, R. A., Levine, T. R., Lachlan, K. A., & Fediuk, T. A. (2002) · Human Communication Research · doi:10.1111/j.1468-2958.2002.tb00821.x · Cited by 77
Statistical power in operations management research
Verma, R., & Goodale, J. C. (1995) · Journal of Operations Management · doi:10.1016/0272-6963(95)00020-s · Cited by 99
The Controversy over Null Hypothesis Significance Testing Revisited
Balluerka, N., Gómez, J., & Hidalgo, D. (2005) · Methodology · doi:10.1027/1614-1881.1.2.55 · Cited by 77
Autogenic Training: A narrative and quantitative review of clinical outcome
von der Linden, W. (1994) · Applied Psychophysiology and Biofeedback · doi:10.1007/bf01721069 · Cited by 77
“Because more outcome studies on AT are now available and indices of standardized treatment effects are explicitly provided in the tables, researchers have more pertinent information on treatment response variance such that power calculations and sample size choices can now be made a priori ( Sedlmeier & Gigerenzer, 19…”
BackgroundEstimating Population Mean Power Under Conditions of Heterogeneity and Selection for Significance
Brunner, J., & Schimmack, U. (2020) · Meta-Psychology · doi:10.15626/mp.2018.874 · Cited by 76
“It is important to distinguish our undertaking from that of Cohen (1962) and follow-up studies by L.J. Chase and R. B. Chase (1976) and Sedlmeier and Gigerenzer (1989).”
Progress in clinical psychology: Is there any?
Rosenthal, R. (1995) · Clinical Psychology: Science and Practice · doi:10.1111/j.1468-2850.1995.tb00035.x · Cited by 75
Are most published social psychological findings false?
Stroebe, W. (2016) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2015.09.017 · Cited by 81
Interpreting Null Results: Improving Presentation and Conclusions with Confidence Intervals 1
Aberson, C. L. (2002) · Cited by 87
Fritz, A., Scherndl, T., & Kühberger, A. (2012) · Theory & Psychology · doi:10.1177/0959354312436870 · Cited by 80
Fidler, F., & Loftus, G. R. (2009) · Zeitschrift für Psychologie / Journal of Psychology · doi:10.1027/0044-3409.217.1.27 · Cited by 72
Toward Improved Statistical Reporting in the Journal of Consulting and Clinical Psychology.
Fidler, F., Cumming, G., Thomason, N., Pannuzzo, D., Smith, J., Fyffe, P., Edmonds, H., Harrington, C., & Schmitt, R. (2005) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.73.1.136 · Cited by 72
Unified power analysis for t-tests through multivariate hypotheses.
OʼBrien, R. G., & Muller, K. E. (1993) · Cited by 87
Probing the effects of individual components in multiple component prevention programs
West, S. G., Aiken, L. S., & Todd, M. (1993) · American Journal of Community Psychology · doi:10.1007/bf00942173 · Cited by 69
“Other reviewers (e.g., Sedlmeier & Gigerenzer, 1989; West, Newsom, & Fenaughty, 1992) have reached similar conclusions about the power of statistical tests in other areas of psychological research.”
BackgroundWhy rehabilitation research does not work (As well as we think it should)
Ottenbacher, K. J. (1995) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/s0003-9993(95)80021-2 · Cited by 73
Data quality over data quantity in computational cognitive neuroscience
Kolossa, A., & Kopp, B. (2018) · NeuroImage · doi:10.1016/j.neuroimage.2018.01.005 · Cited by 68
Feedback: At the Heart of – But Definitely Not All of – Formative Assessment
Wiliam, D. (2018) · Cambridge University Press eBooks · doi:10.1017/9781316832134.003 · Cited by 98
Lilienfeld, S. O., & Penna, S. (2001) · Journal of Anxiety Disorders · doi:10.1016/s0887-6185(01)00070-6 · Cited by 80
“The correlations in Tables 1–4 were not Bonferronicorrected, however, in view of recent arguments by statisticians (e.g., Schmidt, 1992; Sedlmeier & Gigerenzer, 1989) that Bonferroni correction and similar a-adjustment procedures are often ill-advised because of their markedly increased Type 2…”
BackgroundThe significance fallacy in inferential statistics
Kühberger, A., Fritz, A., Lermer, E., & Scherndl, T. (2015) · BMC Research Notes · doi:10.1186/s13104-015-1020-4 · Cited by 85
Ellis, M. V. (1999) · The Counseling Psychologist · doi:10.1177/0011000099274004 · Cited by 84
Vadillo, M. A., Hardwicke, T. E., & Shanks, D. R. (2016) · Journal of Experimental Psychology: General · doi:10.1037/xge0000157 · Cited by 65
Perceived crisis and reforms: Issues, explanations, and remedies.
De Boeck, P., & Jeon, M. (2018) · Psychological Bulletin · doi:10.1037/bul0000154 · Cited by 64
Causal Attributions Following Serious Unexpected Negative Events: A Systematic Review
HALL, S. L., French, D., & Marteau, T. M. (2003) · Journal of Social and Clinical Psychology · doi:10.1521/jscp.22.5.515.22924 · Cited by 78
How to Improve Bayesian Reasoning without Instruction
Gigerenzer, G. (2002) · Adaptive Thinking · doi:10.1093/acprof:oso/9780195153729.003.0006 · Cited by 63
Self-citationNaumann, E., Huber, C. W., Maier, S., Plihal, W., Wustmans, A., Diedrich, O., & Bartussek, D. (1992) · Electroencephalography and Clinical Neurophysiology · doi:10.1016/0013-4694(92)90119-3 · Cited by 74
Quantifying the benefits of using decision models with response time and accuracy data
Stafford, T., Pirrone, A., Croucher, M., & Krystalli, A. (2020) · Behavior Research Methods · doi:10.3758/s13428-020-01372-w · Cited by 70
Rossi, J. S. (2012) · Wiley eBooks · doi:10.1002/9781118133880.hop202003 · Cited by 65
Bite-size science and its undesired side effects
Bertamini, M., & Munafò, M. R. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691611429353 · Cited by 62
“We know that studies in psychology are endemically underpowered (Sedlmeier & Gigerenzer, 1989), and this, therefore, increases the risk of false positives.”
BackgroundOne-year outcomes of a randomized trial of consumer case management
Solomon, P., & Draine, J. (1995) · Evaluation and Program Planning · doi:10.1016/0149-7189(95)00003-t · Cited by 62
Langenberg, B., Janczyk, M., Koob, V., Kliegl, R. M., & Mayer, A. (2022) · Behavior Research Methods · doi:10.3758/s13428-022-01902-8 · Cited by 106
The Effectiveness of Education Technology for Enhancing Reading Achievement: A Meta-Analysis.
Cheung, A., & Slavin, R. E. (2011) · Cited by 93
Determinants of the gait transition speed during human locomotion: kinetic factors
Hreljac, A. P. (1993) · Gait & Posture · doi:10.1016/0966-6362(93)90049-7 · Cited by 68
Significance chasing in research practice: causes, consequences and possible solutions
Ware, J. J., & Munafò, M. R. (2015) · Addiction · doi:10.1111/add.12673 · Cited by 60
Odgaard, E. C., & Fowler, R. L. (2010) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0019294 · Cited by 60
Subjective status and perceived legitimacy across countries
Brandt, M. J., Kuppens, T., Spears, R., Andrighetto, L., Autin, F., Babinčák, P., Badea, C., Bae, J., Batruch, A., Becker, J. C., et al. (2020) · European Journal of Social Psychology · doi:10.1002/ejsp.2694 · Cited by 65
“This helps us avoid low statistical power (Sedlmeier & Gigerenzer, 1989) and uses less-WEIRD sampling (Henrich, Heine, & Norenzayan, 2010).”
MethodPower analysis software for educational researchers
Peng, C.-Y. J., Long, H., & Abaci, S. (2012) · The Journal of Experimental Education · doi:10.1080/00220973.2011.647115 · Cited by 59
“The serious neglect of prospective power analysis may be explained by two reasons (Sedlmeier & Gigerenzer, 1989): (1) the α -adjusted procedures for multiple comparisons of means, and (2) the confusion over the hybrid of Fisherian and Neyman-Pearsonian conceptualization of null hypothesis…”
BackgroundMethodInfluentialOn reporting and interpreting statistical significance and p values in medical research
Aguinis, H., Vassar, M. B., & Wayant, C. (2019) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2019-111264 · Cited by 69
“5 6 In other words, while authors focus on whether their results show an acceptably small type I error rate, type II error—the probability of accepting the null hypothesis erroneously and incorrectly concluding that an effect is absent—looms large.”
BackgroundContext effects and their interaction with development: Area judgments
Gigerenzer, G., & Richter, H. (1990) · Cognitive Development · doi:10.1016/0885-2014(90)90017-n · Cited by 58
Self-citationDecisions about equivalence: A comparison of TOST, HDI-ROPE, and the Bayes factor.
Linde, M., Tendeiro, J. N., Selker, R., Wagenmakers, E., & van Ravenzwaaij, D. (2023) · Psychological Methods · doi:10.1037/met0000402 · Cited by 66
The Modularity of Social Intelligence
Gigerenzer, G. (2002) · Adaptive Thinking · doi:10.1093/acprof:oso/9780195153729.003.0011 · Cited by 57
Self-citationKalinowski, P., & Fidler, F. M. (2010) · Newborn and Infant Nursing Reviews · doi:10.1053/j.nainr.2009.12.007 · Cited by 68
Gervais, W. M., Jewell, J. A., Najle, M. B., & Ng, B. K. L. (2015) · Social Psychological and Personality Science · doi:10.1177/1948550615584199 · Cited by 56
Ueno, T., Fastrich, G. M., & Murayama, K. (2016) · Journal of Experimental Psychology: General · doi:10.1037/xge0000159 · Cited by 56
“…prevalence of underpowered studies in psychology has been repeatedly remarked on in the past (e.g., Cohen, 1988; Sedlmeier & Gigerenzer, 1989), but it is only recently that researchers have started to seriously consider the problems and implications of…”
A Legacy Neglected: Restating the Case for Single-case Research in Cognitive-Behaviour Therapy
Blampied, N. M. (1999) · Behaviour Change · doi:10.1375/bech.16.2.89 · Cited by 56
Accuracy of Effect Size Estimates from Published Psychological Research
Brand, A., Bradley, M. T., Best, L. A., & Stoica, G. (2008) · Perceptual and Motor Skills · doi:10.2466/pms.106.2.645-649 · Cited by 55
Watanabe, E. A. D. M., Torres, C. V., & Alfinito, S. (2019) · Revista de Gestão · doi:10.1108/rege-12-2017-0009 · Cited by 73
Bliese, P. D., & Wang, M. (2019) · Journal of Management · doi:10.1177/0149206319886909 · Cited by 60
A nemesis for heritability estimation
Hirsch, J. (1990) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00077967 · Cited by 55
Koustelios, A. D., & Kousteliou, I. (1998) · Psychological Reports · doi:10.2466/pr0.1998.82.1.131 · Cited by 54
One- versus Two-Tailed Hypothesis Tests in Contemporary Educational Research
Pillemer, D. B. (1991) · Educational Researcher · doi:10.2307/1176244 · Cited by 54
Beyond the Null Ritual: Formal Modeling of Psychological Processes
Marewski, J. N., & Olsson, H. (2009) · Zeitschrift für Psychologie · doi:10.1027/0044-3409.217.1.49 · Cited by 73
The Power of Replicated Measures to Increase Statistical Power
Goulet, M.-A., & Cousineau, D. (2019) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245919849434 · Cited by 67
“Over the years, there have been critiques questioning the lack of statistical power in psychological studies (Bezeau & Graves, 2001; Button et al., 2013; Sedlmeier & Gigerenzer, 1989).”
BackgroundConocimiento social de la mentira y credibilidad
Caballero, A., Sánchez, F., & Becerra, A. (2000) · Psicothema · Cited by 53
Finch, S., Cumming, G., & Thomason, N. (2001) · Educational and Psychological Measurement · doi:10.1177/0013164401612001 · Cited by 52
Kotchoubey, B., & Pavlov, Y. G. (2018) · Frontiers in Neurology · doi:10.3389/fneur.2018.00315 · Cited by 59
Impact of experience on maintenance skills
Jørgensen, M., & Sjøberg, D. I. K. (2002) · Journal of Software Maintenance and Evolution Research and Practice · doi:10.1002/smr.248 · Cited by 63
Schmidt, F., & Hunter, J. (2002) · American Psychologist · doi:10.1037//0003-066x.57.1.65 · Cited by 52
“The evidence is clear that the average level of statistical power in psychological research is between .40 and .60 (e.g., see Cohen, 1962, 1994; Sedlmeier & Gigerenzer, 1989).”
BackgroundCan the behavioral sciences self-correct? A social epistemic study
Romero, F. (2016) · Studies in History and Philosophy of Science Part A · doi:10.1016/j.shpsa.2016.10.002 · Cited by 57
Not null enough: pseudo-null hypotheses in community ecology and comparative psychology
Bausman, W. C., & Halina, M. (2018) · Biology & Philosophy · doi:10.1007/s10539-018-9640-4 · Cited by 62
Gendreau, P., & Smith, P. (2007) · Criminal Justice and Behavior · doi:10.1177/0093854807307025 · Cited by 54
Secchi, D., & Seri, R. (2016) · Computational and Mathematical Organization Theory · doi:10.1007/s10588-016-9218-0 · Cited by 67
Statistical Significance should not be Considered one of Life’s Guarantees: Effect Sizes are Needed
Vacha‐Haase, T. (2001) · Educational and Psychological Measurement · doi:10.1177/00131640121971194 · Cited by 62
Austin, J. T., Scherbaum, C. A., & Mahlman, R. A. (2004) · Wiley eBooks · doi:10.1002/9780470756669.ch1 · Cited by 62
“Sedlmeier and Gigerenzer (1989) hypothesized that statistical power would have increased due to Cohen’s research during the 1960s (cf. Cohen, 1988).”
BackgroundMedical aromatherapy revisited—Basic mechanisms, critique, and a new development
Schneider, R., Singer, N., & Singer, T. (2018) · Human Psychopharmacology Clinical and Experimental · doi:10.1002/hup.2683 · Cited by 60
Woods, S. P., Rippeth, J. D., Conover, E., Carey, C. L., Parsons, T. D., & Tröster, A. I. (2006) · The Clinical Neuropsychologist · doi:10.1080/13854040500203290 · Cited by 52
Giner‐Sorolla, R. (2018) · European Review of Social Psychology · doi:10.1080/10463283.2018.1542902 · Cited by 58
P > .05: The incorrect interpretation of “not significant” results is a significant problem
Smith, R. J. (2020) · American Journal of Physical Anthropology · doi:10.1002/ajpa.24092 · Cited by 57
Lai, K., & Kelley, K. (2011) · Psychological Methods · doi:10.1037/a0021764 · Cited by 49
A Concise Set of Core Recommendations to Improve the Dependability of Psychological Research
Lishner, D. A. (2015) · Review of General Psychology · doi:10.1037/gpr0000028 · Cited by 77
Xuan, Q., Cheung, A., & Sun, D. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.990196 · Cited by 124
STATISTICAL COGNITION: TOWARDS EVIDENCE-BASED PRACTICE IN STATISTICS AND STATISTICS EDUCATION
Marom, T., Fidler, F. M., & Cumming, G. (2008) · Statistics Education Research Journal · doi:10.52041/serj.v7i2.468 · Cited by 50
What Can We Learn from Many Labs Replications?
Stroebe, W. (2019) · Basic and Applied Social Psychology · doi:10.1080/01973533.2019.1577736 · Cited by 52
“Social psychologists have learned to take the complaints about the insufficient statistical power of their research (e.g., Cohen, 1962; Fraley & Vazire, 2014; Sedlmeier & Gigerenzer, 1989) more seriously and have considerably increased the number of participants on which they base their studies.”
BackgroundKempthorne, O. (1990) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00077979 · Cited by 48
Trying to shoot the messenger for his message
Plomin, R. J. (1990) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00078031 · Cited by 47
Cognitive and Neural Correlates of Mathematical Giftedness in Adults and Children: A Review
Myers, T., Carey, E., & Szűcs, D. (2017) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.01646 · Cited by 53
Statistical power of articles published in three health-psychology related journals.
Maddock, J. E., & Rossi, J. S. (2001) · Health Psychology · doi:10.1037//0278-6133.20.1.76 · Cited by 46
Result-Blind Peer Reviews and Editorial Decisions: A Missing Pillar of Scientific Culture
Greve, W., Bröder, A., & Erdfelder, E. (2013) · European Psychologist · doi:10.1027/1016-9040/a000144 · Cited by 46
The Power and Effects of Entrepreneurship Research
Connelly, B. L., Ireland, R. D., Reutzel, C. R., & Coombs, J. E. (2009) · Entrepreneurship Theory and Practice · doi:10.1111/j.1540-6520.2009.00316.x · Cited by 66
Strube, M. (2006) · Behavior Research Methods · doi:10.3758/bf03192746 · Cited by 41
A Tutorial on Hunting Statistical Significance by Chasing N
Szucs, D. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2016.01444 · Cited by 45
“Also note that larger Ns also boost power which is often very low in psychological and neuroscience research (Sedlmeier and Gigerenzer, 1989; Rossi, 1990; Button et al., 2013).”
BackgroundToo Good to be False: Nonsignificant Results Revisited
Hartgerink, C. H. J., Wicherts, J. M., & van Assen, M. A. L. M. (2017) · Collabra: Psychology · doi:10.1525/collabra.71 · Cited by 45
Zinbarg, R. E., Suzuki, S., Uliaszek, A. A., & Lewis, A. R. (2010) · Journal of Abnormal Psychology · doi:10.1037/a0017552 · Cited by 45
Data management and sharing: Practices and perceptions of psychology researchers
Borghi, J. A., & Gulick, A. E. V. (2020) · PLoS ONE · doi:10.1371/journal.pone.0252047 · Cited by 44
“Interrelated issues including an overrepresentation of positive results [1, 2] unreported flexibility in analytical methods [3–5], and low levels of statistical power [6, 7] have resulted in uncertainty about the robustness of results described in the psychology literature.”
BackgroundOn the challenges of drawing conclusions from p -values just below 0.05
Lakens, D. (2015) · PeerJ · doi:10.7717/peerj.1142 · Cited by 44
“At the same time, we can be certain power varies substantially across studies and research disciplines (e.g., Button et al., 2013 ; Sedlmeier & Gigerenzer, 1989 ), and therefore the p -value distribution can be more accurately modeled by summing multiple p -value distributions across di ff erent…”
BackgroundScheffés mixed model for multivariate repeated measures:a relative efficiency evaluation
Boik, R. J. (1991) · Communication in Statistics- Theory and Methods · doi:10.1080/03610929108830562 · Cited by 44
Goals and methods: The study of development versus partitioning of variance
Wahłsten, D. (1990) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00078067 · Cited by 44
The value of RCT evidence depends on the quality of statistical analysis
Faulkner, C., Fidler, F. M., & Cumming, G. (2007) · Behaviour Research and Therapy · doi:10.1016/j.brat.2007.12.001 · Cited by 47
Effect Size Reporting Practices in Published Articles
Alhija, F. N.-A., & Levy, A. (2009) · Educational and Psychological Measurement · doi:10.1177/0013164408315266 · Cited by 43
Scientific rigor and the art of motorcycle maintenance
Munafò, M., Noble, S., Browne, W. J., Brunner, D., Button, K., Ferreira, J., Holmans, P., Langbehn, D., Lewis, G., Lindquist, M., et al. (2014) · Nature Biotechnology · doi:10.1038/nbt.3004 · Cited by 43
Effect size and statistical power in the rodent fear conditioning literature – A systematic review
Carneiro, C. F. D., Moulin, T. C., Macleod, M. R., & Amaral, O. B. (2018) · PLoS ONE · doi:10.1371/journal.pone.0196258 · Cited by 54
Determining Sample Size for Accurate Estimation of the Squared Multiple Correlation Coefficient
Algina, J. J., & Olejnik, S. F. (2000) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr3501_5 · Cited by 43
Davis‐Stober, C. P., & Dana, J. D. (2013) · Behavior Research Methods · doi:10.3758/s13428-013-0342-1 · Cited by 43
Metadata
- Authors
- Sedlmeier, P., & Gigerenzer, G.
- Venue
- Psychological Bulletin
- Published
- 1989-03
- Volume
- 105
- Issue
- 2
- Pages
- 309-316
- ISSN
- 0033-2909 (ISSN-L)
- 1939-1455
- OpenAlex
- W2155449866
- Cited by
- 748
- References
- 55
- Citations by year
- 2026: 3 · 2025: 7 · 2024: 7 · 2023: 14 · 2022: 22 · 2021: 30 · 2020: 23 · 2019: 30 · 2018: 34 · 2017: 35 · 2016: 26 · 2015: 26 · 2014: 30 · 2013: 24 · 2012: 25
- FWCI
- 27.89
- Type
- journal article
- Language
- en
- Open access
- green
- hdl.handle.net
- Licence
- other-oa
- Locations
- Psychological Bulletin (publishedVersion)
- Max Planck Digital Library (submittedVersion, other-oa)
- MPG.PuRe (Max Planck Society) (submittedVersion, other-oa)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › General Psychology › Academic and Historical Perspectives in Psychology
- Keywords
- statistical power, power analysis, effect size, multiple comparisons, non-significant results, null hypothesis significance testing, history of statistics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.