Do studies of statistical power have an effect on the power of studies?
Open accessThe DOI matches its Crossref record; 54 of the 55 references it cites were found in a registry or an index.
Abstract
The long-term impact of studies of statistical power is investigated using J. Cohen's (1962) pioneering work as an example. We argue that the impact is nil; the power of studies in the same journal that Cohen reviewed (now the Journal of Abnormal Psychology) has not increased over the past 24 years. In 1960 the median power (i.e., the probability that a significant result will be obtained if there is a true effect) was.46 for a medium size effect, whereas in 1984 it was only.37. The decline of power is a result of alpha-adjusted procedures. Low power seems to go unnoticed: only 2 out of 64 experi-ments mentioned power, and it was never estimated. Nonsignificance was generally interpreted as confirmation of the null hypothesis (if this was the research hypothesis), although the median power was as low as.25 in these cases. We discuss reasons for the ongoing neglect of power. Since J. Cohen's (1962) classical study on the statistical power of the studies published in the 1960 volume of the Journal of Abnormal and Social Psychology, a number of power analyses have been performed. These studies exhorted researchers to pay attention to the power of their tests rather than to focus exclu-sively on the level of significance. Historically, the concept of
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“A little over two decades after Cohen’s original paper, two meta-analyses by Sedlmeier & Gigerenzer [66] and Rossi [67] examined a total of 25 reviews of statistical power in the psychological and social science literature between 1960 and 1984.”
From The natural selection of bad science (2016) · cites this paper as [66]
In the citing paper
“In fact, the average power of studies published in the 1984 volume of the Journal of Abnormal Psychology (a successor to the JASP) had dropped slightly compared with Cohen's (1962) results.”
From GPOWER: A general power analysis program (1996) · cites this paper as Sedlmeier and Gigerenzer (1989)
In the citing paper
“Sedlmeier and Gigerenzer (1989) investigated the impact of power analysis studies and textbooks on the power of recent psychological studies. Surprisingly, these authors found no significant increase in power values since 1962 when Cohen published his power study of the 1960 volume of the Journal of Abnormal & Social Psychology (JASP).”
From GPOWER: A general power analysis program (1996) · cites this paper as Sedlmeier and Gigerenzer (1989)
Citations
Cites
Statistical power analysis for the behavioral sciences
(1990) · Computers, Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,269
Statistical Power Analysis for the Behavioral Sciences (2nd ed.).
Lachenbruch, P. A., & Cohen, J. (1989) · Journal of the American Statistical Association · doi:10.2307/2290095 · Cited by 37,395
Fisher, R. A. (1936) · BMJ · doi:10.1136/bmj.1.3923.554-a · Cited by 4,980
Fundamental Statistics in Psychology and Education.
Smith, B. B., & Guilford, J. P. (1951) · Journal of the Royal Statistical Society. Series A (General) · doi:10.2307/2980908 · Cited by 3,455
Belief in the law of small numbers.
Tversky, A., & Kahneman, D. (1971) · Psychological Bulletin · doi:10.1037/h0031322 · Cited by 3,246
The statistical power of abnormal-social psychological research: A review.
Cohen, J. (1962) · The Journal of Abnormal and Social Psychology · doi:10.1037/h0045186 · Cited by 1,824
Freiman, J. A., Chalmers, T. C., Smith, H., & Kuebler, R. R. (1978) · New England Journal of Medicine · doi:10.1056/nejm197809282991304 · Cited by 1,628
Statistical Methods and Scientific Inference.
Bartlett, M. S., & Fisher, R. A. (1957) · Biometrika · doi:10.2307/2333276 · Cited by 1,278
Consequences of prejudice against the null hypothesis.
Greenwald, A. G. (1975) · Psychological Bulletin · doi:10.1037/h0076157 · Cited by 1,022
Handbook of clinical psychology
Wolman, B. B. (1965) · McGraw-Hill eBooks · Cited by 960
Cited by
Items 301–400 of 753
Negativland - a home for all findings in psychology
Laws, K. R. (2013) · BMC Psychology · doi:10.1186/2050-7283-1-2 · Cited by 42
What exercise and sport scientists don't understand
Speed, H. D., & Andersen, M. B. (2000) · Journal of science and medicine in sport · doi:10.1016/s1440-2440(00)80051-1 · Cited by 48
Effect Sizes, Power, and Biases in Intelligence Research: A Meta-Meta-Analysis
Nuijten, M. B., van Assen, M. A. L. M., Augusteijn, H. E. M., Crompvoets, E. A. V., & Wicherts, J. M. (2020) · Journal of Intelligence · doi:10.3390/jintelligence8040036 · Cited by 45
“…that sample sizes in psychology research are typically too small to detect the expected true effect sizes (which are also estimated to be small to medium), leading to the conclusion that many psychology studies may be underpowered (Button et al. 2013; Cohen 1962; Sedlmeier and Gigerenzer 1989).”
BackgroundButton, K. S., Chambers, C. D., Lawrence, N., & Munafò, M. R. (2020) · Psychology Learning & Teaching · doi:10.1177/1475725719857659 · Cited by 42
“Studies consistently find statistical power to be low, between 8~50%, across both time and disciplines studied (Button et al., 2013; Cohen, 1962; Sedlmeier & Gigerenzer, 1989).”
BackgroundPower rangers: no improvement in the statistical power of analyses published in Animal Behaviour
Smith, D. R., Hardy, I. C. W., & Gammell, M. P. (2010) · Animal Behaviour · doi:10.1016/j.anbehav.2010.09.026 · Cited by 46
Meta‐analysis or best‐evidence synthesis?
Eysenck, H. J. (1995) · Journal of Evaluation in Clinical Practice · doi:10.1111/j.1365-2753.1995.tb00005.x · Cited by 41
An Analysis of Statistical Power in Behavioral Accounting Research
Borkowski, S. C., Welsh, M. J., & Zhang, Q. (2001) · Behavioral Research in Accounting · doi:10.2308/bria.2001.13.1.63 · Cited by 45
“3 Sedlmeier and Gigerenzer (1989) subsequently found that his judgment of a medium effect size closely corresponded to the actual sample median effect size of the articles in his sample, and his definitions have been supported by post hoc calculations of effect size in related research areas (Haase et al.”
BackgroundPast and Future American Psychological Association Guidelines for Statistical Practice
Finch, S., Thomason, N., & Cumming, G. (2002) · Theory & Psychology · doi:10.1177/0959354302126005 · Cited by 40
Kazdin, A. E. (1996) · Clinical Psychology Science and Practice · doi:10.1111/j.1468-2850.1996.tb00059.x · Cited by 41
Significance Testing in Psychological Research: Some Persisting Issues
Estes, W. K. (1997) · Psychological Science · doi:10.1111/j.1467-9280.1997.tb00538.x · Cited by 46
Osborne, J. W. (2008) · Educational Psychology · doi:10.1080/01443410701491718 · Cited by 39
Reform of statistical inference in psychology: The case ofMemory & Cognition
Finch, S. J., Cumming, G., Williams, J., Palmer, L., Griffith, E., Alders, C., Anderson, J. L., & Goodman, O. (2004) · Behavior Research Methods, Instruments, & Computers · doi:10.3758/bf03195577 · Cited by 43
Further Evidence for Systematic Reliability Differences between Explicit and Implicit Memory Tests
Buchner, A., & Brandt, M. (2003) · The Quarterly Journal of Experimental Psychology Section A · doi:10.1080/02724980244000260 · Cited by 44
Rationality: Why Social Context Matters
Gigerenzer, G. (2002) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780195153729.003.0009 · Cited by 40
Self-citationReplication Unreliability in Psychology: Elusive Phenomena or “Elusive” Statistical Power?
Tressoldi, P. E. (2012) · Frontiers in Psychology · doi:10.3389/fpsyg.2012.00218 · Cited by 41
Statistics as rhetoric in psychology
John, I. D. (1992) · Australian Psychologist · doi:10.1080/00050069208257601 · Cited by 48
The sanctity of p < .05 obfuscates good stuff: A comment on kerr and goss
Andersen, M. B., & Stoove, M. A. (1998) · Journal of Applied Sport Psychology · doi:10.1080/10413209808406384 · Cited by 38
Statistical significance, path dependency, and the culture of journal publication
Altman, M. (2004) · The Journal of Socio-Economics · doi:10.1016/j.socec.2004.09.037 · Cited by 36
Streiner, D. L. (1999) · Schizophrenia Research · doi:10.1016/s0920-9964(98)00126-1 · Cited by 40
ORGANIZATIONAL FACTORS AS PREDICTORS OF TEACHERS' BURNOUT
Koustelios, A. D. (2001) · Psychological Reports · doi:10.2466/pr0.2001.88.3.627 · Cited by 38
Exploring the Meaning of Significance in Experimental EconomicsPreprint
Le, Z., & Ortmann, A. (2013) · SSRN Electronic Journal · doi:10.2139/ssrn.2356018 · Cited by 47
Evaluación del individualismo-colectivismo vertical-horizontal en Brasil: una propuesta de medida
Torres, C. V., & Pérez-Nebra, A. R. (2015) · Redalyc (Universidad Autónoma del Estado de México) · Cited by 37
van Houwelingen, G., van Dijke, M., & Cremer, D. D. (2016) · Journal of Management · doi:10.1177/0149206314530166 · Cited by 43
From theories to models to predictions: A Bayesian model comparison approach
Rouder, J. N., Haaf, J. M., & Aust, F. (2017) · Communication Monographs · doi:10.1080/03637751.2017.1394581 · Cited by 41
Venter, A., Maxwell, S. E., & Bolig, E. E. (2002) · Psychological Methods · doi:10.1037/1082-989x.7.2.194 · Cited by 89
An Interactive Tutorial for Teaching Statistical Power
Aberson, C. L., Berger, D. E., Healy, M. R., & Romero, V. L. (2002) · Journal of Statistics Education · doi:10.1080/10691898.2002.11910682 · Cited by 42
“…studies in many fields is striking evidence of a lack of comprehension of the relevance of statistical power to research design (for example, on average, the Type II error rate in psychology and education is estimated to be 50% or more; Sedlmeier and Gigerenzer 1989; Lipsey and Wilson 1993).”
BackgroundStatistical Power of Within and Between-Subjects Designs in Economic ExperimentsPreprint
Bellemare, C., Bissonnette, L., & Kröger, S. (2014) · SSRN Electronic Journal · doi:10.2139/ssrn.2529895 · Cited by 39
Callahan, J. L., & Reio, T. G. (2006) · Human Resource Development Quarterly · doi:10.1002/hrdq.1167 · Cited by 39
Evaluating Replication ResultsPreprint
Simonsohn, U. (2013) · SSRN Electronic Journal · doi:10.2139/ssrn.2259879 · Cited by 42
Lou, Y. (2004) · Journal of Educational Computing Research · doi:10.2190/xf5a-t29g-x170-7950 · Cited by 34
A meta‐analysis of effect sizes in international marketing experiments
Wang, X., & Yang, Z. (2008) · International Marketing Review · doi:10.1108/02651330810877216 · Cited by 42
Onwuegbuzie, A. J., & Levin, J. R. (2003) · Journal of Modern Applied Statistical Methods · doi:10.22237/jmasm/1051747920 · Cited by 40
“Indeed, the average power of null hypothesis significance tests typically ranges from .40 to .60 in empirical studies (Cohen, 1962, 1965, 1988, 1994; Schmidt, 1996; Sedlmeier & Gigerenzer, 1989).”
BackgroundSupply chain management research: Key elements of study design and statistical testing
Helmuth, C. A., Craighead, C. W., Connelly, B. L., Collier, D. Y., & Hanna, J. B. (2014) · Journal of Operations Management · doi:10.1016/j.jom.2014.12.001 · Cited by 39
The Heuristic Value of p in Inductive Statistical Inference
Krueger, J. I., & Heck, P. R. (2017) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.00908 · Cited by 34
“At the same time, there is also the concern that most empirical samples are not large enough to detect important effects (Cohen, 1962; Sedlmeier and Gigerenzer, 1989).”
BackgroundPOWPAL: A Program for Estimating Effect Sizes, Statistical Power, and Sample Sizes
Gorman, B. S., Primavera, L. H., & Allison, D. B. (1995) · Educational and Psychological Measurement · doi:10.1177/0013164495055005009 · Cited by 33
Rethinking Observed Power: Concept, Practice, and Implications
Sun, S., Pan, W., & Wang, L. L. (2011) · Methodology · doi:10.1027/1614-2241/a000025 · Cited by 32
The Power of Replications and Replications of Power
Ottenbacher, K. J. (1996) · The American Statistician · doi:10.1080/00031305.1996.10474398 · Cited by 32
A Critique of the Usefulness of Inferential Statistics in Applied Behavior Analysis
Hopkins, B. L., Cole, B. L., & Mason, T. L. (1998) · The Behavior Analyst · doi:10.1007/bf03392787 · Cited by 39
“Sedlmeier and Gigerenzer (1989) examined a sample of behavioral research and found that inferential statistics led to incorrect conclusions about effects in fully 60% of the cases (rather than in 5% or 1% of cases, as is commonly assumed, because users of statistics ordinarily op- erate at the .05…”
BackgroundEffective Early Childhood Education Programs: A Systematic Review.
Chambers, B., Cheung, A., Slavin, R. E., Smith, D., & Laurenzano, M. (2010) · Cited by 31
Assessing the Reliability of Ecological Monitoring Data: Power Analysis and Alternative Approaches
Morrison, L. W. (2007) · Natural Areas Journal · doi:10.3375/0885-8608(2007)27[83:atroem]2.0.co;2 · Cited by 32
“Yet, this attempt to fuse opposing theories into a single one has generated much confusion regarding even basic statistical concepts (Sedlmeier and Gigerenzer 1989) .”
BackgroundComparison of A-mode and B-mode Ultrasound for Measurement of Subcutaneous Fat
Wagner, D. R., Teramoto, M., Judd, T., Gordon, J. P., McPherson, C., & Robison, A. (2020) · Ultrasound in Medicine & Biology · doi:10.1016/j.ultrasmedbio.2019.11.018 · Cited by 31
Lim, Y. S., & COLE, K. N. (2002) · Bilingual Research Journal · doi:10.1080/15235882.2002.10668716 · Cited by 42
Book Review of Using Multivariate Statistics by Barbara G. Tabachnick and Linda S. Fidell
Harlow, L. L. (2002) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1207/s15328007sem0904_9 · Cited by 31
The conundrum of depression clinical trials: one size does not fit all
Khan, A., Mar, K. F., & Brown, W. A. (2018) · International Clinical Psychopharmacology · doi:10.1097/yic.0000000000000229 · Cited by 30
“Perhaps we were not heeding the risks of low statistical power well enough, as has been the case in many scientific fields (Cohen, 1962; Freiman et al., 1978; Sedlmeier and Gigerenzer, 1989; Halpern et al., 2002; Maxwell, 2004; Button et al., 2013).”
BackgroundBehavioral Science Foundations of the Rorschach Test: Research and Clinical Applications
Acklin, M. W. (1999) · Assessment · doi:10.1177/107319119900600403 · Cited by 30
Multiple Trials May Yield Exaggerated Effect Size Estimates
Brand, A., Bradley, M. T., Best, L. A., & Stoica, G. (2010) · The Journal of General Psychology · doi:10.1080/00221309.2010.520360 · Cited by 28
“…statistical power of published psychological research to detect small and medium size effects has been shown to be low (Cohen, 1962; Rossi, 1990; Sedlmeier & Gigerenzer, 1989), and, as a consequence, published psychological research tends to overestimate true effect sizes (Brand, Bradley, Best,…”
BackgroundHow does HRM enhance strategic capabilities? Evidence from the Korean management consulting industry
Kim, A., & Lee, C. (2011) · The International Journal of Human Resource Management · doi:10.1080/09585192.2011.561247 · Cited by 41
“37 (Seldmeier and Gigerenzer 1989 , as cited in Colakoglu and Caligiuri 2008 ).”
BackgroundAltoè, G., Bertoldo, G., Callegher, C. Z., Toffalini, E., Calcagnì, A., Finos, L., & Pastore, M. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.02893 · Cited by 34
The Effect Size Statistic: Useful in Health Outcomes Research ?
Hevey, D., & McGee, H. M. (1998) · Journal of Health Psychology · doi:10.1177/135910539800300201 · Cited by 29
Introversion-extraversion and Event-Related Potential (ERP): A test of J.A. Gray's theory
Bartussek, D., Diedrich, O., Naumann, E., & Collet, W. (1993) · Personality and Individual Differences · doi:10.1016/0191-8869(93)90149-w · Cited by 38
From Discovery to Justification: Outline of an Ideal Research Program in Empirical Psychology
Witte, E. H., & Zenker, F. (2017) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.01847 · Cited by 38
“Some 25 years later, Sedlmeier and Gigerenzer (1989) arrived at a similar value.”
BackgroundKing, K. M., Pullmann, M. D., Lyon, A. R., Dorsey, S., & Lewis, C. C. (2019) · Journal of Abnormal Psychology · doi:10.1037/abn0000417 · Cited by 39
An Open Letter to Our Students: Doing Interdisciplinary Moral Psychology
Machery, É., & Doris, J. M. (2017) · Springer eBooks · doi:10.1007/978-3-319-61849-4_7 · Cited by 42
The pros and cons of funnel plots as an aid to risk communication and patient decision making
Rakow, T., Wright, R. J., Spiegelhalter, D. J., & Bull, C. (2014) · British Journal of Psychology · doi:10.1111/bjop.12081 · Cited by 30
There Is a Time and a Place for Significance Testing
Mulaik, S. A., Raju, N. S., & Harshman, R. A. (2016) · doi:10.4324/9781315629049-16 · Cited by 31
Schmidt, F. M., & Hunter, J. E. (1995) · Evaluation & the Health Professions · doi:10.1177/016327879501800405 · Cited by 30
Alcoholism treatment outcome studies, 1980–1992: Methodological characteristics and quality
Morley, J. A., Finney, J. W., Monahan, S. C., & Floyd, A. S. (1996) · Addictive Behaviors · doi:10.1016/0306-4603(95)00073-9 · Cited by 28
Specht, M. (2020) · Sensors · doi:10.3390/s20247144 · Cited by 30
Retrospective median power, false positive meta‐analysis and large‐scale replication
Stanley, T. D., Doucouliagos, H., & Ioannidis, J. P. A. (2021) · Research Synthesis Methods · doi:10.1002/jrsm.1529 · Cited by 34
Panel Attrition and External Validity in the Short-Term Follow-Up Study of Adolescent Substance Use
Tebes, J. K., Snow, D. L., & Arthur, M. W. (1992) · Evaluation Review · doi:10.1177/0193841x9201600203 · Cited by 27
Burgess, E. S., & Haaga, D. A. F. (1998) · Cognitive Therapy and Research · doi:10.1023/a:1018721615712 · Cited by 33
The objections to null hypothesis testing as a means of analysing psychological data
Hammond, G. (1996) · Australian Journal of Psychology · doi:10.1080/00049539608259513 · Cited by 29
Rosenthal, R., & DiMatteo, M. R. (2002) · Wiley eBooks · doi:10.1002/0471214426.pas0410 · Cited by 27
Identifying important management competencies in fitness centres in Greece
Koustelios, A. D. (2003) · Managing Leisure · doi:10.1080/1360671032000123672 · Cited by 26
“Sample sizes were ade-were based on factors with an eigenvalue of quate to assure power of 0.87, an effect size of[1.0; each individual item was correlated 0.25 at a significance level of 0.05 (Cohen,with the the factor concerned at the 0.50 1992; Sedlmeier and Gigerenzer, 1989).”
MethodInfluentialEarly Exposure to Direct Instruction and Subsequent Juvenile Delinquency: A Prospective Examination
Mills, P. E., COLE, K. N., Jenkins, J. R., & Dale, P. S. (2002) · Exceptional Children · doi:10.1177/001440290206900106 · Cited by 26
The Shift from Significance Testing to Effect Size Estimation
Borenstein, M. (1998) · Elsevier eBooks · doi:10.1016/b0080-4270(73)00209-1 · Cited by 27
Evoked and event-related potentials in disorders of consciousness: A quantitative review
Kotchoubey, B. (2017) · Consciousness and Cognition · doi:10.1016/j.concog.2017.05.002 · Cited by 29
Computing and Interpreting Effect Sizes
Hill, C. R., & Thompson, B. (2006) · Kluwer Academic Publishers eBooks · doi:10.1007/1-4020-2456-8_5 · Cited by 26
Mittag, K. C., & Thompson, B. (2000) · Educational Researcher · doi:10.3102/0013189x029004014 · Cited by 69
Between-group psychotherapy outcome research and basic science
Borkovec, T. D., & Miranda, J. (1999) · Journal of Clinical Psychology · doi:10.1002/(sici)1097-4679(199902)55:2<147::aid-jclp2>3.0.co;2-v · Cited by 45
Statistical Power of within and Between-Subjects Designs in Economic ExperimentsPreprint
Bellemare, C., Bissonnette, L., & Kröger, S. (2018) · SSRN Electronic Journal · doi:10.2139/ssrn.3149007 · Cited by 27
Generalized eta squared for multiple comparisons on between-groups designs
Sánchez, M. E. T., & Martínez, R. (2016) · Psicothema · doi:10.7334/psicothema2015.124 · Cited by 30
Sorting the File Drawer: A Typology for Describing Unpublished Studies
Lishner, D. A. (2021) · Perspectives on Psychological Science · doi:10.1177/1745691620979831 · Cited by 25
Job Security and Job Satisfaction among Greek Fitness Instructors
Koustelios, A. D., Kouli, O., & Theodorakis, N. (2003) · Perceptual and Motor Skills · doi:10.2466/pms.2003.97.1.192 · Cited by 24
Effective Beginning Reading Programs: A Best-Evidence Synthesis.
Slavin, R. E., Lake, C., Chambers, B., Cheung, A., & Davis, S. (2009) · Cited by 37
Using Bayes Factors to Test Hypotheses in Developmental Research
Williams, M. N., Bååth, R. A., & Philipp, M. C. (2017) · Research in Human Development · doi:10.1080/15427609.2017.1370964 · Cited by 24
Schroeder, P. A., Nuerk, H., & Plewnia, C. (2017) · Frontiers in Neuroscience · doi:10.3389/fnins.2017.00654 · Cited by 27
Practical power analysis for substance abuse health services research.
Dennis, M. L., Lennox, R., & Foss, M. A. (1997) · American Psychological Association eBooks · doi:10.1037/10222-011 · Cited by 32
In Search Of True Things Worth Knowing: Considerations For A New Article Prototype
Tullett, A. M. (2015) · Social and Personality Psychology Compass · doi:10.1111/spc3.12166 · Cited by 24
“In practice, however, it might not be the case that the expectation of multiple studies results in researchers doubling or tripling the amount of data they would have collected had one study been enough (Cohen, 1992; Maxwell, 2004; Rossi, 1990; Schimmack, 2012; Sedlmeier & Gigerenzer, 1989).”
BackgroundMachery, É. (2012) · Philosophy of Science · doi:10.1086/667877 · Cited by 23
More Powerful Tests of Predictor Subsets in Regression Analysis Under Nonnormality.
Serlin, R. C., & Harwell, M. R. (2004) · Psychological Methods · doi:10.1037/1082-989x.9.4.492 · Cited by 24
Unbalanced Designs to Maximize Statistical Power in Psychotherapy Efficacy Studies
Hsu, L. M. (1994) · Psychotherapy Research · doi:10.1080/10503309412331333932 · Cited by 23
Sample Size Estimation in Research With Dependent Measures and Dichotomous Outcomes
Delucchi, K. (2004) · American Journal of Public Health · doi:10.2105/ajph.94.3.372 · Cited by 23
What are we looking at, and how big is it?
Stoove, M. A., & Andersen, M. B. (2003) · Physical Therapy in Sport · doi:10.1016/s1466-853x(03)00039-7 · Cited by 23
Gigerenzer, G. (2002) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780195153729.003.0007 · Cited by 24
Self-citationReasoning the Fast and Frugal Way
Gigerenzer, G. (2002) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780195153729.003.0008 · Cited by 23
Self-citationValentine, J. C., Wilson, S. J., Rindskopf, D., Lau, T. S., Tanner‐Smith, E. E., Yeide, M., LaSota, R. R., & Foster, L. (2016) · Evaluation Review · doi:10.1177/0193841x16674421 · Cited by 26
Kelley, K., & Maxwell, S. E. (2012) · SAGE Publications Ltd eBooks · doi:10.4135/9781446212165.n11 · Cited by 25
P-Curve and Effect Size: Correcting for Publication Bias Using Only Significant ResultsPreprint
Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2014) · SSRN Electronic Journal · doi:10.2139/ssrn.2377290 · Cited by 22
Fraley, R. C., Chong, J. Y., Baacke, K. A., Greco, A. J., Guan, H., & Vazire, S. (2022) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459221120217 · Cited by 28
Statistical Guidance to Authors at Top-Ranked Journals across Scientific Disciplines
Hardwicke, T. E., Salholz‐Hillel, M., Malički, M., Szűcs, D., Bendixen, T., & Ioannidis, J. P. A. (2022) · The American Statistician · doi:10.1080/00031305.2022.2143897 · Cited by 23
“…in the edifice of scientific claims, but they are widely misused, misreported, and misinterpreted across several scientific disciplines (Altman 1982; Gigerenzer 2018; Nieuwenhuis, Forstmann, and Wagenmakers 2011; Sedlmeier and Gigerenzer 1989; Strasak et al. 2007; Wasserstein and Lazar 2016).”
BackgroundPower to the People: A Beginner’s Tutorial to Power Analysis using jamovi
Bartlett, J. E., & Charles, S. J. (2022) · Meta-Psychology · doi:10.15626/mp.2021.3078 · Cited by 31
Approximate sample size formulas for the two‐sample trimmed mean test with unequal variances
Luh, W., & Guo, J. (2006) · British Journal of Mathematical and Statistical Psychology · doi:10.1348/000711006x100491 · Cited by 21
Single-case research designs and the scientist-practitioner ideal in applied psychology.
Blampied, N. M. (2012) · American Psychological Association eBooks · doi:10.1037/13937-008 · Cited by 29
Estimation accuracy in the psychological sciences
Davis‐Stober, C. P., Dana, J. D., & Rouder, J. N. (2018) · PLoS ONE · doi:10.1371/journal.pone.0207239 · Cited by 22
Statistical Significance, Effect Size, and Replication: What Do the Journals Say?
DeVaney, T. A. (2001) · The Journal of Experimental Education · doi:10.1080/00220970109599490 · Cited by 23
Statistical Power in Plant Pathology Research
Gent, D. H., Esker, P. D., & Kriss, A. B. (2017) · Phytopathology · doi:10.1094/phyto-03-17-0098-le · Cited by 28
Organic vs. mechanistic coordination in distributed New Product Development (NPD) teams
Perea, C., & von Zedtwitz, M. (2018) · Journal of Engineering and Technology Management · doi:10.1016/j.jengtecman.2018.04.005 · Cited by 24
Tahamont, S., Jelveh, Z., Chalfin, A. J., Yan, S., & Hansen, B. (2020) · Journal of Quantitative Criminology · doi:10.1007/s10940-020-09461-x · Cited by 25
“1994; Sedlmeier and Gigerenzer 1989; Sherman 2007).3 Indeed, a common rule of thumb when planning an experiment is to fix statistical power at 80%, meaning that a researcher is willing to make a Type II error and fail to detect a real treatment effect 20% of the time (Cohen 1992).”
BackgroundInadequate reporting of statistical results
Héroux, M. E. (2016) · Journal of Neurophysiology · doi:10.1152/jn.00550.2016 · Cited by 20
“Unfortunately, such guidelines are often ignored (Sedlmeier and Gigerenzer 1989; Tressoldi et al. 2013).”
Metadata
- Authors
- Sedlmeier, P., & Gigerenzer, G.
- Venue
- Psychological Bulletin
- Published
- 1989-03
- Volume
- 105
- Issue
- 2
- Pages
- 309-316
- ISSN
- 0033-2909 (ISSN-L)
- 1939-1455
- OpenAlex
- W2155449866
- Cited by
- 748
- References
- 55
- Citations by year
- 2026: 3 · 2025: 7 · 2024: 7 · 2023: 14 · 2022: 22 · 2021: 30 · 2020: 23 · 2019: 30 · 2018: 34 · 2017: 35 · 2016: 26 · 2015: 26 · 2014: 30 · 2013: 24 · 2012: 25
- FWCI
- 27.89
- Type
- journal article
- Language
- en
- Open access
- green
- hdl.handle.net
- Licence
- other-oa
- Locations
- Psychological Bulletin (publishedVersion)
- Max Planck Digital Library (submittedVersion, other-oa)
- MPG.PuRe (Max Planck Society) (submittedVersion, other-oa)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › General Psychology › Academic and Historical Perspectives in Psychology
- Keywords
- statistical power, power analysis, effect size, multiple comparisons, non-significant results, null hypothesis significance testing, history of statistics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.