The DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
The test of significance in psychological research.
Abstract
The vast majority of investigations which pass for research in the field of psychology today entail the use of statistical tests of significance. The critical assumption involved in testing significance is that, if the experiment is conducted properly, the characteristics of the population have a designably determinative influence on samples drawn from it; that, for example, the mean of a population has a determinative influence on the mean of a sample drawn from it. One of the most viable alternatives to the use of the test of significance involves the theorem of Bayes’. The psychological literature is filled with misinterpretations of the nature of the test of significance. With the acceptance of the Fisher approach the psychological research worker also accepted, and then overused, the test of significance, employing it as the measure of the significance, in the largest sense of the word, of his research efforts.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
SNEDECOR, G. W. (1939) · Soil Science · doi:10.1097/00010694-193902000-00006 · Cited by 19,359
(2001) · Empirical Direction in Design and Analysis · doi:10.4324/9781410600004-24 · Cited by 6,808
Fisher, R. A. (1936) · BMJ · doi:10.1136/bmj.1.3923.554-a · Cited by 4,980
Keynes, J. M. (1973) · Palgrave Macmillan UK eBooks · doi:10.1007/978-1-349-00843-8 · Cited by 2,575
The two disciplines of scientific psychology.
Cronbach, L. J. (1957) · American Psychologist · doi:10.1037/h0043943 · Cited by 2,523
Experimental Design in Psychological Research
(2002) · Foundations of Cognitive Psychology · doi:10.7551/mitpress/3080.003.0011 · Cited by 2,465
Statistical Analysis in Psychology and Education.
Smith, H. F., & Ferguson, G. A. (1967) · Journal of the American Statistical Association · doi:10.2307/2283699 · Cited by 2,213
Statistical Decision Functions
Wald, A. (1949) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177730030 · Cited by 2,176
Tukey, J. W. (1962) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177704711 · Cited by 1,860
The statistical power of abnormal-social psychological research: A review.
Cohen, J. (1962) · The Journal of Abnormal and Social Psychology · doi:10.1037/h0045186 · Cited by 1,824
Cited by
Significance tests and goodness of fit in the analysis of covariance structures.
Bentler, P. M., & Bonett, D. G. (1980) · Psychological Bulletin · doi:10.1037//0033-2909.88.3.588 · Cited by 19,023
Cohen, J. (1994) · American Psychologist · doi:10.1037/0003-066x.49.12.997 · Cited by 4,003
Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations
Greenland, S., Senn, S. J., Rothman, K. J., Carlin, J. B., Poole, C., Goodman, S. N., & Altman, D. G. (2016) · European Journal of Epidemiology · doi:10.1007/s10654-016-0149-3 · Cited by 3,200
Neurocognitive deficit in schizophrenia: A quantitative review of the evidence.
Heinrichs, R. W., & Zakzanis, K. K. (1998) · Neuropsychology · doi:10.1037/0894-4105.12.3.426 · Cited by 2,798
A practical solution to the pervasive problems ofp values
Wagenmakers, E.-J. (2007) · Psychonomic Bulletin & Review · doi:10.3758/bf03194105 · Cited by 2,490
Using confidence intervals in within-subject designs
Loftus, G. R., & Masson, M. E. J. (1994) · Psychonomic Bulletin & Review · doi:10.3758/bf03210951 · Cited by 2,448
Ellis, P. D. (2010) · doi:10.1017/cbo9780511761676 · Cited by 2,461
Mccloskey, D. N. (2008) · The New Palgrave Dictionary of Economics · doi:10.1007/978-1-349-58802-2_1435 · Cited by 1,816
Statistical significance in psychological research.
Lykken, D. T. (1968) · Psychological Bulletin · doi:10.1037/h0026141 · Cited by 1,350
Nosek (2012) · doi:10.1177/1745691612459058 · Cited by 1,301
Jacobson, N. S., Follette, W. C., & Revenstorf, D. (1984) · Behavior Therapy · doi:10.1016/s0005-7894(84)80002-7 · Cited by 1,294
Interpersonal expectancy effects: the first 345 studies
Rosenthal, R., & Rubin, D. B. (1978) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00075506 · Cited by 1,158
The Insignificance of Statistical Significance Testing
Johnson, D. H. (1999) · The Journal of Wildlife Management · doi:10.2307/3802789 · Cited by 1,104
Null hypothesis significance testing: a review of an old and continuing controversy
Nickerson, R. S. (2000) · Psychological Methods · doi:10.1037/1082-989x.5.2.241 · Cited by 1,090
Consequences of prejudice against the null hypothesis.
Greenwald, A. G. (1975) · Psychological Bulletin · doi:10.1037/h0076157 · Cited by 1,022
“They have been expressed, in part, by several previous writers, the article by Bakan (1966) being perhaps closest to the approach I have taken.”
BackgroundInfluentialImpact of a Confounding Variable on a Regression Coefficient
FRANK, K. A. (2000) · Sociological Methods & Research · doi:10.1177/0049124100029002001 · Cited by 1,032
Theory-Testing in Psychology and Physics: A Methodological Paradox
Meehl, P. E. (1967) · Philosophy of Science · doi:10.1086/288135 · Cited by 962
The case against statistical significance testing
Carver, R. (1978) · Harvard Educational Review · doi:10.17763/haer.48.3.t490261645281841 · Cited by 931
Why Summaries of Research on Psychological Theories are Often Uninterpretable
Meehl, P. E. (1990) · Psychological Reports · doi:10.2466/pr0.1990.66.1.195 · Cited by 909
Gigerenzer, G. (2004) · The Journal of Socio-Economics · doi:10.1016/j.socec.2004.09.033 · Cited by 890
Schmidt, F. L. (1996) · Psychological Methods · doi:10.1037/1082-989x.1.2.115 · Cited by 830
Abandon Statistical Significance
McShane, B. B., Gal, D., Gelman, A., Robert, C., & Tackett, J. L. (2019) · The American Statistician · doi:10.1080/00031305.2018.1527253 · Cited by 791
Scientific Guidelines for Conducting Integrative Research Reviews
Cooper, H. M. (1982) · Review of Educational Research · doi:10.2307/1170314 · Cited by 784
Rosnow, R. L., & Rosenthal, R. (1996) · Psychological Methods · doi:10.1037/1082-989x.1.4.331 · Cited by 761
Do studies of statistical power have an effect on the power of studies?
Sedlmeier, P., & Gigerenzer, G. (1989) · Psychological Bulletin · doi:10.1037/0033-2909.105.2.309 · Cited by 748
McCarthy, M. A. (2007) · Cambridge University Press eBooks · doi:10.1017/cbo9780511802454 · Cited by 756
Toward an Integrated Theory of Schizotaxia, Schizotypy, and Schizophrenia
Meehl, P. E. (1990) · Journal of Personality Disorders · doi:10.1521/pedi.1990.4.1.1 · Cited by 751
Meehl, P. E. (1990) · Psychological Inquiry · doi:10.1207/s15327965pli0102_1 · Cited by 673
Facts Are More Important Than Novelty: Replication in the Education Sciences
Makel, M. C., & Plucker, J. A. (2014) · Educational Researcher · doi:10.3102/0013189x14545513 · Cited by 609
“…effect from large initial findings (e.g., Ioannidis, 2005a; Rhine, 1934/1997; Schooler, 2011), overreliance on null hypothesis testing (e.g., Bakan, 1966; Cohen, 1994; Johnson, 2013; LeBel & Peters, 2011; Lykken, 1968; Rozeboom, 1960; Wagenmakers, Wetzels, Borsboom, & van der Maas, 2011),…”
Nelson, L. D., Simmons, J., & Simonsohn, U. (2018) · Annual Review of Psychology · doi:10.1146/annurev-psych-122216-011836 · Cited by 548
Under what conditions does theory obstruct research progress?
Greenwald, A. G., Pratkanis, A. R., Leippe, M. R., & Baumgardner, M. H. (1986) · Psychological Review · doi:10.1037/0033-295x.93.2.216 · Cited by 574
Sequential hypothesis testing with Bayes factors: Efficiently testing mean differences.
Schönbrodt, F. D., Wagenmakers, E.-J., Zehetleitner, M., & Perugini, M. (2017) · Psychological Methods · doi:10.1037/met0000061 · Cited by 567
Onghena, P., & Van Houdenhove, B. (1992) · Pain · doi:10.1016/0304-3959(92)90144-z · Cited by 563
Cumming, G., & Finch, S. (2001) · Educational and Psychological Measurement · doi:10.1177/0013164401614002 · Cited by 528
Cooper, H. M. (1979) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.37.1.131 · Cited by 412
The epistemology of mathematical and statistical modeling: A quiet methodological revolution.
Rodgers, J. L. (2010) · American Psychologist · doi:10.1037/a0018326 · Cited by 402
Tryon, W. W. (2001) · Psychological Methods · doi:10.1037/1082-989x.6.4.371 · Cited by 391
P-Value, a true test of statistical significance? a cautionary note
Dahiru, T. (2011) · Annals of Ibadan Postgraduate Medicine · doi:10.4314/aipm.v6i1.64038 · Cited by 384
Statistical Inference as Severe Testing: How to Get Beyond the Statistics Wars
Mayo, D. G. (2018) · doi:10.1017/9781107286184 · Cited by 376
Psychology Will Be a Much Better Science When We Change the Way We Analyze Data
Loftus, G. R. (1996) · Current Directions in Psychological Science · doi:10.1111/1467-8721.ep11512376 · Cited by 361
Modeling Publication Selection Effects in Meta-Analysis
Hedges, L. V. (1992) · Statistical Science · doi:10.1214/ss/1177011364 · Cited by 359
Halsey, L. G. (2019) · Biology Letters · doi:10.1098/rsbl.2019.0174 · Cited by 357
Zakzanis, K. K. (2001) · Archives of Clinical Neuropsychology · doi:10.1093/arclin/16.7.653 · Cited by 346
AERA Editorial Policies regarding Statistical Significance Testing: Three Suggested Reforms
Thompson, B. (1996) · Educational Researcher · doi:10.2307/1176337 · Cited by 339
The Insignificance of Null Hypothesis Significance Testing
Gill, J. (1999) · Political Research Quarterly · doi:10.1177/106591299905200309 · Cited by 338
A Bayesian Perspective on the Reproducibility Project: Psychology
Etz, A., & Vandekerckhove, J. (2016) · PLOS ONE · doi:10.1371/journal.pone.0149794 · Cited by 295
“Readers who wish to 65 evaluate original findings and replications alike must take into account the fact that our 66 “very publication practices themselves are part and parcel of the probabilistic processes 67 on which we base our conclusions concerning the nature of psychological 68 phenomena” [12] (p.”
Statistical power of psychological research: What have we gained in 20 years?
Rossi, J. S. (1990) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.58.5.646 · Cited by 323
On the relation of clinical research to clinical practice: Current issues, new directions.
Barlow, D. H. (1981) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.49.2.147 · Cited by 312
Magnitude of experimental effect and a table for its rapid estimation.
Friedman, H. (1968) · Psychological Bulletin · doi:10.1037/h0026258 · Cited by 312
Effect-Size Estimates: Issues and Problems in Interpretation
Fern, E. F., & Monroe, K. B. (1996) · Journal of Consumer Research · doi:10.1086/209469 · Cited by 301
When Null Hypothesis Significance Testing Is Unsuitable for Research: A Reassessment
Szucs, D., & Ioannidis, J. P. A. (2017) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2017.00390 · Cited by 298
Null hypothesis significance testing: On the survival of a flawed method.
Krueger, J. (2001) · American Psychologist · doi:10.1037//0003-066x.56.1.16 · Cited by 282
Significance Tests Die Hard: The Amazing Persistence of a Probabilistic Misconception
Falk, R., & Greenbaum, C. W. (1995) · Theory & Psychology · doi:10.1177/0959354395051004 · Cited by 279
ARE WOMEN REALLY MORE RISK‐AVERSE THAN MEN? A RE‐ANALYSIS OF THE LITERATURE USING EXPANDED METHODS
Nelson, J. A. (2015) · Journal of Economic Surveys · doi:10.1111/joes.12069 · Cited by 276
Effect sizes and p values: What should be reported and what should be replicated?
Greenwald, A., Gonzalez, R., Harris, R. J., & Guthrie, D. (1996) · Psychophysiology · doi:10.1111/j.1469-8986.1996.tb02121.x · Cited by 274
Attachment security A meta-analysis of maternal mental health correlates
Atkinson, L. (2000) · Clinical Psychology Review · doi:10.1016/s0272-7358(99)00023-9 · Cited by 272
Statistical Power and Effect Size in Marketing Research
Sawyer, A. G., & Ball, A. D. (1981) · Journal of Marketing Research · doi:10.1177/002224378101800302 · Cited by 269
Methodological Wishes for the Next Decade and How to Make Wishes Come True
Aguinis, H., & Edwards, J. R. (2014) · Journal of Management Studies · doi:10.1111/joms.12058 · Cited by 267
Evaluating Results Using Corrected and Uncorrected Effect Size Estimates
Snyder, P., & Lawson, S. (1993) · The Journal of Experimental Education · doi:10.1080/00220973.1993.10806594 · Cited by 265
The Effect of Sample Size on the Meaning of Significance Tests
Royall, R. M. (1986) · The American Statistician · doi:10.1080/00031305.1986.10475424 · Cited by 257
The reproducibility of research and the misinterpretation of p -values
Colquhoun, D. (2017) · Royal Society Open Science · doi:10.1098/rsos.171085 · Cited by 256
Misinterpretations of Significance: A Problem Students Share with Their Teachers?
Haller, H., & Krauss, S. (2002) · IPN - Institute for Science Education at the University of Kiel, Germany · doi:10.23668/psycharchives.12769 · Cited by 250
Hedges, L. V. (1984) · Journal of Educational Statistics · doi:10.3102/10769986009001061 · Cited by 241
On the logic and purpose of significance testing.
Cortina, J. M., & Dunlap, W. P. (1997) · Psychological Methods · doi:10.1037/1082-989x.2.2.161 · Cited by 166
A historical review of publication bias
Marks‐Anglin, A., & Chen, Y. (2020) · Research Synthesis Methods · doi:10.1002/jrsm.1452 · Cited by 165
EREZ, A., BLOOM, M. C., & WELLS, M. T. (1996) · Personnel Psychology · doi:10.1111/j.1744-6570.1996.tb01801.x · Cited by 161
A Proposal for a New Editorial Policy in the Social Sciences
Walster, G. W., & Cleary, T. A. (1970) · The American Statistician · doi:10.1080/00031305.1970.10478884 · Cited by 144
Significance tests as sorcery: Science is empirical—significance tests are not
Lambdin, C. (2012) · Theory & Psychology · doi:10.1177/0959354311429854 · Cited by 143
Misprescription and misuse of one-tailed tests
LOMBARDI, C. M., & HURLBERT, S. H. (2009) · Austral Ecology · doi:10.1111/j.1442-9993.2009.01946.x · Cited by 93
Bungert, M., Liebke, L., Thome, J., Haeussler, K., Bohus, M., & Lis, S. (2015) · Borderline Personality Disorder and Emotion Dysregulation · doi:10.1186/s40479-015-0025-x · Cited by 92
“However, since we could confirm statistically increased rejection sensitivity in the remitted patients (despite the small sample size and low statistical power), it can be assumed that the population effect is strong [71].”
BackgroundEven statisticians are not immune to misinterpretations of Null Hypothesis Significance Tests
Lecoutre, M., Poitevineau, J., & Lecoutre, B. (2003) · International Journal of Psychology · doi:10.1080/00207590244000250 · Cited by 86
Malignant side effects of null-hypothesis significance testing
Branch, M. (2014) · Theory & Psychology · doi:10.1177/0959354314525282 · Cited by 81
When can we Conclude that Treatments or Programs “Don’t Work”?
Weisburd, D., Lum, C. M., & Yang, S.-M. (2003) · The ANNALS of the American Academy of Political and Social Science · doi:10.1177/0002716202250782 · Cited by 76
Ultrashort-range, high-frequency communication by female mice shapes social interactions
Warren, M. R., Clein, R. S., Spurrier, M. S., Roth, E. D., & Neunuebel, J. P. (2020) · Scientific Reports · doi:10.1038/s41598-020-59418-0 · Cited by 47
Cassidy, S. A., Dimova, R., Giguère, B., Spence, J. R., & Stanley, D. J. (2019) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245919858072 · Cited by 39
(2006) · PubMed · Cited by 23
Verdam, M. G. E., Oort, F. J., & Sprangers, M. A. G. (2014) · Quality of Life Research · doi:10.1007/s11136-013-0437-2 · Cited by 18
Theobald, D. L. (2011) · Biology Direct · doi:10.1186/1745-6150-6-60 · Cited by 16
“Frequentist hypothesis tests (of both Fisher and Neyman-Pearson flavors) have been highly criticized and disparaged for many decades by professional statisticians [44-71].”
BackgroundMonterde-i-Bort, H., Frías-Navarro, D., & Pascual-Llobell, J. (2010) · European Journal of Psychology of Education · doi:10.1007/s10212-010-0021-x · Cited by 14
(2009) · PubMed · Cited by 13
Parapsychology is science, but its findings are inconclusive
Akers, C. (1987) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00054480 · Cited by 11
Metadata
- Authors
- Bakan, D.
- Venue
- Psychological Bulletin
- Published
- 1966
- Volume
- 66
- Issue
- 6
- Pages
- 423-437
- ISSN
- 0033-2909 (ISSN-L)
- 1939-1455
- OpenAlex
- W2074941944
- Cited by
- 911
- References
- 35
- FWCI
- 10.44
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.