Student or educator? Sign up with your school email and get 6 months of Pro free.
Cited by 196

The DOI matches its Crossref record; 4 of the 4 references it cites were found in a registry or an index.

What Statistical Significance Testing Is, and What It Is Not

Shaver, James P.·1993·The Journal of Experimental Education

Abstract

A test of statistical significance addresses the question, How likely is a result, assuming the null hypotheses to be true. Randomness, a central assumption underlying commonly used tests of statistical significance, is rarely attained, and the effects of its absence rarely acknowledged. Statistical significance does not speak to the probability that the null hypothesis or an alternative hypothesis is true or false, to the probability that a result would be replicated, or to treatment effects, nor is it a valid indicator of the magnitude or the importance of a result. The persistence of statistical significance testing is due to many subtle factors. Journal editors are not to blame, but as publishing gatekeepers they could diminish its dysfunctional use.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1080/00220973.1993.10806592

Cites

4 of 4 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 4 references

Research works printed but not linked: 54

  1. Bakan D., 1967, On method: Toward a reconstruction of psychological investigation.
  2. 10.1037/0033-2909.99.3.388 · DOI · Paper page
  3. 10.1126/science.134.3479.596 · DOI · Paper page
  4. 10.1037/0003-066X.46.6.654 · doi:10.1037/0003-066x.46.6.654
  5. Berk R. A., 1978, Evaluation Studies Review Annual, 3, 190
  6. Bracey G. W., 1991, Phi Delta Kappan, 73
  7. 10.2307/1164796 · DOI
  8. Campbell K. E., 1979, Replications in Social Psychology, 1, 3
  9. 10.1177/002246698401800103 · doi:10.1177/002246698401800103
  10. Carver R. P., 1978, Harvard Educational Review, 48, 378, 10.17763/haer.48.3.t490261645281841 · DOI · Paper page
  11. Cohen J., 1969, Statistical power analysis for the behavioral sciences.
  12. Cohen J., 1988, Statistical power analysis for the behavioral sciences
  13. Cohen J., 1983, Applied multiple regression/correlation analysis for the behavioral sciences
  14. Cohen S. A., 1981, Testing research hypotheses with critical ES instead of statistical significance in educational research.
  15. 10.1037/h0076829 · DOI · Paper page
  16. 10.1037/0003-066X.42.2.145 · doi:10.1037/0003-066x.42.2.145
  17. Ferguson G. A., 1989, Statistical analysis in psychology and education
  18. Festinger L., 1957, A theory of cognitive dissonance., 10.1515/9781503620766 · DOI · Paper page
  19. 10.1037/0022-006X.51.1.65 · doi:10.1037/0022-006x.51.1.65
  20. 10.1017/CBO9780511720482 · DOI · Paper page
  21. Glass G. V., 1984, Statistical methods in education and psychology
  22. Glass G. V., 1981, Meta-analysis in social research.
  23. 10.1037/0003-066X.44.6.964 · doi:10.1037/0003-066x.44.6.964
  24. Hays W. L., 1973, Statistics for the social sciences
  25. Hedges L. V., 1985, Statistical methods for meta-analysis.
  26. Hunter J. E., 1990, Methods of meta-analysis: Correcting error and bias in research findings.
  27. 10.3102/00346543050003438 · doi:10.3102/00346543050003438 · Paper page
  28. 10.1037/0003-066X.43.8.635 · DOI
  29. 10.1126/science.255.5045.690 · DOI
  30. McHugh R. B., 1964, The Journal of Experimental Education, 33, 169, 10.1080/00220973.1964.11010869 · doi:10.1080/00220973.1964.11010869
  31. 10.1086/288135 · DOI · Paper page
  32. 10.3102/0013189X018002005 · DOI · Paper page
  33. Morrison D. E., 1970, The significance test controversy: A reader.
  34. Neuliep J. W., 1991, Replication research in the social sciences.
  35. 10.3102/0013189X020009013 · DOI
  36. Rosenthal R., 1991, Replication research in the social sciences, 1
  37. 10.1037/0003-066X.44.10.1276 · DOI · Paper page
  38. Salsburg D. S., 1985, The American Statistician, 39, 220, 10.1080/00031305.1985.10479435 · DOI · Paper page
  39. 10.3102/00346543055003319 · DOI · Paper page
  40. 10.1037/h0035628 · doi:10.1037/h0035628
  41. 10.3102/0013189X008001003 · doi:10.3102/0013189x008001003
  42. Shaver J. P., 1980, Readdressing the role of statistical tests of significance.
  43. Shaver J. P., 1985, Phi Delta Kappan, 67, 57
  44. Shaver J. P., 1985, Phi Delta Kappan, 67, 138
  45. Shaver J. P., 1987, Mandates for curriculum and instruction from educational and psychological research.
  46. Shaver J. P., 1991, Handbook of research on social studies teaching and learning, 83
  47. 10.1080/00933104.1980.10506078 · DOI
  48. Shaver J. P., 1980, American Educational Research Journal. Educational Researcher, 9, 9
  49. 10.1037/h0037351 · DOI
  50. 10.1037/h0047662 · DOI · Paper page
  51. Thompson B., 1989, Advances in social science methodology, 1, 1
  52. 10.1037/h0027108 · DOI · Paper page
  53. Winch R. F., 1969, The American Sociologist, 4, 140
  54. Winer B. J., 1991, Statistical principles in experimental design

Metadata

Authors
Shaver, J. P.
Venue
The Journal of Experimental Education
Published
1993-07
Volume
61
Issue
4
Pages
293-316
ISSN
  • 0022-0973 (ISSN-L)
  • 1940-0683
OpenAlex
W1976163365
Cited by
196
References
4
FWCI
7.41
Type
journal article
Language
en
Open access
closed

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: What Statistical Significance Testing Is, and What It Is Not. https://app.cochat.ai/papers/10.1080/00220973.1993.10806592
The paper
Shaver, J. P. (1993). What Statistical Significance Testing Is, and What It Is Not. The Journal of Experimental Education. https://doi.org/10.1080/00220973.1993.10806592

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.