The DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
The Insignificance of Null Hypothesis Significance Testing
Abstract
The current method of hypothesis testing in the social sciences is under intense criticism, yet most political scientists are unaware of the important issues being raised. Criticisms focus on the construction and interpretation of a procedure that has dominated the reporting of empirical results for over fifty years. There is evidence that null hypothesis significance testing as practiced in political science is deeply flawed and widely misunderstood. This is important since most empirical work argues the value of findings through the use of the null hypothesis significance test. In this article I review the history of the null hypothesis significance testing paradigm in the social sciences and discuss major problems, some of which are logical inconsistencies while others are more interpretive in nature. I suggest alter native techniques to convey effectively the importance of data-analytic findings. These recommendations are illustrated with examples using empirical political science publications.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Gelman, A., Carlin, J. B., Stern, H. S., & Rubin, D. B. (2003) · doi:10.1201/9780429258480 · Cited by 27,276
Methods of Meta-Analysis: Correcting Error and Bias in Research Findings
Schmidt, F. L., & Hunter, J. E. (2015) · doi:10.4135/9781483398105 · Cited by 5,542
Primary, Secondary, and Meta-Analysis of Research
GLASS, G. V. (1976) · Educational Researcher · doi:10.3102/0013189x005010003 · Cited by 4,890
Cohen, J. (1994) · American Psychologist · doi:10.1037/0003-066x.49.12.997 · Cited by 4,003
The statistical power of abnormal-social psychological research: A review.
Cohen, J. (1962) · The Journal of Abnormal and Social Psychology · doi:10.1037/h0045186 · Cited by 1,824
Consequences of prejudice against the null hypothesis.
Greenwald, A. G. (1975) · Psychological Bulletin · doi:10.1037/h0076157 · Cited by 1,022
Sterling, T. D. (1959) · Journal of the American Statistical Association · doi:10.1080/01621459.1959.10501497 · Cited by 964
Wolf, F. (1986) · doi:10.4135/9781412984980 · Cited by 784
Do studies of statistical power have an effect on the power of studies?
Sedlmeier, P., & Gigerenzer, G. (1989) · Psychological Bulletin · doi:10.1037/0033-2909.105.2.309 · Cited by 748
Counterfactuals and Hypothesis Testing in Political Science
Fearon, J. D. (1991) · World Politics · doi:10.2307/2010470 · Cited by 729
Cited by
(2011) · Cited by 675
A solution to minimum sample size for regressions
Jenkins, D. G., & Quintana-Ascencio, P. F. (2020) · PLOS ONE · doi:10.1371/journal.pone.0229345 · Cited by 665
“Power analyses carry forward fundamental problems with null hypothesis inference, which is the long-standing basis for statistical analyses but that has been recently and widely criticized for several reasons [25,26,27,28,29].”
BackgroundThe perils of policy by p-value: Predicting civil conflicts
Ward, M. D., Greenhill, B. D., & Bakke, K. M. (2010) · Journal of Peace Research · doi:10.1177/0022343309356491 · Cited by 476
An Empirical Evaluation of Explanations for State Repression
HILL, D. W., & JONES, Z. M. (2014) · American Political Science Review · doi:10.1017/s0003055414000306 · Cited by 450
Significance tests as sorcery: Science is empirical—significance tests are not
Lambdin, C. (2012) · Theory & Psychology · doi:10.1177/0959354311429854 · Cited by 143
Thiem, A., Baumgartner, M., & Bol, D. (2016) · Comparative Political Studies · doi:10.1177/0010414014565892 · Cited by 138
Duff, B. R. L., & Faber, R. J. (2011) · Journal of Advertising · doi:10.2753/joa0091-3367400204 · Cited by 137
Colaresi, M., & Mahmood, Z. (2017) · Journal of Peace Research · doi:10.1177/0022343316682065 · Cited by 82
Bayesian Data Analysis in Empirical Software Engineering Research
Furia, C. A., Feldt, R., & Torkar, R. (2019) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2019.2935974 · Cited by 42
Do hotter temperatures increase the incidence of self-harm hospitalisations?
Williams, M. N., Hill, S. R., & Spicer, J. (2016) · Psychology, Health & Medicine · doi:10.1080/13548506.2015.1028945 · Cited by 36
Ortega, A., & Navarrete, G. (2017) · Bayesian Inference · doi:10.5772/intechopen.70230 · Cited by 31
Theobald, D. L. (2011) · Biology Direct · doi:10.1186/1745-6150-6-60 · Cited by 16
Metadata
- Authors
- Gill, J.
- Venue
- Political Research Quarterly
- Publisher
- SAGE Publications
- Published
- 1999-09
- Volume
- 52
- Issue
- 3
- Pages
- 647-674
- ISSN
- 1065-9129 (ISSN-L)
- 1938-274X
- OpenAlex
- W2024096835
- Cited by
- 338
- References
- 13
- FWCI
- 78.05
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
