The DOI matches its Crossref record.
The cult of statistical significance: how the standard error costs us jobs, justice, and lives
Abstract
Statistical significance, a technique that dominates medicine, economics, psychology, and many other scientific fields, has been a huge mistake. The outcome is a case study in bad science - how it originates and how it grows. These sciences, from agronomy to zoology, the authors find, engage that doesn't test and estimating that doesn't estimate. Heedless of magnitude and of a genuine engagement with alternative hypotheses, they testimate. Null hypothesis significance testing is in other words a scientific train-wreck, about which a small group of statisticians have been warning for a century.Ziliak and McCloskey's book shows field by field how the wreck happened, reports on the fatalities, and offers a quantitative way forward. The facts will startle the outside reader: how could a group of brilliant scientists wander so far away from scientific magnitudes? And it will inspirit the scientists who seek conscious interpretations of oomph rather than arbitrary columns of t-tests: how can the statistical sciences get back on track, and fulfill their quantitative promise?Ziliak and McCloskey measure the disaster in their home field of economics, and in psychology, epidemiology, and medical science. They touch as well on law, biology, psychiatry, pharmacology, sociology, political science, education, forensics, and other fields in the grip of significance. The book shows for the first time how wide the disaster is, and how bad for science, and it traces the problem to its historical, sociological, and philosophical roots. Many statisticians have complained about it before, but have complained science-by-science.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cited by
A basic introduction to fixed-effect and random-effects models for meta-analysis
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2010) · Research Synthesis Methods · doi:10.1002/jrsm.12 · Cited by 6,793
The ASA Statement on p -Values: Context, Process, and Purpose
Wasserstein, R. L., & Lazar, N. A. (2016) · The American Statistician · doi:10.1080/00031305.2016.1154108 · Cited by 6,306
Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations
Greenland, S., Senn, S. J., Rothman, K. J., Carlin, J. B., Poole, C., Goodman, S. N., & Altman, D. G. (2016) · European Journal of Epidemiology · doi:10.1007/s10654-016-0149-3 · Cited by 3,200
Moving to a World Beyond “ p < 0.05”
Wasserstein, R. L., Schirm, A. L., & Lazar, N. A. (2019) · The American Statistician · doi:10.1080/00031305.2019.1583913 · Cited by 2,976
Ellis, P. D. (2010) · doi:10.1017/cbo9780511761676 · Cited by 2,461
Bayesian Versus Orthodox Statistics: Which Side Are You On?
Dienes, Z. (2011) · Perspectives on Psychological Science · doi:10.1177/1745691611406920 · Cited by 938
QuantCrit: education, policy, ‘Big Data’ and principles for a critical race theory of statistics
Gillborn, D., Warmington, P., & Demack, S. (2018) · Race Ethnicity and Education · doi:10.1080/13613324.2017.1377417 · Cited by 687
Bridging Practice and Theory: A Design Science Approach
Holmström, J., Ketokivi, M., & Hameri, A. (2009) · Decision Sciences · doi:10.1111/j.1540-5915.2008.00221.x · Cited by 674
Rigdon, E. E. (2016) · European Management Journal · doi:10.1016/j.emj.2016.05.006 · Cited by 646
Kline, R. B. (2015) · Basic and Applied Social Psychology · doi:10.1080/01973533.2015.1049349 · Cited by 460
Gronum, S., Verreynne, M.-L., & Kastelle, T. (2012) · Journal of Small Business Management · doi:10.1111/j.1540-627x.2012.00353.x · Cited by 459
Partial Least Squares Methods: Partial Least Squares Correlation and Partial Least Square Regression
Abdi, H., & Williams, L. J. (2013) · Methods in Molecular Biology · doi:10.1007/978-1-62703-059-5_23 · Cited by 448
An overview of Hofstede-inspired country-level culture research in international business since 2006
Beugelsdijk, S., Kostova, T., & Roth, K. (2017) · Journal of International Business Studies · doi:10.1057/s41267-016-0038-8 · Cited by 443
Climate change prediction: Erring on the side of least drama?
Brysse, K., Oreskes, N., O’Reilly, J., & Oppenheimer, M. (2013) · Global Environmental Change · doi:10.1016/j.gloenvcha.2012.10.008 · Cited by 433
Ketokivi, M., & McIntosh, C. N. (2017) · Journal of Operations Management · doi:10.1016/j.jom.2017.05.001 · Cited by 429
Amrhein, V., Trafimow, D., & Greenland, S. (2019) · The American Statistician · doi:10.1080/00031305.2018.1543137 · Cited by 425
Medical students-as-teachers: a systematic review of peer-assisted teaching during medical school
Hill, A., Yu, Wilson, Hawken, Singh, & Lemanu (2011) · Advances in Medical Education and Practice · doi:10.2147/amep.s14383 · Cited by 383
TWO STRATEGIES FOR INDUCTIVE REASONING IN ORGANIZATIONAL RESEARCH.
KETOKIVI, M., & MANTERE, S. (2010) · Academy of Management Review · doi:10.5465/amr.2010.48463336 · Cited by 358
Four reasons to prefer Bayesian analyses over significance testing
Dienes, Z., & Mclatchie, N. (2018) · Psychonomic Bulletin & Review · doi:10.3758/s13423-017-1266-z · Cited by 354
The Cult of Statistical Significance: How the Standard Error Costs Us Jobs, Justice, and Lives
McCloskey, D., & Ziliak, S. (2008) · University of Michigan Press eBooks · doi:10.3998/mpub.186351 · Cited by 334
The effects of financialization on investment: evidence from firm-level data for the UK
Tori, D., & Onaran, Ö. (2018) · Cambridge Journal of Economics · doi:10.1093/cje/bex085 · Cited by 324
The Complexities of Agent-Based Modeling Output Analysis
Lee, J.-S., Filatova, T., Ligmann-Zielinska, A., Hassani-Mahmooei, B., Stonedahl, F., Lorscheid, I., Voinov, A., Polhill, G., Sun, Z., & Parker, D. C. (2015) · Journal of Artificial Societies and Social Simulation · doi:10.18564/jasss.2897 · Cited by 324
Illusions in regression analysis
Armstrong, J. S. (2012) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2012.02.001 · Cited by 322
Greenland, S. (2019) · The American Statistician · doi:10.1080/00031305.2018.1529625 · Cited by 313
The earth is flat ( p > 0.05): significance thresholds and the crisis of unreplicable research
Amrhein, V., Korner-Nievergelt, F., & Roth, T. (2017) · PeerJ · doi:10.7717/peerj.3544 · Cited by 308
Final Collapse of the Neyman-Pearson Decision Theoretic Framework and Rise of the neoFisherian
Hurlbert, S. H., & Lombardi, C. M. (2009) · Annales Zoologici Fennici · doi:10.5735/086.046.0501 · Cited by 301
Swedberg, R. (2020) · The Production of Knowledge · doi:10.1017/9781108762519.002 · Cited by 293
Statistics and Truth in Phylogenomics
Kumar, S., Filipski, A. J., Battistuzzi, F. U., Kosakovsky Pond, S. L., & Tamura, K. (2012) · Molecular Biology and Evolution · doi:10.1093/molbev/msr202 · Cited by 287
Tyler, T. R., Goff, P. A., & MacCoun, R. J. (2015) · Psychological Science in the Public Interest · doi:10.1177/1529100615617791 · Cited by 280
Qualitative generalising in accounting research: concepts and strategies
Parker, L. D., & Northcott, D. (2016) · Accounting, Auditing & Accountability Journal · doi:10.1108/aaaj-04-2015-2026 · Cited by 269
Seven deadly sins of contemporary quantitative political analysis
Schrodt, P. A. (2014) · Journal of Peace Research · doi:10.1177/0022343313499597 · Cited by 264
The environmental effects of the “twin” green and digital transition in European regions
Bianchini, S., Damioli, G., & Ghisetti, C. (2023) · Environmental and Resource Economics · doi:10.1007/s10640-022-00741-7 · Cited by 263
Research Design in Clinical Psychology
Kazdin, A. E. (2021) · doi:10.1017/9781108993647 · Cited by 240
Why People Cooperate: The Role of Social Motivations
Tyler, T. R. (2010) · doi:10.1515/9781400836666 · Cited by 169
Conceptual challenges for interpretable machine learning
Watson, D. S. (2022) · Synthese · doi:10.1007/s11229-022-03485-5 · Cited by 87
Significance testing in empirical finance: a critical review and assessment
Kim, J. H., & Ji, P. I. (2015) · Journal of Empirical Finance · doi:10.1016/j.jempfin.2015.08.006 · Cited by 86
The illusion of predictability: How regression statistics mislead experts
Soyer, E., & Hogarth, R. M. (2012) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2012.02.002 · Cited by 84
Malignant side effects of null-hypothesis significance testing
Branch, M. (2014) · Theory & Psychology · doi:10.1177/0959354314525282 · Cited by 81
Assessing ‘what works’ in international development: meta-analysis for sophisticated dummies
Duvendack, M., Hombrados, J. G., Palmer-Jones, R., & Waddington, H. (2012) · Journal of Development Effectiveness · doi:10.1080/19439342.2012.710642 · Cited by 52
Pictures and words: Spanish and English vocabulary in classrooms.
Branum-Martin, L., Mehta, P. D., Francis, D. J., Foorman, B. R., Cirino, P. T., Miller, J. F., & Iglesias, A. (2009) · Journal of Educational Psychology · doi:10.1037/a0015817 · Cited by 34
Wu, J., Hasselgren, C., Zettergren, A., Zetterberg, H., Blennow, K., Skoog, I., & Halleröd, B. (2020) · Aging & Mental Health · doi:10.1080/13607863.2018.1531368 · Cited by 18
“However, as pointed out by Ziliak and McCloskey (2008), it is important also to take into account the estimate size when interpreting results.”
BackgroundTheobald, D. L. (2011) · Biology Direct · doi:10.1186/1745-6150-6-60 · Cited by 16
Neglect of Multiplicity When Testing Families of Related Hypotheses*
Holland, B., fang, S. W., & Basu, S. (2009) · doi:10.2139/ssrn.1466343 · Cited by 13
Ziliak, S. T. (2012) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2012.02.004 · Cited by 8
Metadata
- Venue
- Choice Reviews Online
- Publisher
- American Library Association
- Published
- 2008-09-01
- Volume
- 46
- Issue
- 01
- Pages
- 46-0372-46-0372
- ISSN
- 0009-4978 (ISSN-L)
- 1523-8253
- OpenAlex
- W39572901
- Cited by
- 713
- References
- 0
- Type
- book review
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.