Statistical Inference as Severe Testing: How to Get Beyond the Statistics Wars
Open accessAbstract
Mounting failures of replication in social and biological sciences give a new urgency to critically appraising proposed reforms. This book pulls back the cover on disagreements between experts charged with restoring integrity to science. It denies two pervasive views of the role of probability in inference: to assign degrees of belief, and to control error rates in a long run. If statistical consumers are unaware of assumptions behind rival evidence reforms, they can't scrutinize the consequences that affect them (in personalized medicine, psychology, etc.). The book sets sail with a simple tool: if little has been done to rule out flaws in inferring a claim, then it has not passed a severe test. Many methods advocated by data experts do not stand up to severe scrutiny and are in tension with successful strategies for blocking or accounting for cherry picking and selective reporting. Through a series of excursions and exhibits, the philosophy and history of inductive inference come alive. Philosophical tools are put to work to solve problems about science and pseudoscience, induction and falsification.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The text we found for this paper looks like another document, so the self-check wasn’t run.
Cites
1 retracted
Genomic signatures to guide the use of chemotherapeutics
Potti, A., Dressman, H. K., Bild, A., Riedel, R. F., Chan, G., Sayer, R. A., Cragun, J. M., Cottrill, H. M., Kelley, M. J., Petersen, R. P., et al. (2006) · Nature Medicine · doi:10.1038/nm1491 · Cited by 595 · see citing works
RetractedControlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482 · see citing works
Bootstrap methods: another look at the jackknife
Efron, B. (1979) · The Annals of Statistics · doi:10.1214/aos/1176344552 · Cited by 17,602 · see citing works
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505 · see citing works
Aad, G., Abajyan, T., Abbott, B., Abdallah, J., Khalek, S. A., Abdelalim, A. A., Abdinov, O., Aben, R. Z., Abi, B., Abolins, M., et al. (2012) · Physics Letters B · doi:10.1016/j.physletb.2012.08.020 · Cited by 10,498 · see citing works
Estimating the reproducibility of psychological science
Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515 · see citing works
Judgment under Uncertainty: Heuristics and Biases
(1982) · doi:10.1017/cbo9780511809477 · Cited by 8,240 · see citing works
The new frontier of genome engineering with CRISPR-Cas9
Doudna, J. A., & Charpentier, E. (2014) · Science · doi:10.1126/science.1258096 · Cited by 7,062 · see citing works
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567 · see citing works
The ASA Statement on <i>p</i> -Values: Context, Process, and Purpose
Wasserstein, R. L., & Lazar, N. A. (2016) · The American Statistician · doi:10.1080/00031305.2016.1154108 · Cited by 6,306 · see citing works
No adjustments are needed for multiple comparisons.
Rothman, K. J. (1990) · Epidemiology · doi:10.1097/00001648-199001000-00010 · Cited by 5,965 · see citing works
Novel Proteinaceous Infectious Particles Cause Scrapie
Prusiner, S. B. (1982) · Science · doi:10.1126/science.6801762 · Cited by 5,121 · see citing works
On the Theory of Scales of Measurement
Stevens, S. S. (1946) · Science · doi:10.1126/science.103.2684.677 · Cited by 4,761 · see citing works
1,500 scientists lift the lid on reproducibility
Baker, M. (2016) · Nature · doi:10.1038/533452a · Cited by 4,209 · see citing works
The Large-Sample Distribution of the Likelihood Ratio for Testing Composite Hypotheses
Wilks, S. S. (1938) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177732360 · Cited by 3,719 · see citing works
Theory and Experiment in Gravitational Physics
Will, C. M. (2018) · doi:10.1017/9781316338612 · Cited by 3,637 · see citing works
Theory and Experiment in Gravitational Physics
Will, C. M. (1993) · doi:10.1017/cbo9780511564246 · Cited by 3,632
A manifesto for reproducible science
Munafò, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., Percie du Sert, N., Simonsohn, U., Wagenmakers, E.-J., Ware, J. J., & Ioannidis, J. P. A. (2017) · Nature Human Behaviour · doi:10.1038/s41562-016-0021 · Cited by 3,397 · see citing works
Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations
Greenland, S., Senn, S. J., Rothman, K. J., Carlin, J. B., Poole, C., Goodman, S. N., & Altman, D. G. (2016) · European Journal of Epidemiology · doi:10.1007/s10654-016-0149-3 · Cited by 3,200 · see citing works
Cox, D. R., & Hinkley, D. V. (1974) · doi:10.1007/978-1-4899-2887-0 · Cited by 3,089 · see citing works
The Methodology of Scientific Research Programmes: Philosophical Papers
Lakatos, I. (1978) · doi:10.1017/cbo9780511621123 · Cited by 3,045 · see citing works
Redefine statistical significance
Benjamin, D. J., Berger, J. O., Johannesson, M., Nosek, B. A., Wagenmakers, E.-J., Berk, R., Bollen, K. A., Brembs, B., Brown, L., Camerer, C., et al. (2017) · Nature Human Behaviour · doi:10.1038/s41562-017-0189-z · Cited by 2,772 · see citing works
A practical solution to the pervasive problems ofp values
Wagenmakers, E.-J. (2007) · Psychonomic Bulletin & Review · doi:10.3758/bf03194105 · Cited by 2,490 · see citing works
Ellis, P. D. (2010) · doi:10.1017/cbo9780511761676 · Cited by 2,461 · see citing works
IX. On the problem of the most efficient tests of statistical hypotheses
Neyman, J., & Pearson, E. S. (1933) · Philosophical Transactions of the Royal Society of London Series A Containing Papers of a Mathematical or Physical Character · doi:10.1098/rsta.1933.0009 · Cited by 2,370 · see citing works
Goldman, A. I. (1999) · doi:10.1093/0198238207.001.0001 · Cited by 2,172 · see citing works
Lakens, D. (2017) · Social Psychological and Personality Science · doi:10.1177/1948550617697177 · Cited by 2,161 · see citing works
The Abuse of Power: The Pervasive Fallacy of Power Calculations for Data Analysis
Hoenig, J. M., & Heisey, D. M. (2001) · The American Statistician · doi:10.1198/000313001300339897 · Cited by 2,046 · see citing works
Harlow, H. F. (1958) · American Psychologist · doi:10.1037/h0047884 · Cited by 2,022 · see citing works
POSTERIOR PREDICTIVE ASSESSMENT OF MODEL FITNESS VIA REALIZED DISCREPANCIES
Gelman, A., Meng, X., & Stern, H. S. (1996) · Cited by 1,982 · see citing works
Shaffer, J. P. (1995) · Annual Review of Psychology · doi:10.1146/annurev.ps.46.020195.003021 · Cited by 1,960 · see citing works
Meehl, P. E. (1978) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.46.4.806 · Cited by 1,952 · see citing works
Testing Statistical Hypotheses
Lehmann, E. L. (1986) · Springer texts in statistics · doi:10.1007/978-1-4757-1923-9 · Cited by 1,876 · see citing works
The statistical power of abnormal-social psychological research: A review.
Cohen, J. (1962) · The Journal of Abnormal and Social Psychology · doi:10.1037/h0045186 · Cited by 1,824 · see citing works
Giere, R. N. (1988) · University of Chicago Press eBooks · doi:10.7208/chicago/9780226292038.001.0001 · Cited by 1,822 · see citing works
Things I have learned (so far).
Cohen, J. (1990) · American Psychologist · doi:10.1037/0003-066x.45.12.1304 · Cited by 1,781 · see citing works
Bayesian statistical inference for psychological research.
Edwards, W., Lindman, H., & Savage, L. J. (1963) · Psychological Review · doi:10.1037/h0044139 · Cited by 1,734 · see citing works
ELECTRICAL WIRING CONFIGURATIONS AND CHILDHOOD CANCER
Wertheimer, N. M., & Leeper, E. (1979) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a112681 · Cited by 1,643 · see citing works
Freiman, J. A., Chalmers, T. C., Smith, H., & Kuebler, R. R. (1978) · New England Journal of Medicine · doi:10.1056/nejm197809282991304 · Cited by 1,628 · see citing works
Longino, H. E. (2002) · doi:10.1515/9780691187013 · Cited by 1,622 · see citing works
Bayesianly Justifiable and Relevant Frequency Calculations for the Applied Statistician
Rubin, D. (1984) · The Annals of Statistics · doi:10.1214/aos/1176346785 · Cited by 1,503 · see citing works
P-curve: A key to the file-drawer.
Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2014) · Journal of Experimental Psychology: General · doi:10.1037/a0033242 · Cited by 1,500 · see citing works
The Mass Production of Redundant, Misleading, and Conflicted Systematic Reviews and Meta‐analyses
IOANNIDIS, J. P. A. (2016) · The Milbank Quarterly · doi:10.1111/1468-0009.12210 · Cited by 1,483 · see citing works
Increasing Transparency Through a Multiverse Analysis
Steegen, S., Tuerlinckx, F., Gelman, A., & Vanpaemel, W. (2016) · Perspectives on Psychological Science · doi:10.1177/1745691616658637 · Cited by 1,388 · see citing works
Beyond Power Calculations: Assessing Type S (Sign) and Type M (Magnitude) Errors
Gelman, A., & Carlin, J. (2014) · Perspectives on Psychological Science · doi:10.1177/1745691614551642 · Cited by 1,343 · see citing works
Error and the Growth of Experimental Knowledge
Mayo, D. G. (1996) · doi:10.7208/chicago/9780226511993.001.0001 · Cited by 1,325 · see citing works
Self-citationThe Logic of Inductive Inference
Fisher, R. A. (1935) · Journal Of The Royal Statistical Society · doi:10.2307/2342435 · Cited by 1,292 · see citing works
The Selection of Prior Distributions by Formal Rules
Kass, R. E., & Wasserman, L. (1996) · Journal of the American Statistical Association · doi:10.1080/01621459.1996.10477003 · Cited by 1,291 · see citing works
Instruments, Randomization, and Learning about Development
Deaton, A. (2010) · Journal of Economic Literature · doi:10.1257/jel.48.2.424 · Cited by 1,289 · see citing works
Kruschke, J. K., & Liddell, T. M. (2018) · Psychonomic Bulletin & Review · doi:10.3758/s13423-016-1221-4 · Cited by 1,288 · see citing works
Model Selection and Accounting for Model Uncertainty in Graphical Models Using Occam's Window
Madigan, D., & Raftery, A. E. (1994) · Journal of the American Statistical Association · doi:10.1080/01621459.1994.10476894 · Cited by 1,205 · see citing works
How to Make Cognitive Illusions Disappear: Beyond “Heuristics and Biases”
Gigerenzer, G. (1991) · European Review of Social Psychology · doi:10.1080/14792779143000033 · Cited by 1,153 · see citing works
Outline of a Theory of Statistical Estimation Based on the Classical Theory of Probability
Neyman, J. (1937) · Philosophical Transactions of the Royal Society of London. Series A, Mathematical and Physical Sciences · doi:10.1098/rsta.1937.0005 · Cited by 1,087 · see citing works
Neyman, J. (1934) · Journal of the Royal Statistical Society · doi:10.2307/2342192 · Cited by 1,086 · see citing works
Dyson, F. W., Eddington, A. S., & Davidson, C. (1920) · Philosophical Transactions of the Royal Society of London Series A Containing Papers of a Mathematical or Physical Character · doi:10.1098/rsta.1920.0009 · Cited by 980 · see citing works
Wagenmakers, E.-J., Wetzels, R., Borsboom, D., & van der Maas, H. L. J. (2011) · Journal of Personality and Social Psychology · doi:10.1037/a0022790 · Cited by 977 · see citing works
The Statistical Crisis in Science
Gelman, A., & Loken, E. (2014) · American Scientist · doi:10.1511/2014.111.460 · Cited by 963 · see citing works
The test of significance in psychological research.
Bakan, D. (1966) · Psychological Bulletin · doi:10.1037/h0020412 · Cited by 957 · see citing works
Freedman, D. A. (2009) · Cambridge University Press eBooks · doi:10.1017/cbo9780511815867 · Cited by 947 · see citing works
Lindley, D. V. (1957) · Biometrika · doi:10.1093/biomet/44.1-2.187 · Cited by 946 · see citing works
Bogen, J., & Woodward, J. (1988) · The Philosophical Review · doi:10.2307/2185445 · Cited by 943 · see citing works
WHY SUMMARIES OF RESEARCH ON PSYCHOLOGICAL THEORIES ARE OFTEN UNINTERPRETABLE
Meehl, P. E. (1990) · Psychological Reports · doi:10.2466/pr0.1990.66.1.195 · Cited by 901 · see citing works
Bem, D. J. (2011) · Journal of Personality and Social Psychology · doi:10.1037/a0021524 · Cited by 893 · see citing works
Gigerenzer, G. (2004) · The Journal of Socio-Economics · doi:10.1016/j.socec.2004.09.033 · Cited by 890 · see citing works
Calibration of ρ Values for Testing Precise Null Hypotheses
Sellke, T., Bayarri, M. J., & Berger, J. O. (2001) · The American Statistician · doi:10.1198/000313001300339950 · Cited by 856 · see citing works
Deep Learning: A Critical Appraisal
Marcus, G. (2018) · arXiv · doi:10.48550/arxiv.1801.00631 · Cited by 853 · see citing works
Philosophy and the practice of Bayesian statistics
Gelman, A., & Shalizi, C. R. (2013) · British Journal of Mathematical and Statistical Psychology · doi:10.1111/j.2044-8317.2011.02037.x · Cited by 825 · see citing works
Revised standards for statistical evidence
Johnson, V. E. (2013) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1313476110 · Cited by 808 · see citing works
An investigation of the false discovery rate and the misinterpretation of <i>p</i> -values
Colquhoun, D. (2014) · Royal Society Open Science · doi:10.1098/rsos.140216 · Cited by 792 · see citing works
The case for objective Bayesian analysis
Berger, J. (2006) · Bayesian Analysis · doi:10.1214/06-ba115 · Cited by 778 · see citing works
Gigerenzer, G., Swijtink, Z. G., Porter, T. M., Daston, L., Beatty, J., & Krüger, L. (1989) · Cambridge University Press eBooks · doi:10.1017/cbo9780511720482 · Cited by 770 · see citing works
Bayesian approaches to randomized trials
Spiegelhalter, D. J., Freedman, L. S., & Parmar, M. K. B. (1994) · Journal of the Royal Statistical Society. Series A (Statistics in Society) · doi:10.2307/2983527 · Cited by 720 · see citing works
Logic of Discovery or Psychology of Research?
Kühn, T. (1970) · Cambridge University Press eBooks · doi:10.1017/cbo9781139171434.003 · Cited by 677 · see citing works
Evidence-Based Policy: A Practical Guide to Doing It Better
Cartwright, N., & Hardie, J. (2012) · doi:10.1093/acprof:osobl/9780199841608.001.0001 · Cited by 672 · see citing works
Principles of Statistical Inference
Cox, D. R. (2006) · doi:10.1017/cbo9780511813559 · Cited by 670 · see citing works
How to Tell When Simpler, More Unified, or LessAd HocTheories will Provide More Accurate Predictions
FORSTER, M., & Sober, E. R. (1994) · The British Journal for the Philosophy of Science · doi:10.1093/bjps/45.1.1 · Cited by 661 · see citing works
Bioequivalence trials, intersection-union tests and equivalence confidence sets
Berger, R. L., & Hsu, J. C. (1996) · Statistical Science · doi:10.1214/ss/1032280304 · Cited by 621 · see citing works
Falsification and the methodology of scientific research programmes
Lakatos, I. (1978) · Cambridge University Press eBooks · doi:10.1017/cbo9780511621123.003 · Cited by 619 · see citing works
Berger, J. O., & Wolpert, R. L. (1988) · doi:10.1214/lnms/1215466210 · Cited by 616 · see citing works
Logic of Statistical Inference
Hacking, I., & Romeijn, J. (2016) · Cambridge University Press eBooks · doi:10.1017/cbo9781316534960 · Cited by 613 · see citing works
Statistical Methods and Scientific Induction
Fisher, R. A. (1955) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.2517-6161.1955.tb00180.x · Cited by 611 · see citing works
Consistency in Statistical Inference and Decision
Smith, C. A. B. (1961) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.2517-6161.1961.tb00388.x · Cited by 560 · see citing works
I.—STUDIES IN THE LOGIC OF CONFIRMATION (I.)
Hempel, C. G. (1945) · Mind · doi:10.1093/mind/liv.213.1 · Cited by 560 · see citing works
Testing Statistical Hypotheses of Equivalence and Noninferiority
Wellek, S. (2010) · doi:10.1201/ebk1439808184 · Cited by 548 · see citing works
Ockham's Razor and Bayesian Analysis
Jefferys, W. H., & Berger, J. O. (1992) · American Scientist · Cited by 541 · see citing works
Some Problems Connected with Statistical Inference
Cox, D. R. (1958) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177706618 · Cited by 536 · see citing works
The interplay of Bayesian and frequentist analysis
Bayarri, M. J., & Berger, J. O. (2004) · Statistical Science · doi:10.1214/088342304000000116 · Cited by 525 · see citing works
Schnall, S., Benton, J., & Harvey, S. (2008) · Psychological Science · doi:10.1111/j.1467-9280.2008.02227.x · Cited by 514 · see citing works
Lambert, P. C., Sutton, A. J., Burton, P. R., Abrams, K. R., & Jones, D. R. (2005) · Statistics in Medicine · doi:10.1002/sim.2112 · Cited by 511 · see citing works
Bayesian Methods for Data Analysis
Carlin, B., & Louis, T. (2008) · Chapman & Hall/CRC eBooks · doi:10.1201/b14884 · Cited by 505 · see citing works
Statistical Foundations of Econometric Modelling
Spanos, A., & Hendry, D. F. (1986) · Cambridge University Press eBooks · doi:10.1017/cbo9780511599293 · Cited by 504 · see citing works
Testing for baseline balance in clinical trials
Senn, S. (1994) · Statistics in Medicine · doi:10.1002/sim.4780131703 · Cited by 503 · see citing works
Risks and Decisions for Conservation and Environmental Management
Burgman, M. A. (2005) · Cambridge University Press eBooks · doi:10.1017/cbo9780511614279 · Cited by 498 · see citing works
The Aim and Structure of Physical Theory
Duhem, P., & Scott, W. T. (1954) · American Journal of Physics · doi:10.1119/1.1933818 · Cited by 495 · see citing works
Two new properties of mathematical likelihood
Fisher, R. A. (1934) · Proceedings of the Royal Society of London Series A Containing Papers of a Mathematical and Physical Character · doi:10.1098/rspa.1934.0050 · Cited by 487 · see citing works
Lakens, D., Adolfi, F., Albers, C. J., Anvari, F., Apps, M. A. J., Argamon, S., Baguley, T. S., Becker, R. B., Benning, S. D., Bradford, D. E., et al. (2018) · Nature Human Behaviour · doi:10.1038/s41562-018-0311-x · Cited by 472 · see citing works
Self-citationGenetic signatures of exceptional longevity in humans
Sebastiani, P., Solovieff, N., DeWan, A. T., Walsh, K. M., Puca, A., Hartley, S. W., Melista, E., Andersen, S., Dworkis, D. A., Wilk, J. B., et al. (2012) · PLoS ONE · doi:10.1371/journal.pone.0029848 · Cited by 464 · see citing works
Fisher, R. A. (1930) · Mathematical Proceedings of the Cambridge Philosophical Society · doi:10.1017/s0305004100016297 · Cited by 454 · see citing works
Statistical Issues in Drug Development
Senn, S. (2007) · doi:10.1002/9780470723586 · Cited by 448 · see citing works
Lindley, D. V. (2000) · Journal of the Royal Statistical Society Series D (The Statistician) · doi:10.1111/1467-9884.00238 · Cited by 437 · see citing works
Reconciling Bayesian and Frequentist Evidence in the One-Sided Testing Problem
Casella, G., & Berger, R. L. (1987) · Journal of the American Statistical Association · doi:10.1080/01621459.1987.10478396 · Cited by 417 · see citing works
The Fisher, Neyman-Pearson Theories of Testing Hypotheses: One Theory or Two?
Lehmann, E. L. (1993) · Journal of the American Statistical Association · doi:10.1080/01621459.1993.10476404 · Cited by 384 · see citing works
Why Isn't Everyone a Bayesian?
Efron, B. (1986) · The American Statistician · doi:10.1080/00031305.1986.10475342 · Cited by 379 · see citing works
Intuitive statistics by 8-month-old infants
Xu, F., & Garcia, V. (2008) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0704450105 · Cited by 353 · see citing works
Could Fisher, Jeffreys and Neyman Have Agreed on Testing?
Berger, J. O. (2003) · Statistical Science · doi:10.1214/ss/1056397485 · Cited by 348 · see citing works
Statistical significance tests.
Cox, D. (1982) · British Journal of Clinical Pharmacology · doi:10.1111/j.1365-2125.1982.tb01987.x · Cited by 335 · see citing works
Probability Theory and Statistical Inference
Spanos, A. (1984) · Cambridge University Press eBooks · doi:10.1017/cbo9780511754081 · Cited by 330 · see citing works
Suppes, P. (1969) · Springer eBooks · doi:10.1007/978-94-017-3173-7_2 · Cited by 330 · see citing works
Achinstein, P. (2001) · doi:10.1093/0195143892.001.0001 · Cited by 322 · see citing works
Confidence Distribution, the Frequentist Distribution Estimator of a Parameter: A Review
Xie, M., & Singh, K. (2013) · International Statistical Review · doi:10.1111/insr.12000 · Cited by 319 · see citing works
In Defence of Objective Bayesianism
Williamson, J. (2010) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780199228003.001.0001 · Cited by 317 · see citing works
Inference, Method and Decision
Rosenkrantz, R. D. (1977) · Springer eBooks · doi:10.1007/978-94-010-1237-9 · Cited by 317 · see citing works
Baggerly, K., & Coombes, K. R. (2009) · The Annals of Applied Statistics · doi:10.1214/09-aoas291 · Cited by 311 · see citing works
Goodman, S. N. (1993) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a116700 · Cited by 310 · see citing works
A comment on replication,<i>P</i>‐values and evidence
Goodman, S. N. (1992) · Statistics in Medicine · doi:10.1002/sim.4780110705 · Cited by 309 · see citing works
What Evidence in Evidence-Based Medicine?
WORRALL, J. L. (2002) · Philosophy of Science · doi:10.1086/341855 · Cited by 307 · see citing works
Prediction Versus Accommodation and the Risk of Overfitting
Hitchcock, C., & Sober, E. R. (2004) · The British Journal for the Philosophy of Science · doi:10.1093/bjps/55.1.1 · Cited by 301 · see citing works
Marginalization Paradoxes in Bayesian and Structural Inference
Dawid, A. P., Stone, M., & Zidek, J. V. (1973) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.2517-6161.1973.tb00952.x · Cited by 296 · see citing works
On the Development of Reference Priors*
Berger, J. O., & Bernardo, J. M. (1992) · doi:10.1093/oso/9780198522669.003.0003 · Cited by 290 · see citing works
Confusion Over Measures of Evidence (p's) Versus Errors (α's) in Classical Statistical Testing
Hubbard, R., & Bayarri, M. J. (2003) · The American Statistician · doi:10.1198/0003130031856 · Cited by 288 · see citing works
Logical versus Historical Theories of Confirmation
Musgrave, A. E. (1974) · The British Journal for the Philosophy of Science · doi:10.1093/bjps/25.1.1 · Cited by 280 · see citing works
Final Collapse of the Neyman-Pearson Decision Theoretic Framework and Rise of the neoFisherian
Hurlbert, S. H., & Lombardi, C. M. (2009) · Annales Zoologici Fennici · doi:10.5735/086.046.0501 · Cited by 277 · see citing works
Seven myths of randomisation in clinical trials
Senn, S. J. (2012) · Statistics in Medicine · doi:10.1002/sim.5713 · Cited by 277 · see citing works
The Role of Exchangeability in Inference
Lindley, D. V., & Novick, M. R. (1981) · The Annals of Statistics · doi:10.1214/aos/1176345331 · Cited by 276 · see citing works
ON GENERALIZED FIDUCIAL INFERENCE
Hannig, J. (2009) · Statistica Sinica · Cited by 276 · see citing works
Severe Testing as a Basic Concept in a Neyman–Pearson Philosophy of Induction
Mayo, D. G., & Spanos, A. (2006) · The British Journal for the Philosophy of Science · doi:10.1093/bjps/axl003 · Cited by 271 · see citing works
Self-citationStatistical Methods, Experimental Design, and Scientific Inference
Fisher, R. A., & Yates, F. (1990) · doi:10.1093/oso/9780198522294.001.0001 · Cited by 259 · see citing works
ON THE USE AND INTERPRETATION OF CERTAIN TEST CRITERIA FOR PURPOSES OF STATISTICAL INFERENCE PART I
Neyman, J., & PEARSON, E. S. (1928) · Biometrika · doi:10.1093/biomet/20a.1-2.175 · Cited by 259 · see citing works
Beyond Subjective and Objective in Statistics
Gelman, A., & Hennig, C. M. (2017) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/rssa.12276 · Cited by 257 · see citing works
Little, R. J. A. (2006) · The American Statistician · doi:10.1198/000313006x117837 · Cited by 240 · see citing works
Relativity and Eclipses: The British Eclipse Expeditions of 1919 and Their Predecessors
Earman, J., & Glymour, C. (1980) · Historical Studies in the Physical Sciences · doi:10.2307/27757471 · Cited by 234 · see citing works
Frequentist probability and frequentist statistics
Neyman, J. (1977) · Synthese · doi:10.1007/bf00485695 · Cited by 230 · see citing works
Flawed science: The fraudulent research practices of social psychologist Diederik Stapel
Levelt, W. J. M., Drenth, J. P. H., & Noort, E. (2012) · Cited by 217 · see citing works
Gelman, A. (2012) · Epidemiology · doi:10.1097/ede.0b013e31827886f7 · Cited by 216 · see citing works
Just Post It: The Lesson From Two Cases of Fabricated Data Detected by Statistics Alone
Simonsohn, U. (2013) · Psychological Science · doi:10.1177/0956797613480366 · Cited by 216 · see citing works
Gigerenzer, G., & Marewski, J. N. (2014) · Journal of Management · doi:10.1177/0149206314547522 · Cited by 215 · see citing works
Statistical Rules of Thumb, Second Edition
van Belle, G. (2008) · Wiley series in probability and statistics · doi:10.1002/9780470377963 · Cited by 209 · see citing works
Statistical Problems in ESP Research
Diaconis, P. W. (1978) · Science · doi:10.1126/science.663642 · Cited by 198 · see citing works
"Inductive Behavior" as a Basic Concept of Philosophy of Science
Neyman, J. (1957) · Revue de l Institut International de Statistique / Review of the International Statistical Institute · doi:10.2307/1401671 · Cited by 198 · see citing works
Is Psychometrics Pathological Science?
Michell, J. (2008) · Measurement Interdisciplinary Research and Perspectives · doi:10.1080/15366360802035489 · Cited by 195 · see citing works
THE EFFICIENCY OF STATISTICAL TOOLS AND A CRITERION FOR THE REJECTION OF OUTLYING OBSERVATIONS
Pearson, E. S., & Sekar, C. C. (1936) · Biometrika · doi:10.1093/biomet/28.3-4.308 · Cited by 184 · see citing works
The Interpretation of Levels of Significance by Psychological Researchers
Rosenthal, R., & Gaito, J. M. (1963) · The Journal of Psychology · doi:10.1080/00223980.1963.9916596 · Cited by 183 · see citing works
Venetian Sea Levels, British Bread Prices, and the Principle of the Common Cause
Sober, E. R. (2001) · The British Journal for the Philosophy of Science · doi:10.1093/bjps/52.2.331 · Cited by 182 · see citing works
Properties of Diagnostic Data Distributions
Dawid, A. P. (1976) · Biometrics · doi:10.2307/2529753 · Cited by 181 · see citing works
R. A. Fisher and Fiducial Argument
Zabell, S. L. (1992) · Statistical Science · doi:10.1214/ss/1177011233 · Cited by 179 · see citing works
Neyman, J., & Pearson, E. S. (1967) · University of California Press eBooks · doi:10.1525/9780520339897 · Cited by 179 · see citing works
Bayesian Interpretation of Standard Inference Statements
Pratt, J. W. (1965) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.2517-6161.1965.tb01486.x · Cited by 176 · see citing works
The Counternull Value of an Effect Size: A New Statistic
Rosenthal, R., & Rubin, D. B. (1994) · Psychological Science · doi:10.1111/j.1467-9280.1994.tb00281.x · Cited by 170 · see citing works
Inferential Models: A Framework for Prior-Free Posterior Probabilistic Inference
Martin, R., & Liu, C. (2013) · Journal of the American Statistical Association · doi:10.1080/01621459.2012.747960 · Cited by 167 · see citing works
Two principles of Bayesian epistemology
Talbott, W. J. (1991) · Philosophical Studies · doi:10.1007/bf00419049 · Cited by 166 · see citing works
Incorporating Bayesian Ideas into Health-Care Evaluation
Spiegelhalter, D. J. (2004) · Statistical Science · doi:10.1214/088342304000000080 · Cited by 160 · see citing works
Novel Evidence and Severe Tests
Mayo, D. G. (1991) · Philosophy of Science · doi:10.1086/289639 · Cited by 154 · see citing works
Self-citationFisher, Neyman, and the Creation of Classical Statistics
Lehmann, E. L. (2011) · Springer eBooks · doi:10.1007/978-1-4419-9500-1 · Cited by 152 · see citing works
Coombes, K. R., Wang, J., & Baggerly, K. (2007) · Nature Medicine · doi:10.1038/nm1107-1276b · Cited by 148 · see citing works
Some Statistical Problems in Research Design
Kish, L. (1959) · American Sociological Review · doi:10.2307/2089381 · Cited by 142 · see citing works
Data, Phenomena, and Reliability
Woodward, J. (2000) · Philosophy of Science · doi:10.1086/392817 · Cited by 139 · see citing works
Pigliucci, M. (2010) · University of Chicago Press eBooks · doi:10.7208/chicago/9780226667874.001.0001 · Cited by 139 · see citing works
Testing relativity from the 1919 eclipse—a question of bias
Kennefick, D. (2009) · Physics Today · doi:10.1063/1.3099578 · Cited by 138 · see citing works
FIDUCIAL ARGUMENT AND THE THEORY OF CONFIDENCE INTERVALS
Neyman, J. (1941) · Biometrika · doi:10.1093/biomet/32.2.128 · Cited by 138 · see citing works
The Seven Pillars of Statistical Wisdom
Stigler, S. M. (2016) · Harvard University Press eBooks · doi:10.2307/jj.41545403 · Cited by 137 · see citing works
THE COMPARISON OF SAMPLES WITH POSSIBLY UNEQUAL VARIANCES
Fisher, R. A. (1939) · Annals of Eugenics · doi:10.1111/j.1469-1809.1939.tb02205.x · Cited by 137 · see citing works
Model Specification: The Views of Fisher and Neyman, and Later Developments
Lehmann, E. L. (1990) · Statistical Science · doi:10.1214/ss/1177012164 · Cited by 136 · see citing works
The information available in small samples
Bartlett, M. S. (1936) · Mathematical Proceedings of the Cambridge Philosophical Society · doi:10.1017/s0305004100019290 · Cited by 129 · see citing works
Is Bayes Posterior just Quick and Dirty Confidence?
Fraser, D. A. S. (2011) · Statistical Science · doi:10.1214/11-sts352 · Cited by 129 · see citing works
Rejection odds and rejection ratios: A proposal for statistical practice in testing hypotheses
Bayarri, M. J., Benjamin, D. J., Berger, J. O., & Sellke, T. M. (2016) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2015.12.007 · Cited by 126 · see citing works
P values are only an index to evidence: 20th‐ vs. 21st‐century statistical science
Burnham, K. P., & Anderson, D. R. (2014) · Ecology · doi:10.1890/13-1066.1 · Cited by 124 · see citing works
THE ESTIMATION OF MANY PARAMETERS
Lindley, D. V. (1970) · ETS Research Bulletin Series · doi:10.1002/j.2333-8504.1970.tb00411.x · Cited by 124 · see citing works
Frequentist statistics as a theory of inductive inference
Mayo, D. G., & Cox, D. (2006) · Institute of Mathematical Statistics eBooks · doi:10.1214/074921706000000400 · Cited by 123 · see citing works
Self-citationExperiments as exhibits and experiments as tests
Sugden, R. (2005) · Journal of Economic Methodology · doi:10.1080/13501780500086248 · Cited by 121 · see citing works
P-Values: Interpretation and Methodology
Gibbons, J. D., & Pratt, J. W. (1975) · The American Statistician · doi:10.2307/2683674 · Cited by 121 · see citing works
Brown, E. N., & Kass, R. E. (2009) · The American Statistician · doi:10.1198/tast.2009.0019 · Cited by 120 · see citing works
Methodology in Practice: Statistical Misspecification Testing
Mayo, D. G., & Spanos, A. (2004) · Philosophy of Science · doi:10.1086/425064 · Cited by 119 · see citing works
Self-citationGreenland, S., & Poole, C. (2012) · Epidemiology · doi:10.1097/ede.0b013e3182785741 · Cited by 115 · see citing works
Statistical Inference: The Big Picture
Kass, R. E. (2011) · Statistical Science · doi:10.1214/10-sts337 · Cited by 114 · see citing works
Why I am not an objective Bayesian; some reflections prompted by Rosenkrantz
Seidenfeld, T. (1979) · Theory and Decision · doi:10.1007/bf00139451 · Cited by 109 · see citing works
Confidence, Likelihood, Probability: Statistical Inference with Confidence Distributions
Schweder, T., & Hjort, N. L. (2016) · Cambridge University Press eBooks · doi:10.1017/cbo9781139046671 · Cited by 109 · see citing works
Confidence distribution (CD) -- distribution estimator of a parameter
Singh, K., Xie, M., & Strawderman, W. E. (2007) · Institute of Mathematical Statistics eBooks · doi:10.1214/074921707000000102 · Cited by 108 · see citing works
Fisher, R. A. (1936) · Proceedings of the American Academy of Arts and Sciences · doi:10.2307/20023225 · Cited by 108 · see citing works
Non-informative priors do not exist A dialogue with José M. Bernardo
Irony, T., & Singpurwalla, N. D. (1997) · Journal of Statistical Planning and Inference · doi:10.1016/s0378-3758(97)00074-8 · Cited by 106 · see citing works
The Path Analysis Controversy: A new statistical approach to strong appraisal of verisimilitude.
Meehl, P. E., & Waller, N. G. (2002) · Psychological Methods · doi:10.1037/1082-989x.7.3.283 · Cited by 106 · see citing works
Some Thoughts on Statistical Inference
Pearson, E. S. (1962) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177704566 · Cited by 105 · see citing works
Must psychologists change the way they analyze their data?
Bem, D. J., Utts, J., & Johnson, W. O. (2011) · Journal of Personality and Social Psychology · doi:10.1037/a0024777 · Cited by 105 · see citing works
Gender differences in implicit self-esteem following a romantic partner’s success or failure.
Ratliff, K. A., & Oishi, S. (2013) · Journal of Personality and Social Psychology · doi:10.1037/a0033769 · Cited by 105 · see citing works
Birnbaum, A. (1977) · Synthese · doi:10.1007/bf00485690 · Cited by 104 · see citing works
Gigerenzer, G. (2002) · DIAL (Catholic University of Leuven) · doi:10.1093/acprof:oso/9780195153729.001.0001 · Cited by 104 · see citing works
Note on a Conditional Property of Student's $t^1$
Buehler, R. J., & Feddersen, A. P. (1963) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177704034 · Cited by 103 · see citing works
Pigliucci, M. (2013) · University of Chicago Press eBooks · doi:10.7208/chicago/9780226051826.003.0002 · Cited by 103 · see citing works
Fraser, D. A. S. (1961) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177704962 · Cited by 101 · see citing works
Senn, S. J. (2001) · Journal of Epidemiology and Biostatistics · doi:10.1080/135952201753172953 · Cited by 98 · see citing works
A Critique of Tests of Significance in Survey Research
Selvin, H. C. (2017) · Routledge eBooks · doi:10.4324/9781315134918-14 · Cited by 98 · see citing works
Gurney, J. G., Mueller, B. A., Davis, S., Schwartz, S. M., Stevens, R. G., & Kopecky, K. J. (1996) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a008718 · Cited by 97 · see citing works
Nonsignificance Plus High Power Does Not Imply Support for the Null Over the Alternative
Greenland, S. (2012) · Annals of Epidemiology · doi:10.1016/j.annepidem.2012.02.007 · Cited by 97 · see citing works
Two Breakthroughs in the Theory of Statistical Decision Making
Neyman, J. (1962) · Revue de l Institut International de Statistique / Review of the International Statistical Institute · doi:10.2307/1402069 · Cited by 96 · see citing works
A new etiquette for replication.
Kahneman, D. J. (2014) · Cited by 96 · see citing works
Make journals report clinical trials properly
Goldacre, B. M. (2016) · Nature · doi:10.1038/530007a · Cited by 94 · see citing works
Kadane, J. B. (2011) · Chapman & Hall/CRC eBooks · doi:10.1201/b11322 · Cited by 93 · see citing works
Statistical Concepts in Their Relation to Reality
Pearson, E. S. (1955) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.2517-6161.1955.tb00194.x · Cited by 92 · see citing works
Note on an Article by Sir Ronald Fisher
Neyman, J. (1956) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.2517-6161.1956.tb00236.x · Cited by 90 · see citing works
The Bayesian Influence, or How to Sweep Subjectivism under the Carpet
Good, I. J. (1976) · Springer eBooks · doi:10.1007/978-94-010-1436-6_5 · Cited by 87 · see citing works
Flagged first, then by citations
13 of the 325 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Moving to a World Beyond “ <i>p</i> < 0.05”
Wasserstein, R. L., Schirm, A. L., & Lazar, N. A. (2019) · The American Statistician · doi:10.1080/00031305.2019.1583913 · Cited by 2,976 · see citing works
Replicability, Robustness, and Reproducibility in Psychological Science
Nosek, B. A., Hardwicke, T. E., Moshontz, H., Allard, A., Corker, K. S., Dreber, A., Fidler, F., Hilgard, J., Kline Struhl, M., Nuijten, M. B., et al. (2022) · Annual Review of Psychology · doi:10.1146/annurev-psych-020821-114157 · Cited by 770 · see citing works
Yarkoni, T. (2022) · Behavioral and Brain Sciences · doi:10.1017/s0140525x20001685 · Cited by 701 · see citing works
Five ways to ensure that models serve society: a manifesto
Saltelli, A., Bammer, G., Bruno, I., Charters, E. M., Fiore, M. D., Didier, E., Espeland, W. N., Kay, J., Piano, S. L., Mayo, D. G., et al. (2020) · Nature · doi:10.1038/d41586-020-01812-9 · Cited by 492 · see citing works
Self-citationLakens, D., Adolfi, F., Albers, C. J., Anvari, F., Apps, M. A. J., Argamon, S., Baguley, T. S., Becker, R. B., Benning, S. D., Bradford, D. E., et al. (2018) · Nature Human Behaviour · doi:10.1038/s41562-018-0311-x · Cited by 472 · see citing works
Self-citationWhy Hypothesis Testers Should Spend Less Time Testing Hypotheses
Scheel, A. M., Tiokhin, L., Isager, P. M., & Lakens, D. (2021) · Perspectives on Psychological Science · doi:10.1177/1745691620966795 · Cited by 382 · see citing works
Model Evaluation: An Adequacy-for-Purpose View
Parker, W. S. (2020) · Philosophy of Science · doi:10.1086/708691 · Cited by 310 · see citing works
Power to Detect What? Considerations for Planning and Evaluating Sample Size
Giner‐Sorolla, R., Montoya, A. K., Reifman, A. S., Carpenter, T., Lewis, N. A., Aberson, C. L., Bostyn, D. H., Conrique, B. G., Ng, B., Schoemann, A. M., et al. (2024) · Personality and Social Psychology Review · doi:10.1177/10888683241228328 · Cited by 292 · see citing works
The Generalizability CrisisPreprint
Yarkoni, T. (2019) · doi:10.31234/osf.io/jqw35 · Cited by 275 · see citing works
Rafi, Z., & Greenland, S. (2020) · BMC Medical Research Methodology · doi:10.1186/s12874-020-01105-9 · Cited by 243 · see citing works
Deep problems with neural network models of human vision
Bowers, J. S., Malhotra, G., Dujmović, M., Llera Montero, M., Tsvetkov, C., Biscione, V., Puebla, G., Adolfi, F., Hummel, J. E., Heaton, R. F., et al. (2023) · Behavioral and Brain Sciences · doi:10.1017/s0140525x22002813 · Cited by 204 · see citing works
Reducing bias, increasing transparency and calibrating confidence with preregistration
Hardwicke, T. E., & Wagenmakers, E. (2023) · Nature Human Behaviour · doi:10.1038/s41562-022-01497-2 · Cited by 190 · see citing works
The Practical Alternative to the p Value Is the Correctly Used p Value
Lakens, D. (2021) · Perspectives on Psychological Science · doi:10.1177/1745691620958012 · Cited by 181 · see citing works
Philosophy of science and the replicability crisis
Romero, F. (2019) · Philosophy Compass · doi:10.1111/phc3.12633 · Cited by 152 · see citing works
How Do I Know What My Theory Predicts?
Dienes, Z. P. (2019) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245919876960 · Cited by 149 · see citing works
Data models, representation and adequacy-for-purpose
Bokulich, A., & Parker, W. S. (2021) · European Journal for Philosophy of Science · doi:10.1007/s13194-020-00345-2 · Cited by 125 · see citing works
Testing by Betting: A Strategy for Statistical and Scientific Communication
Shafer, G. R. (2021) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/rssa.12647 · Cited by 117 · see citing works
Calibrating the Scientific Ecosystem Through Meta-Research
Hardwicke, T. E., Serghiou, S., Janiaud, P., Danchev, V., Crüwell, S., Goodman, S. N., & Ioannidis, J. P. A. (2020) · Annual Review of Statistics and Its Application · doi:10.1146/annurev-statistics-031219-041104 · Cited by 98 · see citing works
When and How to Deviate From a Preregistration
Lakens, D. (2024) · Collabra Psychology · doi:10.1525/collabra.117094 · Cited by 98 · see citing works
Hyperparameter Tuning for Machine and Deep Learning with R
Bartz, E., Bartz–Beielstein, T., Zaefferer, M., & Mersmann, O. (2022) · Springer eBooks · doi:10.1007/978-981-19-5170-1 · Cited by 92 · see citing works
Exploratory hypothesis tests can be more compelling than confirmatory hypothesis tests
Rubin, M., & Donkin, C. (2022) · Philosophical Psychology · doi:10.1080/09515089.2022.2113771 · Cited by 87 · see citing works
Loenneke, J. P., Buckner, S. L., Dankel, S. J., & Abe, T. (2019) · Sports Medicine · doi:10.1007/s40279-019-01106-9 · Cited by 82 · see citing works
Conceptual challenges for interpretable machine learning
Watson, D. (2022) · Synthese · doi:10.1007/s11229-022-03485-5 · Cited by 79 · see citing works
Why and how we should join the shift from significance testing to estimation
Berner, D., & Amrhein, V. (2022) · Journal of Evolutionary Biology · doi:10.1111/jeb.14009 · Cited by 78 · see citing works
van den Akker, O. R., van Assen, M. A. L. M., Bakker, M., Elsherif, M. M., Wong, T. K., & Wicherts, J. M. (2023) · Behavior Research Methods · doi:10.3758/s13428-023-02277-0 · Cited by 78 · see citing works
The benefits of preregistration and Registered Reports
Lakens, D., Mesquida, C., Rasti, S., & Ditroilo, M. (2024) · Evidence-Based Toxicology · doi:10.1080/2833373x.2024.2376046 · Cited by 76 · see citing works
What is (perception of) effort? Objective and subjective effort during attempted task performancePreprint
Steele, J. (2020) · doi:10.31234/osf.io/kbyhm · Cited by 76 · see citing works
The roles, challenges, and merits of the p value
Chén, O. Y., Bodelet, J., Saraiva, R. G., Phan, H., Di, J., Nagels, G., Schwantje, T., Cao, H., Gou, J., Reinen, J., et al. (2023) · Patterns · doi:10.1016/j.patter.2023.100878 · Cited by 74 · see citing works
Errors in Statistical Inference Under Model Misspecification: Evidence, Hypothesis Testing, and AIC
Dennis, B., Ponciano, J. M., Taper, M. L., & Lele, S. R. (2019) · Frontiers in Ecology and Evolution · doi:10.3389/fevo.2019.00372 · Cited by 74 · see citing works
Orben, A. C., & Lakens, D. (2020) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245920917961 · Cited by 68 · see citing works
Vrij, A., Palena, N., Leal, S., & Caso, L. (2021) · The European Journal of Psychology Applied to Legal Context · doi:10.5093/ejpalc2021a7 · Cited by 62 · see citing works
Examining the conceptual and empirical distinctiveness of Agreeableness and “dark” personality items
Vize, C. E., Miller, J. D., & Lynam, D. R. (2020) · Journal of Personality · doi:10.1111/jopy.12601 · Cited by 62 · see citing works
Does preregistration improve the credibility of research findings?
Rubin, M. (2020) · The Quantitative Methods for Psychology · doi:10.20982/tqmp.16.4.p376 · Cited by 61 · see citing works
“A test is severe if it has a low probability of confirming a hypothesis when that hypothesis is false (Mayo, 1996, 2018).”
BackgroundMethodThe replication crisis, scientific revolutions, and linguistics
Sönning, L., & Werner, V. (2021) · Linguistics · doi:10.1515/ling-2019-0045 · Cited by 57 · see citing works
The epistemological foundations of data science: a critical analysis
Desai, J., Watson, D., Wang, V., Taddeo, M., & Floridi, L. (2022) · Synthese · doi:10.1007/s11229-022-03933-2 · Cited by 52 · see citing works
Selective Inference: The Silent Killer of Replicability
Benjamini, Y. (2020) · Harvard Data Science Review · doi:10.1162/99608f92.fc62b261 · Cited by 49 · see citing works
Explaining AI through mechanistic interpretability
Kästner, L., & Crook, B. (2024) · European Journal for Philosophy of Science · doi:10.1007/s13194-024-00614-4 · Cited by 48 · see citing works
False confidence, non-additive beliefs, and valid statistical inference
Martin, R. (2019) · International Journal of Approximate Reasoning · doi:10.1016/j.ijar.2019.06.005 · Cited by 48 · see citing works
Greenland, S. (2022) · Scandinavian Journal of Statistics · doi:10.1111/sjos.12625 · Cited by 47 · see citing works
Rubin, M. (2019) · The British Journal for the Philosophy of Science · doi:10.1093/bjps/axz050 · Cited by 47 · see citing works
Replication crisis or an opportunity to improve scientific production?
Navarro, M. D. F., Pascual‐Llobell, J., Pascual‐Soler, M., Perezgonzalez, J. D., & Berríos-Riquelme, J. (2020) · European Journal of Education · doi:10.1111/ejed.12417 · Cited by 46 · see citing works
On The Philosophy of Unsupervised Learning
Watson, D. (2022) · Philosophy & Technology · doi:10.1007/s13347-023-00635-6 · Cited by 46 · see citing works
Psychology’s Theory Crisis, and why formal modelling cannot solve it
Maatman, F. O. (2025) · Meta-Psychology · doi:10.15626/mp.2024.4224 · Cited by 46 · see citing works
Grossmann, M. (2021) · doi:10.1093/oso/9780197518977.001.0001 · Cited by 45 · see citing works
Time to Assess Bias in Machine Learning Models for Credit Decisions
Brotcke, L. (2022) · Journal of risk and financial management · doi:10.3390/jrfm15040165 · Cited by 45 · see citing works
Zane, S. N., & Pupo, J. A. (2021) · Justice Quarterly · doi:10.1080/07418825.2021.1915364 · Cited by 45 · see citing works
Mayo, D. G., & Hand, D. J. (2022) · Synthese · doi:10.1007/s11229-022-03692-0 · Cited by 44 · see citing works
Self-citationRobust Discovery of Regression Models
Castle, J. L., Doornik, J. A., & Hendry, D. F. (2021) · Econometrics and Statistics · doi:10.1016/j.ecosta.2021.05.004 · Cited by 43 · see citing works
Philosophy of Probability and Statistical Modelling
Suárez, M. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108985826 · Cited by 40 · see citing works
Reliability in Machine Learning
Grote, T., Genin, K., & Sullivan, E. (2024) · Philosophy Compass · doi:10.1111/phc3.12974 · Cited by 39 · see citing works
Improving practices and inferences in developmental cognitive neuroscience
Flournoy, J. C., Vijayakumar, N., Cheng, T. W., Cosme, D., Flannery, J., & Pfeifer, J. H. (2020) · Developmental Cognitive Neuroscience · doi:10.1016/j.dcn.2020.100807 · Cited by 39 · see citing works
Competing narratives in AI ethics: a defense of sociotechnical pragmatism
Watson, D., Mökander, J., & Floridi, L. (2024) · AI & Society · doi:10.1007/s00146-024-02128-2 · Cited by 36 · see citing works
Is Ockham’s razor losing its edge? New perspectives on the principle of model parsimony
Dubova, M., Chandramouli, S., Gigerenzer, G., Grünwald, P., Holmes, W. R., Lombrozo, T., Marelli, M., Musslick, S., Nicenboim, B., Ross, L. N., et al. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2401230121 · Cited by 36 · see citing works
What can mathematical modelling contribute to a sociology of quantification?
Saltelli, A., & Puy, A. (2023) · Humanities and Social Sciences Communications · doi:10.1057/s41599-023-01704-z · Cited by 35 · see citing works
Basic statistical considerations for physiology: The journal Temperature toolbox
Caldwell, A. R., & Cheuvront, S. N. (2019) · Temperature · doi:10.1080/23328940.2019.1624131 · Cited by 35 · see citing works
The epistemic and pragmatic function of dichotomous claims based on statistical hypothesis tests
Tunç, D. U., Tunç, M. N., & Lakens, D. (2023) · Theory & Psychology · doi:10.1177/09593543231160112 · Cited by 34 · see citing works
The variety of evidence thesis and its independence of degrees of independence
Landes, J. (2020) · Synthese · doi:10.1007/s11229-020-02738-5 · Cited by 33 · see citing works
Understanding Replication in a Way That Is True to Science
Haig, B. D. (2021) · Review of General Psychology · doi:10.1177/10892680211046514 · Cited by 33 · see citing works
Statistics in the Service of Science: Don’t Let the Tail Wag the Dog
Singmann, H., Kellen, D., Cox, G. E., Chandramouli, S., Davis‐Stober, C. P., Dunn, J. C., Gronau, Q. F., Kalish, M. L., McMullin, S. D., Navarro, D., et al. (2022) · Computational Brain & Behavior · doi:10.1007/s42113-022-00129-2 · Cited by 33 · see citing works
The economic inequality as normative information model (EINIM)
Sánchez‐Rodríguez, Á., Rodríguez‐Bailón, R., & Willis, G. B. (2023) · European Review of Social Psychology · doi:10.1080/10463283.2022.2160555 · Cited by 33 · see citing works
Falsificationist Treatment of Auxiliary Hypotheses in Social and Behavioral Sciences
Tunç, D. U., & Tunç, M. N. (2023) · Meta-Psychology · doi:10.15626/mp.2021.2756 · Cited by 32 · see citing works
Potentially harmful therapies: A meta-scientific review of evidential value.
Williams, A. J., Botanov, Y., Kilshaw, R. E., Wong, R. E., & Sakaluk, J. K. (2020) · Clinical Psychology Science and Practice · doi:10.1111/cpsp.12331 · Cited by 31 · see citing works
Cox, D. (2019) · Annual Review of Statistics and Its Application · doi:10.1146/annurev-statistics-031219-041051 · Cited by 30 · see citing works
Pay No Attention to the Model Behind the Curtain
Stark, P. B. (2022) · Pure and Applied Geophysics · doi:10.1007/s00024-022-03137-2 · Cited by 29 · see citing works
Luke, T. J., & Alceste, F. (2020) · Law and Human Behavior · doi:10.1037/lhb0000410 · Cited by 28 · see citing works
Thinking About Sum Scores Yet Again, Maybe the Last Time, We Don’t Know, Oh No . . .: A Comment on
Widaman, K. F., & Revelle, W. (2023) · Educational and Psychological Measurement · doi:10.1177/00131644231205310 · Cited by 27 · see citing works
Means to valuable exploration: I. The blending of confirmation and exploration and how to resolve it
Höfler, M., Scherbaum, S., Kanske, P., McDonald, B., & Miller, R. (2022) · Meta-Psychology · doi:10.15626/mp.2021.2837 · Cited by 27 · see citing works
Scientific Models and Decision Making
Winsberg, E. B., & Harvard, S. S. (2024) · Cambridge University Press eBooks · doi:10.1017/9781009029346 · Cited by 27 · see citing works
Simandan, D. (2021) · SSM - Population Health · doi:10.1016/j.ssmph.2021.100971 · Cited by 26 · see citing works
Pedersen, M. K., Díaz, C. M. C., Wang, Q. J., Alba‐Marrugo, M. A., Amidi, A., Basaiawmoit, R. V., Bergenholtz, C., Christiansen, M. H., Gajdacz, M., Hertwig, R., et al. (2023) · Cognitive Science · doi:10.1111/cogs.13308 · Cited by 26 · see citing works
Sawicki, M., & Szóstak, M. (2020) · Applied Sciences · doi:10.3390/app10196690 · Cited by 25 · see citing works
Ballou, N., & Zendle, D. I. (2021) · Computers in Human Behavior · doi:10.1016/j.chb.2021.107140 · Cited by 25 · see citing works
Spotting When Algorithms Are Wrong
Buijsman, S., & Veluwenkamp, H. (2022) · Minds and Machines · doi:10.1007/s11023-022-09591-0 · Cited by 25 · see citing works
A Manifesto for More Productive Psychological Games Research
Ballou, N. (2023) · Games Research and Practice · doi:10.1145/3582929 · Cited by 25 · see citing works
Type I Error Rates are Not Usually Inflated
Rubin, M. (2024) · Journal of Trial and Error · doi:10.36850/4d35-44bd · Cited by 25 · see citing works
Replication value as a function of citation impact and sample size
Isager, P. M., Veer, A. V. '., & Lakens, D. (2025) · Meta-Psychology · doi:10.15626/mp.2022.3300 · Cited by 25 · see citing works
Beyond Neyman–Pearson: E-values enable hypothesis testing with a data-driven alpha
Grünwald, P. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2302098121 · Cited by 23 · see citing works
The epistemic revolution of AI: reconfiguring the foundations of scientific knowledge
Cruz‐Aguilar, M. A. (2025) · AI & Society · doi:10.1007/s00146-025-02658-3 · Cited by 23 · see citing works
Assessing and Improving Robustness of Psychological Research Findings in Four Steps
Nuijten, M. B. (2022) · Springer eBooks · doi:10.1007/978-3-031-04968-2_17 · Cited by 23 · see citing works
Bayesian evaluation of behavior change interventions: a brief introduction and a practical example
Heino, M. T. J., Vuorre, M., & Hankonen, N. E. (2018) · Health Psychology and Behavioral Medicine · doi:10.1080/21642850.2018.1428102 · Cited by 23 · see citing works
Statistical Guidance to Authors at Top-Ranked Journals across Scientific Disciplines
Hardwicke, T. E., Salholz‐Hillel, M., Malički, M., Szűcs, D., Bendixen, T., & Ioannidis, J. P. A. (2022) · The American Statistician · doi:10.1080/00031305.2022.2143897 · Cited by 22 · see citing works
Trustworthiness of Statistical Inference
Hand, D. J. (2021) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/rssa.12752 · Cited by 21 · see citing works
The Challenge of Quantification: An Interdisciplinary Reading
Fiore, M. D., Kuc-Czarnecka, M. E., Piano, S. L., Puy, A., & Saltelli, A. (2022) · Minerva · doi:10.1007/s11024-022-09481-w · Cited by 21 · see citing works
Severe testing of Benford’s law
Cerqueti, R., & Lupi, C. (2023) · Test · doi:10.1007/s11749-023-00848-z · Cited by 21 · see citing works
A Bayesian perspective on severity: risky predictions and specific hypotheses
van Dongen, N. N. N., Sprenger, J., & Wagenmakers, E. (2022) · Psychonomic Bulletin & Review · doi:10.3758/s13423-022-02069-1 · Cited by 21 · see citing works
Wulff, J. N., & Taylor, L. (2023) · Strategic Organization · doi:10.1177/14761270231214429 · Cited by 20 · see citing works
Dunleavy, D. J. (2022) · European Journal for Philosophy of Science · doi:10.1007/s13194-022-00492-8 · Cited by 19 · see citing works
Radžvilas, M., Pretis, F. D., Peden, W., Tortoli, D., & Osimani, B. (2022) · Computational Economics · doi:10.1007/s10614-022-10250-w · Cited by 19 · see citing works
The Reproduction Health Behavior of High School Teenagers in Semarang, Indonesia
Handayani, O. W. K., Wiranti, I., Raharjo, B. B., & Nugroho, E. (2019) · The Open Public Health Journal · doi:10.2174/1874944501912010309 · Cited by 18 · see citing works
Heino, M. T. J., Knittle, K. P., Fried, E. I., Sund, R., Haukkala, A., Borodulin, K., Uutela, A. K., Araújo‐Soares, V., Vasankari, T. J., & Hankonen, N. E. (2019) · Health Psychology and Behavioral Medicine · doi:10.1080/21642850.2019.1646136 · Cited by 18 · see citing works
Reporting and interpreting non-significant results in animal cognition research
Farrar, B. G., Vernouillet, A., Garcia‐Pelegrin, E., Legg, E. W., Brecht, K. F., Lambert, P. J., Elsherif, M. M., Francis, S., O’Neill, L., Clayton, N. S., et al. (2023) · PeerJ · doi:10.7717/peerj.14963 · Cited by 18 · see citing works
Situational sources of rule-breaking acts
Chrysoulakis, A. (2022) · doi:10.24834/isbn.9789178772797 · Cited by 17 · see citing works
Expanding theory testing in general relativity: LIGO and parametrized theories
Patton, L. (2020) · Studies in History and Philosophy of Science Part B Studies in History and Philosophy of Modern Physics · doi:10.1016/j.shpsb.2020.01.001 · Cited by 16 · see citing works
Taper, M. L., Lele, S. R., Ponciano, J. M., Dennis, B., & Jerde, C. L. (2021) · Frontiers in Ecology and Evolution · doi:10.3389/fevo.2021.679155 · Cited by 16 · see citing works
Coenen, L. (2022) · European Journal of Communication · doi:10.1177/02673231221082244 · Cited by 16 · see citing works
Scientific self-correction: the Bayesian way
Romero, F., & Sprenger, J. (2020) · Synthese · doi:10.1007/s11229-020-02697-x · Cited by 16 · see citing works
How to Choose between Different Bayesian Posterior Indices for Hypothesis Testing in Practice
Kelter, R. (2021) · Multivariate Behavioral Research · doi:10.1080/00273171.2021.1967716 · Cited by 16 · see citing works
Improving statistical reporting in psychology
Schubert, A., Snijder-Steinhilber, M., Kang, H., & Quintana, D. S. (2025) · Communications Psychology · doi:10.1038/s44271-025-00356-w · Cited by 15 · see citing works
Circular and unified analysis in network neuroscience
Rubinov, M. (2023) · eLife · doi:10.7554/elife.79559 · Cited by 15 · see citing works
The statistics wars and intellectual conflicts of interest
Mayo, D. G. (2021) · Conservation Biology · doi:10.1111/cobi.13861 · Cited by 14 · see citing works
Self-citation
- Authors
- Mayo, D. G.
- Affiliations
- Virginia Tech (US)
- Corresponding author
- Mayo, Deborah G.
- ORCID
- Mayo, Deborah G.
- Venue
- Cambridge University Press eBooks
- Publisher
- Cambridge University Press
- Published
- 2018-09-14
- OpenAlex
- W3171064310
- Cited by
- 376
- References
- 325
- Citations by year
- 2026: 32 · 2025: 57 · 2024: 35 · 2023: 42 · 2022: 60 · 2021: 72 · 2020: 43 · 2019: 28 · 2018: 5 · 2017: 1
- FWCI
- 54.96
- Type
- book
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Cambridge University Press eBooks (publishedVersion)
- Topics
- Social Sciences › Arts and Humanities › History and Philosophy of Science › Philosophy and History of Science
- Social Sciences › Arts and Humanities › History and Philosophy of Science › Philosophy, Science, and History
- Social Sciences › Arts and Humanities › Philosophy › Literary, Cultural, Historical Analysis
- Keywords
- severe testing, statistical inference, replication crisis, degrees of belief, cherry-picking, selective reporting, inductive reasoning, falsification, pseudoscience, science reform
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
