The DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.
Surrogate Science: The Idol of a Universal Method for Scientific Inference
Open accessAbstract
The application of statistics to science is not a neutral act. Statistical tools have shaped and were also shaped by its objects. In the social sciences, statistical methods fundamentally changed research practice, making statistical inference its centerpiece. At the same time, textbook writers in the social sciences have transformed rivaling statistical systems into an apparently monolithic method that could be used mechanically. The idol of a universal method for scientific inference has been worshipped since the “inference revolution” of the 1950s. Because no such method has ever been found, surrogates have been created, most notably the quest for significant p values. This form of surrogate science fosters delusions and borderline cheating and has done much harm, creating, for one, a flood of irreproducible results. Proponents of the “Bayesian revolution” should be wary of chasing yet another chimera: an apparently universal inference procedure. A better path would be to promote both an understanding of the various devices in the “statistical toolbox” and informed judgment to select among these.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Kass, R. E., & Raftery, A. E. (1995) · Journal of the American Statistical Association · doi:10.1080/01621459.1995.10476572 · Cited by 12,558
Classification And Regression Trees
Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (2017) · doi:10.1201/9781315139470 · Cited by 11,417
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
Neural Networks and the Bias/Variance Dilemma
Geman, S., Bienenstock, E., & Doursat, R. (1992) · Neural Computation · doi:10.1162/neco.1992.4.1.1 · Cited by 3,578 · cited 1×
“One can use computer simulations and formal analysis, grounded, for instance, in the bias-variance dilemma (Geman, Bienenstock, & Doursat, 1992), to specify when less complexity is more— such as when a structural equation model with fewer parameters makes better predictions than one with more…”
MethodHARKing: Hypothesizing after the results are known
Kerr, N. L. (1998) · Personality and Social Psychology Review · doi:10.1207/s15327957pspr0203_4 · Cited by 1,989
Wiltermuth, S. S., & Heath, C. (2009) · Psychological Science · doi:10.1111/j.1467-9280.2008.02253.x · Cited by 1,717
Puzzlingly High Correlations in fMRI Studies of Emotion, Personality, and Social Cognition
Vul, E., Harris, C., Winkielman, P., & Pashler, H. (2009) · Perspectives on Psychological Science · doi:10.1111/j.1745-6924.2009.01125.x · Cited by 1,475 · cited 1×
“Selection of correlations after the fact appears to be common practice in social neurosciences, resulting in “voodoo correlations” (Vul, Harris, Winkielman, & Pashler, 2009: 274)—correla-tions between brain activity and psychological measures that are higher than they can possibly be, given the…”
BackgroundGigerenzer, G. (2004) · The Journal of Socio-Economics · doi:10.1016/j.socec.2004.09.033 · Cited by 890
Self-citationMisinterpretations of Significance: A Problem Students Share with Their Teachers?
Haller, H., & Krauss, S. (2002) · IPN - Institute for Science Education at the University of Kiel, Germany · doi:10.23668/psycharchives.12769 · Cited by 250
Simkin, M. V., Roychowdhury, V. P., & Department of Electrical Engineering, U. O. C. (2024) · Complex Systems · doi:10.25088/complexsystems.14.3.269 · Cited by 192
Research works printed but not linked: 73
- 10.1016/j.cognition.2006.11.001 · DOI
- American Psychological Association, 1974, Publication manual of the American Psychological Association
- 10.1098/rstl.1710.0011 · DOI · Paper page
- Bacon F., 1902, Novum organum
- Boring E. G., 1963, History, psychology, and sciences: Selected papers
- 10.1037/h0038034 · DOI · Paper page
- 10.1017/CBO9780511524059 · DOI · Paper page
- 10.1515/9781400844227 · DOI · Paper page
- Daston L., 2005, Social Research, 72, 1, 10.1353/sor.2005.0016 · DOI
- Earman J., 1992, Bayes or bust? A critical examination of Bayesian confirmation theory
- 10.1037/h0044139 · DOI · Paper page
- 10.1214/06-BA101 · DOI · Paper page
- Fisher R. A., 1955, Journal of the Royal Statistical Society, 17, 69, 10.1111/j.2517-6161.1955.tb00180.x · DOI · Paper page
- Fisher R. A., 1956, Statistical methods and scientific inference
- Fisher R. A., 1971, The design of experiments
- 10.1037/a0022672 · DOI · Paper page
- Gerrig R. J., 2002, Psychology and life
- Gigerenzer G., 1987, The probabilistic revolution: Vol. 2. Ideas in the sciences, 11
- Gigerenzer G., 2002, Calculated risks: How to know when numbers deceive you
- 10.1136/bmj.d6386 · DOI
- 10.1111/j.1756-8765.2008.01006.x · DOI · Paper page
- 10.1037/0096-1523.14.3.513 · doi:10.1037/0096-1523.14.3.513 · Paper page
- 10.1093/acprof:oso/9780199744282.001.0001 · DOI · Paper page
- Gigerenzer G., 1987, Cognition as intuitive statistics
- 10.1017/CBO9780511720482 · DOI · Paper page
- Good I. J., 1971, American Statistician, 25, 62
- 10.1002/0471463760 · doi:10.1002/0471463760
- Guilford J. P., 1942, Fundamental statistics in psychology and education
- Hacking I., 1987, The probabilistic revolution: Vol. 1. Ideas in history, 1860
- Haller H., 2002, Methods of Psychological Research—Online, 7, 1
- 10.1016/0167-8116(94)90003-5 · DOI · Paper page
- Information for contributors, 2005, Psychological Science, 16
- 10.1136/bmj.296.6619.401 · DOI
- 10.1016/S0140-6736(13)62227-8 · DOI · Paper page
- 10.1016/j.jmp.2008.09.004 · DOI
- Jeffreys H., 1961, Theory of probability
- 10.1177/0956797611430953 · DOI · Paper page
- 10.1016/0010-0285(72)90016-3 · DOI · Paper page
- Knight F. H., 1921, Risk, uncertainty, and profit
- 10.1007/978-1-4757-4038-7_9 · doi:10.1007/978-1-4757-4038-7_9
- Krüger L., 1987, The probabilistic revolution: Vol. 1. Ideas in history
- Krüger L., 1987, The probabilistic revolution: Vol. 2. Ideas in the sciences
- Laplace P. S., 1774, Mémoires de mathématique et de physique, presentés a l’Académie Royale des Sciences, par divers savans, & lû dans ses assemblées, 6, 621
- Laplace P. S., 1951, A philosophical essay on probabilities
- Leibniz G. W., 1951, Selections, 17
- 10.2307/1426315 · DOI
- Lindley D. V., 1983, Journal of the Royal Statistical Society. Series D (The Statistician), 32, 1
- Lindley D. V., 1986, American Statistician, 40, 6
- Luce R. D., 1988, Contemporary Psychology, 33, 582, 10.1037/030460 · DOI
- 10.1080/00913367.1995.10673490 · DOI
- Maslow A. H., 1966, The psychology of science
- 10.1037/h0045549 · DOI
- Newton I., 1952, Opticks. Or a treatise of the reflections, refractions, inflections, and colours of light
- 10.2307/1401671 · DOI · Paper page
- Nunnally J. C., 1975, Introduction to statistics for psychology and education
- Oakes M., 1986, Statistical inference: A commentary for the social and behavioral sciences
- Olby R., 2003, Nature, 276, 565
- 10.1097/00001648-199805000-00019 · DOI
- Savage L. J., 1972, The foundations of statistics
- Simmons J. P., 2011, Psychological Science, 20, 1
- Simon H. A., 1979, American Economic Review, 69, 493
- 10.1111/j.1467-9280.1992.tb00017.x · DOI · Paper page
- Soros G., 2009, The crash of 2008 and what it means: The new paradigm for financial markets
- 10.1007/978-3-642-52178-2_2 · doi:10.1007/978-3-642-52178-2_2
- 10.1037/006366 · DOI
- Stigler S. M., 1983, American Statistician, 37, 290
- Stigler S. M., 1999, Statistics on the table: The history of statistical concepts and methods
- 10.5465/amj.54.5.zoj1081 · doi:10.5465/amj.54.5.zoj1081
- 10.1371/journal.pone.0056180 · DOI
- 10.1111/j.1365-2966.2007.11738.x · DOI · Paper page
- Tukey J. W., 1977, Exploratory data analysis
- 10.2307/2393722 · DOI · Paper page
- Ziliak S. T., 2004, American Economic Review. Journal of Socio-Economics, 33, 527
Cited by
Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations
Greenland, S., Senn, S. J., Rothman, K. J., Carlin, J. B., Poole, C., Goodman, S. N., & Altman, D. G. (2016) · European Journal of Epidemiology · doi:10.1007/s10654-016-0149-3 · Cited by 3,200
Kruschke, J. K., & Liddell, T. M. (2018) · Psychonomic Bulletin & Review · doi:10.3758/s13423-016-1221-4 · Cited by 1,288
Brysbaert, M. (2019) · Journal of Cognition · doi:10.5334/joc.72 · Cited by 996
The natural selection of bad science
Smaldino, P. E., & McElreath, R. (2016) · Royal Society Open Science · doi:10.1098/rsos.160384 · Cited by 860
“…for career success in science (Nosek & Lakens, 2014; Nosek et al., 2012; Ioannidis, 2014a; Vankov et al., 2014; Fischer, Ritchie & Hanspach, 2012; Brembs, Button & Munafò, 2013; Begley, Buchan & Dirnagl, 2015; MacCoun & Perlmutter, 2015; Gigerenzer & Marewski, 2015; Sills, 2016; Sarewitz, 2016).”
Saltelli, A., Aleksankina, K., Becker, W., Fennell, P., Ferretti, F., Holst, N., Li, S., & Wu, Q. (2019) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2019.01.012 · Cited by 813
Metadata
- Authors
- Gigerenzer, G., & Marewski, J. N.
- Venue
- Journal of Management
- Publisher
- SAGE Publications
- Published
- 2014-09-02
- Volume
- 41
- Issue
- 2
- Pages
- 421-440
- ISSN
- 0149-2063 (ISSN-L)
- 1557-1211
- OpenAlex
- W2170702398
- Cited by
- 215
- References
- 10
- FWCI
- 22.00
- Type
- journal article
- Language
- en
- Open access
- green
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Surrogate Science: The Idol of a Universal Method for Scientific Inference. https://app.cochat.ai/papers/10.1177/0149206314547522
- The paper
- Gigerenzer, G., & Marewski, J. N. (2015). Surrogate Science: The Idol of a Universal Method for Scientific Inference. Journal of Management. https://doi.org/10.1177/0149206314547522
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.