Beyond Statistical Ritual: Theory in Psychological Science
Open accessThe DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
Abstract
More than 40 years ago, Paul Meehl (1978) published a seminal critique of the state of theorizing in psychological science. According to Meehl, the quality of theories had diminished in the preceding decades, resulting in statistical methods standing in for theoretical rigor. In this introduction to the special issue Theory in Psychological Science, we apply Meehl’s account to contemporary psychological science. We suggest that by the time of Meehl’s writing, psychology found itself in the midst of a crisis that is typical of maturing sciences, in which the theories that had been guiding research were gradually cast into doubt. Psychologists were faced with the same general choice when worldviews fail: Face reality and pursue knowledge in the absence of certainty, or shift emphasis toward sources of synthetic certainty. We suggest that psychologists have too often chosen the latter option, substituting synthetic certainties for theory-guided research, in much the same manner as Scholastic scholars did centuries ago. Drawing from our contributors, we go on to make recommendations for how psychological science may fully reengage with theory-based science.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Estimating the reproducibility of psychological science
Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
The Essential Tension: Selected Studies in Scientific Tradition and Change
Kuhn, T. S. (1977) · doi:10.7208/chicago/9780226217239.001.0001 · Cited by 3,861
Jost, J. T., Banaji, M. R., & Nosek, B. A. (2004) · Political Psychology · doi:10.1111/j.1467-9221.2004.00402.x · Cited by 2,883
Redefine statistical significance
Benjamin, D. J., Berger, J. O., Johannesson, M., Nosek, B. A., Wagenmakers, E.-J., Berk, R., Bollen, K. A., Brembs, B., Brown, L., Camerer, C., et al. (2017) · Nature Human Behaviour · doi:10.1038/s41562-017-0189-z · Cited by 2,772
Part I: Cognitive development in children: Piaget development and learning
Piaget, J. (1964) · Journal of Research in Science Teaching · doi:10.1002/tea.3660020306 · Cited by 1,802
P-curve: A key to the file-drawer.
Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2014) · Journal of Experimental Psychology: General · doi:10.1037/a0033242 · Cited by 1,500
Why the child’s theory of mind really is a theory
GOPNIK, A., & WELLMAN, H. M. (1992) · Mind & Language · doi:10.1111/j.1468-0017.1992.tb00202.x · Cited by 1,158 · cited 1×
“…1977), and identity (Erikson, 1994) development, and rich explanatory accounts of executive function (Zelazo & Müller, 2011) and theory of mind (Gopnik & Wellman, 1992) guide research programs that range from language development (Gooch et al., 2016; Sodian et al., 2020) to the causes of…”
BackgroundAn Agenda for Purely Confirmatory Research
Wagenmakers, E.-J., Wetzels, R., Borsboom, D., van der Maas, H. L. J., & Kievit, R. A. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612463078 · Cited by 1,042
Uncertainty management by means of fairness judgments
van den Bos, K., & Lind, E. A. (2002) · Advances in Experimental Social Psychology · doi:10.1016/s0065-2601(02)80003-x · Cited by 913
Gigerenzer, G. (2004) · The Journal of Socio-Economics · doi:10.1016/j.socec.2004.09.033 · Cited by 890 · cited 1×
“Our faith in statistical ritual has become, in the Popperian sense, an unfalsifiable theory ( Gigerenzer, 2004).”
BackgroundPhilosophy and the practice of Bayesian statistics
Gelman, A., & Shalizi, C. R. (2013) · British Journal of Mathematical and Statistical Psychology · doi:10.1111/j.2044-8317.2011.02037.x · Cited by 825 · cited 1×
“From the Bayesian perspective, Gelman and Shalizi (2013) describe the same iterative process; Morey et al. (2013) again emphasize the fictional nature of statistical models.”
BackgroundJost, J. T., Pelham, B. W., Sheldon, O., & Ni Sullivan, B. (2003) · European Journal of Social Psychology · doi:10.1002/ejsp.127 · Cited by 709
Registered Replication Report: Strack, Martin, & Stepper (1988)
Wagenmakers, E.-J., Beek, T., Dijkhoff, L., Gronau, Q. F., Acosta, A., Adams, R. B., Albohn, D. N., Allard, E. S., Benning, S. D., Blouin-Hudon, E.-M., et al. (2016) · Perspectives on Psychological Science · doi:10.1177/1745691616674458 · Cited by 430
Why Hypothesis Testers Should Spend Less Time Testing Hypotheses
Scheel, A. M., Tiokhin, L., Isager, P. M., & Lakens, D. (2021) · Perspectives on Psychological Science · doi:10.1177/1745691620966795 · Cited by 382
Using prediction markets to estimate the reproducibility of scientific research
Dreber, A., Pfeiffer, T., Almenberg, J., Isaksson, S., Wilson, B., Chen, Y., Nosek, B. A., & Johannesson, M. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1516179112 · Cited by 331
Evaluating the Theory-of-Mind Hypothesis of Autism
Tager-Flusberg, H. (2007) · Current Directions in Psychological Science · doi:10.1111/j.1467-8721.2007.00527.x · Cited by 329
Priming, Replication, and the Hardest Science
Cesario, J. (2014) · Perspectives on Psychological Science · doi:10.1177/1745691613513470 · Cited by 320
Comparing meta-analyses and preregistered multiple-laboratory replication projects
Kvarven, A., Strømland, E., & Johannesson, M. (2019) · Nature Human Behaviour · doi:10.1038/s41562-019-0787-z · Cited by 271 · cited 1×
“, 2014)—on the basis of NHST inferences resting on further NHST assumptions ( Johnson, 2013; Kvarven et al., 2020; Morey, 2013).”
BackgroundGigerenzer, G. (1998) · Theory & Psychology · doi:10.1177/0959354398082006 · Cited by 196 · cited 1×
“In the intervening decades, elements of Meehl’s critique have been applied (Gigerenzer, 1998) and reapplied (Muthukrishna & Henrich, 2019) to a science that remains characterized by the identification of effects (van Rooij & Baggio, 2021; this issue).”
MethodThe case for formal methodology in scientific reform
Devezer, B., Navarro, D. J., Vandekerckhove, J., & Buzbas, E. O. (2020) · doi:10.1101/2020.04.26.048306 · Cited by 195
Meehl, P. E. (1978) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.46.4.806 · Cited by 188
Gronau, Q. F., Ly, A., & Wagenmakers, E.-J. (2020) · The American Statistician · doi:10.1080/00031305.2018.1562983 · Cited by 150
Fiedler, K., McCaughey, L., & Prager, J. (2021) · Perspectives on Psychological Science · doi:10.1177/1745691620970602 · Cited by 73
The Cognitive Neuroscience of Psychological Treatment Action in Depression and Anxiety
Young, K. S., & Craske, M. G. (2018) · Current Behavioral Neuroscience Reports · doi:10.1007/s40473-018-0137-x · Cited by 13
Cited by
Advancing the Network Theory of Mental Disorders: A Computational Model of Panic Disorder
Robinaugh, D., Haslbeck, J. M. B., Waldorp, L., Kossakowski, J. J., Fried, E. I., Millner, A., McNally, R. J., Ryan, O., de Ron, J., van der Maas, H., et al. (2019) · doi:10.31234/osf.io/km37w · Cited by 120
Schumann, F., Steinborn, M. B., Kürten, J., Cao, L., Händel, B. F., & Huestegge, L. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.867978 · Cited by 82
Becker, K. D., & Chorpita, B. F. (2023) · Journal of Clinical Child & Adolescent Psychology · doi:10.1080/15374416.2023.2169926 · Cited by 57
“…statistical identification of effects (van Rooij & Baggio, 2021) and instead committing to robust explanatory paradigms (Lilienfeld, 2010; Proulx & Morey, 2021), which could be accelerated and elevated by collaboration to clarify concepts and measurement, hypothesize associations between…”
Background
Metadata
- Authors
- Proulx, T., & Morey, R. D.
- Venue
- Perspectives on Psychological Science
- Publisher
- SAGE Publications
- Published
- 2021-07
- Volume
- 16
- Issue
- 4
- Pages
- 671-681
- ISSN
- 1745-6916 (ISSN-L)
- 1745-6924
- OpenAlex
- W3175896180
- Cited by
- 53
- References
- 25
- FWCI
- 21.34
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.