The DOI matches its Crossref record; 75 of the 75 references it cites were found in a registry or an index.
How Computational Modeling Can Force Theory Building in Psychological Science
Open accessPublished version of a preprint
Abstract
Psychology endeavors to develop theories of human capacities and behaviors on the basis of a variety of methodologies and dependent measures. We argue that one of the most divisive factors in psychological science is whether researchers choose to use computational modeling of theories (over and above data) during the scientific-inference process. Modeling is undervalued yet holds promise for advancing psychological science. The inherent demands of computational modeling guide us toward better science by forcing us to conceptually analyze, specify, and formalize intuitions that otherwise remain unexamined—what we dub open theory. Constraining our inference process through modeling enables us to build explanatory and predictive theories. Here, we present scientific inference in psychology as a path function in which each step shapes the next. Computational modeling can constrain these steps, thus advancing scientific inference over and above the stewardship of experimental practice (e.g., preregistration). If psychology continues to eschew computational modeling, we predict more replicability crises and persistent failure at coherent theory building. This is because without formal modeling we lack open and transparent theorizing. We also explain how to formalize, specify, and implement a computational model, emphasizing that the advantages of modeling can be achieved by anyone with benefit to all.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Prospect Theory: An Analysis of Decision under Risk
Kahneman, D., & Tversky, A. (1979) · Econometrica · doi:10.2307/1914185 · Cited by 48,524
Prospect theory: An analysis of decision under risk
Kahneman, D., & Tversky, A. (1988) · Decision, Probability and Utility · doi:10.1017/cbo9780511609220.014 · Cited by 32,474
(2011) · Cited by 14,317
Classical conditioning II: Current research and theory.
Black, A. H., & Prokasy, W. F. (1972) · Appleton-Century-Crofts eBooks · Cited by 4,449
Falsification and the Methodology of Scientific Research Programmes
Lakatos, I. (1976) · Can Theories be Refuted? · doi:10.1007/978-94-010-1863-0_14 · Cited by 3,915
The gender similarities hypothesis.
Hyde, J. S. (2005) · American Psychologist · doi:10.1037/0003-066x.60.6.581 · Cited by 3,084
Promoting an open research culture
Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., Buck, S., Chambers, C. D., Chin, G., Christensen, G., et al. (2015) · Science · doi:10.1126/science.aab2374 · Cited by 2,708
Conditioned Reflexes: An Investigation of the Physiological Activity of the Cerebral Cortex.
(1930) · Archives of Neurology And Psychiatry · doi:10.1001/archneurpsyc.1930.02220110256018 · Cited by 2,673
Measuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling
John, L. K., Loewenstein, G., & Prelec, D. (2012) · Psychological Science · doi:10.1177/0956797611430953 · Cited by 2,239
The Atomic Components of Thought
Anderson, J. R., & Lebiere, C. J. (2014) · Psychology Press eBooks · doi:10.4324/9781315805696 · Cited by 2,173
Box, G. E. P. (1976) · Journal of the American Statistical Association · doi:10.1080/01621459.1976.10480949 · Cited by 2,114
The preregistration revolution
Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1708274114 · Cited by 1,998
HARKing: Hypothesizing after the results are known
Kerr, N. L. (1998) · Personality and Social Psychology Review · doi:10.1207/s15327957pspr0203_4 · Cited by 1,989
The Atomic Components of Thought
Taatgen, N. (1999) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(98)01250-9 · Cited by 1,476
The Extent and Consequences of P-Hacking in Science
Head, M. L., Holman, L., Lanfear, R., Kahn, A. T., & Jennions, M. D. (2015) · PLOS Biology · doi:10.1371/journal.pbio.1002106 · Cited by 1,388
Models as Mediators: Perspectives on Natural and Social Science
(1999) · Cambridge University Press eBooks · doi:10.1017/cbo9780511660108 · Cited by 1,233
Baddeley, A. (2010) · Current Biology · doi:10.1016/j.cub.2009.12.014 · Cited by 1,122
Variability in the analysis of a single neuroimaging dataset by many teams
(2020) · doi:10.1038/s41586-020-2314-9 · Cited by 1,097
Theory-Testing in Psychology and Physics: A Methodological Paradox
Meehl, P. E. (1967) · Philosophy of Science · doi:10.1086/288135 · Cited by 962
Vinkhuyzen, R. E., Verschure, P. F. M. J., & Newell, A. (1994) · The American Journal of Psychology · doi:10.2307/1422886 · Cited by 831
YOU CAN'T PLAY 20 QUESTIONS WITH NATURE AND WIN: PROJECTIVE COMMENTS ON THE PAPERS OF THIS SYMPOSIUM
Newell, A. (1973) · Visual Information Processing · doi:10.1016/b978-0-12-170150-5.50012-3 · Cited by 820 · cited 4×
“Psychological scientists typically ascribe to a school of thought that specifies a framework, a theoretical position, or at least some basic hypotheses that they then set out to test using inferential statistics (Meehl, 1967; Newell, 1973).”
BackgroundInfluentialMany Analysts, One Data Set: Making Transparent How Variations in Analytic Choices Affect Results
Silberzahn, R., Uhlmann, E. L., Martin, D. P., Anselmi, P., Aust, F., Awtrey, E., Bahník, Š., Bai, F., Bannard, C., Bonnier, E., et al. (2018) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245917747646 · Cited by 784
PUTNAM, H. (1967) · Art, Mind, and Religion · doi:10.2307/jj.6380610.6 · Cited by 778
SUSTAIN: A Network Model of Category Learning.
Love, B. C., Medin, D. L., & Gureckis, T. M. (2004) · Psychological Review · doi:10.1037/0033-295x.111.2.309 · Cited by 775
Ten simple rules for the computational modeling of behavioral data
Wilson, R. C., & Collins, A. G. (2019) · eLife · doi:10.7554/elife.49547 · Cited by 710
From Understanding Computation to Understanding Neural Circuitry
Marr, D., & Poggio, T. (1976) · Defense Technical Information Center (DTIC) · Cited by 587
Jones, M., & Love, B. C. (2011) · Behavioral and Brain Sciences · doi:10.1017/s0140525x10003134 · Cited by 533
Models as MediatorsPreprint
Morrison, M., & Morgan, M. S. (1999) · RePEc: Research Papers in Economics · Cited by 481
Addressing the theory crisis in psychology
Oberauer, K., & Lewandowsky, S. (2019) · Psychonomic Bulletin & Review · doi:10.3758/s13423-019-01645-2 · Cited by 462
Model-based and model-free Pavlovian reward learning: Revaluation, revision, and revelation
Dayan, P., & Berridge, K. C. (2014) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/s13415-014-0277-8 · Cited by 357 · cited 1×
“Some frameworks appear superficially simple or narrow, such as the concept of working memory (Baddeley, 2010) or dual-systems approaches (Dayan & Berridge, 2014; Kahneman, 2011), whereas others can be all-encompassing such as unified theories of cognition (Newell, 1990) or connectionism (McClelland…”
Backgroundvan Rooij, I., & Baggio, G. (2021) · Perspectives on Psychological Science · doi:10.1177/1745691620970604 · Cited by 354
Morgenbesser, S. (1967) · Cited by 339
Lack of theory building and testing impedes progress in the factor and network literature
Fried, E. I. (2020) · Psychological Inquiry · doi:10.1080/1047840x.2020.1853461 · Cited by 317
Should social science be more solution-oriented?
Watts, D. J. (2017) · Nature Human Behaviour · doi:10.1038/s41562-016-0015 · Cited by 301
The Tractable Cognition Thesis
Van Rooij, I. (2008) · Cognitive Science · doi:10.1080/03640210801897856 · Cited by 285
Psychology of Learning and Motivation
Collectif, C. (2016) · Elsevier eBooks · Cited by 223
Modeling psychopathology: From data models to formal theories.
Haslbeck, J. M. B., Ryan, O., Robinaugh, D. J., Waldorp, L. J., & Borsboom, D. (2021) · Psychological Methods · doi:10.1037/met0000303 · Cited by 212
Is Preregistration Worthwhile?
Szollosi, A., Kellen, D., Navarro, D., Shiffrin, R. M., van Rooij, I., Zandt, T. V., & Donkin, C. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2019.11.009 · Cited by 177
Forscher, B. K. (1963) · Science · doi:10.1126/science.142.3590.339 · Cited by 157
Martin, A. E. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2016.00120 · Cited by 147
Self-citationNavarro, D. (2018) · Computational Brain & Behavior · doi:10.1007/s42113-018-0019-z · Cited by 145
Torretti, R. (1999) · Cambridge University Press eBooks · doi:10.1017/cbo9781139172981 · Cited by 142
Sustainable computational science: the ReScience initiative
Rougier, N. P., Hinsen, K., Alexandre, F., Arildsen, T., Barba, L. A., Benureau, F., Brown, C. T., de Buyl, P., Çağlayan, O., Davison, A. P., et al. (2017) · PeerJ Computer Science · doi:10.7717/peerj-cs.142 · Cited by 141
Self-citationA Compositional Neural Architecture for Language
Martin, A. E. (2020) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_01552 · Cited by 141
Self-citationDevezer, B., Nardin, L. G., Baumgaertner, B. O., & Buzbas, E. O. (2019) · PLoS ONE · doi:10.1371/journal.pone.0216125 · Cited by 127
Miłkowski, M., Hensel, W. M., & Hohol, M. (2018) · Journal of Computational Neuroscience · doi:10.1007/s10827-018-0702-z · Cited by 116
SOAR as a unified theory of cognition: Issues and explanations
Newell, A. (1992) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00069740 · Cited by 108
A systematic methodology for cognitive modelling
Cooper, R., Fox, J., Farringdon, J., & Shallice, T. (1996) · Artificial Intelligence · doi:10.1016/0004-3702(95)00112-3 · Cited by 106
How to play 20 questions with nature and lose: Reflections on 100 years of brain-training research
Katz, B., Shah, P., & Meyer, D. E. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1617102114 · Cited by 96
An Abductive Theory of Scientific Method
Haig, B. D. (2018) · Studies in applied philosophy, epistemology and rational ethics · doi:10.1007/978-3-030-01051-5_3 · Cited by 94
On the adequacy of current empirical evaluations of formal models of categorization.
Wills, A. J., & Pothos, E. M. (2011) · Psychological Bulletin · doi:10.1037/a0025715 · Cited by 92
Models of Success Versus the Success of Models: Reliability without Truth
Winsberg, E. B. (2006) · Synthese · doi:10.1007/s11229-004-5404-6 · Cited by 83
Lack of theory building and testing impedes progress in the factor and network literaturePreprint
Fried, E. I. (2020) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/zg84s · Cited by 82
Multiple time-scales and the developmental dynamics of social systems
Flack, J. C. (2012) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2011.0214 · Cited by 78
Flis, I. (2019) · Theory & Psychology · doi:10.1177/0959354319835322 · Cited by 74
Improving Transparency, Falsifiability, and Rigor by Making Hypothesis Tests Machine-Readable
Lakens, D., & DeBruine, L. M. (2021) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245920970949 · Cited by 69
Cooper, R., & Guest, O. (2013) · Cognitive Systems Research · doi:10.1016/j.cogsys.2013.05.001 · Cited by 62
Self-citationCooper, R. (2007) · Cognitive Science · doi:10.1080/15326900701326592 · Cited by 60
What the success of brain imaging implies about the neural code
Guest, O., & Love, B. C. (2017) · eLife · doi:10.7554/elife.21397 · Cited by 59
Self-citationA Model Hierarchy for Psychological Science
Kellen, D. (2019) · Computational Brain & Behavior · doi:10.1007/s42113-019-00037-y · Cited by 51
Modelling meaning composition from formalism to mechanism
Martin, A. E., & Baggio, G. (2019) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2019.0298 · Cited by 47
Self-citationThe memory tesseract: Mathematical equivalence between composite and separate storage memory models
Kelly, M. A., Mewhort, D. J. K., & West, R. L. (2016) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2016.10.006 · Cited by 37
Modeling accuracy as a function of response time with the generalized linear mixed effects model
Davidson, D. J., & Martin, A. E. (2013) · Acta Psychologica · doi:10.1016/j.actpsy.2013.04.016 · Cited by 30
Self-citationExemplars, Prototypes, and the Flexibility of Classification Models.
Olsson, H., Wennerholm, P., & Lyxzèn, U. (2004) · Journal of Experimental Psychology Learning Memory and Cognition · doi:10.1037/0278-7393.30.4.936 · Cited by 27
SOAR as a world view, not a theory
Hunt, E. B., & Luce, R. D. (1992) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00069570 · Cited by 25
Sculpting Computational‐Level Models
Blokpoel, M. (2017) · Topics in Cognitive Science · doi:10.1111/tops.12282 · Cited by 22
Wimsatt, W. C. (2002) · Philosophy of Science · doi:10.1086/341764 · Cited by 19
Levering, K. R., Conaway, N., & Kurtz, K. J. (2019) · Memory & Cognition · doi:10.3758/s13421-019-00972-y · Cited by 19
Vere, S. A. (1992) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00069703 · Cited by 17
Conceptual Organization is Revealed by Consumer Activity Patterns
Hornsby, A. N., Evans, T., Riefer, P. S., Prior, R., & Love, B. C. (2019) · Computational Brain & Behavior · doi:10.1007/s42113-019-00064-9 · Cited by 16
Conceptual Organization is Revealed by Consumer Activity Patterns
Hornsby, A., Evans, T., Riefer, P., Prior, R., & Love, B. (2019) · UCL Discovery (University College London) · Cited by 16
On Simulating Neural Damage in Connectionist Networks
Guest, O., Caso, A., & Cooper, R. (2020) · Computational Brain & Behavior · doi:10.1007/s42113-020-00081-z · Cited by 16
Self-citationProgress in Modeling Through Distributed Collaboration
Wills, A. J., O’Connell, G., Edmunds, C. E. R., & Inkster, A. B. (2017) · The Psychology of learning and motivation/The psychology of learning and motivation · doi:10.1016/bs.plm.2016.11.007 · Cited by 15
Beyond Statistics: Accepting the Null Hypothesis in Mature Sciences
Morey, R. D., Homer, S., & Proulx, T. (2018) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245918776023 · Cited by 12
Origin Gaps and the Eternal Sunshine of the Second-Order Pendulum
DeDeo, S. (2018) · The frontiers collection · doi:10.1007/978-3-319-75726-1_5 · Cited by 3
Research works printed but not linked: 13
- 10.1093/mind/LXIII.249.49 · doi:10.1093/mind/lxiii.249.49
- Cox G. E., Oxford handbook of human memory
- Epstein J. M. (2008). Why model? Journal of Artificial Societies and Social Simulation, 11(4), Article 12. http://jasss.soc.surrey.ac.uk/11/4/12.html
- Feyerabend P. K. (1957). An attempt at a realistic interpretation of experience. Proceedings of the Aristotelian Society, 58, 143–170. https://www.jstor.org/stable/4544593 · doi:10.1093/aristotelian/58.1.143
- 10.1126/science.142.3590.339.a · doi:10.1126/science.142.3590.339.a
- 10.1080/01621459.1984.10478038 · doi:10.1080/01621459.1984.10478038
- McClelland J. L., 1986, Explorations in the Microstructure of Cognition, 2, 216
- Monperrus M., Jézéquel J.M., Champeau J., Hoeltzener B. (2008). A model-driven measurement approach. In ACM/IEEE 11th International conference on model driven engineering languages and systems (pp. 505–519). Springer. https://doi.org/10.1007/978-3-540-87875-9_36 · doi:10.1007/978-3-540-87875-9_36
- Orwell G., 1945, Animal farm
- 10.1007/s42113-019-00062-x · DOI
- Shiffrin R. M., 2018, Should statistics determine the practice of science, or science determine the practice of statistics?
- Spivey J. M., 1992, The Z notation
- Wiener P. P., 1951, Leibniz: Selections, 1
Cited by
The past, present and future of Registered Reports
Chambers, C. D., & Tzavella, L. (2021) · Nature Human Behaviour · doi:10.1038/s41562-021-01193-7 · Cited by 499
van Rooij, I., & Baggio, G. (2021) · Perspectives on Psychological Science · doi:10.1177/1745691620970604 · Cited by 354
Lack of theory building and testing impedes progress in the factor and network literature
Fried, E. I. (2020) · Psychological Inquiry · doi:10.1080/1047840x.2020.1853461 · Cited by 317
That’s a Lot to Process! Pitfalls of Popular Path Models
Rohrer, J. M., Hünermund, P., Arslan, R. C., & Elson, M. (2022) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459221095827 · Cited by 267
The case for formal methodology in scientific reform
Devezer, B., Navarro, D. J., Vandekerckhove, J., & Buzbas, E. O. (2020) · doi:10.1101/2020.04.26.048306 · Cited by 195
Metadata
- Authors
- Guest, O., & Martin, A. E.
- Venue
- Perspectives on Psychological Science
- Publisher
- SAGE Publications
- Published
- 2021-01-22
- Volume
- 16
- Issue
- 4
- Pages
- 789-802
- ISSN
- 1745-6916 (ISSN-L)
- 1745-6924
- OpenAlex
- W3123354371
- Cited by
- 460
- References
- 75
- FWCI
- 159.60
- Type
- journal article
- Language
- en
- Open access
- gold
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Funders
- Max-Planck-Gesellschaft (Language and Computation in Neural Systems)
- nederlandse organisatie voor wetenschappelijk onderzoek (016.Vidi.188.029)
- Horizon 2020 Framework Programme (739578)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.