CoChat

Invisible Hands and Fine Calipers: A Call to Use Formal Theory as a Toolkit for Theory Construction

Open access
Robinaugh, Donald J., Haslbeck, Jonas M. B., Ryan, Oisín, Fried, Eiko I., Waldorp, Lourens J. · 2021 · Perspectives on Psychological Science

Published version of a preprint

Cited by 209

The DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.

Abstract

In recent years, a growing chorus of researchers has argued that psychological theory is in a state of crisis: Theories are rarely developed in a way that indicates an accumulation of knowledge. Paul Meehl raised this very concern more than 40 years ago. Yet in the ensuing decades, little has improved. We aim to chart a better path forward for psychological theory by revisiting Meehl’s criticisms, his proposed solution, and the reasons his solution failed to meaningfully change the status of psychological theory. We argue that Meehl identified serious shortcomings in our evaluation of psychological theories and that his proposed solution would substantially strengthen theory testing. However, we also argue that Meehl failed to provide researchers with the tools necessary to construct the kinds of rigorous theories his approach required. To advance psychological theory, we must equip researchers with tools that allow them to better generate, evaluate, and develop their theories. We argue that formal theories provide this much-needed set of tools, equipping researchers with tools for thinking, evaluating explanation, enhancing measurement, informing theory development, and promoting the collaborative construction of psychological theories.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1177/1745691620974697

Cites

15 of 15 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 62

  1. 10.1176/appi.books.9780890425596 · DOI · Paper page
  2. 10.1037/0022-3514.54.3.441 · DOI
  3. 10.2307/j.ctt5vkdnq · doi:10.2307/j.ctt5vkdnq
  4. 10.1016/0022-1031(71)90065-5 · DOI · Paper page
  5. 10.2307/2185445 · DOI · Paper page
  6. Borsboom D., van der Maas H., Dalege J., Kievit R., Haig B. (2021). Theory construction methodology: A practical framework for theory formation in psychology. Perspectives on Psychological Science, 16(4), 756–766. https://doi.org/10.1177/1745691620969647 · DOI · Paper page
  7. 10.1037/a0032850 · DOI · Paper page
  8. 10.1016/0162-3095(83)90009-2 · DOI · Paper page
  9. Conroy-Beam D., Buss D. M., Asao K., Sorokowska A., Sorokowski P., Aavik T., Akello G., Alhabahba M. M., Alm C., Amjad N., Anjum A., Atama C. S., Duyar D. A., Ayebare R., Batres C., Bendixen M., Bensafia A., Bizumic B., Boussena M. Zupancˇicˇ M. (2019). Contrasting computational models of mate preference integration across 45 countries. Scientific Reports, 9(1), Article 16885. https://doi.org/10.1038/s41598-019-52748-8 · DOI · Paper page
  10. Denrell J., Le Mens G. (2007). Interdependent sampling and social influence. Psychological Review, 114(2), 398–422. https://doi.org/10.1037/0033-295X.114.2.398 · DOI
  11. Elliott-Graves A. (2014). The role of target systems in scientific practice [Unpublished doctoral dissertation, University of Pennsylvania].
  12. Epstein J. M. (2008). Why model? Journal of Artificial Societies and Social Simulation, 11(4), Article 12. http://jasss.soc.surrey.ac.uk/11/4/12.html
  13. 10.1016/0022-2496(75)90025-5 · DOI
  14. 10.1017/CBO9781316272503 · DOI · Paper page
  15. Feingold A. (1988). Matching for attractiveness in romantic partners and same-sex friends: A metaanalysis and theoretical critique. Psychological Bulletin, 104(2), 226–235. https://doi.org/10.1037/0033-2909.104.2.226 · DOI · Paper page
  16. Fradkin I., Adams R. A., Parr T., Roiser J. P., Huppert J. D. (2020). Searching for an anchor in an unpredictable world: A computational model of obsessive compulsive disorder. Psychological Review, 127(5), 672–699. https://doi.org/10.1037/rev0000188 · DOI · Paper page
  17. Fried E. I., Flake J. K. (2018). Measurement matters. APS Observer, 31(3), 29–30. https://www.psychologicalscience.org/observer/measurement-matters.
  18. 10.1177/0959354310378184 · DOI · Paper page
  19. Guest O., Martin A. E. (2021). How computational modeling can force theory building in psychological science. Perspectives on Psychological Science, 16(4), 789–802. https://doi.org/10.1177/1745691620970585 · DOI · Paper page
  20. 10.1037/1082-989X.10.4.371 · DOI · Paper page
  21. 10.7551/mitpress/9780262027366.001.0001 · DOI
  22. Haslbeck J., Ryan O., Robinaugh D., Waldorp L., Borsboom D. (2019). Modeling psychopathology: From data models to formal theories. PsyArXiv. https://doi.org/10.31234/osf.io/jgm7f · DOI
  23. Jaccard J., 2019, Theory construction and model-building skills: A practical guide for social scientists
  24. Jia T., Spivey R. F., Szymanski B., Korniss G. (2015). An analysis of the matching hypothesis in networks. PLOS ONE, 10(6), Article e0129804. https://doi.org/10.1371/journal.pone.0129804 · DOI
  25. Kalick S. M., Hamilton T. E. (1986). The matching hypothesis reexamined. Journal of Personality and Social Psychology, 51(4), 673–682. https://doi.org/10.1037/0022-3514.51.4.673 · DOI · Paper page
  26. Kalick S. M., Hamilton T. E. (1988). Closer look at a matching simulation: Reply to Aron. Journal of Personality and Social Psychology, 54(3), 447–451. https://doi.org/10.1037/0022-3514.54.3.447 · doi:10.1037//0022-3514.54.3.447
  27. Kellen D., Davis-Stober C., Dunn J. C., Kalish M. (2021). The problem of coordination and the pursuit of structural constraints in psychology. Perspectives on Psychological Science, 16(4), 767–778. https://doi.org/10.1177/1745691620970585 · DOI
  28. 10.1002/casp.2450010108 · DOI
  29. Lewandowsky S., 2010, Computational modeling in cognition: Principles and practice
  30. 10.3758/BF03197461 · DOI · Paper page
  31. Lykken D. T., 1991, Thinking clearly about psychology: Vol. 1. Matters of public interest, 3
  32. 10.1037/0022-006X.46.4.806 · DOI · Paper page
  33. 10.1207/s15327965pli0102_1 · DOI · Paper page
  34. 10.2466/pr0.1990.66.1.195 · DOI · Paper page
  35. 10.1038/s41562-018-0522-1 · DOI · Paper page
  36. Nozick R., 1974, Anarchy, state, and utopia
  37. Orben A., Lakens D. (2019). Crud (re)defined. Advances in Methods and Practices in Psychological Science, 3(2), 238–247. https://doi.org/10.1177/2515245920917961 · DOI · Paper page
  38. Pickering A. D. (2008). Formal and computational models of reinforcement sensitivity theory. In Corr P. J. (Ed.), The reinforcement sensitivity theory of personality (p. 453–481). Cambridge University Press. https://doi.org/10.1017/CBO9780511819384.017 · DOI
  39. 10.1037/0000135-010 · DOI
  40. Ritter F. E., Tehranchi F., Oury J. D. (2019). Act-r: A cognitive architecture for modeling cognition. Wiley Interdisciplinary Reviews: Cognitive Science, 10(3), Article e1488. https://doi.org/10.1002/wcs.1488 · DOI · Paper page
  41. Rodgers J. L. (2010). The epistemology of mathematical and statistical modeling: A quiet methodological revolution. American Psychologist, 65(1), 1–12. https://doi.org/10.1037/a0018326 · DOI · Paper page
  42. 10.1086/287823 · DOI
  43. Rozeboom W. W. (1990). Hypothetico-deductivism is a fraud. American Psychologist, 45(4), 555–556. https://doi.org/10.1037/0003-066X.45.4.555 · doi:10.1037//0003-066x.45.4.555
  44. Rozeboom W. W., 1997, What if there were no significance tests, 335
  45. Schiepek G., 2016, Nonlinear Dynamics, Psychology, and Life Sciences, 20, 369
  46. Schuler K., Ruggero C. J., Mahaffey B., Gonzalez A. L., Callahan J., Boals A., Waszczuk M. A., Luft B. J., Kotov R. (2021). When hindsight is not 20/20: Ecological momentary assessment of PTSD symptoms versus retrospective report. Assessment, 28(1), 238–247. https://doi.org/10.1177/1073191119869826 · DOI
  47. 10.1146/annurev-psych-122216-011845 · DOI · Paper page
  48. Smaldino P. E. (2016). Not even wrong: Imprecision perpetuates the illusion of understanding at the cost of actual understanding. Behavioral and Brain Sciences, 39, Article e163. https://doi.org/10.1017/S0140525X1500151X · DOI
  49. 10.4324/9781315173726-14 · DOI · Paper page
  50. 10.1038/d41586-019-03350-5 · DOI
  51. Smaldino P. E. (2020). How to translate a verbal theory into a formal model. Social Psychology, 51, 207–218. https://doi.org/10.1027/1864-9335/a000425. · DOI
  52. 10.1177/1088868306294789 · DOI · Paper page
  53. 10.1111/j.1747-9991.2009.00261.x · DOI
  54. 10.1086/702029 · doi:10.1086/702029
  55. 10.1007/BF00499820 · DOI · Paper page
  56. van der Maas H. L., Molenaar D., Maris G., Kievit R. A., Borsboom D. (2011). Cognitive psychology meets psychometric theory: On the relation between process models for decision making and latent variable models for individual differences. Psychological Review, 118(2), 339–356. https://doi.org/10.1037/a0022749 · DOI · Paper page
  57. 10.1086/422204 · DOI · Paper page
  58. van Rooij I., Baggio G. (2020). Theory development requires an epistemological sea change. PsyArXiv. https://doi.org/10.31234/osf.io/rnw9q · doi:10.31234/osf.io/rnw9q
  59. van Rooij I., Blokpoel M. (2020, July). Formalizing verbal theories: A tutorial by dialogue. PsyArXiv. https://doi.org/10.31234/osf.io/r2zqy · doi:10.31234/osf.io/r2zqy
  60. 10.3758/s13421-011-0158-0 · DOI · Paper page
  61. Wilensky U., 2015, An introduction to agent-based modeling: Modeling natural, social, and engineered complex systems with netlogo
  62. Wimsatt W. C., 1987, Neutral models in biology, 23

Cited by

See all 209

Metadata

Authors
Robinaugh, D. J., Haslbeck, J. M. B., Ryan, O., Fried, E. I., & Waldorp, L. J.
Venue
Perspectives on Psychological Science
Published
2021-02-16
Volume
16
Issue
4
Pages
725-743
ISSN
  • 1745-6916 (ISSN-L)
  • 1745-6924
OpenAlex
W3129957462
Cited by
209
References
15
FWCI
74.22
Type
journal article
Language
en
Open access
gold
Licence
journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
Funders
  • H2020 European Research Council (Consolidator Grant / 647209)
  • Nederlandse Organisatie voor Wetenschappelijk Onderzoek (Onderzoekstalent Grant / 406-15-128)
  • National Institute of Mental Health (K23MH113805)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.