Lack of theory building and testing impedes progress in the factor and network literature
Open accessThe DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
Abstract
The last decade has brought reforms to improve methodological practices, with the goal to increase the reliability and replicability of effects. However, explanations of effects remain scarce, and a growing chorus of scholars argues that the replicability crisis has distracted from a crisis of theory. In the same decade, the empirical literature using factor and network models has grown rapidly. I discuss three ways in which this literature falls short of theory building and testing. First, statistical and theoretical models are conflated, leading to invalid inferences such as the existence of psychological constructs based on factor models, or recommendations for clinical interventions based on network models. I demonstrate this inferential gap in a simulation study on statistical equivalence: excellent model fit does little to corroborate a theory, regardless of quality or quantity of data. Second, researchers fail to explicate theories about psychological constructs, but use implicit causal beliefs to guide inferences. These latent theories have led to problematic best practices in psychological research where inferences are drawn based on one specific causal model that is assumed, but not explicated. Third, explicated theories are often weak theories: narrative and imprecise descriptions vulnerable to hidden assumptions and unknowns. They fail to make clear predictions, and it remains unclear whether statistical effects corroborate such theories or not. Weak theories are immune to refutation or revision. I argue that these three challenges to theory building and testing are common and harmful, and impede theory formation, failure, and reform. A renewed focus on theoretical psychology and formal models offers a way forward.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
"General Intelligence," Objectively Determined and Measured
Spearman, C. (1904) · The American Journal of Psychology · doi:10.2307/1412107 · Cited by 6,207
Conventional wisdom on measurement: A structural equation perspective.
Bollen, K., & Lennox, R. (1991) · Psychological Bulletin · doi:10.1037/0033-2909.110.2.305 · Cited by 3,618
A network theory of mental disorders
Borsboom, D. (2017) · World Psychiatry · doi:10.1002/wps.20375 · Cited by 3,482 · cited 3×
“However, network theory is based at its core on the idea of self-sustaining interactions among problems, such as the vicious cycle between perceived threat and physiological arousal that can give rise to panic attacks (Borsboom, 2017; Robinaugh, Haslbeck, et al., 2019).”
BackgroundInfluentialBorgatti, S. P. (2005) · Social Networks · doi:10.1016/j.socnet.2004.11.008 · Cited by 3,191
Hammen, C. (2005) · Annual Review of Clinical Psychology · doi:10.1146/annurev.clinpsy.1.102803.143938 · Cited by 2,979
A tutorial on regularized partial correlation networks
Epskamp, S., & Fried, E. I. (2018) · Psychological Methods · doi:10.1037/met0000167 · Cited by 2,918
Self-citationThe theoretical status of latent variables.
Borsboom, D., Mellenbergh, G. J., & van Heerden, J. (2003) · Psychological Review · doi:10.1037/0033-295x.110.2.203 · Cited by 1,566
Fraley, R. C., Heffernan, M. E., Vicary, A. M., & Brumbaugh, C. C. (2011) · Psychological Assessment · doi:10.1037/a0022898 · Cited by 1,198
Lack of group-to-individual generalizability is a threat to human subjects research
Fisher, A. J., Medaglia, J. D., & Jeronimus, B. F. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1711978115 · Cited by 1,099
A dynamical model of general intelligence: The positive manifold of intelligence by mutualism.
Van Der Maas, H. L. J., Dolan, C. V., Grasman, R. P. P. P., Wicherts, J. M., Huizenga, H. M., & Raijmakers, M. E. J. (2006) · Psychological Review · doi:10.1037/0033-295x.113.4.842 · Cited by 1,006
Cited by
How Computational Modeling Can Force Theory Building in Psychological Science
Guest, O., & Martin, A. E. (2021) · Perspectives on Psychological Science · doi:10.1177/1745691620970585 · Cited by 460
On the equivalency of factor and network loadings
Christensen, A. P., & Golino, H. (2021) · Behavior Research Methods · doi:10.3758/s13428-020-01500-6 · Cited by 253
Towards a model of uncertainty distress in the context of Coronavirus (COVID-19)
Freeston, M., Tiplady, A., Mawn, L., Bottesi, G., & Thwaites, S. (2020) · The Cognitive Behaviour Therapist · doi:10.1017/s1754470x2000029x · Cited by 221
“…jump that evidence from between-person studies automatically applies to within-person models (see Molenaar, 2004) and within-person or idiographic studies are required (see Wright and Woods, 2020), despite the challenges of defining, operationalising and analysing such models (e.g. Fried, 2020).”
The case for formal methodology in scientific reform
Devezer, B., Navarro, D. J., Vandekerckhove, J., & Buzbas, E. O. (2020) · doi:10.1101/2020.04.26.048306 · Cited by 195
Burger, J., van der Veen, D. C., Robinaugh, D. J., Quax, R., Riese, H., Schoevers, R. A., & Epskamp, S. (2020) · BMC Medicine · doi:10.1186/s12916-020-01558-1 · Cited by 99
Metadata
- Authors
- Fried, E. I.
- Publisher
- Center for Open Science
- Published
- 2020-02-07
- OpenAlex
- W4250832892
- Cited by
- 53
- References
- 29
- Type
- posted content
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Lack of theory building and testing impedes progress in the factor and network literature. https://app.cochat.ai/papers/10.31234/osf.io/zg84s
- The paper
- Fried, E. I. (2020). Lack of theory building and testing impedes progress in the factor and network literature. https://doi.org/10.31234/osf.io/zg84s
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.