The DOI matches its Crossref record; 62 of the 62 references it cites were found in a registry or an index.
Comparing network structures on three aspects: A permutation test.
Open accessAbstract
Network approaches to psychometric constructs, in which constructs are modeled in terms of interactions between their constituent factors, have rapidly gained popularity in psychology. Applications of such network approaches to various psychological constructs have recently moved from a descriptive stance, in which the goal is to estimate the network structure that pertains to a construct, to a more comparative stance, in which the goal is to compare network structures across populations. However, the statistical tools to do so are lacking. In this article, we present the network comparison test (NCT), which uses resampling-based permutation testing to compare network structures from two independent, cross-sectional data sets on invariance of (a) network structure, (b) edge (connection) strength, and (c) global strength. Performance of NCT is evaluated in simulations that show NCT to perform well in various circumstances for all three tests: The Type I error rate is close to the nominal significance level, and power proves sufficiently high if sample size and difference between networks are substantial. We illustrate NCT by comparing depression symptom networks of males and females. Possible extensions of NCT are discussed. (PsycInfo Database Record (c) 2024 APA, all rights reserved).
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
R: A Language and Environment for Statistical Computing
R Core Team (2025) · doi:10.32614/r.manuals · Cited by 352,181
Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482
Network medicine: a network-based approach to human disease
Barabási, A.-L., Gulbahce, N., & Loscalzo, J. (2011) · Nature Reviews Genetics · doi:10.1038/nrg2918 · Cited by 5,034
Tests for comparing elements of a correlation matrix
Steiger, J. H. (1980) · Psychological Bulletin · doi:10.1037/0033-2909.87.2.245 · Cited by 4,621 · cited 2×
“Steiger’s test is about testing dependent correlations (i.e., in the same sample) but between different variables (Steiger, 1980).”
BackgroundNetwork analysis: An integrative approach to the structure of psychopathology
Borsboom, D., & Cramer, A. O. J. (2013) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-050212-185608 · Cited by 4,359
Self-citationBeitrag zur Theorie des Ferromagnetismus
Ising, E. (1925) · Zeitschrift für Physik · doi:10.1007/bf02980577 · Cited by 3,871
Node centrality in weighted networks: Generalizing degree and shortest paths
Opsahl, T., Agneessens, F., & Skvoretz, J. (2010) · Social Networks · doi:10.1016/j.socnet.2010.03.006 · Cited by 3,718
Permutation inference for the general linear model
Winkler, A. M., Ridgway, G. R., Webster, M. A., Smith, S. M., & Nichols, T. E. (2014) · NeuroImage · doi:10.1016/j.neuroimage.2014.01.060 · Cited by 3,563
Cluster failure: why fMRI inferences for spatial extent have inflated false-positive rates
Eklund, A., Nichols, T. E., & Knutsson, H. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1602413113 · Cited by 3,487
A network theory of mental disorders
Borsboom, D. (2017) · World Psychiatry · doi:10.1002/wps.20375 · Cited by 3,482
Self-citationA tutorial on regularized partial correlation networks
Epskamp, S., & Fried, E. I. (2018) · Psychological Methods · doi:10.1037/met0000167 · Cited by 2,918
High dimensional graphs and variable selection with the Lasso
Meinshausen, N., & Bühlmann, P. (2006) · Cited by 2,539
High-dimensional graphs and variable selection with the Lasso
Meinshausen, N., & Bühlmann, P. (2006) · The Annals of Statistics · doi:10.1214/009053606000000281 · Cited by 2,521
Multiple Comparison Procedures
Hochberg, Y., & Tamhane, A. C. (1987) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316672 · Cited by 2,389
Wittchen, H.-U. (1994) · Journal of Psychiatric Research · doi:10.1016/0022-3956(94)90036-1 · Cited by 2,348
The Inventory of Depressive Symptomatology (IDS): Psychometric properties
Rush, A. J., Gullion, C. M., Basco, M. R., Jarrett, R. B., & Trivedi, M. H. (1996) · Psychological Medicine · doi:10.1017/s0033291700035558 · Cited by 2,177
Extended Bayesian information criteria for model selection with large model spaces
Chen, J., & Chen, Z. (2008) · Biometrika · doi:10.1093/biomet/asn034 · Cited by 2,011
Epidemiology of women and depression
Kessler, R. (2003) · Journal of Affective Disorders · doi:10.1016/s0165-0327(02)00426-3 · Cited by 2,006
Sex differences in unipolar depression: Evidence and theory.
Nolen–Hoeksema, S. (1987) · Psychological Bulletin · doi:10.1037//0033-2909.101.2.259 · Cited by 2,001
Comorbidity: a network perspective
Cramer, A. O. J., Waldorp, L. J., van der Maas, H. L. J., & Borsboom, D. (2010) · Behavioral and Brain Sciences · doi:10.1017/s0140525x09991567 · Cited by 1,575
Self-citationSocial Network Analysis: Methods and Applications
Lazega, E., Wasserman, S., & Faust, K. (1995) · Revue Française de Sociologie · doi:10.2307/3322457 · Cited by 1,558
Statistics for High-Dimensional Data: Methods, Theory and Applications
Bhlmann, P., & van de Geer, S. (2011) · Cited by 1,527
Controlling the familywise error rate in functional neuroimaging: a comparative review
Nichols, T., & Hayasaka, S. (2003) · Statistical Methods in Medical Research · doi:10.1191/0962280203sm341ra · Cited by 1,218
The Gaussian Graphical Model in Cross-Sectional and Time-Series Data
Epskamp, S., Waldorp, L. J., Mõttus, R., & Borsboom, D. (2018) · Multivariate Behavioral Research · doi:10.1080/00273171.2018.1454823 · Cited by 1,134
Self-citationState of the aRt personality research: A tutorial on network analysis of personality data in R
Costantini, G., Epskamp, S., Borsboom, D., Perugini, M., Mõttus, R., Waldorp, L. J., & Cramer, A. O. J. (2015) · Journal of Research in Personality · doi:10.1016/j.jrp.2014.07.003 · Cited by 982
Self-citationThe Netherlands Study of Depression and Anxiety (NESDA): rationale, objectives and methods
Penninx, B. W. J. H., Beekman, A. T. F., Smit, J. H., Zitman, F. G., Nolen, W. A., Spinhoven, P., Cuijpers, P., De Jong, P. J., Van Marwijk, H. W. J., Assendelft, W. J. J., et al. (2008) · International Journal of Methods in Psychiatric Research · doi:10.1002/mpr.256 · Cited by 927
Resampling-Based Multiple Testing: Examples and Methods for p-Value Adjustment
Chaubey, Y. P., Westfall, P. H., & Young, S. S. (1993) · Technometrics · doi:10.2307/1270279 · Cited by 920
High-dimensional Ising model selection using ℓ1-regularized logistic regression
Ravikumar, P., Wainwright, M. J., & Lafferty, J. D. (2010) · The Annals of Statistics · doi:10.1214/09-aos691 · Cited by 892
Association of Symptom Network Structure With the Course of Depression
van Borkulo, C., Boschloo, L., Borsboom, D., Penninx, B. W. J. H., Waldorp, L. J., & Schoevers, R. A. (2015) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2015.2079 · Cited by 846
Self-citationPermutation Methods: A Basis for Exact Inference
Ernst, M. D. (2004) · Statistical Science · doi:10.1214/088342304000000396 · Cited by 802
A new method for constructing networks from binary data
van Borkulo, C. D., Borsboom, D., Epskamp, S., Blanken, T. F., Boschloo, L., Schoevers, R. A., & Waldorp, L. J. (2014) · Scientific Reports · doi:10.1038/srep05918 · Cited by 757
Self-citationHigh-dimensional covariance estimation by minimizing ℓ1-penalized log-determinant divergence
Ravikumar, P., Wainwright, M. J., Raskutti, G., & Yu, B. (2011) · Electronic Journal of Statistics · doi:10.1214/11-ejs631 · Cited by 744
Deconstructing the construct: a network perspective on psychological phenomena
Schmittmann, V. D., Cramer, A. O. J., Waldorp, L. J., Epskamp, S., Kievit, R. A., & Borsboom, D. (2013) · New Ideas in Psychology · doi:10.1016/j.newideapsych.2011.02.007 · Cited by 740
Self-citationA Network Approach to Psychopathology: New Insights into Clinical Longitudinal Data
Bringmann, L. F., Vissers, N., Wichers, M., Geschwind, N., Kuppens, P., Peeters, F., Borsboom, D., & Tuerlinckx, F. (2013) · PLoS ONE · doi:10.1371/journal.pone.0060188 · Cited by 680
Self-citationExtended Bayesian Information Criteria for Gaussian Graphical Models
Foygel, R., & Drton, M. (2010) · arXiv (Cornell University) · doi:10.48550/arxiv.1011.6640 · Cited by 659
Mental Disorders as Causal Systems: A Network Approach to Posttraumatic Stress Disorder
McNally, R. J., Robinaugh, D. J., Wu, G. W. Y., Wang, L., Deserno, M. K., & Borsboom, D. (2015) · Clinical Psychological Science · doi:10.1177/2167702614553230 · Cited by 592
Self-citationValidity and Reliability of the Experience-Sampling Method
Csikszentmihalyi, M., & Larson, R. (2014) · Flow and the Foundations of Positive Psychology · doi:10.1007/978-94-017-9088-8_3 · Cited by 578
The Comparison of Regression Variables
Williams, E. J. (1959) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1959.tb00346.x · Cited by 520
Multiple Testing Procedures with Applications to Genomics
Dudoit, S., & van der Laan, M. J. (2008) · Springer Series in Statistics · doi:10.1007/978-0-387-49317-6 · Cited by 441
Permutation, Parametric, and Bootstrap Tests of Hypotheses
(2005) · Technometrics · doi:10.1198/tech.2005.s333 · Cited by 412
Ledoit, O., & Wolf, M. (2002) · The Annals of Statistics · doi:10.1214/aos/1031689018 · Cited by 399
aan het Rot, M., Hogenelst, K., & Schoevers, R. A. (2012) · Clinical Psychology Review · doi:10.1016/j.cpr.2012.05.007 · Cited by 325
Self-citationEstimating psychopathological networks: Be careful what you wish for
Epskamp, S., Kruis, J., & Marsman, M. (2017) · PLOS ONE · doi:10.1371/journal.pone.0179891 · Cited by 315
From loss to loneliness: The relationship between bereavement and depressive symptoms.
Fried, E. I., Bockting, C., Arjadi, R., Borsboom, D., Amshoff, M., Cramer, A. O. J., Epskamp, S., Tuerlinckx, F., Carr, D., & Stroebe, M. (2015) · Journal of Abnormal Psychology · doi:10.1037/abn0000028 · Cited by 314
Self-citationThe Network Structure of Symptoms of the Diagnostic and Statistical Manual of Mental Disorders
Boschloo, L., van Borkulo, C. D., Rhemtulla, M., Keyes, K. M., Borsboom, D., & Schoevers, R. A. (2015) · PLOS ONE · doi:10.1371/journal.pone.0137621 · Cited by 285
Self-citationSchuch, J. J. J., Roest, A. M., Nolen, W. A., Penninx, B. W. J. H., & de Jonge, P. (2014) · Journal of Affective Disorders · doi:10.1016/j.jad.2013.12.011 · Cited by 280
Emotion-Network Density in Major Depressive Disorder
Pe, M. L., Kircanski, K., Thompson, R. J., Bringmann, L. F., Tuerlinckx, F., Mestdagh, M., Mata, J., Jaeggi, S. M., Buschkuehl, M., Jonides, J., et al. (2015) · Clinical Psychological Science · doi:10.1177/2167702614540645 · Cited by 261
Network analysis of persistent complex bereavement disorder in conjugally bereaved adults.
Robinaugh, D. J., LeBlanc, N. J., Vuletich, H. A., & McNally, R. J. (2014) · Journal of Abnormal Psychology · doi:10.1037/abn0000002 · Cited by 219 · cited 1×
“…has since been applied to many fields, such as psychopathology, intelligence, attitudes, personality, and health sciences (Boschloo et al., 2015; Costantini et al., 2015; Dalege et al., 2016; Kossakowski et al., 2016; Robinaugh et al., 2014; Robinaugh & McNally, 2011; Van Der Maas et al., 2017).”
Multiple Testing for Exploratory Research
Goeman, J. J., & Solari, A. (2011) · Statistical Science · doi:10.1214/11-sts356 · Cited by 205
High-dimensional Ising model selection with Bayesian information criteria
Barber, R. F., & Drton, M. (2015) · Electronic Journal of Statistics · doi:10.1214/15-ejs1012 · Cited by 175
Trauma centrality and PTSD symptom severity in adult survivors of childhood sexual abuse
Robinaugh, D. J., & McNally, R. J. (2011) · Journal of Traumatic Stress · doi:10.1002/jts.20656 · Cited by 175
Network Models for Cognitive Development and Intelligence
van der Maas, H. L. J., Kan, K., Marsman, M., & Stevenson, C. E. (2017) · Journal of Intelligence · doi:10.3390/jintelligence5020016 · Cited by 172
Multiple Testing and Error Control in Gaussian Graphical Model Selection
Drton, M., & Perlman, M. D. (2007) · Statistical Science · doi:10.1214/088342307000000113 · Cited by 171
Wigman, J. T. W., van Os, J., Borsboom, D., Wardenaar, K. J., Epskamp, S., Klippel, A., MERGE, Viechtbauer, W., Myin-Germeys, I., & Wichers, M. (2015) · Psychological Medicine · doi:10.1017/s0033291715000331 · Cited by 170
Self-citationOn the distribution of penalized maximum likelihood estimators: The LASSO, SCAD, and thresholding
Pötscher, B. M., & Leeb, H. (2009) · Journal of Multivariate Analysis · doi:10.1016/j.jmva.2009.06.010 · Cited by 166
Dutilleul, P., Stockwell, J. D., Frigon, D., & Legendre, P. (2000) · Journal of Agricultural, Biological, and Environmental Statistics · doi:10.2307/1400528 · Cited by 158
Kossakowski, J. J., Epskamp, S., Kieffer, J. M., van Borkulo, C. D., Rhemtulla, M., & Borsboom, D. (2015) · Quality of Life Research · doi:10.1007/s11136-015-1127-z · Cited by 154
Self-citationAlcohol use disorders and the course of depressive and anxiety disorders
Boschloo, L., Vogelzangs, N., van den Brink, W., Smit, J. H., Veltman, D. J., Beekman, A. T. F., & Penninx, B. W. J. H. (2012) · The British Journal of Psychiatry · doi:10.1192/bjp.bp.111.097550 · Cited by 131
Self-citationNetworks and Nosology in Posttraumatic Stress Disorder
McNally, R. J. (2016) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2016.3344 · Cited by 65
The Four-Year Course of Major Depressive Disorder: The Role of Staging and Risk Factor Determination
Boschloo, L., Schoevers, R. A., Beekman, A. T. F., Smit, J. H., van Hemert, A. M., & Penninx, B. W. J. H. (2014) · Psychotherapy and Psychosomatics · doi:10.1159/000362563 · Cited by 60
Self-citationCaner, M., & Kock, A. (2017) · Journal of Econometrics · doi:10.1016/j.jeconom.2017.11.005 · Cited by 57
A Network Approach to Bipolar Symptomatology in Patients with Different Course Types
Koenders, M. A., de Kleijn, R., Giltay, E. J., Elzinga, B. M., Spinhoven, P., & Spijker, A. T. (2015) · PLoS ONE · doi:10.1371/journal.pone.0141420 · Cited by 46
Cited by
Robinaugh, D. J., Hoekstra, R. H. A., Toner, E. R., & Borsboom, D. (2020) · Psychological Medicine · doi:10.1017/s0033291719003404 · Cited by 861
Self-citationFried, E. I., Eidhof, M. B., Palic, S., Costantini, G., Huisman-van Dijk, H. M., Bockting, C. L. H., Engelhard, I., Armour, C., Nielsen, A. B. S., & Karstoft, K.-I. (2018) · Clinical Psychological Science · doi:10.1177/2167702617745092 · Cited by 454
Galderisi, S., Rucci, P., Kirkpatrick, B., Mucci, A., Gibertoni, D., Rocca, P., Rossi, A., Bertolino, A., Strauss, G. P., Aguglia, E., et al. (2018) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2017.4607 · Cited by 312
Visualizing Psychological Networks: A Tutorial in R
Jones, P. J., Mair, P., & McNally, R. J. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.01742 · Cited by 270
Psychological distress associated with the COVID-19 lockdown: A two-wave network analysis
Di Blasi, M., Gullo, S., Mancinelli, E., Freda, M. F., Esposito, G., Gelo, O. C. G., Lagetto, G., Giordano, C., Mazzeschi, C., Pazzagli, C., et al. (2021) · Journal of Affective Disorders · doi:10.1016/j.jad.2021.02.016 · Cited by 143
Metadata
- Authors
- van Borkulo, C. D., van Bork, R., Boschloo, L., Kossakowski, J. J., Tio, P., Schoevers, R. A., Borsboom, D., & Waldorp, L. J.
- Venue
- Psychological Methods
- Published
- 2023-12
- Volume
- 28
- Issue
- 6
- Pages
- 1273-1285
- ISSN
- 1082-989X (ISSN-L)
- 1939-1463
- OpenAlex
- W4225598392
- Cited by
- 1,465
- References
- 62
- FWCI
- 208.11
- Type
- journal article
- Language
- en
- Open access
- green
- Funders
- Netherlands Organisation for Scientific Research (VI.Veni.201G.074)
- ERC (647209)
- Netherlands Organisation for Health Research and Development (016-186-139)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.