The DOI matches its Crossref record; 62 of the 62 references it cites were found in a registry or an index.
Comparing network structures on three aspects: A permutation test.
Open accessAbstract
Network approaches to psychometric constructs, in which constructs are modeled in terms of interactions between their constituent factors, have rapidly gained popularity in psychology. Applications of such network approaches to various psychological constructs have recently moved from a descriptive stance, in which the goal is to estimate the network structure that pertains to a construct, to a more comparative stance, in which the goal is to compare network structures across populations. However, the statistical tools to do so are lacking. In this article, we present the network comparison test (NCT), which uses resampling-based permutation testing to compare network structures from two independent, cross-sectional data sets on invariance of (a) network structure, (b) edge (connection) strength, and (c) global strength. Performance of NCT is evaluated in simulations that show NCT to perform well in various circumstances for all three tests: The Type I error rate is close to the nominal significance level, and power proves sufficiently high if sample size and difference between networks are substantial. We illustrate NCT by comparing depression symptom networks of males and females. Possible extensions of NCT are discussed. (PsycInfo Database Record (c) 2024 APA, all rights reserved).
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
R: A Language and Environment for Statistical Computing
R Core Team (2025) · doi:10.32614/r.manuals · Cited by 352,181
Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482
Network medicine: a network-based approach to human disease
Barabási, A.-L., Gulbahce, N., & Loscalzo, J. (2011) · Nature Reviews Genetics · doi:10.1038/nrg2918 · Cited by 5,034 · cited 1×
“…structures on three aspects: A permutation test In the past decades, network analysis has rapidly gained popularity as a method of representing complex relations in large data sets, and has been applied in many different fields, from physics and engineering to medicine and biology (Barabasi, 2011).”
BackgroundTests for comparing elements of a correlation matrix
Steiger, J. H. (1980) · Psychological Bulletin · doi:10.1037/0033-2909.87.2.245 · Cited by 4,621 · cited 2×
“Steiger’s test is about testing dependent correlations (i.e., in the same sample) but between different variables (Steiger, 1980).”
BackgroundNetwork analysis: An integrative approach to the structure of psychopathology
Borsboom, D., & Cramer, A. O. J. (2013) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-050212-185608 · Cited by 4,359
Self-citationBeitrag zur Theorie des Ferromagnetismus
Ising, E. (1925) · Zeitschrift für Physik · doi:10.1007/bf02980577 · Cited by 3,871
Node centrality in weighted networks: Generalizing degree and shortest paths
Opsahl, T., Agneessens, F., & Skvoretz, J. (2010) · Social Networks · doi:10.1016/j.socnet.2010.03.006 · Cited by 3,718
Permutation inference for the general linear model
Winkler, A. M., Ridgway, G. R., Webster, M. A., Smith, S. M., & Nichols, T. E. (2014) · NeuroImage · doi:10.1016/j.neuroimage.2014.01.060 · Cited by 3,563
Cluster failure: why fMRI inferences for spatial extent have inflated false-positive rates
Eklund, A., Nichols, T. E., & Knutsson, H. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1602413113 · Cited by 3,487 · cited 1×
“Permutation tests based on resampling are increasingly used in these situations to control the multiple testing problem and accompanying familywise error rate (FWER; e.g., as in functional neuroimaging; Dudoit & Laan, 2007; Eklund et al., 2016; Nichols & Hayasaka, 2003; Winkler et al., 2014).”
MethodA network theory of mental disorders
Borsboom, D. (2017) · World Psychiatry · doi:10.1002/wps.20375 · Cited by 3,482 · cited 1×
“Network modeling has been proposed as an alternative to latent variable or common cause modeling of psychological test scores (Borsboom, 2017; Borsboom & Cramer, 2013; Cramer et al., 2010; Richard J. McNally, 2017; Schmittmann et al., 2011).”
MethodSelf-citation
Cited by
Robinaugh, D. J., Hoekstra, R. H. A., Toner, E. R., & Borsboom, D. (2020) · Psychological Medicine · doi:10.1017/s0033291719003404 · Cited by 861
Self-citationFried, E. I., Eidhof, M. B., Palic, S., Costantini, G., Huisman-van Dijk, H. M., Bockting, C. L. H., Engelhard, I., Armour, C., Nielsen, A. B. S., & Karstoft, K.-I. (2018) · Clinical Psychological Science · doi:10.1177/2167702617745092 · Cited by 454
Galderisi, S., Rucci, P., Kirkpatrick, B., Mucci, A., Gibertoni, D., Rocca, P., Rossi, A., Bertolino, A., Strauss, G. P., Aguglia, E., et al. (2018) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2017.4607 · Cited by 312
Visualizing Psychological Networks: A Tutorial in R
Jones, P. J., Mair, P., & McNally, R. J. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.01742 · Cited by 270
Psychological distress associated with the COVID-19 lockdown: A two-wave network analysis
Di Blasi, M., Gullo, S., Mancinelli, E., Freda, M. F., Esposito, G., Gelo, O. C. G., Lagetto, G., Giordano, C., Mazzeschi, C., Pazzagli, C., et al. (2021) · Journal of Affective Disorders · doi:10.1016/j.jad.2021.02.016 · Cited by 143
Djelantik, A. A. A. M. J., Robinaugh, D. J., Kleber, R. J., Smid, G. E., & Boelen, P. A. (2020) · Depression and Anxiety · doi:10.1002/da.22880 · Cited by 110
“First, we examined whether the structure of the network differed as a function of violent loss using the package NetworkComparisonTest (v. 2.0.1) (van Borkulo et al., 2018).”
MethodResultInfluentialSmith, K. E., Mason, T. B., Crosby, R. D., Cao, L., Leonard, R. C., Wetterneck, C. T., Smith, B. E. R., Farrell, N. R., Riemann, B. C., Wonderlich, S. A., et al. (2019) · Psychological Medicine · doi:10.1017/s0033291718000867 · Cited by 101
Bai, W., Cai, H., Liu, S., Chen, X., Sha, S., Cheung, T., Lin, J. J., Cui, X., Ng, C. H., & Xiang, Y.-T. (2021) · Translational Psychiatry · doi:10.1038/s41398-021-01738-4 · Cited by 89
Gámez-Guadix, M., Sorrel, M. A., & Martínez-Bacaicoa, J. (2023) · Sexuality Research and Social Policy · doi:10.1007/s13178-022-00775-y · Cited by 65
Ward, J. H., Weir, E., Allison, C., & Baron-Cohen, S. (2023) · Molecular Autism · doi:10.1186/s13229-023-00565-2 · Cited by 60
Inflammatory phenotype of depression symptom structure: A network perspective
Moriarity, D. P., van Borkulo, C., & Alloy, L. B. (2021) · Brain Behavior and Immunity · doi:10.1016/j.bbi.2020.12.005 · Cited by 45
Buelens, T., Costantini, G., Luyckx, K., & Claes, L. (2020) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2020.580922 · Cited by 45
Zeiler, M., Philipp, J., Truttmann, S., Waldherr, K., Wagner, G., & Karwautz, A. (2021) · Nutrients · doi:10.3390/nu13114096 · Cited by 41
“Moreover, we performed the Network Comparison Test (NCT, [48]) using the ‘NetworkComparisonTest’ package in R to compare the network structure and the global network strength between the overweight and underweight samples.”
MethodThe central role of self-esteem in the quality of life of patients with mental disorders
Barbalat, G., Plasse, J., Gauthier, E., Verdoux, H., Quiles, C., Dubreucq, J., Legros-Lafarge, E., Jaafari, N., Massoubre, C., Guillard-Bouhet, N., et al. (2022) · Scientific Reports · doi:10.1038/s41598-022-11655-1 · Cited by 42
Cervin, M., Perrin, S., Olsson, E., Aspvall, K., Geller, D. A., Wilhelm, S., McGuire, J., Lázaro, L., Martínez-González, A. E., Barcaccia, B., et al. (2020) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1016/j.jaac.2019.06.018 · Cited by 37
Anxiety and Depression Symptoms among Youth Survivors of Childhood Sexual Abuse: A Network Analysis
Li, J., Jin, Y., Xu, S., Luo, X., Wilson, A., Li, H., Wang, X., Sun, X., & Wang, Y. (2023) · BMC Psychology · doi:10.1186/s40359-023-01275-3 · Cited by 29
“The centrality of each node was identified according to the expected influence (EI), which represents the whole edges of the weight connecting to this code [53].”
BackgroundMethodA Network Analysis Approach on the Psychological Flexibility/Inflexibility Model
Christodoulou, A., Karekla, M., Costantini, G., & Michaelides, M. P. (2023) · Behavior Therapy · doi:10.1016/j.beth.2023.01.002 · Cited by 22
Examining psychological flexibility in unaccompanied refugee minors: A network analysis
Freymann, J., Morroni, D., Kleinbub, J. R., & Karekla, M. (2024) · Journal of Contextual Behavioral Science · doi:10.1016/j.jcbs.2024.100808 · Cited by 7
Delli Colli, C., Viglione, A., Poggini, S., Cirulli, F., Chiarotti, F., Giuliani, A., & Branchi, I. (2025) · Translational Psychiatry · doi:10.1038/s41398-025-03246-1 · Cited by 7
“The Network Comparison Test (NTC) is a permutation-based test that randomly regroups participants from the network repeatedly [19, 20].”
Network modeling of problematic social media use components in college student social media users
Chen, J., Su, T., Dong, J., Li, Y., Feng, J., Chen, Y., & Liu, G. (2025) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2024.1386845 · Cited by 7
Wu, L., Liu, C., Huang, P., Wang, Z., Cai, M., Fang, P., Liang, W., Sun, K., Tang, X., Ouyang, A., et al. (2025) · BMC Psychiatry · doi:10.1186/s12888-025-07091-w · Cited by 1
Metadata
- Authors
- van Borkulo, C. D., van Bork, R., Boschloo, L., Kossakowski, J. J., Tio, P., Schoevers, R. A., Borsboom, D., & Waldorp, L. J.
- Venue
- Psychological Methods
- Published
- 2023-12
- Volume
- 28
- Issue
- 6
- Pages
- 1273-1285
- ISSN
- 1082-989X (ISSN-L)
- 1939-1463
- OpenAlex
- W4225598392
- Cited by
- 1,465
- References
- 62
- FWCI
- 208.11
- Type
- journal article
- Language
- en
- Open access
- green
- Funders
- Netherlands Organisation for Scientific Research (VI.Veni.201G.074)
- ERC (647209)
- Netherlands Organisation for Health Research and Development (016-186-139)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.