Student or educator? Sign up with your school email and get 6 months of Pro free.

The DOI matches its Crossref record; 76 of the 78 references it cites were found in a registry or an index.

Estimating psychological networks and their accuracy: A tutorial paper

Open access
Epskamp, Sacha, Borsboom, Denny, Fried, Eiko I. · 2018 · Behavior Research Methods

Published version of a preprint

Abstract

The usage of psychological networks that conceptualize behavior as a complex interplay of psychological and other components has gained increasing popularity in various research fields. While prior publications have tackled the topics of estimating and interpreting such networks, little work has been conducted to check how accurate (i.e., prone to sampling variation) networks are estimated, and how stable (i.e., interpretation remains similar with less observations) inferences from the network structure (such as centrality indices) are. In this tutorial paper, we aim to introduce the reader to this field and tackle the problem of accuracy under sampling variation. We first introduce the current state-of-the-art of network estimation. Second, we provide a rationale why researchers should investigate the accuracy of psychological networks. Third, we describe how bootstrap routines can be used to (A) assess the accuracy of estimated network connections, (B) investigate the stability of centrality indices, and (C) test whether network connections and centrality estimates for different variables differ from each other. We introduce two novel statistical methods: for (B) the correlation stability coefficient, and for (C) the bootstrapped difference test for edge-weights and centrality indices. We conducted and present simulation studies to assess the performance of both methods. Finally, we developed the free R-package bootnet that allows for estimating psychological networks in a generalized framework in addition to the proposed bootstrap methods. We showcase bootnet in a tutorial, accompanied by R syntax, in which we analyze a dataset of 359 women with posttraumatic stress disorder available online.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3758/s13428-017-0862-1

Cites

76 of 76 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1 Simulated network structure (left panel) and the importance of each node quantified in centrality indices (right panel). The simulated network is a chain network in which each edge has the same absolute strength. The network model used was a Gaussian graphical model in which each edge represents partial correlation coefficients between two variables after conditioning on all other variables
Fig. 1 Simulated network structure (left panel) and the importance of each node quantified in centrality indices (right panel). The simulated network is a chain network in which each edge has the same absolute strength. The network model used was a Gaussian graphical model in which each edge represents partial correlation coefficients between two variables after conditioning on all other variablesCC BY · PubMed Central
Fig. 2 Estimated network structure based on a sample of 500 people simulated using the true model shown in Fig. 1 (left panel) and computed centrality indices (right panel). Centrality indices are shown as standardized z-scores. Centrality indices show that nodes B and C are the most important nodes, even though the true model does not differentiate in importance between nodes
Fig. 2 Estimated network structure based on a sample of 500 people simulated using the true model shown in Fig. 1 (left panel) and computed centrality indices (right panel). Centrality indices are shown as standardized z-scores. Centrality indices show that nodes B and C are the most important nodes, even though the true model does not differentiate in importance between nodesCC BY · PubMed Central
Fig. 3 Estimated network structure of 17 PTSD symptoms (left panel) and the corresponding centrality indices (right panel). Centrality indices are shown as standardized z-scores. The network structure is a Gaussian graphical model, which is a network of partial correlation coefficients
Fig. 3 Estimated network structure of 17 PTSD symptoms (left panel) and the corresponding centrality indices (right panel). Centrality indices are shown as standardized z-scores. The network structure is a Gaussian graphical model, which is a network of partial correlation coefficientsCC BY · PubMed Central
Fig. 4 Bootstrapped confidence intervals of estimated edge-weights for the estimated network of 17 PTSD symptoms. The red line indicates the sample values and the gray area the bootstrapped CIs. Each horizontal line represents one edge of the network, ordered from the edge with the highest edge-weight to the edge with the lowest edge-weight. In the case of ties (for instance, multiple edge-weights were estimated to be exactly 0), the mean of the bootstrap samples was used in ordering the edges. The y-axis labels have been removed to avoid cluttering
Fig. 4 Bootstrapped confidence intervals of estimated edge-weights for the estimated network of 17 PTSD symptoms. The red line indicates the sample values and the gray area the bootstrapped CIs. Each horizontal line represents one edge of the network, ordered from the edge with the highest edge-weight to the edge with the lowest edge-weight. In the case of ties (for instance, multiple edge-weights were estimated to be exactly 0), the mean of the bootstrap samples was used in ordering the edges. The y-axis labels have been removed to avoid clutteringCC BY · PubMed Central
Fig. 5 Average correlations between centrality indices of networks sampled with persons dropped and the original sample. Lines indicate the means and areas indicate the range from the 2.5th quantile to the 97.5th quantile
Fig. 5 Average correlations between centrality indices of networks sampled with persons dropped and the original sample. Lines indicate the means and areas indicate the range from the 2.5th quantile to the 97.5th quantileCC BY · PubMed Central
Fig. 6 Bootstrapped difference tests (α = 0.05) between edge-weights that were non-zero in the estimated network (above) and node strength of the 17 PTSD symptoms (below). Gray boxes indicate nodes or edges that do not differ significantly from one-another and black boxes represent nodes or edges that do differ significantly from one-another. Colored boxes in the edge-weight plot correspond to the color of the edge in Fig. 3, and white boxes in the centrality plot show the value of node strength
Fig. 6 Bootstrapped difference tests (α = 0.05) between edge-weights that were non-zero in the estimated network (above) and node strength of the 17 PTSD symptoms (below). Gray boxes indicate nodes or edges that do not differ significantly from one-another and black boxes represent nodes or edges that do differ significantly from one-another. Colored boxes in the edge-weight plot correspond to the color of the edge in Fig. 3, and white boxes in the centrality plot show the value of node strengthCC BY · PubMed Central
Fig. 7 Simulation results showing the CS-coefficient of 24,000 simulated datasets. Datasets were generated using chain networks (partial correlations) of 10 nodes with edge-weights set to 0.25 or −0.25. Edges were randomly rewired to obtain a range from networks ranging from networks in which all centralities are equal to networks in which all centralities differ. The CS-coefficient quantifies the maximum proportion of cases that can be dropped at random to retain, with 95 % certainty, a correlation of at least 0.7 with the centralities of the original network. Boxplots show the distribution of CS-coefficients obtained in the simulations. For example, plots on top indicate that the CS-coefficient mostly stays below 0.25 when centralities do not differ from one-another (chain graph as shown in Fig. 1)
Fig. 7 Simulation results showing the CS-coefficient of 24,000 simulated datasets. Datasets were generated using chain networks (partial correlations) of 10 nodes with edge-weights set to 0.25 or −0.25. Edges were randomly rewired to obtain a range from networks ranging from networks in which all centralities are equal to networks in which all centralities differ. The CS-coefficient quantifies the maximum proportion of cases that can be dropped at random to retain, with 95 % certainty, a correlation of at least 0.7 with the centralities of the original network. Boxplots show the distribution of CS-coefficients obtained in the simulations. For example, plots on top indicate that the CS-coefficient mostly stays below 0.25 when centralities do not differ from one-another (chain graph as shown in Fig. 1)CC BY · PubMed Central
Fig. 8 Simulation results showing the rejection rate of the bootstrapped difference test for edge-weights on 6,000 simulated datasets. Datasets were generated using chain networks (partial correlations) of ten nodes with edge-weights set to 0.3. Only networks that were nonzero in the true network were compared to one-another. Lines indicate the proportion of times that two random edge-weights were significantly different (i.e., the null-hypothesis was rejected) and their CI (plus and minus 1.96 times the standard error). Solid horizontal lines indicate the intended significance level and horizontal dashed line the expected significance level. The y-axis is drawn using a logarithmic scale
Fig. 8 Simulation results showing the rejection rate of the bootstrapped difference test for edge-weights on 6,000 simulated datasets. Datasets were generated using chain networks (partial correlations) of ten nodes with edge-weights set to 0.3. Only networks that were nonzero in the true network were compared to one-another. Lines indicate the proportion of times that two random edge-weights were significantly different (i.e., the null-hypothesis was rejected) and their CI (plus and minus 1.96 times the standard error). Solid horizontal lines indicate the intended significance level and horizontal dashed line the expected significance level. The y-axis is drawn using a logarithmic scaleCC BY · PubMed Central
Fig. 9 Simulation results showing the rejection rate of the bootstrapped difference test for centrality indices. Datasets were generated using the same design as in Fig. 7. Lines indicate the proportion of times that two random centralities were significantly different (i.e., the null-hypothesis was rejected at α = 0.05)
Fig. 9 Simulation results showing the rejection rate of the bootstrapped difference test for centrality indices. Datasets were generated using the same design as in Fig. 7. Lines indicate the proportion of times that two random centralities were significantly different (i.e., the null-hypothesis was rejected at α = 0.05)CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 78 references

Research works printed but not linked: 1

  1. Gumbel, E.J. (1939). La probabilité des hypotheses. Comptes Rendus de l’Academie des Sciences (Paris), 209, 645–647.

Cited by

See all 5,715

Metadata

Authors
Epskamp, S., Borsboom, D., & Fried, E. I.
Affiliations
University of Amsterdam (NL)
Corresponding author
Epskamp, Sacha
Venue
Behavior Research Methods
Published
2017-03-24
Volume
50
Issue
1
Pages
195-212
ISSN
  • 1554-351X (ISSN-L)
  • 1554-3528
OpenAlex
W2530237102
Cited by
5,715
References
78
Citations by year
2026: 1331 · 2025: 1249 · 2024: 855 · 2023: 674 · 2022: 509 · 2021: 451 · 2020: 287 · 2019: 181 · 2018: 111 · 2017: 48
FWCI
220.43
Type
journal article
Language
en
Open access
Locations
  • Behavior Research Methods (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • UvA-DARE (University of Amsterdam) (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion)
  • Leiden Repository (Leiden University) (submittedVersion)
  • Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
  • arXiv (Cornell University)
Funders
NWO "research talent" (406-11-066)
Topics
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Mental Health Research Topics
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Functional Brain Connectivity Studies
Keywords
psychological networks, network estimation, sampling variability, bootstrapping, centrality indices, edge weights, post-traumatic stress disorder, network stability
MeSH terms
Female, Humans, Psychophysiology, Stress Disorders, Post-Traumatic, Neural Networks, Computer, Dimensional Measurement Accuracy

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Estimating psychological networks and their accuracy: A tutorial paper. https://app.cochat.ai/papers/10.3758/s13428-017-0862-1
The paper
Epskamp, S., Borsboom, D., & Fried, E. I. (2018). Estimating psychological networks and their accuracy: A tutorial paper. Behavior Research Methods. https://doi.org/10.3758/s13428-017-0862-1

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.