Cited by 147Mental Health Research TopicsPsychometric Methodologies and TestingAnxiety, Depression, Psychometrics, Treatment, Cognitive Processes

Investigating the feasibility of idiographic network models.

Open access
Mansueto, Alessandra C., Wiers, Reinout W., van Weert, Julia C. M., Schouten, Barbara C., Epskamp, Sacha·2023·Psychological Methods

Abstract

Recent times have seen a call for personalized psychotherapy and tailored communication during treatment, leading to the necessity to model the complex dynamics of mental disorders in a single subject. To this aim, time-series data in one patient can be collected through ecological momentary assessment and analyzed with the graphical vector autoregressive model, estimating temporal and contemporaneous idiographic networks. Idiographic networks graph interindividual processes that may be potentially used to tailor psychotherapy and provide personalized feedback to clients and are regarded as a promising tool for clinical practice. However, the question whether we can reliably estimate them in clinical settings remains unanswered. We conducted a large-scale simulation study in the context of psychopathology, testing the performance of personalized networks with different numbers of time points, percentages of missing data, and estimation methods. Results indicate that sensitivity is low with sample sizes feasible for clinical practice (75 and 100 time points). It seems possible to retrieve the global network structure but not to recover the full network. Estimating temporal networks appears particularly challenging; thus, with 75 and 100 observations, it is advisable to reduce the number of nodes to around six variables. With regard to missing data, full information maximum likelihood and the Kalman filter are effective in addressing random item-level missing data; consequently, planned missingness is a valid method to deal with missing data. We discuss possible methodological and clinical solutions to the challenges raised in this work. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1037/met0000466

Cites

0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • A minimum of 75 EMA questionnaires per person is needed for a reliable network with approximately 6 nodes, based on a simulation study.

    “To be included in the present analyses, participants needed to fill out a minimum of 75 EMA questionnaires, as a recent simulation study showed that with 75-100 assessments per person the network can include approximately 6 nodes for it to be reliable.27”

    From Exploring the dynamic interconnectedness of protective and perpetuating factors of cancer-related fatigue (2023) · cites this paper as [27]

    The Summary and Methodological Recommendations (p. 12) say six nodes are reasonable for recovering the global temporal structure with 75–100 data points. But the Simulation Results (p. 9) report low sensitivity at those sample sizes, so the paper does not establish that a six-node network as a whole is reliable with a minimum of 75 questionnaires. (LLM, conf=0.88)

    Experimental — accuracy not yet measured
  • With an average of 88 assessments per person, the number of included nodes was limited to 7 nodes for the network to be reliable.

    “Second, with an average of 88 assessments per person, the number of included nodes was limited to 7 nodes for the network to be reliable.27”

    From Exploring the dynamic interconnectedness of protective and perpetuating factors of cancer-related fatigue (2023) · cites this paper as [27]

    The Summary and Methodological Recommendations (pp. 11–12) advise reducing the number of nodes and say six nodes seem reasonable for recovering the global temporal structure with 75–100 data points. They do not establish a seven-node limit for 88 assessments or say that such a limit makes the full network reliable; p. 11 notes that the full network may not be recovered. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

The publisher hasn’t deposited this paper’s reference list.

Research works printed but not linked: 25

  1. Collins, L. M., Schafer, J. L., & Kam, C.-M. (2001). A comparison of inclusive and restrictive strategies in modern missing data procedures. Psychological Methods , 6 (4), 330 -351. https://doi.org/10.1037/1082 -989X.6.4.330 · DOI
  2. Cramer, A. O. J., & Borsboom, D. (2015). Problems attract problems: A network perspective on mental disorders. Emerging Trends in the Social and Behavioral Sciences . Advance online publication. https://doi.org/10 .1002/9781118900772.etrds0264
  3. Enders, C. K. (2001). A primer on maximum likelihood algorithms available for use with missing data. Structural Equation Modeling , 8 (1), 128 -141. https://doi.org/10.1207/S15328007SEM0801\_7 · DOI
  4. Enders, C., & Bandalos, D. (2001). The relative performance of full information maximum likelihood estimation for missing data in structural equation models. Structural Equation Modeling , 8 (3), 430 -457. https:// doi.org/10.1207/S15328007SEM0803\_5 · DOI
  5. Epskamp, S. (2020b). Psychometric network models from time-series and panel data. Psychometrika , 85 (1), 206 -231. https://doi.org/10.1007/ s11336-020-09697-3
  6. Epskamp, S., Waldorp, L. J., Mõttus, R., & Borsboom, D. (2018). The Gaussian graphical model in cross-sectional and time-series data. Multivariate Behavioral Research , 53 (4), 453 -480. https://doi.org/10.1080/ 00273171.2018.1454823
  7. Foygel, R., & Drton, M. (2010). Extended Bayesian information criteria for Gaussian graphical models. In J. D. Lafferty, C. K. I. Williams, J. Shawe-Taylor, R. S. Zemel, & A. Culotta (Eds.), Advances in neural information processing systems (Vol. 23, pp. 604 -612). Curran Associates, Inc.
  8. Graham, J. W. (2009). Missing data analysis: Making it work in the real world. Annual Review of Psychology , 60 (1), 549 -576. https://doi.org/10 .1146/annurev.psych.58.110405.085530
  9. Graham, J. W., Taylor, B. J., & Cumsille, P. E. (2001). Planned missingdata designs in analysis of change. In L. M. Collins & A. G. Sayer (Eds.), Decade of behavior. New methods for the analysis of change (pp. 335 -353). American Psychological Association. https://doi.org/10.1037/ 10409-011
  10. Harvey, A. C. (1989). Forecasting, structural time series models and the Kalman filter . Cambridge University Press.
  11. Howard, K. I., Moras, K., Brill, P. L., Martinovich, Z., & Lutz, W. (1996). Evaluation of psychotherapy. Efficacy, effectiveness, and patient progress. American Psychologist , 51 (10), 1059 -1064. https://doi.org/10 .1037/0003-066X.51.10.1059
  12. Iorfino, F., Cross, S. P., Davenport, T., Carpenter, J. S., Scott, E., Shiran, S., & Hickie, I. B. (2019). A digital platform designed for youth mental health services to deliver personalized and measurement-based care. Frontiers in Psychiatry , 10 , Article 595. https://doi.org/10.3389/fpsyt .2019.00595 · DOI
  13. Larsen, R. (2011). Missing data imputation versus full information maximum likelihood with second-level dependencies. Structural Equation Modeling , 18 (4), 649 -662. https://doi.org/10.1080/10705511.2011 .607721 · DOI
  14. McLean, D. C., Nakamura, J., & Csikszentmihalyi, M. (2017). Explaining system missing: Missing data and experience sampling method. Social Psychological & Personality Science , 8 (4), 434 -441. https://doi.org/10 .1177/1948550617708015
  15. Molenaar, P. C. M. (2004). A manifesto on psychology as idiographic science: Bringing the person back into scientific psychology, this time forever. Measurement: Interdisciplinary Research and Perspectives , 2 (4), 201 -218. https://doi.org/10.1207/s15366359mea0204\_1 · DOI
  16. Moritz, S., & Bartz-Beielstein, T. (2017). ImputeTS: Time series missing value imputation in R. The R Journal , 9 (1), 207 -218. https://doi.org/10 .32614/RJ-2017-009
  17. Neath, A. A., & Cavanaugh, J. E. (2012). The Bayesian information criterion: Background, derivation, and applications. Wiley Interdisciplinary
  18. Patrick, K., Hekler, E. B., Estrin, D., Mohr, D. C., Riper, H., Crane, D., Godino, J., & Riley, W. T. (2016). The pace of technologic change: Implications for digital health behavior intervention research. American Journal of Preventive Medicine , 51 (5), 816 -824. https://doi.org/10 .1016/j.amepre.2016.05.001
  19. Pomerleau, O. F., Pomerleau, C. S., & Namenek, R. J. (1998). Early experiences with tobacco among women smokers, ex-smokers, and never-smokers. Addiction , 93 (4), 595 -599. https://doi.org/10.1046/j.1360-0443.1998 .93459515.x · DOI
  20. Pomerleau, C. S., Pomerleau, O. F., Namenek, R. J., & Marks, J. L. (1999). Initial exposure to nicotine in college-age women smokers and never-smokers: A replication and extension. Journal of Addictive Diseases , 18 (3), 13 -19. https://doi.org/10.1300/J069v18n03\_02 · DOI
  21. Schafer, J. L., & Graham, J. W. (2002). Missing data: Our view of the state of the art. Psychological Methods , 7 (2), 147 -177. https://doi.org/10 .1037/1082-989X.7.2.147
  22. Trull, T. J., & Ebner-Priemer, U. (2013). Ambulatory assessment. Annual Review of Clinical Psychology , 9 (1), 151 -176. https://doi.org/10.1146/ annurev-clinpsy-050212-185510
  23. Wen, C. K. F., Schneider, S., Stone, A. A., & Spruijt-Metz, D. (2017). Compliance with mobile ecological momentary assessment protocols in children and adolescents: A systematic review and meta-analysis. Journal of Medical Internet Research , 19 (4), Article e132. https://doi.org/10 .2196/jmir.6641
  24. Wilhelm, P., & Schoebi, D. (2007). Assessing mood in daily life: Structural validity, sensitivity to change, and reliability of a short-scale to measure three basic dimensions of mood. European Journal of Psychological Assessment , 23 (4), 258 -267. https://doi.org/10.1027/1015 -5759.23.4.258 · DOI
  25. Yuan, K.-H., Yang-Wallentin, F., & Bentler, P. M. (2012). ML versus MI for missing data with violation of distribution conditions. Sociological Methods & Research , 41 (4), 598 -629. https://doi.org/10.1177/00491 24112460373 · DOI

Cited by

Sorted by citations

Authors
Mansueto, A. C., Wiers, R. W., van Weert, J. C. M., Schouten, B. C., & Epskamp, S.
Affiliations
University of Amsterdam (NL)
Venue
Psychological Methods
Published
2023-10
Volume
28
Issue
5
Pages
1052-1068
ISSN
  • 1082-989X (ISSN-L)
  • 1939-1463
OpenAlex
W4225594736
Cited by
147
References
0
Citations by year
2026: 32 · 2025: 32 · 2024: 27 · 2023: 24 · 2022: 21 · 2021: 10 · 2014: 1
FWCI
23.93
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • Psychological Methods (publishedVersion)
  • PubMed (publishedVersion)
  • UvA-DARE (University of Amsterdam) (submittedVersion, other-oa)
  • PsyArXiv (OSF Preprints) (submittedVersion)
  • PsyArXiv (OSF Preprints) (submittedVersion, cc-by)
Funders
University of Amsterdam
Topics
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Mental Health Research Topics
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Anxiety, Depression, Psychometrics, Treatment, Cognitive Processes
Keywords
personalized psychotherapy, ecological momentary assessment, sample size, missing data, Kalman filter, full information maximum likelihood, dynamic networks, psychopathology, simulation study

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review