Compliance and Retention With the Experience Sampling Method Over the Continuum of Severe Mental Disorders: Meta-Analysis and Recommendations
Open accessAbstract
Background: Despite the growing interest in the experience sampling method (ESM) as a data collection tool for mental health research, the absence of methodological guidelines related to its use has resulted in a large heterogeneity of designs. Concomitantly, the potential effects of the design on the response behavior of the participants remain largely unknown. Objective: The objective of this meta-analysis was to investigate the associations between various sample and design characteristics and the compliance and retention rates of studies using ESM in mental health research. Methods: ESM studies investigating major depressive disorder, bipolar disorder, and psychotic disorder were considered for inclusion. Besides the compliance and retention rates, a number of sample and design characteristics of the selected studies were collected to assess their potential relationships with the compliance and retention rates. Multilevel random/mixed effects models were used for the analyses. Results: Compliance and retention rates were lower for studies with a higher proportion of male participants (P<.001) and individuals with a psychotic disorder (P<.001). Compliance was positively associated with the use of a fixed sampling scheme (P=.02), higher incentives (P=.03), higher time intervals between successive evaluations (P=.02), and fewer evaluations per day (P=.008), while no significant associations were observed with regard to the mean age of the sample, the study duration, or other design characteristics. Conclusions: The findings demonstrate that ESM studies can be carried out in mental health research, but the quality of the data collection depends upon a number of factors related to the design of ESM studies and the samples under study that need to be considered when designing such protocols.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Studies that apply non-personalized experience sampling method questionnaires in clinical samples, including those from our own group, have used a similar or higher number of questions, and compliance rates suggest a manageable burden for participants.
“Studies that apply (non-personalized) ESM questionnaires in clinical samples, including those from our own group, have used a similar or higher number of questions [32–35], and compliance rates suggest a manageable burden for participants.”
From Using person-specific networks in psychotherapy: challenges, limitations, and how we could use them anyway (2020) · cites this paper as [34]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
PRISMA-P Group, Moher, D., Shamseer, L., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., & Stewart, L. A. (2015) · Systematic Reviews · doi:10.1186/2046-4053-4-1 · Cited by 26,223 · see citing works
Conducting Meta-Analyses in <i>R</i> with the <b>metafor</b> Package
Viechtbauer, W. (2010) · Journal of Statistical Software · doi:10.18637/jss.v036.i03 · Cited by 20,464 · see citing works
Self-citationNetwork analysis: An integrative approach to the structure of psychopathology
Borsboom, D., & Cramer, A. O. J. (2013) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-050212-185608 · Cited by 4,359 · see citing works
Some Methods for Strengthening the Common χ 2 Tests
Cochran, W. G. (1954) · Biometrics · doi:10.2307/3001616 · Cited by 3,718 · cited 1× · see citing works
“Heterogeneity was assessed using the Q-test [41] and based on the estimates of the between-study and between-group heterogeneity variance components (with 95% profile likelihood confidence intervals).”
MethodOutlier and influence diagnostics for meta-analysis
Viechtbauer, W., & Cheung, M. W.-L. (2010) · Research Synthesis Methods · doi:10.1002/jrsm.11 · Cited by 2,949 · see citing works
Self-citationThe Handbook of Research Synthesis.
Light, R. J., Cooper, H., & Hedges, L. V. (1994) · Journal of the American Statistical Association · doi:10.2307/2291021 · Cited by 1,880 · see citing works
Experience sampling research in psychopathology: opening the black box of daily life
Myin-Germeys, I., Oorschot, M., Collip, D., Lataster, J., Delespaul, P., & van Os, J. (2009) · Psychological Medicine · doi:10.1017/s0033291708004947 · Cited by 817 · see citing works
Self-citationCapturing momentary, self-report data: A proposal for reporting guidelines
Stone, A. A., & Shiffman, S. (2002) · Annals of Behavioral Medicine · doi:10.1207/s15324796abm2403_09 · Cited by 807 · see citing works
Theory of mind impairment in schizophrenia: meta-analysis
Bora, E., Yucel, M., & Pantelis, C. (2009) · Schizophrenia Research · doi:10.1016/j.schres.2008.12.020 · Cited by 797 · see citing works
Experience sampling methodology in mental health research: new insights and technical developments.
Myin‐Germeys, I., Kasanova, Z., Vaessen, T., Vachon, H., Kirtley, O., Viechtbauer, W., & Reininghaus, U. (2018) · World Psychiatry · doi:10.1002/wps.20513 · Cited by 709 · see citing works
Self-citationFeelings change: Accounting for individual differences in the temporal dynamics of affect.
Kuppens, P., Oravecz, Z., & Tuerlinckx, F. (2010) · Journal of Personality and Social Psychology · doi:10.1037/a0020962 · Cited by 543 · see citing works
Fixed effects and variance components estimation in three-level meta-analysis
Konstantopoulos, S. (2011) · Research Synthesis Methods · doi:10.1002/jrsm.35 · Cited by 521 · see citing works
Assessing schizophrenia in daily life : the experience sampling method
Delespaul, P. (1995) · doi:10.26481/dis.19950504pd · Cited by 349 · see citing works
aan het Rot, M., Hogenelst, K., & Schoevers, R. A. (2012) · Clinical Psychology Review · doi:10.1016/j.cpr.2012.05.007 · Cited by 325 · see citing works
Compliance with ecological momentary assessment protocols in substance users: a meta-analysis
Jones, A., Remmerswaal, D., Verveer, I., Robinson, E., Franken, I. H. A., Wen, C. K. F., & Field, M. (2019) · Addiction · doi:10.1111/add.14503 · Cited by 319 · see citing works
Meta‐analysis of variation: ecological and evolutionary applications and beyond
Nakagawa, S., Poulin, R., Mengersen, K. L., Reinhold, K., Engqvist, L., Lagisz, M., & Senior, A. M. (2014) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.12309 · Cited by 299 · see citing works
Stone, A. A., Broderick, J. E., Schwartz, J. E., Shiffman, S., Litcher-Kelly, L., & Calvanese, P. (2003) · Pain · doi:10.1016/s0304-3959(03)00040-x · Cited by 293 · see citing works
Thompson, R. J., Mata, J., Jaeggi, S. M., Buschkuehl, M., Jonides, J., & Gotlib, I. H. (2012) · Journal of Abnormal Psychology · doi:10.1037/a0027978 · Cited by 292 · see citing works
Torous, J., Friedman, R., & Keshavan, M. (2014) · JMIR mHealth and uHealth · doi:10.2196/mhealth.2994 · Cited by 287 · see citing works
Schizophrenia Patients Are More Emotionally Active Than Is Assumed Based on Their Behavior
Myin‐Germeys, I., Delespaul, P., & deVries, M. W. (2000) · Schizophrenia Bulletin · doi:10.1093/oxfordjournals.schbul.a033499 · Cited by 255 · see citing works
Self-citationRücker, G., Schwarzer, G., Carpenter, J. R., & Olkin, I. (2008) · Statistics in Medicine · doi:10.1002/sim.3511 · Cited by 252 · see citing works
Ruscio, A. M., Gentes, E. L., Jones, J. D., Hallion, L. S., Coleman, E. S., & Swendsen, J. (2015) · Journal of Abnormal Psychology · doi:10.1037/abn0000025 · Cited by 228 · cited 1× · see citing works
“Designs vary from 2 [18] to 50 evaluations per day [19], occurring at fixed [20], semirandom [21], or random time intervals [22], for 1 [23] to 150 days [24], using paper-and-pencil [25] or electronic devices [26], Likert scales [27], or visual analogue scales [28], and with questionnaires varying…”
Feeling blue or turquoise? Emotional differentiation in major depressive disorder
Demiralp, E., Thompson, R. J., Mata, J., Jaeggi, S. M., Buschkuehl, M., Barrett, L. F., Ellsworth, P. C., Demiralp, M., Hernandez-Garcia, L., Deldin, P. J., et al. (2012) · Psychological Science · doi:10.1177/0956797612444903 · Cited by 205 · see citing works
Alliger, G. M., & Williams, K. J. (1993) · Personnel Psychology · doi:10.1111/j.1744-6570.1993.tb00883.x · Cited by 196 · see citing works
Piasecki, T. M., Hufford, M. R., Solhan, M. B., & Trull, T. J. (2007) · Psychological Assessment · doi:10.1037/1040-3590.19.1.25 · Cited by 196 · see citing works
Walk on the bright side: Physical activity and affect in major depressive disorder.
Mata, J., Thompson, R. J., Jaeggi, S. M., Buschkuehl, M., Jonides, J., & Gotlib, I. H. (2012) · Journal of Abnormal Psychology · doi:10.1037/a0023533 · Cited by 191 · see citing works
Feasibility and Validity of Computerized Ecological Momentary Assessment in Schizophrenia
Granholm, E. L., Loh, C., & Swendsen, J. D. (2007) · Schizophrenia Bulletin · doi:10.1093/schbul/sbm113 · Cited by 190 · see citing works
Wichers, M., Peeters, F., Geschwind, N., Jacobs, N., Simons, C. J. P., Derom, C., Thiery, E., Delespaul, P. H., & van Os, J. (2010) · Journal of Affective Disorders · doi:10.1016/j.jad.2009.11.010 · Cited by 189 · see citing works
Cognitive Deficits in Depression and Functional Specificity of Regional Brain Activity
Levin, R. L., Heller, W., Mohanty, A., Herrington, J. D., & Miller, G. A. (2007) · Cognitive Therapy and Research · doi:10.1007/s10608-007-9128-z · Cited by 187 · see citing works
Compliance with momentary pain measurement using electronic diaries: A systematic review
Morren, M., van Dulmen, S., Ouwerkerk, J. W., & Bensing, J. M. (2008) · European Journal of Pain · doi:10.1016/j.ejpain.2008.05.010 · Cited by 177 · see citing works
Response compliance and predictors thereof in studies using the experience sampling method.
Rintala, A., Wampers, M., Myin‐Germeys, I., & Viechtbauer, W. (2018) · Psychological Assessment · doi:10.1037/pas0000662 · Cited by 176 · see citing works
Self-citationLee-Flynn, S. C., Pomaki, G., DeLongis, A., Biesanz, J. C., & Puterman, E. (2011) · Personality and Social Psychology Bulletin · doi:10.1177/0146167210394204 · Cited by 148 · cited 1× · see citing works
“However, 2 studies with very low compliance rates [43,44] and 3 studies with very low retention rates [44-46] were found to be overly influential based on their Cook distances (larger than the median Cook distance plus 2.5 times the interquartile range) and were excluded from further analyses…”
InfluentialFeasibility and validity of ecological momentary assessment in the investigation of suicide risk
Husky, M., Olié, E., Guillaume, S., Genty, C., Swendsen, J., & Courtet, P. (2014) · Psychiatry Research · doi:10.1016/j.psychres.2014.08.019 · Cited by 124 · see citing works
Johnson, E. I., Grondin, O., Barrault, M., Faytout, M., Helbig, S., Husky, M., Granholm, E. L., Loh, C., Nadeau, L., Wittchen, H., et al. (2009) · International Journal of Methods in Psychiatric Research · doi:10.1002/mpr.276 · Cited by 122 · see citing works
Myin-Germeys, I., Marcelis, M., Krabbendam, L., Delespaul, P., & van Os, J. (2005) · Biological Psychiatry · doi:10.1016/j.biopsych.2005.02.012 · Cited by 115 · see citing works
Self-citationA qualitative approach to guide choices for designing a diary study
Janssens, K. A. M., Bos, E. H., Rosmalen, J. G. M., Wichers, M. C., & Riese, H. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-018-0579-6 · Cited by 105 · see citing works
Ficek, S. K., Blasey, C. M., & Schatzberg, A. F. (2003) · Journal of Psychiatric Research · doi:10.1016/s0022-3956(03)00100-6 · Cited by 104 · see citing works
Aldinger, M., Stopsack, M., Ulrich, I., Appel, K., Reinelt, E., Wolff, S., Grabe, H. J., Lang, S., & Barnow, S. (2014) · BMC Psychiatry · doi:10.1186/s12888-014-0210-2 · Cited by 103 · see citing works
van Roekel, E., Bennik, E. C., Bastiaansen, J. A., Verhagen, M., Ormel, J., Engels, R. C. M. E., & Oldehinkel, A. J. (2015) · Research on Child and Adolescent Psychopathology · doi:10.1007/s10802-015-0090-z · Cited by 102 · see citing works
Silvia, P. J., Kwapil, T. R., Eddington, K. M., & Brown, L. H. (2013) · Social Science Computer Review · doi:10.1177/0894439313479902 · Cited by 100 · see citing works
Ramsey, A. T., Wetherell, J. L., Depp, C. A., Dixon, D. J., & Lenze, E. J. (2016) · Journal of Technology in Human Services · doi:10.1080/15228835.2016.1170649 · Cited by 91 · see citing works
Factors Predicting Compliance to Ecological Momentary Assessment Among Adolescent Smokers
Sokolovsky, A. W., Mermelstein, R. J., & Hedeker, D. (2013) · Nicotine & Tobacco Research · doi:10.1093/ntr/ntt154 · Cited by 82 · see citing works
Effects of Antidepressant Treatment on the Quality of Daily Life
Barge‐Schaapveld, D. Q. C. M., & Nicolson, N. A. (2002) · The Journal of Clinical Psychiatry · doi:10.4088/jcp.v63n0603 · Cited by 81 · see citing works
Havermans, R., Nicolson, N. A., Berkhof, J., & deVries, M. W. (2011) · Psychoneuroendocrinology · doi:10.1016/j.psyneuen.2010.07.016 · Cited by 79 · see citing works
Fuller‐Tyszkiewicz, M., Skouteris, H., Richardson, B., Blore, J. D., Holmes, M., & Mills, J. (2013) · Body Image · doi:10.1016/j.bodyim.2013.06.003 · Cited by 72 · see citing works
Computerized ambulatory monitoring in mood disorders: Feasibility, compliance, and reactivity
Husky, M. M., Gindre, C., Mazure, C. M., Brebant, C., Nolen-Hoeksema, S., Sanacora, G., & Swendsen, J. (2010) · Psychiatry Research · doi:10.1016/j.psychres.2010.04.045 · Cited by 62 · see citing works
Reward and Affective Regulation in Depression-Prone Smokers
Audrain-McGovern, J. E., Wileyto, E. P., Ashare, R. L., Cuevas, J., & Strasser, A. A. (2014) · Biological Psychiatry · doi:10.1016/j.biopsych.2014.04.018 · Cited by 57 · see citing works
Husen, K., Rafaeli, E., Rubel, J. A., Bar‐Kalifa, E., & Lutz, W. (2016) · Journal of Affective Disorders · doi:10.1016/j.jad.2016.08.025 · Cited by 55 · see citing works
Hypomanic Personality, Stability of Self‐Esteem and Response Styles to Negative Mood
Bentall, R. P., Myin‐Germeys, I., Smith, A., Knowles, R. J., Jones, S. H., Smith, T., & Tai, S. (2011) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.780 · Cited by 43 · see citing works
Self-citationMessiah, A., Grondin, O., & Encrenaz, G. (2010) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2010.09.016 · Cited by 42 · see citing works
Blum, L. H., Vakhrusheva, J., Saperstein, A. M., Khan, S., Chang, R. W., Hansen, M. C., Zemon, V. M., & Kimhy, D. (2015) · Psychiatry Research · doi:10.1016/j.psychres.2015.03.008 · Cited by 40 · see citing works
Detecting improvements in acute psychotic symptoms using experience sampling methodology
So, S. H., PETERS, E. R., Swendsen, J. D., Garety, P. A., & Kapur, S. (2013) · Psychiatry Research · doi:10.1016/j.psychres.2013.05.010 · Cited by 37 · see citing works
Sagar, K. A., Dahlgren, M. K., Racine, M. T., Dreman, M. W., Olson, D. P., & Gruber, S. A. (2016) · PLoS ONE · doi:10.1371/journal.pone.0157060 · Cited by 31 · see citing works
Vachon, H., Fortes-Bourbousson, M., Deschamps, T., Doron, J., Bulteau, S., Sauvaget, A., & Thomas-Ollivier, V. (2016) · Psychiatry Research · doi:10.1016/j.psychres.2016.08.034 · Cited by 27 · see citing works
Self-citationOhmuro, N., Matsumoto, K., Katsura, M., Obara, C., Kikuchi, T., Hamaie, Y., Sakuma, A., Iizuka, K., Ito, F., & Matsuoka, H. (2015) · Schizophrenia Research · doi:10.1016/j.schres.2015.01.008 · Cited by 25 · see citing works
Colombo, D., García‐Palacios, A., Fernández‐Álvarez, J., Patanè, A., Semonella, M., Cipresso, P., Kwiatkowska, M., Riva, G., & Botella, C. (2018) · Systematic Reviews · doi:10.1186/s13643-018-0899-y · Cited by 21 · see citing works
Kuepper, R., Oorschot, M., Myin‐Germeys, I., Smits, M., van Os, J., & Henquet, C. (2013) · Acta Psychiatrica Scandinavica · doi:10.1111/acps.12078 · Cited by 19 · see citing works
Self-citationVachon, H., Rintala, A., Viechtbauer, W., & Myin‐Germeys, I. (2018) · Systematic Reviews · doi:10.1186/s13643-018-0673-1 · Cited by 9 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Wrzus, C., & Neubauer, A. B. (2023) · Assessment · doi:10.1177/10731911211067538 · Cited by 500 · see citing works
Eisele, G., Vachon, H., Lafit, G., Kuppens, P., Houben, M., Myin-Germeys, I., & Viechtbauer, W. (2022) · Assessment · doi:10.1177/1073191120957102 · Cited by 459 · see citing works
Self-citationRoefs, A., Fried, E. I., Kindt, M., Martijn, C., Elzinga, B., Evers, A. W. M., Wiers, R. W., Borsboom, D., & Jansen, A. (2022) · Behaviour Research and Therapy · doi:10.1016/j.brat.2022.104096 · Cited by 185 · see citing works
Hepp, J., Carpenter, R. W., Störkel, L. M., Schmitz, S. E., Schmahl, C. G., & Niedtfeld, I. (2020) · Clinical Psychology Review · doi:10.1016/j.cpr.2020.101888 · Cited by 149 · see citing works
Use of Ecological Momentary Assessment to Study Suicidal Thoughts and Behavior: a Systematic Review
Sedano-Capdevila, A., Porras-Segovia, A., Bello, H. J., Baca-García, E., & Barrigon, M. L. (2021) · Current Psychiatry Reports · doi:10.1007/s11920-021-01255-7 · Cited by 148 · see citing works
Kirtley, O. J., Lafit, G., Achterhof, R., Hiekkaranta, A. P., & Myin‐Germeys, I. (2021) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245920924686 · Cited by 130 · see citing works
Self-citationNakagawa, S., Noble, D. W. A., Lagisz, M., Spake, R., Viechtbauer, W., & Senior, A. M. (2022) · Ecology Letters · doi:10.1111/ele.14144 · Cited by 123 · see citing works
Self-citationWang, S. B., Coppersmith, D. D. L., Kleiman, E. M., Bentley, K. H., Millner, A. J., Fortgang, R., Mair, P., Dempsey, W., Huffman, J. C., & Nock, M. K. (2021) · JAMA Network Open · doi:10.1001/jamanetworkopen.2021.0591 · Cited by 98 · see citing works
A qualitative approach to guide choices for designing a diary study
Janssens, K. A. M., Bos, E. H., Rosmalen, J. G. M., Wichers, M. C., & Riese, H. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-018-0579-6 · Cited by 105 · see citing works
A Systematic Review of Momentary Assessment Designs for Mood and Anxiety Symptoms
Hall, M., Scherner, P. V., Kreidel, Y., & Rubel, J. A. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.642044 · Cited by 100 · see citing works
Götzl, C., Hiller, S., Rauschenberg, C., Schick, A., Fechtelpeter, J., Fischer-Abaigar, U., Koppe, G., Durstewitz, D., Reininghaus, U., & Krumm, S. (2022) · Child and Adolescent Psychiatry and Mental Health · doi:10.1186/s13034-022-00522-6 · Cited by 84 · see citing works
Early warning signals and critical transitions in psychopathology: challenges and recommendations
Helmich, M. A., Olthof, M., Oldehinkel, A. J., Wichers, M., Bringmann, L. F., & Smit, A. C. (2021) · Current Opinion in Psychology · doi:10.1016/j.copsyc.2021.02.008 · Cited by 80 · see citing works
Digital phenotyping adherence, feasibility, and tolerability in outpatients with schizophrenia
Raugh, I. M., James, S. H., Gonzalez, C. M., Chapman, H. C., Cohen, A. S., Kirkpatrick, B. W., & Strauss, G. P. (2021) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2021.04.022 · Cited by 78 · see citing works
Ambulatory assessment in psychopathology research: Current achievements and future ambitions
Mestdagh, M., & Dejonckheere, E. (2021) · Current Opinion in Psychology · doi:10.1016/j.copsyc.2021.01.004 · Cited by 79 · see citing works
Hasselhorn, K., Ottenstein, C., & Lischetzke, T. (2021) · Behavior Research Methods · doi:10.3758/s13428-021-01683-6 · Cited by 72 · see citing works
Momentary predictors of compliance in studies using the experience sampling method
Rintala, A., Wampers, M., Myin-Germeys, I., & Viechtbauer, W. (2020) · Psychiatry Research · doi:10.1016/j.psychres.2020.112896 · Cited by 68 · see citing works
Self-citationvon Klipstein, L., Riese, H., van der Veen, D. C., Servaas, M. N., & Schoevers, R. A. (2020) · BMC Medicine · doi:10.1186/s12916-020-01818-0 · Cited by 63 · see citing works
Bos, F. M., Schreuder, M. J., George, S. V., Doornbos, B., Bruggeman, R., van der Krieke, L., Haarman, B. C. M., Wichers, M., & Snippe, E. (2022) · International Journal of Bipolar Disorders · doi:10.1186/s40345-022-00258-4 · Cited by 64 · see citing works
Sun, J., Rhemtulla, M., & Vazire, S. (2020) · Personality and Social Psychology Bulletin · doi:10.1177/0146167220964639 · Cited by 62 · see citing works
Bos, F. M., von Klipstein, L., Emerencia, A. C., Veermans, E., Verhage, T., Snippe, E., Doornbos, B., Hadders-Prins, G., Wichers, M., & Riese, H. (2022) · JMIR Mental Health · doi:10.2196/36430 · Cited by 62 · see citing works
Eisele, G. V., Vachon, H., Lafit, G., Kuppens, P., Houben, M., Myin‐Germeys, I., & Viechtbauer, W. (2020) · doi:10.31234/osf.io/zf4nm · Cited by 60 · see citing works
Self-citationBlanchard, M. A., Contreras, A., Kalkan, R. B., & Heeren, A. (2022) · Behavior Research Methods · doi:10.3758/s13428-022-01839-y · Cited by 59 · see citing works
Remote Methods for Conducting Tobacco-Focused Clinical Trials
Dahne, J., Tomko, R. L., McClure, E. A., Obeid, J. S., & Carpenter, M. J. (2020) · Nicotine & Tobacco Research · doi:10.1093/ntr/ntaa105 · Cited by 57 · see citing works
Interoceptive attention facilitates emotion regulation strategy use
Tan, Y., Wang, X., Blain, S. D., Jia, L., & Qiu, J. (2022) · International Journal of Clinical and Health Psychology · doi:10.1016/j.ijchp.2022.100336 · Cited by 58 · see citing works
Ottenstein, C., & Werner, L. (2022) · Assessment · doi:10.1177/10731911211032718 · Cited by 52 · see citing works
Bos, F. M., Snippe, E., Bruggeman, R., Doornbos, B., Wichers, M., & van der Krieke, L. (2020) · International Journal of Bipolar Disorders · doi:10.1186/s40345-020-00201-5 · Cited by 54 · see citing works
Bell, I. H., Eisner, E., Allan, S., Cartner, S., Torous, J. B., Bucci, S., & Thomas, N. (2023) · Schizophrenia Bulletin · doi:10.1093/schbul/sbad127 · Cited by 52 · see citing works
Tonkin, S., Gass, J. C., Wray, J. M., Maguin, E., Mahoney, M. C., Colder, C. R., Tiffany, S. T., & Hawk, L. W. (2023) · Journal of Medical Internet Research · doi:10.2196/43826 · Cited by 43 · see citing works
Shevchenko, Y., Kuhlmann, T., & Reips, U. (2021) · Behavior Research Methods · doi:10.3758/s13428-020-01527-9 · Cited by 43 · see citing works
Rabasco, A., & Sheehan, K. G. (2021) · Archives of Suicide Research · doi:10.1080/13811118.2021.1903635 · Cited by 41 · see citing works
Ortiz, A., Maslej, M. M., Husain, M. I., Daskalakis, Z. J., & Mulsant, B. H. (2021) · Journal of Affective Disorders · doi:10.1016/j.jad.2021.08.140 · Cited by 41 · see citing works
Investigating Best Practices for Ecological Momentary Assessment: Nationwide Factorial Experiment
Businelle, M. S., Hébert, E. T., Shi, D., Benson, L., Kezbers, K. M., Tonkin, S., Piper, M. E., & Qian, T. (2024) · Journal of Medical Internet Research · doi:10.2196/50275 · Cited by 37 · see citing works
Slow down and be critical before using early warning signals in psychopathology
Helmich, M. A., Schreuder, M. J., Bringmann, L. F., Riese, H., Snippe, E., & Smit, A. C. (2024) · Nature Reviews Psychology · doi:10.1038/s44159-024-00369-y · Cited by 39 · see citing works
Tutunji, R., Kogias, N., Kapteijns, B., Krentz, M., Krause, F., Vassena, E., & Hermans, E. J. (2023) · Journal of Medical Internet Research · doi:10.2196/39995 · Cited by 41 · see citing works
Murray, A. L., Brown, R., Zhu, X., Speyer, L. G., Yang, Y., Xiao, Z., Ribeaud, D., & Eisner, M. (2022) · Journal of Affective Disorders · doi:10.1016/j.jad.2022.11.014 · Cited by 32 · see citing works
Ecological momentary assessment of mind-wandering: meta-analysis and systematic review
Kawashima, I., Hinuma, T., & Tanaka, S. (2023) · Scientific Reports · doi:10.1038/s41598-023-29854-9 · Cited by 32 · see citing works
Myin‐Germeys, I., Schick, A., Ganslandt, T., Hajdúk, M., Heretik, A., Hoyweghen, I. V., Kiekens, G., Koppe, G., Marelli, L., Nagyová, I., et al. (2024) · Psychological Medicine · doi:10.1017/s0033291724001454 · Cited by 33 · see citing works
Self-citationHardy, A., Ward, T., Emsley, R., Greenwood, K., Freeman, D. H., Fowler, D. G., Kuipers, E., Bebbington, P., & Garety, P. A. (2022) · JMIR Human Factors · doi:10.2196/29725 · Cited by 31 · see citing works
Lenferink, L. I. M., van Eersel, J. H. W., & Franzen, M. (2022) · Comprehensive Psychiatry · doi:10.1016/j.comppsych.2022.152351 · Cited by 28 · see citing works
Snippe, E., Smit, A. C., Kuppens, P., Burger, H., & Ceulemans, E. (2023) · Journal of Psychopathology and Clinical Science · doi:10.1037/abn0000812 · Cited by 31 · see citing works
Mölsä, M., Lax, M., Korhonen, J., Gumpel, T. P., & Söderberg, P. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.844698 · Cited by 26 · see citing works
Webb, C. A., Forgeard, M. J. C., Israel, E. S., Lovell-Smith, N., Beard, C., & Björgvinsson, Þ. (2021) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000555 · Cited by 28 · see citing works
Porras‐Segovia, A., Díaz-Oliván, I., Barrigón, M. L., Moreno, M., Artés-Rodrı́guez, A., Pérez-Rodríguez, M. M., & Baca‐García, E. (2022) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2022.02.026 · Cited by 27 · see citing works
Kraft, R., Reichert, M., & Pryss, R. C. (2024) · Sensors · doi:10.3390/s24020472 · Cited by 25 · see citing works
Castro‐Alvarez, S., Tendeiro, J. N., de Jonge, P., Meijer, R. R., & Bringmann, L. F. (2021) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705511.2021.1961587 · Cited by 28 · see citing works
How people think about being alone shapes their experience of loneliness
Rodriguez, M., Schertz, K. E., & Kross, E. (2025) · Nature Communications · doi:10.1038/s41467-025-56764-3 · Cited by 29 · see citing works
Emotion Detection through Smartphone's Accelerometer and Gyroscope Sensors
Piskioulis, O., Tzafilkou, K., & Economides, A. A. (2021) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3450613.3456822 · Cited by 25 · see citing works
Bahlinger, K., Lincoln, T. M., & Clamor, A. (2021) · Cognitive Therapy and Research · doi:10.1007/s10608-021-10253-1 · Cited by 25 · see citing works
Hart, A., Reis, D., Prestele, E., & Jacobson, N. C. (2022) · Journal of Medical Internet Research · doi:10.2196/34015 · Cited by 22 · see citing works
Girolamo, G., Barattieri di San Pietro, C., Bulgari, V., Dagani, J., Ferrari, C., Hotopf, M., Iannone, G., Macis, A., Matcham, F., Myin‐Germeys, I., et al. (2020) · Depression and Anxiety · doi:10.1002/da.23023 · Cited by 24 · see citing works
Self-citationWrzus, C., & Neubauer, A. B. (2021) · doi:10.31234/osf.io/vmr8q · Cited by 25 · see citing works
A Systematic Study into the Factors that Affect the Predictive Accuracy of Multilevel VAR(1) Models
Lafit, G., Meers, K., & Ceulemans, E. (2021) · Psychometrika · doi:10.1007/s11336-021-09803-z · Cited by 24 · see citing works
Kronkvist, K., & Engström, A. (2020) · Methodological Innovations · doi:10.1177/2059799120980306 · Cited by 21 · see citing works
von Klipstein, L., Servaas, M. N., Schoevers, R. A., van der Veen, D. C., & Riese, H. (2023) · Heliyon · doi:10.1016/j.heliyon.2023.e14507 · Cited by 21 · see citing works
A Pilot Open Trial of a Digital Mindfulness-Based Intervention for Anorexia Nervosa
Sala, M., Levinson, C. A., Kober, H., & Roos, C. R. (2023) · Behavior Therapy · doi:10.1016/j.beth.2023.01.003 · Cited by 19 · see citing works
Wang, S. D., Yang, C., Brown, D. M. Y., Cheng, A., & KWAN, M. Y. W. (2025) · JMIR mhealth and uhealth · doi:10.2196/52887 · Cited by 20 · see citing works
Mewes, R., Feneberg, A. C., Doerr, J. M., & Nater, U. M. (2022) · Psychosomatic Medicine · doi:10.1097/psy.0000000000001006 · Cited by 19 · see citing works
Diest, A. P.-V., Mortier, P., Ballester, L., Amigo, F., Carrasco, P., Falcó, R., Gili, M., Kiekens, G., Machancoses, F. H., Piqueras, J. A., et al. (2024) · Journal of Medical Internet Research · doi:10.2196/55712 · Cited by 20 · see citing works
Garety, P. A., Ward, T., Emsley, R., Greenwood, K., Freeman, D. H., Fowler, D. G., Kuipers, E., Bebbington, P., Dunn, G., & Hardy, A. (2021) · Efficacy and Mechanism Evaluation · doi:10.3310/eme08110 · Cited by 18 · see citing works
Schreuder, M. J., Hartman, C. A., Groen, R. N., Smit, A. C., Wichers, M., & Wigman, J. T. W. (2022) · Clinical Psychological Science · doi:10.1177/21677026221103138 · Cited by 18 · see citing works
Reiter, T., & Schoedel, R. (2023) · Behavior Research Methods · doi:10.3758/s13428-023-02252-9 · Cited by 23 · see citing works
Lenferink, L. I. M., Terbrack, E., van Eersel, J. H. W., Zuidersma, M., Franzen, M., & Riese, H. (2024) · Current Psychology · doi:10.1007/s12144-024-06987-2 · Cited by 19 · see citing works
van Genugten, C., Schuurmans, J., Hoogendoorn, A. W., Araya, R., Andersson, G., Baños, R., Berger, T., Botella, C., Pashoja, A. C., Cieślak, R., et al. (2022) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2022.755809 · Cited by 18 · see citing works
Murray, A. L., Yang, Y., Zhu, X., Speyer, L. G., Brown, R., Eisner, M., & Ribeaud, D. (2023) · International Journal of Methods in Psychiatric Research · doi:10.1002/mpr.1972 · Cited by 21 · see citing works
Wright, A. C., Browne, J., Skiest, H. E., Bhiku, K., Baker, J. T., & Cather, C. (2021) · Schizophrenia Research · doi:10.1016/j.schres.2021.04.010 · Cited by 18 · see citing works
Missing not at random intensive longitudinal data with dynamic structural equation models.
McNeish, D. (2025) · Psychological Methods · doi:10.1037/met0000742 · Cited by 17 · see citing works
Carpenter, R. W., Squeglia, L. M., Emery, N. N., McClure, E. A., Gray, K. M., Miranda, R., & Tomko, R. L. (2020) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2020.107897 · Cited by 17 · see citing works
Bootsma, T. I., Schellekens, M. P. J., van Woezik, R. A. M., Slatman, J., & van der Lee, M. L. (2022) · Internet Interventions · doi:10.1016/j.invent.2022.100568 · Cited by 16 · see citing works
Correlates of real-world goal-directed behavior in schizophrenia
Merchant, J. T., Moran, E. K., Strube, M. J., & Barch, D. M. (2021) · Psychological Medicine · doi:10.1017/s0033291721004281 · Cited by 16 · see citing works
Merchant, J. T., Moran, E. K., & Barch, D. M. (2022) · Schizophrenia Research · doi:10.1016/j.schres.2022.01.012 · Cited by 16 · see citing works
Niemeijer, K., Mestdagh, M., Verdonck, S., Meers, K., & Kuppens, P. (2023) · JMIR Formative Research · doi:10.2196/43296 · Cited by 16 · see citing works
Radley, J., Barlow, J., & Johns, L. (2022) · British Journal of Clinical Psychology · doi:10.1111/bjc.12389 · Cited by 16 · see citing works
Davanzo, A., d’Huart, D., Seker, S., Moessner, M., Zimmermann, R., Schmeck, K., & Behn, A. J. (2023) · Journal of Medical Internet Research · doi:10.2196/44853 · Cited by 18 · see citing works
Cowan, T., Cohen, A. S., Raugh, I. M., & Strauss, G. P. (2022) · Psychiatry Research · doi:10.1016/j.psychres.2022.114485 · Cited by 14 · see citing works
Scholten, S., Schemer, L., Herzog, P., Haas, J. W., Heider, J., Winter, D., Reis, D., & Glombiewski, J. A. (2024) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-024-01363-5 · Cited by 18 · see citing works
The experience sampling methodology in psychosis risk states: A systematic review
Bogudzińska, B., Jaworski, A., Zajdel, A., Skrzypek, K., & Misiak, B. (2024) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2024.04.050 · Cited by 19 · see citing works
Complex PTSD symptoms predict positive symptoms of psychosis in the flow of daily life
Panayi, P., PETERS, E. R., Bentall, R. P., Hardy, A., Berry, K., Sellwood, W., Dudley, R. E. J., Longden, E., Underwood, R., Steel, C., et al. (2024) · Psychological Medicine · doi:10.1017/s0033291724001934 · Cited by 15 · see citing works
van Doorn, M., Nijhuis, L. A., Monsanto, A., van Amelsvoort, T., Popma, A., Jaspers, M., Noordzij, M. L., Öry, F. G., Álvarez‐Jiménez, M., & Nieman, D. H. (2022) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2022.871813 · Cited by 13 · see citing works
Dejonckheere, E., Verdonck, S., Andries, J. I., Röhrig, N., Piot, M., kilani, G., & Mestdagh, M. (2024) · Computers in Human Behavior · doi:10.1016/j.chb.2024.108360 · Cited by 14 · see citing works
The Evolution of Psychiatric Epidemiology: Where to Next?
Plana‐Ripoll, O., Lasgaard, M., Mneimneh, Z., & McGrath, J. J. (2021) · The Canadian Journal of Psychiatry · doi:10.1177/0706743721996110 · Cited by 13 · see citing works
Perceived disturbance and predictors thereof in studies using the experience sampling method
Rintala, A., Wampers, M., Lafit, G., Myin-Germeys, I., & Viechtbauer, W. (2021) · Current Psychology · doi:10.1007/s12144-021-01974-3 · Cited by 12 · see citing works
Self-citationHruska, B. J., Piccirillo, M. L., Lenferink, L. I. M., Pacella‐LaBarbara, M. L., Contractor, A. A., Price, M., & Greene, T. (2025) · European Journal of Psychotraumatology · doi:10.1080/20008066.2025.2477423 · Cited by 12 · see citing works
Stadel, M., Bringmann, L. F., Stulp, G., Elmer, T., Verdonck, S., Mestdagh, M., & van Duijn, M. A. J. (2024) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459241244745 · Cited by 11 · see citing works
Ecological Momentary Assessment Using Smartphones in Patients With Depression: Feasibility Study
Maatoug, R., Peiffer‐Smadja, N., Delval, G., Brochu, T., Pitrat, B., & Millet, B. (2021) · JMIR Formative Research · doi:10.2196/14179 · Cited by 13 · see citing works
Fortea, L., Tortella‐Feliu, M., Juaneda‐Seguí, A., la Peña‐Arteaga, V. D., Chavarría-Elizondo, P., Prat-Torres, L., Soriano‐Mas, C., Lane, S. P., Raduà, J., & Fullana, M. A. (2021) · Assessment · doi:10.1177/10731911211065166 · Cited by 12 · see citing works
How you think about an emotion predicts how you regulate: an experience-sampling study
Wittkamp, M. F., Nowak, U., Clamor, A., & Lincoln, T. M. (2022) · Cognition & Emotion · doi:10.1080/02699931.2022.2027744 · Cited by 11 · see citing works
Jennen, L., Mazereel, V., Vansteelandt, K., Menne‐Lothmann, C., Decoster, J., Derom, C. A., Thiery, E. W., Rutten, B. P. F., Jacobs, N., van Os, J., et al. (2022) · Mental health and physical activity · doi:10.1016/j.mhpa.2022.100499 · Cited by 10 · see citing works
Mintz, E. H., Toner, E. R., Skolnik, A. M., Pan, A., Frumkin, M., Baker, A. W., Simon, N. M., & Robinaugh, D. (2024) · Death Studies · doi:10.1080/07481187.2024.2433109 · Cited by 12 · see citing works
Schricker, I. F., Nayman, S., Reinhard, I., & Kuehner, C. (2022) · Behavior Therapy · doi:10.1016/j.beth.2022.09.001 · Cited by 10 · see citing works
Zarbo, C., Zamparini, M., Nielssen, O., Casiraghi, L., Rocchetti, M., Starace, F., de Girolamo, G., & Collaborators, D. (2023) · Journal of Medical Internet Research · doi:10.2196/42093 · Cited by 9 · see citing works
Hogenelst, K., Krone, T., Maarse, B. C. E., Warnke, I., Snabel, J. M., van den Broek, T. J., Schuren, F. H. J., Moerland, M., & Hoevenaars, F. P. M. (2024) · Brain Behavior & Immunity - Health · doi:10.1016/j.bbih.2024.100918 · Cited by 9 · see citing works
Drexl, K., Ralisa, V., Amoussou, J. R., Wen, C. K. F., Urben, S., Plessen, K. J., & Glaus, J. (2025) · Journal of Medical Internet Research · doi:10.2196/65710 · Cited by 10 · see citing works
Wu, Y.-H., Stangl, E., Oleson, J., Caraher, K. J., & Dunn, C. C. (2021) · Journal of the American Academy of Audiology · doi:10.1055/a-1674-0060 · Cited by 10 · see citing works
Schricker, I. F., Nayman, S., Reinhard, I., & Kuehner, C. (2023) · Behaviour Research and Therapy · doi:10.1016/j.brat.2023.104383 · Cited by 9 · see citing works
Schreuder, M. J., Groen, R. N., Wigman, J. T. W., Wichers, M., & Hartman, C. A. (2022) · Psychological Assessment · doi:10.1037/pas0001197 · Cited by 9 · see citing works
A Time-Varying Dynamic Partial Credit Model to Analyze Polytomous and Multivariate Time Series Data
Castro‐Alvarez, S., Bringmann, L. F., Meijer, R. R., & Tendeiro, J. N. (2023) · Multivariate Behavioral Research · doi:10.1080/00273171.2023.2214787 · Cited by 10 · see citing works
Tutunji, R., Kogias, N., Kapteijns, B., Krentz, M., Krause, F., Vassena, E., & Hermans, E. J. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.06.29.450360 · Cited by 9 · see citing works
Provost, E. M., Sperry, S. H., Tavernor, J., Anderau, S., Yocum, A. K., & McInnis, M. G. (2024) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2024.3407683 · Cited by 10 · see citing works
Capturing Flow Experiences in Everyday Life: A Comparison of Recall and Momentary Measurement
Bartholomeyczik, K., Knierim, M. T., Weinhardt, C., Oettingen, G., & Ebner‐Priemer, U. (2024) · Journal of Happiness Studies · doi:10.1007/s10902-024-00776-1 · Cited by 8 · see citing works
Specker, P., Pociūnaitė-Ott, J., Rosenblum, A. L., Marcolini, S., Waschnig, P., Magoon, C., Joseph, A.-L. C., Nijborg, L. C. J., Pan, X., & Lenferink, L. I. M. (2025) · European Journal of Psychotraumatology · doi:10.1080/20008066.2025.2515705 · Cited by 8 · see citing works
- Authors
- Vachon, H., Viechtbauer, W., Rintala, A., & Myin-Germeys, I.
- Affiliations
- KU Leuven (BE)
- Maastricht University (NL)
- Corresponding author
- Vachon, Hugo
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2019-12-06
- Volume
- 21
- Issue
- 12
- Pages
- e14475
- ISSN
- 1438-8871 (ISSN-L)
- PMC
- PMC6925392
- OpenAlex
- W2993886326
- Cited by
- 252
- References
- 58
- Citations by year
- 2026: 38 · 2025: 44 · 2024: 41 · 2023: 33 · 2022: 46 · 2021: 37 · 2020: 12 · 2018: 1
- FWCI
- 3.65
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Journal of Medical Internet Research (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Lirias (KU Leuven) (submittedVersion, cc-by)
- Research Publications (Maastricht University) (submittedVersion, cc-by)
- Research Publications (Maastricht University) (submittedVersion, other-oa)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Mental Health Research Topics
- Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Keywords
- ecological momentary assessment, compliance, participant retention, severe mental illness, major depressive disorder, bipolar disorder, psychosis, study design characteristics, sampling plans, incentives, gender differences, methodological recommendations
- MeSH terms
- Adolescent, Adult, Aged, Female, Humans, Male, Mental Health, Middle Aged, Psychotic Disorders, Research Design, Young Adult
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
