Smartphone-based ecological momentary assessment (EMA) in psychiatric patients and student controls: a real-world feasibility study
Open accessAbstract
Smartphone-based ecological momentary assessment (EMA) is a promising methodology for mental health research. The objective of this study is to determine the feasibility of smartphone-based active and passive EMA in psychiatric outpatients and student controls. Two smartphone applications —MEmind and eB2— were developed for behavioral active and passive monitoring. The applications were tested in psychiatric patients with a history of suicidal thoughts and/or behaviors (STB), psychiatric patients without a history of STB, and student controls. Main outcome was feasibility, measured as response to recruitment, retention, and EMA compliance. Secondary outcomes were patterns of smartphone usage. Response rate was 87.3% in patients with a history of STB, 85.1% in patients without a history of STB, and 75.0% in student controls. 457 participants installed the MEmind app (120 patients with a history of STB and 337 controls) and 1,708 installed the eB2 app (139 patients with a history of STB, 1,224 patients with no history of STB and 346 controls). For the MEmind app, participants were followed-up for a median of 49.5, resulting in 22,622 person-days. For the eB2 application, participants were followed-up for a median of 48.9 days, resulting in 83,521 person-days. EMA compliance rate was 65.00% in suicidal patients and 75.21% in student controls. At the end of the follow-up, over 60% of participants remained in the study. Cases and controls were not matched by age and sex. Cases were patients who were receiving adequate psychopharmacological treatment and attending their appointments, which may result in an overstatement of clinical compliance. Smartphone-based active and passive monitoring are feasible methods in psychiatric patients in real-world settings. The development of applications with friendly interfaces and directly useful features can help increase engagement without using incentives. The MEmind and eB2 applications are promising clinical tools that could contribute to the management of mental disorders. In the near future, these applications could serve as risk monitoring devices in the clinical practice.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Recent EMA studies in samples of individuals with SI found compliance rates of around 65% per assessment.
“Recent EMA studies in samples of individuals with SI found compliance rates of around 65% per assessment, although the role of incentives (such as increased compensation for compliance), the length of the study, and the number of daily assessments may affect those rates (33, 47).”
From New methods for assessing rapid changes in suicide risk (2021) · cites this paper as [47]
Two reviewed studies used a turn-over system where a subset of questions was randomly presented at a certain time point to limit repetition.
“Two studies used a turn-over system where a subset of questions was randomly presented at a certain time point to limit repetition (83, 87).”
From Don't Miss the Moment: A Systematic Review of Ecological Momentary Assessment in Suicide Research (2022) · cites this paper as [87]
A similar pattern emerged when comparing psychiatric patients (65%) and student controls (75%).
“A similar pattern emerged when comparing psychiatric patients (65%) and student controls (75%) (87).”
From Don't Miss the Moment: A Systematic Review of Ecological Momentary Assessment in Suicide Research (2022) · cites this paper as [87]
In contrast to findings of lower compliance rates among psychiatric patients, dropout was lower among clinical cases than controls.
“In contrast to findings of lower compliance rates among psychiatric patients, dropout was lower among clinical cases than controls (87).”
From Don't Miss the Moment: A Systematic Review of Ecological Momentary Assessment in Suicide Research (2022) · cites this paper as [87]
First applications of using EMA in clinical practice to monitor and manage symptoms are already underway.
“First applications of using EMA in clinical practice to monitor and manage symptoms are already underway (87).”
From Don't Miss the Moment: A Systematic Review of Ecological Momentary Assessment in Suicide Research (2022) · cites this paper as [87]
Cites
(1998) · The Journal of Clinical Psychiatry · Cited by 19,139 · see citing works
A Rating Scale for Mania: Reliability, Validity and Sensitivity
Young, R. C., Biggs, J. T., Ziegler, V. E., & Meyer, D. A. (1978) · British Journal of Psychiatry · doi:10.1192/bjp.133.5.429 · Cited by 8,744 · see citing works
Validation of the Insomnia Severity Index as an outcome measure for insomnia research
Bastien, C. (2001) · Sleep Medicine · doi:10.1016/s1389-9457(00)00065-4 · Cited by 8,067 · see citing works
Ecological momentary assessment
Shiffman, S., Stone, A. A., & Hufford, M. R. (2008) · Annual Review of Clinical Psychology · doi:10.1146/annurev.clinpsy.3.022806.091415 · Cited by 6,454 · see citing works
Posner, K., Brown, G. K., Stanley, B., Brent, D. A., Yershova, K. V., Oquendo, M. A., Currier, G. W., Melvin, G. A., Greenhill, L., Shen, S., et al. (2011) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2011.10111704 · Cited by 5,157 · see citing works
State-Trait Anxiety Inventory for Adults
Spielberger, C. D. (2012) · PsycTESTS Dataset · doi:10.1037/t06496-000 · Cited by 3,479 · see citing works
Webb, T. L., Joseph, J., Yardley, L., & Michie, S. (2010) · Journal of Medical Internet Research · doi:10.2196/jmir.1376 · Cited by 2,517 · see citing works
The Inventory of Depressive Symptomatology (IDS): Psychometric properties
Rush, A. J., Gullion, C. M., Basco, M. R., Jarrett, R. B., & Trivedi, M. H. (1996) · Psychological Medicine · doi:10.1017/s0033291700035558 · Cited by 2,177 · see citing works
Kleiman, E. M., Turner, B. J., Fedor, S., Beale, E. E., Huffman, J. C., & Nock, M. K. (2017) · Journal of Abnormal Psychology · doi:10.1037/abn0000273 · Cited by 831 · see citing works
Wilson, M. G., Thomas, D. R., Rubenstein, L. Z., Chibnall, J. T., Anderson, S., Baxi, A., Diebold, M. R., & Morley, J. E. (2005) · American Journal of Clinical Nutrition · doi:10.1093/ajcn/82.5.1074 · Cited by 812 · see citing works
Saeb, S., Zhang, M., Karr, C. J., Schueller, S. M., Corden, M. E., Kording, K. P., & Mohr, D. C. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.4273 · Cited by 782 · see citing works
Torous, J. B., Onnela, J., & Keshavan, M. S. (2017) · Translational Psychiatry · doi:10.1038/tp.2017.25 · Cited by 349 · see citing works
A Systematic Review of Cognitive Behavioral Therapy and Behavioral Activation Apps for Depression
Huguet, A., Rao, S., McGrath, P. J., Wozney, L., Wheaton, M., Conrod, J., & Rozario, S. (2016) · PLOS ONE · doi:10.1371/journal.pone.0154248 · Cited by 328 · see citing works
Versión española de una escala de evaluación de la manía: validez y fiabilidad de la Escala de Young
Colom, F., Vieta, E., Martínez‐Arán, A., García-García, M., Reinares, M., Torrent, C., Goikolea, J. M., Banús, S., & Salamero, M. (2002) · Medicina Clínica · doi:10.1016/s0025-7753(02)73419-2 · Cited by 291 · see citing works
Smartphone Apps for Schizophrenia: A Systematic Review
Firth, J., & Torous, J. (2015) · JMIR mHealth and uHealth · doi:10.2196/mhealth.4930 · Cited by 268 · see citing works
Burke, L. E., Shiffman, S., Music, E., Styn, M. A., Kriska, A., Smailagic, A., Siewiorek, D., Ewing, L. J., Chasens, E., French, B., et al. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.7138 · Cited by 267 · see citing works
What’s in a name? Compliance, adherence and concordance in chronic psychiatric disorders
Chakrabarti, S. (2014) · World Journal of Psychiatry · doi:10.5498/wjp.v4.i2.30 · Cited by 257 · see citing works
Daily emotional dynamics in depressed youth: A cell phone ecological momentary assessment study
Silk, J. S., Forbes, E. E., Whalen, D. J., Jakubcak, J. L., Thompson, W. K., Ryan, N. D., Axelson, D., Birmaher, B. B., & Dahl, R. E. (2010) · Journal of Experimental Child Psychology · doi:10.1016/j.jecp.2010.10.007 · Cited by 215 · see citing works
Hallensleben (2019) · doi:10.1016/j.eurpsy.2018.11.003 · Cited by 210 · see citing works
Littlewood, D. L., Kyle, S. D., Carter, L.-A., Peters, S., Pratt, D., & Gooding, P. (2019) · Psychological Medicine · doi:10.1017/s0033291718001009 · Cited by 209 · see citing works
The Spanish version of the Insomnia Severity Index: A confirmatory factor analysis
Fernández‐Mendoza, J., Rodríguez‐Muñoz, A., Vela‐Bueno, A., Olavarrieta‐Bernardino, S., Calhoun, S. L., Bixler, E. O., & Vgontzas, A. N. (2011) · Sleep Medicine · doi:10.1016/j.sleep.2011.06.019 · Cited by 205 · see citing works
Hita‐Contreras, F., López, E. J. M., Román, P. Á. L., Garrido, F., Campos, M. A. S. E., & Martínez‐Amat, A. (2014) · Rheumatology International · doi:10.1007/s00296-014-2960-z · Cited by 204 · see citing works
The application of mHealth to mental health: opportunities and challenges
Marzano, L., Bardill, A., Fields, B., Herd, K., Veale, D., Grey, N., & Moran, P. (2015) · The Lancet Psychiatry · doi:10.1016/s2215-0366(15)00268-0 · Cited by 200 · see citing works
Asselbergs, J., Ruwaard, J., Ejdys, M., Schrader, N., Sijbrandij, M., & Riper, H. (2016) · Journal of Medical Internet Research · doi:10.2196/jmir.5505 · Cited by 192 · see citing works
Do Incentives Exert Undue Influence on Survey Participation? Experimental Evidence
Singer, E., & Couper, M. P. (2008) · Journal of Empirical Research on Human Research Ethics · doi:10.1525/jer.2008.3.3.49 · Cited by 173 · see citing works
Forkmann, T., Spangenberg, L., Rath, D., Hallensleben, N., Hegerl, U., Kersting, A., & Glaesmer, H. (2018) · Journal of Abnormal Psychology · doi:10.1037/abn0000381 · Cited by 165 · see citing works
Czyz, E. K., King, C. A., & Nahum-Shani, I. (2018) · Psychiatry Research · doi:10.1016/j.psychres.2018.06.031 · Cited by 112 · see citing works
Predictors of re-attempt in a cohort of suicide attempters: A survival analysis
Irigoyen-Otiñano, M., Porras‐Segovia, A., Galván, L., Puigdevall-Ruestes, M., Giner, L., de León-Martínez, S., & Baca‐García, E. (2018) · Journal of Affective Disorders · doi:10.1016/j.jad.2018.12.050 · Cited by 96 · see citing works
Self-citationHidalgo‐Mazzei, D., Reinares, M., Mateu, A., Nikolova, V. L., Bonnín, C. D. M., Samalin, L., García-Estela, A., Pérez, V., Young, A. H., Strejilevich, S., et al. (2018) · Journal of Affective Disorders · doi:10.1016/j.jad.2018.08.048 · Cited by 92 · see citing works
Al‐Halabí, S., Sáiz, P. A., Burón, P., Garrido, M., Benabarre, A., Jiménez, E., Cervilla, J. A., Navarrete, M. I., Díaz-Mesa, E., García-Álvarez, L., et al. (2016) · Revista de Psiquiatría y Salud Mental · doi:10.1016/j.rpsm.2016.02.002 · Cited by 91 · see citing works
Schwartz, S., Schultz, S., Reider, A., & Saunders, E. F. H. (2015) · Journal of Affective Disorders · doi:10.1016/j.jad.2015.11.013 · Cited by 90 · see citing works
Yang, Y. S., Ryu, G. W., & Choi, M. (2019) · JMIR mhealth and uhealth · doi:10.2196/11215 · Cited by 85 · see citing works
Berrouiguet, S., Barrigón, M. L., Castroman, J. L., Courtet, P., Artés-Rodríguez, A., & Baca-García, E. (2019) · BMC Psychiatry · doi:10.1186/s12888-019-2260-y · Cited by 84 · see citing works
Self-citationShort form of the Spanish adaptation of the State-Trait Anxiety Inventory
Buela‐Casal, G., & Guillén‐Riquelme, A. (2017) · International Journal of Clinical and Health Psychology · doi:10.1016/j.ijchp.2017.07.003 · Cited by 78 · see citing works
Eisner, E., Drake, R. J., Berry, N., Barrowclough, C., Emsley, R., Machin, M., & Bucci, S. (2019) · JMIR mHealth and uHealth · doi:10.2196/11568 · Cited by 73 · see citing works
User profiles of an electronic mental health tool for ecological momentary assessment: MEmind
Barrigón, M. L., Berrouiguet, S., Carballo, J. J., Bonal-Giménez, C., Fernández‐Navarro, P., Pfang, B., Delgado‐Gómez, D., Courtet, P., Aroca, F., López‐Castromán, J., et al. (2017) · International Journal of Methods in Psychiatric Research · doi:10.1002/mpr.1554 · Cited by 68 · see citing works
Self-citationWhat might get in the way: Barriers to the use of apps for depression
Stiles-Shields, C., Montague, E., Lattie, E. G., Kwasny, M. J., & Mohr, D. C. (2017) · DIGITAL HEALTH · doi:10.1177/2055207617713827 · Cited by 68 · see citing works
Nuij, C., van Ballegooijen, W., Ruwaard, J., Beurs, D. P. D., Mokkenstorm, J. K., van Duijn, E., de Winter, R. F. P., O’Connor, R. C., Smit, J. H., Riper, H. M., et al. (2018) · Internet Interventions · doi:10.1016/j.invent.2018.04.005 · Cited by 58 · see citing works
Spangenberg, L., Glaesmer, H., Hallensleben, N., Rath, D., & Forkmann, T. (2019) · Suicide and Life-Threatening Behavior · doi:10.1111/sltb.12547 · Cited by 57 · see citing works
Personal financial incentives in health promotion: where do they fit in an ethic of autonomy?
Ashcroft, R. E. (2011) · Health Expectations · doi:10.1111/j.1369-7625.2011.00664.x · Cited by 52 · see citing works
Real-Time Monitoring of Non-linear Suicidal Dynamics: Methodology and a Demonstrative Case Report
Fartacek, C., Schiepek, G. K., Kunrath, S., Fartacek, R. R., & Plöderl, M. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2016.00130 · Cited by 50 · see citing works
Hung, S., Li, M.-S., Chen, Y.-L., Chiang, J.-H., Chen, Y.-Y., & Hung, G. C.-L. (2016) · Asian Journal of Psychiatry · doi:10.1016/j.ajp.2016.08.003 · Cited by 46 · see citing works
Berrouiguet, S., Ramírez, D., Barrigón, M. L., Moreno-Muñoz, P., Camacho, R. C., Baca‐García, E., & Artés-Rodrı́guez, A. (2018) · JMIR mhealth and uhealth · doi:10.2196/mhealth.9472 · Cited by 46 · see citing works
Self-citationHonorarium or coercion: use of incentives for participants in clinical research.
Groth, S. W. (2010) · PubMed · Cited by 39 · see citing works
Hendrikoff, L., Kambeitz‐Ilankovic, L., Pryss, R. C., Senner, F., Falkai, P., Pogarell, O. M., Hasan, A., & Peters, H. (2018) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2018.11.025 · Cited by 35 · see citing works
Assessing and Improving Clinical Insight Among Patients “in Denial”
Williams, A. R., Olfson, M., & Galanter, M. S. (2015) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2014.2684 · Cited by 33 · see citing works
Mackesy-Amiti, M. E., & Boodram, B. (2018) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2018.03.016 · Cited by 32 · see citing works
Hartmann, R., Sander, C., Lorenz, N., Böttger, D., & Hegerl, U. (2019) · JMIR Mental Health · doi:10.2196/11671 · Cited by 28 · see citing works
Electronic monitoring of self-reported mood: the return of the subjective?
Ortiz, A., & Grof, P. (2016) · International Journal of Bipolar Disorders · doi:10.1186/s40345-016-0069-x · Cited by 24 · see citing works
Staccini, P., Fernández-Luque, L., & Informatics, S. E. F. T. I. Y. S. O. C. H. (2016) · Yearbook of Medical Informatics · doi:10.15265/iy-2016-045 · Cited by 23 · see citing works
Feasibility of Ecological Momentary Assessment of Negative Emotion in Girls With ADHD: A Pilot Study
Babinski, D. E., & Welkie, J. M. (2019) · Psychological Reports · doi:10.1177/0033294119838757 · Cited by 19 · see citing works
Trujols, J., de Diego-Adeliño, J., Feliu‐Soler, A., Castillo, I. I., Puigdemont, D., Álvarez, E., Pérez, V., & Portella, M. J. (2014) · Journal of Affective Disorders · doi:10.1016/j.jad.2014.08.011 · Cited by 15 · see citing works
An Open Source Mobile Platform for Psychophysiological Self Tracking
Gaggioli, A., Cipresso, P., Serino, S., Pioggia, G., Tartarisco, G., Baldus, G., Daniele, C., & Riva, G. (2012) · Studies in health technology and informatics · doi:10.3233/978-1-61499-022-2-136 · Cited by 13 · see citing works
Lemey, C., Larsen, M. E., DeVylder, J., Courtet, P., Billot, R., Lenca, P., Walter, M., Baca‐García, E., & Berrouiguet, S. (2018) · Journal of Medical Internet Research · doi:10.2196/10111 · Cited by 13 · see citing works
Self-citationFaurholt‐Jepsen, M., Frost, M. M., Christensen, E. M., Bardram, J. E., Vinberg, M., & Kessing, L. V. (2019) · Journal of Affective Disorders · doi:10.1016/j.jad.2019.07.029 · Cited by 11 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Jakob, R., Harperink, S., Rudolf, A. M., Fleisch, E., Haug, S., Mair, J. L., Salamanca-Sanabria, A., & Kowatsch, T. (2022) · Journal of Medical Internet Research · doi:10.2196/35371 · Cited by 446 · see citing works
Don't Miss the Moment: A Systematic Review of Ecological Momentary Assessment in Suicide Research
Kivelä, L., van der Does, W. A. J., Riese, H., & Antypa, N. (2022) · Frontiers in Digital Health · doi:10.3389/fdgth.2022.876595 · Cited by 163 · see citing works
Use of Ecological Momentary Assessment to Study Suicidal Thoughts and Behavior: a Systematic Review
Sedano-Capdevila, A., Porras-Segovia, A., Bello, H. J., Baca-García, E., & Barrigon, M. L. (2021) · Current Psychiatry Reports · doi:10.1007/s11920-021-01255-7 · Cited by 148 · see citing works
Self-citationNew methods for assessing rapid changes in suicide risk
Ballard, E. D., Gilbert, J. R., Wusinich, C., & Zarate, C. A. (2021) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2021.598434 · Cited by 63 · see citing works
Cobo, A., Porras-Segovia, A., Pérez-Rodríguez, M. M., Artés-Rodríguez, A., Barrigón, M. L., Courtet, P., & Baca-García, E. (2021) · BJPsych Open · doi:10.1192/bjo.2021.43 · Cited by 18 · see citing works
Self-citationMultidimensional variability in ecological assessments predicts two clusters of suicidal patients.
Bonilla-Escribano, P., Ramírez, D., Baca-García, E., Courtet, P., Artés-Rodríguez, A., & López-Castromán, J. (2023) · Scientific Reports · doi:10.1038/s41598-023-30085-1 · Cited by 17 · see citing works
Self-citation
- Authors
- Porras-Segovia, A., Molina-Madueño, R. M., Berrouiguet, S., López-Castroman, J., Barrigón, M. L., Pérez-Rodríguez, M. S., Marco, J. H., Díaz-Oliván, I., de León, S., Courtet, P., Artés-Rodríguez, A., & Baca-García, E.
- Affiliations
- Centre Hospitalier Régional Universitaire de Brest (FR)
- Universitat de València (ES)
- Instituto de Investigación Sanitaria Fundación Jiménez Díaz (ES)
- Université de Montpellier (FR)
- Venue
- Journal of Affective Disorders
- Publisher
- Elsevier BV
- Published
- 2020-09
- Volume
- 274
- Pages
- 733-741
- ISSN
- 0165-0327 (ISSN-L)
- OpenAlex
- W3032703209
- Cited by
- 85
- References
- 55
- Citations by year
- 2026: 10 · 2025: 13 · 2024: 11 · 2023: 12 · 2022: 16 · 2021: 18 · 2020: 5
- FWCI
- 11.55
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Journal of Affective Disorders (acceptedVersion, other-oa)
- PubMed (publishedVersion)
- HAL (submittedVersion, cc-by-nc)
- Repository of Digital Objects for Teaching Research and Culture (University of Valencia) (submittedVersion, cc-by-nc)
- RUIdeRA - Institutional University Repository (University of Castilla-La Mancha) (submittedVersion)
- Funders
- American Foundation for Suicide Prevention
- Topics
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Mental Health Research Topics
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Keywords
- ecological momentary assessment, mobile application, psychiatric outpatients, suicidal ideation, suicidal behavior, passive monitoring, active monitoring, student retention, smartphone usage patterns, mental health monitoring, risk monitoring
- MeSH terms
- Smartphone, Ecological Momentary Assessment, Feasibility Studies, Humans, Students, Mobile Applications
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
