Compliance to a cell phone-based ecological momentary assessment study: the effect of time and personality characteristics
Abstract
Ecological momentary assessment (EMA) is a method that is now widely used to study behavior and mood in the settings in which they naturally occur. It maximizes ecological validity and avoids the limitations of retrospective self-reports. Compliance patterns across time have not been studied. Consistent compliance patterns could lead to data not missing at random and bias the results of subsequent analyses. In order to use modern statistical approaches for handling missing data, it is important to include variables predicting missing values into the statistical analysis. Therefore, these predictors have to be known and measured. The authors collected data on 3 four-item mood scales measuring well-being, wakefulness, and nervousness on 6 occasions per day for 7 days (N = 305) and examined compliance rate across time, within day, and within week. Results show good global compliance (mean compliance: 74.9% of calls answered). Compliance varied more within day than within week. Within day, it was lower for the first call of the day between 9 p.m. and 11 p.m. and higher for the call between 5 p.m. and 7 p.m. Within week, calls were equally answered across days of the week, but, as the study progressed, there was a slight drop in compliance with a progressive decrease that was stronger for the first 2 calls. Compliance on the person level did not depend on personality or on satisfaction with life. Practical consequences of the results for conducting ambulatory assessment studies are discussed, and some recommendations are given.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Compliance was good and not substantially lower than in other clinical or non-clinical populations.
“Meanwhile, compliance was good and not substantially lower than in other clinical (103) or non-clinical populations [see e.g., (104)].”
From Don't Miss the Moment: A Systematic Review of Ecological Momentary Assessment in Suicide Research (2022) · cites this paper as [104]
Cites
The satisfaction with life scale
Diener, E., Emmons, R. A., Larsen, R. J., & Griffin, S. (1985) · Journal of Personality Assessment · doi:10.1207/s15327752jpa4901_13 · Cited by 31,768 · see citing works
Missing data: our view of the state of the art
Schafer, J. L., & Graham, J. W. (2002) · Psychological Methods · doi:10.1037/1082-989x.7.2.147 · Cited by 11,638 · see citing works
Well-being : the foundations of hedonic psychology
Kahneman, D. J., Diener, E. F., & Schwarz, N. (1999) · Cited by 7,472 · see citing works
(2010) · Cited by 7,034 · see citing works
Ecological momentary assessment
Shiffman, S., Stone, A. A., & Hufford, M. R. (2008) · Annual Review of Clinical Psychology · doi:10.1146/annurev.clinpsy.3.022806.091415 · Cited by 6,454 · see citing works
Ecological Momentary Assessment (Ema) in Behavioral Medicine
Stone, A. A., & Shiffman, S. (1994) · Annals of Behavioral Medicine · doi:10.1093/abm/16.3.199 · Cited by 1,931 · see citing works
Pavot, W., Diener, E., Colvin, C. R., & Sandvik, E. (1991) · Journal of Personality Assessment · doi:10.1207/s15327752jpa5701_17 · Cited by 1,866 · see citing works
Revised NEO Personality Inventory (NEO PI-R) and NEO Five-Factor Inventory (NEO-FFI)
Costa, P. T., & McCrea, R. B. (1992) · Medical Entomology and Zoology · Cited by 1,758 · see citing works
Experience Sampling Method: Measuring the Quality of Everyday Life
Hektner, J. M., Schmidt, J. A., & Csíkszentmihályi, M. (2006) · Cited by 1,255 · see citing works
Patient compliance with paper and electronic diaries
Stone, A. A., Shiffman, S., Schwartz, J. E., Broderick, J. E., & Hufford, M. R. (2003) · Controlled Clinical Trials · doi:10.1016/s0197-2456(02)00320-3 · Cited by 910 · see citing works
Experience Sampling: Promises and Pitfalls, Strengths and Weaknesses
Scollon, C. N., Kim‐Prieto, C., & Diener, E. (2003) · Journal of Happiness Studies · doi:10.1023/a:1023605205115 · Cited by 827 · see citing works
Patient non-compliance with paper diaries
Stone, A. A., Shiffman, S., Schwartz, J. E., Broderick, J. E., & Hufford, M. R. (2002) · BMJ · doi:10.1136/bmj.324.7347.1193 · Cited by 810 · see citing works
Happiness in Everyday Life: The Uses of Experience Sampling
Csikszentmihalyi, M., & Hunter, J. (2003) · Journal of Happiness Studies · doi:10.1023/a:1024409732742 · Cited by 717 · see citing works
Continuity, stability, and change in daily emotional experience across adolescence
Larson, R. W., Moneta, G., Richards, M. H., & Wilson, S. (2002) · Child Development · doi:10.1111/1467-8624.00464 · Cited by 560 · see citing works
Ecological momentary assessment research in behavioral medicine
Smyth, J. M., & Stone, A. A. (2003) · Journal of Happiness Studies · doi:10.1023/a:1023657221954 · Cited by 439 · see citing works
Ambulatory Assessment - Monitoring Behavior in Daily Life Settings
Fahrenberg, J., Myrtek, M., Pawlik, K., & Perrez, M. (2007) · European Journal of Psychological Assessment · doi:10.1027/1015-5759.23.4.206 · Cited by 317 · see citing works
Stone, A. A., Broderick, J. E., Schwartz, J. E., Shiffman, S., Litcher-Kelly, L., & Calvanese, P. (2003) · Pain · doi:10.1016/s0304-3959(03)00040-x · Cited by 293 · see citing works
The Science of Real-Time Data Capture: Self-Reports in Health Research
Atienza, A. A. (2007) · Cited by 289 · see citing works
Collins, R. L., Kashdan, T. B., & Gollnisch, G. (2003) · Experimental and Clinical Psychopharmacology · doi:10.1037//1064-1297.11.1.73 · Cited by 185 · see citing works
Cell phones for ecological momentary assessment with cocaine-addicted homeless patients in treatment
Freedman, M. J., Lester, K. M., McNamara, C. L., Milby, J. B., & Schumacher, J. E. (2006) · Journal of Substance Abuse Treatment · doi:10.1016/j.jsat.2005.10.005 · Cited by 168 · see citing works
Technology and Methods in Behavioral Medicine
Krantz, D. S., Baum, A. E., & Baum, A. S. (1998) · Psychology Press eBooks · doi:10.4324/9781410602770 · Cited by 157 · see citing works
Signaling does not adequately improve diary compliance
Broderick, J. E., Schwartz, J. E., Shiffman, S. S., Hufford, M. R., & Stone, A. A. (2003) · Annals of Behavioral Medicine · doi:10.1207/s15324796abm2602_06 · Cited by 151 · see citing works
Meldrum, S. J. (2002) · Physiological Measurement · doi:10.1088/0967-3334/23/3/701 · Cited by 111 · see citing works
Ecological momentary assessment: A new tool for behavioral medicine research.
Shiffman, S. M., & Stone, A. A. (1998) · Cited by 75 · see citing works
Ecological Momentary Assessment of Fatigue in Patients Receiving Intensive Cancer Therapy
Hacker, E. D., & Ferrans, C. E. (2007) · Journal of Pain and Symptom Management · doi:10.1016/j.jpainsymman.2006.08.007 · Cited by 70 · see citing works
Psychometric properties of a computerized mobile phone method for assessing mood in daily life
Courvoisier, D. S., Eid, M., Lischetzke, T., & Schreiber, W. H. (2010) · Emotion · doi:10.1037/a0017813 · Cited by 56 · see citing works
Measuring Affective Clarity Indirectly: Individual Differences in Response Latencies of State.
Lischetzke, T., Cuccodoro, G., Gauger, A., Todeschini, L., & Eid, M. (2005) · Emotion · doi:10.1037/1528-3542.5.4.431 · Cited by 46 · see citing works
Self-citationAiemagno, S. A., Cochran, D., Feucht, T. E., et al. (1996) · American Journal of Public Health · doi:10.2105/ajph.86.11.1626 · Cited by 33 · see citing works
Litcher‐Kelly, L., Kellerman, Q. D., Stone, A. A., & Hanauer, S. B. (2007) · Annals of Behavioral Medicine · doi:10.1007/bf02879902 · Cited by 24 · see citing works
DIFFERENCES BETWEEN VOLUNTEERS AND NONVOLUNTEERS IN A HIGH-DEMAND SELF-RECORDING STUDY
Waite, B. M., Claffey, R., & Hillbrand, M. (1998) · Psychological Reports · doi:10.2466/pr0.1998.83.1.199 · Cited by 17 · see citing works
Salamon, R., Johnson, E. I., Grondin, O. S., et al. (2009) · Journal de Thérapie Comportementale et Cognitive · doi:10.1016/j.jtcc.2009.04.004 · Cited by 4 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Trull, T. J., & Ebner-Priemer, U. (2013) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-050212-185510 · Cited by 761 · see citing works
Wrzus, C., & Neubauer, A. B. (2023) · Assessment · doi:10.1177/10731911211067538 · Cited by 500 · see citing works
Eisele, G., Vachon, H., Lafit, G., Kuppens, P., Houben, M., Myin-Germeys, I., & Viechtbauer, W. (2022) · Assessment · doi:10.1177/1073191120957102 · Cited by 459 · see citing works
Don't Miss the Moment: A Systematic Review of Ecological Momentary Assessment in Suicide Research
Kivelä, L., van der Does, W. A. J., Riese, H., & Antypa, N. (2022) · Frontiers in Digital Health · doi:10.3389/fdgth.2022.876595 · Cited by 163 · see citing works
μEMA: Microinteraction-based ecological momentary assessment (EMA) using a smartwatch
Intille, S., Haynes, C., Maniar, D., Ponnada, A., & Manjourides, J. (2016) · Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing · doi:10.1145/2971648.2971717 · Cited by 130 · see citing works
Silvia, P. J., Kwapil, T. R., Walsh, M. A., & Myin-Germeys, I. (2014) · Behavior Research Methods · doi:10.3758/s13428-013-0353-y · Cited by 97 · see citing works
Momentary predictors of compliance in studies using the experience sampling method
Rintala, A., Wampers, M., Myin-Germeys, I., & Viechtbauer, W. (2020) · Psychiatry Research · doi:10.1016/j.psychres.2020.112896 · Cited by 68 · see citing works
Ottenstein, C., & Werner, L. (2022) · Assessment · doi:10.1177/10731911211032718 · Cited by 52 · see citing works
Mackesy-Amiti, M. E., & Boodram, B. (2018) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2018.03.016 · Cited by 32 · see citing works
Gort, C., Marcusson-Clavertz, D., & Kuehner, C. (2021) · Journal of Rational-Emotive & Cognitive-Behavior Therapy · doi:10.1007/s10942-020-00353-4 · Cited by 29 · see citing works
Experiential Avoidance and Meaning in Life as Predictors of Valued Living: A daily diary study
Pavlacic, J. M., Schulenberg, S. E., & Buchanan, E. M. (2021) · Journal of Prevention and Health Promotion · doi:10.1177/2632077021998261 · Cited by 18 · see citing works
Fischer, A., Rosner, R., Renneberg, B., & Steil, R. (2022) · Borderline Personality Disorder and Emotion Dysregulation · doi:10.1186/s40479-021-00172-8 · Cited by 18 · see citing works
- Authors
- Courvoisier, D. S., Eid, M., & Lischetzke, T.
- Affiliations
- University of Geneva (CH)
- Freie Universität Berlin (DE)
- Venue
- Psychological Assessment
- Published
- 2012-09
- Volume
- 24
- Issue
- 3
- Pages
- 713-720
- ISSN
- 1040-3590 (ISSN-L)
- 1939-134X
- OpenAlex
- W2027102872
- Cited by
- 153
- References
- 31
- Citations by year
- 2026: 7 · 2025: 16 · 2024: 9 · 2023: 11 · 2022: 10 · 2021: 21 · 2020: 23 · 2019: 5 · 2018: 13 · 2017: 5 · 2016: 13 · 2015: 5 · 2014: 6 · 2013: 7 · 2012: 2
- FWCI
- 8.57
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Assessment (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Swiss National Foundation
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Mental Health Research Topics
- Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
- Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
- Keywords
- ecological momentary assessment, compliance, ambulatory assessment, missing data, within-day variation, mood, well-being, wakefulness, anxiety, life satisfaction, personality
- MeSH terms
- Adolescent, Adult, Affect, Female, Humans, Longitudinal Studies, Male, Middle Aged, Personality, Personality Inventory, Surveys and Questionnaires, Research Design, Time Factors, Monitoring, Ambulatory, Cell Phone, Young Adult
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review