Ecological Momentary Assessment: A Meta-Analysis on Designs, Samples, and Compliance Across Research Fields
Open accessAbstract
Ecological Momentary Assessments (i.e., EMA, repeated assessments in daily life) are widespread in many fields of psychology and related disciplines. Yet, little knowledge exists on how differences in study designs and samples predict study compliance and dropout—two central parameters of data quality in (micro-)longitudinal research. The current meta-analysis included k = 477 articles (496 samples, total N = 677,536). For each article, we coded the design, sample characteristics, compliance, and dropout rate. The results showed that on average EMA studies scheduled six assessments per day, lasted for 7 days, and obtained a compliance of 79%. Studies with more assessments per day scheduled fewer assessment days, yet, the number of assessments did not predict compliance or dropout rates. Compliance was significantly higher in studies providing financial incentives. Otherwise, design or sample characteristics had little effects. We discuss the implications of the findings for planning, reporting, and reviewing EMA studies.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Most of the studies (54/70, 77%) paid the participants to enhance the compliance rates, which turned out to be in line with the average compliance in AA literature (approximately 75%).
“In addition, most of the studies (54/70, 77%) paid the participants to enhance the compliance rates, which turned out to be in line with the average compliance in AA literature (approximately 75% [108]).”
From Studies of Social Anxiety Using Ambulatory Assessment: Systematic Review (2024) · cites this paper as [108]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Shamseer, L., Moher, D., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., Stewart, L. A., & the PRISMA-P Group (2015) · BMJ · doi:10.1136/bmj.g7647 · Cited by 13,120 · cited 2× · see citing works
“We designed and reported the results of the meta-analysis in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist (Shamseer et al., 2015).”
MethodIdentifying careless responses in survey data.
Meade, A. W., & Craig, S. B. (2012) · Psychological Methods · doi:10.1037/a0028085 · Cited by 3,865 · see citing works
Hunter, J., & Schmidt, F. (2004) · doi:10.4135/9781412985031 · Cited by 3,023 · see citing works
The interpretation of interaction in contingency tables
Simpson, E. H. (1951) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1951.tb00088.x · Cited by 2,301 · cited 1× · see citing works
“It needs to be acknowledged although that inferring such recommendations for individual studies still entails uncertainty because the current findings refer to the level of a population of studies (e.g., see the Simpson paradox, Kievit et al., 2013; Simpson, 1951).”
BackgroundHandbook of research methods for studying daily life
Mehl, M. R., Conner, T. S., & Csíkszentmihályi, M. (2012) · Choice Reviews Online · doi:10.5860/choice.50-1159 · Cited by 2,121 · see citing works
Molenaar, P. C. M. (2004) · Measurement: Interdisciplinary Research & Perspective · doi:10.1207/s15366359mea0204_1 · Cited by 1,943 · see citing works
Belief and feeling: Evidence for an accessibility model of emotional self-report.
Robinson, M. D., & Clore, G. L. (2002) · Psychological Bulletin · doi:10.1037/0033-2909.128.6.934 · Cited by 1,806 · see citing works
Appelbaum, M., Cooper, H., Kline, R. B., Mayo-Wilson, E., Nezu, A. M., & Rao, S. M. (2018) · American Psychologist · doi:10.1037/amp0000191 · Cited by 1,449 · see citing works
Schmitt, D. P., Realo, A., Voracek, M., & Allik, J. (2008) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.94.1.168 · Cited by 1,391 · see citing works
Kleiman, E. M., Turner, B. J., Fedor, S., Beale, E. E., Huffman, J. C., & Nock, M. K. (2017) · Journal of Abnormal Psychology · doi:10.1037/abn0000273 · Cited by 831 · see citing works
Shcherbina, A., Mattsson, C., Waggott, D., Salisbury, H., Christle, J., Hastie, T., Wheeler, M., & Ashley, E. (2017) · Journal of Personalized Medicine · doi:10.3390/jpm7020003 · Cited by 625 · see citing works
Feelings change: Accounting for individual differences in the temporal dynamics of affect.
Kuppens, P., Oravecz, Z., & Tuerlinckx, F. (2010) · Journal of Personality and Social Psychology · doi:10.1037/a0020962 · Cited by 543 · see citing works
Simpson's paradox in psychological science: a practical guide
Kievit, R. A., Frankenhuis, W. E., Waldorp, L. J., & Borsboom, D. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00513 · Cited by 542 · see citing works
Harari, G. M., Lane, N. D., Wang, R., Crosier, B. S., Campbell, A. T., & Gosling, S. D. (2016) · Perspectives on Psychological Science · doi:10.1177/1745691616650285 · Cited by 515 · see citing works
Eisele, G., Vachon, H., Lafit, G., Kuppens, P., Houben, M., Myin-Germeys, I., & Viechtbauer, W. (2022) · Assessment · doi:10.1177/1073191120957102 · Cited by 459 · see citing works
Beal, D. J. (2015) · Annual Review of Organizational Psychology and Organizational Behavior · doi:10.1146/annurev-orgpsych-032414-111335 · Cited by 436 · see citing works
The Experience Sampling Method on Mobile Devices
van Berkel, N., Ferreira, D., & Kostakos, V. (2018) · ACM Computing Surveys · doi:10.1145/3123988 · Cited by 432 · see citing works
McShane, B. B., Böckenholt, U., & Hansen, K. T. (2016) · Perspectives on Psychological Science · doi:10.1177/1745691616662243 · Cited by 398 · see citing works
Wen, C. K. F., Schneider, S., Stone, A. A., & Spruijt-Metz, D. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.6641 · Cited by 350 · see citing works
Trull, T. J., & Ebner‐Priemer, U. (2019) · Journal of Abnormal Psychology · doi:10.1037/abn0000473 · Cited by 350 · see citing works
Riediger, M., Schmiedek, F., Wagner, G. G., & Lindenberger, U. (2009) · Psychological Science · doi:10.1111/j.1467-9280.2009.02473.x · Cited by 325 · see citing works
Compliance with ecological momentary assessment protocols in substance users: a meta-analysis
Jones, A., Remmerswaal, D., Verveer, I., Robinson, E., Franken, I. H. A., Wen, C. K. F., & Field, M. (2019) · Addiction · doi:10.1111/add.14503 · Cited by 319 · see citing works
Stone, A. A., Broderick, J. E., Schwartz, J. E., Shiffman, S., Litcher-Kelly, L., & Calvanese, P. (2003) · Pain · doi:10.1016/s0304-3959(03)00040-x · Cited by 293 · see citing works
Heron, K. E., Everhart, R. S., McHale, S. M., & Smyth, J. M. (2017) · Journal of Pediatric Psychology · doi:10.1093/jpepsy/jsx078 · Cited by 283 · see citing works
Vachon, H., Viechtbauer, W., Rintala, A., & Myin-Germeys, I. (2019) · Journal of Medical Internet Research · doi:10.2196/14475 · Cited by 252 · see citing works
Ecological momentary assessment methodology in chronic pain research: a systematic review
May, M., Junghaenel, D. U., Ono, M., Stone, A. A., & Schneider, S. (2018) · The Journal of Pain · doi:10.1016/j.jpain.2018.01.006 · Cited by 247 · see citing works
The Electronically Activated Recorder (EAR)
Mehl, M. R. (2017) · Current Directions in Psychological Science · doi:10.1177/0963721416680611 · Cited by 246 · see citing works
The stability of mood variability: A spectral analytic approach to daily mood assessments.
Larsen, R. J. (1987) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.52.6.1195 · Cited by 219 · see citing works
Lab And/Or Field? Measuring Personality Processes and Their Social Consequences
Wrzus, C., & Mehl, M. R. (2015) · European Journal of Personality · doi:10.1002/per.1986 · Cited by 200 · see citing works
Self-citationvan Roekel, E., Keijsers, L., & Chung, J. M. (2019) · Journal of Research on Adolescence · doi:10.1111/jora.12471 · Cited by 191 · see citing works
Initial elevation bias in subjective reports
Shrout, P. E., Stadler, G., Lane, S. P., McClure, M. J., Jackson, G. L., Clavél, F. D., Iida, M., Gleason, M. E. J., Xu, J. H., & Bolger, N. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1712277115 · Cited by 190 · see citing works
Compliance with momentary pain measurement using electronic diaries: A systematic review
Morren, M., van Dulmen, S., Ouwerkerk, J. W., & Bensing, J. M. (2008) · European Journal of Pain · doi:10.1016/j.ejpain.2008.05.010 · Cited by 177 · see citing works
Response compliance and predictors thereof in studies using the experience sampling method.
Rintala, A., Wampers, M., Myin‐Germeys, I., & Viechtbauer, W. (2018) · Psychological Assessment · doi:10.1037/pas0000662 · Cited by 176 · see citing works
Intensive Longitudinal Methods
Wade, T. D. (2017) · Springer eBooks · doi:10.1007/978-981-287-104-6_100140 · Cited by 173 · see citing works
Collins, L. M., & Graham, J. A. (2002) · Drug and Alcohol Dependence · doi:10.1016/s0376-8716(02)00217-x · Cited by 169 · see citing works
Analyzing developmental processes on an individual level using nonstationary time series modeling.
Molenaar, P. C. M., Sinclair, K. O., Rovine, M. J., et al. (2009) · Developmental Psychology · doi:10.1037/a0014170 · Cited by 156 · see citing works
Courvoisier, D. S., Eid, M., & Lischetzke, T. (2012) · Psychological Assessment · doi:10.1037/a0026733 · Cited by 153 · see citing works
μEMA: Microinteraction-based ecological momentary assessment (EMA) using a smartwatch
Intille, S., Haynes, C., Maniar, D., Ponnada, A., & Manjourides, J. (2016) · Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing · doi:10.1145/2971648.2971717 · Cited by 130 · see citing works
Doherty, S. T., Lemieux, C. J., & Canally, C. (2014) · Social Science & Medicine · doi:10.1016/j.socscimed.2014.01.048 · Cited by 110 · see citing works
Ono, M., Schneider, S., Junghaenel, D. U., & Stone, A. A. (2018) · Journal of Medical Internet Research · doi:10.2196/11398 · Cited by 109 · see citing works
A qualitative approach to guide choices for designing a diary study
Janssens, K. A. M., Bos, E. H., Rosmalen, J. G. M., Wichers, M. C., & Riese, H. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-018-0579-6 · Cited by 105 · see citing works
Neubauer, A. B., Scott, S., Sliwinski, M. J., & Smyth, J. M. (2019) · Journal of Personality and Social Psychology · doi:10.1037/pspp0000248 · Cited by 101 · see citing works
Self-citationSilvia, P. J., Kwapil, T. R., Eddington, K. M., & Brown, L. H. (2013) · Social Science Computer Review · doi:10.1177/0894439313479902 · Cited by 100 · see citing works
Wrzus, C., Müller, V., Wagner, G. G., Lindenberger, U., & Riediger, M. (2012) · Developmental Psychology · doi:10.1037/a0028325 · Cited by 96 · see citing works
Self-citationvan Berkel, N., Gonçalves, J., Lovén, L., Ferreira, D., Hosio, S. J., & Kostakos, V. (2018) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2018.12.002 · Cited by 96 · see citing works
A direct comparison of the day reconstruction method (DRM) and the experience sampling method (ESM).
Lucas, R. E., Wallsworth, C., Anusic, I., & Donnellan, M. B. (2020) · Journal of Personality and Social Psychology · doi:10.1037/pspp0000289 · Cited by 96 · see citing works
TaskMe: Toward a dynamic and quality-enhanced incentive mechanism for mobile crowd sensing
Guo, B., Chen, H., Yu, Z., et al. (2016) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2016.09.002 · Cited by 90 · see citing works
Himmelstein, P. H., Woods, W. C., & Wright, A. G. C. (2019) · Psychological Assessment · doi:10.1037/pas0000718 · Cited by 82 · see citing works
Experiments in the Wild: Introducing the Within-Person Encouragement Design
Schmiedek, F., & Neubauer, A. B. (2019) · Multivariate Behavioral Research · doi:10.1080/00273171.2019.1627660 · Cited by 77 · see citing works
Self-citationMcCarthy, D. E., Minami, H., Yeh, V. M., & Bold, K. W. (2015) · Addiction · doi:10.1111/add.12996 · Cited by 76 · see citing works
Riediger, M., Wrzus, C., Schmiedek, F., Wagner, G. G., & Lindenberger, U. (2011) · Emotion · doi:10.1037/a0022756 · Cited by 75 · see citing works
Self-citationCarpenter, R. W., Wycoff, A. M., & Trull, T. J. (2016) · Assessment · doi:10.1177/1073191116632341 · Cited by 72 · see citing works
Hasselhorn, K., Ottenstein, C., & Lischetzke, T. (2021) · Behavior Research Methods · doi:10.3758/s13428-021-01683-6 · Cited by 72 · see citing works
Ambulatory Assessment in Lifespan Psychology
Hoppmann, C. A., & Riediger, M. (2009) · European Psychologist · doi:10.1027/1016-9040.14.2.98 · Cited by 65 · see citing works
Everyday Empathic Accuracy in Younger and Older Couples
Rauers, A., Blanke, E. S., & Riediger, M. (2013) · Psychological Science · doi:10.1177/0956797613490747 · Cited by 57 · see citing works
Ashurst, J., van Woerden, I., Dunton, G. F., Todd, M. W., Ohri‐Vachaspati, P., Swan, P. D., & Bruening, M. M. (2018) · BMC Public Health · doi:10.1186/s12889-018-5447-0 · Cited by 56 · see citing works
Moving in Sync: Hourly Physical Activity and Sedentary Behavior are Synchronized in Couples
Pauly, T., Keller, J., Knoll, N., Michalowski, V. I., Hohl, D. H., Ashe, M. C., Gerstorf, D., Madden, K. M., & Hoppmann, C. A. (2019) · Annals of Behavioral Medicine · doi:10.1093/abm/kaz019 · Cited by 54 · see citing works
Ottenstein, C., & Werner, L. (2022) · Assessment · doi:10.1177/10731911211032718 · Cited by 52 · see citing works
Lange, S., & Süß, H. (2014) · Consciousness and Cognition · doi:10.1016/j.concog.2013.12.008 · Cited by 49 · see citing works
Noninvasive Psychophysiological Ambulatory Recordings
Houtveen, J. H., & de Geus, E. J. C. (2009) · European Psychologist · doi:10.1027/1016-9040.14.2.132 · Cited by 46 · see citing works
The stability of social interaction.
Nezlek, J. B. (1993) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.65.5.930 · Cited by 45 · see citing works
Neubauer, A. B., Lerche, V., & Voß, A. (2017) · Journal of Personality · doi:10.1111/jopy.12351 · Cited by 45 · see citing works
Self-citationNew Frontiers in Ambulatory Assessment
Timmons, A. C., Baucom, B., Han, S. C., Perrone, L., Chaspari, T., Narayanan, S. S., & Margolin, G. (2017) · Social Psychological and Personality Science · doi:10.1177/1948550617709115 · Cited by 44 · see citing works
Soyster, P. D., Bosley, H. G., Reeves, J. W., Altman, A. D., & Fisher, A. J. (2019) · Journal for Person-Oriented Research · doi:10.17505/jpor.2019.06 · Cited by 33 · see citing works
McKee, S. E., Smith, H. J., Koch, A., Balzarini, R. N., Georges, M., & Callahan, M. P. (2013) · Psychology of Women Quarterly · doi:10.1177/0361684312469792 · Cited by 31 · see citing works
Epstein, D. H., & Preston, K. L. (2012) · The American Journal on Addictions · doi:10.1111/j.1521-0391.2012.00230.x · Cited by 28 · see citing works
Ludwigs, K., Lucas, R. E., Veenhoven, R., Richter, D., & Arends, L. R. (2019) · Applied Research in Quality of Life · doi:10.1007/s11482-019-09723-2 · Cited by 14 · see citing works
Flagged first, then by citations
Research works printed but not linked: 5
- Hufford M. R. Shields A. L. (2002). Electronic diaries. Applied Clinical Trials, 11(4), 46– 56.
- Mehl M. R. Conner T. (Eds.). (2012). Handbook of research methods for studying daily life. Guilford Press.
- Reis H. T. (2012). Why researchers should think “real-world”: A conceptual rationale. In Mehl M. R. Conner T. (Eds.), Handbook of research methods for studying daily life (pp. 3– 21). Guilford Press.
- Schwarz N. (2012). Why researchers should think “real-time”: A cognitive rationale. In Mehl M. R. Conner T. (Eds.), Handbook of research methods for studying daily life (pp. 22– 42). Guilford Press.
- van Laerhoven K. (2022). Beyond the smartphone I: The future of wearables as mobile sensors. In Mehl M. R. Eid M. Wrzus C. Harari G. Ebner-Priemer U. W. (Eds.), Mobile Sensing in Psychology: Methods and Applications. New York: Guilford.
Cited by
Sorted by citations
Studies of Social Anxiety Using Ambulatory Assessment: Systematic Review
Fernández-Álvarez, J., Colombo, D., Gómez Penedo, J. M., Pierantonelli, M., Baños, R. M., & Botella, C. (2024) · JMIR Mental Health · doi:10.2196/46593 · Cited by 8 · see citing works
- Authors
- Wrzus, C., & Neubauer, A. B.
- Affiliations
- Heidelberg University (DE)
- DIPF | Leibniz Institute for Research and Information in Education (DE)
- Corresponding author
- Wrzus, Cornelia
- Venue
- Assessment
- Publisher
- SAGE Publications
- Published
- 2022-01-11
- Volume
- 30
- Issue
- 3
- Pages
- 825-846
- ISSN
- 1073-1911 (ISSN-L)
- 1552-3489
- PMC
- PMC9999286
- OpenAlex
- W4205268398
- Cited by
- 500
- References
- 67
- Citations by year
- 2026: 149 · 2025: 131 · 2024: 112 · 2023: 77 · 2022: 29
- FWCI
- 12.73
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Assessment (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Mental Health Research Topics
- Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Keywords
- ecological momentary assessment, study compliance, participant dropout, research design, sample characteristics, financial incentives, repeated assessment
- MeSH terms
- Data Accuracy, Ecological Momentary Assessment, Humans, Research Design, Knowledge
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
