The DOI matches its Crossref record; 56 of the 57 references it cites were found in a registry or an index.
Ecological Momentary Assessment in Behavioral Research: Addressing Technological and Human Participant Challenges
Open accessAbstract
BACKGROUND: Ecological momentary assessment (EMA) assesses individuals' current experiences, behaviors, and moods as they occur in real time and in their natural environment. EMA studies, particularly those of longer duration, are complex and require an infrastructure to support the data flow and monitoring of EMA completion. OBJECTIVE: Our objective is to provide a practical guide to developing and implementing an EMA study, with a focus on the methods and logistics of conducting such a study. METHODS: The EMPOWER study was a 12-month study that used EMA to examine the triggers of lapses and relapse following intentional weight loss. We report on several studies that informed the implementation of the EMPOWER study: (1) a series of pilot studies, (2) the EMPOWER study's infrastructure, (3) training of study participants in use of smartphones and the EMA protocol and, (4) strategies used to enhance adherence to completing EMA surveys. RESULTS: The study enrolled 151 adults and had 87.4% (132/151) retention rate at 12 months. Our learning experiences in the development of the infrastructure to support EMA assessments for the 12-month study spanned several topic areas. Included were the optimal frequency of EMA prompts to maximize data collection without overburdening participants; the timing and scheduling of EMA prompts; technological lessons to support a longitudinal study, such as proper communication between the Android smartphone, the Web server, and the database server; and use of a phone that provided access to the system's functionality for EMA data collection to avoid loss of data and minimize the impact of loss of network connectivity. These were especially important in a 1-year study with participants who might travel. It also protected the data collection from any server-side failure. Regular monitoring of participants' response to EMA prompts was critical, so we built in incentives to enhance completion of EMA surveys. During the first 6 months of the 12-month study interval, adherence to completing EMA surveys was high, with 88.3% (66,978/75,888) completion of random assessments and around 90% (23,411/25,929 and 23,343/26,010) completion of time-contingent assessments, despite the duration of EMA data collection and challenges with implementation. CONCLUSIONS: This work informed us of the necessary preliminary steps to plan and prepare a longitudinal study using smartphone technology and the critical elements to ensure participant engagement in the potentially burdensome protocol, which spanned 12 months. While this was a technology-supported and -programmed study, it required close oversight to ensure all elements were functioning correctly, particularly once human participants became involved.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“However, these barriers have fallen with the explosion of smartphones and multitude of low-cost EMA options;99 we argue that a tipping point has been reached, and the time to shift toward EMA is now.”
From Precision clinical trials: a framework for getting to precision medicine for neurobehavioural disorders (2021) · cites this paper as [99]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Ecological momentary assessment
Shiffman, S., Stone, A. A., & Hufford, M. R. (2008) · Annual Review of Clinical Psychology · doi:10.1146/annurev.clinpsy.3.022806.091415 · Cited by 6,454
Self-citationRelapse Prevention: Maintenance Strategies in the Treatment of Addictive Behaviors
Washton, A. M. (1986) · Journal of Studies on Alcohol · doi:10.15288/jsa.1986.47.260 · Cited by 2,531
Long-term weight loss maintenance
Wing, R. R., & Phelan, S. (2005) · The American Journal of Clinical Nutrition · doi:10.1093/ajcn/82.1.222s · Cited by 1,941 · cited 1×
“Relapse and weight regain are major issues in the treatment of obesity [16], yet knowledge of how relapses occur following intentional weight loss is rather limited.”
Self-monitoring in weight loss: a systematic review of the literature
Burke, L. E., Wang, J., & Sevick, M. A. (2011) · Journal of the American Dietetic Association · doi:10.1016/j.jada.2010.10.008 · Cited by 1,203 · cited 1×
“The core behavioral change strategies are based on social cognitive theory and include goal setting, self-monitoring, cognitive restructuring, self-efficacy enhancement, and social support with feedback and guidance provided by interventionists to assist with development of problem-solving skills [23-28].”
Self-citationPatient compliance with paper and electronic diaries
Stone, A. A., Shiffman, S., Schwartz, J. E., Broderick, J. E., & Hufford, M. R. (2003) · Controlled Clinical Trials · doi:10.1016/s0197-2456(02)00320-3 · Cited by 910
Self-citationA self-regulation program for maintenance of weight loss
Wing, R. R., Tate, D. F., Gorin, A. A., Raynor, H. A., & Fava, J. L. (2006) · New England Journal of Medicine · doi:10.1056/nejmoa061883 · Cited by 830
Patient non-compliance with paper diaries
Stone, A. A., Shiffman, S., Schwartz, J. E., Broderick, J. E., & Hufford, M. R. (2002) · BMJ · doi:10.1136/bmj.324.7347.1193 · Cited by 810
Self-citationSelf‐Recording of Everyday Life Events: Origins, Types, and Uses
Wheeler, L., & Reis, H. (1991) · Journal of Personality · doi:10.1111/j.1467-6494.1991.tb00252.x · Cited by 697
Behavioral treatment of obesity
Butryn, M. L., Webb, V., & Wadden, T. A. (2011) · Psychiatric Clinics of North America · doi:10.1016/j.psc.2011.08.006 · Cited by 636
Negative Affect and Smoking Lapses: A Prospective Analysis.
Shiffman, S., & Waters, A. J. (2004) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.72.2.192 · Cited by 427
Self-citationImmediate antecedents of cigarette smoking: An analysis from ecological momentary assessment.
Shiffman, S., Gwaltney, C. J., Balabanis, M. H., Liu, K. S., Paty, J. A., Kassel, J. D., Hickcox, M., & Gnys, M. (2002) · Journal of Abnormal Psychology · doi:10.1037//0021-843x.111.4.531 · Cited by 405
Self-citationRelapse prevention training and problem-solving therapy in the long-term management of obesity.
Perri, M. G., Nezu, A. M., McKelvey, W. F., Shermer, R. L., Renjilian, D. A., & Viegener, B. J. (2001) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.69.4.722 · Cited by 402
Burke, L. E., Conroy, M. B., Sereika, S. M., Elci, O. U., Styn, M. A., Acharya, S. D., Sevick, M. A., Ewing, L. J., & Glanz, K. (2011) · Obesity · doi:10.1038/oby.2010.208 · Cited by 329
Self-citationWeight Regain in U.S. Adults Who Experienced Substantial Weight Loss, 1999–2002
Weiss, E. C., Galuska, D. A., Kettel Khan, L., Gillespie, C., & Serdula, M. K. (2007) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2007.02.040 · Cited by 307
Stone, A. A., Broderick, J. E., Schwartz, J. E., Shiffman, S., Litcher-Kelly, L., & Calvanese, P. (2003) · Pain · doi:10.1016/s0304-3959(03)00040-x · Cited by 293
Self-citationRandomized Clinical Trials of Weight Loss Maintenance: A Review
Turk, M. W., Yang, K., Hravnak, M., Sereika, S. M., Ewing, L. J., & Burke, L. E. (2009) · Journal of Cardiovascular Nursing · doi:10.1097/01.jcn.0000317471.58048.32 · Cited by 258
Self-citationShiffman, S. M., Hickcox, M., Paty, J. A., Gnys, M., Kassel, J. D., & Richards, T. J. (1996) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.64.5.993 · Cited by 258
Self-citationShiffman, S., Hickcox, M., Paty, J. A., Gnys, M., Kassel, J. D., & Richards, T. J. (1996) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.64.5.993 · Cited by 254
Self-citationShiffman, S., Balabanis, M. H., Gwaltney, C., Paty, J., Gnys, M., Kassel, J. D., Hickcox, M., & Paton, S. M. (2007) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2007.05.017 · Cited by 219
Self-citationChanges in self-efficacy and dietary adherence: the impact on weight loss in the PREFER study
Warziski, M., Sereika, S. M., Styn, M. A., Music, E., & Burke, L. E. (2007) · Journal of Behavioral Medicine · doi:10.1007/s10865-007-9135-2 · Cited by 199
Self-citationDynamic Influences on Smoking Relapse Process
Shiffman, S. (2005) · Journal of Personality · doi:10.1111/j.0022-3506.2005.00364.x · Cited by 191
Self-citationEcological momentary assessment of stressful events and negative affect in bulimia nervosa.
Goldschmidt, A. B., Wonderlich, S. A., Crosby, R. D., Engel, S. G., Lavender, J. M., Peterson, C. B., Crow, S. J., Cao, L., & Mitchell, J. E. (2013) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0034974 · Cited by 179
Relapse crises and coping among dieters.
Grilo, C. M., Shiffman, S., & Wing, R. R. (1989) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.57.4.488 · Cited by 175
Self-citationWeight-Loss Diets for the Prevention and Treatment of Obesity
Katan, M. (2009) · New England Journal of Medicine · doi:10.1056/nejme0810291 · Cited by 166
Introduction to the special section: Ecological momentary assessment in health psychology.
Shiffman, S. M., & Stone, A. A. (1998) · Health Psychology · doi:10.1037/h0092706 · Cited by 157
Self-citationModeling the complexity of post-treatment drinking: It's a rocky road to relapse
Witkiewitz, K., & Marlatt, G. A. (2007) · Clinical Psychology Review · doi:10.1016/j.cpr.2007.01.002 · Cited by 146
Burke, L. E., Styn, M. A., Glanz, K., Ewing, L. J., Elci, O. U., Conroy, M. B., Sereika, S. M., Acharya, S. D., Music, E., Keating, A. L., et al. (2009) · Contemporary Clinical Trials · doi:10.1016/j.cct.2009.07.003 · Cited by 121
Self-citationA Momentary Sampling Study of the Affective Experience Following Coital Events in Adolescents
Shrier, L. A., Shih, M., Hacker, L., & de Moor, C. (2007) · Journal of Adolescent Health · doi:10.1016/j.jadohealth.2006.10.014 · Cited by 108
Natural history of nicotine withdrawal
Shiffman, S., Patten, C., Gwaltney, C., Paty, J., Gnys, M., Kassel, J., Hickcox, M., Waters, A., & Balabanis, M. (2006) · Addiction · doi:10.1111/j.1360-0443.2006.01635.x · Cited by 104
Self-citationMomentary Assessment of Adults’ Physical Activity and Sedentary Behavior: Feasibility and Validity
Dunton, G. F., Liao, Y., Kawabata, K., & Intille, S. S. (2012) · Frontiers in Psychology · doi:10.3389/fpsyg.2012.00260 · Cited by 103
Hair cortisol and self-reported stress in healthy, working adults
Gidlow, C. J., Randall, J., Gillman, J. C., Silk, S., & Jones, M. V. (2015) · Psychoneuroendocrinology · doi:10.1016/j.psyneuen.2015.09.022 · Cited by 97
Stress-induced laboratory eating behavior in obese women with binge eating disorder
Schulz, S., & Laessle, R. G. (2011) · Appetite · doi:10.1016/j.appet.2011.12.007 · Cited by 92
Liao, Y., Intille, S. S., & Dunton, G. F. (2014) · International Journal of Behavioral Medicine · doi:10.1007/s12529-014-9400-z · Cited by 89
Zenk, S. N., Horoi, I., McDonald, A. R., Corte, C. M., Riley, B., & Odoms‐Young, A. M. (2014) · Appetite · doi:10.1016/j.appet.2014.09.008 · Cited by 84
Using Smartphone Apps to Promote Psychiatric and Physical Well-Being
Macias, C., Panch, T., Hicks, Y. M., Scolnick, J. S., Weene, D. L., Öngür, D., & Cohen, B. M. (2015) · Psychiatric Quarterly · doi:10.1007/s11126-015-9337-7 · Cited by 76
An Ecological Momentary Assessment of Lapse Occurrences in Dieters
McKee, H. C., Ntoumanis, N., & Taylor, I. M. (2014) · Annals of Behavioral Medicine · doi:10.1007/s12160-014-9594-y · Cited by 73
Self-Monitoring as a Mediator of Weight Loss in the SMART Randomized Clinical Trial
Turk, M. W., Elci, O. U., Wang, J., Sereika, S. M., Ewing, L. J., Acharya, S. D., Glanz, K., & Burke, L. E. (2013) · International Journal of Behavioral Medicine · doi:10.1007/s12529-012-9259-9 · Cited by 66
Self-citationDunbar, M. S., Scharf, D., Kirchner, T., & Shiffman, S. (2010) · Nicotine & Tobacco Research · doi:10.1093/ntr/ntp198 · Cited by 64
Self-citationShiffman, S. S., Shumaker, S. A., Abrams, D. B., Cohen, S., Garvey, A. J., Grunberg, N. E., & Swan, G. E. (1986) · PubMed · Cited by 57
Self-citationSmoking Relapse Situations: A Preliminary Typology
Shiftman, S., Read, L. L., & Jarvik, M. E. (1985) · International Journal of the Addictions · doi:10.3109/10826088509044913 · Cited by 55
Burke, L. E., Choo, J., Music, E., Warziski, M., Styn, M. A., Kim, Y., & Sevick, M. A. (2005) · Contemporary Clinical Trials · doi:10.1016/j.cct.2005.08.002 · Cited by 50
Self-citationTask Force 2: Models of smoking relapse.
Shiffman, S. S., Shumaker, S. A., Abrams, D. B., Cohen, S., & al, E. (1986) · Health Psychology · doi:10.1037/0278-6133.5.suppl.13 · Cited by 48
Self-citationThe SELF trial: A self-efficacy-based behavioral intervention trial for weight loss maintenance
Burke, L. E., Ewing, L. J., Ye, L., Styn, M. A., Zheng, Y., Music, E., Loar, I., Mancino, J., Imes, C. C., Hu, L., et al. (2015) · Obesity · doi:10.1002/oby.21238 · Cited by 45
Self-citationStefano, E. C., Hudson, D. L., Whisenhunt, B., Buchanan, E., & Latner, J. (2016) · Eating Behaviors · doi:10.1016/j.eatbeh.2016.03.026 · Cited by 40
van Os, J., Delespaul, P., Barge‐Schaapveld, D. Q. C. M., & Bakker, P. R. (2014) · PLoS ONE · doi:10.1371/journal.pone.0115254 · Cited by 39
Bromberg, M. H., Connelly, M. A., Anthony, K. K., Gil, K. M., & Schanberg, L. E. (2015) · Clinical Journal of Pain · doi:10.1097/ajp.0000000000000298 · Cited by 36
Real-Time Assessment of the Effect of Biofeedback Therapy with Migraine: A Pilot Study
Odawara, M., Hashizume, M., Yoshiuchi, K., & Tsuboi, K. (2015) · International Journal of Behavioral Medicine · doi:10.1007/s12529-015-9469-z · Cited by 28
Physical Activity, Mind Wandering, Affect, and Sleep: An Ecological Momentary Assessment
Fanning, J. T., MacKenzie, M. J., Roberts, S. A., Crato, I., Ehlers, D. K., & McAuley, E. (2016) · JMIR mhealth and uhealth · doi:10.2196/mhealth.5855 · Cited by 27
Coping with dietary relapse crises and their aftermath
Grilo, C. M., Shiffman, S. M., & Wing, R. R. (1993) · Addictive Behaviors · doi:10.1016/0306-4603(93)90012-x · Cited by 26
Self-citationModelling the effects of partially observed covariates on Poisson process intensity
Rathbun, S. L., Shiffman, S. S., & Gwaltney, C. J. (2007) · Biometrika · doi:10.1093/biomet/asm009 · Cited by 25
Self-citationCohn, A. M., Brandon, T. H., Armeli, S., Ehlke, S., & Bowers, M. (2015) · BMJ Open · doi:10.1136/bmjopen-2014-007046 · Cited by 12
Inada, S., Yoshiuchi, K., Iizuka, Y., Ohashi, K., Kikuchi, H., Yamamoto, Y., Kadowaki, T., & Akabayashi, A. (2016) · International Journal of Behavioral Medicine · doi:10.1007/s12529-016-9535-1 · Cited by 12
Social contexts of momentary craving to smoke among Korean American emerging adults
Huh, J., Cerrada, C. J., Kirkpatrick, M. G., Dunton, G. F., & Leventhal, A. M. (2016) · Addictive Behaviors · doi:10.1016/j.addbeh.2016.01.006 · Cited by 10
Ehlers, D. K., Huberty, J., Buman, M. P., Hooker, S. P., Todd, M. W., & de Vreede, G. (2015) · Journal of Physical Activity and Health · doi:10.1123/jpah.2015-0059 · Cited by 10
Neustifter, B., Rathbun, S. L., & Shiffman, S. S. (2011) · Journal of Applied Statistics · doi:10.1080/02664763.2011.626848 · Cited by 5
Self-citationZheng, Y., Sereika, S. M., Imes, C. C., Ye, L., Mattos, M. K., Mancino, J., Danford, C. A., & Burke, L. E. (2014) · Circulation · doi:10.1161/circ.129.suppl_1.07 · Cited by 1
Self-citation
Flagged first, then by citations
1 of the 57 references couldn’t be matched to a known paper.
Research works printed but not linked: 6
- Shiffman, S, 2000, The Science of Self Report: Implication for Research and Practice, 277
- Grilo, CM, 1989, J Consult Clin Psychol, 57, 488, 10.1037/0022-006X.57.4.488 · DOI
- Wing, R, 2004, Handbook of Obesity: Clinical Applications. 2nd edition, 147
- Music, E, 2013, Ann Behav Med, 45, s123
- Burke, L, 2010, Obesity, 18, s160
- Shiffman, S, 2002, J Abnorm Psychol, 111, 531, 10.1037/0021-843X.111.4.531 · DOI · Paper page
Cited by
Sorted by citations
Short, C. E., DeSmet, A., Woods, C., Williams, S. L., Maher, C., Middelweerd, A., Müller, A. M., Wark, P. A., Vandelanotte, C., Poppe, L., et al. (2018) · Journal of Medical Internet Research · doi:10.2196/jmir.9397 · Cited by 400
Trull, T. J., & Ebner‐Priemer, U. (2019) · Journal of Abnormal Psychology · doi:10.1037/abn0000473 · Cited by 383
Compliance with ecological momentary assessment protocols in substance users: a meta-analysis
Jones, A., Remmerswaal, D., Verveer, I., Robinson, E., Franken, I. H. A., Wen, C. K. F., & Field, M. (2019) · Addiction · doi:10.1111/add.14503 · Cited by 319
The common sense model of illness self-regulation: a conceptual review and proposed extended model
Hagger, M. S., & Orbell, S. (2021) · Health Psychology Review · doi:10.1080/17437199.2021.1878050 · Cited by 378
The Design of Ecological Momentary Assessment Technologies
Doherty, K., Balaskas, A., & Doherty, G. J. (2020) · Interacting with Computers · doi:10.1093/iwcomp/iwaa019 · Cited by 202
Metadata
- Authors
- Burke, L. E., Shiffman, S., Music, E., Styn, M. A., Kriska, A., Smailagic, A., Siewiorek, D., Ewing, L. J., Chasens, E., French, B., Mancino, J., Mendez, D., Strollo, P., & Rathbun, S. L.
- Affiliations
- University of Pittsburgh (US)
- Carnegie Mellon University (US)
- University of Georgia (US)
- Corresponding author
- Burke, Lora E
- ORCID
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2017-03-15
- Volume
- 19
- Issue
- 3
- Pages
- e77
- ISSN
- 1438-8871 (ISSN-L)
- PMC
- PMC5371716
- OpenAlex
- W2601018405
- Cited by
- 267
- References
- 57
- Citations by year
- 2026: 35 · 2025: 25 · 2024: 33 · 2023: 26 · 2022: 31 · 2021: 45 · 2020: 31 · 2019: 21 · 2018: 18 · 2017: 1
- FWCI
- 25.28
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Journal of Medical Internet Research (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Funders
- National Heart, Lung, and Blood Institute (R01HL107370)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Mental Health Research Topics
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Keywords
- ecological momentary assessment, smartphone data collection, behavioral interventions, participant retention, network connectivity, participant training, pilot study, participant engagement
- MeSH terms
- Smartphone, Ecological Momentary Assessment, Adult, Female, Humans, Male, Behavioral Research, Weight Reduction Programs
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
