Characterizing user engagement with health app data: a data mining approach
Open accessAbstract
The use of mobile health applications (apps) especially in the area of lifestyle behaviors has increased, thus providing unprecedented opportunities to develop health programs that can engage people in real-time and in the real-world. Yet, relatively little is known about which factors relate to the engagement of commercially available apps for health behaviors. This exploratory study examined behavioral engagement with a weight loss app, Lose It! and characterized higher versus lower engaged groups. Cross-sectional, anonymized data from Lose It! were analyzed (n = 12,427,196). This dataset was randomly split into 24 subsamples and three were used for this study (total n = 1,011,008). Classification and regression tree methods were used to identify subgroups of user engagement with one subsample, and descriptive analyses were conducted to examine other group characteristics associated with engagement. Data mining validation methods were conducted with two separate subsamples. On average, users engaged with the app for 29 days. Six unique subgroups were identified, and engagement for each subgroup varied, ranging from 3.5 to 172 days. Highly engaged subgroups were primarily distinguished by the customization of diet and exercise. Those less engaged were distinguished by weigh-ins and the customization of diet. Results were replicated in further analyses. Commercially-developed apps can reach large segments of the population, and data from these apps can provide insights into important app features that may aid in user engagement. Getting users to engage with a mobile health app is critical to the success of apps and interventions that are focused on health behavior change.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Being younger than 44 years was associated with a higher level of uptake and engagement than older adults.
“Being younger than 44 years was associated with a higher level of uptake and engagement [38,41,42,44,49,53,55,63,64] than older adults.”
From Influences on the Uptake of and Engagement With Health and Well-Being Smartphone Apps: Systematic Review (2020) · cites this paper as [63]
The Limitations section (page unavailable) reports that app users were younger on average than a nationally representative sample, and Data preparation notes that adults aged 65 and older are less likely to use health-related smartphone apps. These passages offer some support for greater uptake among younger people, but the paper does not establish a cutoff of 44 years or show that people below it had higher engagement than older adults. (LLM, conf=0.82)
Experimental — accuracy not yet measuredApps were more frequently downloaded by women than men, with the percentage ranging from 59% to 74%.
“Apps were more frequently downloaded by women than men, with the percentage ranging from 59% to 74% [38,41,49,53,55,63], although 1 study found that being male was associated with using an app to manage alcohol consumption [65].”
From Influences on the Uptake of and Engagement With Health and Well-Being Smartphone Apps: Systematic Review (2020) · cites this paper as [63]
The Results state that most app users were women, and the Limitations report that women made up 75% of the app sample. This supports the female-majority point, but the paper measures app users rather than downloads and its reported percentage falls outside the cited 59%–74% range. Page numbers are unavailable in our copy. (LLM, conf=0.86)
Experimental — accuracy not yet measuredA large study found that 30% of the most frequently engaged group customized the app more (eg, uploaded pictures) than the least engaged group (2%).
“Furthermore, a large study found that 30% of the most frequently engaged group customized the app more, for example, uploaded pictures, than the least engaged group (2%) [63].”
From Influences on the Uptake of and Engagement With Health and Well-Being Smartphone Apps: Systematic Review (2020) · cites this paper as [63]
Studies of over 1 million participants showed that power users demonstrated the greatest weight loss.
“Weight loss was the subject of other studies of over 1 million participants [129], which showed that power users demonstrated the greatest weight loss.”
From Human-centred artificial intelligence for mobile health sensing: challenges and opportunities (2023) · cites this paper as [129]
The longest reviewed observational study conducted by Serrano et al was 7 years long with over 1 million participants.
“The longest reviewed observational study conducted by Serrano et al was 7 years long with over 1 million participants [34].”
From A Library of Analytic Indicators to Evaluate Effective Engagement with Consumer mHealth Apps for Chronic Conditions: Scoping Review (2019) · cites this paper as [34]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970 · see citing works
The Elements of Statistical Learning
Hastie, T., Friedman, J., & Tibshirani, R. (2001) · Springer Series in Statistics · doi:10.1007/978-0-387-21606-5 · Cited by 24,162 · see citing works
Classification and Regression Trees.
Ryzin, J. V., Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1986) · Journal of the American Statistical Association · doi:10.2307/2288003 · Cited by 21,028 · see citing works
The Elements of Statistical Learning: Data Mining, Inference, and Prediction
Lu, Z. Q. J. (2010) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/j.1467-985x.2010.00646_6.x · Cited by 12,904 · see citing works
Classification and regression trees
Praagman, J. (1985) · European Journal of Operational Research · doi:10.1016/0377-2217(85)90321-2 · Cited by 10,053 · see citing works
Classification and regression trees
Loh, W. (2011) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.8 · Cited by 2,464 · see citing works
What is user engagement? A conceptual framework for defining user engagement with technology
O'Brien, H. L., & Toms, E. G. (2008) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.20801 · Cited by 1,397 · see citing works
Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery
(2011) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/(issn)1942-4795 · Cited by 1,007 · see citing works
An Introduction to Recursive Partitioning Using the RPART Routines
Therneau, T. M., & Atkinson, E. J. (2015) · Cited by 1,006 · see citing works
Lemon, S. C., Roy, J., Clark, M. A., Friedmann, P. D., & Rakowski, W. (2003) · Annals of Behavioral Medicine · doi:10.1207/s15324796abm2603_02 · Cited by 878 · see citing works
Kim, Y. H., Kim, D. J., & Wachter, K. (2013) · Decision Support Systems · doi:10.1016/j.dss.2013.07.002 · Cited by 781 · see citing works
Evaluating Medical Tests: Objective and Quantitative Guidelines.
Guthrie, D., & Kraemer, H. C. (1992) · Journal of the American Statistical Association · doi:10.2307/2290671 · Cited by 636 · see citing works
Personalization versus Customization: The Importance of Agency, Privacy, and Power Usage
Sundar, S. S., & Marathe, S. S. (2010) · Human Communication Research · doi:10.1111/j.1468-2958.2010.01377.x · Cited by 588 · see citing works
Patrick, K., Griswold, W. G., Raab, F., & Intille, S. S. (2008) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2008.05.001 · Cited by 479 · see citing works
mHealth 2.0: Experiences, Possibilities, and Perspectives
Becker, S., Miron-Shatz, T., Schumacher, N., Krocza, J., Diamantidis, C., & Albrecht, U.-V. (2014) · JMIR mHealth and uHealth · doi:10.2196/mhealth.3328 · Cited by 354 · see citing works
Laing, B. Y., Mangione, C. M., Tseng, C.-H., Leng, M., Vaisberg, E., Mahida, M., Bholat, M., Glazier, E., Morisky, D. E., & Bell, D. S. (2014) · Annals of Internal Medicine · doi:10.7326/m13-3005 · Cited by 353 · see citing works
The use of classification trees for bioinformatics
Chen, X. '., Wang, M., & Heping Zhang, D. (2011) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.14 · Cited by 235 · see citing works
An Evaluation of Mobile Health Application Tools
Sama, P. R., Eapen, Z. J., Weinfurt, K. P., Shah, B. R., & Schulman, K. A. (2014) · JMIR mHealth and uHealth · doi:10.2196/mhealth.3088 · Cited by 230 · see citing works
Marathe, S. S., & Sundar, S. S. (2011) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1978942.1979056 · Cited by 157 · see citing works
Atienza, A. A., & Patrick, K. (2011) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2011.01.008 · Cited by 76 · see citing works
Self-citationKing, A. C., Goldberg, J. H., Salmon, J., Owen, N. G., Dunstan, D. W., Weber, D., Doyle, C., & Robinson, T. N. (2009) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2009.08.032 · Cited by 69 · see citing works
Mining Health App Data to Find More and Less Successful Weight Loss Subgroups
Serrano, K. J., Yu, M., Coa, K. I., Collins, L. M., & Atienza, A. A. (2016) · Journal of Medical Internet Research · doi:10.2196/jmir.5473 · Cited by 53 · see citing works
Self-citationIdentifying Sedentary Subgroups
Atienza, A. A., Yaroch, A. L., Mâsse, L. C., Moser, R. P., Hesse, B. W., & King, A. C. (2006) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2006.07.024 · Cited by 30 · see citing works
Self-citationExploratory data mining using CART in the behavioral sciences.
McArdle, J. J. (2012) · American Psychological Association eBooks · doi:10.1037/13621-020 · Cited by 24 · see citing works
Yang, C., Maher, J. P., & Conroy, D. E. (2015) · Preventive Medicine Reports · doi:10.1016/j.pmedr.2015.08.009 · Cited by 14 · see citing works
Dunton, G. F., Atienza, A. A., Tscherne, J., & Rodriguez, D. (2011) · Pediatric Exercise Science · doi:10.1123/pes.23.1.106 · Cited by 8 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Challenges in Participant Engagement and Retention Using Mobile Health Apps: Literature Review
Amagai, S., Pila, S., Kaat, A. J., Nowinski, C. J., & Gershon, R. C. (2022) · Journal of Medical Internet Research · doi:10.2196/35120 · Cited by 343 · see citing works
Szinay, D., Jones, A., Chadborn, T., Brown, J., & Naughton, F. (2020) · Journal of Medical Internet Research · doi:10.2196/17572 · Cited by 289 · see citing works
“The focus of some studies was very specific and targeted a certain health behavior or condition, including alcohol reduction [38,46,54,58,59,64], smoking cessation [40,58,67,72,77], increasing physical activity [39,45,48,49,53,62,65,68], weight management [47,48,51,53,63,65,66,71,78], depression [52,61], mindfulness […”
BackgroundInfluentialBidargaddi, N., Almirall, D., Murphy, S., Nahum-Shani, I., Kovalcik, M., Pituch, T., Maaieh, H., & Strecher, V. (2018) · JMIR mHealth and uHealth · doi:10.2196/10123 · Cited by 189 · see citing works
Best practices for analyzing large-scale health data from wearables and smartphone apps
Hicks, J. L., Althoff, T., Sosic, R., Kuhar, P., Bostjancic, B., King, A. C., Leskovec, J., & Delp, S. L. (2019) · npj Digital Medicine · doi:10.1038/s41746-019-0121-1 · Cited by 183 · see citing works
Effect of self-monitoring on long-term patient engagement with mobile health applications
Lee, K., Kwon, H., Lee, B., Lee, G., Lee, J. H., Park, Y. R., & Shin, S.-Y. (2018) · PLOS ONE · doi:10.1371/journal.pone.0201166 · Cited by 132 · see citing works
Pham, Q., Graham, G., Carrion, C., Morita, P. P., Seto, E., Stinson, J. N., & Cafazzo, J. A. (2019) · JMIR mHealth and uHealth · doi:10.2196/11941 · Cited by 131 · see citing works
Edney, S., Ryan, J. C., Olds, T., Monroe, C., Fraysse, F., Vandelanotte, C., Plotnikoff, R., Curtis, R., & Maher, C. (2019) · Journal of Medical Internet Research · doi:10.2196/14645 · Cited by 104 · see citing works
What Works Best to Engage Participants in Mobile App Interventions and e-Health: A Scoping Review
Oakley-Girvan, I., Yunis, R., Longmire, M., & Ouillon, J. S. (2022) · Telemedicine and e-Health · doi:10.1089/tmj.2021.0176 · Cited by 92 · see citing works
Bidargaddi, N., Pituch, T., Maaieh, H., Short, C., & Strecher, V. (2018) · Preventive Medicine Reports · doi:10.1016/j.pmedr.2018.07.004 · Cited by 77 · see citing works
Human-centred artificial intelligence for mobile health sensing: challenges and opportunities
Dang, T., Spathis, D., Ghosh, A., & Mascolo, C. (2023) · Royal Society Open Science · doi:10.1098/rsos.230806 · Cited by 30 · see citing works
- Authors
- Serrano, K. J., Coa, K. I., Yu, M., Wolff-Hughes, D. L., & Atienza, A. A.
- Affiliations
- National Institutes of Health (US)
- ICF International (United States) (US)
- Corresponding author
- Serrano, Katrina J.
- Venue
- Translational Behavioral Medicine
- Publisher
- Oxford University Press (OUP)
- Published
- 2017-06
- Volume
- 7
- Issue
- 2
- Pages
- 277-285
- ISSN
- 1613-9860 (ISSN-L)
- 1869-6716
- PMC
- PMC5526821
- OpenAlex
- W2670643325
- Cited by
- 67
- References
- 26
- Citations by year
- 2026: 2 · 2025: 6 · 2024: 6 · 2023: 5 · 2022: 12 · 2021: 16 · 2020: 4 · 2019: 6 · 2018: 9 · 2017: 1
- FWCI
- 8.87
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Translational Behavioral Medicine (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Keywords
- mobile health apps, user engagement, weight loss apps, lifestyle modification, app usage patterns, weigh-in
- MeSH terms
- Smartphone, Adolescent, Adult, Aged, Cross-Sectional Studies, Female, Humans, Male, Middle Aged, Patient Participation, Health Behavior, Telemedicine, Young Adult, Data Mining, Self Report, Weight Reduction Programs, Mobile Applications
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
