CoChat

To Prompt or Not to Prompt? A Microrandomized Trial of Time-Varying Push Notifications to Increase Proximal Engagement With a Mobile Health App

Open access

Published version of a preprint

The DOI matches its Crossref record; 46 of the 46 references it cites were found in a registry or an index.

Abstract

BACKGROUND: Mobile health (mHealth) apps provide an opportunity for easy, just-in-time access to health promotion and self-management support. However, poor user engagement with these apps remains a significant unresolved challenge. OBJECTIVE: This study aimed to assess the effect of sending versus not sending a push notification containing a contextually tailored health message on proximal engagement, measured here as self-monitoring via the app. Secondary aims were to examine whether this effect varies by the number of weeks enrolled in the program or by weekday versus weekend. An exploratory aim was to describe how the effect on proximal engagement differs between weekday versus weekend by the time of day. METHODS: The study analyzes the causal effects of push notifications on proximal engagement in 1255 users of a commercial workplace well-being intervention app over 89 days. The app employs a microrandomized trial (MRT) design to send push notifications. At 1 of 6 times per day (8:30 am, 12:30 pm, 5:30 pm, 6:30 pm, 7:30 pm, and 8:30 pm; selected randomly), available users were randomized with equal probability to be sent or not sent a push notification containing a tailored health message. The primary outcome of interest was whether the user self-monitored behaviors and feelings at some time during the next 24 hours via the app. A generalization of log-linear regression analysis, adapted for use with data arising from an MRT, was used to examine the effect of sending a push notification versus not sending a push notification on the probability of engagement over the next 24 hours. RESULTS: Users were estimated to be 3.9% more likely to engage with the app in the next 24 hours when a tailored health message was sent versus when it was not sent (risk ratio 1.039; 95% CI 1.01 to 1.08; P<.05). The effect of sending the message attenuated over the course of the study, but this effect was not statistically significant (P=.84). The effect of sending the message was greater on weekends than on weekdays, but the difference between these effects was not statistically significant (P=.18). When sent a tailored health message on weekends, the users were 8.7% more likely to engage with the app (95% CI 1.01 to 1.17), whereas on weekdays, the users were 2.5% more likely to engage with the app (95% CI 0.98 to 1.07). The effect of sending a tailored health message was greatest at 12:30 pm on weekends, when the users were 11.8% more likely to engage (90% CI 1.02 to 1.13). CONCLUSIONS: Sending a push notification containing a tailored health message was associated with greater engagement in an mHealth app. Results suggested that users are more likely to engage with the app within 24 hours when push notifications are sent at mid-day on weekends.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.2196/10123

Cites

46 of 46 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 Different types of contextually tailored messages.
Figure 1 Different types of contextually tailored messages.CC BY · PubMed Central
Figure 2 Micro-randomized trial design in JOOL app. Each decision point j (i)=(d, t) where i=1, 2,…,534, corresponding to a time of day t=1,2,…,6; 8:30 am, 12:30 pm, 5:30 pm, 6:30 pm, 7:30 pm, 8:30 pm with in a day d=1, 2, …, 89.
Figure 2 Micro-randomized trial design in JOOL app. Each decision point j (i)=(d, t) where i=1, 2,…,534, corresponding to a time of day t=1,2,…,6; 8:30 am, 12:30 pm, 5:30 pm, 6:30 pm, 7:30 pm, 8:30 pm with in a day d=1, 2, …, 89.CC BY · PubMed Central
Figure 3 Percentage of individuals available by day in study.
Figure 3 Percentage of individuals available by day in study.CC BY · PubMed Central
Figure 4 Effects of push notification over course of the trial.
Figure 4 Effects of push notification over course of the trial.CC BY · PubMed Central
Figure 5 Effects of push notification over different times in a day.
Figure 5 Effects of push notification over different times in a day.CC BY · PubMed Central
Figure 6 Effects of push notification over different times in a day (weekday and weekends).
Figure 6 Effects of push notification over different times in a day (weekday and weekends).CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 46 references

Research works printed but not linked: 2

  1. Strecher, VJ, 2016, Life on Purpose
  2. Zhang, S, 2016, AMIA Annu Symp Proc, 2016, 2090

Cited by

See all 189

Metadata

Authors
Bidargaddi, N., Almirall, D., Murphy, S., Nahum-Shani, I., Kovalcik, M., Pituch, T., Maaieh, H., & Strecher, V.
Affiliations
  • University of Michigan (US)
  • Harvard University (US)
Corresponding author
Bidargaddi, Niranjan
Venue
JMIR mHealth and uHealth
Published
2018-11-29
Volume
6
Issue
11
Pages
e10123
ISSN
2291-5222 (ISSN-L)
OpenAlex
W2836815702
Cited by
189
References
45
Citations by year
2026: 30 · 2025: 22 · 2024: 24 · 2023: 31 · 2022: 33 · 2021: 22 · 2020: 21 · 2019: 6
FWCI
18.71
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • JMIR mhealth and uhealth (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • Flinders Academic Commons (Flinders University) (submittedVersion)
  • Flinders Academic Commons (Flinders University) (submittedVersion, cc-by)
Funders
  • National Institute on Alcohol Abuse and Alcoholism (R01AA023187)
  • National Institute on Drug Abuse (P50DA039838, R01DA039901)
Topics
  • Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
  • Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
  • Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
Keywords
push notifications, self-monitoring, tailored health messages, workplace well-being interventions, time of day effects, user retention, digital health interventions, health behavior change, user-generated images, user-generated video

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.