The DOI matches its Crossref record; 41 of the 41 references it cites were found in a registry or an index.
The Effect of Timing and Frequency of Push Notifications on Usage of a Smartphone-Based Stress Management Intervention: An Exploratory Trial
Open accessAbstract
Push notifications offer a promising strategy for enhancing engagement with smartphone-based health interventions. Intelligent sensor-driven machine learning models may improve the timeliness of notifications by adapting delivery to a user's current context (e.g. location). This exploratory mixed-methods study examined the potential impact of timing and frequency on notification response and usage of Healthy Mind, a smartphone-based stress management intervention. 77 participants were randomised to use one of three versions of Healthy Mind that provided: intelligent notifications; daily notifications within pre-defined time frames; or occasional notifications within pre-defined time frames. Notification response and Healthy Mind usage were automatically recorded. Telephone interviews explored participants' experiences of using Healthy Mind. Participants in the intelligent and daily conditions viewed (d = .47, .44 respectively) and actioned (d = .50, .43 respectively) more notifications compared to the occasional group. Notification group had no meaningful effects on percentage of notifications viewed or usage of Healthy Mind. No meaningful differences were indicated between the intelligent and non-intelligent groups. Our findings suggest that frequent notifications may encourage greater exposure to intervention content without deterring engagement, but adaptive tailoring of notification timing does not always enhance their use. Hypotheses generated from this study require testing in future work. TRIAL REGISTRATION NUMBER: ISRCTN67177737.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“The studies that ranked high for selective reporting bias tended to report only user satisfaction data and treatment adherence, despite their methods considering the collection and assessment of psychiatric scale data and other effectiveness outcomes (29, 76, 79, 85–88, 92).”
From Examining the role of AI technology in online mental healthcare: opportunities, challenges, and implications, a mixed-methods review (2024) · cites this paper as [79]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In terms of allegiance bias, because of the scarcity and novelty of AI-augmented interventions and other AI algorithms, most of the included articles developed and studied their own AI tools, instead of AI tools developed by other researchers (29, 33, 73, 74, 76, 79, 80, 82–87, 90).”
From Examining the role of AI technology in online mental healthcare: opportunities, challenges, and implications, a mixed-methods review (2024) · cites this paper as [79]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“There were a large variety of study types among the 17 studies, including pilot randomised controlled trials (RCTs) [28–30], observational studies [31–35], model evaluations [36,37], and a qualitative study [38].”
From Artificial intelligence and machine learning in mobile apps for mental health: A scoping review (2022) · cites this paper as [28]
The cited source says what the claim attributes to it.
In the citing paper
“They found that compared to occasional notifications, intelligent notifications and daily notifications encouraged a higher exposure to the therapeutic content while maintaining engagement levels. However, they noted that the timing of the notification did not seem to support treatment engagement, when comparing intelligent notifications against daily notifications (79).”
From Examining the role of AI technology in online mental healthcare: opportunities, challenges, and implications, a mixed-methods review (2024) · cites this paper as [79]
The cited source says what the claim attributes to it.
In the citing paper
“In terms of improvement to the delivery of online therapy, the included studies [n=4 (30, 75, 79, 80)] have reported on the implementation of intelligent notifications, momentary sensing, monitoring of treatment outcomes, and triage decision support.”
From Examining the role of AI technology in online mental healthcare: opportunities, challenges, and implications, a mixed-methods review (2024) · cites this paper as [79]
The cited source says what the claim attributes to it.
In the citing paper
“Though, in the case of Morrison et al. (2017) they found that there was no significant improvement comparing daily to intelligent notifications (79, 80).”
From Examining the role of AI technology in online mental healthcare: opportunities, challenges, and implications, a mixed-methods review (2024) · cites this paper as [79]
The cited source says what the claim attributes to it.
In the citing paper
“In terms of treatment engagement support, Burns et al. (2011) and Morrison et al. (2017) implemented an AI algorithm that could learn from the user and their environment, to deliver intelligent treatment reminders (79, 80).”
From Examining the role of AI technology in online mental healthcare: opportunities, challenges, and implications, a mixed-methods review (2024) · cites this paper as [79]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“12 of the included studies (41, 73–83) presented data on treatment dropout, online or remote psychotherapy interventions reported a dropout rate between 0% and 61%, psychoeducation a dropout rate between 4% and 60% and no intervention a dropout rate between 5% and 34%.”
From Examining the role of AI technology in online mental healthcare: opportunities, challenges, and implications, a mixed-methods review (2024) · cites this paper as [79]
Experimental — accuracy not yet measuredIn the citing paper
“or to personalise aspects of the app such as notification delivery [28]”
From Artificial intelligence and machine learning in mobile apps for mental health: A scoping review (2022) · cites this paper as [28]
In the citing paper
“12 of the 17 included studies had sample sizes smaller than 100 [28–31,33–35,37–39,41,42].”
From Artificial intelligence and machine learning in mobile apps for mental health: A scoping review (2022) · cites this paper as [28]
In the citing paper
“predicting or helping manage stress [28,32,33]”
From Artificial intelligence and machine learning in mobile apps for mental health: A scoping review (2022) · cites this paper as [28]
In the citing paper
“One study using AI/ML to send context-specific notifications found no difference between the ‘intelligent’ notifications and non-intelligent notifications [28].”
From Artificial intelligence and machine learning in mobile apps for mental health: A scoping review (2022) · cites this paper as [28]
In the citing paper
“Only three randomised controlled trials of an AI-enabled mental health app were identified in this review [28–30]”
From Artificial intelligence and machine learning in mobile apps for mental health: A scoping review (2022) · cites this paper as [28]
In the citing paper
“All of the studies reviewed used data collected via mobile devices to inform AI models—for natural language analysis and output [29,30,38,40,42,43], risk stratification and prediction [31,41] or for personalising the user’s app experience [28,44].”
From Artificial intelligence and machine learning in mobile apps for mental health: A scoping review (2022) · cites this paper as [28]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414
Tong, A., Sainsbury, P., & Craig, J. (2007) · International Journal for Quality in Health Care · doi:10.1093/intqhc/mzm042 · Cited by 42,509
Hoffmann, T. C., Glasziou, P. P., Boutron, I., Milne, R., Perera, R., Moher, D., Altman, D. G., Barbour, V., Macdonald, H., Johnston, M., et al. (2014) · BMJ · doi:10.1136/bmj.g1687 · Cited by 10,548
Sontrop, J. M., Dixon, S. N., Garg, A. X., Jiménez, I. B., Dohein, O., Huang, S. S., & Clark, W. F. (2013) · American Journal of Nephrology · doi:10.1159/000350377 · Cited by 5,903
A behavior model for persuasive design
Fogg, B. (2009) · Proceedings of the 4th International Conference on Persuasive Technology · doi:10.1145/1541948.1541999 · Cited by 2,328
Confidence intervals rather than P values: estimation rather than hypothesis testing.
Gardner, M. J., & Altman, D. G. (1986) · BMJ · doi:10.1136/bmj.292.6522.746 · Cited by 2,014
Yardley, L., Morrison, L., Bradbury, K., & Muller, I. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.4055 · Cited by 1,442
Self-citationDennison, L., Morrison, L., Conway, G., & Yardley, L. (2013) · Journal of Medical Internet Research · doi:10.2196/jmir.2583 · Cited by 895
Self-citationMohr, D. C., Burns, M. N., Schueller, S. M., Clarke, G., & Klinkman, M. (2013) · General Hospital Psychiatry · doi:10.1016/j.genhosppsych.2013.03.008 · Cited by 693 · cited 1×
“Notifications can also prompt access to more intensive support provided by other platforms [12,13].”
BackgroundA meta-analysis of web-delivered tailored health behavior change interventions
Lustria, M. L. A., Noar, S. M., Cortese, J., Van Stee, S. K., Glueckauf, R. L., & Lee, J. (2013) · Journal of Health Communication · doi:10.1080/10810730.2013.768727 · Cited by 689
Research works printed but not linked: 1
- Hargood C, Michaelides D, Weal M, Pejovic V, Musolesi M, Morrison L, et al. The UBhave framework: Developing dynamic mobile applications for digital behavioural interventions. Poster presented at: 11th International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services; 2014 Dec 02–05; London, UK.
Cited by
Sorted by citations
Applications of Machine Learning in Real-Life Digital Health Interventions: Review of the Literature
Triantafyllidis, A. K., & Tsanas, A. (2019) · Journal of Medical Internet Research · doi:10.2196/12286 · Cited by 243
Fineberg, N. A., Menchón, J. M., Hall, N., Dell’Osso, B. M., Brand, M., Potenza, M. N., Chamberlain, S. R., Cirnigliaro, G., Löchner, C., Billieux, J., et al. (2022) · Comprehensive Psychiatry · doi:10.1016/j.comppsych.2022.152346 · Cited by 276
khan, Z. F., & Alotaibi, S. R. (2020) · Journal of Healthcare Engineering · doi:10.1155/2020/8894694 · Cited by 235
Bidargaddi, N., Almirall, D., Murphy, S., Nahum-Shani, I., Kovalcik, M., Pituch, T., Maaieh, H., & Strecher, V. (2018) · JMIR mHealth and uHealth · doi:10.2196/10123 · Cited by 189
Engagement in digital interventions.
Nahum-Shani, I., Shaw, S. D., Carpenter, S. M., Murphy, S. A., & Yoon, C. (2022) · American Psychologist · doi:10.1037/amp0000983 · Cited by 177
Metadata
- Authors
- Morrison, L. G., Hargood, C., Pejovic, V., Geraghty, A. W. A., Lloyd, S., Goodman, N., Michaelides, D. T., Weston, A., Musolesi, M., Weal, M. J., & Yardley, L.
- Affiliations
- University of Southampton (GB)
- Bournemouth University (GB)
- Durham University (GB)
- Gateshead Council (GB)
- University College London (GB)
- Corresponding author
- Morrison, Leanne G.
- ORCID
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2017-01-03
- Volume
- 12
- Issue
- 1
- Pages
- e0169162
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC5207732
- OpenAlex
- W1771486365
- Cited by
- 162
- References
- 41
- Citations by year
- 2026: 13 · 2025: 19 · 2024: 18 · 2023: 15 · 2022: 20 · 2021: 28 · 2020: 19 · 2019: 16 · 2018: 9 · 2017: 3
- FWCI
- 14.68
- Type
- journal article
- Language
- en
- Update notices
- correction (2018-05-22)
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Teesside University Research Portal (Teesside University) (publishedVersion, cc-by)
- Explore Bristol Research (publishedVersion, cc-by)
- Bournemouth University Research Online (Bournemouth University) (acceptedVersion)
- ePrints Soton (University of Southampton) (acceptedVersion)
- Archivio istituzionale della ricerca (Alma Mater Studiorum Università di Bologna) (submittedVersion, cc-by)
- TeesRep (Teesside University) (submittedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- UCL Discovery (University College London) (submittedVersion)
- Figshare (submittedVersion, cc-by)
- Bristol Research (University of Bristol) (submittedVersion, cc-by)
- Funders
- Engineering and Physical Sciences Research Council (EP/I032673/1)
- Topics
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Social Sciences › Psychology › Clinical Psychology › Mindfulness and Compassion Interventions
- Keywords
- push notifications, notification timing, smartphone-based intervention, stress management, healthy mind, user engagement, context-aware notifications, mobile health
- MeSH terms
- Smartphone, Machine Learning, Adolescent, Adult, Algorithms, Automation, Female, United Kingdom, Health Promotion, Humans, Male, Middle Aged, Public Health, Quality of Life, Stress, Psychological, Health Behavior, Telemedicine, Geographic Information Systems, Young Adult, Text Messaging, Accelerometry
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
