The Effect of Timing and Frequency of Push Notifications on Usage of a Smartphone-Based Stress Management Intervention: An Exploratory Trial
Open accessAbstract
Push notifications offer a promising strategy for enhancing engagement with smartphone-based health interventions. Intelligent sensor-driven machine learning models may improve the timeliness of notifications by adapting delivery to a user's current context (e.g. location). This exploratory mixed-methods study examined the potential impact of timing and frequency on notification response and usage of Healthy Mind, a smartphone-based stress management intervention. 77 participants were randomised to use one of three versions of Healthy Mind that provided: intelligent notifications; daily notifications within pre-defined time frames; or occasional notifications within pre-defined time frames. Notification response and Healthy Mind usage were automatically recorded. Telephone interviews explored participants' experiences of using Healthy Mind. Participants in the intelligent and daily conditions viewed (d = .47, .44 respectively) and actioned (d = .50, .43 respectively) more notifications compared to the occasional group. Notification group had no meaningful effects on percentage of notifications viewed or usage of Healthy Mind. No meaningful differences were indicated between the intelligent and non-intelligent groups. Our findings suggest that frequent notifications may encourage greater exposure to intervention content without deterring engagement, but adaptive tailoring of notification timing does not always enhance their use. Hypotheses generated from this study require testing in future work. TRIAL REGISTRATION NUMBER: ISRCTN67177737.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Studies that ranked high for selective reporting bias tended to report only user satisfaction data and treatment adherence, despite their methods considering collection and assessment of psychiatric scale data and other effectiveness outcomes.
“The studies that ranked high for selective reporting bias tended to report only user satisfaction data and treatment adherence, despite their methods considering the collection and assessment of psychiatric scale data and other effectiveness outcomes (29, 76, 79, 85–88, 92).”
From Examining the role of AI technology in online mental healthcare: opportunities, challenges, and implications, a mixed-methods review (2024) · cites this paper as [79]
Split into 2 atoms — 0 confirmed, 0 partially supported, 1 disputed, 0 misattributed, 1 unverifiable. Expand the claim to see per-atom verdicts.
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredIn terms of allegiance bias, most included articles developed and studied their own AI tools instead of AI tools developed by other researchers.
“In terms of allegiance bias, because of the scarcity and novelty of AI-augmented interventions and other AI algorithms, most of the included articles developed and studied their own AI tools, instead of AI tools developed by other researchers (29, 33, 73, 74, 76, 79, 80, 82–87, 90).”
From Examining the role of AI technology in online mental healthcare: opportunities, challenges, and implications, a mixed-methods review (2024) · cites this paper as [79]
The Method (Design and Intervention; page unavailable) describes a trial of Healthy Mind’s intelligent notification algorithm and says the app was created using Life Guide Toolbox software. It does not establish that this study’s authors developed the AI algorithm themselves; the Design section says its sensing and model-generation functions used independent open-source libraries. This paper also cannot establish the claim that most of the citing review’s included articles studied their own tools. (LLM, conf=0.80)
Experimental — accuracy not yet measuredMorrison et al. (2017) found that compared to occasional notifications, intelligent notifications and daily notifications encouraged a higher exposure to therapeutic content while maintaining engagement levels; however, timing of notification did not seem to support treatment engagement when comparing intelligent notifications against daily notifications.
“They found that compared to occasional notifications, intelligent notifications and daily notifications encouraged a higher exposure to the therapeutic content while maintaining engagement levels. However, they noted that the timing of the notification did not seem to support treatment engagement, when comparing intelligent notifications against daily notifications (79).”
From Examining the role of AI technology in online mental healthcare: opportunities, challenges, and implications, a mixed-methods review (2024) · cites this paper as [79]
The included studies (n=4) reported on implementation of intelligent notifications, momentary sensing, monitoring of treatment outcomes, and triage decision support.
“In terms of improvement to the delivery of online therapy, the included studies [n=4 (30, 75, 79, 80)] have reported on the implementation of intelligent notifications, momentary sensing, monitoring of treatment outcomes, and triage decision support.”
From Examining the role of AI technology in online mental healthcare: opportunities, challenges, and implications, a mixed-methods review (2024) · cites this paper as [79]
Morrison et al. (2017) found that there was no significant improvement comparing daily to intelligent notifications.
“Though, in the case of Morrison et al. (2017) they found that there was no significant improvement comparing daily to intelligent notifications (79, 80).”
From Examining the role of AI technology in online mental healthcare: opportunities, challenges, and implications, a mixed-methods review (2024) · cites this paper as [79]
Burns et al. (2011) and Morrison et al. (2017) implemented an AI algorithm that could learn from the user and their environment, to deliver intelligent treatment reminders.
“In terms of treatment engagement support, Burns et al. (2011) and Morrison et al. (2017) implemented an AI algorithm that could learn from the user and their environment, to deliver intelligent treatment reminders (79, 80).”
From Examining the role of AI technology in online mental healthcare: opportunities, challenges, and implications, a mixed-methods review (2024) · cites this paper as [79]
12 of the included studies presented data on treatment dropout; online or remote psychotherapy interventions reported a dropout rate between 0% and 61%, psychoeducation between 4% and 60%, and no intervention between 5% and 34%.
“12 of the included studies (41, 73–83) presented data on treatment dropout, online or remote psychotherapy interventions reported a dropout rate between 0% and 61%, psychoeducation a dropout rate between 4% and 60% and no intervention a dropout rate between 5% and 34%.”
From Examining the role of AI technology in online mental healthcare: opportunities, challenges, and implications, a mixed-methods review (2024) · cites this paper as [79]
The Results (Sample characteristics and Notification response and intervention usage; page unavailable) describe one Healthy Mind notification trial, including participants who ceased app use. Searches for treatment dropout, psychotherapy, psychoeducation, and the stated rate ranges found no passage reporting dropout across 12 studies or the three intervention categories. (LLM, conf=0.77)
Experimental — accuracy not yet measuredThere were a large variety of study types among the 17 studies, including pilot randomised controlled trials, observational studies, model evaluations, and a qualitative study.
“There were a large variety of study types among the 17 studies, including pilot randomised controlled trials (RCTs) [28–30], observational studies [31–35], model evaluations [36,37], and a qualitative study [38].”
From Artificial intelligence and machine learning in mobile apps for mental health: A scoping review (2022) · cites this paper as [28]
In some studies, AI/ML was used to personalise aspects of the app such as notification delivery.
“or to personalise aspects of the app such as notification delivery [28]”
From Artificial intelligence and machine learning in mobile apps for mental health: A scoping review (2022) · cites this paper as [28]
12 of the 17 included studies had sample sizes smaller than 100.
“12 of the 17 included studies had sample sizes smaller than 100 [28–31,33–35,37–39,41,42].”
From Artificial intelligence and machine learning in mobile apps for mental health: A scoping review (2022) · cites this paper as [28]
3 of the 17 studies focused on predicting or helping manage stress.
“predicting or helping manage stress [28,32,33]”
From Artificial intelligence and machine learning in mobile apps for mental health: A scoping review (2022) · cites this paper as [28]
One study using AI/ML to send context-specific notifications found no difference between the 'intelligent' notifications and non-intelligent notifications.
“One study using AI/ML to send context-specific notifications found no difference between the ‘intelligent’ notifications and non-intelligent notifications [28].”
From Artificial intelligence and machine learning in mobile apps for mental health: A scoping review (2022) · cites this paper as [28]
Only three randomised controlled trials of an AI-enabled mental health app were identified in this review.
“Only three randomised controlled trials of an AI-enabled mental health app were identified in this review [28–30]”
From Artificial intelligence and machine learning in mobile apps for mental health: A scoping review (2022) · cites this paper as [28]
All of the studies reviewed used data collected via mobile devices to inform AI models-for natural language analysis and output, risk stratification and prediction or for personalising the user's app experience.
“All of the studies reviewed used data collected via mobile devices to inform AI models—for natural language analysis and output [29,30,38,40,42,43], risk stratification and prediction [31,41] or for personalising the user’s app experience [28,44].”
From Artificial intelligence and machine learning in mobile apps for mental health: A scoping review (2022) · cites this paper as [28]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414 · see citing works
Tong, A., Sainsbury, P., & Craig, J. (2007) · International Journal for Quality in Health Care · doi:10.1093/intqhc/mzm042 · Cited by 42,509 · see citing works
Hoffmann, T. C., Glasziou, P. P., Boutron, I., Milne, R., Perera, R., Moher, D., Altman, D. G., Barbour, V., Macdonald, H., Johnston, M., et al. (2014) · BMJ · doi:10.1136/bmj.g1687 · Cited by 10,548 · see citing works
Sontrop, J. M., Dixon, S. N., Garg, A. X., Jiménez, I. B., Dohein, O., Huang, S. S., & Clark, W. F. (2013) · American Journal of Nephrology · doi:10.1159/000350377 · Cited by 5,903 · see citing works
A behavior model for persuasive design
Fogg, B. (2009) · Proceedings of the 4th International Conference on Persuasive Technology · doi:10.1145/1541948.1541999 · Cited by 2,328 · see citing works
Confidence intervals rather than P values: estimation rather than hypothesis testing.
Gardner, M. J., & Altman, D. G. (1986) · BMJ · doi:10.1136/bmj.292.6522.746 · Cited by 2,014 · see citing works
Yardley, L., Morrison, L., Bradbury, K., & Muller, I. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.4055 · Cited by 1,442 · see citing works
Self-citationDennison, L., Morrison, L., Conway, G., & Yardley, L. (2013) · Journal of Medical Internet Research · doi:10.2196/jmir.2583 · Cited by 895 · see citing works
Self-citationMohr, D. C., Burns, M. N., Schueller, S. M., Clarke, G., & Klinkman, M. (2013) · General Hospital Psychiatry · doi:10.1016/j.genhosppsych.2013.03.008 · Cited by 693 · cited 1× · see citing works
“Notifications can also prompt access to more intensive support provided by other platforms [12,13].”
BackgroundA meta-analysis of web-delivered tailored health behavior change interventions
Lustria, M. L. A., Noar, S. M., Cortese, J., Van Stee, S. K., Glueckauf, R. L., & Lee, J. (2013) · Journal of Health Communication · doi:10.1080/10810730.2013.768727 · Cited by 689 · see citing works
Maxwell, S. E. (2004) · Psychological Methods · doi:10.1037/1082-989x.9.2.147 · Cited by 655 · see citing works
Nahum-Shani, I., Hekler, E. B., & Spruijt-Metz, D. (2015) · Health Psychology · doi:10.1037/hea0000306 · Cited by 568 · see citing works
The Jigsaw continuous sensing engine for mobile phone applications
Lu, H., Yang, J., Liu, Z., Lane, N. D., Choudhury, T., & Campbell, A. T. (2010) · ACM Conference on Embedded Networked Sensor Systems (SenSys) · doi:10.1145/1869983.1869992 · Cited by 561 · see citing works
Mobile Health Apps to Facilitate Self-Care: A Qualitative Study of User Experiences
Anderson, K., Burford, O., & Emmerton, L. (2016) · PLOS ONE · doi:10.1371/journal.pone.0156164 · Cited by 502 · see citing works
Brouwer, W., Kroeze, W., Crutzen, R., de Nooijer, J., de Vries, N. K., Brug, J., & Oenema, A. (2011) · Journal of Medical Internet Research · doi:10.2196/jmir.1639 · Cited by 455 · see citing works
Fry, J. P., & Neff, R. A. (2009) · Journal of Medical Internet Research · doi:10.2196/jmir.1138 · Cited by 447 · see citing works
Online Prevention Aimed at Lifestyle Behaviors: A Systematic Review of Reviews
Kohl, L. F., Crutzen, R., & de Vries, N. K. (2013) · Journal of Medical Internet Research · doi:10.2196/jmir.2665 · Cited by 388 · see citing works
An in-situ study of mobile phone notifications
Pielot, M., Church, K., & de Oliveira, R. C. (2014) · International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI) · doi:10.1145/2628363.2628364 · Cited by 356 · see citing works
Predicting human interruptibility with sensors
Fogarty, J. A., Hudson, S. E., Atkeson, C. G., Avrahami, D., Forlizzi, J., Kiesler, S. B., Lee, J. C., & Yang, J. (2005) · ACM Transactions on Computer-Human Interaction · doi:10.1145/1057237.1057243 · Cited by 344 · see citing works
Morrison, L. G., Yardley, L., Powell, J., & Michie, S. (2012) · Telemedicine and e-Health · doi:10.1089/tmj.2011.0062 · Cited by 290 · see citing works
Self-citationPejović, V., & Musolesi, M. (2014) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/2632048.2632062 · Cited by 283 · see citing works
Fischer, J. E., Greenhalgh, C., & Benford, S. (2011) · International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI) · doi:10.1145/2037373.2037402 · Cited by 189 · see citing works
Designing content-driven intelligent notification mechanisms for mobile applications
Mehrotra, A., Musolesi, M., Hendley, R., & Pejović, V. (2015) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/2750858.2807544 · Cited by 186 · see citing works
Pielot, M., de Oliveira, R., Kwak, H., & Oliver, N. (2014) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2556288.2556973 · Cited by 176 · see citing works
Strategies for mHealth Research: Lessons from 3 Mobile Intervention Studies
Ben‐Zeev, D., Schueller, S. M., Begale, M. J., Duffecy, J. L., Kane, J. M., & Mohr, D. C. (2014) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-014-0556-2 · Cited by 167 · see citing works
Challenges and opportunities for promoting physical activity in the workplace
Marshall, A. L. (2004) · Journal of science and medicine in sport · doi:10.1016/s1440-2440(04)80279-2 · Cited by 164 · see citing works
Effects of content and time of delivery on receptivity to mobile interruptions
Fischer, J. E., Yee, N., Bellotti, V. M. E., Good, N., Benford, S., & Greenhalgh, C. (2010) · International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI) · doi:10.1145/1851600.1851620 · Cited by 148 · see citing works
How behavioral science can advance digital health
Pagoto, S., & Bennett, G. G. (2013) · Translational Behavioral Medicine · doi:10.1007/s13142-013-0234-z · Cited by 145 · see citing works
Sarker, H., Sharmin, M., Ali, A. A., Rahman, M. M., Bari, R., Hossain, S. M., & Kumar, S. (2014) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/2632048.2636082 · Cited by 121 · see citing works
Danaher, B. G., Brendryen, H., Seeley, J. R., Tyler, M. S., & Woolley, T. (2015) · Internet Interventions · doi:10.1016/j.invent.2015.01.002 · Cited by 102 · see citing works
The power of mobile notifications to increase wellbeing logging behavior
Bentley, F. R., & Tollmar, K. (2013) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2470654.2466140 · Cited by 100 · see citing works
Characterizing Periodic Messaging Interventions Across Health Behaviors and Media: Systematic Review
León, E. D., Fuentes, L. W., & Cohen, J. E. (2014) · Journal of Medical Internet Research · doi:10.2196/jmir.2837 · Cited by 81 · see citing works
Morrison, L. G., Hargood, C., Lin, S. X., Dennison, L., Joseph, J., Hughes, S., Michaelides, D. T., Johnston, D., Johnston, M., Michie, S., et al. (2014) · Journal of Medical Internet Research · doi:10.2196/jmir.3579 · Cited by 79 · see citing works
Self-citationRiley, W. T., Serrano, K. J., Nilsen, W., & Atienza, A. A. (2015) · Current Opinion in Psychology · doi:10.1016/j.copsyc.2015.03.024 · Cited by 76 · see citing works
Bardus, M., Blake, H., Lloyd, S., & Suzanne Suggs, L. (2014) · International Journal of Workplace Health Management · doi:10.1108/ijwhm-11-2013-0040 · Cited by 54 · see citing works
Self-citationFukuoka, Y., Lindgren, T., & Jong, S. (2012) · Public Health Nursing · doi:10.1111/j.1525-1446.2011.00997.x · Cited by 49 · see citing works
Braun, T., Bambra, C. L., Booth, M., Adetayo, K., & Milne, E. M. G. (2014) · Journal of Public Health · doi:10.1093/pubmed/fdu043 · Cited by 45 · see citing works
Open source smartphone libraries for computational social science
Lathia, N., Rachuri, K. K., Cecilia Mascolo, C., & Roussos, G. (2013) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/2494091.2497345 · Cited by 40 · see citing works
Willoughby, J. F., & Furberg, R. (2015) · Journal of Health Communication · doi:10.1080/10810730.2014.977468 · Cited by 37 · see citing works
Suggs, L. S., Blake, H., Bardus, M., & Lloyd, S. (2013) · Journal of Health Services Research & Policy · doi:10.1177/1355819613478001 · Cited by 20 · see citing works
Interest in behaviour change interventions: A conceptual model
Crutzen, R., & Ruiter, R. A. C. (2015) · Research Publications (Maastricht University) · Cited by 13 · see citing works
Flagged first, then by citations
Research works printed but not linked: 4
- [24]Pejovic V, Musolesi, M. InterruptMe. In: Brush AJ, Friday A, Kientz J, Scott J, Song J, editors. Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing; 2014 Sep 13–17; Seattle, WA, USA. New York, NY: ACM; 2014. p. 897–908.
- [32]Hargood C, Michaelides D, Weal M, Pejovic V, Musolesi M, Morrison L, et al. The UBhave framework: Developing dynamic mobile applications for digital behavioural interventions. Poster presented at: 11th International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services; 2014 Dec 02–05; London, UK.
- [38]Cohen J. Statistical Power Analysis for the Behavioural Sciences. New York, NY: Academic Press; 1969.
- [42]Mehrotra A, Musolesi M, Hendley R, Pejovic V. Designing content-driven intelligent notication mechamisms for mobile applications. In: Mase K, Lanheinrich M, Gatica-Perez D, Gellersen H, Choudhury T, Yatani K, editors. Proceedings of the 2015 ACM InternationalJoint Conference on Pervasive and Ubiquitous Computing; 2015 Sep 07–11; Osaka, Japan. New York, NY: ACM; 2015. p. 813–824.
Cited by
Sorted by citations
Applications of Machine Learning in Real-Life Digital Health Interventions: Review of the Literature
Triantafyllidis, A. K., & Tsanas, A. (2019) · Journal of Medical Internet Research · doi:10.2196/12286 · Cited by 243 · see citing works
Fineberg, N. A., Menchón, J. M., Hall, N., Dell’Osso, B. M., Brand, M., Potenza, M. N., Chamberlain, S. R., Cirnigliaro, G., Löchner, C., Billieux, J., et al. (2022) · Comprehensive Psychiatry · doi:10.1016/j.comppsych.2022.152346 · Cited by 253 · see citing works
khan, Z. F., & Alotaibi, S. R. (2020) · Journal of Healthcare Engineering · doi:10.1155/2020/8894694 · Cited by 235 · see citing works
Bidargaddi, N., Almirall, D., Murphy, S., Nahum-Shani, I., Kovalcik, M., Pituch, T., Maaieh, H., & Strecher, V. (2018) · JMIR mHealth and uHealth · doi:10.2196/10123 · Cited by 189 · see citing works
Engagement in digital interventions.
Nahum-Shani, I., Shaw, S. D., Carpenter, S. M., Murphy, S. A., & Yoon, C. (2022) · American Psychologist · doi:10.1037/amp0000983 · Cited by 177 · see citing works
Timmers, T., Janssen, L., Kool, R. B., & Kremer, J. A. M. (2020) · Journal of Medical Internet Research · doi:10.2196/17342 · Cited by 127 · see citing works
Designing m-Health interventions for precision mental health support
Bidargaddi, N., Schrader, G., Klasnja, P., Licinio, J., & Murphy, S. (2020) · Translational Psychiatry · doi:10.1038/s41398-020-00895-2 · Cited by 92 · see citing works
Detecting Receptivity for mHealth Interventions in the Natural Environment
Mishra, V., Künzler, F., Kramer, J.-N., Fleisch, E., Kowatsch, T., & Kotz, D. F. (2021) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3463492 · Cited by 85 · see citing works
Exploring the State-of-Receptivity for mHealth Interventions
Künzler, F., Mishra, V., Kramer, J.-N., Kotz, D. F., Fleisch, E., & Kowatsch, T. (2019) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3369805 · Cited by 83 · see citing works
Bidargaddi, N., Pituch, T., Maaieh, H., Short, C., & Strecher, V. (2018) · Preventive Medicine Reports · doi:10.1016/j.pmedr.2018.07.004 · Cited by 77 · see citing works
Gutierrez, G., Stephenson, C., Eadie, J., Asadpour, K., & Alavi, N. (2024) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2024.1356773 · Cited by 71 · see citing works
Adolescent HIV treatment in South Africa's national HIV programme: a retrospective cohort study
Maskew, M., Bor, J. H., MacLeod, W., Carmona, S., Sherman, G. G., & Fox, M. P. (2019) · The Lancet HIV · doi:10.1016/s2352-3018(19)30234-6 · Cited by 71 · see citing works
Artificial intelligence and machine learning in mobile apps for mental health: A scoping review
Milne-Ives, M., Selby, E., Inkster, B., Lam, C., & Meinert, E. (2022) · PLOS Digital Health · doi:10.1371/journal.pdig.0000079 · Cited by 68 · see citing works
Thornton, L. K., Gardner, L. A., Osman, B., Green, O., Champion, K. E., Bryant, Z., Teesson, M., KAY‐LAMBKIN, F. J., Chapman, C., & The Health4Life Team (2021) · JMIR Formative Research · doi:10.2196/25513 · Cited by 66 · see citing works
Comparing usage of a web and app stress management intervention: An observational study
Morrison, L. G., Geraghty, A. W. A., Lloyd, S., Goodman, N., Michaelides, D. T., Hargood, C., Weal, M., & Yardley, L. (2018) · Internet Interventions · doi:10.1016/j.invent.2018.03.006 · Cited by 57 · see citing works
Self-citationNeCamp, T., Sen, S., Frank, E., Walton, M. A., Ionides, E. L., Fang, Y., Tewari, A., & Wu, Z. (2020) · Journal of Medical Internet Research · doi:10.2196/15033 · Cited by 55 · see citing works
Dose–response research in digital health interventions: Concepts, considerations, and challenges.
McVay, M. A., Bennett, G. G., Steinberg, D., & Voils, C. I. (2019) · Health Psychology · doi:10.1037/hea0000805 · Cited by 55 · see citing works
Bardus, M., Blake, H., Lloyd, S., & Suzanne Suggs, L. (2014) · International Journal of Workplace Health Management · doi:10.1108/ijwhm-11-2013-0040 · Cited by 54 · see citing works
Self-citationZiesemer, K., König, L., Boushey, C., Villinger, K., Wahl, D., Butscher, S., Müller, J., Reiterer, H., Schupp, H., & Renner, B. (2019) · JMIR mHealth and uHealth · doi:10.2196/15430 · Cited by 37 · see citing works
Hernández-Reyes, A., Cámara‐Martos, F., Molina‐Recio, G., Molina‐Luque, R., Romero‐Saldaña, M., & Rojas, R. M. (2019) · JMIR mhealth and uhealth · doi:10.2196/13747 · Cited by 46 · see citing works
Nahum-Shani, I., Rabbi, M., Yap, J., Philyaw-Kotov, M. L., Klasnja, P., Bonar, E. E., Cunningham, R. M., Murphy, S. A., & Walton, M. A. (2021) · Health Psychology · doi:10.1037/hea0001101 · Cited by 41 · see citing works
Current Multi-factor of Authentication: Approaches, Requirements, Attacks and Challenges
Mohammed, A. H. Y., Dziyauddin, R. A., & Latiff, L. A. (2023) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2023.0140119 · Cited by 42 · see citing works
Militello, L. K., Sobolev, M., Okeke, F., Adler, D. A., & Nahum-Shani, I. B. (2021) · JMIR Formative Research · doi:10.2196/30606 · Cited by 36 · see citing works
Machine Learning in Health Promotion and Behavioral Change: Scoping Review
Goh, Y. S., Yong, J. Q. Y. O., Chee, B. Q. H., Kuek, J. H. L., & Ho, C. S. H. (2022) · Journal of Medical Internet Research · doi:10.2196/35831 · Cited by 38 · see citing works
Alhasani, M., & Orji, R. (2024) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2023.2297330 · Cited by 48 · see citing works
Nyberg, A., Tistad, M., & Wadell, K. (2019) · Scandinavian Journal of Primary Health Care · doi:10.1080/02813432.2019.1569415 · Cited by 36 · see citing works
Flaherty, S. J., McCarthy, M. B., Collins, A., McCafferty, C., & McAuliffe, F. M. (2021) · European Journal of Marketing · doi:10.1108/ejm-06-2019-0531 · Cited by 35 · see citing works
Kolenik, T. (2022) · Internet of things · doi:10.1007/978-3-030-91181-2_7 · Cited by 34 · see citing works
Bell, L., Garnett, C. V., Qian, T., Perski, O., Williamson, E., & POTTS, H. W. W. (2020) · Journal of Medical Internet Research · doi:10.2196/23369 · Cited by 33 · see citing works
Just-in-time adaptive intervention to promote fluid consumption in patients with kidney stones.
Conroy, D. E., West, A. B., Brunke-Reese, D., Thomaz, E., & Streeper, N. M. (2020) · Health Psychology · doi:10.1037/hea0001032 · Cited by 34 · see citing works
Bucher, A., Blazek, E. S., & Symons, C. T. (2024) · Mayo Clinic Proceedings Digital Health · doi:10.1016/j.mcpdig.2024.05.007 · Cited by 37 · see citing works
Pereira‐Figueiredo, D., Nascimento, A., Cunha-Rodrigues, M. C., Brito, R., & Calaza, K. D. C. (2021) · Cellular and Molecular Neurobiology · doi:10.1007/s10571-021-01077-4 · Cited by 33 · see citing works
Context-Aware Delivery of Ecological Momentary Assessment
Aminikhanghahi, S., Schmitter‐Edgecombe, M., & Cook, D. J. (2019) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2019.2937116 · Cited by 29 · see citing works
Teixeira, R. G., Belisario, D. C., Fontrodona, X., Romero, I., Tomaz, A. I., García, M. H., Riganti, C., & Valente, A. (2021) · Inorganic Chemistry Frontiers · doi:10.1039/d0qi01344g · Cited by 31 · see citing works
Smartphone Sensing for the Well-Being of Young Adults: A Review
Meegahapola, L., & Gática-Pérez, D. (2020) · IEEE Access · doi:10.1109/access.2020.3045935 · Cited by 30 · see citing works
Kunkel, T., Hayduk, T., & Lock, D. (2023) · European Journal of Marketing · doi:10.1108/ejm-06-2021-0388 · Cited by 26 · see citing works
Singh, G., MacGillivray, M. K., Mills, P. B., Adams, J. R., Sawatzky, B., & Mortenson, W. B. (2019) · Journal of Medical Systems · doi:10.1007/s10916-019-1487-y · Cited by 26 · see citing works
When Do Drivers Interact with In-Vehicle Well-being Interventions?
Koch, K., Mishra, V., Liu, S., Berger, T., Fleisch, E., Kotz, D. F., & Wortmann, F. (2021) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3448116 · Cited by 25 · see citing works
Issom, D.-Z., Henriksen, A., Woldaregay, A. Z., Rochat, J., Lovis, C., & Hartvigsen, G. (2020) · JMIR Human Factors · doi:10.2196/14599 · Cited by 25 · see citing works
Reduced stress defence responses contribute to the higher toxicity of a pesticide under warming
Delnat, V., Swaegers, J., Asselman, J., & Stoks, R. (2020) · Molecular Ecology · doi:10.1111/mec.15667 · Cited by 24 · see citing works
Elvitigala, D. S., Scholl, P., Suriyaarachchi, H., Dissanayake, V., & Nanayakkara, S. C. (2021) · International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI) · doi:10.1145/3447526.3472023 · Cited by 24 · see citing works
The impact of customised mobile marketing on passenger shopping behaviour in the airport terminal
Wu, C., & Ma, N. K. (2022) · Journal of Retailing and Consumer Services · doi:10.1016/j.jretconser.2022.102941 · Cited by 24 · see citing works
Mobile applications for endometriosis management functionalities: Analysis and potential
Moumane, K., & Idri, A. (2023) · Scientific African · doi:10.1016/j.sciaf.2023.e01833 · Cited by 22 · see citing works
Chen, Y.-P., Woodward, J., Bista, D., Zhang, X., Singh, I., Obajemu, O., Shankar, M. N., Ross, K. M., Ruiz, J., & Anthony, L. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3641993 · Cited by 21 · see citing works
Workplace nomophobia: a systematic literature review
Hessari, H., Daneshmandi, F., Busch, P., & Smith, S. (2024) · Current Psychology · doi:10.1007/s12144-024-06222-y · Cited by 19 · see citing works
Künzler, F., Kramer, J.-N., & Kowatsch, T. (2017) · IEEE International Conference on Wireless and Mobile Computing, Networking and Communications (WiMob) · doi:10.1109/wimob.2017.8115839 · Cited by 21 · see citing works
Interaction Modalities for Notification Signals in Augmented Reality
Lazaro, M. J., Kim, S.-H., Lee, J. Y., Chun, J., & Yun, M. H. (2021) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3462244.3479898 · Cited by 20 · see citing works
Trần, T. B., Ambrens, M., Nguyễn, J., Coleman, E., Gilanyi, Y. L., Letton, M. E., Pandit, A., Lock, M. L., Thom, J. M., Sen, S., et al. (2024) · BMJ Open · doi:10.1136/bmjopen-2023-082345 · Cited by 19 · see citing works
Driven by notifications – exploring the effects of badge notifications on user experience
Bartoli, N., & Benedetto, S. (2022) · PLOS ONE · doi:10.1371/journal.pone.0270888 · Cited by 17 · see citing works
Parsons, C. E., Jensen, K. L., Roepstorff, A., Fjorback, L. O., & Linehan, C. (2019) · Digital Health · doi:10.1177/2055207619868550 · Cited by 17 · see citing works
Swain, S., Muduli, K., Kumar, A., V, & Luthra, S. (2023) · International Journal of Industrial Engineering and Operations Management · doi:10.1108/ijieom-12-2022-0067 · Cited by 19 · see citing works
Slade, C. R., Sun, Y., Chen, W., Chen, C.-C., Benzo, R. M., & Washington, P. Y. (2025) · JAMIA Open · doi:10.1093/jamiaopen/ooaf025 · Cited by 17 · see citing works
Kim, J., & Park, S. (2025) · IEEE Access · doi:10.1109/access.2025.3549033 · Cited by 14 · see citing works
Künzler, F., Kramer, J.-N., & Kowatsch, T. (2017) · Alexandria (UniSG) (University of St.Gallen) · doi:10.3929/ethz-b-000218431 · Cited by 14 · see citing works
de Menezes, M. P., Facin, A. C., Cardozo, M. V., Tinucci‐Costa, M., & Moraes, P. C. (2020) · Topics in companion animal medicine · doi:10.1016/j.tcam.2020.100489 · Cited by 15 · see citing works
King, Z., Yu, H., Vaessen, T., Myin‐Germeys, I., & Sano, A. (2023) · JMIR mhealth and uhealth · doi:10.2196/46347 · Cited by 13 · see citing works
Lekkas, D., Price, G. D., McFadden, J. R., & Jacobson, N. C. (2021) · Mindfulness · doi:10.1007/s12671-021-01723-4 · Cited by 13 · see citing works
Nordby, E. S., Gjestad, R., Kenter, R. M. F., Guribye, F., Mukhiya, S. K., Lundervold, A. J., & Nordgreen, T. (2022) · Frontiers in Digital Health · doi:10.3389/fdgth.2022.821031 · Cited by 14 · see citing works
Sher, V., Hatala, M., & Gašević, D. (2022) · Journal of Learning Analytics · doi:10.18608/jla.2022.6697 · Cited by 12 · see citing works
“A prominent ongoing debate, in relation to the process of tailoring notification delivery in the aforementioned personalized learning systems, concerns the effects of timing of push notifications—random versus fixed (Fischer et al., 2010; Tabuenca et al., 2015; Morrison et al., 2017).”
BackgroundJankovič, A., Kolenik, T., & Pejović, V. (2022) · Behavioral Sciences · doi:10.3390/bs12050116 · Cited by 13 · see citing works
Context-aware notification management systems for just-in-time adaptive interventions
Künzler, F. (2019) · IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops) · doi:10.1109/percomw.2019.8730874 · Cited by 12 · see citing works
Swain, S., Oyekola, P. O., & Muduli, K. (2022) · International Journal of Social Ecology and Sustainable Development · doi:10.4018/ijsesd.313963 · Cited by 12 · see citing works
Shim, Y., Scotney, V. S., & Tay, L. (2021) · The Journal of Positive Psychology · doi:10.1080/17439760.2021.1913642 · Cited by 11 · see citing works
Attrition rate and predictors of a monitoring mHealth application in adolescents with obesity
Desmet, M., Fillon, A., Thivel, D., Tanghe, A., & Braet, C. (2023) · Pediatric Obesity · doi:10.1111/ijpo.13071 · Cited by 11 · see citing works
Cardiovascular Risk in Women Deprived of Freedom from a Public Prison in Mato Grosso, Brazil
Hachbardt, N. B., Hattori, T. Y., do Nascimento, V. F., da Silva, J. H., Terças‐Trettel, A. C. P., Vivi, V. K., & Atanaka, M. (2020) · High Blood Pressure & Cardiovascular Prevention · doi:10.1007/s40292-020-00365-2 · Cited by 11 · see citing works
He, Q., & Zeng, Z. (2024) · Communication and the Public · doi:10.1177/20570473241239359 · Cited by 13 · see citing works
Informing nursing policy: An exploration of digital health research by nurses in England
O’Connor, S., Cave, L., & Philips, N. (2024) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2024.105381 · Cited by 10 · see citing works
Categorization Framework for Usability Issues of Smartwatches and Pedometers for the Older Adults
Khakurel, J., Knutas, A., Melkas, H., Penzenstadler, B., Fu, B., & Porras, J. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-92049-8_7 · Cited by 10 · see citing works
Røed, M. H., Vik, F. N., Hillesund, E. R., Lippevelde, W. V., Medin, A. C., & Øverby, N. C. (2020) · JMIR Human Factors · doi:10.2196/18171 · Cited by 9 · see citing works
Zhou, X., Rau, P. P., & Liu, X. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449077 · Cited by 9 · see citing works
Hatfield, M., Ashcroft, E., Maguire, S., Kershaw, L., & Ciccarelli, M. (2022) · Journal of Autism and Developmental Disorders · doi:10.1007/s10803-022-05542-x · Cited by 9 · see citing works
Caffeine exposure ameliorates acute ischemic cell death in avian developing retina
Pereira-Figueiredo, D., Brito, R., Araújo, D., Nascimento, A. A., Lyra, E. B., Cheibub, A. M. S. S., Netto, A. D. P., Ventura, A., Paes-de-Carvalho, R., & Calaza, K. (2020) · Purinergic Signalling Purinergic Signalling · doi:10.1007/s11302-020-09687-1 · Cited by 9 · see citing works
Optimizing Forecasted Activity Notifications with Reinforcement Learning
Fikry, M., & Inoue, S. (2023) · Sensors · doi:10.3390/s23146510 · Cited by 8 · see citing works
A survey of cyber-physical system implementations of real-time personalized interventions
Steele, R. J., Hillsgrove, T., Khoshavi, N., & Jaimes, L. G. (2021) · Journal of Ambient Intelligence and Humanized Computing · doi:10.1007/s12652-021-03263-0 · Cited by 8 · see citing works
Chaudry, B., Islam, A., & Matthieu, M. M. (2021) · JMIR Formative Research · doi:10.2196/30640 · Cited by 8 · see citing works
AI-Enabled Proactive mHealth: A Review
Sulaiman, M., Håkansson, A., & Karlsen, R. (2021) · Communications in computer and information science · doi:10.1007/978-3-030-94209-0_9 · Cited by 8 · see citing works
Receptivity to mobile health interventions
Keller, R., von Wangenheim, F., Mair, J. L., & Kowatsch, T. (2022) · Elsevier eBooks · doi:10.1016/b978-0-323-90045-4.00006-x · Cited by 8 · see citing works
Smartphone-Related Accidents in Children and Adolescents
Wagner, R., Gosemann, J., Sorge, I., Hubertus, J., Lacher, M., & Mayer, S. (2019) · Pediatric Emergency Care · doi:10.1097/pec.0000000000001781 · Cited by 8 · see citing works
Morrison, L. G., Hargood, C., Pejović, V., Geraghty, A. W. A., Lloyd, S., Goodman, N., Michaelides, D. T., Weston, A., Musolesi, M., Weal, M. J., et al. (2018) · PLoS ONE · doi:10.1371/journal.pone.0198008 · Cited by 8 · see citing works
Self-citationWhy Machines Matter for Survey and Social Science Researchers
Buskirk, T. D., & Kirchner, A. (2020) · doi:10.1002/9781118976357.ch1 · Cited by 8 · see citing works
Alhasani, M., & Orji, R. (2024) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2024.2434172 · Cited by 8 · see citing works
Digital Health Interventions to Promote Physical Activity Among Adolescents: Systematic Review
Fan, R. S., Jiang, J. J., Zhou, Q. Y., Zhang, X. Y., Wu, Z. H., & Ji, L. (2026) · Journal of Medical Internet Research · doi:10.2196/82395 · Cited by 7 · see citing works
Scrutable and Persuasive Push-Notifications
Fraser, K., Yousuf, B., & Conlan, O. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-17287-9_6 · Cited by 7 · see citing works
Weerasekara, M., & Smedberg, Å. B. (2019) · International Conference on Advanced Computer Science and Information Systems · doi:10.1109/icacsis47736.2019.8979813 · Cited by 7 · see citing works
Yi, J., Lee, S., & Lee, S. H. (2024) · Frontiers in Public Health · doi:10.3389/fpubh.2024.1442064 · Cited by 6 · see citing works
Nomeikaitė, A., Geležėlytė, O., Böttche, M., Andersson, G., & Kazlauskas, E. (2024) · JMIR Mental Health · doi:10.2196/62782 · Cited by 5 · see citing works
Evaluation of Priority-Dependent Notifications for Smart Glasses Based on Peripheral Visual Cues
Faulhaber, A. K., Hoppe, M., & Schmidt, L. (2022) · i-com · doi:10.1515/icom-2022-0022 · Cited by 6 · see citing works
Nguyen, B., Lai, J., Qureshi, H., Marcotullio, C., Labbaf, S., Wang, Y., Jafarlou, S., Dutt, N. D., Rahmani, A. M., & Borelli, J. L. (2025) · JMIR Formative Research · doi:10.2196/70528 · Cited by 6 · see citing works
Shiroyama, R., Wang, M., & Yoshimura, C. (2020) · Annales de Limnologie - International Journal of Limnology · doi:10.1051/limn/2020010 · Cited by 6 · see citing works
Pérez, C., Díaz-Roa, A., Bernal-Rosas, Y., Arenas, N. E., Kalume, D. E., Côrtes, L. M. D. C., da Silva, P. I., Varela, Y., Patarroyo, M. A., Torres, O., et al. (2021) · Memórias do Instituto Oswaldo Cruz · doi:10.1590/0074-02760200587 · Cited by 6 · see citing works
White, K. M., Carr, E., Leightley, D., Matcham, F., Conde, P., Ranjan, Y., Simblett, S. K., Dawe‐Lane, E., Williams, L., Henderson, C., et al. (2023) · JMIR mhealth and uhealth · doi:10.2196/44214 · Cited by 7 · see citing works
From sensing to intervention for mental and behavioral health
Mishra, V. (2019) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/3341162.3349304 · Cited by 6 · see citing works
Crafting Usable Quantified Self-wearable Technologies for Older Adult
Khakurel, J., Knutas, A., Melkas, H., Penzenstadler, B., & Porras, J. (2018) · Advances in intelligent systems and computing · doi:10.1007/978-3-319-94619-1_8 · Cited by 6 · see citing works
Wiratunga, N., Cooper, K., Wijekoon, A., Palihawadana, C., Mendham, V., Reiter, E., & Martin, K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.14067 · Cited by 19 · see citing works
Luštrek, M., Lukan, J., Bolliger, L., Lauwerier, E., & Clays, E. (2023) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/3594739.3611326 · Cited by 5 · see citing works
O’Connor, S. R., Kee, F., Thompson, D. R., Cupples, M., Donnelly, M., & Heron, N. (2021) · Digital Health · doi:10.1177/20552076211065271 · Cited by 5 · see citing works
From Sensing to Acting—Can Pervasive Computing Change the World?
Bardram, J. E. (2022) · IEEE Pervasive Computing · doi:10.1109/mprv.2022.3182489 · Cited by 5 · see citing works
Proposing Design Recommendations for an Intelligent Recommender System Logging Stress
Visuri, A., Poguntke, R., & Kuosmanen, E. (2018) · International Conference on Mobile and Ubiquitous Multimedia (MUM) · doi:10.1145/3282894.3289733 · Cited by 5 · see citing works
Brogly, C., Shoemaker, J. K., Lizotte, D. J., Kueper, J. K., & Bauer, M. (2021) · JMIR Formative Research · doi:10.2196/29160 · Cited by 5 · see citing works
Inagaki, S., Kato, K., Matsuda, T., Abe, K., Kurebayashi, S., Mihara, M., Azuma, D., Takabe, M., Abe, Y., & Yasuda, H. (2025) · Technology and Health Care · doi:10.1177/09287329251332454 · Cited by 4 · see citing works
- Authors
- Morrison, L. G., Hargood, C., Pejovic, V., Geraghty, A. W. A., Lloyd, S., Goodman, N., Michaelides, D. T., Weston, A., Musolesi, M., Weal, M. J., & Yardley, L.
- Affiliations
- University of Southampton (GB)
- Bournemouth University (GB)
- Durham University (GB)
- Gateshead Council (GB)
- University College London (GB)
- Corresponding author
- Morrison, Leanne G.
- ORCID
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2017-01-03
- Volume
- 12
- Issue
- 1
- Pages
- e0169162
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC5207732
- OpenAlex
- W1771486365
- Cited by
- 162
- References
- 41
- Citations by year
- 2026: 13 · 2025: 19 · 2024: 18 · 2023: 15 · 2022: 20 · 2021: 28 · 2020: 19 · 2019: 16 · 2018: 9 · 2017: 3 · 2014: 1
- FWCI
- 14.68
- Type
- journal article
- Language
- en
- Update notices
- correction (2018-05-22)
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Teesside University Research Portal (Teesside University) (publishedVersion, cc-by)
- Explore Bristol Research (publishedVersion, cc-by)
- Bournemouth University Research Online (Bournemouth University) (acceptedVersion)
- ePrints Soton (University of Southampton) (acceptedVersion)
- Archivio istituzionale della ricerca (Alma Mater Studiorum Università di Bologna) (submittedVersion, cc-by)
- TeesRep (Teesside University) (submittedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- UCL Discovery (University College London) (submittedVersion)
- Figshare (submittedVersion, cc-by)
- Bristol Research (University of Bristol) (submittedVersion, cc-by)
- Funders
- Engineering and Physical Sciences Research Council (EP/I032673/1)
- Topics
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Social Sciences › Psychology › Clinical Psychology › Mindfulness and Compassion Interventions
- Keywords
- push notifications, notification timing, smartphone-based intervention, stress management, healthy mind, user engagement, context-aware notifications, mobile health
- MeSH terms
- Smartphone, Machine Learning, Adolescent, Adult, Algorithms, Automation, Female, United Kingdom, Health Promotion, Humans, Male, Middle Aged, Public Health, Quality of Life, Stress, Psychological, Health Behavior, Telemedicine, Geographic Information Systems, Young Adult, Text Messaging, Accelerometry
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
