Assessing Real-Time Moderation for Developing Adaptive Mobile Health Interventions for Medical Interns: Micro-Randomized Trial
Open accessAbstract
Background: Individuals in stressful work environments often experience mental health issues, such as depression. Reducing depression rates is difficult because of persistently stressful work environments and inadequate time or resources to access traditional mental health care services. Mobile health (mHealth) interventions provide an opportunity to deliver real-time interventions in the real world. In addition, the delivery times of interventions can be based on real-time data collected with a mobile device. To date, data and analyses informing the timing of delivery of mHealth interventions are generally lacking. Objective: This study aimed to investigate when to provide mHealth interventions to individuals in stressful work environments to improve their behavior and mental health. The mHealth interventions targeted 3 categories of behavior: mood, activity, and sleep. The interventions aimed to improve 3 different outcomes: weekly mood (assessed through a daily survey), weekly step count, and weekly sleep time. We explored when these interventions were most effective, based on previous mood, step, and sleep scores. Methods: We conducted a 6-month micro-randomized trial on 1565 medical interns. Medical internship, during the first year of physician residency training, is highly stressful, resulting in depression rates several folds higher than those of the general population. Every week, interns were randomly assigned to receive push notifications related to a particular category (mood, activity, sleep, or no notifications). Every day, we collected interns’ daily mood valence, sleep, and step data. We assessed the causal effect moderation by the previous week’s mood, steps, and sleep. Specifically, we examined changes in the effect of notifications containing mood, activity, and sleep messages based on the previous week’s mood, step, and sleep scores. Moderation was assessed with a weighted and centered least-squares estimator. Results: We found that the previous week’s mood negatively moderated the effect of notifications on the current week’s mood with an estimated moderation of −0.052 (P=.001). That is, notifications had a better impact on mood when the studied interns had a low mood in the previous week. Similarly, we found that the previous week’s step count negatively moderated the effect of activity notifications on the current week’s step count, with an estimated moderation of −0.039 (P=.01) and that the previous week’s sleep negatively moderated the effect of sleep notifications on the current week’s sleep with an estimated moderation of −0.075 (P<.001). For all three of these moderators, we estimated that the treatment effect was positive (beneficial) when the moderator was low, and negative (harmful) when the moderator was high. Conclusions: These findings suggest that an individual’s current state meaningfully influences their receptivity to mHealth interventions for mental health. Timing interventions to match an individual’s state may be critical to maximizing the efficacy of interventions. Trial Registration: ClinicalTrials.gov NCT03972293; http://clinicaltrials.gov/ct2/show/NCT03972293
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In a microrandomized trial, delivering an intervention when individuals experienced lower levels of mental health indicated by low mood, poor sleep, and reduced activity was beneficial, whereas intervening when the individuals were doing well had a negative effect.
“The authors found that delivering an intervention when individuals experienced lower levels of MH indicated by low mood, poor sleep, and reduced activity was beneficial, whereas intervening when the individuals were doing well had a negative effect [55].”
From Optimization of a Transdiagnostic Mobile Emotion Regulation Intervention for University Students: Protocol for a Microrandomized Trial (2023) · cites this paper as [55]
These factors seem to play a role in intervention functioning.
“These factors seem to play a role in intervention functioning [51,55,56].”
From Optimization of a Transdiagnostic Mobile Emotion Regulation Intervention for University Students: Protocol for a Microrandomized Trial (2023) · cites this paper as [55]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The PHQ-9: Validity of a brief depression severity measure
Kroenke, K., Spitzer, R. L., & Williams, J. B. W. (2001) · Journal of General Internal Medicine · doi:10.1046/j.1525-1497.2001.016009606.x · Cited by 44,910 · see citing works
Statistical Analysis with Missing Data
Little, R. J. A., & Rubin, D. B. (2002) · Wiley Series in Probability and Statistics · doi:10.1002/9781119013563 · Cited by 8,473 · see citing works
Selective Cortical Undercutting As a Means of Modifying and Studying Frontal Lobe Function in Man
Scoville, W. B. (1949) · Journal of neurosurgery · doi:10.3171/jns.1949.6.1.0065 · Cited by 7,373 · see citing works
Ecological momentary assessment
Shiffman, S., Stone, A. A., & Hufford, M. R. (2008) · Annual Review of Clinical Psychology · doi:10.1146/annurev.clinpsy.3.022806.091415 · Cited by 6,454 · see citing works
Prince, S. A., Adamo, K. B., Hamel, M., Hardt, J., Connor Gorber, S., & Tremblay, M. (2008) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/1479-5868-5-56 · Cited by 3,162 · see citing works
Baglioni, C., Battagliese, G., Feige, B., Spiegelhalder, K., Nissen, C., Voderholzer, U., Lombardo, C., & Riemann, D. (2011) · Journal of Affective Disorders · doi:10.1016/j.jad.2011.01.011 · Cited by 2,779 · see citing works
Mediators and Moderators of Treatment Effects in Randomized Clinical Trials
Kraemer, H. C., Wilson, G. T., Fairburn, C. G., & Agras, W. S. (2002) · Archives of General Psychiatry · doi:10.1001/archpsyc.59.10.877 · Cited by 2,562 · cited 1× · see citing works
“Standard single–time point RCTs only inform moderation by baseline variables [31] and do not permit the discovery of time-varying moderators.”
BackgroundA behavior model for persuasive design
Fogg, B. (2009) · Proceedings of the 4th International Conference on Persuasive Technology · doi:10.1145/1541948.1541999 · Cited by 2,328 · see citing works
Nahum-Shani, I., Smith, S. N., Spring, B. J., Collins, L. M., Witkiewitz, K., Tewari, A., & Murphy, S. A. (2018) · Annals of Behavioral Medicine · doi:10.1007/s12160-016-9830-8 · Cited by 2,171 · see citing works
Self-citationTheRPackagegeepackfor Generalized Estimating Equations
Halekoh, U., Højsgaard, S., & Yan, J. (2006) · Journal of Statistical Software · doi:10.18637/jss.v015.i02 · Cited by 1,921 · see citing works
The Economic Burden of Adults With Major Depressive Disorder in the United States (2005 and 2010)
Greenberg, P. E., Fournier, A.-A., Sisitsky, T., Pike, C. T., & Kessler, R. C. (2015) · The Journal of Clinical Psychiatry · doi:10.4088/jcp.14m09298 · Cited by 1,779 · see citing works
Toward a theory of motivational interviewing
Miller, W. R., & Rose, G. S. (2009) · American Psychologist · doi:10.1037/a0016830 · Cited by 1,603 · see citing works
Detecting and monitoring depression with a two-item questionnaire (PHQ-2)
Löwe, B., Kroenke, K., & Gräfe, K. (2005) · Journal of Psychosomatic Research · doi:10.1016/j.jpsychores.2004.09.006 · Cited by 1,473 · see citing works
Physical activity, exercise, depression and anxiety disorders
Ströhle, A. (2009) · Journal of Neural Transmission · doi:10.1007/s00702-008-0092-x · Cited by 1,269 · see citing works
Mata, D. A., Ramos, M. A., Bansal, N., Khan, R., Guille, C., Di Angelantonio, E., & Sen, S. (2015) · JAMA · doi:10.1001/jama.2015.15845 · Cited by 1,255 · see citing works
Self-citationCollins, L. M., Murphy, S. A., & Strecher, V. (2007) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2007.01.022 · Cited by 1,082 · see citing works
Health behavior models in the age of mobile interventions: are our theories up to the task?
Riley, W. T., Rivera, D. E., Atienza, A. A., Nilsen, W., Allison, S. M., & Mermelstein, R. (2011) · Translational Behavioral Medicine · doi:10.1007/s13142-011-0021-7 · Cited by 978 · see citing works
Understanding tailoring in communicating about health
Hawkins, R. P., Kreuter, M., Resnicow, K., Fishbein, M., & Dijkstra, A. (2008) · Health Education Research · doi:10.1093/her/cyn004 · Cited by 934 · see citing works
A meta-analysis of computer-tailored interventions for health behavior change
Krebs, P., Prochaska, J. O., & Rossi, J. S. (2010) · Preventive Medicine · doi:10.1016/j.ypmed.2010.06.004 · Cited by 854 · see citing works
Microrandomized trials: An experimental design for developing just-in-time adaptive interventions.
Klasnja, P., Hekler, E. B., Shiffman, S., Boruvka, A., Almirall, D., Tewari, A., & Murphy, S. A. (2015) · Health Psychology · doi:10.1037/hea0000305 · Cited by 671 · see citing works
Self-citationHarnessing context sensing to develop a mobile intervention for depression
Burns, M. N., Begale, M., Duffecy, J., Gergle, D., Karr, C. J., Giangrande, E., & Mohr, D. C. (2011) · Journal of Medical Internet Research · doi:10.2196/jmir.1838 · Cited by 646 · cited 2× · see citing works
“In two studies [21,22], the authors demonstrated the benefits of timing mHealth intervention delivery based on real-time variables.”
BackgroundWork-related stress and depressive disorders
Tennant, C. (2001) · Journal of Psychosomatic Research · doi:10.1016/s0022-3999(01)00255-0 · Cited by 605 · see citing works
Psychosocial stress and impaired sleep
Åkerstedt, T. (2006) · Scandinavian Journal of Work, Environment & Health · doi:10.5271/sjweh.1054 · Cited by 585 · see citing works
What Is Sufficient Evidence for the Reliability and Validity of Patient-Reported Outcome Measures?
Frost, M. H., Reeve, B. B., Liepa, A. M., Stauffer, J. W., & Hays, R. D. (2007) · Value in Health · doi:10.1111/j.1524-4733.2007.00272.x · Cited by 572 · see citing works
Does monitoring goal progress promote goal attainment? A meta-analysis of the experimental evidence.
Harkin, B., Webb, T. L., Chang, B. P. I., Prestwich, A., Conner, M., Kellar, I., Benn, Y., & Sheeran, P. (2016) · Psychological Bulletin · doi:10.1037/bul0000025 · Cited by 564 · see citing works
The Psychology of Fatigue: Work, Effort and Control
Hockey, R. (2013) · doi:10.1017/cbo9781139015394 · Cited by 350 · see citing works
Motivational Interviewing: moving from why to how with autonomy support
Resnicow, K., & McMaster, F. (2012) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/1479-5868-9-19 · Cited by 320 · see citing works
Practical and statistical issues in missing data for longitudinal patient-reported outcomes
Bell, M. L., & Fairclough, D. L. (2013) · Statistical Methods in Medical Research · doi:10.1177/0962280213476378 · Cited by 285 · see citing works
Rizvi, S. L., Dimeff, L. A., Skutch, J., Carroll, D., & Linehan, M. M. (2011) · Behavior Therapy · doi:10.1016/j.beth.2011.01.003 · Cited by 271 · cited 5× · see citing works
“Mobile devices hold the power to deliver just-in-time adaptive interventions (JITAIs) [7] to individuals during times when they are able to receive and respond to them.”
MethodInfluentialProudfoot, J., Clarke, J., Birch, M.-R., Whitton, A. E., Parker, G., Manicavasagar, V., Harrison, V., Christensen, H., & Hadzi-Pavlovic, D. (2013) · BMC Psychiatry · doi:10.1186/1471-244x-13-312 · Cited by 264 · see citing works
Just-in-the-Moment Adaptive Interventions (JITAI): A Meta-Analytical Review
Wang, L., & Miller, L. C. (2020) · Health Communication · doi:10.1080/10410236.2019.1652388 · Cited by 264 · see citing works
An overview of self-monitoring research in assessment and treatment.
Korotitsch, W. J., & Nelson-Gray, R. O. (1999) · Psychological Assessment · doi:10.1037/1040-3590.11.4.415 · Cited by 241 · see citing works
Klasnja, P., Smith, S., Seewald, N. J., Lee, A., Hall, K., Luers, B., Hekler, E. B., & Murphy, S. A. (2019) · Annals of Behavioral Medicine · doi:10.1093/abm/kay067 · Cited by 241 · see citing works
The Role of Feedback in the Process of Health Behavior Change
DiClemente, C. C., Marinilli, A. S., Singh, M., & Bellino, L. (2001) · American Journal of Health Behavior · doi:10.5993/ajhb.25.3.8 · Cited by 215 · see citing works
Bidargaddi, N., Almirall, D., Murphy, S., Nahum-Shani, I., Kovalcik, M., Pituch, T., Maaieh, H., & Strecher, V. (2018) · JMIR mHealth and uHealth · doi:10.2196/10123 · Cited by 189 · see citing works
Dynamic Models of Behavior for Just-in-Time Adaptive Interventions
Spruijt‐Metz, D., & Nilsen, W. (2014) · IEEE Pervasive Computing · doi:10.1109/mprv.2014.46 · Cited by 188 · see citing works
Miner, A., Kuhn, E., Hoffman, J. E., Owen, J. E., Ruzek, J. I., & Taylor, C. B. (2016) · Psychological Trauma: Theory, Research, Practice, and Policy · doi:10.1037/tra0000092 · Cited by 185 · see citing works
Treatment burden and treatment fatigue as barriers to health.
Heckman, B. W., Mathew, A. R., & Carpenter, M. J. (2015) · Current Opinion in Psychology · doi:10.1016/j.copsyc.2015.03.004 · Cited by 179 · cited 4× · see citing works
“Treatment fatigue is pervasive in mHealth [7] and for individuals with heavy workloads [9].”
BackgroundInfluentialKristjánsdóttir, Ó. B., Fors, E. A., Eide, E., Finset, A., Stensrud, T. L., van Dulmen, S., Wigers, S. H., & Eide, H. (2013) · Journal of Medical Internet Research · doi:10.2196/jmir.2249 · Cited by 176 · see citing works
Possemato, K., Kuhn, E., Johnson, E., Hoffman, J. E., Owen, J. E., Kanuri, N., De Stefano, L., & Brooks, E. (2016) · General Hospital Psychiatry · doi:10.1016/j.genhosppsych.2015.09.005 · Cited by 167 · see citing works
Strategies for mHealth Research: Lessons from 3 Mobile Intervention Studies
Ben‐Zeev, D., Schueller, S. M., Begale, M. J., Duffecy, J. L., Kane, J. M., & Mohr, D. C. (2014) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-014-0556-2 · Cited by 167 · see citing works
Morrison, L. G., Hargood, C., Pejovic, V., Geraghty, A. W. A., Lloyd, S., Goodman, N., Michaelides, D. T., Weston, A., Musolesi, M., Weal, M. J., et al. (2017) · PLoS ONE · doi:10.1371/journal.pone.0169162 · Cited by 162 · see citing works
Assessing time-varying causal effect moderation in mobile health
Boruvka, A., Almirall, D., Witkiewitz, K., & Murphy, S. A. (2018) · Journal of the American Statistical Association · doi:10.1080/01621459.2017.1305274 · Cited by 152 · see citing works
Working conditions and health behaviours among employed women and men: the Helsinki Health Study.
Lallukka, T., Sarlio-Lähteenkorva, S., Roos, E., Laaksonen, M., Rahkonen, O., & Lahelma, E. (2004) · Preventive Medicine · doi:10.1016/j.ypmed.2003.09.027 · Cited by 151 · see citing works
Pielot, M., Cardoso, B., Katevas, K., Serrà, J., Matic, A., & Oliver, N. (2017) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3130956 · Cited by 149 · see citing works
Sarker, H., Sharmin, M., Ali, A. A., Rahman, M. M., Bari, R., Hossain, S. M., & Kumar, S. (2014) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/2632048.2636082 · Cited by 121 · see citing works
Danaher, B. G., Brendryen, H., Seeley, J. R., Tyler, M. S., & Woolley, T. (2015) · Internet Interventions · doi:10.1016/j.invent.2015.01.002 · Cited by 102 · see citing works
Kalmbach, D. A., Fang, Y., Arnedt, J. T., Cochran, A. L., Deldin, P. J., Kaplin, A. I., & Sen, S. (2018) · Journal of General Internal Medicine · doi:10.1007/s11606-018-4373-2 · Cited by 95 · see citing works
Self-citationUnleashing the Power of Prevention
Hawkins, J. D., Jenson, J. M., Catalano, R. F., Fräser, M. W., Botvin, G. J., Shapiro, V. B., Brown, C. H., Beardslee, W. R., Brent, D. A., Leslie, L. K., et al. (2015) · NAM Perspectives · doi:10.31478/201506c · Cited by 89 · see citing works
Smyth, J. M., & Heron, K. E. (2016) · Wireless Health · doi:10.1109/wh.2016.7764561 · Cited by 87 · see citing works
Characterizing Periodic Messaging Interventions Across Health Behaviors and Media: Systematic Review
León, E. D., Fuentes, L. W., & Cohen, J. E. (2014) · Journal of Medical Internet Research · doi:10.2196/jmir.2837 · Cited by 81 · see citing works
Newman, M. G., Przeworski, A., Consoli, A. J., & Taylor, C. B. (2014) · Psychotherapy · doi:10.1037/a0032519 · Cited by 71 · see citing works
Depressive symptoms, daily stress, and adherence in late adolescents with type 1 diabetes.
Baucom, K. J. W., Queen, T. L., Wiebe, D. J., Turner, S. L., Wolfe, K. L., Godbey, E., Fortenberry, K. T., Mansfield, J. H., & Berg, C. A. (2015) · Health Psychology · doi:10.1037/hea0000219 · Cited by 67 · see citing works
A multiple imputation strategy for sequential multiple assignment randomized trials
Shortreed, S. M., Laber, E., Scott Stroup, T., Pineau, J., & Murphy, S. A. (2014) · Statistics in Medicine · doi:10.1002/sim.6223 · Cited by 58 · see citing works
The Relation of Stress and Depression to Interdialytic Weight Gain in Hemodialysis Patients
Everett, K. D., Brantley, P. J., Sletten, C. D., Jones, G. N., & McKnight, G. T. (1995) · Behavioral Medicine · doi:10.1080/08964289.1995.9933739 · Cited by 51 · see citing works
Ahmedani, B. K., Crotty, N., Abdulhak, M. M., & Ondersma, S. J. (2013) · Behavioral Medicine · doi:10.1080/08964289.2013.867827 · Cited by 31 · see citing works
Suggs, L. S., Blake, H., Bardus, M., & Lloyd, S. (2013) · Journal of Health Services Research & Policy · doi:10.1177/1355819613478001 · Cited by 20 · see citing works
Standardized Effect Sizes for Preventive Mobile Health Interventions in Micro-randomized Trials
Luers, B., Klasnja, P., & Murphy, S. A. (2018) · Prevention Science · doi:10.1007/s11121-017-0862-5 · Cited by 20 · see citing works
Theoretical Models to Inform Technology-Based Health Behavior Interventions
Riley, W. T. (2014) · Oxford University Press eBooks · doi:10.1093/med/9780199314027.003.0002 · Cited by 10 · see citing works
Zhang, S., & Elhadad, N. (2016) · PubMed Central · Cited by 10 · see citing works
Flagged first, then by citations
2 of the 62 references couldn’t be matched to a known paper.
Research works printed but not linked: 3
- [46]Miller WR Rollnick S Motivational Interviewing: Preparing People for Change. Second Edition 2002 New York, US The Guilford Press
- [49]Bandura A Social Foundations of Thought and Action: A Social Cognitive Theory 1986 Upper Saddle River, New Jersey Prentice Hall
- [65]Cohen J Statistical Power Analysis for the Behavioral Sciences. Second Edition 1988 Mahwah, New Jersey, United States Lawrence Erlbaum Associates
Cited by
Sorted by citations
Individual-level interventions for reducing occupational stress in healthcare workers
Tamminga, S. J., Emal, L. M., Boschman, J. S., Levasseur, A., Thota, A. B., Ruotsalainen, J. H., Schelvis, R. M. C., Nieuwenhuijsen, K., & van der Molen, H. F. (2023) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd002892.pub6 · Cited by 123 · see citing works
Liu, J., Goetz, J., Sen, S., & Tewari, A. (2021) · JMIR mhealth and uhealth · doi:10.2196/23728 · Cited by 61 · see citing works
Self-citationA method for characterizing daily physiology from widely used wearables
Bowman, C., Huang, Y., Walch, O., Fang, Y., Frank, E., Tyler, J., Mayer, C., Stockbridge, C., Goldstein, C. A., Sen, S., et al. (2021) · Cell Reports Methods · doi:10.1016/j.crmeth.2021.100058 · Cited by 59 · see citing works
Self-citationDigital Phenotyping in Health Using Machine Learning Approaches: Scoping Review
Dlima, S. D., Shevade, S., Menezes, S. R., & Ganju, A. (2022) · JMIR Bioinformatics and Biotechnology · doi:10.2196/39618 · Cited by 53 · see citing works
Hsu, T. C., Whelan, P., Gandrup, J., Armitage, C. J., Cordingley, L., & McBeth, J. (2024) · British Journal of Health Psychology · doi:10.1111/bjhp.12766 · Cited by 52 · see citing works
An Ethics Checklist for Digital Health Research in Psychiatry: Viewpoint
Shen, F. X., Silverman, B. C., Monette, P., Kimble, S., Rauch, S. L., & Baker, J. T. (2022) · Journal of Medical Internet Research · doi:10.2196/31146 · Cited by 43 · see citing works
Bell, L., Garnett, C., Bao, Y., Cheng, Z., Qian, T., Perski, O., Potts, H. W. W., & Williamson, E. (2023) · JMIR mHealth and uHealth · doi:10.2196/38342 · Cited by 35 · see citing works
Microrandomized Trials: Developing Just-in-Time Adaptive Interventions for Better Public Health
Liu, X., Deliu, N., & Chakraborty, B. (2022) · American Journal of Public Health · doi:10.2105/ajph.2022.307150 · Cited by 36 · see citing works
Horwitz, A. G., Czyz, E. K., Al‐Dajani, N., Dempsey, W., Zhao, Z., Nahum-Shani, I. B., & Sen, S. (2022) · Journal of Affective Disorders · doi:10.1016/j.jad.2022.06.064 · Cited by 30 · see citing works
Self-citationComparative Effectiveness of Three Digital Interventions for Adults Seeking Psychiatric Services
Horwitz, A. G., Mills, E. D., Sen, S., & Bohnert, A. S. B. (2024) · JAMA Network Open · doi:10.1001/jamanetworkopen.2024.22115 · Cited by 27 · see citing works
Self-citationWang, J., Fang, Y., Frank, E., Walton, M. A., Burmeister, M., Tewari, A., Dempsey, W., NeCamp, T., Sen, S., & Wu, Z. (2023) · npj Digital Medicine · doi:10.1038/s41746-022-00746-y · Cited by 24 · see citing works
Self-citationGolbus, J. R., Shi, J., Gupta, K., Stevens, R., Jeganathan, V. S. E., Luff, E., Boyden, T. F., Mukherjee, B., Kohnstamm, S., Taralunga, V., et al. (2024) · Circulation Cardiovascular Quality and Outcomes · doi:10.1161/circoutcomes.123.010731 · Cited by 17 · see citing works
Prentice, C., Peven, K., Zhaunova, L., Nayak, V., Radović, T., Klepchukova, A., POTTS, H. W. W., & Ponzo, S. (2024) · BMC Digital Health · doi:10.1186/s44247-024-00092-x · Cited by 17 · see citing works
Takeuchi, H., Ishizawa, T., Kishi, A., Nakamura, T., Yoshiuchi, K., & Yamamoto, Y. (2024) · Journal of Medical Internet Research · doi:10.2196/49669 · Cited by 16 · see citing works
Wang, J., Wu, Z., Choi, S. W., Sen, S., Yan, X., Miner, J. A., Sander, A. M., Lyden, A. K., Troost, J. P., & Carlozzi, N. E. (2023) · JMIR Formative Research · doi:10.2196/43099 · Cited by 16 · see citing works
Self-citationGilley, K. N., Baroudi, L., Yu, M., Gainsburg, I., Reddy, N., Bradley, C. M., Cislo, C., Rozwadowski, M., Clingan, C., DeMoss, M., et al. (2022) · JMIR Mental Health · doi:10.2196/34645 · Cited by 16 · see citing works
Carlozzi, N. E., Choi, S. W., Wu, Z., Troost, J. P., Lyden, A. K., Miner, J. A., GRAVES, C. M., Wang, J., Yan, X., & Sen, S. (2022) · Rehabilitation Psychology · doi:10.1037/rep0000472 · Cited by 13 · see citing works
Self-citationCleary, J. L., Fang, Y., Sen, S., & Wu, Z. (2022) · PLoS ONE · doi:10.1371/journal.pone.0277350 · Cited by 13 · see citing works
Self-citationYao, J., Brunskill, E., Pan, W., Murphy, S. A., & Doshi‐Velez, F. (2020) · PubMed · doi:10.48550/arxiv.2004.06230 · Cited by 13 · see citing works
Horwitz, A. G., Hong, V., Eisenberg, D., Zheng, K., Albucher, R. C., Coryell, W. H., Pistorello, J., Favorite, T., & King, C. A. (2021) · Behavior Therapy · doi:10.1016/j.beth.2021.10.001 · Cited by 10 · see citing works
Participant Engagement in Microrandomized Trials of mHealth Interventions: Scoping Review
Leong, U., & Chakraborty, B. (2023) · JMIR mhealth and uhealth · doi:10.2196/44685 · Cited by 7 · see citing works
The effect of cognitive behavioral therapy text messages on mood: A micro-randomized trial
Avalos, M. R. A., Xu, J., Figueroa, C. A., Haro‐Ramos, A. Y., Chakraborty, B., & Aguilera, A. (2024) · PLOS Digital Health · doi:10.1371/journal.pdig.0000449 · Cited by 7 · see citing works
Laure, T., Remmerswaal, D., Konigorski, S., Engels, R. C. M. E., & Boffo, M. (2024) · Stress and Health · doi:10.1002/smi.3507 · Cited by 7 · see citing works
McCarthy, K., & Horwitz, A. G. (2025) · Journal of American College Health · doi:10.1080/07448481.2025.2458085 · Cited by 7 · see citing works
Laure, T., Engels, R. C. M. E., Remmerswaal, D., Spruijt-Metz, D., Konigorski, S., & Boffo, M. (2023) · JMIR Research Protocols · doi:10.2196/46603 · Cited by 6 · see citing works
“Within the field of MH, NeCamp et al [55] investigated the optimal timing for delivering a mobile MH intervention to medical interns.”
BackgroundStatistically Efficient Advantage Learning for Offline Reinforcement Learning in Infinite Horizons
Shi, C., Luo, S., Le, Y., Hongtu Zhu, B., & Song, R. (2022) · Journal of the American Statistical Association · doi:10.1080/01621459.2022.2106868 · Cited by 6 · see citing works
Shi, J., Wu, Z., & Dempsey, W. (2022) · Biometrika · doi:10.1093/biomet/asac065 · Cited by 6 · see citing works
Self-citationShapiro, B., Fang, Y., Sen, S., & Forger, D. B. (2024) · PLOS Digital Health · doi:10.1371/journal.pdig.0000439 · Cited by 7 · see citing works
Self-citationCarlozzi, N. E., Sander, A. M., Choi, S. W., Wu, Z., Miner, J. A., Lyden, A. K., GRAVES, C. M., & Sen, S. (2022) · PLoS ONE · doi:10.1371/journal.pone.0268726 · Cited by 7 · see citing works
Self-citationAgarwal, A. K., Ali, Z. S., Shofer, F. S., Xiong, R. A., Hemmons, J. E., Spencer, E. A., Abdel-Rahman, D., Sennett, B. J., & Delgado, M. K. (2022) · JMIR Formative Research · doi:10.2196/31894 · Cited by 6 · see citing works
Zhao, Y., & Choudhury, T. (2024) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/3675094.3678373 · Cited by 5 · see citing works
A meta-learning method for estimation of causal excursion effects to assess time-varying moderation
Shi, J., & Dempsey, W. (2025) · Biometrics · doi:10.1093/biomtc/ujaf129 · Cited by 5 · see citing works
Hassinger, A. B., Velez, C., Wang, J., Mador, M. J., Wilding, G. E., & Mishra, A. (2023) · Journal of Clinical Sleep Medicine · doi:10.5664/jcsm.10820 · Cited by 5 · see citing works
Valle, C. G., Nezami, B. T., O’Shea, N. G., Hatley, K. E., & Tate, D. F. (2025) · DIGITAL HEALTH · doi:10.1177/20552076251353267 · Cited by 4 · see citing works
Zhang, S., Dieciuc, M., Dilanchian, A. T., Lustria, M. L. A., Carr, D. C., Charness, N. H., He, Z., & Boot, W. R. (2024) · Gerontology and Geriatric Medicine · doi:10.1177/23337214231224571 · Cited by 5 · see citing works
Yaraghi, N., Ramesh, R., & Tayi, G. K. (2024) · Management Science · doi:10.1287/mnsc.2023.02603 · Cited by 4 · see citing works
A Scoping Review of Physical Activity Interventions for Physician Wellness
Huecker, M. R., Genova, G., & Shreffler, J. R. (2023) · American Journal of Lifestyle Medicine · doi:10.1177/15598276231178485 · Cited by 4 · see citing works
Testing stationarity and change point detection in reinforcement learning
Li, M., Shi, C., Wu, Z., & Fryźlewicz, P. (2025) · The Annals of Statistics · doi:10.1214/25-aos2501 · Cited by 4 · see citing works
Self-citationYan, X., Newman, M., Park, S. Y., Sander, A. M., Choi, S. W., Miner, J. A., Wu, Z., & Carlozzi, N. E. (2023) · JMIR Formative Research · doi:10.2196/47813 · Cited by 3 · see citing works
Self-citationMultivariate dynamic mediation analysis under a reinforcement learning framework
Luo, L., Shi, C., Wang, J., Wu, Z., & Li, L. (2025) · The Annals of Statistics · doi:10.1214/24-aos2475 · Cited by 3 · see citing works
Self-citationTransfusion probability as an alternative measure of lab‐guided medical decision‐making
Risk, M., Callum, J. L., Trentino, K. M., Murray, K., Zhao, L., Shi, X., Verma, A. A., Razak, F. A., & Raza, S. (2025) · Transfusion · doi:10.1111/trf.18235 · Cited by 3 · see citing works
Feasibility and acceptability of collecting passive phone usage and sensor data via Apple SensorKit
Funk, C., Zhao, Z., Horwitz, A. G., Fang, Y., Pereira‐Lima, K., Kheterpal, V., Sen, S., & Frank, E. (2025) · PLoS ONE · doi:10.1371/journal.pone.0329707 · Cited by 3 · see citing works
Self-citationLi, S., Li, X., Nan, N., Zhang, Y., Yan, Z., Xu, M., Zhang, Y., Chen, R., & Qin, G. (2026) · Reviews of Environmental Contamination and Toxicology · doi:10.1007/s44169-025-00108-1 · Cited by 2 · see citing works
Shi, J., Wu, Z., & Dempsey, W. (2025) · Journal of the American Statistical Association · doi:10.1080/01621459.2025.2516197 · Cited by 2 · see citing works
Self-citationPutting it all together: Connecting mobile technology to systems biology
Walch, O. (2020) · Current Opinion in Systems Biology · doi:10.1016/j.coisb.2020.07.012 · Cited by 2 · see citing works
Mobile Mental Health for Depression Assistance
Rodrigues, J. M., Oliveira, F., Ribeiro, C. P., & Santos, R. C. (2021) · Advances in psychology, mental health, and behavioral studies (APMHBS) book series · doi:10.4018/978-1-7998-8634-1.ch002 · Cited by 2 · see citing works
Wang, J., Fang, Y., Frank, E., Walton, M. A., Burmeister, M., Tewari, A., Dempsey, W., NeCamp, T., Sen, S., & Wu, Z. (2022) · medRxiv · doi:10.1101/2022.03.11.22272278 · Cited by 2 · see citing works
Self-citationDandil, Y., & Kingston, J. L. (2025) · Journal of Technology in Human Services · doi:10.1080/15228835.2025.2491347 · Cited by 1 · see citing works
Horwitz, A. G., Anand, S., Chen, M., McCarthy, K., Schueller, S. M., Walton, M., Sen, S., & King, C. A. (2026) · Internet Interventions · doi:10.1016/j.invent.2026.100912 · Cited by 2 · see citing works
Self-citationStress fitness: A neuroscientific approach to building emotional resilience
Akil, H., Maras, P. M., & Turner, C. A. (2026) · Neuron · doi:10.1016/j.neuron.2026.03.032 · Cited by 1 · see citing works
Shi, J., Wu, Z., & Dempsey, W. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.01681
Self-citationStrategies in Ecological Momentary Interventions for Mental Health Care in Adults: A Scoping Review
Ryu, G. W. (2024) · Journal of Digital Contents Society · doi:10.9728/dcs.2024.25.4.961
Generalized Fitted Q‐Iteration With Clustered Data
Hu, L., Wang, J., Wu, Z., & Shi, C. (2025) · Stat · doi:10.1002/sta4.70112
Self-citationA Personalised Well-Being App for Staff Working in Healthcare Settings: A Pilot Feasibility Study
Dandil, Y., McGillivray, H., & Kingston, J. L. (2026) · Journal of Technology in Human Services · doi:10.1080/15228835.2026.2629258
Valle, C. G., Nezami, B. T., Hatley, K. E., Diamond, M. A., Polzien, K. M., Wasser, H., Gottfredson, N. C., & Tate, D. F. (2026) · Contemporary Clinical Trials · doi:10.1016/j.cct.2026.108318
- Authors
- NeCamp, T., Sen, S., Frank, E., Walton, M. A., Ionides, E. L., Fang, Y., Tewari, A., & Wu, Z.
- Affiliations
- University of Michigan (US)
- Corresponding author
- NeCamp, Timothy
- ORCID
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2020-03-31
- Volume
- 22
- Issue
- 3
- Pages
- e15033
- ISSN
- 1438-8871 (ISSN-L)
- PMC
- PMC7157494
- OpenAlex
- W3013613965
- Cited by
- 55
- References
- 62
- Citations by year
- 2026: 5 · 2025: 10 · 2024: 12 · 2023: 9 · 2022: 11 · 2021: 6 · 2020: 2
- FWCI
- 6.33
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Journal of Medical Internet Research (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- National Science Foundation (CAREER, 1452099)
- American Foundation for Suicide Prevention (LSRG-0-059-16)
- National Institutes of Health
- Michigan Institute for Data Science, University of Michigan
- Rogel Cancer Center, University of Michigan (P30 CA046592)
- National Institute of Mental Health (R01MH101459)
- National Cancer Institute (P30CA046592)
- Topics
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Mental Health Research Topics
- Keywords
- push notifications, medical interns, physician depression, occupational stress, mood, physical activity, sleep, moderation, micro-randomized trial, step count, sleep duration, mood valence, physician well-being, smartphone data collection, wearable activity trackers, sleep trackers
- MeSH terms
- Female, Humans, Internship and Residency, Male, Telemedicine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
