Developing a Hypothetical Implementation Framework of Expectations for Monitoring Early Signs of Psychosis Relapse Using a Mobile App: Qualitative Study
Open accessPublished version of a preprint
The DOI matches its Crossref record; 62 of the 62 references it cites were found in a registry or an index.
Abstract
Background: Relapse is a common experience for people diagnosed with psychosis, which is associated with increased service costs and profound personal and familial distress. EMPOWER (Early signs Monitoring to Prevent relapse in psychosis and prOmote Well-being, Engagement, and Recovery) is a peer worker–supported digital intervention that aims to enable service users to self-monitor their mental health with the aim of encouraging self-management and the shared use of personal data to promote relapse prevention. Digital interventions have not been widely used in relapse prevention and, therefore, little is currently known about their likely implementation—both within trials and beyond. Objective: Seeking the perspectives of all relevant stakeholder groups is recommended in developing theories about implementation because this can reveal important group differences in understandings and assumptions about whether and for whom the intervention is expected to work. However, the majority of intervention implementation research has been retrospective. This study aimed to discover and theoretically frame implementation expectations in advance of testing and synthesize these data into a framework. Methods: To develop a hypothetical implementation framework, 149 mental health professionals, carers, and people diagnosed with psychosis participated in 25 focus groups in both Australia and the United Kingdom. An interview schedule informed by the normalization process theory was used to explore stakeholders’ expectations about the implementation of the EMPOWER intervention. Data were analyzed using thematic analysis and then theoretically framed using the Medical Research Council guidelines for understanding the implementation of complex interventions. Results: All groups expected that EMPOWER could be successfully implemented if the intervention generated data that were meaningful to mental health staff, carers, and service users within their unique roles. However, there were key differences between staff, carers, and service users about what facilitators and barriers that stakeholders believe exist for intervention implementation in both the cluster randomized controlled trial stage and beyond. For example, service user expectations mostly clustered around subjective user experiences, whereas staff and carers spoke more about the impact upon staff interactions with service users. Conclusions: A hypothetical implementation framework synthesized from stakeholder implementation expectations provides an opportunity to compare actual implementation data gathered during an ongoing clinical trial, giving valuable insights into the accuracy of these stakeholders’ previous expectations. This is among the first studies to assess and record implementation expectations for a newly developed digital intervention for psychosis in advance of testing in a clinical trial. Trial Registration: ISRCTN Registry ISRCTN99559262; http://www.isrctn.com/ISRCTN99559262
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“A negative prior experience formed a barrier to engaging with a DMHI [74,117,122], whereas a positive experience increased participants’ engagement [54,72,104,115,116,118-121].”
From Barriers to and facilitators of user engagement with digital mental health interventions: Systematic review (2021) · cites this paper as [117]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“A positive prior experience with technology [115,116], mental health services [54,74,104,117-119], and mental health technology [72,120,121] facilitated people’s intention to use interventions, as well as actual engagement.”
From Barriers to and facilitators of user engagement with digital mental health interventions: Systematic review (2021) · cites this paper as [117]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“and whether the intervention provided a clear advantage over past or current care received [103,117,121,155,171].”
From Barriers to and facilitators of user engagement with digital mental health interventions: Systematic review (2021) · cites this paper as [117]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“a feeling of empowerment over being in control of their health [14,32,84,93,95,104,117,122,124,170,174,175,196,206,207]”
From Barriers to and facilitators of user engagement with digital mental health interventions: Systematic review (2021) · cites this paper as [117]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“This perception was facilitated by whether users were able to understand the data presented to them [104,117,170]”
From Barriers to and facilitators of user engagement with digital mental health interventions: Systematic review (2021) · cites this paper as [117]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414 · cited 2×
“Thematic analysis is a qualitative method used to construct, analyze, and report on patterns within text data [34].”
MethodTong, A., Sainsbury, P., & Craig, J. (2007) · International Journal for Quality in Health Care · doi:10.1093/intqhc/mzm042 · Cited by 42,509 · cited 1×
“The methods are reported in line with the consolidated criteria for reporting qualitative research reporting recommendations for qualitative work [30]; a full checklist can be seen in Multimedia Appendix 1.”
Book Review: Focus Groups: A Practical Guide for Applied Research
McQuarrie, E. F., & Krueger, R. A. (1989) · Journal of Marketing Research · doi:10.2307/3172912 · Cited by 15,462
Gale, N. K., Heath, G., Cameron, E., Rashid, S., & Redwood, S. (2013) · BMC Medical Research Methodology · doi:10.1186/1471-2288-13-117 · Cited by 10,865 · cited 1×
“For stage 2, the MRC process evaluation framework [18] was identified as a suitable deductive coding framework [38] for placing the themes in an implementation theory context more relevant to the needs for a feasibility study where it may be too early to decide if normalization should be the goal.”
MethodProcess evaluation of complex interventions: Medical Research Council guidance
Moore, G. F., Audrey, S., Barker, M., Bond, L., Bonell, C., Hardeman, W., Moore, L., O'Cathain, A., Tinati, T., Wight, D., et al. (2015) · BMJ · doi:10.1136/bmj.h1258 · Cited by 6,204 · cited 9×
“The MRC framework for process evaluations recommends that implementation research should proactively include key stakeholders because those expected to engage with an intervention are likely to have relevant experiential knowledge, which is useful in understanding the implementation process during a trial [18].”
BackgroundMethodInfluentialMaking sense of implementation theories, models and frameworks
Nilsen, P. (2015) · Implementation Science · doi:10.1186/s13012-015-0242-0 · Cited by 4,447 · cited 1×
“Theory in implementation science implies some predictive capacity [23].”
BackgroundGreenhalgh, T., Wherton, J., Papoutsi, C., Lynch, J., Hughes, G., A'Court, C., Hinder, S., Fahy, N., Procter, R., & Shaw, S. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.8775 · Cited by 2,887 · cited 3×
“However, many effective digital interventions have failed to generalize from clinical trials into clinical practice [15,16].”
BackgroundMethodMurray, E., Treweek, S., Pope, C., MacFarlane, A., Ballini, L., Dowrick, C., Finch, T., Kennedy, A., Mair, F., O'Donnell, C., et al. (2010) · BMC Medicine · doi:10.1186/1741-7015-8-63 · Cited by 1,478
Development of a theory of implementation and integration: Normalization Process Theory
May, C. R., Mair, F., Finch, T., MacFarlane, A., Dowrick, C., Treweek, S., Rapley, T., Ballini, L., Ong, B. N., Rogers, A., et al. (2009) · Implementation Science · doi:10.1186/1748-5908-4-29 · Cited by 1,409 · cited 3×
“One such implementation theory, normalization process theory (NPT) [25] focuses on the work that groups and individuals do when interacting with an intervention and how they make sense of it, many intervention studies have successfully utilized NPT as a framework to guide research to more fully understand the implemen…”
MethodRobinson, D., Woerner, M. G., Alvir, J. M. J., Bilder, R., Goldman, R., Geisler, S., Koreen, A., Sheitman, B., Chakos, M., Mayerhoff, D., et al. (1999) · Archives of General Psychiatry · doi:10.1001/archpsyc.56.3.241 · Cited by 1,327 · cited 1×
“Relapse is common for many people diagnosed with schizophrenia [1].”
BackgroundGreenhalgh, T., Hinton, L., Finlay, T., Macfarlane, A., Fahy, N., Clyde, B., & Chant, A. (2019) · Health Expectations · doi:10.1111/hex.12888 · Cited by 1,059 · cited 1×
“Finally, PPI can range from consultation to stakeholders having decision making over the aims and conduct of a study [56].”
BackgroundImplementation, context and complexity
May, C. R., Johnson, M., & Finch, T. (2016) · Implementation Science · doi:10.1186/s13012-016-0506-3 · Cited by 978
Studying complexity in health services research: desperately seeking an overdue paradigm shift
Greenhalgh, T., & Papoutsi, C. (2018) · BMC Medicine · doi:10.1186/s12916-018-1089-4 · Cited by 962 · cited 1×
“A recent recommendation for undertaking complexity science–informed implementation research within health care services is to abandon attempts to simplify implementation research but rather explore implementation more inductively from multiple perspectives [55].”
BackgroundGalletly, C., Castle, D., Dark, F., Humberstone, V., Jablensky, A., Killackey, E., Kulkarni, J., McGorry, P., Nielssen, O., & Tran, N. (2016) · Australian & New Zealand Journal of Psychiatry · doi:10.1177/0004867416641195 · Cited by 877 · cited 8×
“Because I think the next generation of people coming through are going to have been grown up with technology and are going to be okay with using it... [Participant 8, Staff group 11, Australia] J Med Internet Res 2019 | vol. 21 | iss.”
BackgroundInfluentialMichie, S., Yardley, L., West, R., Patrick, K., & Greaves, F. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.7126 · Cited by 865 · cited 1×
“User experience has been described as a neglected area within digital intervention research [47] and psychosis more specifically [48].”
BackgroundBraithwaite, J., Churruca, K., Long, J. C., Ellis, L. A., & Herkes, J. (2018) · BMC Medicine · doi:10.1186/s12916-018-1057-z · Cited by 845 · cited 1×
“However, within a complexity science–informed understanding, context is defined by an intervention interacting with multiple enacted environments of different social actors [43].”
BackgroundThe dark side of coproduction: do the costs outweigh the benefits for health research?
Oliver, K., Kothari, A., & Mays, N. (2019) · Health Research Policy and Systems · doi:10.1186/s12961-019-0432-3 · Cited by 754 · cited 1×
“Despite the recommended involvement of patients and members of the public within implementation research [27] and widespread assumptions that consultation work can help researchers anticipate stakeholders’ needs, capacities, and priorities [28], the MRC guidelines on process evaluation [18] report substantial empirica…”
BackgroundMay, C. R., Cummings, A., Girling, M., Bracher, M., Mair, F. S., May, C. M., Murray, E., Myall, M., Rapley, T., & Finch, T. (2018) · Implementation Science · doi:10.1186/s13012-018-0758-1 · Cited by 683 · cited 1×
“One such implementation theory, normalization process theory (NPT) [25] focuses on the work that groups and individuals do when interacting with an intervention and how they make sense of it, many intervention studies have successfully utilized NPT as a framework to guide research to more fully understand the implemen…”
BackgroundTowards a general theory of implementation
May, C. (2013) · Implementation Science · doi:10.1186/1748-5908-8-18 · Cited by 656 · cited 1×
“Typically, implementation theory aims to create conceptual tools that enable researchers to describe, identify, and explain crucial elements of the implementation process and its outcomes [24].”
BackgroundBonell, C., Fletcher, A., Morton, M., Lorenc, T., & Moore, L. (2012) · Social Science & Medicine · doi:10.1016/j.socscimed.2012.08.032 · Cited by 582 · cited 1×
“Qualitative research carried out during a trial (eg, asking service users about their experiences) can aid in understanding why an intervention might work and how context affects implementation [21].”
BackgroundReed, M. W. R. (2007) · Clinical Oncology · doi:10.1016/j.clon.2007.06.006 · Cited by 571
Evaluating patient and stakeholder engagement in research: moving from theory to practice
Esmail, L., Moore, E., & Rein, A. (2015) · Journal of Comparative Effectiveness Research · doi:10.2217/cer.14.79 · Cited by 502 · cited 1×
“Despite the recommended involvement of patients and members of the public within implementation research [27] and widespread assumptions that consultation work can help researchers anticipate stakeholders’ needs, capacities, and priorities [28], the MRC guidelines on process evaluation [18] report substantial empirica…”
BackgroundFeedback from research participants: are member checks useful in qualitative research?
Thomas, D. R. (2017) · Qualitative Research in Psychology · doi:10.1080/14780887.2016.1219435 · Cited by 487
The nature of relapse in schizophrenia
Emsley, R., Chiliza, B., Asmal, L., & Harvey, B. H. (2013) · BMC Psychiatry · doi:10.1186/1471-244x-13-50 · Cited by 478 · cited 1×
“Research into reliable and valid signs of relapse is essential for early intervention aimed at minimizing the harms associated with relapse [7].”
BackgroundMember Checking With Older Women: A Framework for Negotiating Meaning
Doyle, S. (2007) · Health Care for Women International · doi:10.1080/07399330701615325 · Cited by 449 · cited 1×
“Resource limitations meant that strategies such as member checking (where participants check over themes proposed by the researcher as an interpretation validity check [40]) were not utilized.”
BackgroundMcGinn, C. A., Grenier, S., Duplantie, J., Shaw, N., Sicotte, C., Mathieu, L., Leduc, Y., Légaré, F., & Gagnon, M.-P. (2011) · BMC Medicine · doi:10.1186/1741-7015-9-46 · Cited by 393
Greenhalgh, T., Shaw, S., Wherton, J., Vijayaraghavan, S., Morris, J., Bhattacharya, S., Hanson, P., Campbell-Richards, D., Ramoutar, S., Collard, A., et al. (2018) · Journal of Medical Internet Research · doi:10.2196/jmir.9897 · Cited by 371 · cited 1×
“The context of health care settings is constructed as being institutionalized [44] because behaviors by social actors are described in terms of the roles people are expected to act out.”
BackgroundGreenhalgh, T., Shaw, S., Wherton, J., Vijayaraghavan, S., Morris, J., Bhattacharya, S., Hanson, P., Campbell-Richards, D., Ramoutar, S., Collard, A., et al. (2018) · doi:10.2196/preprints.9897 · Cited by 371
Berry, N., Lobban, F., Emsley, R., & Bucci, S. (2016) · Journal of Medical Internet Research · doi:10.2196/jmir.5250 · Cited by 230 · cited 1×
“Hypothetical acceptability is measured by key stakeholders’ willingness to engage with a proposed intervention and in previous trials of digital interventions for severe mental health problems actual acceptability (assessed postintervention) is typically higher than hypothetical acceptability [22].”
BackgroundComplex interventions and their implications for systematic reviews: a pragmatic approach
Petticrew, M., Anderson, L., Elder, R., Grimshaw, J., Hopkins, D., Hahn, R., Krause, L., Kristjansson, E., Mercer, S., Sipe, T., et al. (2013) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2013.06.004 · Cited by 206
Bell, I. H., Lim, M. H., Rossell, S. L., & Thomas, N. (2017) · Psychiatric Services · doi:10.1176/appi.ps.201600523 · Cited by 189 · cited 2×
“User experience has been described as a neglected area within digital intervention research [47] and psychosis more specifically [48].”
BackgroundRole of patient and public involvement in implementation research: a consensus study
Gray-Burrows, K. A., Willis, T. A., Foy, R., Rathfelder, M., Bland, P., Chin, A., Hodgson, S., Ibegbuna, G., Prestwich, G., Samuel, K., et al. (2018) · BMJ Quality & Safety · doi:10.1136/bmjqs-2017-006954 · Cited by 176
Berry, N., Lobban, F., & Bucci, S. (2019) · BMC Psychiatry · doi:10.1186/s12888-018-1979-1 · Cited by 170 · cited 3×
“Previous qualitative research conducted with service users exploring potential [29,41,45] and actual [46] acceptability of digital self-management interventions for psychosis has reported that having access to personal data may have positive impacts such as enhancing self-management.”
BackgroundSutcliffe, K., Thomas, J., Stokes, G., Hinds, K., & Bangpan, M. (2015) · Systematic Reviews · doi:10.1186/s13643-015-0126-z · Cited by 151 · cited 1×
“Typically, the majority of implementation research on engagement with interventions has been retrospective [20].”
BackgroundAref-Adib, G., McCloud, T., Ross, J., O'Hanlon, P., Appleton, V., Rowe, S., Murray, E., Johnson, S., & Lobban, F. (2019) · The Lancet Psychiatry · doi:10.1016/s2215-0366(18)30302-x · Cited by 118 · cited 2×
“Understanding the context behind empirical outcomes from novel digital mental health interventions is key in deciding if an intervention can be easily implemented within current practice [16] or will require significant resources and effort to do so [42].”
BackgroundBerry, N., Bucci, S., & Lobban, F. (2017) · JMIR Mental Health · doi:10.2196/mental.8311 · Cited by 112 · cited 1×
“Furthermore, staff foreseeing digital interventions having an impact on staff roles and responsibilities is similar to previous qualitative research work conducted with mental health staff [29,51].”
ResultThe Cost of Relapse in Schizophrenia
Pennington, M., & McCrone, P. (2017) · PharmacoEconomics · doi:10.1007/s40273-017-0515-3 · Cited by 103 · cited 1×
“In the United States, excess costs from relapse range from US $6033-$32,753 [4].”
BackgroundEarly intervention in psychotic relapse.
Birchwood, M., & Spencer, E. (2001) · Clinical Psychology Review · doi:10.1016/s0272-7358(01)00105-2 · Cited by 101 · cited 1×
“Delayed help-seeking narrows the window for timely intervention [10] and can result in the use of coercive treatment measures that confirm negative expectations [11] and make disclosure of EWS more threatening for service users.”
BackgroundGumley, A., Karatzias, A., Power, K., Reilly, J., McNay, L., & O'Grady, M. (2006) · British Journal of Clinical Psychology · doi:10.1348/014466505x49925 · Cited by 97 · cited 1×
“Delayed help-seeking narrows the window for timely intervention [10] and can result in the use of coercive treatment measures that confirm negative expectations [11] and make disclosure of EWS more threatening for service users.”
BackgroundSelf-citationCarper, M. M., McHugh, R. K., & Barlow, D. H. (2013) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-011-0377-5 · Cited by 92 · cited 1×
“These findings are in line with previous research demonstrating that different stakeholders can hold different perspectives on digital mental health interventions [29,52,53] and suggest value in seeking out all relevant stakeholder perspectives.”
ResultAssessing early signs of relapse in psychosis: Review and future directions
Eisner, E., Drake, R., & Barrowclough, C. (2013) · Clinical Psychology Review · doi:10.1016/j.cpr.2013.04.001 · Cited by 84 · cited 1×
“A review [8] to determine the validity of early signs as predictors of relapse in people with nonaffective psychosis found that the sensitivity (correct relapse prediction by staff) ranged from 10% to 80% (median 61%), and specificity (nonrelapses correctly identified) ranged from 38% to 100% (median 81%).”
BackgroundEarly Psychosis Service User Views on Digital Technology: Qualitative Analysis
Bucci, S., Morris, R., Berry, K., Berry, N., Haddock, G., Barrowclough, C., Lewis, S., & Edge, D. (2018) · JMIR Mental Health · doi:10.2196/10091 · Cited by 78 · cited 1×
“Previous qualitative research conducted with service users exploring potential [29,41,45] and actual [46] acceptability of digital self-management interventions for psychosis has reported that having access to personal data may have positive impacts such as enhancing self-management.”
BackgroundKumar, D., Tully, L. M., Iosif, A., Zakskorn, L. N., Nye, K. E., Zia, A., & Niendam, T. A. (2018) · JMIR Mental Health · doi:10.2196/mental.8551 · Cited by 76 · cited 1×
“A recent study examining a mobile health platform for clinical monitoring in psychosis indicates that implementation was low because of the app frequently crashing [49], perhaps highlighting the importance of exploring user experience in implementation research.”
BackgroundEisner, E., Drake, R. J., Berry, N., Barrowclough, C., Emsley, R., Machin, M., & Bucci, S. (2019) · JMIR mHealth and uHealth · doi:10.2196/11568 · Cited by 73 · cited 3×
“The field of digital self-monitoring interventions in psychosis is rapidly expanding [46,48], and there is a need to optimize interventions for implementation.”
BackgroundKooij, L., Groen, W. G., & van Harten, W. H. (2018) · Journal of Medical Internet Research · doi:10.2196/jmir.8989 · Cited by 71 · cited 1×
“However, the quantity of discussion of barriers and facilitators may not equal their importance or relevance [54].”
BackgroundHardy, A., Wojdecka, A., West, J., Matthews, E., Golby, C., Ward, T., Lopez, N. D., Freeman, D., Waller, H., Kuipers, E., et al. (2018) · JMIR Mental Health · doi:10.2196/11222 · Cited by 71
Garety, P. A., Ward, T., Freeman, D., Fowler, D., Emsley, R., Dunn, G., Kuipers, E., Bebbington, P., Waller, H., Greenwood, K., et al. (2017) · Trials · doi:10.1186/s13063-017-2242-7 · Cited by 66 · cited 1×
“Digital interventions can help address clinical priorities in psychosis, such as increasing access to psychological interventions for symptoms such as paranoia [14].”
BackgroundA Framework for the Study of Complex mHealth Interventions in Diverse Cultural Settings
Maar, M. A., Yeates, K., Perkins, N. J., Boesch, L., Hua‐Stewart, D., Liu, P. P., Sleeth, J., & Tobe, S. W. (2017) · JMIR mhealth and uhealth · doi:10.2196/mhealth.7044 · Cited by 60 · cited 1×
“This is commonly utilized within qualitative aspects of process evaluations to identify key barriers and facilitators for implementation of a diverse range of digital interventions [35-37].”
BackgroundDigital Platforms in the Assessment and Monitoring of Patients with Bipolar Disorder
Rajagopalan, A., Shah, P. R., Zhang, M. W., & Ho, R. (2017) · Brain Sciences · doi:10.3390/brainsci7110150 · Cited by 49 · cited 1×
“One emerging application of technology in mental health care is remote self-monitoring [13].”
BackgroundFear of recurrence: Results of a randomized trial of relapse detection in schizophrenia
Gumley, A. I., MacBeth, A., Reilly, J. D., O'Grady, M., White, R. G., McLeod, H., Schwannauer, M., & Power, K. G. (2015) · British Journal of Clinical Psychology · doi:10.1111/bjc.12060 · Cited by 46 · cited 1×
“Therefore, existing systems used to identify EWS have an uncertain prognostic utility and may result in an unnecessary intervention that engenders fear of relapse in service users and carers [9].”
MethodSelf-citationBiosafety in Handling Gene Transfer Vectors
Braun, A. (2006) · Current Protocols in Human Genetics · doi:10.1002/0471142905.hg1201s50 · Cited by 36
Complex interventions and their implications for systematic reviews: A pragmatic approach
Petticrew, M. P., Anderson, L. M., Elder, R. W., Grimshaw, J., Hopkins, D. P., Hahn, R. A., Krause, L. K., Kristjansson, E., Mercer, S. L., Sipe, T., et al. (2015) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2015.01.004 · Cited by 34 · cited 1×
“The effectiveness of interventions (including their success in reaching the target population) can be influenced by how an intervention interacts with the context in which it was implemented [18,19].”
BackgroundMorton, K., Dennison, L., Bradbury, K. J., Band, R. J., May, C. R., Raftery, J. P., Little, P. S., McManus, R. J., & Yardley, L. (2018) · BMJ Open · doi:10.1136/bmjopen-2017-020843 · Cited by 33 · cited 1×
“This is commonly utilized within qualitative aspects of process evaluations to identify key barriers and facilitators for implementation of a diverse range of digital interventions [35-37].”
Bridging the dichotomy of actual versus aspirational digital health
Torous, J., & Firth, J. (2018) · World Psychiatry · doi:10.1002/wps.20464 · Cited by 31 · cited 1×
“Understanding the context behind empirical outcomes from novel digital mental health interventions is key in deciding if an intervention can be easily implemented within current practice [16] or will require significant resources and effort to do so [42].”
BackgroundStrand, M., Gammon, D., Eng, L. S., & Ruland, C. M. (2017) · JMIR Mental Health · doi:10.2196/mental.8491 · Cited by 29
McClelland, G. T., & Fitzgerald, M. (2018) · Health Education Journal · doi:10.1177/0017896918773790 · Cited by 29 · cited 1×
“One critical implementation-focused strategy is intervention co-design with stakeholders to develop digital psychosis interventions more suitable to the needs of end users [57,58].”
BackgroundThomas, N., Bless, J. J., Alderson‐Day, B., Bell, I., Cella, M., Craig, T., Delespaul, P., Hugdahl, K., Laloyaux, J., Larøi, F., et al. (2018) · Schizophrenia Bulletin · doi:10.1093/schbul/sby103 · Cited by 29
Psychosis‐risk criteria in the general population: frequent misinterpretations and current evidence
Schultze‐Lutter, F., Klosterkötter, J., Gaebel, W., & Schmidt, S. J. (2018) · World Psychiatry · doi:10.1002/wps.20498 · Cited by 25
Huerta-Ramos, E., Escobar-Villegas, M. S., Rubinstein, K., Unoka, Z. S., Grasa, E., Hospedales, M., Jääskeläinen, E., Rubio-Abadal, E., Caspi, A., Bitter, I., et al. (2016) · JMIR mHealth and uHealth · doi:10.2196/mhealth.5716 · Cited by 20 · cited 6×
“Furthermore, staff foreseeing digital interventions having an impact on staff roles and responsibilities is similar to previous qualitative research work conducted with mental health staff [29,51].”
BackgroundResultInfluentialQualitative investigation of targets for and barriers to interventions to prevent psychosis relapse
Eisner, E., Barrowclough, C., Lobban, F. A., & Drake, R. (2014) · BMC Psychiatry · doi:10.1186/1471-244x-14-201 · Cited by 15 · cited 1×
“EWS-based prevention strategies assume that identifying relapse early enough enables preventative action and averts full relapse [5].”
BackgroundPerspectives of Family Members on Using Technology in Youth Mental Health Care: A Qualitative Study
Lal, S., Daniel, W., & Rivard, L. (2017) · JMIR Mental Health · doi:10.2196/mental.7296 · Cited by 14 · cited 1×
“Carers (similar to findings from previous qualitative work [50]) and staff generally reflected how they foresee EMPOWER influencing service user interactions with staff.”
Resultvan Reijen, M., Asscheman, M., Vriend, I. I., van Mechelen, W., & Verhagen, E. A. L. M. (2018) · JMIR Rehabilitation and Assistive Technologies · doi:10.2196/rehab.8638 · Cited by 6 · cited 1×
“This is commonly utilized within qualitative aspects of process evaluations to identify key barriers and facilitators for implementation of a diverse range of digital interventions [35-37].”
Background
Research works printed but not linked: 1
- Hardy, A, 2018, JMIR Ment Health, -
Cited by
Borghouts, J., Eikey, E., Mark, G., De Leon, C., Schueller, S. M., Schneider, M., Stadnick, N., Zheng, K., Mukamel, D., & Sorkin, D. H. (2021) · Journal of Medical Internet Research · doi:10.2196/24387 · Cited by 1,069
“This perception was facilitated by whether users were able to understand the data presented to them [104,117,170], whether it was clear what action they should take [129,133,154,155,166], and whether the intervention provided a clear advantage over past or current care received [103,117,121,155,171].”
BackgroundInfluentialZidaru, T., Morrow, E. M., & Stockley, R. (2021) · Health Expectations · doi:10.1111/hex.13299 · Cited by 169
Gumley, A. I., Bradstreet, S., Ainsworth, J., Allan, S., Alvarez-Jimenez, M., Aucott, L., Birchwood, M., Briggs, A., Bucci, S., Cotton, S. M., et al. (2022) · The Lancet Psychiatry · doi:10.1016/s2215-0366(22)00103-1 · Cited by 71
“27,28 We identified important safety signals related to the EMPOWER intervention.”
BackgroundSelf-citationEisner, E., Faulkner, S., Allan, S., Ball, H., Basilio, D. D., Nicholas, J., Priyam, A., Wilson, P. M., Zhang, X., & Bucci, S. (2025) · JMIR Mental Health · doi:10.2196/65246 · Cited by 77
“Others described intrinsic motivations: commitment to participating [44], wanting to provide accurate data [31], agreement with DMHI tasks or goals [83], less “controlled motivations for treatment” [133], and interest in technology [83].”
InfluentialSelf-citationUnderstanding Mental Health App Use Among Community College Students: Web-Based Survey Study
Borghouts, J. W., Eikey, E. V., Mark, G. J., Leon, C. D., Schueller, S. M., Schneider, M. L., Stadnick, N. A., Zheng, K., Mukamel, D. B., & Sorkin, D. H. (2021) · Journal of Medical Internet Research · doi:10.2196/27745 · Cited by 68
“However, it will likely matter how they experienced using mental health services, as people with a positive experience are generally more open to trying new mental health technologies [26-28], but people with a negative experience are less interested in mental health technology [49-51].”
Metadata
- Authors
- Allan, S., Bradstreet, S., Mcleod, H., Farhall, J., Lambrou, M., Gleeson, J., Clark, A., EMPOWER Group, & Gumley, A.
- Affiliations
- University of Glasgow (GB)
- La Trobe University (AU)
- Australian Catholic University (AU)
- Corresponding author
- EMPOWER Group
- ORCID
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2019-10-24
- Volume
- 21
- Issue
- 10
- Pages
- e14366
- ISSN
- 1438-8871 (ISSN-L)
- PMC
- PMC6838692
- OpenAlex
- W2981804184
- Cited by
- 44
- References
- 61
- Citations by year
- 2026: 2 · 2025: 14 · 2024: 2 · 2023: 5 · 2022: 7 · 2021: 9 · 2020: 4 · 2019: 1
- FWCI
- 7.15
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Journal of Medical Internet Research (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- ENLIGHTEN (Jurnal Bimbingan dan Konseling Islam) (acceptedVersion, cc-by-sa)
- Monash University Research Portal (Monash University) (submittedVersion, cc-by)
- Funders
- Northwestern University
- Manchester Metropolitan University
- National Institute for Health and Care Research (13/154/04, HTA/13/154/04)
- University of Warwick
- University of Glasgow
- University of Aberdeen
- Deakin University
- Monash University
- Medical Research Council
- Scottish Mental Health Research Network
- Topics
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Health Sciences › Health Professions › General Health Professions › Mental Health and Patient Involvement
- Keywords
- psychosis, relapse prevention, mobile application, digital interventions, implementation framework, Normalization Process Theory, stakeholder perspectives, self-monitoring, self-management, caregivers, mental health professionals, service users
- MeSH terms
- Self-Management, Humans, Psychotic Disorders, Recurrence, Retrospective Studies, Qualitative Research, Mobile Applications
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
