The DOI matches its Crossref record; 62 of the 62 references it cites were found in a registry or an index.
Developing a Hypothetical Implementation Framework of Expectations for Monitoring Early Signs of Psychosis Relapse Using a Mobile App: Qualitative Study
Open accessPublished version of a preprint
Abstract
Background: Relapse is a common experience for people diagnosed with psychosis, which is associated with increased service costs and profound personal and familial distress. EMPOWER (Early signs Monitoring to Prevent relapse in psychosis and prOmote Well-being, Engagement, and Recovery) is a peer worker–supported digital intervention that aims to enable service users to self-monitor their mental health with the aim of encouraging self-management and the shared use of personal data to promote relapse prevention. Digital interventions have not been widely used in relapse prevention and, therefore, little is currently known about their likely implementation—both within trials and beyond. Objective: Seeking the perspectives of all relevant stakeholder groups is recommended in developing theories about implementation because this can reveal important group differences in understandings and assumptions about whether and for whom the intervention is expected to work. However, the majority of intervention implementation research has been retrospective. This study aimed to discover and theoretically frame implementation expectations in advance of testing and synthesize these data into a framework. Methods: To develop a hypothetical implementation framework, 149 mental health professionals, carers, and people diagnosed with psychosis participated in 25 focus groups in both Australia and the United Kingdom. An interview schedule informed by the normalization process theory was used to explore stakeholders’ expectations about the implementation of the EMPOWER intervention. Data were analyzed using thematic analysis and then theoretically framed using the Medical Research Council guidelines for understanding the implementation of complex interventions. Results: All groups expected that EMPOWER could be successfully implemented if the intervention generated data that were meaningful to mental health staff, carers, and service users within their unique roles. However, there were key differences between staff, carers, and service users about what facilitators and barriers that stakeholders believe exist for intervention implementation in both the cluster randomized controlled trial stage and beyond. For example, service user expectations mostly clustered around subjective user experiences, whereas staff and carers spoke more about the impact upon staff interactions with service users. Conclusions: A hypothetical implementation framework synthesized from stakeholder implementation expectations provides an opportunity to compare actual implementation data gathered during an ongoing clinical trial, giving valuable insights into the accuracy of these stakeholders’ previous expectations. This is among the first studies to assess and record implementation expectations for a newly developed digital intervention for psychosis in advance of testing in a clinical trial. Trial Registration: ISRCTN Registry ISRCTN99559262; http://www.isrctn.com/ISRCTN99559262
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“A negative prior experience formed a barrier to engaging with a DMHI [74,117,122], whereas a positive experience increased participants’ engagement [54,72,104,115,116,118-121].”
From Barriers to and facilitators of user engagement with digital mental health interventions: Systematic review (2021) · cites this paper as [117]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“A positive prior experience with technology [115,116], mental health services [54,74,104,117-119], and mental health technology [72,120,121] facilitated people’s intention to use interventions, as well as actual engagement.”
From Barriers to and facilitators of user engagement with digital mental health interventions: Systematic review (2021) · cites this paper as [117]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“and whether the intervention provided a clear advantage over past or current care received [103,117,121,155,171].”
From Barriers to and facilitators of user engagement with digital mental health interventions: Systematic review (2021) · cites this paper as [117]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“a feeling of empowerment over being in control of their health [14,32,84,93,95,104,117,122,124,170,174,175,196,206,207]”
From Barriers to and facilitators of user engagement with digital mental health interventions: Systematic review (2021) · cites this paper as [117]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“This perception was facilitated by whether users were able to understand the data presented to them [104,117,170]”
From Barriers to and facilitators of user engagement with digital mental health interventions: Systematic review (2021) · cites this paper as [117]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414 · cited 2×
“Thematic analysis is a qualitative method used to construct, analyze, and report on patterns within text data [34].”
MethodTong, A., Sainsbury, P., & Craig, J. (2007) · International Journal for Quality in Health Care · doi:10.1093/intqhc/mzm042 · Cited by 42,509 · cited 1×
“The methods are reported in line with the consolidated criteria for reporting qualitative research reporting recommendations for qualitative work [30]; a full checklist can be seen in Multimedia Appendix 1.”
Book Review: Focus Groups: A Practical Guide for Applied Research
McQuarrie, E. F., & Krueger, R. A. (1989) · Journal of Marketing Research · doi:10.2307/3172912 · Cited by 15,462
Gale, N. K., Heath, G., Cameron, E., Rashid, S., & Redwood, S. (2013) · BMC Medical Research Methodology · doi:10.1186/1471-2288-13-117 · Cited by 10,865 · cited 1×
“For stage 2, the MRC process evaluation framework [18] was identified as a suitable deductive coding framework [38] for placing the themes in an implementation theory context more relevant to the needs for a feasibility study where it may be too early to decide if normalization should be the goal.”
MethodProcess evaluation of complex interventions: Medical Research Council guidance
Moore, G. F., Audrey, S., Barker, M., Bond, L., Bonell, C., Hardeman, W., Moore, L., O'Cathain, A., Tinati, T., Wight, D., et al. (2015) · BMJ · doi:10.1136/bmj.h1258 · Cited by 6,204 · cited 9×
“The MRC framework for process evaluations recommends that implementation research should proactively include key stakeholders because those expected to engage with an intervention are likely to have relevant experiential knowledge, which is useful in understanding the implementation process during a trial [18].”
BackgroundMethodInfluentialMaking sense of implementation theories, models and frameworks
Nilsen, P. (2015) · Implementation Science · doi:10.1186/s13012-015-0242-0 · Cited by 4,447 · cited 1×
“Theory in implementation science implies some predictive capacity [23].”
BackgroundGreenhalgh, T., Wherton, J., Papoutsi, C., Lynch, J., Hughes, G., A'Court, C., Hinder, S., Fahy, N., Procter, R., & Shaw, S. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.8775 · Cited by 2,887 · cited 3×
“However, many effective digital interventions have failed to generalize from clinical trials into clinical practice [15,16].”
BackgroundMethodMurray, E., Treweek, S., Pope, C., MacFarlane, A., Ballini, L., Dowrick, C., Finch, T., Kennedy, A., Mair, F., O'Donnell, C., et al. (2010) · BMC Medicine · doi:10.1186/1741-7015-8-63 · Cited by 1,478
Development of a theory of implementation and integration: Normalization Process Theory
May, C. R., Mair, F., Finch, T., MacFarlane, A., Dowrick, C., Treweek, S., Rapley, T., Ballini, L., Ong, B. N., Rogers, A., et al. (2009) · Implementation Science · doi:10.1186/1748-5908-4-29 · Cited by 1,409 · cited 3×
“One such implementation theory, normalization process theory (NPT) [25] focuses on the work that groups and individuals do when interacting with an intervention and how they make sense of it, many intervention studies have successfully utilized NPT as a framework to guide research to more fully understand the implemen…”
MethodRobinson, D., Woerner, M. G., Alvir, J. M. J., Bilder, R., Goldman, R., Geisler, S., Koreen, A., Sheitman, B., Chakos, M., Mayerhoff, D., et al. (1999) · Archives of General Psychiatry · doi:10.1001/archpsyc.56.3.241 · Cited by 1,327 · cited 1×
“Relapse is common for many people diagnosed with schizophrenia [1].”
Background
Research works printed but not linked: 1
- Hardy, A, 2018, JMIR Ment Health, -
Cited by
Borghouts, J., Eikey, E., Mark, G., De Leon, C., Schueller, S. M., Schneider, M., Stadnick, N., Zheng, K., Mukamel, D., & Sorkin, D. H. (2021) · Journal of Medical Internet Research · doi:10.2196/24387 · Cited by 1,069
“This perception was facilitated by whether users were able to understand the data presented to them [104,117,170], whether it was clear what action they should take [129,133,154,155,166], and whether the intervention provided a clear advantage over past or current care received [103,117,121,155,171].”
BackgroundInfluentialZidaru, T., Morrow, E. M., & Stockley, R. (2021) · Health Expectations · doi:10.1111/hex.13299 · Cited by 169
Gumley, A. I., Bradstreet, S., Ainsworth, J. D., Allan, S., Álvarez‐Jiménez, M., Aucott, L. S., Birchwood, M. J., Briggs, A. H., Bucci, S., Cotton, S., et al. (2022) · The Lancet Psychiatry · doi:10.1016/s2215-0366(22)00103-1 · Cited by 85
“27,28 We identified important safety signals related to the EMPOWER intervention.”
BackgroundSelf-citationEisner, E., Faulkner, S., Allan, S., Ball, H., Basilio, D. D., Nicholas, J., Priyam, A., Wilson, P. M., Zhang, X., & Bucci, S. (2025) · JMIR Mental Health · doi:10.2196/65246 · Cited by 76
“Others described intrinsic motivations: commitment to participating [44], wanting to provide accurate data [31], agreement with DMHI tasks or goals [83], less “controlled motivations for treatment” [133], and interest in technology [83].”
InfluentialSelf-citationUnderstanding Mental Health App Use Among Community College Students: Web-Based Survey Study
Borghouts, J. W., Eikey, E. V., Mark, G. J., Leon, C. D., Schueller, S. M., Schneider, M. L., Stadnick, N. A., Zheng, K., Mukamel, D. B., & Sorkin, D. H. (2021) · Journal of Medical Internet Research · doi:10.2196/27745 · Cited by 68
“However, it will likely matter how they experienced using mental health services, as people with a positive experience are generally more open to trying new mental health technologies [26-28], but people with a negative experience are less interested in mental health technology [49-51].”
Metadata
- Authors
- Allan, S., Bradstreet, S., Mcleod, H., Farhall, J., Lambrou, M., Gleeson, J., Clark, A., EMPOWER Group, & Gumley, A.
- Affiliations
- University of Glasgow (GB)
- La Trobe University (AU)
- Australian Catholic University (AU)
- Corresponding author
- EMPOWER Group
- ORCID
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2019-10-24
- Volume
- 21
- Issue
- 10
- Pages
- e14366
- ISSN
- 1438-8871 (ISSN-L)
- PMC
- PMC6838692
- OpenAlex
- W2981804184
- Cited by
- 44
- References
- 61
- Citations by year
- 2026: 2 · 2025: 14 · 2024: 2 · 2023: 5 · 2022: 7 · 2021: 9 · 2020: 4 · 2019: 1
- FWCI
- 7.15
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Journal of Medical Internet Research (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- ENLIGHTEN (Jurnal Bimbingan dan Konseling Islam) (acceptedVersion, cc-by-sa)
- Monash University Research Portal (Monash University) (submittedVersion, cc-by)
- Funders
- Northwestern University
- Manchester Metropolitan University
- National Institute for Health and Care Research (13/154/04, HTA/13/154/04)
- University of Warwick
- University of Glasgow
- University of Aberdeen
- Deakin University
- Monash University
- Medical Research Council
- Scottish Mental Health Research Network
- Topics
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Health Sciences › Health Professions › General Health Professions › Mental Health and Patient Involvement
- Keywords
- psychosis, relapse prevention, mobile application, digital interventions, implementation framework, Normalization Process Theory, stakeholder perspectives, self-monitoring, self-management, caregivers, mental health professionals, service users
- MeSH terms
- Self-Management, Humans, Psychotic Disorders, Recurrence, Retrospective Studies, Qualitative Research, Mobile Applications
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Developing a Hypothetical Implementation Framework of Expectations for Monitoring Early Signs of Psychosis Relapse Using a Mobile App: Qualitative Study. https://app.cochat.ai/papers/10.2196/14366
- The paper
- Allan, S., Bradstreet, S., Mcleod, H., Farhall, J., Lambrou, M., Gleeson, J., Clark, A., EMPOWER Group, & Gumley, A. (2019). Developing a Hypothetical Implementation Framework of Expectations for Monitoring Early Signs of Psychosis Relapse Using a Mobile App: Qualitative Study. Journal of Medical Internet Research. https://doi.org/10.2196/14366
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
