Implementation of eMental Health care: viewpoints from key informants from organizations and agencies with eHealth mandates
Open accessAbstract
BACKGROUND: The use of technology such as computers, tablets, and smartphones to improve access to and the delivery of mental health care (eMental Health care) is growing worldwide. However, despite the rapidly expanding evidence base demonstrating the efficacy of eMental Health care, its implementation in clinical practice and health care systems remains fragmented. To date, no peer-reviewed, key-informant studies have reported on the perspectives of decision-makers concerned with whether and how to implement eMental Health care. METHODS: From September to November 2015, we conducted 31 interviews with key informants responsible for leadership, policy, research, and/ or information technology in organizations influential in the adoption of technology for eMental Health care. Deductive and inductive thematic analyses of transcripts were conducted using the Behavior Change Wheel as an organizing framework. Frequency and intensity effect sizes were calculated for emerging themes to further explore patterns within the data. RESULTS: Key informant responses (n = 31) representing 6 developed countries and multiple organizations showed consensus on common factors impacting implementation: individual and organizational capacities (e.g., computer literacy skills [patients and providers], knowledge gaps about cyber security, limited knowledge of available services); motivational drivers of technology-based care (e.g., extending care, data analytics); and opportunities for health systems to advance eMental Health care implementation (e.g., intersectoral research, rapid testing cycles, sustainable funding). Frequency effect sizes showed strong associations between implementation and credibility, knowledge, workflow, patient empowerment, electronic medical record (EMR) integration, sustained funding and intersectoral networks. Intensity effect sizes showed the highest concentration of statements (>10% of all comments) related to funding, credibility, knowledge gaps, and patient empowerment. CONCLUSION: This study provides previously unavailable information about key informant perspectives on eMental Health care implementation. The themes that emerged, namely the need to intensify intersectoral research, measure/monitor readiness to implement, define cost-utility benchmarks, raise awareness about available technologies, and test assumptions that 'proven' technologies will be easily integrated can inform the design and evaluation of eMental Health care implementation models.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Participants' mental health literacy, digital literacy, and digital health literacy influenced the extent to which they were able to adapt and engage with DMHIs: higher literacy was associated with higher engagement.
“Participants’ mental health literacy [123], digital literacy [88,103,104,122,124-127], and digital health literacy [103,127] influenced the extent to which they were able to adapt and engage with DMHIs: for each type of literacy; higher literacy was associated with higher engagement.”
From Barriers to and facilitators of user engagement with digital mental health interventions: Systematic review (2021) · cites this paper as [126]
The Results section, “Developing capacity for eMental Health knowledge mobilization” (page unavailable), identifies computer literacy among patients and providers as a barrier to implementation. The Discussion section, “Assess eHealth literacy skills of all stakeholder groups” (page unavailable), recommends promoting patients’ eHealth literacy. The paper does not establish that participants with higher digital or digital health literacy engaged more with DMHIs; the mental-health-literacy portion is cited to a different source. (LLM, conf=0.84)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Basics of Qualitative Research: Grounded Theory Procedures and Techniques.
Ellis, C., Strauss, A., & Corbin, J. M. (1992) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2074814 · Cited by 29,279 · see citing works
How Many Interviews Are Enough?: An Experiment with Data Saturation and Variability
Guest, G., Bunce, A., & Johnson, L. (2006) · Field Methods · doi:10.1177/1525822x05279903 · Cited by 20,808 · see citing works
Michie, S., van Stralen, M. M., & West, R. (2011) · Implementation Science · doi:10.1186/1748-5908-6-42 · Cited by 13,697 · see citing works
The SAGE handbook of qualitative research
(2005) · Choice Reviews Online · doi:10.5860/choice.43-1330 · Cited by 10,173 · see citing works
The Sage Handbook of Qualitative Research[1]
(2006) · Qualitative Research in Organizations and Management An International Journal · doi:10.1108/17465640610666642 · Cited by 5,854 · see citing works
(1990) · Cited by 5,210 · see citing works
Qualitative Data Analysis for Health Services Research: Developing Taxonomy, Themes, and Theory
Bradley, E. H., Curry, L. A., & Devers, K. J. (2007) · Health Services Research · doi:10.1111/j.1475-6773.2006.00684.x · Cited by 3,437 · see citing works
Understanding and Validity in Qualitative Research
Maxwell, J. (1992) · Harvard Educational Review · doi:10.17763/haer.62.3.8323320856251826 · Cited by 2,951 · cited 1× · see citing works
“pertaining to one theme together in this way preserves the complexity of the findings and optimizes the descriptive validity of the thematic analysis [43].”
BackgroundThe global prevalence of common mental disorders: a systematic review and meta-analysis 1980–2013
Steel, Z., Marnane, C., Iranpour, C., Chey, T., Jackson, J. W., Patel, V., & Silove, D. (2014) · International Journal of Epidemiology · doi:10.1093/ije/dyu038 · Cited by 2,846 · see citing works
(2007) · PubMed Central · Cited by 2,521 · see citing works
Eysenbach, G. (2001) · Journal of Medical Internet Research · doi:10.2196/jmir.3.2.e20 · Cited by 2,449 · see citing works
Andrews, G., Cuijpers, P., Craske, M. G., McEvoy, P., & Titov, N. (2010) · PLoS ONE · doi:10.1371/journal.pone.0013196 · Cited by 1,311 · see citing works
Weiner, B. J., Amick, H., & Lee, S.-Y. D. (2008) · Medical Care Research and Review · doi:10.1177/1077558708317802 · Cited by 786 · see citing works
Mair, F. S., May, C., O’Donnell, C., Finch, T., Sullivan, F., & Murray, E. (2012) · Bulletin of the World Health Organization · doi:10.2471/blt.11.099424 · Cited by 553 · cited 1× · see citing works
“An understanding of professional behaviour, local context, personal and organizational development, change management and diffusion of innovation all impact health systems’ capacity to implement eMental Health care technologies [28, 29].”
BackgroundUsing qualitative metasummary to synthesize qualitative and quantitative descriptive findings
Sandelowski, M., Barroso, J. V., & Voils, C. I. (2007) · Research in Nursing & Health · doi:10.1002/nur.20176 · Cited by 541 · see citing works
Phonological rehabilitation in acquired aphasia
Luzzatti, C., Molinari, A., Zanobio, M., & Rizzi, G. (2015) · Frontiers in Psychology · doi:10.3389/conf.fpsyg.2015.65.00036 · Cited by 433 · see citing works
How many qualitative interviews is enough
Baker, S., & Edwards, R. (2012) · Cited by 409 · see citing works
Technological innovations in mental healthcare: harnessing the digital revolution
Hollis, C., Morriss, R., Martin, J., Amani, S., Cotton, R., Denis, M., & Lewis, S. (2015) · British Journal of Psychiatry · doi:10.1192/bjp.bp.113.142612 · Cited by 375 · see citing works
Current and Future Trends in Internet-Supported Mental Health Interventions
Barak, A., & Grohol, J. M. (2011) · Journal of Technology in Human Services · doi:10.1080/15228835.2011.616939 · Cited by 373 · cited 1× · see citing works
“conducted and published regularly to keep up with the ever-changing landscape [34].”
BackgroundLluch, M. (2011) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2011.09.005 · Cited by 358 · see citing works
Modeling Patients' Acceptance of Provider-delivered E-health
Wilson, E. V. (2004) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1475 · Cited by 352 · see citing works
Arnberg, F. K., Linton, S. J., Hultcrantz, M., Heintz, E., & Jonsson, U. (2014) · PLoS ONE · doi:10.1371/journal.pone.0098118 · Cited by 330 · see citing works
A Systematic Review of Cognitive Behavioral Therapy and Behavioral Activation Apps for Depression
Huguet, A., Rao, S., McGrath, P. J., Wozney, L., Wheaton, M., Conrod, J., & Rozario, S. (2016) · PLOS ONE · doi:10.1371/journal.pone.0154248 · Cited by 328 · see citing works
Self-citationEmerging mHealth and eHealth interventions for serious mental illness: a review of the literature
Naslund, J. A., Marsch, L. A., McHugo, G. J., & Bartels, S. J. (2015) · Journal of Mental Health · doi:10.3109/09638237.2015.1019054 · Cited by 327 · see citing works
American Psychological Association ( APA )
Hsu, K. J., & Davison, G. C. (2015) · The Encyclopedia of Clinical Psychology · doi:10.1002/9781118625392.wbecp359 · Cited by 264 · see citing works
Effect Sizes in Qualitative Research: A Prolegomenon
Onwuegbuzie, A. J. (2003) · Quality & Quantity · doi:10.1023/a:1027379223537 · Cited by 255 · see citing works
Health Care Provider Adoption of eHealth: Systematic Literature Review
Li, J., Talaei-Khoei, A., Seale, H., Ray, P., & MacIntyre, C. R. (2013) · interactive Journal of Medical Research · doi:10.2196/ijmr.2468 · Cited by 250 · see citing works
Lavis, J. N., Boyko, J. A., Oxman, A. D., Lewin, S. A., & Fretheim, A. (2009) · Health Research Policy and Systems · doi:10.1186/1478-4505-7-s1-s14 · Cited by 194 · see citing works
Lewis, C., Pearce, J., & Bisson, J. I. (2012) · British Journal of Psychiatry · doi:10.1192/bjp.bp.110.084756 · Cited by 189 · see citing works
Issues in mHealth: Findings From Key Informant Interviews
Whittaker, R. (2012) · Journal of Medical Internet Research · doi:10.2196/jmir.1989 · Cited by 173 · see citing works
Online interventions for depression and anxiety – a systematic review
Saddichha, S., Al-Desouki, M., Lamia, A., Linden, I. A., & Krausz, M. (2014) · Health Psychology and Behavioral Medicine · doi:10.1080/21642850.2014.945934 · Cited by 133 · cited 1× · see citing works
“Though not universal, both short- and long-term positive outcomes have been reported for a diverse range of eMental Health technologies [10, 11].”
BackgroundFischer, P., Schulz‐Hardt, S., & Frey, D. (2008) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.94.2.94.2.231 · Cited by 129 · see citing works
Online Mental Health Treatment: Concerns and Considerations
Wells, M., Mitchell, K. J., David Finkelhor, B., & Becker‐Blease, K. A. (2007) · CyberPsychology & Behavior · doi:10.1089/cpb.2006.9933 · Cited by 126 · see citing works
Carr, S. M., Lhussier, M., Forster, N., Geddes, L., Deane, K. H., Pennington, M., Visram, S., White, M. J., Michie, S., Donaldson, C., et al. (2011) · Health Technology Assessment · doi:10.3310/hta15090 · Cited by 114 · see citing works
Online Communication Between Doctors and Patients in Europe: Status and Perspectives
Santana, S., Lausen, B., Bujnowska-Fedak, M., Chronaki, C., Kummervold, P. E., Rasmussen, J., & Sorensen, T. (2010) · Journal of Medical Internet Research · doi:10.2196/jmir.1281 · Cited by 110 · see citing works
The generalizability of salience effects.
Taylor, S. E., Crocker, J., Fiske, S. T., Sprinzen, M., & Winkler, J. D. (1979) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.37.3.357 · Cited by 108 · see citing works
Meurk, C., Leung, J., Hall, W., Head, B. W., & Whiteford, H. (2016) · Journal of Medical Internet Research · doi:10.2196/jmir.4827 · Cited by 105 · see citing works
Whitfield, G., & Williams, C. (2004) · Behavioural and Cognitive Psychotherapy · doi:10.1017/s1352465804001031 · Cited by 83 · see citing works
Developing a roadmap for the translation of e-mental health services for depression
Batterham, P. J., Sunderland, M., Calear, A. L., Davey, C. G., Christensen, H., Teesson, M., Kay-Lambkin, F., Andrews, G., Mitchell, P. B., Herrman, H., et al. (2015) · Australian & New Zealand Journal of Psychiatry · doi:10.1177/0004867415582054 · Cited by 77 · see citing works
Boulding, T., Wu, F., McCuaig, R., Dunn, J., Sutton, C. R., Hardy, K., Tu, W., Bullman, A., Yip, D., Dahlstrom, J. E., et al. (2016) · PLoS ONE · doi:10.1371/journal.pone.0148065 · Cited by 73 · see citing works
What Are Young Adults Saying About Mental Health? An Analysis of Internet Blogs
Marcus, M., Westra, H. A., Eastwood, J. D., & Barnes, K. L. (2012) · Journal of Medical Internet Research · doi:10.2196/jmir.1868 · Cited by 63 · see citing works
Gäebel, W., Großimlinghaus, I., Kerst, A., Cohen, Y., Hinsche-Böckenholt, A., Johnson, B., Mucić, D., Petrea, I., Wulf Rössler, A., Thornicroft, G., et al. (2016) · European Archives of Psychiatry and Clinical Neuroscience · doi:10.1007/s00406-016-0677-6 · Cited by 47 · see citing works
Stephens-Reicher, J., Metcalf, A., Blanchard, M., Mangan, C., & Burns, J. (2011) · Australasian Psychiatry · doi:10.3109/10398562.2011.583077 · Cited by 46 · see citing works
Evaluating eHealth: Undertaking Robust International Cross-Cultural eHealth Research
Bates, D. W., & Wright, A. T. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000105 · Cited by 45 · see citing works
Castelnuovo, G., Zoppis, I. F., Santoro, E., Ceccarini, M., Pietrabissa, G., Manzoni, G. M., Corti, S. P., Borrello, M., Giusti, E. M., Cattivelli, R., et al. (2015) · Frontiers in Psychology · doi:10.3389/fpsyg.2015.00407 · Cited by 44 · see citing works
Wild West eHealth: Time to hold our horses?
Ruwaard, J., & Kok, R. N. (2015) · Data Archiving and Networked Services (DANS) · Cited by 28 · see citing works
Struthers, A., Charette, C., Bapuji, S. B., Winters, S., Ye, X., Metge, C., Kreindler, S., Raynard, M., Lemaire, J., Synyshyn, M., et al. (2015) · Canadian Journal of Community Mental Health · doi:10.7870/cjcmh-2015-006 · Cited by 27 · see citing works
Wao, H., Dedrick, R. F., & Ferron, J. M. (2010) · Quality & Quantity · doi:10.1007/s11135-010-9404-y · Cited by 25 · see citing works
Jones, R. B., & Ashurst, E. J. (2013) · Health Informatics Journal · doi:10.1177/1460458212474908 · Cited by 15 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- [48]Robert G Greenhalgh T MacFarlane F Peacock R Organisational factors influencing technology adoption and assimilation in the NHS: a systematic literature review 2009 London National Institute for Health Research Service Delivery and Organization Programme
Cited by
Sorted by citations
Borghouts, J., Eikey, E., Mark, G., De Leon, C., Schueller, S. M., Schneider, M., Stadnick, N., Zheng, K., Mukamel, D., & Sorkin, D. H. (2021) · Journal of Medical Internet Research · doi:10.2196/24387 · Cited by 1,069 · see citing works
“As seen in Multimedia Appendix 2, the 208 articles included in this review [27-237] reported on 2 types of user studies: (1) 69 studies were needs assessments that aimed to understand user needs and attitudes toward DMHIs without or before engaging with a specific intervention as part of a study and (2) 135 studies we…”
BackgroundVis, C., Mol, M., Kleiboer, A., Bührmann, L., Finch, T., Smit, J., & Riper, H. (2018) · JMIR Mental Health · doi:10.2196/mental.9769 · Cited by 183 · see citing works
Recent developments in the use of smartphone interventions for mental health
Mehrotra, S., & Tripathi, R. (2018) · Current Opinion in Psychiatry · doi:10.1097/yco.0000000000000439 · Cited by 45 · see citing works
Hill, C., Creswell, C., Vigerland, S., Nauta, M. H., March, S., Donovan, C., Wolters, L., Spence, S. H., Martin, J. L., Wozney, L., et al. (2018) · Internet Interventions · doi:10.1016/j.invent.2018.02.002 · Cited by 44 · see citing works
Self-citationApolinário-Hagen, J., Hennemann, S., Kück, C., Wodner, A., Geibel, D., Riebschläger, M., Zeißler, M., & Breil, B. (2020) · Health Services Insights · doi:10.1177/1178632920911061 · Cited by 26 · see citing works
Neher, M., Nygårdh, A., Broström, A., Lundgren, J., & Johansson, P. (2022) · Journal of Medical Internet Research · doi:10.2196/28870 · Cited by 21 · see citing works
“Wozney et al [31] interviewed key informants who were influential in the adoption of technology for e-Mental health and found similar knowledge gaps.”
Kuso, S., Nitsch, M., Zeiler, M., Simek, M., Adamcik, T., Dey, M., Berger, T., Krieger, T., Weisel, K. K., Zarski, A.-C., et al. (2021) · European Journal of Public Health · doi:10.1093/eurpub/ckab043 · Cited by 16 · see citing works
Abi Ramia, J., Abi Hana, R., Noun, P., Cuijpers, P., Carswell, K., van't Hof, E., Heim, E., Zoghbi, E., Sijbrandij, M., & El Chammay, R. (2024) · Frontiers in Public Health · doi:10.3389/fpubh.2023.1293187 · Cited by 15 · see citing works
- Authors
- Wozney, L., Newton, A. S., Gehring, N. D., Bennett, K., Huguet, A., Hartling, L., Dyson, M. P., & McGrath, P.
- Affiliations
- Izaak Walton Killam Health Centre (CA)
- University of Alberta (CA)
- McMaster University (CA)
- Dalhousie University (CA)
- Corresponding author
- Newton, Amanda S.
- ORCID
- Venue
- BMC Medical Informatics and Decision Making
- Published
- 2017-06-02
- Volume
- 17
- Issue
- 1
- Pages
- 78-78
- ISSN
- 1472-6947 (ISSN-L)
- PMC
- PMC5455087
- OpenAlex
- W2620955834
- Cited by
- 58
- References
- 49
- Citations by year
- 2025: 6 · 2024: 8 · 2023: 2 · 2022: 11 · 2021: 12 · 2020: 10 · 2019: 4 · 2018: 5
- FWCI
- 5.95
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- BMC Medical Informatics and Decision Making (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, other-oa)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Canadian Institutes of Health Research (201404KRS)
- Topics
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Telemedicine and Telehealth Implementation
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Keywords
- implementation barriers, implementation facilitators, key informants, decision-makers, Behaviour Change Wheel, computer literacy, patient empowerment, electronic health record integration, intersectoral collaboration, funding, implementation readiness, health policy
- MeSH terms
- Health Services Research, Humans, Mental Health Services, Organizational Innovation, Telemedicine, Qualitative Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
