Use of Human-Centered Design to Improve Implementation of Evidence-Based Psychotherapies in Low-Resource Communities: Protocol for Studies Applying a Framework to Assess Usability
Open accessThe DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index.
Abstract
Background: This paper presents the protocol for the National Institute of Mental Health (NIMH)–funded University of Washington’s ALACRITY (Advanced Laboratories for Accelerating the Reach and Impact of Treatments for Youth and Adults with Mental Illness) Center (UWAC), which uses human-centered design (HCD) methods to improve the implementation of evidence-based psychosocial interventions (EBPIs). We propose that usability—the degree to which interventions and implementation strategies can be used with ease, efficiency, effectiveness, and satisfaction—is a fundamental, yet poorly understood determinant of implementation. Objective: We present a novel Discover, Design/Build, and Test (DDBT) framework to study usability as an implementation determinant. DDBT will be applied across Center projects to develop scalable and efficient implementation strategies (eg, training tools), modify existing EBPIs to enhance usability, and create usable and nonburdensome decision support tools for quality delivery of EBPIs. Methods: Stakeholder participants will be implementation practitioners/intermediaries, mental health clinicians, and patients with mental illness in nonspecialty mental health settings in underresourced communities. Three preplanned projects and 12 pilot studies will employ the DDBT model to (1) identify usability challenges in implementing EBPIs in underresourced settings; (2) iteratively design solutions to overcome these challenges; and (3) compare the solution to the original version of the EPBI or implementation strategy on usability, quality of care, and patient-reported outcomes. The final products from the center will be a streamlined modification and redesign model that will improve the usability of EBPIs and implementation strategies (eg, tools to support EBPI education and decision making); a matrix of modification targets (ie, usability issues) that are both common and unique to EBPIs, strategies, settings, and patient populations; and a compilation of redesign strategies and the relative effectiveness of the redesigned solution compared to the original EBPI or strategy. Results: The UWAC received institutional review board approval for the three separate studies in March 2018 and was funded in May 2018. Conclusions: The outcomes from this center will inform the implementation of EBPIs by identifying cross-cutting features of EBPIs and implementation strategies that influence the use and acceptability of these interventions, actively involving stakeholder clinicians and implementation practitioners in the design of the EBPI modification or implementation strategy solution and identifying the impact of HCD-informed modifications and solutions on intervention effectiveness and quality. Trial Registration: ClinicalTrials.gov NCT03515226 (https://clinicaltrials.gov/ct2/show/NCT03515226), NCT03514394 (https://clinicaltrials.gov/ct2/show/NCT03514394), and NCT03516513 (https://clinicaltrials.gov/ct2/show/NCT03516513). International Registered Report Identifier (IRRID): DERR1-10.2196/14990
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The PHQ-9: Validity of a brief depression severity measure
Kroenke, K., Spitzer, R. L., & Williams, J. B. W. (2001) · Journal of General Internal Medicine · doi:10.1046/j.1525-1497.2001.016009606.x · Cited by 44,910
Three Approaches to Qualitative Content Analysis
Hsieh, H.-F., & Shannon, S. E. (2005) · Qualitative Health Research · doi:10.1177/1049732305276687 · Cited by 44,601
Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009) · Implementation Science · doi:10.1186/1748-5908-4-50 · Cited by 14,496
Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., Griffey, R., & Hensley, M. (2011) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-010-0319-7 · Cited by 8,721 · cited 2×
“The emphasis of this testing phase is on user experience, satisfaction with the end design, reported benefit over alternative or existing processes, and implementation outcomes [43].”
BackgroundPowell, B. J., Waltz, T. J., Chinman, M. J., Damschroder, L. J., Smith, J. L., Matthieu, M. M., Proctor, E. K., & Kirchner, J. E. (2015) · Implementation Science · doi:10.1186/s13012-015-0209-1 · Cited by 4,963 · cited 1×
“Numerous implementation strategies have been developed to facilitate EBPI delivery [19-21], but the interventions and their accompanying implementation strategies are complex processes that are often difficult to deliver [22,23].”
BackgroundCurran, G. M., Bauer, M., Mittman, B., Pyne, J. M., & Stetler, C. (2012) · Medical Care · doi:10.1097/mlr.0b013e3182408812 · Cited by 4,058
Advancing a Conceptual Model of Evidence-Based Practice Implementation in Public Service Sectors
Aarons, G. A., Hurlburt, M., & Horwitz, S. M. (2011) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-010-0327-7 · Cited by 3,240 · cited 1×
“An extensive implementation science literature supports the importance of organizations, individuals, and contexts to the adoption and sustainment of EBPIs [91,92].”
BackgroundA Guide to Conducting Consensual Qualitative Research
Hill, C. E., Thompson, B. J., & Williams, E. N. (1997) · The Counseling Psychologist · doi:10.1177/0011000097254001 · Cited by 2,884
Implementation strategies: recommendations for specifying and reporting
Proctor, E. K., Powell, B. J., & McMillen, J. C. (2013) · Implementation Science · doi:10.1186/1748-5908-8-139 · Cited by 2,585 · cited 3×
“Numerous implementation strategies have been developed to facilitate EBPI delivery [19-21], but the interventions and their accompanying implementation strategies are complex processes that are often difficult to deliver [22,23].”
BackgroundMethodA mathematical model of the finding of usability problems
Nielsen, J., & Landauer, T. K. (1993) · Proceedings of the SIGCHI conference on Human factors in computing systems - CHI '93 · doi:10.1145/169059.169166 · Cited by 1,961
Merikangas, K. R., He, J.-P., Burstein, M., Swendsen, J., Avenevoli, S., Case, B., Georgiades, K., Heaton, L., Swanson, S., & Olfson, M. (2011) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1016/j.jaac.2010.10.006 · Cited by 1,579 · cited 1×
“EBPI implementation is particularly challenging because most people receive treatment for mental illness in nontraditional or integrated settings such as primary care [12] and schools [13,14].”
BackgroundCognitive processing therapy for sexual assault victims.
Resick, P. A., & Schnicke, M. K. (1992) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.60.5.748 · Cited by 1,490 · cited 2×
“10 | e14990 | p. 4http://www.researchprotocols.org/2019/10/e14990/ (page number not for citation purposes) XSL•FO RenderX of problem-solving therapy (PST) [45] and cognitive processing therapy (CPT) [46] interventions for the management of depression and anxiety by master’s degree–level social workers from seven feder…”
BackgroundSheehan, D. V., Harnett-Sheehan, K., & Raj, B. A. (1996) · International Clinical Psychopharmacology · doi:10.1097/00004850-199606003-00015 · Cited by 1,440
Dissemination and Implementation Research in HealthTranslating Science to Practice
Brownson, R. C., Colditz, G. A., & Proctor, E. K. (2012) · doi:10.1093/acprof:oso/9780199751877.001.0001 · Cited by 1,355
Wiltsey Stirman, S., Baumann, A. A., & Miller, C. J. (2019) · Implementation Science · doi:10.1186/s13012-019-0898-y · Cited by 1,329
Mixed method designs in implementation research
Palinkas, L. A., Aarons, G. A., Horwitz, S., Chamberlain, P., Hurlburt, M., & Landsverk, J. (2011) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-010-0314-z · Cited by 1,187 · cited 2×
“We will also determine the need to conduct additional qualitative interviews with patients, clinicians, and implementation practitioners for the purpose of quantitative data explanation and elaboration [59] or each project.”
BackgroundMethodFlottorp, S. A., Oxman, A. D., Krause, J., Musila, N. R., Wensing, M., Godycki-Cwirko, M., Baker, R., & Eccles, M. P. (2013) · Implementation Science · doi:10.1186/1748-5908-8-35 · Cited by 1,138
Beidas, R. S., & Kendall, P. C. (2010) · Clinical Psychology: Science and Practice · doi:10.1111/j.1468-2850.2009.01187.x · Cited by 1,085
Methods to support human-centred design
MAGUIRE, M. (2001) · International Journal of Human-Computer Studies · doi:10.1006/ijhc.2001.0503 · Cited by 1,057 · cited 1×
“The first step of our framework leverages important aspects of HCD [39,47,48], including identification of current and potential stakeholders, their needs, influential aspects of the target setting(s), and understanding what facilitates and inhibits the usability of current tools and workflows.”
BackgroundBeyond the five-user assumption: benefits of increased sample sizes in usability testing
Faulkner, L. (2003) · Behavior Research Methods, Instruments, & Computers · doi:10.3758/bf03195514 · Cited by 1,053 · cited 1×
“There is debate in the HCD literature about the appropriate sample size for user testing and growing agreement that sample size depends on the goals of the test, the complexity of its elements, and the number of groups or strata being compared [67,85].”
BackgroundAssessing Psychiatric Impairment in Primary Care with the Sheehan Disability Scale
Leon, A. C., Olfson, M., Portera, L., Farber, L., & Sheehan, D. V. (1997) · The International Journal of Psychiatry in Medicine · doi:10.2190/t8em-c8yh-373n-1uwd · Cited by 987 · cited 1×
“The Sheehan Disability Scale [74,75], a brief analog scale measuring functioning in work, social, and health domains will be administered to patient participants in the Test phase across projects.”
MethodMethods to Improve the Selection and Tailoring of Implementation Strategies
Powell, B. J., Beidas, R. S., Lewis, C. C., Aarons, G. A., McMillen, J. C., Proctor, E. K., & Mandell, D. S. (2017) · The Journal of Behavioral Health Services & Research · doi:10.1007/s11414-015-9475-6 · Cited by 967
McHugh, R. K., Whitton, S. W., Peckham, A. D., Welge, J. A., & Otto, M. W. (2013) · The Journal of Clinical Psychiatry · doi:10.4088/jcp.12r07757 · Cited by 800
Pathways Into and Through Mental Health Services for Children and Adolescents
Farmer, E. M. Z., Burns, B. J., Phillips, S. D., Angold, A., & Costello, E. J. (2003) · Psychiatric Services · doi:10.1176/appi.ps.54.1.60 · Cited by 658
Lewis, C. C., Klasnja, P., Powell, B. J., Lyon, A. R., Tuzzio, L., Jones, S., Walsh-Bailey, C., & Weiner, B. (2018) · Frontiers in Public Health · doi:10.3389/fpubh.2018.00136 · Cited by 588
Self-citationChambers, D. A., & Norton, W. E. (2016) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2016.05.011 · Cited by 581
User‐centered design for psychosocial intervention development and implementation.
Lyon, A. R., & Koerner, K. (2016) · Clinical Psychology: Science and Practice · doi:10.1111/cpsp.12154 · Cited by 478 · cited 6×
“HCD is a field of science that has produced methods to develop compelling, intuitive, and easily adopted products and tools [22,33].”
BackgroundMethodInfluentialSelf-citationMohr, D. C., Lyon, A. R., Lattie, E. G., Reddy, M., & Schueller, S. M. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.7725 · Cited by 432
Self-citationLewis, C. C., Fischer, S., Weiner, B. J., Stanick, C., Kim, M., & Martinez, R. G. (2015) · Implementation Science · doi:10.1186/s13012-015-0342-x · Cited by 430 · cited 1×
“Indeed, recent reviews of implementation measurement instruments [30] and implementation strategies [31] indicate that attention to intervention-level determinants has been sparse.”
BackgroundTrends in Office-Based Mental Health Care Provided by Psychiatrists and Primary Care Physicians
Olfson, M., Kroenke, K., Wang, S., & Blanco, C. (2014) · The Journal of Clinical Psychiatry · doi:10.4088/jcp.13m08834 · Cited by 221 · cited 1×
“EBPI implementation is particularly challenging because most people receive treatment for mental illness in nontraditional or integrated settings such as primary care [12] and schools [13,14].”
BackgroundLyon, A. R., & Bruns, E. J. (2019) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2018.3060 · Cited by 178
Self-citationWilliams, N. J. (2016) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-015-0693-2 · Cited by 164
Interagency collaborative team model for capacity building to scale-up evidence-based practice
Hurlburt, M., Aarons, G. A., Fettes, D., Willging, C., Gunderson, L., & Chaffin, M. J. (2014) · Children and Youth Services Review · doi:10.1016/j.childyouth.2013.10.005 · Cited by 132
“Engage” Therapy: Behavioral Activation and Improvement of Late-Life Major Depression
Alexopoulos, G. S., Raue, P. J., Gunning, F., Kiosses, D. N., Kanellopoulos, D., Pollari, C., Banerjee, S., & Arean, P. A. (2016) · The American Journal of Geriatric Psychiatry · doi:10.1016/j.jagp.2015.11.006 · Cited by 84 · cited 1×
“Recent work has applied HCD to EBPIs to enhance usability, decrease burden, and increase contextual appropriateness [22,34,35].”
MethodComparing Engage with PST in Late-Life Major Depression: A Preliminary Report
Alexopoulos, G. S., Raue, P. J., Kiosses, D. N., Seirup, J. K., Banerjee, S., & Arean, P. A. (2015) · The American Journal of Geriatric Psychiatry · doi:10.1016/j.jagp.2014.06.008 · Cited by 51 · cited 1×
“Usability standards sit in striking contrast to the current state of EBPIs, which are generally difficult to learn, requiring several months of training and supervision [26-28]; impose a high degree of user burden or cognitive load [22]; and do not fit well into typical provider and patient workflows [29].”
BackgroundArean, P. A., Raue, P. J., Sirey, J. A., & Snowden, M. (2012) · Psychiatric Services · doi:10.1176/appi.ps.201100078 · Cited by 33 · cited 1×
“Psychosocial interventions (eg, psychotherapy, counseling, and case management) are a preferred mode of treatment by most people seeking care for mental health problems, particularly among low-income, minority, geriatric, and rural populations [1-7].”
Background
Cited by
Lyon, A. R., Brewer, S. K., & Areán, P. A. (2020) · American Psychologist · doi:10.1037/amp0000652 · Cited by 152
Self-citationReilly, K. L., Kennedy, S., Porter, G., & Estabrooks, P. (2020) · Frontiers in Public Health · doi:10.3389/fpubh.2020.00430 · Cited by 102
“When applied to service recipients, the population that would have health benefits from the evidencebased intervention, understanding these variables can provide valuable information relative to the potential for an evidencebased approach to achieve high reach (23).”
BackgroundAlexopoulos, G. S., Raue, P. J., Banerjee, S., Marino, P., Renn, B. N., Solomonov, N., Adeagbo, A., Sirey, J. A., Hull, T. D., Kiosses, D. N., et al. (2021) · Molecular Psychiatry · doi:10.1038/s41380-020-0832-3 · Cited by 56
Self-citationLewis, C. C., Mettert, K., & Lyon, A. R. (2021) · Implementation Research and Practice · doi:10.1177/2633489521994197 · Cited by 38
Self-citationEscobar-Viera, C. G., Porta, G., Coulter, R. W. S., Martina, J., Goldbach, J., & Rollman, B. L. (2023) · Internet Interventions · doi:10.1016/j.invent.2023.100668 · Cited by 34
Metadata
- Authors
- Lyon, A. R., Munson, S. A., Renn, B. N., Atkins, D. C., Pullmann, M. D., Friedman, E., & Areán, P. A.
- Venue
- JMIR Research Protocols
- Publisher
- JMIR Publications Inc.
- Published
- 2019-10-09
- Volume
- 8
- Issue
- 10
- Pages
- e14990
- ISSN
- 1929-0748 (ISSN-L)
- OpenAlex
- W2980059451
- Cited by
- 187
- References
- 36
- FWCI
- 16.33
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.