A Digital Mental Health Intervention in an Orthopedic Setting for Patients With Symptoms of Depression and/or Anxiety: Feasibility Prospective Cohort Study
Open accessPublished version of a preprint
The DOI matches its Crossref record; 44 of the 44 references it cites were found in a registry or an index; 17 of the 46 sources its claims cite were checked for support.
Abstract
Background: Symptoms of depression and anxiety commonly coexist with chronic musculoskeletal pain, and when this occurs, standard orthopedic treatment is less effective. However, mental health intervention is not yet a routine part of standard orthopedic treatment, in part because of access-related barriers. Digital mental health intervention is a potential scalable resource that could be feasibly incorporated into orthopedic care. Objective: This study’s primary purpose was to assess the feasibility of introducing a digital mental health intervention (Wysa) in an outpatient orthopedic setting to patients with coexisting symptoms of depression and/or anxiety. The secondary purpose was to perform a preliminary effectiveness analysis of the intervention. Methods: In this single-arm, prospective cohort study, participants included adult patients (18 years and older) who presented to a nonsurgical orthopedic specialist at a single tertiary care academic center for evaluation of a musculoskeletal condition and who self-reported symptoms of depression and/or anxiety (Patient-Reported Outcomes Measurement Information System [PROMIS] Depression and/or Anxiety score ≥55). Face-to-face enrollment was performed by a research coordinator immediately after the participant’s encounter with an orthopedic clinician. Participants were provided 2 months of access to a mobile app called Wysa, which is an established, multicomponent digital mental health intervention that uses chatbot technology and text-based access to human counselors to deliver cognitive behavioral therapy, mindfulness training, and sleep tools, among other features. For this study, Wysa access also included novel, behavioral activation–based features specifically developed for users with chronic pain. Primary feasibility outcomes included the study recruitment rate, retention rate, and engagement rate with Wysa (defined as engagement with a therapeutic Wysa tool at least once during the study period). Secondary effectiveness outcomes were between-group differences in mean longitudinal PROMIS mental and physical health score changes at 2-month follow-up between high and low Wysa users, defined by a median split. Results: The recruitment rate was 29.3% (61/208), retention rate was 84% (51/61), and engagement rate was 72% (44/61). Compared to low users, high users reported greater improvement in PROMIS Anxiety scores (between-group difference −4.2 points, 95% CI −8.1 to −0.2; P=.04) at the 2-month follow-up. Between-group differences in PROMIS Depression (−3.2 points, 95% CI −7.5 to 1.2; P=.15) and Pain Interference scores (−2.3 points, 95% CI −6.3 to 1.7; P=.26) favored high users but did not meet statistical significance. Improvements in PROMIS Physical Function scores were comparable between groups. Conclusions: Delivery of a digital mental health intervention within the context of orthopedic care is feasible and has the potential to improve mental health and pain-related impairment to a clinically meaningful degree. Participants’ engagement rates exceeded industry standards, and additional opportunities to improve recruitment and retention were identified. Further pilot study followed by a definitive, randomized controlled trial is warranted. Trial Registration: ClinicalTrials.gov NCT04640090; https://clinicaltrials.gov/ct2/show/NCT04640090
Overview
CoChat Summary
## TL;DR
This single-arm prospective cohort study found that offering the Wysa digital mental health intervention in an outpatient orthopedic clinic was feasible, with 29.3% recruitment, 84% retention, and 72% of enrolled participants engaging with the app. High users improved more in anxiety than low users at 2 months, with a between-group difference of −4.2 points on PROMIS Anxiety.
## Key findings
- Recruitment rate was 29.3% (61/208), retention rate was 84% (51/61), and engagement with Wysa was 72% (44/61).
- Among enrolled participants, the weekly engagement rate was 57% (35/61) and the coach engagement rate was 33% (20/61).
- Compared with low users, high users showed greater 2-month improvement in PROMIS Anxiety: −4.2 points (95% CI −8.1 to −0.2; P=.04).
- High users also favored improvement in PROMIS Depression (−3.2 points, 95% CI −7.5 to 1.2; P=.15) and Pain Interference (−2.3 points, 95% CI −6.3 to 1.7; P=.26), but these were not statistically significant; PROMIS Physical Function improved similarly in both groups.
## Methods
- Design: Single-arm, prospective cohort, pilot feasibility study at a single tertiary care academic medical center in the United States; participants were enrolled from December 8, 2020, through July 14, 2021.
- Sample/data: Adults 18+ seeing a nonsurgical orthopedic provider for a musculoskeletal condition, with PROMIS Depression and/or Anxiety ≥55; excluded if planning in-person mental health treatment or without a mobile device. Data came from electronic medical records, self-report, and Wysa usage logs.
- Main analysis: Feasibility was assessed via recruitment, retention, and engagement rates; preliminary effectiveness used linear mixed-effects models comparing high vs low Wysa users (median split) on 2-month changes in PROMIS Depression, Anxiety, Pain Interference, and Physical Function.
## Limitations
- The paper states the primary limitation was no true control arm because of limited resources.
- The paper notes possible healthy participant bias: high users may have been more ready than low users to address mental health, and similar bias may have affected who enrolled.
- Evident from the text: this was a single-center study with nonsurgical orthopedic providers only, so generalizability is limited.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures






Verification
Claims the paper makes
6 Valid · 1 Not valid · 15 Not checked · 38 Own results and interpretations
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“Participants’ engagement rates exceeded industry standards”
- Also stated in: § Implications These findings suggest that a fully powered, randomized controlled trial is feasible and warranted to assess the clinical benefit of offering a digital mental health intervention in the context of orthopedic care for patients with coexisting symptoms of depression and/or anxiety. Engagement in our study was notably higher than digital health industry standards, which measure engagement on the order of days instead of weeks or months [40]. The high engagement rates and positive clinical findings in this study are consistent with those in a study of 10,000 participants by Bailey et al [41], in which people with chronic low back and/or knee pain were given a 12-week digital health intervention sponsored by their employers [41]. In that study, 73% of participants continued engagement with the program into the third month. We hypothesize that introducing a digital health intervention through a trusted resource such as a physician or employer may contribute to these higher engagement rates. Furthermore, if an orthopedic provider referred a patient to a digital mental health intervention as part of standard care (as opposed to a research study), we hypothesize that rates of onboarding to the intervention would be even greater than the enrollment rates we observed in this study. Patients who present to an orthopedic clinic for a musculoskeletal condition have demonstrated a willingness to engage with digital (physical) health interventions [42], and incorporation of digital mental health support is a logical next step to improving clinical outcomes in orthopedic care. This line of study may also have implications for ambulatory care settings other than orthopedic clinics. We believe that clinical encounters related to chronic pain are a particularly opportune setting to address mental health because when patients understand that their physical pain experience is directly linked to their mental health, their desire to reduce pain becomes a tangible (and potentially nonstigmatizing) motivator for addressing mental health. This approach of inserting a mental health access point and resources could be applied to other frequently used ambulatory care settings that address chronic pain (eg, primary care, rheumatology, gastroenterology, headache clinics) [43], and it would create opportunities to provide mental health screening and intervention for patients with symptoms of depression and/or anxiety who might not have otherwise sought mental health treatment. In our experience, the orthopedic setting poses some especially challenging barriers to the discussion of mental health with patients, which include patients’ expectations regarding appropriate content to discuss during the clinic encounter and providers’ lack of training and possible lack of inherent interest or budgeted time to discuss mental health with patients. Therefore, evidence of the feasibility of delivering mental health intervention within the orthopedic setting is certainly encouraging for potential success in other ambulatory care settings that address chronic pain. Furthermore, although no coordination was made with patients’ established mental health care providers in this study (for those who were already receiving some form of mental health treatment), better care coordination and communication between nontraditional mental health “access points” (eg, within orthopedics) and patients’ established mental health providers should be focused on in the future. · plain
“Engagement in our study was notably higher than digital health industry standards, which measure engagement on the order of days instead of weeks or months [40].”
§ A Digital Mental Health Intervention in an Orthopedic Setting for Patients With Symptoms of Depression and/or Anxiety: Feasibility Prospective Cohort Study > Abstract > Conclusions
What the sources say
- Disputedcites [40] sourceExperimental — accuracy not yet measured
Our copy of Baumel et al. (2019) reports the opposite direction: the Principal Findings state 'general user retention is poor, with a median 15-day retention of 3.9% and 30-day retention of 3.3%' and that these figures 'reflect the lower ranges of real-world retention rates reported in developer-led studies', while the Abstract concludes that 'only a small portion of users actually used the apps for a long period of time'. The App Usage section likewise gives a median daily open rate of only 4.0% and the discussion refers to 'the poor active user rates found in our analysis'. No passage frames the observed engagement as exceeding any industry standard or benchmark; the only 'relatively high' engagement noted is for the two peer-support apps relative to other techniques within the sample. (LLM, conf=0.72)
Checked by a reasoning model — may be wrong.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Implications These findings suggest that a fully powered, randomized controlled trial is feasible and warranted to assess the clinical benefit of offering a digital mental health intervention in the context of orthopedic care for patients with coexisting symptoms of depression and/or anxiety. Engagement in our study was notably higher than digital health industry standards, which measure engagement on the order of days instead of weeks or months [40]. The high engagement rates and positive clinical findings in this study are consistent with those in a study of 10,000 participants by Bailey et al [41], in which people with chronic low back and/or knee pain were given a 12-week digital health intervention sponsored by their employers [41]. In that study, 73% of participants continued engagement with the program into the third month. We hypothesize that introducing a digital health intervention through a trusted resource such as a physician or employer may contribute to these higher engagement rates. Furthermore, if an orthopedic provider referred a patient to a digital mental health intervention as part of standard care (as opposed to a research study), we hypothesize that rates of onboarding to the intervention would be even greater than the enrollment rates we observed in this study. Patients who present to an orthopedic clinic for a musculoskeletal condition have demonstrated a willingness to engage with digital (physical) health interventions [42], and incorporation of digital mental health support is a logical next step to improving clinical outcomes in orthopedic care. This line of study may also have implications for ambulatory care settings other than orthopedic clinics. We believe that clinical encounters related to chronic pain are a particularly opportune setting to address mental health because when patients understand that their physical pain experience is directly linked to their mental health, their desire to reduce pain becomes a tangible (and potentially nonstigmatizing) motivator for addressing mental health. This approach of inserting a mental health access point and resources could be applied to other frequently used ambulatory care settings that address chronic pain (eg, primary care, rheumatology, gastroenterology, headache clinics) [43], and it would create opportunities to provide mental health screening and intervention for patients with symptoms of depression and/or anxiety who might not have otherwise sought mental health treatment. In our experience, the orthopedic setting poses some especially challenging barriers to the discussion of mental health with patients, which include patients’ expectations regarding appropriate content to discuss during the clinic encounter and providers’ lack of training and possible lack of inherent interest or budgeted time to discuss mental health with patients. Therefore, evidence of the feasibility of delivering mental health intervention within the orthopedic setting is certainly encouraging for potential success in other ambulatory care settings that address chronic pain. Furthermore, although no coordination was made with patients’ established mental health care providers in this study (for those who were already receiving some form of mental health treatment), better care coordination and communication between nontraditional mental health “access points” (eg, within orthopedics) and patients’ established mental health providers should be focused on in the future. · plain
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Furthermore, a growing body of evidence supports the effectiveness of digital mental health interventions in improving depression and anxiety symptoms, sometimes to a degree equivalent to that obtained from in-person treatment [17-19].”
§ A Digital Mental Health Intervention in an Orthopedic Setting for Patients With Symptoms of Depression and/or Anxiety: Feasibility Prospective Cohort Study > Introduction > Background
What the sources say
- Supportedcites [17] source
The introductory passage (p. 3) describes more than 200 randomized trials, often finding internet-delivered CBT effective against controls, with anxiety and depression among its typical targets. The Discussion (p. 16) reports that internet-delivered and face-to-face CBT were equally effective for depressive symptoms and several anxiety disorders. This supports the cited sentence’s qualified claim that digital interventions can sometimes achieve comparable results. (LLM, conf=0.90)
- Partly supportedcites [18] sourceExperimental — accuracy not yet measured
The Introduction reports that computerized therapies can reduce anxiety symptoms with efficacy similar to face-to-face treatment, and the Discussion concludes that smartphone interventions can reduce anxiety. The paper does not establish improvement in depression symptoms: the study it describes as targeting major depression is discussed for its anxiety outcome (section 3.6). The Discussion also says smartphone-only therapy had not yet been compared directly with face-to-face therapy. (LLM, conf=0.87)
- Partly supportedcites [19] sourceExperimental — accuracy not yet measured
The Abstract and Discussion report that app-supported smartphone interventions improved depressive and anxiety symptoms compared with controls. The Abstract also says they did not differ significantly from active interventions, including face-to-face treatment, but notes that few studies made those comparisons (n≤13). A nonsignificant difference does not establish equivalent effectiveness to in-person treatment. (LLM, conf=0.86)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
None of the sources cited for this claim have been checked yet.
In the paper
“when this occurs, standard orthopedic treatment is less effective.”
- Also stated in: § Background · plain
“when a person has both conditions simultaneously, the effectiveness of standard treatment for either condition in isolation is reduced [5-7].”
§ A Digital Mental Health Intervention in an Orthopedic Setting for Patients With Symptoms of Depression and/or Anxiety: Feasibility Prospective Cohort Study > Abstract > Background
What the sources say
- Not checked yetcites [5] source
- Could not verifycites [6] sourceExperimental — accuracy not yet measured
The Results and Discussion (pp. 732–741) address the effect of pain on depression-treatment outcomes, not the effect of depression or anxiety on orthopedic treatment. A search for orthopedic treatment and musculoskeletal-pain outcomes found no passage assessing whether standard orthopedic care is less effective in patients with these conditions. (LLM, conf=0.76)
- Not checked yetcites [7] source
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Background · plain
None of the sources cited for this claim have been checked yet.
In the paper
“Chronic pain and psychological impairment such as depression and anxiety are driven by shared biological pathways and neurotransmitters”
§ A Digital Mental Health Intervention in an Orthopedic Setting for Patients With Symptoms of Depression and/or Anxiety: Feasibility Prospective Cohort Study > Introduction > Background
What the sources say
- Not checked yetcites [5] source
- Could not verifycites [6] sourceExperimental — accuracy not yet measured
The Introduction (paper pp. 732–741) describes pain and depression as frequently co-occurring and discusses antidepressants that can treat certain pain conditions. The Discussion mentions panic symptoms, but neither passage says that pain, depression, and anxiety are driven by shared biological pathways or neurotransmitters. Searches for shared pathways and neurotransmitter mechanisms found no passage addressing that claim. (LLM, conf=0.76)
- Not checked yetcites [7] source
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Minimum clinically meaningful effect sizes were a priori chosen in accordance with previously published literature involving patients with chronic musculoskeletal pain who were managed nonoperatively.”
§ A Digital Mental Health Intervention in an Orthopedic Setting for Patients With Symptoms of Depression and/or Anxiety: Feasibility Prospective Cohort Study > Methods > Outcomes
What the sources say
- Supportedcites [34] source
Lee et al. (2017) is exactly the kind of work the citing sentence invokes: its METHODS ('Minimally important differences') state that 'only MIDs that met 3 a priori criteria were selected,' the third being that 'Effect size (Cohen's d) of each MID falls within plausible range (0.2≤ x≤ 0.8)', reiterated in the DISCUSSION as an effect size 'both large enough to be considered important (≥0.2) and small enough to be considered minimal (0.8≥).' Its cohort (METHODS, 'Study participants') are adults with symptomatic knee osteoarthritis (mean pain duration 8.7 years, i.e. chronic musculoskeletal pain) enrolled in a randomized trial of Tai Chi versus physical therapy — nonoperative management, with intra-articular steroid injection or knee replacement within three months an exclusion. So the cited paper both pre-specifies minimum clinically meaningful effect-size thresholds and does so in a nonoperatively managed chronic musculoskeletal pain population, supporting the identification/exemplification the citation offers. (The 0.2–0.8 bounds themselves are credited in the source to its references 55 and 79, but the claim only asserts that such a priori thresholds appear in prior literature in this population, which the source satisfies.) (LLM, conf=0.82)
- Partly supportedcites [35] sourceExperimental — accuracy not yet measured
The source does fit the descriptive frame of the citation: it is previously published literature on 244 primary care patients with chronic musculoskeletal pain managed nonoperatively (Abstract: secondary analysis of the SCOPE randomized trial of optimized analgesic therapy at the Roudebush VA), and it reports minimum clinically important change values — MID of 2 points (depression) and 2.5 points (anxiety), with a three-point change called a conservative estimate (Discussion). What it does not support is the specific thing attributed: our copy nowhere reports or chooses 'minimum clinically meaningful effect sizes' a priori — the Statistical Analysis section derives the MID post hoc from the standard error of measurement (1 or 2 SEMs, refs 33,34) in raw scale points, not standardized effect sizes, and the paper is a psychometric operating-characteristics analysis rather than a trial protocol setting a priori thresholds. (LLM, conf=0.72)
- Supportedcites [36] source
Chen et al. (2018) is exactly the kind of published literature the citing sentence invokes: its Methods (2.1, Design and Participants) describe Samples 1 and 2 as primary-care RCTs of non-surgical treatments (pharmacological vs behavioral for chronic low back pain; opioid vs non-opioid medication for chronic back pain or hip/knee osteoarthritis), and the Discussion states that 'pain in the two pain trials was chronic and musculoskeletal.' The paper supplies minimum clinically meaningful thresholds for those chronic musculoskeletal pain samples: effect-size-based bounds of 0.2/0.35/0.5 SD (Methods 2.4.1) and recommended MIDs of 2.0–3.0 points (Results 3.4). The only element not stated verbatim is 'managed nonoperatively,' which follows from the trial designs our copy reports; the citing authors' own a priori choice is their statement, not attributed content. (LLM, conf=0.82)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
29 more claims · 1 with their cited sources checked
Create a free account to see all 60 claims and to verify the 29 references still open.
- Probably not a substantial claim. 16
- It may state two separate claims as one. 14
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Our previous pilot work related to this digital mental health intervention suggested that depression symptoms continued to improve between 1- and 2-month follow-ups [25].”
From Behavioral Mechanisms That Mediate Mental and Physical Health Improvements in People With Chronic Pain Who Receive a Digital Health Intervention: Prospective Cohort Pilot Study (2023) · cites this paper as [25]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“This study adds clinical context to our previously reported finding that it is feasible to deliver a digital mental health intervention in the setting of orthopedic care [14].”
The cited source says what the claim attributes to it.
In the citing paper
“Our preliminary work suggests that a digital mental health intervention (Wysa for Chronic Pain) which was designed to address these 3 behavioral targets can improve self-reported mental and physical health in patients who present for orthopedic management of chronic musculoskeletal pain and have coexisting symptoms of depression and anxiety [25,26].”
Citations
Cites
Harris, P. A., Taylor, R., Thielke, R., Payne, J., Gonzalez, N., & Conde, J. G. (2009) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2008.08.010 · Cited by 51,384 · cited 1×
“All electronic medical record and self-reported data were stored in a secure electronic Research Electronic Data Capture (REDCap; Vanderbilt University) database [32,33].”
MethodThe REDCap consortium: building an international community of software platform partners
Harris, P. A., Taylor, R., Minor, B. L., Elliott, V., Fernandez, M., O'Neal, L., McLeod, L., Delacqua, G., Delacqua, F., Kirby, J., et al. (2019) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2019.103208 · Cited by 24,013 · cited 1×
“All electronic medical record and self-reported data were stored in a secure electronic Research Electronic Data Capture (REDCap; Vanderbilt University) database [32,33].”
MethodThe brief resilience scale: Assessing the ability to bounce back
Smith, B. W., Dalen, J., Wiggins, K., Tooley, E., Christopher, P., & Bernard, J. (2008) · International Journal of Behavioral Medicine · doi:10.1080/10705500802222972 · Cited by 6,439 · cited 3×
“3.2 (2.5-3.7)Brief Resilience Scale scorec, median (IQR) Potential for behavior changed, median (IQR) 76 (68-90)Importance of change 69 (50-82)Readiness for change 68 (50-74)Confidence in the ability to change 46 (75)Any pain medication use, n (”
BackgroundInfluentialDepression and Pain Comorbidity: A Literature Review
Bair, M. J., Robinson, R. L., Katon, W., & Kroenke, K. (2003) · Archives of Internal Medicine · doi:10.1001/archinte.163.20.2433 · Cited by 3,463
Cella, D., Yount, S., Rothrock, N., Gershon, R., Cook, K., Reeve, B., Ader, D., Fries, J. F., Bruce, B., & Rose, M. (2007) · Medical Care · doi:10.1097/01.mlr.0000258615.42478.55 · Cited by 3,118
Area Deprivation and Widening Inequalities in US Mortality, 1969–1998
Singh, G. K. (2003) · American Journal of Public Health · doi:10.2105/ajph.93.7.1137 · Cited by 1,423 · cited 1×
“aThe national Area Deprivation Index is a community-level measure of social disadvantage based on a person’s 9-digit zip code [37,38].”
BackgroundNeighborhood Socioeconomic Disadvantage and 30-Day Rehospitalization: A Retrospective Cohort Study
Kind, A. J. H., Jencks, S., Brock, J., Yu, M., Bartels, C., Ehlenbach, W., Greenberg, C., & Smith, M. (2014) · Annals of Internal Medicine · doi:10.7326/m13-2946 · Cited by 1,379
Barriers to mental health treatment: results from the WHO World Mental Health surveys
Andrade, L. H., Alonso, J., Mneimneh, Z., Wells, J. E., Al-Hamzawi, A., Borges, G., Bromet, E., Bruffaerts, R., de Girolamo, G., de Graaf, R., et al. (2014) · Psychological Medicine · doi:10.1017/s0033291713001943 · Cited by 1,289 · cited 1×
“Among people who recognize their need for mental health treatment, 64% are resistant to reaching out to another person for assistance and 23% have financial, transportation, or time-related barriers to seeking care [13,14].”
BackgroundCarlbring, P., Andersson, G., Cuijpers, P., Riper, H., & Hedman-Lagerlöf, E. (2018) · Cognitive Behaviour Therapy · doi:10.1080/16506073.2017.1401115 · Cited by 1,232 · cited 1×
“Furthermore, a growing body of evidence supports the effectiveness of digital mental health interventions in improving depression and anxiety symptoms, sometimes to a degree equivalent to that obtained from in-person treatment [17-19].”
BackgroundTorous, J., Bucci, S., Bell, I. H., Kessing, L. V., Faurholt‐Jepsen, M., Whelan, P., Carvalho, A. F., Keshavan, M., Linardon, J., & Firth, J. (2021) · World Psychiatry · doi:10.1002/wps.20883 · Cited by 1,199 · cited 1×
“With the widespread, everyday use of smartphones, these emerging digital tools could function as efficient, widely available mental health resources for orthopedic clinicians to offer their patients [16,20-22].”
Background
Research works printed but not linked: 3
- Zelaya, C, 2020, NCHS Data Brief, 1
- Vranceanu, AM, 2017, Arch Bone Jt Surg, 5, 2
- Zimmerman, GL, 2000, Am Fam Physician, 61, 1409
Cited by
Li, H., Zhang, R., Lee, Y.-C., Kraut, R. E., & Mohr, D. C. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00979-5 · Cited by 598
“…CAs in mental health, only 15 studies incorporated safety assessment or protection measures in CAs, such as access to human experts 20,23,39 , onboarding processes 19,20,25,26,31 , assessment of adverse events 25,29,31,34 and automatic crises or harm identi fi cation 17,19,25,31,34,36,41,42,46,47 .”
BackgroundEngagement and retention in digital mental health interventions: a narrative review
Boucher, E. M., & Raiker, J. S. (2024) · BMC Digital Health · doi:10.1186/s44247-024-00105-9 · Cited by 181
“Some studies have reported above industry standard engagement rates when participants learned about the DMHI through their employer [101] or during a visit to an orthopedic clinic [64].”
BackgroundHua, Y., Siddals, S., Ma, Z., Galatzer‐Levy, I., Xia, W., Hau, C., Na, H., Flathers, M., Linardon, J., Ayubcha, C., et al. (2025) · World Psychiatry · doi:10.1002/wps.21352 · Cited by 117
Mitsea, E., Drigas, A., & Skianis, C. (2023) · Behavioral Sciences · doi:10.3390/bs13121008 · Cited by 101
Artificial intelligence for psychotherapy: a review of the current state and future directions
Beg, M. J., Verma, M., M., V. C. K. M., & Verma, M. K. (2025) · Indian Journal of Psychological Medicine · doi:10.1177/02537176241260819 · Cited by 82
“30 Merry et al. found that SPARX computerized cognitive-behavioral treatment effectively reduced depressive symptoms in teen-agers despite the lack of blinding, highlighting the potential benefits of digital mental health therapy in various settings.”
Robinson, C. L., D'Souza, R. S., Yazdi, C., Diejomaoh, E. M., Schatman, M. E., Emerick, T. D., & Orhurhu, V. (2024) · Journal of Pain Research · doi:10.2147/jpr.s439452 · Cited by 70
“14,15 One potential area of development is the integration of NLP, the field that utilizes computational linguistics, and ML to process human language, potentially resulting in improved conversational AI interfaces.”
BackgroundDing, H., Simmich, J., Vaezipour, A., Andrews, N. E., & Russell, T. G. (2023) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad222 · Cited by 72
Inkster, B., Kadaba, M., & Subramanian, V. (2023) · Frontiers in Global Women's Health · doi:10.3389/fgwh.2023.1084302 · Cited by 53
Sinha, C., Meheli, S., & Kadaba, M. (2022) · JMIR Formative Research · doi:10.2196/41913 · Cited by 63
“Additionally, Wysa has been found to improve mental health outcomes in chronic conditions [20,21], with high engagement and adherence [22].”
BackgroundMacNeill, A. L., Doucet, S. A., & Luke, A. (2024) · JMIR Formative Research · doi:10.2196/50025 · Cited by 96
“Less is known about their effectiveness in patient populations, although some research indicates that they may be beneficial for people with musculoskeletal conditions [23] and chronic pain [24].”
BackgroundSinha, C., Cheng, A. L., & Kadaba, M. (2022) · JMIR Formative Research · doi:10.2196/37302 · Cited by 51
“The pilot study that used this intervention has also examined the app’s clinical efficacy while sustaining this engagement [16].”
BackgroundSelf-citationLeo, A. J., Schuelke, M. J., Hunt, D. M., Miller, J. P., Areán, P. A., & Cheng, A. L. (2022) · JMIR Formative Research · doi:10.2196/36203 · Cited by 40
“Furthermore, in a single-arm prospective pilot study, we demonstrated that delivery of a digital mental health intervention (Wysa) within an outpatient orthopedic care setting is feasible and preliminary effectiveness analyses are promising [14].”
InfluentialSelf-citationMitsea, E., Drigas, A. S., & Skianis, C. A. (2024) · Electronics · doi:10.3390/electronics13122253 · Cited by 48
Alavi, N., Moghimi, E., Stephenson, C., Gutierrez, G., Jagayat, J., Kumar, A., Shao, Y., Miller, S., Yee, C. S., Stefatos, A., et al. (2023) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2023.1113956 · Cited by 36
Gupta, M., Malik, T., & Sinha, C. (2022) · JMIR Research Protocols · doi:10.2196/36910 · Cited by 43
“Wysa for Chronic Pain has proven high engagement and efficacy when the intervention uses a conversational agent enhanced by a human coach [87].”
BackgroundMethodBeyond Doomsday Fears: Why We Need to Consider the Potential Harms of AI Psychotherapy
Tekin, Ş., & Delehanty, M. (2025) · The American Journal of Bioethics · doi:10.1080/15265161.2025.2457724 · Cited by 55
“In that study, Leo et al. (2022) Interference and Physical Function.”
The first AI ‐based Chatbot to promote HIV self‐management: A mixed methods usability study
Ma, Y., Achiche, S., Tu, G., Vicente, S., Lessard, D., Engler, K., Lemire, B., Laymouna, M. A., de Pokomandy, A., Cox, J., et al. (2024) · HIV Medicine · doi:10.1111/hiv.13720 · Cited by 35
“Concerning feasibility, the recruitment rate was 74% (40/54), with 70% of participants completing at least 20 rounds of conversation with MARVIN in 3 weeks, rates comparable to other successful healthcare chatbot studies [80, 81].”
Le, K. B. Q., & Cayrat, C. (2024) · Journal of service management · doi:10.1108/josm-09-2023-0408 · Cited by 36
Barreveld, A. M., Rosén Klement, M. L., Cheung, S., Axelsson, U., Basem, J. I., Reddy, A. S., Borrebaeck, C. A. K., & Mehta, N. (2023) · Pain Medicine · doi:10.1093/pm/pnad049 · Cited by 22
AI-Led Mental Health Support (Wysa) for Health Care Workers During COVID-19: Service Evaluation
Chang, C. L., Sinha, C., Roy, M., & Wong, J. C. M. (2024) · JMIR Formative Research · doi:10.2196/51858 · Cited by 27
“The high initial engagement rate of 93.9% of onboarded users completing at least 1 in-app session may have been due to this intervention being introduced through a trusted source—a health care institution or employer [14].”
BackgroundMethodInfluentialIglesias, M., Sinha, C., Vempati, R., Grace, S. E., Roy, M., Chapman, W. C., & Rinaldi, M. L. (2022) · Journal of Occupational and Environmental Medicine · doi:10.1097/jom.0000000000002762 · Cited by 27
“Patients who received Wysa reported greater improvements on measures of anxiety, depression, pain interference, and physical function than those who received usual orthopedic care.(28,29) In this study, we examine the feasibility and acceptability of a digital psychosocial intervention, Wysa for Return to Work, for in…”
BackgroundMazurek, J., Cieślik, B., Wrzeciono, A., Gajda, R., & Szczepańska-Gieracha, J. (2023) · Journal of Clinical Medicine · doi:10.3390/jcm12247681 · Cited by 25
“Depressive and anxiety disorders have been identified in the literature as potential factors that can impede the efficacy of rehabilitation programs [12].”
BackgroundLi, H. Z., Zhang, R., Lee, Y., Kraut, R. E., & Mohr, D. C. (2023) · doi:10.31234/osf.io/m3vjt · Cited by 19
(2023) · doi:10.2196/47350 · Cited by 18
“Previous studies have shown the efficacy of using deep breathing techniques as a part of stress and anxiety management programs [47,48].”
Oh, K. M., Hong, S.-S., Beran, K., Sanders, L., Park, J. Y., & Lee, J. (2025) · Journal of Applied Gerontology · doi:10.1177/07334648251341629 · Cited by 18
“Leo et al. (2022) reported an 84% engagement rate, indicating successful participant engagement.”
InfluentialArtificial Intelligence (AI) Generated Health Counseling For Mental Illness Patients
Shankar Ganesh, M., & Venkateswaramurthy, N. (2024) · Current Psychiatry Research and Reviews · doi:10.2174/0126660822277500240109050359 · Cited by 25
Striegl, J., Richter, J., Grossmann, L., Bråstad, B., Gotthardt, M., Rück, C., Wallert, J., & Loitsch, C. (2024) · PeerJ Computer Science · doi:10.7717/peerj-cs.2104 · Cited by 27
Jin, S., Kim, B., & Han, K. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713732 · Cited by 28
The rise of artificial intelligence for cognitive behavioral therapy: A bibliometric overview
Vanhée, L., Andersson, G., Garcia, D., & Sikström, S. (2025) · Applied Psychology Health and Well-Being · doi:10.1111/aphw.70033 · Cited by 14
“…support innovative experimental designs (e.g., biometrics, adaptive support systems) and automate tasks such as chatbot-driven pre-screening of research participants (Leo et al., 2022), making previously complex research processes more feasible and opening up new possibilities for investigation.”
Cheng, A. L., Agarwal, M., Armbrecht, M. A., Abraham, J., Calfee, R. P., & Goss, C. W. (2023) · JMIR Formative Research · doi:10.2196/51422 · Cited by 10
“…health intervention (Wysa for Chronic Pain) which was designed to address these 3 behavioral targets can improve self-reported mental and physical health in patients who present for orthopedic management of chronic musculoskeletal pain and have coexisting symptoms of depression and anxiety [25,26].”
BackgroundSelf-citationChaudhry, F., Morgan, S. B., Kruse, C. C., Wolfstadt, J. I., & Ekhtiari, S. (2024) · Journal of the American Academy of Orthopaedic Surgeons · doi:10.5435/jaaos-d-23-01130 · Cited by 8
Ethical Considerations Surrounding the Use of AI-Interventions in Application of Mental Healthcare
Verma, M., Chanthar, K. M. M. V., & Beg, M. J. (2025) · Advances in medical technologies and clinical practice book series · doi:10.4018/979-8-3693-3112-5.ch009 · Cited by 9
Revolutionizing Mental Healthcare Through Artificial Intelligence
Beg, M. J., & Verma, M. K. (2025) · Advances in medical technologies and clinical practice book series · doi:10.4018/979-8-3693-3112-5.ch002 · Cited by 14
Dreaming big with little therapy devices: automated therapy from India
Lang, C. (2024) · Anthropology and Medicine · doi:10.1080/13648470.2024.2378727 · Cited by 9
“Claiming quality and authority and engineering evidence of efficacy and impact, Wysa has published efficacy studies in the form of clinical trials for chronic pain and other conditions with external academic institutions (Leo et al. 2022).”
Conversational Agents to Support Pain Management: A Scoping Review
Souza, F. L., Bowman, H., Yang, F., Hesam‐Shariati, N., Linke, J., Gilanyi, Y. L., Jones, M. D., Z‐Pinto, R., McAuley, J. H., & Rizzo, R. R. N. (2025) · European Journal of Pain · doi:10.1002/ejp.70016 · Cited by 8
Cheng, A. L., Leo, A. J., Calfee, R. P., Dy, C. J., Armbrecht, M. A., & Abraham, J. (2023) · BMC Psychiatry · doi:10.1186/s12888-023-04868-9 · Cited by 6
“The digital intervention, called Wysa for Chronic Pain, is an evidence-based mental health app that addresses the interplay between mental health and chronic pain [19, 20, 40].”
MethodSelf-citationMapping the Process of Engagement With Digital Health Interventions: A Cross-Case Synthesis
Milne‐Ives, M., Homer, S. R., Andrade, J., & Meinert, E. (2025) · Mayo Clinic Proceedings Innovations Quality & Outcomes · doi:10.1016/j.mayocpiqo.2025.100625 · Cited by 7
“Previous evaluations of Wysa have reported more positive evidence of engagement (median user retention ¼ 51 days, 57 weekly engagement rate ¼ 57%, 58 and more expressions of gratitude (n ¼ 212) than dissatisfaction (n ¼ 15) toward the CA 59 ); however, differences in levels of engagement likely…”
Chatbot-Based Mood and Activity Journaling for Resource-Oriented CBT Support of Students
Striegl, J., Fekih, F., Weber, G., & Loitsch, C. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-61063-9_12 · Cited by 6
Striegl, J., Rottmann, S., & Loitsch, C. (2024) · Lecture notes in networks and systems · doi:10.1007/978-3-031-66329-1_14 · Cited by 4
Cheng, A. L., Abraham, J., Hartz, S. M., Laber, E. B., & Miller, J. P. (2025) · JMIR Research Protocols · doi:10.2196/77532 · Cited by 8
“Preliminary studies have demonstrated encouraging signals of effectiveness for Wysa for Chronic Pain among adults across the age spectrum [32-34].”
Self-citationTheory of Mind in Artificial Intelligence Applications
Garcia-Lopez, A. (2023) · Logic, argumentation & reasoning · doi:10.1007/978-3-031-46742-4_23 · Cited by 10
Ye, Q., Tang, W., Xu, T., Zhu, D., & Hua, W. (2025) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2025.105243 · Cited by 6
Yang, H., Chang, F., Muroi, F., Liu, Z., Zhang, W., & Cai, J. (2026) · Psychotherapy and Psychosomatics · doi:10.1159/000552072 · Cited by 3
Assistive Chatbots for healthcare: a succinct reviewPreprint
Bhattacharya, B. S. (2023) · doi:10.36227/techrxiv.23741469 · Cited by 5
Voice Assistant-Based Cognitive Behavioral Therapy for Test Anxiety in Students
Striegl, J., Loitsch, C., & Weber, G. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-35602-5_28 · Cited by 2
Denecke, K., Rivera-Romero, O., Wynn, R., & Gabarrón, E. (2025) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2025.1429304 · Cited by 4
Davis, A., Landers, S. E., Sun, Y., Nguyen, V. O., Bolger, N. P., Lewis, R. A., & Rosenthal, S. L. (2025) · Clinical Pediatrics · doi:10.1177/00099228251371903 · Cited by 5
“26,27 However, the feasibility and acceptability of using Wysa in youth has not yet been tested.”
Alexander, D., Alexander, B., Martin, T. C., & Christoph, J. (2025) · Scientific Reports · doi:10.1038/s41598-025-21290-1 · Cited by 4
“Outliers, defined as 1.5 times interquartile range, were removed in experiment 1 for uncanniness (95), human likeness (28), and UTAUT acceptance (16), and in experiment 2 for uncanniness (17), human likeness (8), and UTAUT acceptance (6).”
InfluentialReconfiguring Psy Expertise in the Digital Age: Two Cases from India
Lang, C. (2025) · Medicine Anthropology Theory · doi:10.17157/mat.12.1.9060 · Cited by 2
“The studies on perceived needs and digital engagement of chronic pain patients (Leo et al. 2022; Sinha, Cheng and Kadaba 2022) stress ‘user-led research’, which is ‘guided by the voice of the user through co-design and qualitative methodologies.’”
Application of AI in Addressing Challenges of Primary Healthcare in Hong Kong
Li, L. W., & Ma, C. (2025) · Quality of life in Asia · doi:10.1007/978-981-96-0817-1_36 · Cited by 2
TRANSFORMATION BASED DIGITAL TECHNOLOGY: THE EFFECTIVITY OF PSYCHOSOCIAL INTERVENTION METHOD
Putra, I. K. A. H., Ardhaneswari, P. P. N., Ariasih, N. W. D., & Kanaya, I. K. K. (2022) · Smart City · doi:10.56940/sc.v2.i1.3 · Cited by 2
Assistive Chatbots for healthcare: a succinct reviewPreprint
Bhattacharya, B. S. (2023) · doi:10.36227/techrxiv.23741469.v1 · Cited by 4
Clark, D. S., Moore, B. P., & Somerson, J. S. (2024) · Journal of Shoulder and Elbow Surgery · doi:10.1016/j.jse.2024.05.059 · Cited by 1
Alam, A. (2025) · IGI Global eBooks · doi:10.4018/979-8-3373-5007-3.ch001 · Cited by 1
Diel, A., Hölsken, S., Christoph, J., Lalgi, T. J., Lena, H., Martin, T. C., & Bäuerle, A. (2026) · Mental Health & Prevention · doi:10.1016/j.mhp.2026.200503 · Cited by 1
Sinha, C., Meheli, S., & Kadaba, M. (2022) · doi:10.2196/preprints.41913 · Cited by 1
Roige, A. (2026) · The American Journal of Bioethics · doi:10.1080/15265161.2025.2608646 · Cited by 1
Artificial Intelligence Interventions in Chronic Pain Management: A Scoping Review (Preprint)Preprint
Abiodun, A. O., Das, S., & Taylor, A. (2024) · doi:10.2196/preprints.57883
USO DO CHATBOT NO ENFRENTAMENTO DA ANSIEDADE: UMA REVISÃO INTEGRATIVA
Ferreira, D. D. S., Martins, C. B., Barbalho, I. M. P., Maciel, N. D. S., de Oliveira, B. S. B., & Jorge, M. S. B. (2024) · Revista Enfermagem Atual In Derme · doi:10.31011/reaid-2024-v.98-n.2-art.2140
Use of Artificial Intelligence Interventions in Chronic Pain Management: A Scoping Review (Preprint)
Abiodun, A. O., Das, S., & Taylor, A. (2024) · Journal of Medical Internet Research · doi:10.2196/57883
Feedback in Rehabilitation Settings
Knauer, J., Terhorst, Y., & Baumeister, H. (2025) · CBT: science into practice · doi:10.1007/978-3-031-90428-8_10
Impact of Physician Mindfulness on Patient Care Quality: A Study from an Indian Corporate Hospital
Bandi, V., Parepalli, K., & Radhakrishna, G. (2025) · International Research Journal of Multidisciplinary Scope · doi:10.47857/irjms.2025.v06i03.04542
Yang, Y., Fan, L., Wang, T., & Yan, R. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5472396
Precision Psychiatry in the Age of AI: Early Detection and Personalized Treatment of Mood Disorders
Verma, M., & Beg, M. J. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-1325-2.ch004
van de Kuit, A., van Maarseveen, O. E. C., Kempen, D. H. R., Scholtes, V. A. B., Broekman, B. F. P., & Poolman, R. W. (2026) · BMJ Open · doi:10.1136/bmjopen-2024-095699 · Cited by 1
Huang, H., Zhang, G., Hu, X., Zhang, Z., Xie, H., & Wang, C. (2026) · BMC Psychology · doi:10.1186/s40359-026-04470-0
“Depression is a mood disorder that causes a persistent feeling of sadness and loss of interest that interferes with daily activities, whereas anxiety is an emotional state characterised by feelings of tension and intrusive thoughts [29].”
Scoping Review: Intervention Methods in Conversational Agent-Based Cognitive Behavioral Therapy
Striegl, J., Butz, N., & Loitsch, C. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-29842-3_16
Yang, Y., Fan, L., Wang, T. R., & Yan, R. (2026) · Acta Psychologica · doi:10.1016/j.actpsy.2026.107325 · Cited by 1
Artificial intelligence in psychotherapy and counseling
Bhatt, S., & Dandona, A. (2026) · Elsevier eBooks · doi:10.1016/b978-0-443-43934-6.00019-5
Cheng, A. L., Gou, C. Y., Martin, A., Hartz, S. M., & Abraham, J. (2026) · JMIR Human Factors · doi:10.2196/87365 · Cited by 1
Self-citation
Metadata
- Authors
- Leo, A. J., Schuelke, M. J., Hunt, D. M., Metzler, J. P., Miller, J. P., Areán, P. A., Armbrecht, M. A., & Cheng, A. L.
- Affiliations
- Washington University in St. Louis (US)
- University of Washington (US)
- Corresponding author
- Cheng, Abby L
- ORCID
- Venue
- JMIR Formative Research
- Publisher
- JMIR Publications Inc.
- Published
- 2022-02-21
- Volume
- 6
- Issue
- 2
- Pages
- e34889
- ISSN
- 2561-326X (ISSN-L)
- PMC
- PMC8902664
- OpenAlex
- W4205207624
- Cited by
- 70
- References
- 44
- Citations by year
- 2026: 10 · 2025: 20 · 2024: 18 · 2023: 15 · 2022: 7
- FWCI
- 13.25
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- JMIR Formative Research (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Funders
- Doris Duke Charitable Foundation
- National Institutes of Health
- National Institute of Arthritis and Musculoskeletal and Skin Diseases (K23AR074520)
- Topics
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Health Sciences › Medicine › Psychiatry and Mental health › Mental Health Treatment and Access
- Keywords
- digital mental health interventions, depression, anxiety, chronic musculoskeletal pain, orthopedic patients, Wysa, chatbots, mobile application, behavioral activation, cognitive behavioral therapy, mindfulness, PROMIS, feasibility, retention, engagement, treatment-resistant depression, digital therapeutics, mental health outcomes, patient-reported outcomes
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.