Behavioral Activation and Depression Symptomatology: Longitudinal Assessment of Linguistic Indicators in Text-Based Therapy Sessions

May combine two claims.

All LIWC variables together explain the most variance without detracting from model fit.

“All LIWC variables together explain the most variance without detracting from model fit.”

May combine two claims.

Less depressed individuals used more "we" pronouns and fewer "I" pronouns.

“Less depressed individuals used more “we” pronouns and fewer “I” pronouns.”

May combine two claims.

Health and biology categories appear to show increased usage in more depressed individuals but have remarkably large confidence intervals for the least depressed group.

“The health and biology categories appear to show increased usage in more depressed individuals but have remarkably large confidence intervals for the least depressed group (none/minimal).”

May combine two claims.

Changes in established LIWC variables are consistent with case-control differences demonstrated in prior research, with tone and first-person plural pronouns increasing as symptoms decrease and first-person singular pronoun usage decreasing with improvement in symptoms.

“Changes in established LIWC variables are consistent with case-control differences demonstrated in prior research [14,19,20], with tone and first-person plural pronouns increasing as symptoms decrease and first-person singular pronoun usage decreasing with improvement in symptoms.”

• cites [14] sourceNot checked yet

  Not checked in the free run.

• cites [19] sourceNot checked yet

  Not checked in the free run.

• cites [20] De ChoudhuryMGamonMCountsSHorvitzEPredicting Depression via Social Media2013Seventh International AAAI Conference on Weblogs and Social MediaJuly 8-11, 2013Cambridge, MANot checked yet

  Not checked in the free run.

May combine two claims.

Mean activation is different between moderately depressed individuals and those with only minimal symptoms, while tone was not.

“mean activation is different between moderately depressed individuals and those with only minimal symptoms, while tone was not.”

May combine two claims.

Topic-related word categories were significantly different between severity groups, with sadness having the most pronounced differences between groups.

“The topic-related word categories were also significantly different between severity groups, with sadness having the most pronounced differences between groups.”

May combine two claims.

Our dataset of naturally occurring language contains chat conversations from over 10,000 individuals over almost 3 years, with over 74 million words.

“Containing chat conversations from over 10,000 individuals over almost 3 years, with over 74 million words, our dataset of naturally occurring language”

May combine two claims.

Tone appears to be clearly different between the moderately severe and severe groups at baseline, while activation is not.

“tone appears to be clearly different between the moderately severe and severe groups at baseline, while activation is not.”

May combine two claims.

The 3 topics together explain more variance than the tone and pronoun variables together and also have better fit.

“The health category is the most informative topic category; the 3 topics together explain more variance than the tone and pronoun variables together and also have better fit.”

May combine two claims.

Neither the breadth of activities discussed nor mentions of feelings of accomplishment were different between the improving and nonimproving groups.

“Neither the breadth of activities discussed nor mentions of feelings of accomplishment were different between groups.”

May combine two claims.

Satisfaction, decisions, long-term planning, effort, and structure all showed significant differences between the improving and nonimproving groups (P<.05).

“Satisfaction, decisions, long-term planning, effort, and structure all showed significant differences between the improving and nonimproving groups (P<.05).”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Our results indicate that context is more important for some concepts than others; for example, breadth is only informative when considered alongside other markers such as emotional affect to contextualize it.

“our results indicate that context is more important for some concepts than others; for example, breadth is only informative when considered alongside other markers such as emotional affect to contextualize it.”

May combine two claims.

Both the well-established LIWC measures as well as the novel BA measures have utility in predicting longitudinal patient trajectories.

“we showed that both the well-established LIWC measures as well as the novel BA measures have utility in predicting longitudinal patient trajectories.”

May combine two claims.

Our study population was predominantly young (8014/10,718, or 74.77%, were 35 years old or younger) and, considering that they used a paid online therapy service, presumably financially stable; in addition, 2679 (2679/10,718, 25.00%) were residents of California or New York, and 8340 (8340/10,718, 77.81%) were female.

“Our study population was predominantly young (8014/10,718, or 74.77%, were 35 years old or younger) and, considering that they used a paid online therapy service, presumably financially stable; in addition, 2679 (2679/10,718, 25.00%) were residents of California or New York, and 8340 (8340/10,718, 77.81%) were female.”

May combine two claims.

Reactions are more informative when considered alongside tone than without, whereas adding tone makes comparatively little difference for activities.

“reactions are more informative when considered alongside tone than without, whereas adding tone makes comparatively little difference for activities.”

May combine two claims.

Our work here demonstrated that theme and affect analysis conducted via established, straightforward methods such as word counting both reflect and potentially predict depression status.

“Our work here demonstrated that theme and affect analysis conducted via established, straightforward methods such as word counting both reflect and potentially predict depression status.”

Classified as probably not a substantial claim.

Satisfaction, accomplishment (a sense of having accomplished something), and effort (enjoyment due to effort exerted) are similar to each other.

“Similarly, satisfaction, accomplishment (a sense of having accomplished something), and effort (enjoyment due to effort exerted) are similar to each other.”

May combine two claims.

We validated established linguistic markers of depression in a large corpus of naturally occurring language collected as part of psychotherapy sessions, presenting differences between participants with low and high PHQ-9 scores.

“we validated established linguistic markers of depression in a large corpus of naturally occurring language collected as part of psychotherapy sessions, presenting differences between participants with low and high PHQ-9 scores.”

May combine two claims.

Our metrics of the individual subconstructs of BA capture distinct dimensions of the underlying mechanisms and may lend themselves to unique clinical insights.

“we demonstrated that our metrics of the individual subconstructs of BA capture distinct dimensions of the underlying mechanisms and may lend themselves to unique clinical insights.”

May combine two claims.

Activities alone were more explanatory of improvement than reactions alone; in other words, activities are explanatory even without considering emotional tone.

“Activities alone were more explanatory of improvement than reactions alone; in other words, activities are explanatory even without considering emotional tone.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The PHQ-9 is a validated self-report questionnaire commonly used to assess depression severity, scored on a scale of 0-27.

“The PHQ-9 is a validated self-report questionnaire commonly used to assess depression severity, scored on a scale of 0-27 [4].”

• cites [4] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Engage is as effective as problem-solving therapy in treating late-life major depression, while having the advantage of being less complex.

“A recent randomized controlled trial showed that Engage is as effective as problem-solving therapy in treating late-life major depression, while having the advantage of being less complex; Engage required 30% less training time compared to problem-solving therapy [10].”

• cites [10] sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim. May combine two claims.

The behavioral theory of depression states that depressed individuals participate in fewer pleasant activities and derive less pleasure and feelings of accomplishment from such activities.

“The behavioral theory of depression states that depressed individuals participate in fewer pleasant activities and derive less pleasure and feelings of accomplishment from such activities [6]; in other words, they exhibit reduced behavioral activation (BA).”

• cites [6] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Increased first-person singular pronoun use indicates increased self-focused attention, consistent with Pyszcynski and Greenberg's integrative model of depression.

“For example, depressed individuals use more first-person singular pronouns (eg, “I,” “me,” “my”) than nondepressed individuals [14,15], indicating increased self-focused attention, a language use consistent with Pyszcynski and Greenberg’s [16] integrative model of depression.”

• cites [16] sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

The BADS Activation subscale is a validated instrument used to identify subjective engagement levels.

“We constructed a set of 66 unique representative seed terms, informed by the Activation subscale of the Behavioral Activation for Depression Scale (BADS) [27], a validated instrument used to identify subjective engagement levels.”

• cites [27] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Linguistic indicators of positive and negative affect differ in depression and have shown utility in social media-based predictive models of depression.

“Linguistic indicators of positive and negative affect differ in depression and have shown utility in social media–based predictive models of depression [14,20].”

• cites [14] sourceNot checked yet

  Not checked in the free run.

• cites [20] De ChoudhuryMGamonMCountsSHorvitzEPredicting Depression via Social Media2013Seventh International AAAI Conference on Weblogs and Social MediaJuly 8-11, 2013Cambridge, MANot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

Term sets constructed using Empath have a strong correlation with corresponding LIWC categories.

“Term sets constructed in this way have a strong correlation with the corresponding LIWC categories, which were constructed in a completely manual process.”

• cites [25] FastEChenBBernsteinMSEmpath: Understanding Topic Signals in Large-Scale Text2016CHI Conference on Human Factors in Computing SystemsMay 7-12, 2016San Jose, CA sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Prior work has found that depressed individuals use words related to sadness and somatic health concerns (health and biology words).

“These include words related to sadness, as well as words related to somatic health concerns (health and biology words, with the biology category combining body, health, sexual, and ingestion words) [22,23].”

• cites [22] sourceNot checked yet

  Not checked in the free run.

• cites [23] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Suicidal poets used fewer first-person plural pronouns than nonsuicidal poets.

“Stirman and Pennebaker [19] found that suicidal poets used fewer first-person plural pronouns than nonsuicidal poets.”

• cites [19] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

BA therapies have been shown to be as effective as learning-based therapies while being easier to understand for patients and easier to deliver for therapists.

“They aim to reduce depression symptoms by activating the reward system and have been shown to be as effective as learning-based therapies while being easier to understand for patients and easier to deliver for therapists [6].”

• cites [6] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

LIWC categories have been validated in numerous experiments.

“As reviewed by Tausczik and Pennebaker [24], numerous experiments have validated the LIWC categories.”

• cites [24] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Engage aims to systematically address disengagement from participation in pleasurable activities in a structured approach by incorporating reward exposure and addressing barriers in 3 behavioral domains: negativity bias, apathy, and emotional dysregulation.

“An example is the streamlined, evidence-based psychotherapeutic strategy called Engage [7], which aims to systematically address disengagement from participation in pleasurable activities in a structured approach by incorporating reward exposure and addressing barriers in 3 behavioral domains: negativity bias, apathy, and emotional dysregulation.”

• cites [7] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Of the 6760 patients used for trajectory analysis, 47.2% improved and 52.8% did not improve.

“Of these patients, which Hull et al [38] identified as strictly “improving” or “nonimproving,” 47.2% (3189/6760) improved (classified as Recovery or Acute Recovery), and 52.8% (3571/6760) did not improve (classified as Chronic or Elevated Chronic).”

• cites [38] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Engage required 30% less training time compared to problem-solving therapy.

“A recent randomized controlled trial showed that Engage is as effective as problem-solving therapy in treating late-life major depression, while having the advantage of being less complex; Engage required 30% less training time compared to problem-solving therapy [10].”

• cites [10] sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim. May combine two claims.

Reward exposure aiming to induce BA has been thought to reactivate and retrain reward networks and improve depression.

“For this reason, “reward exposure” aiming to induce BA has been thought to reactivate and retrain reward networks and improve depression [9].”

• cites [9] sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

BA therapies aim to reduce depression symptoms by activating the reward system.

“They aim to reduce depression symptoms by activating the reward system and have been shown to be as effective as learning-based therapies while being easier to understand for patients and easier to deliver for therapists [6].”

• cites [6] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

LIWC constructs such as leisure are related to activation but do not provide a comprehensive account of how engagement might manifest in language.

“However, while LIWC constructs such as “leisure” are related to the notion of activation, they do not provide a comprehensive account of how engagement might manifest in language.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Customized in-domain training corpora produce better word representations in clinical domains.

“Though generally harder to come by, customized in-domain training corpora are known to produce better word representations in clinical domains [26].”

• cites [26] RobertsKAssessing the Corpus Size vs. Similarity Trade-off for Word Embeddings in Clinical NLP2017Clinical Natural Language Processing WorkshopDecember 11, 2016Osaka, JapanNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

Manos et al previously showed via factor analysis that a modified version of the breadth item ("I engaged in many different activities") should be retained in a short version of the BADS.

“Manos et al [41] previously showed via factor analysis that a modified version of the breadth item (“I engaged in many different activities”) should be retained in a short version of the BADS.”

• cites [41] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Patients on the path to recovery discuss these topics less as time passes, which may indicate that they experience fewer somatic symptoms.

“That patients on the path to recovery discuss these topics less as time passes may indicate that they experience fewer somatic symptoms.”

May combine two claims.

Many patients only had 1 or 2 scores, and some participants with identical sets of scores were assigned to different groups.

“Many patients only had 1 or 2 scores, and some participants with identical sets of scores were assigned to different groups.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Counting words belonging to semantic and syntactic categories is simple yet effective, as demonstrated in countless experiments across several fields.

“Counting words belonging to semantic and syntactic categories is simple yet effective, as demonstrated in countless experiments across several fields [14,20,22,24,43,44].”

• cites [14] sourceNot checked yet

  Not checked in the free run.

• cites [20] De ChoudhuryMGamonMCountsSHorvitzEPredicting Depression via Social Media2013Seventh International AAAI Conference on Weblogs and Social MediaJuly 8-11, 2013Cambridge, MANot checked yet

  Not checked in the free run.

• cites [22] sourceNot checked yet

  Not checked in the free run.

• cites [24] sourceNot checked yet

  Not checked in the free run.

• cites [43] sourceNot checked yet

  Not checked in the free run.

• cites [44] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The shift to a more positive tone may be an effect of participating in therapy.

“which may be an effect of participating in therapy”

Classified as probably not a substantial claim.

The mechanisms underlying improvement due to BA therapies are not sufficiently well understood.

“The mechanisms underlying improvement due to BA therapies are not sufficiently well understood.”

• cites [6] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The nonimproving group's PHQ slope being different from 0 is an effect of unsupervised learning coupled with the fact that therapy and the simple passage of time (regression to the mean) is at least somewhat effective for most people.

“we believe that this is an effect of unsupervised learning (ie, the trajectories clustered into groups labeled as “chronic” are distinct from the other clusters, but not necessarily entirely without improvement) coupled with the fact that therapy and just the simple passage of time (regression to the mean) is at least somewhat effective for most people.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Some elements of the various activation approaches may be unnecessary.

“suggesting that some elements of the various activation approaches may be unnecessary [6].”

• cites [6] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

At baseline, the number of words written did not vary significantly with depression severity (P=.33).

“At baseline, the number of words written did not vary significantly with depression severity (P=.33).”

May combine two claims.

The trajectories used to categorize patients as improving or otherwise were assigned via unsupervised learning and do not directly correspond to the total change in PHQ over the course of treatment.

“The trajectories used to categorize patients as improving or otherwise were assigned via unsupervised learning and do not directly correspond to the total change in PHQ over the course of treatment.”

May combine two claims.

These metrics indeed reflect depression symptom status in this dataset, providing further strong support for their relationship with depression.

“we showed that these metrics indeed reflect depression symptom status in this dataset, providing further strong support for their relationship with depression.”

May combine two claims.

While breadth alone is not explanatory, it is informative in the context of other predictors such as emotional affect.

“while breadth alone is not explanatory, it is informative in the context of other predictors such as emotional affect.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Reliance on the PHQ-9 is a limitation because this instrument focuses exclusively on symptoms of depression.

“Reliance on the PHQ-9 is a limitation because this instrument focuses exclusively on symptoms of depression.”

• AssumptionRests on an assumption: The PHQ-9 focuses exclusively on symptoms of depression.

May combine two claims.

Discussing activities without positive feelings is not indicative of improvement; activities must also be perceived as rewarding.

“This observation may indicate that discussing activities without positive feelings is not indicative of improvement; activities must also be perceived as rewarding.”

May combine two claims.

Analyses were repeated with the excluded "gray area" trajectories included, and results, though less clear, were not different and our conclusions held.

“analyses were repeated with these included, and results, though less clear, were not different and our conclusions held”

Probably not a substantial claim: may not pass S3 (load-bearing).

Another limitation of the study is the absence of qualitative review that could examine the accuracy of our ratings.

“Another limitation of the study is the absence of qualitative review that could examine the accuracy of our ratings.”

Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing). May combine two claims.

Metrics of activation may predict longer-term changes more accurately than word use analysis, which may be confounded by the mood of the moment, and thus capture a separate and clinically meaningful dimension of symptomatology and treatment success.

“one may expect metrics of activation to predict longer-term changes more accurately than word use analysis, which may be confounded by the mood of the moment, and thus capture a separate and clinically meaningful dimension of symptomatology and treatment success.”

Classified as probably not a substantial claim.

Merely discussing feelings of satisfaction, or lack thereof, may not correspond to improvement.

“A possible explanation is that merely discussing feelings of satisfaction, or lack thereof, may not correspond to improvement.”

Classified as probably not a substantial claim.

This approach ensured that the concept represents language usage in our specific study population.

“This approach ensured that the concept represents language usage in our specific study population.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Mentions of being satisfied may only be triggered positively once the positive feedback loop is set in motion.

“Mentions of being satisfied may only be triggered positively once the positive feedback loop is set in motion.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Geographically and demographically diverse groups may significantly differ in their word choice and usage; as a result, the lexicon may not be generalizable to other groups.

“Geographically and demographically diverse groups may significantly differ in their word choice and usage; as a result, the lexicon may not be generalizable to other groups.”

• AssumptionRests on an assumption: Domain-specific texts generally work better for distributional semantics than more general corpora, but they may also result in artifacts with limited generalizability.

Classified as probably not a substantial claim.

The lexicon of words collectively representing activation developed in this work was based solely on the messages in this dataset.

“The lexicon of words collectively representing activation developed in this work was based solely on the messages in this dataset.”

Probably not a substantial claim: may not pass S3 (load-bearing).

A potential limitation is that the therapy sessions in this work did not specifically use BA therapy.

“A potential limitation is that the therapy sessions in this work did not specifically use BA therapy.”

Probably not a substantial claim: may not pass S1 (asserted), S4 (based). May combine two claims.

Discussing activities, plans for activities, or even avoidance of activities shows that a patient engages with the ideas surrounding BA and may indicate that a patient is moving towards or already part of a positive feedback loop of self-perpetuating improvements.

“Discussing activities, plans for activities, or even avoidance of activities shows that a patient engages with the ideas surrounding BA and may indicate that a patient is moving towards or already part of a positive feedback loop of self-perpetuating improvements.”

Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing), S4 (based).

An additional implication of the potentially causative mechanism of activation is that it should occur before symptom improvement.

“An additional implication of the potentially causative mechanism of activation is that it should occur before symptom improvement.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The striking gender imbalance is consistent with previously reported gender imbalances in both online and face-to-face therapy.

“This striking gender imbalance is consistent with previously reported gender imbalances in both online and face-to-face therapy.”

• cites [45] sourceNot checked yet

  Not checked in the free run.

• cites [46] sourceNot checked yet

  Not checked in the free run.

• cites [47] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S4 (based).

It requires additional information about the patient's tone to be informative of current symptom severity.

“it requires additional information about the patient’s tone to be informative of current symptom severity.”

Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

This work enables further work in automated diagnosis and assessment of depression, as well as refinement of BA psychotherapeutic strategies.

“This work therefore enables further work in automated diagnosis and assessment of depression, as well as refinement of BA psychotherapeutic strategies.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Word count approaches such as the one employed by LIWC have limitations.

“Word count approaches such as the one employed by LIWC have limitations.”

Probably not a substantial claim: may not pass S1 (asserted), S2 (checkable). May combine two claims.

While more direct measures of sentiment and mindset reflect an individual's current thoughts and feelings, "activity" topic analysis may reveal long-term dimensions of patient trajectory.

“While more direct measures of sentiment and mindset reflect an individual’s current thoughts and feelings, “activity” topic analysis may reveal long-term dimensions of patient trajectory.”

Probably not a substantial claim: may not pass S1 (asserted), S2 (checkable), S3 (load-bearing), S4 (based).

Ascertaining the timeline of changes in these different activation metrics relative to patient trajectories is a prerequisite to the translation of linguistic indicators into clinical insights.

“Ascertaining the timeline of changes in these different activation metrics relative to patient trajectories is a prerequisite to the translation of linguistic indicators into clinical insights.”