Breaking open the black box: isolating the most potent features of a web and mobile phone-based intervention for depression, anxiety, and stress

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Of 720 participants screened eligible, 242 were randomly allocated to myCompass, 11 withdrew, leaving a final sample of 231 for analysis.

“Of the 720 participants who screened eligible to participate in the myCompass RCT, 242 were randomly allocated to receive the myCompass program for seven weeks. There were eleven participants in this group that withdrew from the RCT, leaving a final sample of 231 for the current analysis.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Across 266 days, the site recorded 58,296 page views, with an average of 12.34 page views per visit and an average visiting time of 10 minutes and 37 seconds.

“Across the 266 days, the site recorded 58,296 page views, with an average of 12.34 page views per visit and an average visiting time of 10 minutes and 37 seconds.”

Probably not a substantial claim: may not pass S3 (load-bearing).

During the RCT period, there were 4724 visits to the site, of which 1182 (25.02%) were unique first-time visits.

“During the RCT period, there were a total of 4724 visits to the site, 1182/4724 (25.02%) of which were unique (ie, first time) visits.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Of those who used tracking reminders, 29 received at least one reminder via email, 97 via SMS, and 8 via both modalities.

“Of those who used tracking reminders, 29 received at least one reminder via email, 97 received at least one reminder via SMS, and 8 subjects received reminders via both modalities.”

May combine two claims.

The total number of tracking reminders was positively correlated with the number of program log-ins and with the frequency of symptom tracking.

“The total number of tracking reminders was positively correlated with the number of program log-ins, as well as the frequency of symptom tracking (Table 4).”

May combine two claims.

After controlling for baseline severity, frequency of tracking reminders received via email (r=-.27, P=.28) and via SMS (r=-.12, P=.37) were both associated with decreased functional impairment at post intervention, although neither correlation was significant when examined individually.

“After controlling for baseline severity, frequency of tracking reminders received via email (r=-.27, P=.28), and via SMS (r=-.12, P=.37) were both associated with decreased functional impairment at post intervention, although neither correlation was significant when examined individually.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The least commonly used feature was the diary, used by 61 of 231 participants (26.4%).

“The least commonly used feature was the diary, used by 61/231 (26.4%) participants.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Females chose to receive more snippets than males (t155=2.15, P=.03).

“Females chose to receive more “snippets” than males t 155 = 2.15, P=.03”

May combine two claims.

The mean number of modules started was 1.2 (SD 1.6) and the mean number completed was 0.6 (SD 1.3).

“The mean number of modules started was 1.2 (SD 1.6) and the mean number completed was 0.6 (SD 1.3).”

May combine two claims.

Among those who used the program, log-in frequency (r=.19, P=.02) and total instances of symptom tracking (r=.18, P=.03) increased with participant age.

“for those who did use the program, log-in frequency (r=.19, P=.02) and total instances of symptom tracking (r=.18, P=.03) increased with participant age.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Of those who used snippets, 153 received at least one via email and 15 received at least one via SMS.

“Of those who used snippets, 153 received at least one via email and 15 received at least one via SMS.”

Probably not a substantial claim: may not pass S3 (load-bearing).

118 of 231 participants (51.1%) used the optional tracking reminder feature.

“and 118/231 (51.1%) also used the optional tracking reminder feature.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The second most commonly used feature was the symptom tracking function.

“The second most commonly used feature was the symptom tracking function.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Those who used myCompass logged in a total of 2463 times, with a mean of 14.9 log-ins per user (SD 16.5), ranging from 1 to 105.

“Those who did use myCompass logged in a total of 2463 times, with the mean number of log-ins per user being 14.9 (SD 16.5), ranging from 1 to 105.”

May combine two claims.

118 of 231 participants (51.1%) used the modules, and 57 of those completed at least one module.

“There were 118/231 (51.1%) participants who also used the modules, 57 of whom completed at least one module.”

May combine two claims.

A greater number of tracking reminders received via email was associated with increased use and completion of modules and with use of the diary.

“A greater number of tracking reminders received via email were also associated with increased use and completion of the modules, as well as use of the diary.”

May combine two claims.

Neither initiation nor completion of the psychotherapy modules was associated with treatment outcome.

“Interestingly, neither initiation nor completion of the psychotherapy modules was associated with treatment outcome.”

May combine two claims.

Sex, education, and age were not found to be strong predictors of program use.

“Although sex, education, and age were not found to be strong predictors of program use, it is possible that rates of program engagement, or use of certain features, may differ in other populations.”

May combine two claims.

The sample used in the current study was predominantly female (160/231, 69.3%), university-educated (135/231, 58.4%), and around 40 years of age.

“Finally, the sample used in the current study was predominantly female (160/231, 69.3%), university-educated (135/231, 58.4%), and around 40 years of age.”

Probably not a substantial claim: may not pass S3 (load-bearing).

In a study of a personally controlled health management record, increased use of a diary feature, an Internet poll feature, and an Internet appointment booking feature correlated with increased help-seeking for emotional well-being and overall health service utilization.

“Similarly, in a study examining the relationship between use of features of a personally controlled health management record and help-seeking behaviors, Lau et al [9] found that increased use of a diary feature, an Internet poll feature, and an Internet appointment booking feature, correlated with increased help-seeking for emotional well-being as well as overall health service utilization.”

• cites [9] sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

As eMental Health interventions become more sophisticated, multifaceted, and user-driven, heterogeneity in patterns of usage and the number of treatment components increases.

“As these interventions become increasingly more sophisticated, multifaceted, and user-driven, the heterogeneity in patterns of usage, and subsequently, the number of treatment components, also increases.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

In a trial of a Web-based intervention for diabetes self-management, use of a self-monitoring feature was associated with improvements in healthy eating, reductions in dietary fat, and increased exercise, but not with medication compliance.

“For example, in a trial of a Web-based intervention for diabetes self-management, use of a self-monitoring feature was associated with improvements in healthy eating, reductions in dietary fat, and increased exercise, but not with medication compliance [8].”

• cites [8] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Use of reminder emails predicts treatment outcome.

“and use of reminder emails [13].”

• cites [13] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

The DASS-21 has adequate internal consistency and test-retest reliability.

“The DASS-21 has adequate internal consistency and test-retest reliability [17]”

• cites [17] Lovibond, PF, 1995, Behav Res Ther sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The WSAS has sound psychometric properties when administered on the Internet.

“The WSAS has been shown to have sound psychometric properties when administered on the Internet [20]”

• cites [20] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The DASS-21 has sound psychometric properties when administered in an Internet format.

“and has sound psychometric properties when administered in an Internet format [18].”

• cites [18] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

When used correctly, Internet-delivered mental health (eMental Health) interventions produce treatment effects similar to those observed in face-to-face treatment.

“When used correctly, Internet-delivered mental health (eMental Health) interventions produce treatment effects similar to those observed in face-to-face treatment [1-4].”

• cites [1] sourceNot checked yet

  Not checked in the free run.

• cites [2] sourceNot checked yet

  Not checked in the free run.

• cites [3] sourceNot checked yet

  Not checked in the free run.

• cites [4] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.

The appeal of the "snippets" may lie in the fact that they are supportive, normalizing, and instill hope, yet demand little of the user in terms of behavior change, attention, or time.

“The appeal of the “snippets” may lie in the fact that they are supportive, normalizing, and instill hope, yet demand little of the user in terms of behavior change, attention, or time.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Frequency of program use decreasing over the course of the intervention is common in nontherapist-assisted eMental Health interventions.

“However, as is common in nontherapist-assisted eMental Health interventions [22,25,26], frequency of program use decreased over the course of the intervention, emphasizing the need to identify program features that promote more regular program engagement.”

• cites [22] sourceNot checked yet

  Not checked in the free run.

• cites [25] sourceNot checked yet

  Not checked in the free run.

• cites [26] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Research suggests that a tunnelled format may facilitate greater program engagement (e.g., greater number of website pages visited, greater time spent on the website, and greater knowledge gained from the website).

“Although research suggests that a tunnelled format may facilitate greater program engagement (eg, greater number of website pages visited, greater time spent on the website, and greater knowledge gained from the website) [27], subjects were often required to complete the between-session homework tasks before being able to progress to the next session within a module.”

• cites [27] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

The lack of difference between the email and SMS delivery modes is possibly due to a high degree of overlap between the two, as 10 of the 15 users who received snippets via SMS also received snippets via email.

“The lack of difference between the email and SMS delivery modes is possibly due to a high degree of overlap between the two, as 10 of the 15 users who received snippets via SMS also received snippets via email.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Users chose to track symptom dimensions most closely tied to the primary goal of the study, which was to reduce symptoms of depression, anxiety, and stress.

“Among the most common symptom dimensions tracked were stress, worry, anxiety, motivation, and depression, indicating that users chose to track symptom dimensions most closely tied to the primary goal of the study, which was to reduce symptoms of depression, anxiety, and stress.”

May combine two claims.

Although there was evidence of a stronger association between symptom improvement and use of alerts via the mobile phone platform, the degree of overlap between use of email and SMS alerts may have precluded identification of the alert delivery modalities most strongly associated with symptom reduction.

“Although there was evidence of a stronger association between symptom improvement and use of alerts via the mobile phone platform, the degree of overlap between use of email and SMS alerts may have precluded identification of alert delivery modalities that were most strongly associated with symptom reduction.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The finding that the majority chose SMS reminders may represent the perception that reminders are likely to be received in a more timely fashion if delivered to the mobile phone, given that mobile phones are generally carried on the person.

“Given that mobile phones are generally carried on the person, this finding may represent the perception that reminders are likely to be received in a more timely fashion if delivered to the mobile phone.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Low module completion rates may also have arisen because the module sessions were delivered in a tunnelled fashion, whereby subjects were required to complete the module sessions in a predetermined sequence.

“Low module completion rates may also have arisen because the module sessions were delivered in a tunnelled fashion, whereby subjects were required to complete the module sessions in a predetermined sequence.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

In an Internet intervention for smoking cessation, users who received email reminders made 1.2 times as many visits, viewed 1.58 times as many pages, and spent 1.51 times as many minutes compared to users who did not receive email reminders.

“In particular, one study showed that users who received email reminders made 1.2 times as many visits, viewed 1.58 times as many pages, and spent 1.51 times as many minutes in an Internet intervention for smoking cessation [13], compared to users who did not receive email reminders.”

• cites [13] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The average number of log-ins per user (14.9) is somewhat higher than that reported in other eMental Health interventions (e.g., 6.55 log-ins for highest quartile in an Internet intervention for depression).

“The average number of log-ins per user over the course of the intervention was 14.9, which is somewhat higher than that reported in other eMental Health interventions (eg, a mean of 6.55 log-ins for individuals in the highest quartile of usage in an Internet intervention for depression) [10].”

• cites [10] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Alerts may be an especially potent component of eMental Health interventions, both via their association with enhanced program usage and independently.

“These findings indicate that alerts may be an especially potent component of eMental Health interventions, both via their association with enhanced program usage, as well as independently.”

Probably not a substantial claim: may not pass S2 (checkable).

Program utilization may be enhanced (particularly use of features that involve alerts) via incorporating a mobile phone platform into future eMental Health interventions.

“It also suggests that program utilization may be enhanced (particularly use of features that involve alerts) via incorporating a mobile phone platform into future eMental Health interventions.”

Probably not a substantial claim: may not pass S3 (load-bearing).

A lower-than-expected rate of module completion is not an issue that is unique to our intervention.

“However, a lower-than-expected rate of module completion is not an issue that is unique to our intervention.”

• AssumptionRests on an assumption: Many self-paced eMental Health interventions have low rates of module completion.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

Alerts may also be useful for encouraging more ongoing program engagement in users who show a tendency to prematurely disengage from the program once they have begun to achieve symptom remission ("e-attainers").

“Alerts may also be useful for encouraging more ongoing program engagement in users who show a tendency to prematurely disengage from the program once they have begun to achieve symptom remission (“e-attainers”).”

Probably not a substantial claim: may not pass S3 (load-bearing).

It is possible that rates of module use only within a specific period of the intervention (ie, in the first few weeks) may be more strongly associated with therapeutic gain than total module completion rate.

“Indeed, it is possible that rates of module use only within a specific period of the intervention (ie, in the first few weeks) may be more strongly associated with therapeutic gain than total module completion rate.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

It is possible that rates of program usage may be lower in individuals with poorer literacy, or indeed, those who present with more severe forms of psychopathology.

“It is possible that rates of program usage may be lower in individuals with poorer literacy, or indeed, those who present with more severe forms of psychopathology.”

May combine two claims.

Receiving regular reminders, motivational messages, and tips may lead to reductions in symptomatology and functional impairment because they regularly and consistently cue the expectation of symptom improvement, create a sense of being continuously supported and encouraged, and remind the user that they are actively taking steps to gain control over their symptoms.

“Therefore, receiving regular reminders, motivational messages, and tips may lead to reductions in symptomatology and functional impairment because they regularly and consistently cue the expectation of symptom improvement, create a sense of being continuously supported and encouraged, and remind the user that they are actively taking steps to gain control over their symptoms.”

Probably not a substantial claim: may not pass S3 (load-bearing).

In a study examining the association between usage metrics and treatment outcome in an RCT of an Internet depression treatment program, Donkin et al [12] found that the proportion of modules completed was not associated with treatment outcome.

“In a study examining the association between usage metrics and treatment outcome in an RCT of an Internet depression treatment program, Donkin et al [12] found that the proportion of modules completed was not associated with treatment outcome.”

• cites [12] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Alerts act as prompts that enhance program engagement, and so may help to combat the high rates of nonusage attrition common to many Internet interventions.

“First, in terms of program usage, alerts act as prompts that enhance program engagement, and so may help to combat the high rates of nonusage attrition common to many Internet interventions [31,32].”

• cites [31] sourceNot checked yet

  Not checked in the free run.

• cites [32] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

The greater association between email reminders and treatment outcome may be because users who chose to receive alerts via email spend more time on their computer, and as a result, could more easily access the program features that were only available via the computer platform (such as the modules).

“Furthermore, the greater association between email reminders and treatment outcome may be because users who chose to receive alerts via email spend more time on their computer, and as a result, could more easily access the program features that were only available via the computer platform (such as the modules).”

Probably not a substantial claim: may not pass S3 (load-bearing).

Ours is not the first study to find a lack of association between module completion and treatment outcome.

“However, ours is not the first study to find a lack of association between module completion and treatment outcome.”

• cites [12] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Cognitive models of depression posit that maladaptive responses to negative automatic thoughts (such as rumination and decreased engagement in pleasurable activities) are a primary factor that maintains perceptions of low self-worth and associated depressive symptomatology.

“For example, cognitive models of depression posit that maladaptive responses to negative automatic thoughts (such as rumination and decreased engagement in pleasurable activities) are a primary factor that maintains perceptions of low self-worth and associated depressive symptomatology [34].”

• cites [34] Beck, AT, 2002, Clinical advances in cognitive psychotherapy: Theory and applicationNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

It is possible that rates of program engagement, or use of certain features, may differ in other populations.

“Although sex, education, and age were not found to be strong predictors of program use, it is possible that rates of program engagement, or use of certain features, may differ in other populations.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The alert-based functions, such as the "snippets", may provide a means of buffering against or breaking the negative feedback cycles that contribute to the maintenance of many psychiatric conditions.

“Finally, the alert-based functions, such as the “snippets”, may provide a means of buffering against or breaking the negative feedback cycles that contribute to the maintenance of many psychiatric conditions.”

May combine two claims.

Alerts may serve multiple therapeutic functions and may therefore be an especially potent feature in eMental Health interventions.

“This finding indicates that alerts may serve multiple therapeutic functions and may therefore be an especially potent feature in eMental Health interventions.”

Probably not a substantial claim: may not pass S3 (load-bearing).

It may be that many users prefer symptom tracking to completion of more in-depth modules.

“Studies suggest that individuals may derive substantial benefit from symptom tracking alone [35], and it may be that many users prefer this to completion of more in-depth modules.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Self-monitoring is one of the most important components of psychotherapy for anxiety and other mental illnesses, and is a key step in helping individuals to change unhelpful cognitions, beliefs, and behaviors.

“In one respect, this is unsurprising, as self-monitoring is one of the most important components of psychotherapy for anxiety and other mental illnesses [29], and is a key step in helping individuals to change unhelpful cognitions, beliefs, and behaviors [30].”

• cites [29] sourceNot checked yet

  Not checked in the free run.

• cites [30] Beck, AT, 1997, J Psychother Pract ResNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing), S4 (based).

Another possibility is that user-specific factors, such as the degree of therapeutic alliance with the program, may contribute more strongly to symptom improvement than the actual skills taught in the modules.

“Another possibility is that user-specific factors, such as the degree of therapeutic alliance with the program, may contribute more strongly to symptom improvement than the actual skills taught in the modules.”