CoChat
Sign Up
Cited by 9Digital Mental Health InterventionsWorkplace Health and Well-beingMindfulness and Compassion Interventions

Increasing engagement with, and effectiveness of, an online CBT-based stress management intervention for employees through the use of an online facilitated bulletin board: study protocol for a pilot randomised controlled trial

Open access
Carolan, Stephany, Harris, Peter R., Greenwood, Kathryn, Cavanagh, Kate·2016·Trials
DOI 10.1186/s13063-016-1733-2PMC PMC5159960
Open this paper in CoChat — free accountOpen PDF

Abstract

BACKGROUND: The evidence for the benefits of online cognitive behaviour therapy (CBT)-based programmes delivered in a clinical context is clear, but this evidence does not translate to online CBT-based stress management programmes delivered within a workplace context. One of the challenges to the delivery of online interventions is programme engagement; this challenge is even more acute for interventions delivered in real-world settings such as the workplace. The purpose of this pilot study is to explore the effect of an online facilitated discussion group on engagement, and to estimate the potential effectiveness of an online CBT-based stress management programme. METHODS: This study is a three-arm randomised controlled trial (RCT) comparing a minimally guided, online, CBT-based stress management intervention delivered with and without an online facilitated bulletin board, and a wait list control group. Up to 90 employees from six UK-based organisations will be recruited to the study. Inclusion criteria will include age 18 years or over, elevated levels of stress (as measured on the PSS-10 scale), access to a computer or a tablet and the Internet. The primary outcome measure will be engagement, as defined by the number of logins to the site; secondary outcome measures will include further measures of engagement (the number of pages visited, the number of modules completed and self-report engagement) and measures of effectiveness (psychological distress and subjective wellbeing). Possible moderators will include measures of intervention quality (satisfaction, acceptability, credibility, system usability), time pressure, goal conflict, levels of distress at baseline and job autonomy. Measures will be taken at baseline, 2 weeks (credibility and expectancy measures only), 8 weeks (completion of intervention) and 16 weeks (follow-up). Primary analysis will be conducted on intention-to-treat principles. DISCUSSION: To our knowledge this is the first study to explore the effect of an online discussion group on the engagement and effectiveness of an online CBT-based stress management intervention. This study could provide a solution to the growing problem of poor employee psychological health and begin to address the challenge of increasing engagement with Internet-delivered health interventions. TRIAL REGISTRATION: ClinicalTrials.gov Identifier: NCT02729987 . Registered on 18 Mar 2016.

AI snapshot

Written from the paper’s full text.

TL;DR

This is a protocol for a three-arm pilot randomised controlled trial testing whether adding an online facilitated bulletin board increases engagement with, and effectiveness of, an online CBT-based workplace stress management programme (WorkGuru). It aims to recruit up to 90 employees with elevated stress from six UK organisations and will compare the intervention with and without the bulletin board against a waitlist control.

Key findings

  • No outcome results are reported; the paper describes a planned pilot RCT. The primary outcome is engagement, defined as the number of logins to the site; secondary engagement measures include number of pages visited, number of modules completed, and self-report engagement.
  • Effectiveness measures are psychological distress (DASS-21) and subjective wellbeing at work (IWP Multi-affect Indicator). The authors hypothesise that the bulletin board group will have better engagement than the minimal support group, leading to decreased psychological distress and increased subjective wellbeing at work.
  • Planned moderators include intervention quality (satisfaction, acceptability, credibility, system usability), time pressure, goal conflict, baseline distress, and job autonomy.
  • Pilot objectives are to assess recruitment rate, study attrition, robustness of engagement measures, provide an effect size prediction, refine threshold levels of adherence, and identify challenges of conducting research and delivering an online intervention in the workplace.

Methods

  • Three-arm RCT comparing a minimally guided online CBT-based stress management intervention (WorkGuru) delivered with an online facilitated bulletin board, the same intervention without the bulletin board (minimal support group, MSG), and a waitlist control group. All participants have unrestricted access to care as usual; randomisation is individual-level, 1:1:1, with researchers blind to allocation. Assessments occur at baseline, 2 weeks (credibility/expectancy only), 8 weeks (intervention completion), and 16 weeks (follow-up).
  • Up to 90 employees from six UK-based organisations will be recruited, with 30 participants per arm. Inclusion criteria: age 18 years or over, employed by a participating organisation, willingness to engage with an online CBT-based stress management intervention, access to a computer or tablet, access to the Internet, and a score of ≥20 on the PSS-10. No exclusion criteria are set. WorkGuru has 10 modules (7 core), coach contact three times over 8 weeks, and the bulletin board runs for 8 weeks in groups of 30.
  • Primary and secondary analyses will be conducted on intention-to-treat principles. Missing data will be imputed using Last Observation Carried Forward, and group mean for engagement measures where no previous data are available. Sensitivity analysis will include per-protocol analysis, defined as three or more logins. Analysis will use descriptive statistics, 95% confidence intervals, Cohen’s d, odds ratios where appropriate, and exploratory t tests, ANOVAs, and correlations; baseline differences will be explored with chi-square tests and ANOVA.

Limitations

  • The paper states that this is a pilot study and is underpowered, so it cannot allow definitive conclusions; it is intended to provide parameters for a definitive trial.
  • The authors state that the quantitative nature of the study does not allow exploration of why participants may or may not engage; a follow-up qualitative study is planned. They also note that participants cannot be blinded to intervention type, which could bias self-reported measures, although including two active groups may limit this bias.
  • The authors state that the efficacy of the specific online intervention (WorkGuru) has not been established, and that predicting effect size is difficult because a stress management intervention recruiting participants with elevated but not necessarily clinically significant stress is likely to report a smaller effect size than a clinical intervention. They also note potential workplace recruitment and attrition challenges, including fear of demonstrating vulnerability and time pressure.
  • Evident from the text: as a study protocol, no outcome data are available to support the hypothesised effects of the bulletin board on engagement or effectiveness.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1186/s13063-016-1733-2

Cites

61 of 61 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

0 of 55 cited sources checked · 0 supported · 0 flagged

  • Premisecitation

    As few as 1% of registered users complete all sessions of a freely available online CBT programme for people with panic disorder and agoraphobia.

    “with as few as 1% of registered users completing all sessions of a freely available online CBT programme for people with panic disorder and agoraphobia [22]”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [22] Farvolden P, Denisoff E, Selby P, Bagby RM, Rudy L. Usage and longitudinal effectiveness of a web-based self-help cognitive behavioral therapy program for panic disorder. J Med Internet Res. 2005;7(1):e7. source

      Not checked in the free run.

  • Premisecitation

    For Internet interventions delivered in real-world settings, challenges of adherence, engagement and attrition can be even more acute than in clinical research settings.

    “For Internet interventions delivered in real-world settings (as opposed to clinical research settings), these challenges can be even more acute [20, 21]”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [20] Christensen H, Griffiths KM, Korten AE, Brittliffe K, Groves C. A comparison of changes in anxiety and depression symptoms of spontaneous users and trial participants of a cognitive behavior therapy website. J Med Internet Res. 2004;6(4):e46. http://doi.org/ 10.2196/jmir.6.4.e46 . source

      Not checked in the free run.

    • cites [21] Gilbody S, Littlewood E, Hewitt C, Brierley G, Tharmanathan P, Barkham M, et al. Computerised cognitive behaviour therapy (cCBT) as treatment for depression in primary care (REEACT trial): large scale pragmatic randomised controlled trial. BMJ. 2015;351:h5627. doi: 10.1136/bmj.h5627 . source

      Not checked in the free run.

  • Study design and conductreasoning

    Randomisation at individual level rather than organisation or team level allows control for group stressors such as large-scale redundancies and team deadlines.

    “Randomisation is being conducted at an individual level rather than at organisation or team level. This allows us to control for group stressors such as large-scale redundancies and team deadlines.”
    Quote in the paperSubstantial claim
  • Premisecitation

    Evidence for online psychological interventions delivered in workplace settings is less convincing than evidence from clinical or community settings.

    “but this evidence does not translate to online CBT-based stress management programmes delivered within a workplace context”
    • Also stated in: § Background · plain
      “but the evidence for online psychological interventions delivered in workplace settings is less convincing [13–15]”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [13] Geraedts AS, Kleiboer AM, Twisk J, Wiezer NM, van Mechelen W, Cuijpers P. Long-term results of a web-based guided self-help intervention for employees with depressive symptoms: randomized controlled trial. J Med Internet Res. 2014;16(7):e168. http://doi.org/ 10.2196/jmir.3539 . source

      Not checked in the free run.

    • cites [14] Grime PR. Computerized cognitive behavioural therapy at work: a randomized controlled trial in employees with recent stress related absenteeism. Occup Med. 2004;54:353–9. source

      Not checked in the free run.

    • cites [15] Philips R, Schneider J, Molosankwe I, Leese M, Foroushani PS, Grime P, Thornicroft G. Randomized controlled trial of computerized cognitive behavioural therapy for depressive symptoms: effectiveness and costs of a workplace intervention. Psychol Med. 2014;44(4):741–52. source

      Not checked in the free run.

  • Premisecitation

    Evidence for the benefits of online CBT-based programmes delivered in clinical or community settings for depression and anxiety is established.

    “The evidence for the benefits of online cognitive behaviour therapy (CBT)-based programmes delivered in a clinical context is clear”
    • Also stated in: § Background · plain
      “A large number of meta-analyses have found evidence for the delivery of online CBT-based programmes delivered in clinical or community settings for individuals with depression and anxiety [8–12]”
    Quote in the paperSubstantial claim0 of 5 sources checked
    • cites [8] Andersson G, Cuijpers P. Internet-based and other computerized psychological treatments for adult depression: a meta-analysis. Cogn Behav Ther. 2009;38(4):196–205. http://doi.org/ 10.1080/16506070903318960 . source

      Not checked in the free run.

    • cites [9] Andrews G, Cuijpers P, Craske MG, McEvoy P, Titov N. Computer therapy for the anxiety and depressive disorders is effective, acceptable and practical health care: a meta-analysis. PLoS One. 2010;5(10):e13196. http://doi.org/ 10.1371/journal.pone.0013196 . source

      Not checked in the free run.

    • cites [10] Culjpers P, Marks IM, van Straten A, Cavanagh K, Gega L, Andersson G. Computer‐aided psychotherapy for anxiety disorders: a meta‐analytic review. Cogn Behav Ther. 2009;38(2):66–82. source

      Not checked in the free run.

    • cites [11] Richards D, Richardson T. Computer-based psychological treatments for depression: a systematic review and meta-analysis. Clin Psychol Rev. 2012;32(4):329–42. http://doi.org/ 10.1016/j.cpr.2012.02.004 . source

      Not checked in the free run.

    • cites [12] Spek V, Cuijpers P, Nyklícek I, Riper H. Internet-based cognitive behaviour therapy for symptoms of depression and anxiety: a meta-analysis. Psychol Med. 2007;37:319–28. source

      Not checked in the free run.

  • Classified as probably not a substantial claim. 35
  • May combine two claims. 20

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    DASS-21 has a Cronbach's alpha of .93.

    “In a review of the scale Henry and Crawford [40] reported a Cronbach’s alpha of .93.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [40] Henry JD, Crawford JR. The short-form version of the Depression Anxiety Stress Scales (DASS-21): construct validity and normative data in a large non-clinical sample. Br J Clin Psychol. 2005;44(Pt 2):227–39. http://doi.org/ 10.1348/014466505X29657 . source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Other studies included discussion groups as part of an online intervention but failed to include the groups as a unique research variable, so could not identify the impact of the group on intervention effectiveness.

    “A number of other studies [28–30] have also included discussion groups delivered in the form of bulletin boards as part of an online intervention but have failed to include the groups as a unique research variable so have been unable to identify the impact of the group on the effectiveness of the intervention.”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [28] Andersson G, Bergstrom J, Hollandare F, Carlbring P, Kaldo V, Ekselius L. Internet-based self-help for depression: randomised controlled trial. Br J Psychiatry. 2005;187(5):456–61. http://doi.org/ 10.1192/bjp.187.5.456 . source

      Not checked in the free run.

    • cites [29] Berger T, Caspar F, Richardson R, Kneubuhler B, Sutter D, Andersson G. Internet-based treatment of social phobia: a randomized controlled trial comparing unguided with two types of guided self-help. Behav Res Ther. 2011;49(3):158–69. http://doi.org/ 10.1016/j.brat.2010.12.007 . source

      Not checked in the free run.

    • cites [30] El Alaoui S, Ljótsson B, Hedman E, Kaldo V, Andersson E, Rück C, Lindefors N. Predictors of symptomatic change and adherence in Internet-based cognitive behaviour therapy for social anxiety disorder in routine psychiatric care. PLoS One. 2015;10(4):e0124258. http://doi.org/ 10.1371/journal.pone.0124258 . source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    PSS-10 psychometric properties are superior to PSS-14 and PSS-10 is recommended for use in practice and research.

    “The author concluded that the psychometric properties of PSS-10 are superior to those of PSS-14, and recommend that PSS-10 should be used both in practice and research.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [37] Eun-Hyun L. Review of the psychometric evidence of the Perceived Stress Scale. Asian Nurs Res. 2012;6(4):121–7. http://doi.org/ 10.1016/j.anr.2012.08.004 . source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    Primary analysis will be conducted on intention-to-treat principles.

    “Primary analysis will be conducted on intention-to-treat principles.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Study design and conductcitation

    In a review, Eun-Hyun reported a Cronbach's alpha and a test-retest reliability of PSS-10 greater than .70.

    “In a review of the scale Eun-Hyun [37] reported a Cronbach’s alpha and a test-retest reliability of > .70.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [37] Eun-Hyun L. Review of the psychometric evidence of the Perceived Stress Scale. Asian Nurs Res. 2012;6(4):121–7. http://doi.org/ 10.1016/j.anr.2012.08.004 . source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The allocation schedule will be created using a computer-generated randomisation sequence (random.org).

    “The first author will create an allocation schedule using a computer-generated randomisation sequence (random.org).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductcitation

    The credibility/expectancy questionnaire has high internal consistency (α > 0.84) and good test-retest reliability (α = 0.82 for expectancy and α = 0.75 for credibility).

    “The authors reported high internal consistency (α > 0.84) and good test-retest reliability (α = 0.82 for expectancy and α = 0.75 for credibility).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [45] Devilly GJ, Borkovec TD. Psychometric properties of the credibility/expectancy questionnaire. J Behav Ther Exp Psychiatry. 2000;31(2):73–86. http://doi.org/ 10.1016/S0005-7916(00)00012-4 . source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    The extent of contamination between the study groups will be monitored.

    “The extent of contamination between the study groups will be monitored.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    IWP Multi-affect Indicator alpha coefficients range from .75 to .90.

    “Alpha coefficients for this scale range from .75 to .90 [42].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [42] Warr P, Bindl UK, Parker SK, Inceoglu I. Four-quadrant investigation of job-related affects and behaviours. Eur J Work Organ Psy. 2014;23(3):342–63. source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    An independent researcher not otherwise involved in the research will allocate each group as an active condition or as the wait list control.

    “An independent researcher not otherwise involved in the research will allocate each group (A, B or C) as an active condition (with or without a facilitated bulletin board) or as the WLC.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductnone given

    The trial will be a three-arm randomised controlled trial comparing a minimally guided online CBT-based stress management intervention delivered with and without an online facilitated bulletin board, and a wait list control group.

    “A three-arm randomised controlled trial (RCT) will be conducted to compare engagement and effectiveness of a minimally guided, online, CBT-based stress management intervention (WorkGuru) delivered with and without an online facilitated bulletin board. Both active conditions will be compared with a wait list control group (WLC).”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    The trial will be conducted and reported in line with CONSORT 2010 guidelines.

    “The trial will be conducted and reported in line with Consolidated Standards of Reporting Trials (CONSORT) 2010 guidelines [31].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [31] Schulz KF, Altman DG, Moher D, for the CONSORT Group. CONSORT 2010. Statement: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340(mar23 1):c332. http://doi.org/ 10.1136/bmj.c332 . source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductcitation

    The Online Support Group Questionnaire has Cronbach alphas of .84 for Support, .77 for Relevance, and .82 for Comfort.

    “The authors report Cronbach alphas of the Support, Relevance and Comfort subscales as α = 0.84, α = 0.77 and α = 0.82, respectively.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [46] Chang T, Yeh CJ, Krumboltz JD. Process and outcome evaluation of an on-line support group for Asian American male college students. J Couns Psychol. 2001;48(3):319–29. http://doi.org/ 10.1037//0022-0167.48.3.319 . source

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductnone given

    All participants will have unrestricted access to Care as Usual, which will be monitored to control for potential confounding effects.

    “All participants will have unrestricted access to Care as Usual (CAU), such as counselling and medication, which will be monitored to control for potential confounding effects.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    Participants will be randomly allocated on a ratio of 1:1:1 to the groups.

    “Participants will be randomly allocated on a ratio of 1:1:1 to these groups.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    DASS-21 was described as moderately sensitive to change in a clinical sample of depressed subjects.

    “The scale was described as moderately sensitive to change in a clinical sample of depressed subjects [41].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [41] Page AC, Hooke GR, Morrison DL. Psychometric properties of the Depression Anxiety Stress Scales (DASS) in depressed clinical samples. Br J Clin Psychol. 2007;46(pt3):283–97. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    Participants using the bulletin board will be required to use a pseudonym to maintain researcher blindness.

    “Participants using the bulletin board will be required to use a pseudonym to maintain researcher blindness.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The study researchers will be blind to group allocation.

    “The study researchers will be blind to the group allocation.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductcitation

    The sample size of 30 participants per arm is based on the optimum number of discussion group participants identified by WorkGuru and is equal to the medium per arm sample size identified in an audit of sample sizes for pilot and feasibility studies.

    “The sample size of 30 participants per arm is based on the optimum number of discussion group participants identified by WorkGuru, and is equal to the medium per arm sample size identified in an audit of sample sizes for pilot and feasibility studies [32].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [32] Billingham SA, Whitehead AL, Julious SA. An audit of sample sizes for pilot and feasibility trials being undertaken in the United Kingdom registered in the United Kingdom Clinical Research Network database. BMC Med Res Methodol. 2013;13:104. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    The Client Satisfaction Questionnaire has a high degree of internal consistency (α = 0.93).

    “It has a high degree of internal consistency (α = 0.93; [43]).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [43] Larsen DL, Attkisson CC, Hargreaves WA, Nguyen TD. Assessment of client/patient satisfaction: development of a general scale. Eval Program Plann. 1979;2(3):197–207. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Increasing guidance from a therapist can lead to greater adherence to online interventions and improved outcomes.

    “Evidence suggests that increasing guidance from a therapist can lead to greater adherence to online interventions, and result in improved outcomes [8, 12, 16, 23–26]”
    Quote in the paperSubstantial claim0 of 7 sources checked
    • cites [8] Andersson G, Cuijpers P. Internet-based and other computerized psychological treatments for adult depression: a meta-analysis. Cogn Behav Ther. 2009;38(4):196–205. http://doi.org/ 10.1080/16506070903318960 . source

      Not checked in the free run.

    • cites [12] Spek V, Cuijpers P, Nyklícek I, Riper H. Internet-based cognitive behaviour therapy for symptoms of depression and anxiety: a meta-analysis. Psychol Med. 2007;37:319–28. source

      Not checked in the free run.

    • cites [16] Kelders SM, Kok RN, Ossebaard HC, Van Gemert-Pijnen JE. Persuasive system design does matter: a systematic review of adherence to web-based interventions. J Med Internet Res. 2012;14(6):e152. http://doi.org/ 10.2196/jmir.2104 . source

      Not checked in the free run.

    • cites [23] Brouwer W, Kroeze W, Crutzen R, de Nooijer J, de Vries NK, Brug J, Oenema A. Which intervention characteristics are related to more exposure to Internet-delivered healthy lifestyle promotion interventions? A systematic review. J Med Internet Res. 2011;13(1):e2. http://doi.org/ 10.2196/jmir.1639 . source

      Not checked in the free run.

    • cites [24] Hilvert-Bruce Z, Rossouw PJ, Wong N, Sunderland M, Andrews G. Adherence as a determinant of effectiveness of Internet cognitive behavioural therapy for anxiety and depressive disorders. Behav Res Ther. 2012;50(7–8):463–8. http://doi.org/ 10.1016/j.brat.2012.04.001 . source

      Not checked in the free run.

    • cites [25] Mohr DC, Cuijpers P, Lehman K. Supportive accountability: a model for providing human support to enhance adherence to eHealth interventions. J Med Internet Res. 2011;13(1):e30. source

      Not checked in the free run.

    • cites [26] Palmqvist B, Carlbring P, Andersson G. Internet-delivered treatments with or without therapist input: does the therapist factor have implications for efficacy and cost? Expert Rev Pharmacoecon Outcomes Res. 2007;7(3):291–7. http://doi.org/ 10.1586/14737167.7.3.291 . source

      Not checked in the free run.

  • May combine two claims.

    Study design and conductcitation

    PSS-10 is a widely used and validated scale designed to measure the extent to which individuals perceive aspects of their life in the last month as uncontrollable, unpredictable and overloading.

    “PSS-10 was chosen because it was felt that stress was something that employees could readily relate to (as opposed to anxiety or depression), and it is a widely used and validated scale that has been designed to measure the extent to which individuals perceive aspects of their life in that last month as being uncontrollable, unpredictable and overloading.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [37] Eun-Hyun L. Review of the psychometric evidence of the Perceived Stress Scale. Asian Nurs Res. 2012;6(4):121–7. http://doi.org/ 10.1016/j.anr.2012.08.004 . source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductcitation

    Job autonomy is measured using the nine-item autonomy subscale of the 77-item Work Design Questionnaire, which has good internal consistency (α > 0.85).

    “Job autonomy is measured using the nine-item autonomy subscale of the more comprehensive 77-item Work Design Questionnaire [50]. The authors report good internal consistency for the autonomy subscale (α > 0.85).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [50] Morgeson FP, Humphrey SE. The Work Design Questionnaire (WDQ): developing and validating a comprehensive measure for assessing job design and the nature of work. J Appl Psychol. 2006;91(6):1321–39. http://doi.org/ 10.1037/0021-9010.91.6.1321 . source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    Care as Usual will be monitored using the Client Service Receipt Inventory adapted for this study.

    “CAU (including medication for depression or anxiety) will be monitored using the Client Service Receipt Inventory ([52]; adapted for this study).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [52] Beecham J, Knapp M. Costing psychiatric interventions. In: Thornicorft G, Brewin C, Wing J, editors. Measuring mental health needs. 1st ed. London: Gaskell/Royal College of Psychiatrists; 1992. p. 163–83.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Contamination between groups will be monitored with a question about discussion with colleagues allocated to a different research group.

    “Contamination between the groups will be monitored with the question: ‘During the course of this study to what extent have you discussed the research with colleagues who were allocated a different research group? (For example, if you are in the control group have you spoken with colleagues who are using the online programme?)’.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    The time perception measure has a good level of reliability (α = 0.89).

    “The time perception measure [49] is a five-item questionnaire that has a good level of reliability (α = 0.89).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [49] Etkin J, Evangelidis I, Aaker J. Pressed for time? Goal conflict shapes how time is perceived, spent, and valued. J Market Res. 2015;52(3):394–406. http://doi.org/ 10.1509/jmr.14.013 . source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    One of the stated benefits of Internet-based or eHealth interventions is that utilisation or dose can be objectively measured.

    “One of the stated benefits of Internet-based or eHealth (the use of information and communication technology for health) interventions is that utilisation or dose can be objectively measured [59].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [59] Norman GJ, Zabinski MF, Adams MA, Rosenberg DE, Yaroch AL, Atienza AA. A review of eHealth interventions for physical activity and dietary behavior change. Am J Prev Med. 2007;33(4):336–45. doi: 10.1016/j.amepre.2007.05.007 . source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Where possible, the demographics of trial participants will be compared with the workforce of each organisation to check whether trial participants are representative of the workforce.

    “Baseline differences between groups will be explored using the chi-square test and ANOVA (as appropriate) and, where possible, we will compare demographics of trial participants with the workforce of each organisation to see if trial participants are representative of the workforce.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Previous experience of stress management training will be monitored.

    “Previous experience of stress management training will be monitored with the question: ‘Have you previously received stress management training including training on relaxation techniques and time management?’”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    Participants in the online discussion group will be anonymous in the group by selecting a user name.

    “Participants will be required to select a user name, so that they will be anonymous in the group.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Other studies that included online discussion groups as part of an online intervention failed to report usage data, to differentiate usage of the groups across treatment groups, or to analyse the impact of the discussion groups on study outcomes.

    “Other studies have included online discussion groups as part of an online intervention (for example: [27, 29, 30]), but failed to either report usage data, differentiate the usage of the groups across the treatment groups, or analyse the impact of the discussion groups on the study outcomes.”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [27] Titov N, Andrews G, Choi I, Schwencke G, Mahoney A. Shyness 3: randomized controlled trial of guided versus unguided Internet-based CBT for social phobia. Aust N Z J Psychiatry. 2008;42(12):1030–40. http://doi.org/ 10.1080/00048670802512107 . source

      Not checked in the free run.

    • cites [29] Berger T, Caspar F, Richardson R, Kneubuhler B, Sutter D, Andersson G. Internet-based treatment of social phobia: a randomized controlled trial comparing unguided with two types of guided self-help. Behav Res Ther. 2011;49(3):158–69. http://doi.org/ 10.1016/j.brat.2010.12.007 . source

      Not checked in the free run.

    • cites [30] El Alaoui S, Ljótsson B, Hedman E, Kaldo V, Andersson E, Rück C, Lindefors N. Predictors of symptomatic change and adherence in Internet-based cognitive behaviour therapy for social anxiety disorder in routine psychiatric care. PLoS One. 2015;10(4):e0124258. http://doi.org/ 10.1371/journal.pone.0124258 . source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    Levels of sickness absence will be assessed by asking participants if they had taken time off work for a stress-related complaint in the previous 8 weeks at time points 1, 2 and 3.

    “To assess levels of sickness absence, participants will be asked if they had taken time off work for a stress-related complaint in the previous 8 weeks at time points 1, 2 and 3.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    The primary outcome (number of logins to the site) will be analysed on an intention-to-treat basis, with participants analysed in the group to which they were randomised regardless of treatment received or extent of engagement.

    “Analysis of the primary outcome measure (number of logins to the site) will be conducted on an intention-to-treat basis (participants’ data is analysed in the group that they are randomised to, regardless of treatment they receive or the extent to which they engage with the intervention).”
    • Also stated in: § Statistical analyses · plain
      “Analysis of the secondary measures (psychological distress and subjective wellbeing) will also be conducted on an intention-to-treat basis.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    The most common objective exposure measure used in studies is login rates; other measures include number of pages visited, length of visit and sessions or modules completed.

    “The most common objective exposure measure used in studies is login rates [23], other measures include number of pages visited, length of visit and sessions or modules completed.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [23] Brouwer W, Kroeze W, Crutzen R, de Nooijer J, de Vries NK, Brug J, Oenema A. Which intervention characteristics are related to more exposure to Internet-delivered healthy lifestyle promotion interventions? A systematic review. J Med Internet Res. 2011;13(1):e2. http://doi.org/ 10.2196/jmir.1639 . source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given

    One explanation for the failure of clinical-setting efficacy evidence to translate to workplace online stress management interventions might be the additional challenge of achieving engagement and adherence to an online intervention delivered within a dynamic and busy working environment.

    “One explanation for this failure might be the additional challenge of achieving engagement and adherence to an online intervention that is delivered within a dynamic and busy working environment.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The evidence for the efficacy of online CBT-based interventions delivered within clinical settings has not translated to CBT-based online stress management interventions delivered within workplace settings.

  • May combine two claims.

    Study design and conductreasoning

    This pilot study is underpowered and therefore cannot allow definitive conclusions to be drawn.

    “This pilot study, while underpowered so unable to allow us to draw definitive conclusions, will provide the parameters to inform the methods of a definitive trial.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    The adherence threshold of three logins to the WorkGuru site was derived from average login data from the site and reflects login data from other studies of online health promotion interventions.

    “This baseline has been identified from average login data from the site and reflects login data from other studies (for a summary of login data for online health promotion interventions see [23]).”
    • Also stated in: § Discussion · boosted
      “This baseline has been established from current WorkGuru usage and in reference to login data from studies on other health promotion sites (see [23]).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [23] Brouwer W, Kroeze W, Crutzen R, de Nooijer J, de Vries NK, Brug J, Oenema A. Which intervention characteristics are related to more exposure to Internet-delivered healthy lifestyle promotion interventions? A systematic review. J Med Internet Res. 2011;13(1):e2. http://doi.org/ 10.2196/jmir.1639 . source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    The System Usability Scale has demonstrated good reliability (α = 0.91).

    “The test has demonstrated good reliability (α = 0.91; [48]).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [48] Bangor A, Kortum PT, Miller JT. An empirical evaluation of the System Usability Scale. Int J Hum Comput Interact. 2008;24(6):574–94. http://doi.org/ 10.1080/10447310802205776 . source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductcitation

    As recommended by Rozental et al., a question to identify possible negative effects of the intervention was included, and deterioration between pre and post treatment will be reported.

    “As recommended by Rozental et al. [51], we have included a question to identify any possible negative effects of the intervention. Deterioration between pre and post treatment will be reported, and the self-report question: ‘What, if any, positive or negative effects caused by the programme/being in the control group did you experience?’ will be asked.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [51] Rozental A, Andersson G, Boettcher J, Ebert DD, Cuijpers P, Knaevelsrud C, Carlbring P. Consensus statement on defining and measuring negative effects of Internet interventions. Internet Interventions. 2014;1(1):12–9. http://doi.org/ 10.1016/j.invent.2014.02.001 . source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    To the authors' knowledge, this is the first study that will isolate the effect of an online facilitated discussion group on adherence, engagement and effectiveness of an online CBT-based stress management intervention.

    “To our knowledge this is the first study that will isolate the effect of an online facilitated discussion group on adherence, engagement and effectiveness of an online CBT-based stress management intervention.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    In the study by Hilvert-Bruce et al., adherence to an online CBT programme increased from 37.9% to 60% after changes to the way the intervention was delivered.

    “Adherence increased after the changes had been made from 37.9% to 60%, an increase of 58%.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [24] Hilvert-Bruce Z, Rossouw PJ, Wong N, Sunderland M, Andrews G. Adherence as a determinant of effectiveness of Internet cognitive behavioural therapy for anxiety and depressive disorders. Behav Res Ther. 2012;50(7–8):463–8. http://doi.org/ 10.1016/j.brat.2012.04.001 . source

      Not checked in the free run.

  • May combine two claims.

    Study design and conductreasoning

    It is not possible to blind participants to the type of intervention they receive, which could bias the self-reported measures, although the inclusion of two active groups may limit this bias.

    “While it is not possible to blind participants to the type of intervention that they receive, which could result in a bias in the self-reported measures, the inclusion of two active groups may limit this bias.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    The design of the WorkGuru stress management intervention is based on theoretical psychological principles whose efficacy has been proven in other studies for both face-to-face and online delivery.

    “The design of the stress management intervention (WorkGuru) is based on clear theoretical psychological principles, the efficacy for which has been proven in other studies for both face to face delivery (for example: [55, 56]), and for online delivery (for example: [57, 58]).”
    Quote in the paperSubstantial claim0 of 4 sources checked
    • cites [55] Blonk R, Houtman I, Brenninkmeijer V, Lagerveld SE. Return to work: a comparison of two cognitive behavioural interventions in cases of work-related psychological complaints among the self-employed. Work Stress. 2006;20(2):129–44. http://doi.org/ 10.1080/02678370600856615 . source

      Not checked in the free run.

    • cites [56] van der Klink J, Blonk R, Schene AH, van Dijk FJ. The benefits of interventions for work-related stress. Am J Public Health. 2001;91(2):270–6. source

      Not checked in the free run.

    • cites [57] Hasson D, Anderberg UM, Theorell T, Arnetz BB. Psychophysiological effects of a web-based stress management system: a prospective, randomized controlled intervention study of IT and media workers [ISRCTN54254861]. BMC Public Health. 2005;5:78. source

      Not checked in the free run.

    • cites [58] Rose RD. Self-guided multimedia stress management and resilience training. J Posit Psychol. 2014;9(6):489–93. http://doi.org/ 10.1080/17439760.2014.927907 . source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    The study by Hilvert-Bruce et al. found that noncompleters still benefited from the intervention but that greater adherence resulted in greater benefit.

    “The study by Hilvert-Bruce et al. [24] found that noncompleters still benefited from the intervention but that greater adherence resulted in greater benefit.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [24] Hilvert-Bruce Z, Rossouw PJ, Wong N, Sunderland M, Andrews G. Adherence as a determinant of effectiveness of Internet cognitive behavioural therapy for anxiety and depressive disorders. Behav Res Ther. 2012;50(7–8):463–8. http://doi.org/ 10.1016/j.brat.2012.04.001 . source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Existing psychological illness will be monitored with a question about diagnosis of mental illness from a GP or health care professional.

    “Existing psychological illness will be monitored with the question: ‘Have you received a diagnosis of mental illness from your GP or a health care professional?’”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    In the study by Hilvert-Bruce et al., the difference in average number of lessons completed before and after the changes was statistically significant (t(1106) = 8.8, p < .001), with a Cohen's d effect size of 0.53.

    “This was a significant difference t(1106) = 8.8, p < .001. The Cohen’s d effect size was d = 0.53.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [24] Hilvert-Bruce Z, Rossouw PJ, Wong N, Sunderland M, Andrews G. Adherence as a determinant of effectiveness of Internet cognitive behavioural therapy for anxiety and depressive disorders. Behav Res Ther. 2012;50(7–8):463–8. http://doi.org/ 10.1016/j.brat.2012.04.001 . source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    A sensitivity analysis including a per-protocol analysis will be performed to check the robustness of the primary findings.

    “To check the robustness of the primary findings, sensitivity analysis including a per-protocol analysis will be performed.”
    Quote in the paperNot a substantial claim
47 of 47

  • The authors state this as an assumption.

    This is the first study to explore the effect of an online discussion group on the engagement and effectiveness of an online CBT-based stress management intervention.

    “To our knowledge this is the first study to explore the effect of an online discussion group on the engagement and effectiveness of an online CBT-based stress management intervention.”

  • The paper relies on this without support.

    Workplace psychological ill-health is a growing problem for both employers and their employees.

    “Workplace psychological ill-health is a growing problem for both employers and their employees”

  • The paper relies on this without support.

    The evidence for the efficacy of online CBT-based interventions delivered within clinical settings has not translated to CBT-based online stress management interventions delivered within workplace settings.

    “while there is clear and convincing evidence for the efficacy of delivering CBT-based online interventions within clinical settings, that evidence has not translated to CBT-based online stress management interventions delivered within workplace settings”
3 of 3

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Statistics recomputed

Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.

  • Consistentt(1106) = 8.8reported p < .001 · recomputed p < .001

    “t(1106) = 8.8, p < .001”

1 of 1

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Self-check results appear here once an independent check has confirmed them.

Cites

Flagged first, then by citations

2 of the 63 references couldn’t be matched to a known paper.

Cited by

Sorted by citations

Related

Authors
Carolan, S., Harris, P. R., Greenwood, K., & Cavanagh, K.
Affiliations
University of Sussex (GB)
Corresponding author
Carolan, Stephany
ORCID
Venue
Trials
Publisher
Springer Science and Business Media LLC
Published
2016-12
Volume
17
Issue
1
Pages
598-598
ISSN
1745-6215 (ISSN-L)
DOI
10.1186/s13063-016-1733-2
PMC
PMC5159960
OpenAlex
W2566954800
Semantic Scholar
892f4521257e5e37e38560674e6dfd09ef0b50ea
Cited by
9
References
63
Citations by year
2026: 1 · 2025: 1 · 2024: 1 · 2022: 1 · 2019: 1 · 2018: 2 · 2017: 2
FWCI
2.56
Type
journal article
Language
en
Open access
Licence
Locations
  • Trials (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Sussex Research Online (University of Sussex) (submittedVersion, cc-by)
  • Figshare (submittedVersion, cc-by)
Topics
  • Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
  • Health Sciences › Health Professions › General Health Professions › Workplace Health and Well-being
  • Social Sciences › Psychology › Clinical Psychology › Mindfulness and Compassion Interventions
Keywords
stress management, occupational stress, workplace mental health, intervention engagement, online forums, bulletin boards, online intervention, psychological distress, subjective well-being, employee autonomy, time pressure, goal conflict, treatment acceptability
MeSH terms
Attitude of Health Personnel, Clinical Protocols, United Kingdom, Humans, Health Knowledge, Attitudes, Practice, Mental Health, Occupational Diseases, Occupational Health Services, Patient Participation, Pilot Projects, Psychotherapy, Group, Surveys and Questionnaires, Research Design, Stress, Psychological, Therapy, Computer-Assisted, Time Factors, Cognitive Behavioral Therapy, Occupational Health, Treatment Outcome, Internet, Intention to Treat Analysis

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-10.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-10.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review