Effectiveness of eHealth interventions for reducing mental health conditions in employees: A systematic review and meta-analysis
Open accessThe DOI matches its Crossref record; 74 of the 75 references it cites were found in a registry or an index; 2 of the 20 sources its claims cite were checked for support.
Abstract
BACKGROUND: Many organisations promote eHealth applications as a feasible, low-cost method of addressing mental ill-health and stress amongst their employees. However, there are good reasons why the efficacy identified in clinical or other samples may not generalize to employees, and many Apps are being developed specifically for this group. The aim of this paper is to conduct the first comprehensive systematic review and meta-analysis evaluating the evidence for the effectiveness and examine the relative efficacy of different types of eHealth interventions for employees. METHODS: Systematic searches were conducted for relevant articles published from 1975 until November 17, 2016, of trials of eHealth mental health interventions (App or web-based) focused on the mental health of employees. The quality and bias of all identified studies was assessed. We extracted means and standard deviations from published reports, comparing the difference in effect sizes (Hedge's g) in standardized mental health outcomes. We meta-analysed these using a random effects model, stratified by length of follow up, intervention type, and whether the intervention was universal (unselected) or targeted to selected groups e.g. "stressed". RESULTS: 23 controlled trials of eHealth interventions were identified which overall suggested a small positive effect at both post intervention (g = 0.24, 95% CI 0.13 to 0.35) and follow up (g = 0.23, 95% CI 0.03 to 0.42). There were differential short term effects seen between the intervention types whereby Mindfulness based interventions (g = 0.60, 95% CI 0.34 to 0.85, n = 6) showed larger effects than the Cognitive Behaviour Therapy (CBT) based (g = 0.15, 95% CI 0.02 to 0.29, n = 11) and Stress Management based (g = 0.17, 95%CI -0.01 to 0.34, n = 6) interventions. The Stress Management interventions however differed by whether delivered to universal or targeted groups with a moderately large effect size at both post-intervention (g = 0.64, 95% CI 0.54 to 0.85) and follow-up (g = 0.69, 95% CI 0.06 to 1.33) in targeted groups, but no effect in unselected groups. INTERPRETATION: There is reasonable evidence that eHealth interventions delivered to employees may reduce mental health and stress symptoms post intervention and still have a benefit, although reduced at follow-up. Despite the enthusiasm in the corporate world for such approaches, employers and other organisations should be aware not all such interventions are equal, many lack evidence, and achieving the best outcomes depends upon providing the right type of intervention to the correct population.
Overview
CoChat Summary
TL;DR
This systematic review and meta-analysis found that eHealth interventions for employees had a small overall benefit for mental health outcomes post-intervention and at follow-up, but effects varied substantially by intervention type and target population. Mindfulness-based interventions were strongest for universal employees, while Stress Management appeared beneficial mainly for selected employees with elevated stress.
Key findings
- Across 23 controlled trials, eHealth interventions showed a small positive effect on standardized mental health outcomes post-intervention (g = 0.24, 95% CI 0.13 to 0.35) and at follow-up (g = 0.23, 95% CI 0.03 to 0.42).
- By intervention type, post-intervention effects differed: Mindfulness was larger (g = 0.60, 95% CI 0.34 to 0.85, n = 6) than CBT (g = 0.15, 95% CI 0.02 to 0.29, n = 11) and Stress Management (g = 0.17, 95% CI -0.01 to 0.34, n = 6).
- Stress Management worked differently by population: in selected groups it showed a moderate-to-large effect post-intervention (g = 0.64, 95% CI 0.43 to 0.85) and at follow-up (g = 0.69, 95% CI 0.06 to 1.33), but in unselected groups it showed no effect; one unselected follow-up result was negative (g = -0.16, 95% CI -0.41 to 0.09).
- The paper reports moderate to large heterogeneity overall (I2 = 67.6% post-intervention; I2 = 77.7% at follow-up) and evidence of a small study effect at post-intervention (Egger’s intercept 2.53, p = 0.01; trim-and-fill adjusted effect g = 0.12, 95% CI 0.01 to 0.25).
Methods
- Systematic review and meta-analysis conducted according to PRISMA, searching MEDLINE, PsycINFO, CENTRAL, and EMBASE for studies published from 1975 to 17 November 2016.
- Included peer-reviewed English-language trials of web-based or smartphone/tablet App eHealth interventions for employees aged 18 to 65, with standardized outcomes for depression, anxiety, and/or stress; studies without controls were excluded from the meta-analysis.
- Effect sizes were pooled as Hedge’s g using a random-effects model, with subgroup analyses by intervention type (CBT, Stress Management, Mindfulness) and by universal vs targeted participants; quality was assessed with the Downs and Black checklist and risk of bias with Cochrane guidelines.
Limitations
- The paper states that the number of workplace eHealth studies was surprisingly small, with only 23 studies in the meta-analysis, and some subgroups had too few studies for strong conclusions.
- It also states that outcomes were based on self-reported measures, so conclusions are limited to self-reported symptom reduction rather than clinical diagnosis.
- Evident from the text: there was substantial heterogeneity and a small study effect, which may inflate apparent benefits; the paper also notes that initial screening of some titles for exclusion was only completed by one author.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
2 Valid · 0 Not valid · 12 Not checked · 55 Own results and interpretations
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Previous research indicates that large effect sizes are unlikely to be found in interventions delivered to general populations who may not be unwell [19, 73].”
§ Effectiveness of eHealth interventions for reducing mental health conditions in employees: A systematic review and meta-analysis > Discussion
What the sources say
- Supportedcites [19] M Deady, 2017, eHealth interventions for the prevention of depression and anxiety in the general population: A systematic review and meta-analysis, BMC Psychiatry source
The Discussion of Deady et al. (2017) states outright that 'Prevention interventions are never likely to produce large individual effect sizes, as they are delivered to the mass populations who are not unwell,' matching the claim's substance (small effects expected in general, non-unwell populations). The same section reports the review's own pooled effect sizes as small (universal 0.29; indicated/selective 0.25, Results > Secondary analyses), and the Methods note that 'At a population level, when considering prevention interventions, small effect sizes are considered relevant.' The trailing clause about population-level impact being substantial is credited to reference [58], but the likelihood-of-small-effect-size point is the paper's own assertion, so the attribution holds. (LLM, conf=0.90)
- Not checked yetcites [73] G Rose, 1994, The strategy of prevention medicine
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“selected (n = 3) (g = 0.09, 95% CI -0.09 to 0.26, p = 3.40, I2 = 6.2%)”
§ Effectiveness of eHealth interventions for reducing mental health conditions in employees: A systematic review and meta-analysis > Results > Meta-analysis results
plainn = 3- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“At follow-up there was only study in an unselected group showing a small non-significant negative effect (g = -0.16, 95% CI -0.41 to 0.09, p = 0.21, I2 = 0.0%)”
§ Effectiveness of eHealth interventions for reducing mental health conditions in employees: A systematic review and meta-analysis > Results > Meta-analysis results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For stress levels it is evident that Stress Management interventions are most effective at post-intervention.”
§ Effectiveness of eHealth interventions for reducing mental health conditions in employees: A systematic review and meta-analysis > Discussion
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Mindfulness based interventions (n = 6) showed a moderate to large positive effect size and no heterogeneity (g = 0.60, 95% CI 0.34 to 0.85, p = <0.00, I2 = 0.0%)”
§ Effectiveness of eHealth interventions for reducing mental health conditions in employees: A systematic review and meta-analysis > Results > Meta-analysis results
plainstates a causen = 6 ·- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
33 more claims
Create a free account to see all 69 claims and to verify the 18 references still open.
- It may state two separate claims as one. 21
- Probably not a substantial claim. 12
- The quote only partly states this claim. 1
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The review that focused on employees observed a small positive effect for web-based congnitive behavioural therapy (CBT) with remote monitoring in reducing stress and anxiety in employees, but concluded that interventions targeted to individual employees were more effective than universal interventions [37].”
From Telemedicine in the OECD: An umbrella review of clinical and cost-effectiveness, patient experience and implementation (2020) · cites this paper as [37]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Previous meta-analyses of self-guided programs found small effects in working populations (Carolan et al., 2017; Stratton et al., 2017) and in general populations (Heber et al., 2017) on stress and psychological distress.”
From Effects of self-guided stress management interventions in college students: A systematic review and meta-analysis (2022) · cites this paper as Stratton et al., 2017
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Web-based interventions have proven to be effective not only in clinical populations but also in the occupational setting.”
From Acceptability of Web-Based Mental Health Interventions in the Workplace: Systematic Review (2022) · cites this paper as [13]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Clinical populations differ significantly in symptom severity, level of risks, functionality, and response to treatment; thus, the results might not be generalizable to occupational populations [13].”
From Acceptability of Web-Based Mental Health Interventions in the Workplace: Systematic Review (2022) · cites this paper as [13]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Similarly, tailoring interventions to the environment and employees’ needs could increase their general interest and willingness to use them [13].”
From Acceptability of Web-Based Mental Health Interventions in the Workplace: Systematic Review (2022) · cites this paper as [13]
Experimental — accuracy not yet measured
7 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & The PRISMA Group (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000097 · Cited by 64,042
Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.
(2009) · PubMed Central · Cited by 44,859
Cochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series
(2008) · doi:10.1002/9780470712184 · Cited by 25,048
Wang, G., Ma, R., Qiao, G., Wada, K., Aizawa, Y., & Satoh, T. (2015) · Integrative Medicine International · doi:10.1159/000368408 · Cited by 18,657
Duval, S., & Tweedie, R. (2000) · Biometrics · doi:10.1111/j.0006-341x.2000.00455.x · Cited by 14,099 · cited 1×
“In the case where at least 10 studies were available for analysis and a small study effect was found we used a Duval and Tweedie trim and fill to quantify the magnitude of small study effect [44].”
MethodZhang, Q., Wang, C., Zheng, M., Li, Y., Li, J., Zhang, L., Shang, X., & Yan, C. (2015) · Cerebrovascular Diseases · doi:10.1159/000369778 · Cited by 11,533
Projections of Global Mortality and Burden of Disease from 2002 to 2030
Mathers, C. D., & Loncar, D. (2006) · PLoS Medicine · doi:10.1371/journal.pmed.0030442 · Cited by 11,386
Downs, S. H., & Black, N. (1998) · Journal of Epidemiology and Community Health · doi:10.1136/jech.52.6.377 · Cited by 8,252 · cited 1×
“The quality of the identified studies was assessed using the Downs and Black checklist [38].”
MethodPublication Bias in Meta‐Analysis: Prevention, Assessment and Adjustments
(2005) · doi:10.1002/0470870168 · Cited by 2,368
Computer-based psychological treatments for depression: A systematic review and meta-analysis
Richards, D., & Richardson, T. (2012) · Clinical Psychology Review · doi:10.1016/j.cpr.2012.02.004 · Cited by 1,442 · cited 2×
“Similar results were found in a systematic review and meta-analysis of eHealth interventions in a clinical setting [20].”
BackgroundResultInternet-Based and Other Computerized Psychological Treatments for Adult Depression: A Meta-Analysis
Andersson, G., & Cuijpers, P. (2009) · Cognitive Behaviour Therapy · doi:10.1080/16506070903318960 · Cited by 1,368 · cited 1×
“Sensitivity analysis results were also assessed by whether the intervention was unguided (selfhelp) or guided (feedback, rather than just technical support, as this is known to affect the outcome [37].”
BackgroundAndrews, G., Cuijpers, P., Craske, M. G., McEvoy, P., & Titov, N. (2010) · PLoS ONE · doi:10.1371/journal.pone.0013196 · Cited by 1,311 · cited 1×
“Both groups showed very small if any effect at post-intervention or followup, indicating that CBT interventions appeared not to have any statistical benefits either universal or targeted working populations which seems to differ from large results seen in both universal [76] and clinical [77], populations.”
ResultAndersson, G., Cuijpers, P., Carlbring, P., Riper, H., & Hedman, E. (2014) · World Psychiatry · doi:10.1002/wps.20151 · Cited by 1,077 · cited 1×
“Thus there is currently ample evidence for the short and long term benefits for eHealth delivered Cognitive Behavioural Therapy (CBT) for treating anxiety and depressive conditions in both general population and clinical settings [21–23], with greater improvement shown for those interventions that include guidance thr…”
BackgroundSpijkerman, M. P. J., Pots, W. T. M., & Bohlmeijer, E. T. (2016) · Clinical Psychology Review · doi:10.1016/j.cpr.2016.03.009 · Cited by 975 · cited 1×
“Evidence has also emerged for the effectiveness of Mindfulness based eHealth interventions in improving symptoms in both unselected and symptomatic people [25].”
BackgroundHarvey, S. B., Modini, M., Joyce, S., Milligan-Saville, J. S., Tan, L., Mykletun, A., Bryant, R. A., Christensen, H., & Mitchell, P. B. (2017) · Occupational and Environmental Medicine · doi:10.1136/oemed-2016-104015 · Cited by 799
Self-citationEffective and viable mind-body stress reduction in the workplace: A randomized controlled trial.
Wolever, R. Q., Bobinet, K. J., McCabe, K., Mackenzie, E. R., Fekete, E., Kusnick, C. A., & Baime, M. (2012) · Journal of Occupational Health Psychology · doi:10.1037/a0027278 · Cited by 539 · cited 1×
“Two papers did not have a control group [56,57], and one study used a face-to-face control group [68].”
BackgroundWorkplace interventions for common mental disorders: A systematic meta-review
Joyce, S., Modini, M., Christensen, H., Mykletun, A., Bryant, R., Mitchell, P. B., & Harvey, S. B. (2016) · Psychological Medicine · doi:10.1017/s0033291715002408 · Cited by 525 · cited 2×
“Despite mental illness now being the leading cause of work incapacity and sickness absence [14], and depression estimated to be the leading cause of disability at work by 2020 [15], a recent systematic meta-review found that there no workplace mental health interventions that could be considered evidence-based [16].”
BackgroundMethodSelf-citationWorking for a healthier tomorrow
Hulshof, C. T. J. (2009) · Occupational and Environmental Medicine · doi:10.1136/oem.2008.040899 · Cited by 497
Stewart, W. F., Ricci, J. A., Chee, E., & Morganstein, D. (2003) · Journal of Occupational and Environmental Medicine · doi:10.1097/01.jom.0000099999.27348.78 · Cited by 427 · cited 2×
“Overall pooled eHealth interventions showed a small significant positive effect at post intervention and follow-up for reducing overall mental health (depression, anxiety and stress) in both universal and selected employees which is similar to a recent review of eHealth interventions for general population [19], provi…”
BackgroundResultGilbody, S., Littlewood, E., Hewitt, C., Brierley, G., Tharmanathan, P., Araya, R., Barkham, M., Bower, P., Cooper, C., Gask, L., et al. (2015) · BMJ · doi:10.1136/bmj.h5627 · Cited by 409
Work and common psychiatric disorders
Henderson, M., Harvey, S., Øverland, S., Mykletun, A., & Hotopf, M. (2011) · Journal of the Royal Society of Medicine · doi:10.1258/jrsm.2011.100231 · Cited by 379 · cited 1×
“Despite mental illness now being the leading cause of work incapacity and sickness absence [14], and depression estimated to be the leading cause of disability at work by 2020 [15], a recent systematic meta-review found that there no workplace mental health interventions that could be considered evidence-based [16].”
BackgroundSelf-citationSamoocha, D., Bruinvels, D. J., Elbers, N. A., Anema, J. R., & van der Beek, A. J. (2010) · Journal of Medical Internet Research · doi:10.2196/jmir.1286 · Cited by 374 · cited 1×
“The checklist was modified to simplify the power item to a score of either zero = no or one = yes based on whether the studies reported sufficient power to detect a clinically significant effect, as reported in previous studies [11,39].”
MethodMindfulness Goes to Work: Impact of an Online Workplace Intervention
Aikens, K. A., Astin, J., Pelletier, K. R., Levanovich, K., Baase, C. M., Park, Y. Y., & Bodnar, C. M. (2014) · Journal of Occupational & Environmental Medicine · doi:10.1097/jom.0000000000000209 · Cited by 338
Wagner, B., Horn, A. B., & Maercker, A. (2014) · Journal of Affective Disorders · doi:10.1016/j.jad.2013.06.032 · Cited by 311 · cited 1×
“To mitigate the cost of these, many organisations are now providing eHealth interventions and applications into their workplace health management systems for employees, either by creating their own programs for internal use or buying commercially available products [29,30].”
BackgroundMobile mental health: Review of the emerging field and proof of concept study
Harrison, V., Proudfoot, J., Wee, P. P., Parker, G., Pavlovic, D. H., & Manicavasagar, V. (2011) · Journal of Mental Health · doi:10.3109/09638237.2011.608746 · Cited by 267 · cited 1×
“The past decade has seen an explosion in the delivery of eHealth interventions targeting common mental illnesses such as depression and anxiety, as well as those focused upon broader concepts of stress and distress [17] through Apps and via the Internet.”
BackgroundDunn, A. G., Leask, J., Zhou, X., Mandl, K. D., & Coiera, E. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.4343 · Cited by 256
Deady, M., Choi, I., Calvo, R. A., Glozier, N., Christensen, H., & Harvey, S. B. (2017) · BMC Psychiatry · doi:10.1186/s12888-017-1473-1 · Cited by 222 · cited 2×
“Previous research indicates that large effect sizes are unlikely to be found in interventions delivered to general populations who may not be unwell [19, 73].”
BackgroundResultSelf-citationWeb-Based and Mobile Stress Management Intervention for Employees: A Randomized Controlled Trial
Heber, E., Lehr, D., Ebert, D. D., Berking, M., & Riper, H. (2016) · Journal of Medical Internet Research · doi:10.2196/jmir.5112 · Cited by 213
Tan, L., Wang, M.-J., Modini, M., Joyce, S., Mykletun, A., Christensen, H., & Harvey, S. B. (2014) · BMC Medicine · doi:10.1186/1741-7015-12-74 · Cited by 208 · cited 3×
“Scores were pooled into four categories as used in other reviews [11]: Effectiveness of eHealth interventions for reducing mental health conditions in employees”
BackgroundMethodSelf-citationAndersson, G., Hesser, H., Veilord, A., Svedling, L., Andersson, F., Sleman, O., Mauritzson, L., Sarkohi, A., Claesson, E., Zetterqvist, V., et al. (2013) · Journal of Affective Disorders · doi:10.1016/j.jad.2013.08.022 · Cited by 195 · cited 1×
“Some eHealth interventions offer fully unguided self-help programs while others involve supported guidance throughout and are thus more costly [18].”
BackgroundKnudsen, A. K., Harvey, S. B., Mykletun, A., & Øverland, S. (2013) · Acta Psychiatrica Scandinavica · doi:10.1111/j.1600-0447.2012.01902.x · Cited by 185 · cited 1×
“However, common mental disorders may go unnoticed in the workplace, as they can often be characterized as work stress or other stress related conditions [7].”
BackgroundSelf-citationSheffield, J. K., Spence, S. H., Rapee, R. M., Kowalenko, N., Wignall, A., Davis, A., & McLoone, J. (2006) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.74.1.66 · Cited by 175 · cited 1×
“There was no difference between universal and targeted CBT interventions, although, previous evidence suggests that selected interventions may be more effective than universal interventions [20,74,75].”
BackgroundAhtinen, A., Mattila, E., Välkkynen, P., Kaipainen, K., Vanhala, T., Ermes, M., Sairanen, E., Myllymäki, T., & Lappalainen, R. (2013) · JMIR mhealth and uhealth · doi:10.2196/mhealth.2596 · Cited by 173 · cited 1×
“Two papers did not have a control group [56,57], and one study used a face-to-face control group [68].”
BackgroundCook, R. F., Billings, D. W., Hersch, R. K., Back, A. S., & Hendrickson, A. (2007) · Journal of Medical Internet Research · doi:10.2196/jmir.9.2.e17 · Cited by 170
A randomized controlled pilot study of a brief web-based mindfulness training
Glück, T. M., & Maercker, A. (2011) · BMC Psychiatry · doi:10.1186/1471-244x-11-175 · Cited by 168
Birney, A. J., Gunn, R., Russell, J. K., & Ary, D. V. (2016) · JMIR mHealth and uHealth · doi:10.2196/mhealth.4231 · Cited by 159
Mental health and employment: much work still to be done
Harvey, S. B., Henderson, M., Lelliott, P., & Hotopf, M. (2009) · British Journal of Psychiatry · doi:10.1192/bjp.bp.108.055111 · Cited by 159
Self-citationAllexandre, D., Bernstein, A. M., Walker, E., Hunter, J., Roizen, M. F., & Morledge, T. J. (2016) · Journal of Occupational & Environmental Medicine · doi:10.1097/jom.0000000000000680 · Cited by 157
Small-Study Effects in Meta-Analysis
Schwarzer, G., Carpenter, J. R., & Rücker, G. (2015) · Use R! · doi:10.1007/978-3-319-21416-0_5 · Cited by 145 · cited 1×
“There is evidence of a small study effect in the overall 23 studies at post-intervention, a possible cause for this is the clinical heterogeneity between the participants in the large and small studies, the smaller studies were targeted at the selected group (n = 1328 participants) so that a favourable outcome of the…”
ResultSpek, V., Cuijpers, P., Nyklíček, I., Smits, N., Riper, H., Keyzer, J., & Pop, V. (2008) · Psychological Medicine · doi:10.1017/s0033291707002590 · Cited by 139 · cited 1×
“Both groups showed very small if any effect at post-intervention or followup, indicating that CBT interventions appeared not to have any statistical benefits either universal or targeted working populations which seems to differ from large results seen in both universal [76] and clinical [77], populations.”
ResultLy, K. H., Asplund, K., & Andersson, G. (2014) · Internet Interventions · doi:10.1016/j.invent.2014.06.003 · Cited by 133
So, M., Yamaguchi, S., Hashimoto, S., Sado, M., Furukawa, T. A., & McCrone, P. (2013) · BMC Psychiatry · doi:10.1186/1471-244x-13-113 · Cited by 129
Does Psychoeducation Help Prevent Post Traumatic Psychological Distress?
Wessely, S., Bryant, R. A., Greenberg, N., Earnshaw, M., Sharpley, J., & Hughes, J. H. (2008) · Psychiatry: Interpersonal and Biological Processes · doi:10.1521/psyc.2008.71.4.287 · Cited by 125 · cited 1×
“It is not clear why a Stress Management intervention would not be effective when used on an unselected population, though it may relate to the dangers of providing non-help seeking populations with too much information, leading them to feel more anxious and vulnerable, as suggested by the authors of these studies[79],…”
ResultThe impact of online resilience training for sales managers on wellbeing and performance
Abbott, J.-A., Klein, B., Hamilton, C., & Rosenthal, A. J. (2009) · E-Journal of Applied Psychology · doi:10.7790/ejap.v5i1.145 · Cited by 119 · cited 1×
“The two studies using cognitive training [59,60] were not included for the meta-analysis as the outcomes were not standardized mental health outcomes.”
MethodA Web-Based Approach to Managing Stress and Mood Disorders in the Workforce
Billings, D. W., Cook, R. F., Hendrickson, A., & Dove, D. C. (2008) · Journal of Occupational & Environmental Medicine · doi:10.1097/jom.0b013e31816c435b · Cited by 117 · cited 2×
“Three authors were contacted, and two of those responded with relevant data [33,34].”
BackgroundMethodEbert, D. D., Lehr, D., Boß, L., Riper, H., Cuijpers, P., Andersson, G., Thiart, H., Heber, E., & Berking, M. (2014) · Scandinavian Journal of Work, Environment & Health · doi:10.5271/sjweh.3449 · Cited by 115
Common mental disorders and disability pension award: Seven year follow-up of the HUSK study
Knudsen, A. K., Øverland, S., Aakvaag, H. F., Harvey, S. B., Hotopf, M., & Mykletun, A. (2010) · Journal of Psychosomatic Research · doi:10.1016/j.jpsychores.2010.03.007 · Cited by 115
Self-citationPsychological debriefing is a waste of time
Wessely, S., & Deahl, M. (2003) · British Journal of Psychiatry · doi:10.1192/bjp.183.1.12 · Cited by 115 · cited 1×
“Only one study reported a follow up and suggested psychological harm, which may, if replicated, be another example of well-meaning intentions going awry, as some believe to occur with psychological debriefing [78].”
BackgroundPhillips, R., Schneider, J., Molosankwe, I., Leese, M., Foroushani, P. S., Grime, P., McCrone, P., Morriss, R., & Thornicroft, G. (2014) · Psychological Medicine · doi:10.1017/s0033291713001323 · Cited by 110
Mak, W. W., Chan, A. T., Cheung, E. Y., Lin, C. L., & Ngai, K. C. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.3746 · Cited by 110
Internet interventions for depression: new developments
Schröder, J., Berger, T., Westermann, S., Klein, J. P., & Moritz, S. (2016) · Dialogues in Clinical Neuroscience · doi:10.31887/dcns.2016.18.2/jschroeder · Cited by 105
Imamura, K., Kawakami, N., Furukawa, T. A., Matsuyama, Y., Shimazu, A., Umanodan, R., Kawakami, S., & Kasai, K. (2014) · PLoS ONE · doi:10.1371/journal.pone.0097167 · Cited by 104 · cited 1×
“Four papers were identified as reporting on different follow-up periods and were merged [48,49] and [50,53].”
BackgroundGrime, P. R. (2004) · Occupational Medicine · doi:10.1093/occmed/kqh077 · Cited by 104 · cited 1×
“Search strategy A thorough literature search was conducted using the electronic databases MEDLINE, PsycINFO, Cochrane Register of Controlled Trials (CENTRAL) and EMBASE for relevant articles published from 1975 (the first eHealth intervention for employees was identified in 2004 [32]) until 17 November 2016.”
MethodEfficacy of Vortioxetine on Cognitive Functioning in Working Patients With Major Depressive Disorder
McIntyre, R. S., Florea, I., Tonnoir, B., Loft, H., Lam, R. W., & Christensen, M. C. (2017) · The Journal of Clinical Psychiatry · doi:10.4088/jcp.16m10744 · Cited by 100 · cited 2×
“However, we know employed people differ systemically from both general or clinical populations used in most eHealth studies (in terms of symptom profile, risks, function and response [26]) Therefore, this evidence may not be generalisable to the occupational setting.”
BackgroundBolier, L., Ketelaar, S. M., Nieuwenhuijsen, K., Smeets, O., Gärtner, F. R., & Sluiter, J. K. (2014) · Internet Interventions · doi:10.1016/j.invent.2014.10.002 · Cited by 98
Hasson, D., Anderberg, U. M., Theorell, T., & Arnetz, B. B. (2005) · BMC Public Health · doi:10.1186/1471-2458-5-78 · Cited by 98 · cited 1×
“In one study [35] the SD could not be obtained from the paper or the authors; effect sizes for this study were derived from 2-tailed p-value for Group × Time interaction and sample sizes.”
MethodGeraedts, A. S., Kleiboer, A. M., Twisk, J., Wiezer, N. M., van Mechelen, W., & Cuijpers, P. (2014) · Journal of Medical Internet Research · doi:10.2196/jmir.3539 · Cited by 89
Eisen, K. P., Allen, G. J., Bollash, M., & Pescatello, L. S. (2008) · Computers in Human Behavior · doi:10.1016/j.chb.2007.02.003 · Cited by 81 · cited 2×
“One paper was excluded due to missing data [36].”
BackgroundHarvey, S. B., Deady, M., Wang, M., Mykletun, A., Butterworth, P., Christensen, H., & Mitchell, P. B. (2017) · Medical Journal of Australia · doi:10.5694/mja16.00295 · Cited by 79 · cited 1×
“The majority of mental ill-health seen in the workforce is due to common mental disorders, most notably, depression and anxiety [6–8], accounting for up to 12% of the Australian working population [9].”
BackgroundSelf-citationLappalainen, P., Kaipainen, K., Lappalainen, R., Hoffrén, H., Myllymäki, T., Kinnunen, M.-L., Mattila, E., Happonen, A. P., Rusko, H., & Korhonen, I. (2013) · JMIR Research Protocols · doi:10.2196/resprot.2389 · Cited by 79
Mental ill health and fitness for work
Glozier, N. (2002) · Occupational and Environmental Medicine · doi:10.1136/oem.59.10.714 · Cited by 77
Self-citationA Web-based health promotion program for older workers : Randomized controlled trial
Cook, R. F., Hersch, R. K., Schlossberg, D., & Leaf, S. L. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.3399 · Cited by 71 · cited 1×
“Three authors were contacted, and two of those responded with relevant data [33,34].”
BackgroundImamura, K., Kawakami, N., Furukawa, T. A., Matsuyama, Y., Shimazu, A., Umanodan, R., Kawakami, S., & Kasai, K. (2015) · Psychological Medicine · doi:10.1017/s0033291714003006 · Cited by 61
Effect of web-based assertion training for stress management of Japanese nurses
YAMAGISHI, M., KOBAYASHI, T., KOBAYASHI, T., NAGAMI, M., SHIMAZU, A., & KAGEYAMA, T. (2007) · Journal of Nursing Management · doi:10.1111/j.1365-2834.2007.00739.x · Cited by 61
Shimazu, A., Kawakami, N., Irimajiri, H., Sakamoto, M., & Amano, S. (2005) · Journal of Occupational Health · doi:10.1539/joh.47.405 · Cited by 56 · cited 1×
“After the quality assessment of all studies using the Down’ & Black Checklist, two studies were excluded from the analysis as they were found to be of ‘poor’ quality [45,55].”
MethodStansfeld, S. A., Kerry, S., Chandola, T., Russell, J., Berney, L., Hounsome, N., Lanz, D., Costelloe, C., Smuk, M., & Bhui, K. (2015) · BMJ Open · doi:10.1136/bmjopen-2015-007981 · Cited by 53
Geraedts, A. S., Kleiboer, A. M., Wiezer, N. M., van Mechelen, W., & Cuijpers, P. (2014) · Journal of Medical Internet Research · doi:10.2196/jmir.3185 · Cited by 48 · cited 1×
“Four papers were identified as reporting on different follow-up periods and were merged [48,49] and [50,53].”
BackgroundHarvey, S. B., Modini, M., Christensen, H. M., & Glozier, N. (2013) · Australian & New Zealand Journal of Psychiatry · doi:10.1177/0004867413476351 · Cited by 47 · cited 1×
“More severe mental illness is less common in the workplace, due to lower levels of unemployment in those with severe mental illness [10].”
BackgroundSelf-citationBorness, C., Proudfoot, J., Crawford, J., & Valenzuela, M. (2013) · PLoS ONE · doi:10.1371/journal.pone.0059982 · Cited by 40 · cited 1×
“The two studies using cognitive training [59,60] were not included for the meta-analysis as the outcomes were not standardized mental health outcomes.”
MethodYamagishi, M., Kobayashi, T., & Nakamura, Y. (2008) · Journal of Occupational Health · doi:10.1539/joh.l7086 · Cited by 36 · cited 1×
“After the quality assessment of all studies using the Down’ & Black Checklist, two studies were excluded from the analysis as they were found to be of ‘poor’ quality [45,55].”
MethodUMANODAN, R., SHIMAZU, A., MINAMI, M., & KAWAKAMI, N. (2014) · Industrial Health · doi:10.2486/indhealth.2013-0209 · Cited by 36
Mori, M., Tajima, M., Kimura, R., Sasaki, N., Somemura, H., Ito, Y., Okanoya, J., Yamamoto, M., Nakamura, S., & Tanaka, K. (2014) · JMIR Research Protocols · doi:10.2196/resprot.3629 · Cited by 21
Effectiveness of a Cognitive Behavioral Therapy-Based Exercise Program for Healthy Employees
Takechi, S., Kikuchi, T., Horisawa, S., Nakagawa, A., Yoshimura, K., & Mimura, M. (2015) · Advances in Physical Education · doi:10.4236/ape.2015.54031 · Cited by 6 · cited 1×
“Two papers did not have a control group [56,57], and one study used a face-to-face control group [68].”
Background
Research works printed but not linked: 4
- C Black, 2008, Dame Carol Black’s Review of the health of Britain’s working age population
- E Ahthes, 2016, Mobile mental-health apps have exploded onto the market, but few have been thoroughly tested. Pocket Psychiatry, Nature, 532
- K Aikens, 2014, Mindfulness goes to work, JOEM, 56
- G Andrews, 2010, Computer therapy for the anxiety and depressive disorders is effective, acceptable and practical health care: A meta-analysis, PLoS ONE, 5, 10.1371/journal.pone.0013196 · DOI · Paper page
Cited by
Bringing health care to the patient: An overview of the use of telemedicine in OECD countries
(2020) · OECD Health Working Papers · doi:10.1787/8e56ede7-en · Cited by 539
Goldberg, S. B., Riordan, K. M., Sun, S., & Davidson, R. J. (2022) · Perspectives on Psychological Science · doi:10.1177/1745691620968771 · Cited by 520
Eze, N. D., Mateus, C., & Cravo Oliveira Hashiguchi, T. (2020) · PLoS ONE · doi:10.1371/journal.pone.0237585 · Cited by 345
“The review that focused on employees observed a small positive effect for web-based congnitive behavioural therapy (CBT) with remote monitoring in reducing stress and anxiety in employees, but concluded that interventions targeted to individual employees were more effective than universal interventions [37].”
BackgroundMobile Apps for Mental Health Issues: Meta-Review of Meta-Analyses
Lecomte, T., Potvin, S., Corbière, M., Guay, S., Samson, C., Cloutier, B., Francoeur, A., Pennou, A., & Khazaal, Y. (2020) · JMIR mHealth and uHealth · doi:10.2196/17458 · Cited by 336
“CorNoNo follow-up described 481RCT rected effect size: g=0.23, 95% CI: 0.04-0.42 (considerably smaller) poorsmall-mediumunknown for this subgroup of studies NoNo follow-up described 468PC +RCT poorsmall-mediumunknown for this subgroup of studies NoNo follow-up described 870PC poorsmall-mediumunknown for this subgroup…”
BackgroundInfluentialMental illness and suicide among physicians
Harvey, S. B., Epstein, R. M., Glozier, N., Petrie, K., Strudwick, J., Gayed, A., Dean, K., & Henderson, M. (2021) · The Lancet · doi:10.1016/s0140-6736(21)01596-8 · Cited by 307
Self-citation
Metadata
- Authors
- Stratton, E., Lampit, A., Choi, I., Calvo, R. A., Harvey, S. B., & Glozier, N.
- Affiliations
- The University of Sydney (AU)
- Black Dog Institute (AU)
- Corresponding author
- Stratton, Elizabeth
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2017-12-21
- Volume
- 12
- Issue
- 12
- Pages
- e0189904
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC5739441
- OpenAlex
- W2776696250
- Semantic Scholar
- e29967f7aa2be1ce851585dac9d486fc8f10d172
- Cited by
- 252
- References
- 75
- Citations by year
- 2026: 15 · 2025: 33 · 2024: 36 · 2023: 35 · 2022: 36 · 2021: 31 · 2020: 30 · 2019: 24 · 2018: 11 · 2017: 1
- FWCI
- 2.09
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Minerva Access (University of Melbourne) (submittedVersion, other-oa)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Spiral (Imperial College London) (submittedVersion, cc-by)
- UNSWorks (University of New South Wales, Sydney, Australia) (submittedVersion, cc-by-nc-nd)
- Funders
- Beyondblue & Movember Foundation
- Australian Research Council Future Fellowship
- NHMRC-ARC Dementia Research Development Fellowship
- Topics
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Workplace Health and Well-being
- Keywords
- digital health interventions, employee mental health, workplace mental health, mobile health apps, online intervention, mindfulness-based interventions, cognitive behavioral therapy, stress management interventions, occupational stress, anxiety, burnout, job satisfaction, work-life balance, work-related musculoskeletal pain
- MeSH terms
- Mental Disorders, Humans, Stress, Psychological, Program Evaluation, Cognitive Behavioral Therapy, Telemedicine, Mindfulness
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.


