Cited by 219Disability Education and EmploymentSchizophrenia research and treatmentEmployment and Welfare Studies

Supported employment: Meta-analysis and review of randomized controlled trials of individual placement and support

Open access
Frederick, Donald E., VanderWeele, Tyler J.·2019·PLOS ONE

Abstract

Supported employment is a treatment whereby those with severe mental illness (or other disabilities) receive aid searching for competitive employment and mental health (or other) treatments concurrently. The most popular implementation of supported employment is individual placement and support (IPS). We conducted meta-analytic analyses of the randomized controlled trials of IPS. We found that subjects in IPS, compared to usual treatment conditions, had better vocational outcomes (obtained any competitive employment: RR = 1.63, 95%CI = [1.46, 1.82]; job tenure: d = 0.55, 95%CI = [0.33, 0.79]; job length: d = 0.46, 95%CI = [0.35, 0.57]; income: d = 0.48, 95%CI = [0.36, 0.59]) Non-vocational outcomes estimates, while favoring IPS, included the null (quality of life: d = 0.30, 95%CI = [-0.07, 0.67]; global functioning: d = 0.09, 95%CI = [-0.09, 0.27]; mental health: d = 0.03, 95%CI = [-0.15, 0.21]). Analysis of the expected proportion of studies with a true effect on non-vocational outcomes with d>0.2 showed some reason to expect a possible improvement for quality of life for at least some settings (Prop = 0.57, 95%CI = [0.30, 0.84]).

AI snapshot

Written from the paper’s full text.

TL;DR

This meta-analysis of randomized controlled trials found that individual placement and support (IPS) supported employment reliably improves vocational outcomes compared with usual treatment, while non-vocational outcomes were smaller and mostly included the null. A novel proportion-of-settings analysis suggested possible quality-of-life benefits in some settings.

Key findings

  • IPS increased any competitive employment (RR = 1.63, 95% CI [1.46, 1.82]) and end-of-study competitive employment (reported as 1.8x more likely; log(RR) = 0.58, 95% CI [0.42, 0.74]), and shortened time to first competitive employment (d = -0.31, 95% CI [-0.46, -0.15]).
  • Secondary vocational outcomes favored IPS: job tenure d = 0.55, 95% CI [0.33, 0.79]; job length d = 0.46, 95% CI [0.35, 0.57]; income d = 0.48, 95% CI [0.36, 0.59].
  • Non-vocational pooled estimates favored IPS but all confidence intervals included the null: quality of life d = 0.30, 95% CI [-0.07, 0.67]; global functioning d = 0.09, 95% CI [-0.09, 0.27]; mental health d = 0.03, 95% CI [-0.15, 0.21].
  • The estimated proportion of settings with a true quality-of-life effect d > 0.2 was 0.57, 95% CI [0.30, 0.84], suggesting possible benefit in some settings.

Methods

  • Searched PsycINFO; the primary query “supported employment AND randomized controlled trial” returned 111 results, a broader search returned 313, and after de-duplication 368 unique publications were screened. The final analysis included 30 articles: 25 original RCTs, 2 follow-up studies, and 3 secondary analyses comparing IPS with treatment as usual.
  • Random-effects meta-analyses were run in R using the metafor package; binary outcomes used log relative risk and continuous outcomes used standardized mean differences, with intent-to-treat analyses using randomized Ns.
  • The authors also estimated the proportion of settings with scientifically meaningful effects (RR > 1.2 or < 0.8; d > 0.2 or < -0.2) using the EValue library for samples with five or more articles.

Limitations

  • The paper states that included articles varied greatly in quality, that some non-significant results were reported without estimates and could not be included, and that sample sizes were often powered only for the main outcome, limiting secondary outcomes.
  • The paper reports substantial heterogeneity for some outcomes (e.g., quality of life I² = 94.90%; job tenure I² = 68.68%; mental health I² = 64.55%), and some analyses had few studies (job tenure k = 5; global functioning k = 5; mental health k = 6), making estimates imprecise.
  • Evident from the text: the search and data extraction were done by the first author alone, and the discussion notes that cost-benefit evidence for SE/IPS is mixed and that SE/IPS is “not a panacea.”

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1371/journal.pone.0212208

Cites

72 of 72 references verified0 citation errors · 0 not checkable

Statistics

18 tests recomputed · 0 with the reported and recomputed p on opposite sides of .05

Claims the paper makes

0 of 33 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    For job tenure, the estimated proportion of studies with a scientifically meaningful effect d>0.2 was 0.90 (95% CI [0.68, 1]).

    “We estimated that scientific meaningful effects that we expect to be observed with d>0.2 was 0.90 (95%CI = [0.68, 1])”
    Quote in the paperNot a substantial claim
  • Findingown data

    The estimated proportion of settings with effect size d < -0.2 for total time worked in competitive employment was essentially zero (Prop < 0.001, 95% CI [0, <0.001]).

    “and the proportion with d<-0.2 to be essentially zero (Prop = <0.001, 95%CI = [0, <0.001]).”
    Quote in the paperNot a substantial claim
  • Findingown data

    There was significant heterogeneity in the underlying study populations for competitive employment outcomes.

    “Our estimates showed that there was significant heterogeneity in the underlying study populations.”
    Quote in the paperNot a substantial claim
  • Findingown data

    For income from competitive employment, participants in IPS conditions had significantly more income than controls (k = 12, d = 0.48, 95% CI [0.36, 0.59], p < 0.0001).

    “We estimated that those in IPS conditions would have significantly more income (Fig 5, k = 12; d = 0.48, SE = 0.06, Z = 8.31, p<0.0001, 95%CI = [0.36, 0.59], τ2 (SE) = 0.0004(0.02), I2 = 0.90%, H2 = 1.01, Q(11) = 8.3404, p = 0.6825).”
    • Also stated in: § Discussion · plain
      “and higher total income (d = 0.48, 95%CI = [0.36, 0.59], Fig 5).”
    Quote in the paperSubstantial claim
  • Findingown data

    Length was not a significant moderator of the effect on income from competitive employment (Length = -0.004, 95% CI [-0.01, 0.005]).

    “Length was not a significant moderator (Length = -0.004, 95%CI = [-0.01, 0.005]).”
    Quote in the paperNot a substantial claim
  • May combine two claims. 15
  • Classified as probably not a substantial claim. 24
  • May combine two claims.

    Findingown data

    In randomized controlled trials of IPS, the non-vocational outcome estimate for global functioning favored IPS but included the null (d = 0.09, 95% CI [-0.09, 0.27]).

    “global functioning: d = 0.09, 95%CI = [-0.09, 0.27]”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    In randomized controlled trials of IPS, the non-vocational outcome estimate for mental health favored IPS but included the null (d = 0.03, 95% CI [-0.15, 0.21]).

    “mental health: d = 0.03, 95%CI = [-0.15, 0.21]”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingreasoning

    Because researchers often did not report estimates for non-significant results, the authors were unable to include all outcomes that researchers said they had measured.

    “Because of this, we were unable to include all outcomes that researchers said that they had measured.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    For global functioning, participants in IPS conditions had slightly higher global functioning than controls, but the confidence interval included the null (k = 5, d = 0.09, 95% CI [-0.0891, 0.2731], p = 0.3194).

    “We estimated that those in IPS had a slightly higher global functioning, but that the confidence intervals of the estimate include the null (Fig 7A, k = 5, d = 0.09, SE = 0.09, Z = 0.9957, p = 0.3194, 95%CI = [-0.0891, 0.2731]).”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    For quality of life, participants in IPS conditions had higher levels than controls, but the confidence interval included the null (k = 9, d = 0.30, 95% CI [-0.07, 0.67], p = 0.11).

    “We estimated that those in IPS conditions had a higher levels of QOL, but that the confidence intervals for the estimate included the null (Fig 6, k = 9, d = 0.30, SE = 0.19, Z = 1.60, p = 0.11, 95%CI = [-0.07 0.67], τ2 (SE) = 0.2955(0.1602), I2 = 94.90%, H2 = 19.59, Q(8) = 62.1751, p<0.0001).”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    For general mental health, IPS had a very small effect and the confidence interval included the null (k = 6, d = 0.03, 95% CI [-0.15, 0.21], p = 0.7399).

    “We estimated a very small effect and that the confidence intervals of the estimate include the null (Fig 7B, k = 6, d = 0.03, SE = 0.09, Z = 0.3320, p = 0.7399, 95%CI-0.15, 0.21], τ2 (SE) = 0.03(0.03), I2 = 64.55%, H2 = 2.82, Q(5) = 15.67, p = 0.0078).”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    Scale may be a significant moderator of the quality-of-life effect, with the Personal Wellbeing Index having the highest estimate.

    “Given the heterogeneity and the different scales used, we also tested to see if scale was a significant moderator and we estimated that it may be with the Personal Wellbeing Index having the highest estimate (Table 11).”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    A prior meta-analysis found no significant effect on mental health symptoms (d = -0.20, 95% CI [-0.62, 0.23]).

    “or mental health symptoms (d = -0.20, 95%CI = [-0.62, 0.23]).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [53] RM van Rijn, 2016, Work as treatment? The effectiveness of re-employment programmes for unemployed persons with severe mental health problems on health and quality of life: a systematic review and meta-analysis, Occupational and Environmental Medicine source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Past meta-analyses estimated that supported employment increases the likelihood of finding any employment by 3.24-fold (RR = 3.24, 95% CI [2.17, 4.82]).

    “and of finding any employment by 3.24-fold (RR = 3.24 95%CI = [2.17, 4.82]) [15].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [15] Y Kinoshita, 2013, Supported employment for adults with severe mental illness, The Cochrane database of systematic reviews

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    For reproducibility, the relevant data were saved as R data files and included as supplemental materials along with the needed R script file.

    “For reproducibility, we saved the relevant data as R data files and have included them as supplemental materials along with the needed R script file (S2 and S3 Files, respectively).”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Study design and conductown data

    All articles included in the meta-analysis reported randomized controlled trials comparing supported employment as implemented via IPS to a control condition, which was usually the locally available best usual practice.

    “All articles reported the results of randomized controlled trials (RCT) that focused on comparing supported employment (SE) as implemented via individual placement and support (IPS) to a control condition, which was usually the locally available best usual practice.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    A prior meta-analysis found no significant effect on global functioning (d = -0.01, 95% CI [-0.13, -0.11]).

    “but no significant effects on global functioning (d = -0.01, 95%CI = [-0.13, -0.11])”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [53] RM van Rijn, 2016, Work as treatment? The effectiveness of re-employment programmes for unemployed persons with severe mental health problems on health and quality of life: a systematic review and meta-analysis, Occupational and Environmental Medicine source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    A prior meta-analysis found a significant effect on quality of life, with those in IPS having a higher level (d = 0.28, 95% CI [0.04, 0.52]).

    “The researchers found a significant effect on quality of life with those in IPS having a higher level (d = 0.28, 95%CI = [0.04, 0.52])”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [53] RM van Rijn, 2016, Work as treatment? The effectiveness of re-employment programmes for unemployed persons with severe mental health problems on health and quality of life: a systematic review and meta-analysis, Occupational and Environmental Medicine source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Articles varied by whether or not they reported measures of fidelity to the study protocol.

    “Articles also varied by whether or not they reported measures of fidelity to the study protocol.”
    Quote in the paperNot a substantial claim
  • Classified as probably not a substantial claim.

    Study design and conductnone given

    If a study reported both intent-to-treat and conditional statistics for secondary vocational outcomes, the intent-to-treat values were selected.

    “If a study reported both intent-to-treat and conditional statistics, we selected to use the intent-to-treat values.”
    Quote in the paperSubstantial claim
  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductnone given

    For secondary vocational outcomes, meta-analysis was performed only from papers that used competitive employment for these statistics, and statistics that combined all types of employment were omitted.

    “We only performed meta-analysis from papers that only used competitive employment for these statistics. We omit from analysis statistics that combined all types of employment (i.e., competitive and not).”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Nordt et al. estimated that one in five suicides might be attributable to unemployment for the period 2000-2011 within sixty-three countries sampled.

    “In that study, Nordt et al. estimated that one in five suicides might be attributable to unemployment for the period of 2000–2011 within sixty-three countries sampled [8].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [8] C Nordt, 2015, Modelling suicide and unemployment: a longitudinal analysis covering 63 countries, 2000–11, The Lancet Psychiatry source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Researchers measured the same outcome variable in a number of ways; for example, competitive employment was often measured as any competitive employment during the study period and/or employment at the end of the study.

    “Researchers measured the same outcome variable in a number of ways. For example, competitive employment was often measured as having had any competitive employment during the study period and/or as having been employed at the end of the study.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Past meta-analyses estimated that supported employment increases the likelihood of achieving competitive employment by 2.4-fold (RR = 2.40, 95% CI [1.99, 2.90]).

    “Past meta-analyses have estimated that supported employment increases the likelihood of achieving competitive employment by 2.4-fold (RR = 2.40 95%CI = [1.99, 2.90]) [14]”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [14] M Modini, 2016, Supported employment for people with severe mental illness: Systematic review and meta-analysis of the international evidence, Br J Psychiatry source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    The research articles included in the review varied greatly in quality.

    “In general, research articles varied greatly in quality, as has been previously reported [15].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [15] Y Kinoshita, 2013, Supported employment for adults with severe mental illness, The Cochrane database of systematic reviews

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductown data

    The analyses followed the intent-to-treat paradigm, using the total number of subjects randomized for the Ns needed to be specified.

    “Finally, given that we had access to the total number of subjects randomized, we performed the analyses following the intent-to-treat paradigm.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Employment may be associated with positive outcomes such as increased physical and mental health.

    “Employment may be associated with positive outcomes such as increased physical and mental health [11–13].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [11] RE Lucas, 2004, Unemployment Alters the Set Point for Life Satisfaction, Psychological Science source

      Not checked in the free run.

    • cites [12] 2012, Association of Returning to Work With Better Health in Working-Aged Adults: A Systematic Review, Am J Public Health source

      Not checked in the free run.

    • cites [13] 2006, Is Work Good for your Health and Well-being?

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Unemployment may be associated with negative outcomes, such as suicide in the most extreme.

    “Unemployment may be associated with negative outcomes [9, 10], such as suicide, in the most extreme.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [9] F McKee-Ryan, 2005, Psychological and physical well-being during unemployment: a meta-analytic study, J Appl Psychol source

      Not checked in the free run.

    • cites [10] K Paul, 2009, Unemployment impairs mental health: Meta-analyses, J Vocat Behav source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    A recent study using mediation analysis suggests improvement in quality of life may be fully mediated via competitive employment.

    “A recent study using mediation analysis suggest improvement in quality of life may be fully mediated via competitive employment [54].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [54] D Jäckel, 2017, Effects of Sustained Competitive Employment on Psychiatric Hospitalizations and Quality of Life, Psychiatric Services

      Not checked in the free run.

  • May combine two claims.

    Interpretationown data

    There is some evidence that IPS may also improve quality of life and maybe global functioning, but more research with larger sample sizes is needed to know for sure.

    “Furthermore, there is some evidence that IPS may also improve quality of life and maybe global functioning, but more research with larger sample sizes are needed to know for sure.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Skills training in conjunction with IPS increased the test group's knowledge of fundamental concepts of the workplace (e.g., job performance).

    “They found that while the test group increased their knowledge of fundamental concepts of the workplace (e.g., job performance), they did not differ from controls on a host of other outcomes.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [66] KT Mueser, 2005, The effectiveness of skills training for improving outcomes in supported employment, Psychiatric services (Washington, DC) source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationreasoning

    In the limit, IPS could be provided to everyone, including those without any noticeable or known disability.

    “The answer appears to be “yes” that, in the limit, IPS could be provided to everyone.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    If there is a true effect of IPS on quality of life and other non-vocational outcomes, a possible route of causation could be via competitive employment.

    “If this is a true effect, a possible route of causation for increased quality of life and other non-vocational outcomes could be via the competitive employment.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Interpretationreasoning

    Given the effectiveness of IPS, additional groups could benefit; the answer appears to be yes from the above results and past literature.

    “Given the effectiveness of IPS, are there additional groups that could benefit. The answer appears to be a ‘yes’ from our above results and past literature.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: SE and IPS were originally developed for and used with populations that had severe mental illness.
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Skills training in conjunction with IPS did not differ from controls on a host of other outcomes.

    “They found that while the test group increased their knowledge of fundamental concepts of the workplace (e.g., job performance), they did not differ from controls on a host of other outcomes.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [66] KT Mueser, 2005, The effectiveness of skills training for improving outcomes in supported employment, Psychiatric services (Washington, DC) source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Previous research reported that for populations with schizophrenia or schizoaffective disorder, IPS proved more effective than treatment as usual as measured by subjects finding competitive employment (57% vs. 29%).

    “Previous research reported that for populations with schizophrenia or schizoaffective disorder, IPS proved more effective than TAU as measured by subjects finding competitive employment (57% vs. 29%)[44].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [44] EW Twamley, 2012, The efficacy of supported employment for middle-aged and older people with schizophrenia, Schizophrenia research source

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductown data

    For the income analyses, the authors used only values derived from competitive employment; if the reported statistic was from income of competitive and non-competitive employment sources, they did not use it.

    “For the following analyses, as above, we used only values that were derived from competitive employment (i.e., if the reported statistic was from income of competitive and non-competitive employment sources, we did not use it).”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Premisecitation

    For those who did not respond to IPS, cognitive enhancement treatment had higher employment (60% vs. 36%), worked longer (23.9 vs. 9.2 weeks), and earned more ($3,421 vs. $1,728) than enhanced SE controls.

    “Most recently, they found that for those that didn’t respond to IPS, the individual receiving cognitive enhancement treatments (the “Thinking Skills for Work Program”) had higher employment (60% vs. 36%), worked longer (23.9 vs. 9.2 weeks), and earned more ($3,421 vs. $1,728) than the control group that had an enhanced version of SE [65].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [65] SR McGurk, 2015, Cognitive Enhancement Treatment for People With Mental Illness Who Do Not Respond to Supported Employment: A Randomized Controlled Trial, The American journal of psychiatry source

      Not checked in the free run.

  • May combine two claims.

    Interpretationown data

    The result that 0.57 of settings would have an effect size d > 0.2 for quality of life is suggestive that there may be a significant effect of IPS on these outcomes, but that the effect is sensitive to the setting.

    “This latter result is suggestive that there may be a significant effect of IPS on these outcomes, but that the effect is sensitive to the setting.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    There may be a difference in structural white matter between bipolar and unipolar depression.

    “For example, there may be a difference in the structural (white matter) between bipolar and unipolar depression [63].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [63] G Serafini, 2014, Brain changes in early-onset bipolar and unipolar depressive disorders: a systematic review in children and adolescents, Eur Child Adolesc Psychiatry source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    Given heterogeneity and the different scales used, the authors tested whether scale was a significant moderator of the quality-of-life effect.

    “Given the heterogeneity and the different scales used, we also tested to see if scale was a significant moderator”
    Quote in the paperNot a substantial claim
36 of 36
  • The paper relies on this without support.

    Working is better than not working.

    “In our society, it is generally thought that working is better than not working.”
  • The paper relies on this without support.

    SE and IPS were originally developed for and used with populations that had severe mental illness.

    “SE and IPS were originally developed for and used with populations that had severe mental illness.”
  • The paper relies on this without support.

    Subjects still work below full time and still suffer from their underlying disorder.

    “Subjects still work below full time and still suffer from their underlying disorder.”
3 of 3

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Statistics that don’t add up

Each reported p-value recomputed from its test statistic and degrees of freedom.

  • Inconsistent p-value: Q(22) = 29.64, p = .1277

    Minor

    The paper reports p = .1277 here; recomputing from the test statistic and its degrees of freedom gives p = .127. The check cannot tell whether the statistic or the p-value is the misprint.

    Reported p = .1277 · recomputed p = .127

1 of 1

Statistics recomputed

Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.

  • Doesn’t add upQ(22) = 29.64reported p = .1277 · recomputed p = .127

    “Q(22) = 29.64, p = .1277”

  • Consistentz = 8.7reported p < .0001 · recomputed p < .001

    “Z = 8.70, p < .0001”

  • Consistentz = 7.23reported p < .0001 · recomputed p < .001

    “Z = 7.23, p < .0001”

  • ConsistentQ(12) = 14.93reported p = .2451 · recomputed p = .245

    “Q(12) = 14.93, p = .2451”

  • Consistentz = -3.93reported p < .0001 · recomputed p < .001

    “Z = -3.93, p < .0001”

  • ConsistentQ(8) = 5.8994reported p = .6585 · recomputed p = .658

    “Q(8) = 5.8994, p = .6585”

  • Consistentz = 1.42reported p = .16 · recomputed p = .156

    “Z = 1.42, p = .16”

  • Consistentz = 4.72reported p < .0001 · recomputed p < .001

    “Z = 4.72, p < .0001”

  • ConsistentQ(7) = 21.477reported p = .0031 · recomputed p = .003

    “Q(7) = 21.4770, p = .0031”

  • Consistentz = 8.23reported p < .0001 · recomputed p < .001

    “Z = 8.23, p < .0001”

  • Consistentz = 8.31reported p < .0001 · recomputed p < .001

    “Z = 8.31, p < .0001”

  • ConsistentQ(11) = 8.3404reported p = .6825 · recomputed p = .683

    “Q(11) = 8.3404, p = .6825”

  • Consistentz = 1.6reported p = .11 · recomputed p = .110

    “Z = 1.60, p = .11”

  • ConsistentQ(8) = 62.1751reported p < .0001 · recomputed p < .001

    “Q(8) = 62.1751, p < .0001”

  • Consistentz = 0.9957reported p = .3194 · recomputed p = .319

    “Z = 0.9957, p = .3194”

  • Consistentz = 0.332reported p = .7399 · recomputed p = .740

    “Z = 0.3320, p = .7399”

  • ConsistentQ(5) = 15.67reported p = .0078 · recomputed p = .008

    “Q(5) = 15.67, p = .0078”

  • Consistentz = 1.46reported p = .14 · recomputed p = .144

    “Z = 1.46, p = .14”

18 of 18

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Drake 1996 employment count exceeds the randomized group

    Major

    Table 5 reports 526 IPS participants obtaining competitive employment in Drake 1996, but Table 2 says only 74 participants were randomized to IPS in that study.

  • Quoted prior-study confidence interval contradicts its significance description

    Minor

    The text describes the prior global-functioning result as nonsignificant, but the quoted 95% confidence interval lies entirely below zero. The quoted point estimate also falls outside that interval.

  • Full-text exclusion totals disagree

    Minor

    The flow-chart description says 24 articles were excluded after full-text review, whereas the accompanying account says 26 were removed at that stage.

  • Job-tenure study count disagrees with the heterogeneity degrees of freedom

    Minor

    The job-tenure analysis reports five studies, but its Q statistic has seven degrees of freedom, corresponding to eight studies in the stated meta-analysis. Table 7 also lists eight studies.

4 of 4

Cites

Flagged first, then by citations

1 of the 73 references couldn’t be matched to a known paper.

Research works printed but not linked: 1

  1. GR Bond, 1997, A fidelity scale for the Individual Placement and Support model of supported employment, Rehabilitation Counseling Bulletin, 40, 265

Cited by

Sorted by citations

Authors
Frederick, D. E., & VanderWeele, T. J.
Affiliations
Harvard University (US)
Corresponding author
Frederick, Donald E.
Venue
PLOS ONE
Published
2019-02-20
Volume
14
Issue
2
Pages
e0212208
ISSN
1932-6203 (ISSN-L)
OpenAlex
W2915389400
Cited by
219
References
73
Citations by year
2026: 26 · 2025: 31 · 2024: 41 · 2023: 32 · 2022: 39 · 2021: 25 · 2020: 18 · 2019: 6 · 2017: 1
FWCI
5.78
Type
journal article
Language
en
Open access
Locations
  • PLoS ONE (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • Figshare (submittedVersion, cc-by)
Topics
  • Social Sciences › Safety Research › Disability Education and Employment
  • Health Sciences › Medicine › Psychiatry and Mental health › Schizophrenia research and treatment
  • Health Sciences › Health Professions › General Health Professions › Employment and Welfare Studies
Keywords
supported employment, Individual Placement and Support, severe mental illness, competitive employment, career outcomes, job tenure, income, quality of life, global functioning, mental health, randomized controlled trials
MeSH terms
Mental Disorders, Female, Humans, Income, Male, Mental Health, Quality of Life, Randomized Controlled Trials as Topic, Employment, Supported

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-10.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-10.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review