CoChat
Sign Up
Cited by 179Mindfulness and Compassion InterventionsDigital Mental Health InterventionsAnxiety, Depression, Psychometrics, Treatment, Cognitive Processes

Feasibility of a Prototype Web-Based Acceptance and Commitment Therapy Prevention Program for College Students.

Open access
Levin, Michael E., Pistorello, Jacqueline, Seeley, John R., Hayes, Steven C.·2014·Journal of American College Health
DOI 10.1080/07448481.2013.843533PMC PMC3858823
Open this paper in CoChat — free account

Abstract

Objective: This study examined the feasibility of a prototype Web-based acceptance and commitment therapy (ACT) program for preventing mental health problems among college students.Participants: Undergraduate first-year students (N= 76) participated between May and November 2011.Methods: Participants were randomized to ACT or a waitlist, with assessments conducted at baseline, posttherapy, and 3-week follow-up. Waitlist participants accessed the program after the second assessment.Results: Program usability/usage data indicated high program acceptability. Significant improvements were found for ACT knowledge, education values, and depression with ACT relative to waitlist. Subgroup analyses indicated that ACT decreased depression and anxiety relative to waitlist among students with at least minimal distress. Within the ACT condition, significant improvements were observed from baseline to 3-week follow-up on all outcome and process measures.Conclusions: Results provide preliminary support for the feasibility of a Web-based ACT prevention program.

Integrity signals

Registry

Crossref · 10.1080/07448481.2013.843533

Cites

44 of 44 references verified0 citation errors · 0 not checkable

Statistics

13 tests recomputed · 0 with the reported and recomputed p on opposite sides of .05

Claims the paper makes

12 of 63 cited sources checked · 5 supported · 0 flagged · accuracy not yet measured

  • Premisecitation

    A substantial literature indicates psychological flexibility is a common protective factor in the development and exacerbation of many psychological problems, including depression, anxiety, substance abuse, eating disorders, stress, adjustment to trauma, adjustment to chronic medical conditions, school performance, employment performance, and burnout.

    “There is a substantial literature base indicating psychological flexibility as a common protective factor in the development and exacerbation of a variety of psychological problems including depression, anxiety, substance abuse, eating disorders, stress, adjustment to traumatic experiences, adjustment to chronic medical conditions, school performance, employment performance, and burnout.18–20”
    Quote in the paperSubstantial claim1 of 3 sources checked
    • cites [18] source
    • cites [19] source

      The Research on ACT section (pp. 12–13) reports that greater psychological flexibility is associated with better mental-health outcomes and predicts better subsequent mental health, job performance, and outcomes in chronic pain. The claim extends that evidence to a specific list of problems; the passages do not establish the protective role for every item, notably eating disorders and school performance, and the paper characterizes much of the evidence as correlational. (LLM, conf=0.85)

      Experimental — accuracy not yet measured
    • cites [20]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Premisecitation

    Early efforts to create and test these programs have shown promise.

    “Researchers have begun to develop and evaluate such programs, with promising results.15,16”
    Quote in the paperSubstantial claim1 of 2 sources checked
    • cites [15] source
    • cites [16] source
  • Premisecitation

    Psychological flexibility is a possible target for a transdiagnostic prevention approach.

    “A powerful possible target for a transdiagnostic prevention approach is psychological flexibility,17–19”
    Quote in the paperSubstantial claim2 of 3 sources checked
    • cites [17] source
    • cites [18] source
    • cites [19] source

      The ACT/RFT theory of psychopathology (p. 6) presents psychological inflexibility as a broad source of psychological problems, and Six core processes of ACT (p. 7) says ACT targets those problems by increasing psychological flexibility. Research on ACT (p. 12) also suggests flexibility affects subsequent mental health. These passages support it as a possible target across problems, but our copy does not specifically propose a transdiagnostic prevention approach. (LLM, conf=0.82)

      Experimental — accuracy not yet measured
  • Findingown data

    Usage and usability data from the prototype indicated high acceptability.

    “Program usability/usage data indicated high program acceptability.”
    Quote in the paperSubstantial claim
  • Findingown data

    ACT produced significant improvements in education values relative to waitlist.

    “Significant improvements were found for ACT knowledge, education values and depression with ACT relative to waitlist.”
    Quote in the paperSubstantial claim
  • May combine two claims. 34
  • Classified as probably not a substantial claim. 32
  • The quote only partly states this claim. 1

  • May combine two claims.

    Premisecitation

    Psychological flexibility and experiential avoidance can be targeted by acceptance and mindfulness interventions such as ACT, producing improvements across various psychological problems.

    “Psychological flexibility/experiential avoidance can be targeted through acceptance and mindfulness-based interventions such as Acceptance and Commitment Therapy (ACT)19 to produce improvements across a range of psychological problems.”
    Quote in the paperSubstantial claim1 of 1 sources checked

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    About 2 to 4% of young adult women develop an eating disorder, with onset peaking between ages 16 and 20.

    “Approximately 2 to 4% of young adult females acquire an eating disorder, with the peak age of onset between 16 and 20 years of age.4,5”
    Quote in the paperNot a substantial claim1 of 2 sources checked
    • cites [4] source
    • cites [5] source

      The Prevalence Studies Using Self-Report Instruments section (p. 4) reports that 3.8% of women in a student sample met criteria for binge eating disorder, which lends limited support to the percentage. But this is prevalence of a specific disorder, not the proportion of young adult women who acquire any eating disorder. The passages on age (pp. 5–6) do not establish a peak onset between 16 and 20 years. (LLM, conf=0.83)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Study design and conductcitation

    Prior work indicates the DASS subscales are not unidimensional, so an overall total score is not appropriate.

    “Previous studies have found that these subscales do not load onto a single latent variable and thus cannot be calculated as an overall total score.36”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [36] source

      Table 4 (page unavailable) reports separate correlations for the DASS depression, anxiety, and stress subscales. The factor analyses in Participants and procedure and Results and Discussion concern the AAQ-II, not whether the DASS subscales load onto one latent variable. A search for DASS factor structure and an overall total score found no passage addressing that claim. (LLM, conf=0.76)

      Experimental — accuracy not yet measured

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    The DASS has shown good reliability and validity in past research.

    “The DASS has been found to have good reliability and validity in past research.36,37”
    Quote in the paperNot a substantial claim1 of 2 sources checked
    • cites [36] source

      The Measures section (page unavailable) includes the DASS among measures described as well-validated and widely used, supporting the validity part of the claim. It does not state that the DASS has good reliability; the reliability passages concern the AAQ-II instead. (LLM, conf=0.84)

      Experimental — accuracy not yet measured
    • cites [37]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Premisecitation

    Prevention programs with evidence of effectiveness are available for eating disorders, harmful drinking, and depression and anxiety.

    “Effective prevention programs exist for eating disorders,10 harmful alcohol use,11 and depression and anxiety disorders.12”
    Quote in the paperSubstantial claim1 of 3 sources checked
    • cites [10] source
    • cites [11] source
    • cites [12] source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    20% of college students have an alcohol use disorder.

    “Severe drinking problems are also common, with over 45% engaging in binge drinking3 and 20% experiencing an alcohol use disorder.1”
    Quote in the paperNot a substantial claim1 of 1 sources checked

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Almost half of college students have a diagnosable psychiatric disorder in a given year.

    “Mental health problems are highly prevalent in college with almost 50% of college students having a diagnosable psychiatric disorder (e.g., depressive, anxiety, substance use, eating disorders) within a given year.1”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [1] source

      The Results report that almost one-half of college students met DSM-IV criteria for at least one psychiatric disorder in the previous year; Table 2 gives the figure as 45.79%. The Method section lists mood, anxiety, and substance use disorders among the diagnoses assessed, but not eating disorders, so that example in the citing sentence is not supported by this paper. (LLM, conf=0.86)

      Experimental — accuracy not yet measured

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    Past research with college students found adequate internal consistency for DASS depression, anxiety, and stress subscales with alphas of .87, .79, and .84.

    “Past research with college students indicates adequate internal consistency with Cronbach’s alpha of .87, .79 and .84 for the depression, anxiety and stress subscales.38”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [38] source

      The passages from the paper’s References section concern depression-prevention studies, not the DASS subscales. Searches for the DASS alpha values and for internal consistency in college students found no passage reporting the claimed values. (LLM, conf=0.76)

      Experimental — accuracy not yet measured

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    When asked what they learned, 54% described content from the values lesson and 46% from the acceptance lesson.

    “Participants appeared to be split when asked “What was the most important thing you learned from this program?” with 54% describing content from the values lesson and 46% from the acceptance lesson.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    58.1% of participants logged on twice, 21.6% three times, and 16.3% four to five times.

    “The majority logged onto the program twice (58.1%), with most others logging in 3 (21.6%) or 4 to 5 (16.3%) times.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Participants took an average of 3.89 days (SD = 2.57) to complete each lesson, and 85% completed both lessons within the 3-week intervention period.

    “Although participants took 3.89 days on average to complete each lesson after it was made available (SD = 2.57), 85% completed both lessons within the designated 3-week intervention period.”
    Quote in the paperSubstantial claim

  • The quote only partly states this claim.

    Findingown data

    The mean SUS usability rating for ACT-CL was 84.55 (SD = 10.91).

    “The mean usability rating of ACT-CL on the SUS combining both ACT-CL and waitlist participants was 84.55 (SD = 10.91).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    85.3% reported reading the tailored emails, and 69.0% of those readers reported engaging in the suggested exercises.

    “Regarding the tailored emails, 85.3% reported reading the emails and 69.0% of those who read the emails reported engaging in the suggested exercises.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    For ACT knowledge, HLM failed to converge, but MMRM found significant improvement over time, F(2, 33.28) = 29.56, p < .001, d = 1.88.

    “HLM failed to converge for ACT knowledge, but MMRM analysis found a significant improvement over time on ACT knowledge, F(2, 33.28) = 29.56, p < .001, Cohen’s d = 1.88.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Within-group findings were not replicated in the waitlist condition after using the website, with only ACT knowledge improving significantly from pre to post.

    “However, within group findings were not replicated in the waitlist condition after using the website, with only ACT knowledge improving significantly from pre to post.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Among distressed students, a significant time by condition interaction was found for depression, F(1, 30) = 6.23, p = .018, d = .91, with greater reductions in ACT-CL than waitlist.

    “and depression, F(1, 30) = 6.23, p = .018, Cohen’s d = .91, in the ACT-CL condition relative to waitlist.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    The ACT knowledge measure drew on prior knowledge questionnaires from ACT bibliotherapy research.

    “The ACT knowledge questionnaire was based on existing knowledge questionnaires used in ACT bibliotherapy studies.29”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [29] source

  • May combine two claims.

    Premisecitation

    People who tend to avoid unpleasant thoughts and feelings may have few problems without stress, but under stress they may engage in self-amplifying suppression that worsens their problems.

    “In the absence of stress, people prone to avoiding unpleasant thoughts and feelings may have few problems. However, when encountering stress (such as developmental, social, financial, or academic stressors), they may lock into self-amplifying efforts to suppress unwanted thoughts and feelings, thus exacerbating their problems.18”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [18] source

  • May combine two claims.

    Interpretationcitation

    ACT appears effective in non-clinical samples and may reduce the onset of psychological problems.

    “Thus, ACT appears to be effective with non-clinical samples and can potentially reduce the onset of psychological problems.29”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [29] source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    One in eight college students have had an anxiety disorder in the past 12 months.

    “Nearly one-fifth of college students have suffered from depression in their lifetime and one-eighth from anxiety disorders in the last 12 months.2”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [2]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductcitation

    Research across 206 studies indicates SUS items load onto one latent factor, have high internal consistency (alpha .91), and can distinguish more from less usable programs.

    “Research summarized across 206 studies indicates that the SUS items load onto a single latent factor, have a high level of internal consistency (Cronbach’s alpha = .91), and that the measure can be used to distinguish between more and less usable program.34”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [34] source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The ACT-CL material drew on established ACT treatment and self-help protocols for several mental health issues, including non-clinical worksite and college wellness programs.

    “Program content was adapted from empirically validated ACT treatment and self-help protocols for a variety of mental health problems, including non-clinical worksite and college student wellness programs.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    More than 45% of college students engage in binge drinking.

    “Severe drinking problems are also common, with over 45% engaging in binge drinking3 and 20% experiencing an alcohol use disorder.1”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [3] source

  • May combine two claims.

    Premisecitation

    Nearly every higher education institution already has ways to support freshmen transition, which could serve as a framework for a new prevention program.

    “Due in part to these issues, almost all higher education institutions have existing mechanisms to help freshmen with the transition to college, which can provide a framework for implementing a new prevention program.32”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [32] source

  • Classified as probably not a substantial claim.

    Study design and conductown data

    Participants were randomized to ACT-CL or waitlist.

    “Participants were then randomized to the ACT-CL (n = 37) or waitlist condition (n = 39).”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    College campuses face high demand for counseling services, making it hard to meet the needs of all students.

    “Given the large numbers of students who experience psychiatric disorders, the demand for counseling services on college campuses is high, which has created a challenge for campuses to meet the needs of the student body.8,9”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [8] source
    • cites [9] source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Many freshmen drop out for emotional or psychological reasons.

    “In addition, many freshmen dropout for emotional/psychological reasons.31”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [31] source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    Empirically derived cutoff scores for the SUS identify below and above average program usability ratings.

    “Empirically derived cutoff scores have been established for the SUS to identify below and above average program usability ratings.35”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [35] source

  • May combine two claims.

    Premisecitation

    A recent study found an ACT self-help book improved mental health and reduced incidence of clinical distress at 2-month follow-up compared with waitlist among native Japanese students in US higher education.

    “a recent study found that an ACT self-help book produced improvements in mental health as well as reduced the incidence of clinical levels of distress at 2-month follow up relative to a waitlist condition among native Japanese students attending higher education in the US.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [29] source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Among college students, suicide ranks as a leading cause of death.

    “Furthermore, suicide is a leading cause of death among college students.7”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [7] source

  • May combine two claims.

    Interpretationreasoning

    A comprehensive set of disorder-specific programs would be complicated, expensive, and burdensome for students.

    “Implementing a full range of problem-specific programs would be complex, costly, and over-burden students.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    All participants completed the second web survey three weeks after baseline.

    “All participants completed a second web-based survey 3 weeks after baseline.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Nearly one in five college students have experienced depression in their lifetime.

    “Nearly one-fifth of college students have suffered from depression in their lifetime and one-eighth from anxiety disorders in the last 12 months.2”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [2]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationreasoning

    Web-based transdiagnostic programs reduce delivery cost, staff burden, and training needs while increasing dissemination capacity and participant convenience.

    “The cost of delivery, burden on staff resources, and training needs would all be reduced at the same time that the capacity to disseminate the program across the student body with greater convenience to participants would be increased.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    The SUS is a common 10-item measure of program usability and acceptability.

    “The SUS is a widely used 10-item scale designed to assess program usability and acceptability.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [33] source

  • May combine two claims.

    Interpretationown data

    High program utilization and system usability suggest the approach is potentially feasible and acceptable for first-year college students.

    “There was high program utilization and system usability with the prototype program, attesting to the potential feasibility and acceptability of this approach with first year college students.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductnone given

    The within-group data for ACT-CL included three time points, whereas the waitlist condition had only two.

    “Another explanation is that the within group data for the ACT-CL condition included 3 time points compared to only 2 time points in the waitlist condition.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationown data

    The need for replication is particularly important given the lack of improvements in the waitlist condition after viewing the website.

    “This is particularly important given the lack of improvements observed in the waitlist condition after viewing the website.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationcitation

    The lack of condition effects on psychological flexibility at post raises a concern.

    “The lack of condition effects on psychological flexibility/experiential avoidance at post raises a concern”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    A recent study with test anxiety found ACT was as effective as traditional cognitive behavioral methods for reducing anxiety but superior for improving academic test performance.

    “A recent study with test anxiety found that ACT did as well as traditional cognitive behavioral methods in reducing anxiety but was superior in improving academic test performance.41”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [41] source

  • May combine two claims.

    Study design and conductown data

    The sample size was too small to determine differences across institution types, although some students came from other schools including a community college.

    “Although a portion of students participated from other schools, including a local community college, the sample size was too small to determine if there were differences across types of institutions.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    Cognitive patterns can become overextended in ways that elicit and support experiential avoidance, and cognitive defusion interventions in ACT are particularly crucial for increasing psychological flexibility.

    “Cognitive patterns are known to become overextended in ways that elicit and support experiential avoidance (i.e., rumination, worry, problem solving), and cognitive defusion interventions in ACT are particularly crucial to increasing psychological flexibility (for a recent review of laboratory tests of cognitive defusion see Levin et al.42).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Levin et al.42

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationown data

    Within-group results in ACT-CL were positive, with medium to large effect sizes and continued improvement over the three-week follow-up.

    “The within group results in the ACT-CL condition were positive with medium to large effect sizes for all outcome and processes tested showing continued improvement over the three-week follow up.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    A full ACT program with multiple modules and mobile/email support holds promise as a transdiagnostic program for college students and institutions.

    “A full program, with multiple modules targeting all aspects of ACT and supported by mobile applications and email features, holds out promise for a transdiagnostic program that could meet the needs of college students and institutions of higher education alike.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductown data

    An intent-to-treat approach was used, including all dropouts in the analyses.

    “An intent-to-treat analytic approach was taken such that study dropouts were included in all analyses.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    Preliminary results suggest the program may impact at least some mental health problems, based on between-group differences in depression and large effect sizes for depression and anxiety among students with at least minimal symptoms.

    “Preliminary results also suggest the program may have an impact on at least some mental health problems given the between group differences on depression symptoms as well as large effect sizes for ACT-CL on depression and anxiety symptoms among those with at least minimal symptoms.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Study design and conductcitation

    The AAQ-II has adequate reliability and validity in past research, including non-clinical college samples, with alpha .84 and 3-month test-retest reliability .81.

    “The AAQ-II has been found to have adequate reliability and validity in past research, including in non-clinical college student samples, with a Cronbach’s alpha of .84 and a test-retest reliability of .81 at 3 month follow up.21”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [21] source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    One ACT-CL participant who experienced a major non-research-related adverse event was identified as an outlier and excluded from all reported analyses.

    “Outlier analyses identified an outlier in the ACT-CL condition who reported experiencing a major non-research related adverse event (accidental death of a close family member) during the program-testing phase. This participant was excluded from all reported analyses.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    Past research indicated adequate discriminant and convergent validity for the AAQ-II on measures of psychological distress, functioning, and related variables such as thought suppression.

    “Past research has also indicated adequate discriminant and convergent validity with the AAQ-II on measures of psychological distress, functioning, and related variables (i.e., thought suppression).21”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [21] source

  • May combine two claims.

    Interpretationcitation

    The mean SUS rating of 84.55 is above the 'A' score cutoff of 80.3, which represents the top 10% of program scores.

    “This is above the cutoff for an “A” score (80.3), which represents the top 10% of program scores and those programs which people are likely to recommend to their friends.35”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [35] source

  • May combine two claims.

    Interpretationown data

    The program may be acceptable to and impactful with college freshmen.

    “Results suggest that the program may be acceptable to, and impactful with, college freshmen.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Study design and conductown data

    There was some attrition: two participants (one per condition) dropped out after baseline, and one additional ACT-CL participant dropped out before the third survey.

    “There was some attrition across assessment phases. Two participants (one from each condition) dropped out of the study after completing the baseline survey, reportedly because they were too busy with school. An additional participant in the ACT-CL condition dropped out before completing the third web-based survey also citing time constraints.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductreasoning

    The pilot study was not designed to determine preventative effects and focused on a narrower range of mental health problems than the program's broader transdiagnostic goals.

    “The pilot study was not designed to determine preventative effects per se and focused on a more limited range of mental health problems than befits the broader transdiagnostic goals of the program.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    Because universities emphasize student retention and academic success, improving academic functioning may be an important institutional outcome beyond mental health prevention.

    “Given the emphasis at most universities on student retention and academic success, the ability to improve academic functioning may be an important institutional outcome above and beyond mental health prevention and amelioration.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductown data

    In the waitlist condition, the PVQ values variables showed 3-week test-retest reliability, with correlations of .55 for intrinsic/positive motivation for relationship values, .57 for education values, .45 for relationship values success, and .73 for education values success (all p < .001).

    “The 3 week test-retest reliability within the waitlist condition for the values variables used in the analyses were as follows: intrinsic/positive motivation for relationship values, r (36) = .55, p < .001, intrinsic/positive motivation for education values, r (36) = .57, p < .001, relationship values success, r (36) = .45, p < .001, education values success, r (36) = .73, p < .001.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    The lack of condition effects on psychological flexibility may be attributable to the limited treatment components in the prototype program.

    “This may be attributable to the limited treatment components included in the prototype program.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    The DASS subscales have been shown to separate clinical depression from anxiety presentations accurately.

    “Studies have found that the DASS subscales accurately distinguish depressed and anxious clinical presentations.37”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [37]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationreasoning

    The lack of replication in the waitlist condition may simply indicate a failure to replicate, which is concerning given the small sample and less stringent significance cutoff (p < .10).

    “Lastly, this may indicate simply a failure to replicate results, which is particularly a concern given the small sample size and less stringent significance cutoff (p < .10).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    ACT has been shown to improve innovation at work, persistence in difficult tasks, social functioning, and exercise.

    “ACT has also been shown to improve innovation at work,25 persistence in difficult tasks,42 social functioning,43 and exercise.44”
    Quote in the paperNot a substantial claim0 of 4 sources checked
    • cites [25] source
    • cites [42] source
    • cites [43]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [44]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Study design and conductreasoning

    The trial had a short follow-up, no follow-up in the control group, and a focus on proximal programmatic outcomes.

    “As a methodological reflection of that purpose, follow up was short, no follow up was collected in the control group, and there was a focus on proximal programmatic outcomes.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The full ACT model includes considerably more elements than the two modules tested.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    The AAQ-II had a Cronbach's alpha of .84 in the current sample.

    “The Cronbach’s alpha for the current sample was .84.”
    Quote in the paperNot a substantial claim
61 of 61

  • The paper relies on this without support.

    Higher education already has an infrastructure for preventive and ameliorative programming through student orientation and success programs.

    “An infrastructure exists in higher education to provide preventive and ameliorative programming through student orientation and success programs.”

  • The paper relies on this without support.

    Experiential avoidance is a key process affecting psychological flexibility.

    “A key process that impacts psychological flexibility is experiential avoidance, the unwillingness to experience difficult emotions, thoughts, or sensations, even when avoiding these experiences is harmful.”

  • The paper relies on this without support.

    Pilot studies tend to be underpowered because resources are split between development and feasibility testing.

    “Pilot studies such as the current one tend to be underpowered because resources and time have to be distributed between program development and conducting a feasibility trial to determine “proof of concept.””

  • The paper relies on this without support.

    The full ACT model includes considerably more elements than the two modules tested.

    “However, the full ACT model is composed of considerably more elements, including a focus on mindfulness, contact with the present moment, perspective taking, cognitive defusion (learning to notice a thought as a thought), and committed action.”
4 of 4

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The change in DASS-21 total score from baseline to post-assessment in this study was more significant than in the Levin study.

    “However, the change in the DASS-21 total score from baseline to post-assessment (–6.36) in our study was more significant than in the Levin study (–4.59).”

    From Effectiveness of an Internet-Based Acceptance and Commitment Therapy Intervention for Reducing Psychological Distress in Health Care Professionals: Randomized Controlled Trial (2024) · cites this paper as [53]

    The Methods > Measures passage says the DASS-21 subscales cannot be calculated as an overall total score. The Results > Between Group Analyses instead report depression, anxiety, and stress separately; our copy does not report a −4.59 change in a DASS-21 total score for the comparison claimed. (LLM, conf=0.88)

    Checked by a reasoning model — may be wrong.

    Experimental — accuracy not yet measured

  • The two studies testing ACT as a transdiagnostic treatment found positive effects on depression and anxiety, but did not assess a broader range of problems.

    “The two studies testing ACT as a transdiagnostic treatment found positive effects on depression and anxiety, but did not assess a broader range of problems (Levin et al., 2014;2015).”

    From Web-Based Acceptance and Commitment Therapy for Mental Health Problems in College Students: A Randomized Controlled Trial (2017) · cites this paper as Levin et al., 2014

    The Results (Between Group Analyses; page unavailable) report greater reductions in depression and anxiety for distressed students receiving ACT than for those on the waitlist, but no significant anxiety difference among nondistressed students. The Methods (Measures; page unavailable) show that the study also assessed stress, so it did not restrict assessment to depression and anxiety; it did not assess a wide range of other mental health problems. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

  • The AAQ-II may be less sensitive to detecting treatment effects in this context.

    “Some previous web-based ACT research with college students has similarly found a lack of effect with the AAQ-II suggesting it may be less sensitive to detecting treatment effects in this context (Levin et al., 2014).”

    From Web-Based Acceptance and Commitment Therapy for Mental Health Problems in College Students: A Randomized Controlled Trial (2017) · cites this paper as Levin et al., 2014

    The Methods (Measures; page unavailable) identify the AAQ-II as the measure of psychological flexibility and experiential avoidance. The Results (Between Group Analyses; page unavailable) report no significant between-group effect on that measure, but the paper does not suggest that the AAQ-II lacks sensitivity; the Comment instead raises the limited components of the prototype program as a possible explanation. (LLM, conf=0.85)

    Experimental — accuracy not yet measured

  • The findings corroborate post-treatment outcomes of recent studies among non-clinical populations, which showed the efficacy of ACT self-help interventions in increasing general mental health compared to a wait-list condition.

    “The findings corroborate post-treatment outcomes of recent studies among non-clinical populations, which showed the efficacy of ACT selfhelp interventions in increasing general mental health compared to a wait-list condition (Jeffcoat &amp; Hayes, 2012; Levin et al., 2014; Pots et al., 2016).”

    From A randomised controlled trial of brief web-based acceptance and commitment Therapy on the general mental health, depression, anxiety and stress of college Students (2022) · cites this paper as Levin et al., 2014

    The Results (Between Group Analyses; page not provided) report greater post-treatment improvement in depression with the web-based ACT program than with waitlist, and reduced depression and anxiety among students with at least minimal distress. They do not establish an increase in general mental health across the non-clinical sample: other between-group effects were not significant, and the Limitations describe the study as a pilot examining a limited range of mental health problems. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

  • Levin et al. (2014) found significant and large improvements in depression, anxiety, and stress symptoms for those exposed to a web-based ACT intervention compared to a wait-list control among 76 first-year college students.

    “Overall, initial results were promising with significant and large improvements in depression ( p = .001, d = 0. 97), anxiety ( p = .003, d = 0.95) and stress ( p = .001, d = 0.81) symptoms for those exposed to the intervention compared to the control group.”

    From A randomised controlled trial of brief web-based acceptance and commitment Therapy on the general mental health, depression, anxiety and stress of college Students (2022) · cites this paper as Levin et al., 2014

    The Results (Between Group Analyses) report significant, large reductions in depression (p = .018, d = .91) and anxiety (p = .033, d = .81) relative to waitlist, but only among students with at least minimal distress—not all 76 participants. The paper reports no significant between-group difference in stress in either subgroup, and its reported statistics do not match those in the citing sentence. (LLM, conf=0.93)

    Experimental — accuracy not yet measured

Statistics that don’t add up

Each reported p-value recomputed from its test statistic and degrees of freedom.

  • Inconsistent p-value: r (36) = .45, p < .001

    Minor

    The paper reports p < .001 here; recomputing from the test statistic and its degrees of freedom gives p = .005 (two-tailed). The check cannot tell whether the statistic or the p-value is the misprint.

    Reported p < .001 · recomputed p = .005

1 of 1

Statistics recomputed

Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.

  • Doesn’t add upr(36) = 0.45reported p < .001 · recomputed p = .005

    “r (36) = .45, p < .001”

  • Consistentr(36) = 0.64reported p < .001 · recomputed p < .001

    “r (36) = .64, p < .001”

  • Consistentr(36) = 0.55reported p < .001 · recomputed p < .001

    “r (36) = .55, p < .001”

  • Consistentr(36) = 0.57reported p < .001 · recomputed p < .001

    “r (36) = .57, p < .001”

  • Consistentr(36) = 0.73reported p < .001 · recomputed p < .001

    “r (36) = .73, p < .001”

  • ConsistentF(1, 72.23) = 2.87reported p = .095 · recomputed p = .095

    “F(1, 72.23) = 2.87, p = .095”

  • ConsistentF(1, 71.85) = 38.85reported p < .001 · recomputed p < .001

    “F(1, 71.85) = 38.85, p < .001”

  • ConsistentF(1, 71.94) = 5.31reported p = .024 · recomputed p = .024

    “F(1, 71.94) = 5.31, p = .024”

  • ConsistentF(1, 72.01) = 4.62reported p = .035 · recomputed p = .035

    “F(1, 72.01) = 4.62, p = .035”

  • ConsistentF(1, 30) = 4.98reported p = .033 · recomputed p = .033

    “F(1, 30) = 4.98, p = .033”

  • ConsistentF(1, 30) = 6.23reported p = .018 · recomputed p = .018

    “F(1, 30) = 6.23, p = .018”

  • ConsistentF(2, 33.28) = 29.56reported p < .001 · recomputed p < .001

    “F(2, 33.28) = 29.56, p < .001”

  • Consistentt(36.14) = -8.2reported p < .001 · recomputed p < .001

    “t(36.14) = −8.20, p < .001”

13 of 13

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Number of dropouts conflicts with later accounting

    Minor

    One section says two participants dropped out after baseline and one more dropped out before the third survey, implying three dropouts. The Limitations section instead refers to only two of the four study dropouts being from the community college, which conflicts with the earlier count if only three dropouts occurred.

  • Dropout count changes from three to four

    Minor

    The methods describe two participants dropping out after baseline and one more before the third survey, for three dropouts. The limitations section instead refers to four study dropouts; the separately excluded outlier is not described as a dropout.

2 of 2

Cites

Flagged first, then by citations

Research works printed but not linked: 7

  1. American College Health Association, 2012, American College Health Association–National College Health Assessment II: Reference Group Data Report Fall 2011
  2. Turner J C, Paper presented at the annual meeting of the American Public Health Association
  3. Gallagher R., 2011, National Survey of Counseling Center Directors
  4. Ruiz F J, 2010, Int J Psychol Psychol Ther, 10, 125
  5. Fledderus M, 2011, Psychol Med, 1
  6. Levin M E, 2012, Sex Addict Compuls, 19, 168
  7. Ciarrochi J, Presented at the Second World Conference on ACT, RFT, and Contextual Behavioural Science

Cited by

Sorted by citations

Related

Authors
Levin, M. E., Pistorello, J., Seeley, J. R., & Hayes, S. C.
Affiliations
  • Utah State University (US)
  • Contextual Change (United States) (US)
  • Oregon Research Institute (US)
  • University of Nevada, Reno (US)
ORCID
Venue
Journal of American College Health
Publisher
Informa UK Limited
Published
2013-12-07
Volume
62
Issue
1
Pages
20-30
ISSN
  • 0164-4300 (ISSN-L)
  • 0744-8481
  • 1940-3208
DOI
10.1080/07448481.2013.843533
PMC
PMC3858823
OpenAlex
W2090082457
Cited by
179
References
43
Citations by year
2026: 7 · 2025: 11 · 2024: 13 · 2023: 20 · 2022: 27 · 2021: 13 · 2020: 22 · 2019: 16 · 2018: 13 · 2017: 11 · 2016: 21 · 2015: 4 · 2014: 1
FWCI
9.63
Type
journal article
Language
en
Open access
Locations
  • Journal of American College Health (publishedVersion)
  • PubMed (publishedVersion)
  • PubMed Central (acceptedVersion)
  • Digital Commons - USU (Utah State University) (submittedVersion)
Funders
National Institute of Mental Health (R43MH085336)
Topics
  • Social Sciences › Psychology › Clinical Psychology › Mindfulness and Compassion Interventions
  • Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Anxiety, Depression, Psychometrics, Treatment, Cognitive Processes
Keywords
acceptance and commitment therapy, online intervention, prevention programs, university students, depression, anxiety, mental health prevention, program feasibility, program acceptability, first-year students, psychological distress
MeSH terms
Adaptation, Psychological, Adolescent, Anxiety, Depression, Female, Humans, Health Knowledge, Attitudes, Practice, Male, Mental Health, Students, Universities, Patient Satisfaction, Internet, Young Adult, Acceptance and Commitment Therapy

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-09-28.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-09-28.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review