Cited by 21Education Discipline and InequalityLegal Issues in EducationBehavioral and Psychological Studies

The Impact of Restorative Practices on the Use of Out-of-School Suspensions: Results from a Cluster Randomized Controlled Trial

Open access
Huang, Francis L., Gregory, Anne, Ward-Seidel, Allison Rae·2023·Prevention Science

Abstract

The overuse of exclusionary discipline practices, such as out-of-school suspensions (OSS), has consistently been documented over several decades. The resulting racial discipline disparities and the negative outcomes related to OSS have led policy makers and educators to consider other approaches to school discipline. One such approach, which has gained increasing use in the USA, is restorative practices (RP). However, despite its popularity, the experimental evidence base of the effects of RP and suspensions is extremely limited. To add to this knowledge base, we present findings from a cluster randomized controlled trial with 18 K-12 schools in an urban district in the US Northeast with 5878 students. Multilevel regression results, after 1 year of the intervention, showed no differences in the likelihood of suspension between students in the intervention and control schools and that the effects of the intervention did not vary by race/ethnicity, gender, or student disability status. However, for students in the intervention group, we show reductions in the likelihood of receiving an OSS for students who had previously been suspended.

Integrity signals

Registry

Crossref · 10.1007/s11121-023-01507-3

Cites

48 of 48 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

15 of 49 cited sources checked · 6 supported · 0 flagged · accuracy not yet measured

  • Premisecitation

    In a two-year cluster randomized controlled trial with 13 middle schools in Maine, Acosta et al. (2019) reported no statistically significant differences in 11 outcomes related to school connectedness, school climate, social skills, and bullying victimization.

    “In a 2-year CRCT with 13 middle schools (seven intervention and six control) in Maine (students = 2834), Acosta et al. (2019) reported no statistically significant differences in terms of 11 outcomes related to school connectedness, school climate, social skills, and bullying victimization.”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Acosta et al. (2019) source
  • Findingown data

    The effect of the intervention on suspension did not vary by student disability status.

    “the effects of the intervention did not vary by race/ethnicity, gender, or student disability status”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The restorative practices intervention was implemented with sufficient fidelity in the intervention schools.
    • Rests on an assumption: Control schools did not implement restorative practices or similar practices during the study period.
    • Rests on an assumption: Out-of-school suspension records accurately reflect suspensions and are comparable across intervention and control schools.
    • Rests on an assumption: The study had sufficient statistical power to detect meaningful effects on out-of-school suspensions.
    • Rests on an assumption: One year of intervention is sufficient to produce detectable changes in out-of-school suspensions.
  • Findingown data

    The effect of the intervention on suspension did not vary by gender.

    “the effects of the intervention did not vary by race/ethnicity, gender, or student disability status”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The restorative practices intervention was implemented with sufficient fidelity in the intervention schools.
    • Rests on an assumption: Control schools did not implement restorative practices or similar practices during the study period.
    • Rests on an assumption: Out-of-school suspension records accurately reflect suspensions and are comparable across intervention and control schools.
    • Rests on an assumption: The study had sufficient statistical power to detect meaningful effects on out-of-school suspensions.
    • Rests on an assumption: One year of intervention is sufficient to produce detectable changes in out-of-school suspensions.
  • Findingown data

    After one year of the intervention, the likelihood of suspension did not differ between students in intervention schools and students in control schools.

    “Multilevel regression results, after 1 year of the intervention, showed no differences in the likelihood of suspension between students in the intervention and control schools”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The restorative practices intervention was implemented with sufficient fidelity in the intervention schools.
    • Rests on an assumption: Control schools did not implement restorative practices or similar practices during the study period.
    • Rests on an assumption: Out-of-school suspension records accurately reflect suspensions and are comparable across intervention and control schools.
    • Rests on an assumption: The study had sufficient statistical power to detect meaningful effects on out-of-school suspensions.
    • Rests on an assumption: One year of intervention is sufficient to produce detectable changes in out-of-school suspensions.
  • Findingown data

    The effect of the intervention on suspension did not vary by race/ethnicity.

    “the effects of the intervention did not vary by race/ethnicity, gender, or student disability status”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The restorative practices intervention was implemented with sufficient fidelity in the intervention schools.
    • Rests on an assumption: Control schools did not implement restorative practices or similar practices during the study period.
    • Rests on an assumption: Out-of-school suspension records accurately reflect suspensions and are comparable across intervention and control schools.
    • Rests on an assumption: The study had sufficient statistical power to detect meaningful effects on out-of-school suspensions.
    • Rests on an assumption: One year of intervention is sufficient to produce detectable changes in out-of-school suspensions.
  • Classified as probably not a substantial claim. 39
  • May combine two claims. 34
  • The quote only partly states this claim. 1
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    In a two-year cluster randomized controlled trial with 40 middle schools in England, Bonell et al. (2018) found that schools assigned to a restorative practices and social-emotional learning intervention had students who experienced less bullying victimization, less substance abuse, and better psychological well-being.

    “In a 2-year CRCT with 40 middle schools (20 intervention and 20 control) in England (students = 5960), Bonell et al. (2018) found that schools assigned to an RP and a social-emotional learning (SEL) intervention had students who experienced less bullying victimization, less substance abuse (e.g., smoking, alcohol), and better psychological well-being.”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Bonell et al. (2018) source

      Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

      Experimental — accuracy not yet measured
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Acosta et al. noted that the lack of differences may be due to intervention schools only having a modest amount of restorative practices experience, which was not that different from what control schools received on their own.

    “Acosta et al. noted that the lack of differences may be due to intervention schools only having a modest amount of RP experience—which was not that different from what control schools received on their own.”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Acosta et al. (2019) source
  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    Students with a history of suspensions were less likely to receive an out-of-school suspension and felt more respected by teachers who had received a brief intervention compared to students whose teachers were in the control group.

    “Students with a history of suspensions were less likely to receive an OSS and felt more respected by teachers who had received a brief intervention compared to the students whose teachers were in the control group (Okonofua et al., 2016).”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Okonofua et al., 2016 source

      The paper’s Experiment 3 results (page not available in our copy) report lower suspension rates for previously suspended students whose math teachers received the intervention (29.4% versus 51.2%) and higher perceived respect (mean 4.70 versus 3.85). The paper calls the outcome a suspension, but does not establish in these passages that it was specifically an out-of-school suspension (OSS). (LLM, conf=0.85)

      Experimental — accuracy not yet measured
  • Classified as probably not a substantial claim.

    Premisecitation

    In evaluating two separate empathetic mindset interventions for teachers, an interaction effect for the intervention condition and prior student suspension was statistically significant, showing benefits for students who had been suspended in the prior year.

    “Another set of studies also demonstrated effects for students with prior suspensions: in evaluating two separate empathetic mindset interventions for teachers with the aim of improving student–teacher relationships, an interaction effect for the intervention condition and prior student suspension was statistically significant showing benefits for students who had been suspended in the prior year (”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Okonofua et al., 2022 source

      Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

      Experimental — accuracy not yet measured
  • May combine two claims.

    Premisecitation

    Bonell et al. (2018) characterized the effects as small (d = 0.08) and found no statistically significant differences in aggression.

    “Effects were characterized as small (d = 0.08) and no statistically significant differences in terms of aggression were found.”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Bonell et al. (2018) source
  • May combine two claims.

    Premisecitation

    At the end of 2020, only two cluster randomized controlled trials involving restorative practices had been conducted in the USA, with mixed results.

    “At the end of the last decade (i.e., 2020), only two cluster randomized controlled (CRCTs) studies involving RPs, long considered the gold standard in providing evidence of a causal relationship in evaluation studies, have been conducted, specifically in the USA, with mixed results (Acosta et al., 2019; Augustine et al., 2018).”
    Quote in the paperSubstantial claim1 of 2 sources checked
    • cites Acosta et al., 2019 source

      The Methods (Study Design; page unavailable) describe a school-randomized restorative-practices trial in Maine. The Conclusion (page unavailable) reports no significant intervention effects but positive associations between students’ restorative-practices experiences and outcomes. This supports citing Acosta et al. as one US trial with mixed findings; the paper does not establish that only two such trials had been conducted by the end of 2020. (LLM, conf=0.86)

      Experimental — accuracy not yet measured
    • cites Augustine et al., 2018

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Premisecitation

    Restorative practices are already being adopted rapidly by schools despite a lack of experimental evidence, highlighting the need for the current study.

    “the proverbial “train has left the station,” given RPs are already being adopted at a fast rate by schools, despite the lack of an experimental evidence base which highlights the need for the current study (Diliberti et al., 2019; Song & Swearer, 2016).”
    Quote in the paperSubstantial claim1 of 2 sources checked
    • cites Diliberti et al., 2019

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Song & Swearer, 2016 source
  • May combine two claims.

    Premisecitation

    Prior suspensions may damage trust and respect between students and teachers and can trigger a cycle of misconduct and further suspensions.

    “Prior suspensions may lead to a breakdown of trust and respect for the teacher and can trigger a vicious cycle of misconduct and suspensions (Okonofua et al., 2016; Rosenbaum, 2020).”
    Quote in the paperSubstantial claim2 of 2 sources checked
    • cites Okonofua et al., 2016 source
    • cites Rosenbaum, 2020 source

      The Hypotheses and Discussion sections say suspension may lead to school disengagement and further deviance, supporting the misconduct part of the proposed cycle. They do not describe a breakdown of trust or respect between students and teachers, or establish that the further deviance leads to further suspensions. Page numbers are unavailable in our copy. (LLM, conf=0.83)

      Experimental — accuracy not yet measured
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Other studies have also found intervention effects for students with a prior out-of-school suspension history.

    “The finding of intervention effects for those students with a prior history of OSS has been shown by others as well (Glenn et al., 2020; Okonofua et al., 2022).”
    Quote in the paperNot a substantial claim1 of 2 sources checked
    • cites Glenn et al., 2020

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Okonofua et al., 2022 source

      The Results section ('Additional student subgroups'; page unavailable) reports a significant intervention effect for students with at least one suspension the previous year. It does not distinguish prior out-of-school suspensions from prior in-school suspensions, so it does not establish the effect specifically for students with an OSS history. (LLM, conf=0.88)

      Experimental — accuracy not yet measured
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The resource and commitment requirements of restorative practices may be similar to those of other school-wide or teacher interventions such as positive behavior intervention supports or MyTeaching Partner.

    “these requirements may be similar to other school-wide or teacher interventions such as positive behavior intervention supports (PBIS; Bradshaw et al., 2010) or MyTeaching Partner (Gregory et al., 2015).”
    Quote in the paperNot a substantial claim1 of 2 sources checked
    • cites Bradshaw et al., 2010 source
    • cites Gregory et al., 2015 source
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Teachers and administrators may be less reluctant to suspend students who already have a suspension history, particularly Black students, who are especially affected by implicit racial bias.

    “Teachers and administrations may be less reluctant to issue suspensions to students who already have a track record of suspensions (Blakely & Woodward, 2000) and especially to already-suspended Black students—a group particularly impacted by implicit racial bias (Okonofua et al., 2016).”
    Quote in the paperNot a substantial claim2 of 2 sources checked
    • cites Blakely & Woodward, 2000 source

      The citation to Blakely and Woodward is attached to the claim that teachers and administrators may be less reluctant to suspend students with prior suspensions; the racial-bias clause cites a different source. The Background and bias sections discuss multilevel epidemiological research, not school discipline. Searches for suspension history and school discipline found no relevant passage in our copy. (LLM, conf=0.77)

      Experimental — accuracy not yet measured
    • cites Okonofua et al., 2016 source

      The Experiment 3 results report higher suspension rates for African American and Latino students and, separately, for students with prior suspensions (page not available in our copy). They do not establish a particular risk for previously suspended Black students or identify implicit racial bias as the cause. The broader claim about teachers and administrators issuing repeat suspensions is attached to a different citation. (LLM, conf=0.83)

      Experimental — accuracy not yet measured
  • May combine two claims.

    Interpretationcitation

    Because outcomes are worse for Black students with a suspension history, the current findings suggest some promise for restorative practices in dismantling the school-to-prison pipeline.

    “Given outcomes are worse for Black students with a history of suspension relative to suspended students from other racial/ethnic groups (e.g., Rosenbaum, 2020), the current study’s findings also indicate some promise for RP to aid in dismantling the school-to-prison pipeline.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Rosenbaum, 2020 source

      The Discussion says Black youth “seem to have worse outcomes from school suspension,” supporting the cited premise in broad terms. The Results compare suspended Black youth with similar non-suspended Black youth, not directly with suspended students from other racial or ethnic groups. The citing sentence states that cross-group comparison more definitively than the paper does; its claim about restorative practices is the citing authors’ own conclusion. (LLM, conf=0.82)

      Experimental — accuracy not yet measured
  • May combine two claims.

    Findingown data

    In robustness checks, the intervention effect did not vary by subgroup, as all interactions were not statistically significant (all ps > 0.20).

    “Results also indicated that the intervention effect did not vary by subgroup as all the interactions were not statistically significant (all ps > 0.20).”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    In both the pre- and post-intervention periods, out-of-school suspension rates were lowest in elementary schools (1.2% to 1.3%) and highest in high schools (9.1% to 10.2%).

    “In both pre- and post-intervention periods, the OSS rate was lowest in elementary schools (i.e., 1.2 to 1.3%) and highest in high schools (i.e., 9.1 to 10.2%).”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    Likelihood ratio tests indicated that including random slopes and allowing residual variances to vary by school type both improved model fit.

    “LRTs indicated that the inclusion of random slopes (χ2 = 74.6, p < 0.001) and allowing residual variances to vary by school type (χ2 = 1231.4, p < 0.001) were both warranted”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    The marginal R2 based on fixed effects increased from 0.108 in the main effects model to 0.142 in model 3, an increase of 30% from one additional predictor.

    “the marginal R2, based on the model fixed effects (Nakagawa & Schielzeth, 2013), increased from 0.108 (in the main effects model) to 0.142 in model 3, which is substantial (i.e., an increase of 30% based on only one additional predictor).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Nakagawa & Schielzeth, 2013 source
  • May combine two claims.

    Findingown data

    The intervention effect did not vary by student gender, race/ethnicity, or disability status; the likelihood ratio test was not significant (χ2(5) = 8.4, p = 0.14), and none of the interactions was statistically significant.

    “An LRT, which simultaneously tested if including the interaction terms improved model fit, indicated that the effect of the intervention did not vary by the gender, race, or disability status; χ2(5) = 8.4, p = 0.14. This is also evident in model 2 (see Table 3) where none of the interactions was statistically significant (all ps > 0.05).”
    • Also stated in: § Conclusion · plain
      “In addition, no differential intervention effects were found based on race, gender, and disability status.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Study design and conductcitation

    Of those that attrited, 4.1% and 2.7% left from the control and intervention groups, respectively, which is not considered problematic based on What Works Clearinghouse guidelines.

    “Of those that attrited, 4.1% and 2.7% left from the control and intervention groups, respectively, which is not considered problematic based on WWC guidelines.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites WWC, 2020

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim.

    Premisecitation

    Fronius et al. (2019) characterized the restorative practices research base as still in its infancy due to the inability of most studies to make strong causal claims about restorative practices effectiveness.

    “However, despite the growth in RP studies focusing on various outcomes (e.g., school climate, student victimization), Fronius et al. (2019) characterized the RP research base as still in its infancy due to the inability of most studies to make strong causal claims about RPs effectiveness.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Fronius et al. (2019)

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    Kindergarteners were excluded as they would not have records of prior suspensions.

    “As preregistered, kindergarteners were excluded as they would not have records of prior suspensions.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    In all intervention schools, RP coordinators helped facilitate schoolwide planning through diverse Restorative Intervention leadership teams, which met an average of 8 times (range 4 to 13).

    “in all the interventions schools, RP coordinators helped facilitate schoolwide planning through diverse Restorative Intervention (RI) leadership teams, which met an average of 8 times ranging from 4 to 13 meetings during the year.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Restorative practices use is related to improved attendance rates.

    “improved attendance rates (Gonzalez, 2012)”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Gonzalez, 2012
  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    In a quasi-experimental study in New Orleans, Glenn et al. (2020) reported that suspensions did not decline for the general student population that used restorative practices but out-of-school suspensions declined for students that had been suspended at least once prior to treatment.

    “In a study in New Orleans using a quasi-experimental design (i.e., difference in difference models using propensity scores), Glenn et al. (2020) reported that suspensions did not decline for the general student population that used RP but OSS declined for students that had been suspended at least once prior to treatment.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Glenn et al. (2020)

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Restorative practices use is related to higher academic achievement.

    “higher academic achievement (Darling-Hammond et al., 2021)”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Darling-Hammond et al., 2021

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The Institute of Education Sciences recommends the dummy variable indicator method for missing baseline data in cluster randomized trials.

    “the Institute of Education Sciences (IES, WWC, 2020, p. 38) suggests the use of the dummy variable indicator method.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites IES, WWC, 2020, p. 38

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    No schools dropped out of the one-year evaluation, indicating no school-level attrition.

    “No schools dropped out of the 1-year evaluation (i.e., there was no school-level attrition).”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Restorative practices use is related to fewer behavioral incidents.

    “fewer behavioral incidents (Boulton & Mirsky, 2006)”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Boulton & Mirsky, 2006
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    Robustness checks using logistic regression with cluster robust standard errors were prespecified.

    “We had also prespecified that robustness checks would be conducted using logistic regression analysis using cluster robust standard errors.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    208 students (3.4%) were considered attritors who completely left the school district.

    “In addition, 208 students (3.4%) were considered attritors who completely left the school district for various reasons (e.g., transferred out of state, withdrew from school after repeated absences, was deceased).”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Premisecitation

    Prior studies have found that restorative practices may not reduce suspension gaps between Black and non-Black students or between students with and without disabilities.

    “Other studies have shown that RP may not necessarily lower suspension gaps between Black and non-Black students as well as students with and without disabilities (Anyon et al., 2016; Hashim et al., 2018).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Anyon et al., 2016 source
    • cites Hashim et al., 2018 source
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    A possible reason for the absence of main effects on suspension is that one year was too short for the intervention to fully take effect.

    “One possible reason why RP main effects for suspension may not have materialized is that the time frame may have been too short (i.e., 1 year).”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Study design and conductown data

    Baseline equivalence analysis showed minimal differences across all school- and student-level characteristics (all ps > 0.10), suggesting successful randomization.

    “Baseline equivalence analysis, based on both school- and student-level characteristics, showed minimal differences across all characteristics (e.g., all ps > 0.10, see Table 1) suggesting successful randomization.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Premisecitation

    In a two-year cluster randomized controlled trial in 44 public schools in Pittsburgh, Augustine et al. (2018) reported that students in intervention schools reported improved perceptions of school climate and fewer days suspended compared to students in control schools.

    “In contrast, another 2-year CRCT was conducted in 44 public schools in Pittsburg (22 treatment and 22 control) and Augustine et al. (2018) reported that students in intervention schools reported improved perceptions of school climate and fewer days suspended compared to students in control schools.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Augustine et al. (2018) source
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    The dummy variable method for missing baseline data is simple, yields unbiased treatment effects under randomization, and is acceptable in cluster randomized trials when only the baseline measure is missing.

    “Using the dummy variable method for missing data is simple, produces unbiased treatment effects due to the randomization, and is acceptable when a CRCT is used and only if the baseline measure is missing (Puma et al., 2009).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Puma et al., 2009

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductreasoning

    The final analytic sample retained 253 students who transferred to another school within the wider school system, in line with the prespecified intention-to-treat analysis which preserved the original randomized design.

    “The final analytic sample consisted of 5878 students, of which 253 students had transferred to another school within the wider school system (the school system was comprised of over 30 school districts with over a million students) but were retained in the analytic sample in line with the prespecified intention-to-treat (ITT) analysis which preserved the original randomized design.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    The linear probability model was prespecified in the preregistered research protocol and approved by the grant funding agency's technical advisor.

    “the LPM was prespecified as part of the preregistered research protocol and was approved by the technical advisor of the grant funding agency”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    421 late joiners were excluded as prespecified based on What Works Clearinghouse guidelines.

    “Out of the 6507 students, 421 enrolled 6 weeks after the start of class and were considered “late joiners” and were excluded as was prespecified based on What Works Clearinghouse (WWC, 2020) guidelines.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites WWC, 2020

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    For the Schoolwide Staff Development component, RP coordinators in all nine schools held the 15-hour training and conducted individualized coaching, averaging 5 sessions per trained circle facilitator (range 1 to 24).

    “For the Schoolwide Staff Development training component, RP coordinators, in all 9 schools, held the 15-h training for staff and conducted individualized coaching (an average of 5 sessions per trained circle facilitator with a range of 1 session to 24 sessions per person).”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    All nine intervention school principals attended the two-day RP Leadership Team training and individual coaching sessions, averaging 11 sessions (range 7 to 14).

    “for the RP Leadership Team training component, all 9 principals attended the 2-day training and individual coaching sessions (they averaged 11 sessions ranging from 7 to 14 sessions).”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Study design and conductown data

    Missing data were minimal (5.4%) and occurred only in the baseline measure.

    “As there was little missing data (5.4%) and only coming from the baseline measure (e.g., OSS recorded in the prior year)”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    Training and support for the Family RP Opportunities component were variable: four of nine schools trained their parent coordinators in RP, and six of nine schools held RP family events.

    “training and support for the Family RP Opportunities component were more variable with 4 of the 9 schools training their parent coordinators in RP and 6 of the 9 schools holding RP family events.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    The study completed a preregistered research protocol prior to randomizing schools to the intervention and control conditions.

    “For the current study, and as required by the funder, we completed a preregistered research protocol (submitted to Abt Associates, the technical advisors of the US Department of Education) prior to randomizing schools to the intervention and control conditions.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Premisecitation

    Augustine et al. (2018) found no change in overall student behavior, student arrest rates, and daily attendance.

    “However, the study noted no change in overall student behavior, student arrest rates (which were already low at 1.2%), and daily attendance.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Augustine et al. (2018) source
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    The implementation fidelity measures were designed and refined during a pilot year with three schools not enrolled in the current study.

    “Implementation measures were designed and refined during a pilot year of the project working with three schools (not enrolled in the current study).”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    Only three of the nine intervention schools had multiple staff members participate in the formal two-day Restorative Intervention training.

    “However, only 3 of the 9 schools had multiple staff members participate in the formal 2-day RI training.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Premisenone given

    Preregistration increases the credibility of findings based on a priori hypotheses.

    “Preregistration increases the credibility of findings based on a priori hypotheses.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Interpretationown data

    After one year of implementation, restorative practices did not have a direct effect on reducing suspensions, and the effect did not differ by student race, disability status, or gender.

    “We show, using a rigorous CRCT, that after 1 year of implementation, RP did not have a direct effect in the reduction of suspensions, and effects were consistent regardless of student race, disability status, or gender (i.e., there was no differential effect of RP).”
    Quote in the paperSubstantial claim
  • Classified as probably not a substantial claim.

    Study design and conductown data

    Implementation fidelity of the training was monitored throughout the project through close contact with RP coordinators and Morningside Center staff.

    “Implementation fidelity of the training was measured throughout the project in close contact with RP coordinators and Morningside Center staff.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    Blocking by school type resulted in an equal number of elementary, middle, combined, and high schools randomized into the business-as-usual and intervention conditions.

    “Blocking by school type (e.g., elementary) resulted in an equal number of elementary, middle, combined, and high schools randomized into the BAU and intervention conditions.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Interpretationown data

    Although no main effect was found, restorative practices may be a promising approach for reducing suspensions, especially among students already at greater risk of academic failure.

    “Although main effects were not found, RP may be a promising approach for reducing suspensions in schools, especially for a subsample of students who are already at greater risk of academic failure.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Interpretationreasoning

    During remote schooling, out-of-school suspensions were not issued because students were already not in school, making the 2019-2020 data incomparable with prior years.

    “during remote schooling, out-of-school suspensions were not being issued (as students were already not in school), which made 2019–2020 incomparable with prior years”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The Whole School RP Project combined restorative practices, a social-emotional learning curriculum, and activities to shift adult mindsets and practices to increase racial equity in discipline.

    “the Whole School RP Project weaved together RP, an SEL curricula, and activities to shift adult mindsets and practices to increase racial equity in discipline.”
    Quote in the paperNot a substantial claim
  • Classified as probably not a substantial claim.

    Study design and conductnone given

    The evaluation was conducted after one year instead of the originally planned two years because of the COVID-19 pandemic, following a preregistered analytic plan.

    “Based on the preregistered analytic plan, of which the evaluation was conducted after 1 year instead of the 2 years as originally planned as a result of the COVID-19 pandemic”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Restorative practices and other interventions are being more widely adopted across the country.

    “this is reflective of discipline reform efforts across the country as RPs (and other types of interventions) are more widely adopted (Diliberti et al., 2019).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Diliberti et al., 2019

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    In the wider school system, suspension rates fell from approximately 8% to 5% between 2012 and 2019.

    “suspension rates in the school system fell from approximately 8 to 5% from 2012 to 2019 (Rodriguez & Welsh, 2022).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Rodriguez & Welsh, 2022 source
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    In the third-year extension, schools used a hybrid mode where suspensions were rarely used, and many students originally in the project had progressed to the next level of schooling or graduated.

    “Although we attempted to collect data in the third year (i.e., through an extension of the implementation to 2021, which was not planned), schools were using a hybrid mode where suspensions were rarely used (since students were mostly at home) and students who were originally part of the project had progressed to the next level of schooling or beyond”
    Quote in the paperNot a substantial claim
  • May combine two claims. The quote only partly states this claim.

    Interpretationown data

    Despite low prevalence and some control schools using restorative approaches, the Whole School RP Project reduced suspension likelihood among students with a prior suspension.

    “Yet, even when faced with this comparison and low prevalence rates, the Whole School RP Project had an impact on those with a prior history of suspension.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning

    The Whole School RP Project requires substantial resources, commitment, and effort, including RP coordinators in schools several days a week and buy-in from teachers, administrators, and students.

    “an intervention such as the Whole School RP Project requires resources, commitment, and effort (e.g., RP coordinators are in schools several days a week; teacher, administrator, and student buy-in is required).”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Restorative practices have been described as an effective but exhausting alternative to suspensions.

    “RP has been previously described as “an effective but exhausting alternative” to suspensions (Dominus, 2016).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Dominus, 2016

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationown data

    Among students with a prior suspension, the large reduction in suspension rates observed in intervention schools is practically meaningful and represents breaking the cycle of suspensions.

    “In the current study, of those who had been suspended previously, the large reduction in suspension rates is practically meaningful and represents the breaking of the cycle of suspensions.”
    Quote in the paperSubstantial claim
60 of 60
  • The paper relies on this without support.

    The restorative practices intervention was implemented with sufficient fidelity in the intervention schools.

    “The nine intervention schools were assigned to one of eight Morningside Center RP Coordinators and one of three RP/equity principal coaches.”
  • The paper relies on this without support.

    Control schools did not implement restorative practices or similar practices during the study period.

    “control/business-as-usual condition (BAU)”
  • The paper relies on this without support.

    Out-of-school suspension records accurately reflect suspensions and are comparable across intervention and control schools.

    “likelihood of receiving an OSS”
  • The paper relies on this without support.

    The study had sufficient statistical power to detect meaningful effects on out-of-school suspensions.

    “Multilevel regression results, after 1 year of the intervention, showed no differences in the likelihood of suspension between students in the intervention and control schools”
  • The paper relies on this without support.

    One year of intervention is sufficient to produce detectable changes in out-of-school suspensions.

    “outcomes were measured after a single year of the initiative, at the end of the 2018–2019 school year, given the second year was interrupted by the COVID-19 pandemic.”
  • The paper relies on this without support.

    Students' prior suspension history is accurately recorded and available.

    “for students who had previously been suspended”
  • The paper relies on this without support.

    The district's suspension records accurately and completely capture all out-of-school suspension events for students.

    “Suspension data, as well as student demographic data, were provided by the district.”
  • The paper relies on this without support.

    The observed reduction in suspension likelihood among students with a prior suspension is not due to regression to the mean or other statistical artifacts.

    “The final model (see model 3, Table 3) showed that the intervention effect varied as a function of whether the student had received a prior OSS or not (B = − 0.14, p = 0.04).”
  • The paper relies on this without support.

    The prior out-of-school suspension variable from the prior school year accurately identifies students with a suspension history.

    “In addition, we also use suspension data from the prior school year.”
9 of 9

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Two included evaluations of restorative programmes, Augustine et al. (2018) and Huang et al. (2023), could be described as underpowered cluster-RCTs.

    “For example, two included evaluations of RP, Augustine et al. (2018) and Huang et al. (2023), could be described as underpowered cluster‐RCTs.”

    From School‐Based Interventions for Reducing Disciplinary School Exclusion. An Updated Systematic Review (2025) · cites this paper as Huang et al. (2023)

    The Methodology (Participants; Analytic Strategy, page unavailable) confirms that Huang et al. conducted a cluster-randomized trial with 18 schools and notes the limited number of clusters. The Conclusion (page unavailable) says the evaluation ended after one year rather than the planned two, but the paper does not establish that the trial was statistically underpowered. It identifies Augustine et al. (2018) as another cluster-randomized trial without establishing that trial’s power either. (LLM, conf=0.83)

    Experimental — accuracy not yet measured
  • Huang et al. (2023) measured the impact of restorative programmes with 18 clusters (schools) and 5878 students, averaging approximately 327 students per school.

    “This evaluation measured the impact of RP with 18 clusters (in this case, schools) and a total of 5878 students, which averages to approximately 327 students per school.”
  • The estimated statistical power of Huang et al. (2023) is only 40%, assuming a normally distributed outcome and without considering potential benefits from stratification or baseline controls.

    “When we assess the statistical power of this study, and assume a normally distributed outcome, without considering potential benefits from factors like stratification or baseline controls, the estimated statistical power of the study is only 40%.”

    From School‐Based Interventions for Reducing Disciplinary School Exclusion. An Updated Systematic Review (2025) · cites this paper as Huang et al. (2023)

    The Methodology describes the 18-school trial, blocking by school type, and baseline covariates, but gives no 40% power estimate. Searches for statistical power, a normally distributed outcome, and a power analysis found no passage that verifies the citing authors’ calculation. (LLM, conf=0.74)

    Experimental — accuracy not yet measured

Statistics recomputed

Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.

  • Consistentχ²(5) = 8.4reported p = .14 · recomputed p = .136

    “χ2(5) = 8.4, p = .14”

1 of 1

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

  • Trial-study percentage does not match the stated review size

    Minor

    The paper says two of the 71 reviewed studies used a cluster randomized controlled trial, but two divided by 71 is about 2.8%, not 7.7%.

    Reported 7.7 · recomputed 2.816901

1 of 1

Cites

Flagged first, then by citations

Research works printed but not linked: 2

  1. Boyes-Watson, C., & Pranis, K. (2015). Circle forward: Building a restorative school community. Living Justice Press.
  2. Evans, K., & Vaandering, D. (2016). The little book of restorative justice in education: Fostering responsibility, healing, and hope in schools. Simon and Schuster.

Cited by

Sorted by citations

Authors
Huang, F. L., Gregory, A., & Ward-Seidel, A. R.
Affiliations
  • University of Missouri (US)
  • Rutgers, The State University of New Jersey (US)
  • University of Virginia (US)
Corresponding author
Huang, Francis L.
Venue
Prevention Science
Published
2023-02-28
Volume
24
Issue
5
Pages
962-973
ISSN
  • 1389-4986 (ISSN-L)
  • 1573-6695
OpenAlex
W4322617060
Cited by
21
References
46
Citations by year
2026: 5 · 2025: 8 · 2024: 3 · 2023: 5
FWCI
15.49
Type
journal article
Language
en
Open access
Locations
  • Prevention Science (publishedVersion)
  • PubMed Central (publishedVersion)
  • PubMed (publishedVersion)
Funders
U.S. Department of Education (U411C160035)
Topics
  • Social Sciences › Education › Education Discipline and Inequality
  • Social Sciences › Law › Legal Issues in Education
  • Social Sciences › Psychology › Developmental and Educational Psychology › Behavioral and Psychological Studies
Keywords
restorative practices, out-of-school suspension, student discipline, exclusionary discipline, K-12 schools, urban schools, gender differences
MeSH terms
Ethnicity, Humans, Schools, Students, Suspensions

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-09-28.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-09-28.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review