The Impact of Restorative Practices on the Use of Out-of-School Suspensions: Results from a Cluster Randomized Controlled Trial
Open accessAbstract
The overuse of exclusionary discipline practices, such as out-of-school suspensions (OSS), has consistently been documented over several decades. The resulting racial discipline disparities and the negative outcomes related to OSS have led policy makers and educators to consider other approaches to school discipline. One such approach, which has gained increasing use in the USA, is restorative practices (RP). However, despite its popularity, the experimental evidence base of the effects of RP and suspensions is extremely limited. To add to this knowledge base, we present findings from a cluster randomized controlled trial with 18 K-12 schools in an urban district in the US Northeast with 5878 students. Multilevel regression results, after 1 year of the intervention, showed no differences in the likelihood of suspension between students in the intervention and control schools and that the effects of the intervention did not vary by race/ethnicity, gender, or student disability status. However, for students in the intervention group, we show reductions in the likelihood of receiving an OSS for students who had previously been suspended.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
15 of 49 cited sources checked · 6 supported · 0 flagged · accuracy not yet measured
- Premisecitation
In a two-year cluster randomized controlled trial with 13 middle schools in Maine, Acosta et al. (2019) reported no statistically significant differences in 11 outcomes related to school connectedness, school climate, social skills, and bullying victimization.
“In a 2-year CRCT with 13 middle schools (seven intervention and six control) in Maine (students = 2834), Acosta et al. (2019) reported no statistically significant differences in terms of 11 outcomes related to school connectedness, school climate, social skills, and bullying victimization.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites Acosta et al. (2019) source
- Findingown data
The effect of the intervention on suspension did not vary by student disability status.
“the effects of the intervention did not vary by race/ethnicity, gender, or student disability status”
Quote in the paperSubstantial claim- Rests on an assumption: The restorative practices intervention was implemented with sufficient fidelity in the intervention schools.
- Rests on an assumption: Control schools did not implement restorative practices or similar practices during the study period.
- Rests on an assumption: Out-of-school suspension records accurately reflect suspensions and are comparable across intervention and control schools.
- Rests on an assumption: The study had sufficient statistical power to detect meaningful effects on out-of-school suspensions.
- Rests on an assumption: One year of intervention is sufficient to produce detectable changes in out-of-school suspensions.
- Findingown data
The effect of the intervention on suspension did not vary by gender.
“the effects of the intervention did not vary by race/ethnicity, gender, or student disability status”
Quote in the paperNot a substantial claim- Rests on an assumption: The restorative practices intervention was implemented with sufficient fidelity in the intervention schools.
- Rests on an assumption: Control schools did not implement restorative practices or similar practices during the study period.
- Rests on an assumption: Out-of-school suspension records accurately reflect suspensions and are comparable across intervention and control schools.
- Rests on an assumption: The study had sufficient statistical power to detect meaningful effects on out-of-school suspensions.
- Rests on an assumption: One year of intervention is sufficient to produce detectable changes in out-of-school suspensions.
- Findingown data
After one year of the intervention, the likelihood of suspension did not differ between students in intervention schools and students in control schools.
“Multilevel regression results, after 1 year of the intervention, showed no differences in the likelihood of suspension between students in the intervention and control schools”
Quote in the paperSubstantial claim- Rests on an assumption: The restorative practices intervention was implemented with sufficient fidelity in the intervention schools.
- Rests on an assumption: Control schools did not implement restorative practices or similar practices during the study period.
- Rests on an assumption: Out-of-school suspension records accurately reflect suspensions and are comparable across intervention and control schools.
- Rests on an assumption: The study had sufficient statistical power to detect meaningful effects on out-of-school suspensions.
- Rests on an assumption: One year of intervention is sufficient to produce detectable changes in out-of-school suspensions.
- Findingown data
The effect of the intervention on suspension did not vary by race/ethnicity.
“the effects of the intervention did not vary by race/ethnicity, gender, or student disability status”
Quote in the paperSubstantial claim- Rests on an assumption: The restorative practices intervention was implemented with sufficient fidelity in the intervention schools.
- Rests on an assumption: Control schools did not implement restorative practices or similar practices during the study period.
- Rests on an assumption: Out-of-school suspension records accurately reflect suspensions and are comparable across intervention and control schools.
- Rests on an assumption: The study had sufficient statistical power to detect meaningful effects on out-of-school suspensions.
- Rests on an assumption: One year of intervention is sufficient to produce detectable changes in out-of-school suspensions.
- Findingown data
Among students in the intervention group who had previously been suspended, the likelihood of receiving an out-of-school suspension was reduced.
“However, for students in the intervention group, we show reductions in the likelihood of receiving an OSS for students who had previously been suspended.”
Quote in the paperSubstantial claim- Rests on an assumption: The restorative practices intervention was implemented with sufficient fidelity in the intervention schools.
- Rests on an assumption: Control schools did not implement restorative practices or similar practices during the study period.
- Rests on an assumption: Out-of-school suspension records accurately reflect suspensions and are comparable across intervention and control schools.
- Rests on an assumption: The study had sufficient statistical power to detect meaningful effects on out-of-school suspensions.
- Rests on an assumption: One year of intervention is sufficient to produce detectable changes in out-of-school suspensions.
- Rests on an assumption: Students' prior suspension history is accurately recorded and available.
- Findingown data
An additional test indicated that the intervention effect did not vary by school type (χ2 = 0.76, p = 0.86).
“An additional test was conducted to determine if the intervention effect varied by type of school. An LRT (χ2 = 0.76, p = 0.86) indicated that the intervention effect did not vary by school type.”
Quote in the paperNot a substantial claim - Findingown data
In robustness checks, students in intervention schools were less likely to receive an out-of-school suspension (OR = 0.57, p = 0.09; one-tailed p = 0.045, d = 0.31) while controlling for all other variables.
“Robustness checks (see Online Appendix C) indicated that students in the intervention schools were less likely to receive an OSS (OR = 0.57, p = 0.09; one-tailed p = 0.045, d = 0.31) while controlling for all other variables in the model.”
Quote in the paperNot a substantial claim - Findingown data
A null multilevel model indicated that the intraclass correlation coefficient for suspensions was 0.04.
“A null multilevel model (MLM) indicated that the intraclass correlation coefficient for suspensions was 0.04.”
Quote in the paperNot a substantial claim - Findingown data
In robustness checks, the interaction with prior out-of-school suspension showed lower rates of suspension for students with a prior suspension (B = −0.98, p = 0.08; one-tailed p = 0.04).
“The interaction with prior OSS also showed lower rates of suspension for those with a prior OSS (B = − 0.98, p = 0.08; one-tailed p = 0.04).”
Quote in the paperNot a substantial claim - Findingown data
For students with a prior out-of-school suspension, the probability of suspension was 33% in control schools and 11% in intervention schools (d = 0.77).
“students in the control school with a prior OSS had a 33% probability of being suspended; in the intervention schools, the probability was only 11% (d = 0.77).”
Quote in the paperSubstantial claim - Findingown data
In four of the nine control schools, principals reported that some staff had been trained in restorative approaches to discipline.
“In four out of the nine schools, principals noted some staff were trained in restorative approaches to discipline”
- Also stated in: § Discussion · plain
“four (out of nine) BAU/control schools, based on administrator interviews, were using some form of restorative approach to discipline as well”
Quote in the paperNot a substantial claim - Also stated in: § Discussion · plain
- Findingown data
The final model showed that the intervention effect varied as a function of whether the student had received a prior out-of-school suspension (B = −0.14, p = 0.04).
“The final model (see model 3, Table 3) showed that the intervention effect varied as a function of whether the student had received a prior OSS or not (B = − 0.14, p = 0.04).”
Quote in the paperSubstantial claim - Findingown data
At the start of the project, principal interviews indicated that none of the control schools was implementing an integrated restorative practices initiative focused on leadership capacity building, social-emotional learning skills, and school-based teams delivering circle-process training.
“Interviews indicated that none of the control schools was engaged with integrated RP initiatives which focused on capacity building of the school leadership, taught SEL skills, and had school-based teams develop and deliver training programs using the circle process.”
Quote in the paperSubstantial claim - Findingown data
In the 2017-2018 school year, the average out-of-school suspension rate across the 18 schools was 4.4%.
“In the year prior to the intervention (school year 2017–2018), the average OSS rate for the 18 schools was 4.4%.”
Quote in the paperNot a substantial claim - Findingown data
The main effects multilevel model showed no statistically significant difference in the likelihood of suspension between students in intervention and control schools (B = −0.001, p = 0.95).
“Main effects of the MLM (see Table 3) indicated no statistically significant differences in the likelihood of suspension between students in the intervention and control schools (B = − 0.001, p = 0.95).”
- Also stated in: § Conclusion · plain
“the current study found no differences in a student’s likelihood of being suspended over the course of the academic year between schools randomized to the intervention and control conditions.”
Quote in the paperSubstantial claim - Also stated in: § Conclusion · plain
- Findingown data
In the 2018-2019 school year, the average out-of-school suspension rate across the 18 schools was 4.5%.
“In the year of the intervention (school year 2018–2019), the OSS rate was 4.5% (see Table 2).”
Quote in the paperNot a substantial claim - Findingown data
In the 2018-2019 school year, 42% of suspensions were issued in March or later.
“in the 2018–2019 school year, 42% of suspensions were issued in March or later”
Quote in the paperSubstantial claim - Findingown data
A prior suspension is one of the strongest predictors of a future suspension.
“As a prior suspension is one of the strongest predictors of a future suspension (see Table 2)”
Quote in the paperSubstantial claim - Premisenone given
Overall, the findings of the cluster randomized controlled trials investigating restorative practices are mixed.
“Overall, the findings of the CRCTs investigating RPs are mixed.”
Quote in the paperSubstantial claim - Premisecitation
In a review of 71 studies on school-based restorative practices published between 2000 and 2020, Zakszeski and Rutherford (2021) noted that only two peer-reviewed studies published in journals (7.7%) used a cluster randomized controlled trial while the rest were mostly qualitative or correlational/descriptive studies.
“In a review of 71 studies on school-based RP published in academic journals between 2000 and 2020 (i.e., the last 20 years), Zakszeski and Rutherford (2021) noted that only two peer-reviewed studies published in journals (7.7%) used a cluster randomized controlled trial (CRCT) while the rest of the studies were mostly qualitative or correlational/descriptive studies.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Zakszeski and Rutherford (2021) source
- Premisecitation
In prior research using the same sample, students in RP Project schools were less likely than comparison students to receive a discipline incident record.
“We found that overall, students in the RP Project schools were less likely to receive a discipline incident record as compared to students in the comparison schools (Gregory et al., 2022).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Gregory et al., 2022 source
- Study design and conductown data
Schools were randomly assigned to either the treatment/intervention condition or the control/business-as-usual condition.
“we conducted a CRCT where schools were randomly assigned to be in either the treatment/intervention or the control/business-as-usual condition (BAU).”
Quote in the paperSubstantial claim - Premisecitation
The prior study using the same sample found that students in restorative practices intervention schools were less likely to receive a discipline incident record (11.1%) compared to students in comparison schools (18.2%).
“The prior study focused solely on the reduction of office disciplinary referrals (ODRs; the preregistered primary outcome) as a result of an RP intervention and findings indicated that students in the RP intervention schools were less likely to receive a discipline incident record (11.1%) as compared to students in the comparison schools (18.2%; Gregory et al., 2022).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Gregory et al., 2022 source
- Premisecitation
Augustine et al. (2018) found that lower suspensions were due to reductions in suspensions in elementary but not middle or high schools.
“and the lower suspensions were due to reductions in suspensions in the elementary but not the middle or high schools.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Augustine et al. (2018) source
- Premisecitation
Augustine et al. (2018) found that reductions in days suspended for Black students were greater compared to White students.
“The reductions in days suspended for Black students were greater compared to White students”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Augustine et al. (2018) source
- Premisecitation
Augustine et al. (2018) found that academic achievement (math but not reading) of middle school students in schools with predominantly Black student enrolment in the intervention condition worsened.
“In addition, academic achievement (math but not reading) of middle school students, in schools with a predominantly Black student enrolment, in the intervention condition worsened.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Augustine et al. (2018) source
- Premisecitation
Linear probability models are valid for experimental studies.
“LPMs are valid for experimental studies (Deke, 2014; Huang, 2021, 2022)”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites Deke, 2014
- cites Huang, 2021, 2022
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Study design and conductown data
Despite variability across schools, every intervention school met the overall threshold for adequate implementation of training, a threshold developed with project designers and the research team based on a pilot year.
“Despite some variability across schools, each intervention school met the overall threshold for adequate implementation of training—a threshold developed in collaboration with project designers and the research team and based on a pilot year of implementation (with three different schools).”
Quote in the paperSubstantial claim - Premisecitation
Quantitative studies on restorative practices have generally shown that restorative practices use is related to lower suspensions.
“Quantitative (descriptive or correlational) studies on RP have generally shown that RP use is related to lower suspensions (Anyon et al., 2016; Gregory et al., 2018)”
Quote in the paperNot a substantial claim0 of 2 sources checked - Interpretationown data
Among students with a prior suspension, those in intervention schools were less likely to be suspended than those in control schools.
“This indicates that students with a prior suspension were less likely to be suspended in the intervention schools.”
- Also stated in: § Conclusion · plain
“Noteworthy was that after a single year of implementation, the current experimental study found a reduction in the likelihood of receiving an OSS for students with a prior history of suspension.”
Quote in the paperSubstantial claim - Also stated in: § Conclusion · plain
- Interpretationreasoning
Students with a prior out-of-school suspension are a subpopulation already at greater risk of academic failure.
“Students who had received an OSS previously represent a subpopulation of students who are already at greater risk of academic failure (i.e., they had already experienced being suspended).”
Quote in the paperSubstantial claim - Study design and conductreasoning
The integrated intervention did not isolate whether specific well-defined restorative practices such as circles or conferences caused the observed findings.
“The integrated initiative did not isolate whether specific well-defined RP such as circles or conferences caused the findings.”
Quote in the paperNot a substantial claim - Premisecitation
Restorative practice initiatives may require several years (e.g., 3 to 5) to be fully implemented.
“Considering that RP initiatives may take several years (e.g., 3 to 5 years) to fully implement (Glenn et al., 2020; Hashim et al., 2018; Mansfield et al., 2018)”
Quote in the paperSubstantial claim0 of 3 sources checked - Study design and conductreasoning
The COVID-19 pandemic forced schools to transition to remote schooling in March 2020, truncating data collection.
“the global Coronavirus pandemic forced schools to transition to remote schooling in March 2020, truncating data collection efforts.”
Quote in the paperSubstantial claim
- Classified as probably not a substantial claim. 39
- May combine two claims. 34
- The quote only partly states this claim. 1
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationIn a two-year cluster randomized controlled trial with 40 middle schools in England, Bonell et al. (2018) found that schools assigned to a restorative practices and social-emotional learning intervention had students who experienced less bullying victimization, less substance abuse, and better psychological well-being.
“In a 2-year CRCT with 40 middle schools (20 intervention and 20 control) in England (students = 5960), Bonell et al. (2018) found that schools assigned to an RP and a social-emotional learning (SEL) intervention had students who experienced less bullying victimization, less substance abuse (e.g., smoking, alcohol), and better psychological well-being.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites Bonell et al. (2018) source
Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationAcosta et al. noted that the lack of differences may be due to intervention schools only having a modest amount of restorative practices experience, which was not that different from what control schools received on their own.
“Acosta et al. noted that the lack of differences may be due to intervention schools only having a modest amount of RP experience—which was not that different from what control schools received on their own.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites Acosta et al. (2019) source
Classified as probably not a substantial claim. May combine two claims.
PremisecitationStudents with a history of suspensions were less likely to receive an out-of-school suspension and felt more respected by teachers who had received a brief intervention compared to students whose teachers were in the control group.
“Students with a history of suspensions were less likely to receive an OSS and felt more respected by teachers who had received a brief intervention compared to the students whose teachers were in the control group (Okonofua et al., 2016).”
Quote in the paperSubstantial claim1 of 1 sources checked- cites Okonofua et al., 2016 source
The paper’s Experiment 3 results (page not available in our copy) report lower suspension rates for previously suspended students whose math teachers received the intervention (29.4% versus 51.2%) and higher perceived respect (mean 4.70 versus 3.85). The paper calls the outcome a suspension, but does not establish in these passages that it was specifically an out-of-school suspension (OSS). (LLM, conf=0.85)
Experimental — accuracy not yet measured
Classified as probably not a substantial claim.
PremisecitationIn evaluating two separate empathetic mindset interventions for teachers, an interaction effect for the intervention condition and prior student suspension was statistically significant, showing benefits for students who had been suspended in the prior year.
“Another set of studies also demonstrated effects for students with prior suspensions: in evaluating two separate empathetic mindset interventions for teachers with the aim of improving student–teacher relationships, an interaction effect for the intervention condition and prior student suspension was statistically significant showing benefits for students who had been suspended in the prior year (”
Quote in the paperSubstantial claim1 of 1 sources checked- cites Okonofua et al., 2022 source
Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
Experimental — accuracy not yet measured
May combine two claims.
PremisecitationBonell et al. (2018) characterized the effects as small (d = 0.08) and found no statistically significant differences in aggression.
“Effects were characterized as small (d = 0.08) and no statistically significant differences in terms of aggression were found.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites Bonell et al. (2018) source
May combine two claims.
PremisecitationAt the end of 2020, only two cluster randomized controlled trials involving restorative practices had been conducted in the USA, with mixed results.
“At the end of the last decade (i.e., 2020), only two cluster randomized controlled (CRCTs) studies involving RPs, long considered the gold standard in providing evidence of a causal relationship in evaluation studies, have been conducted, specifically in the USA, with mixed results (Acosta et al., 2019; Augustine et al., 2018).”
Quote in the paperSubstantial claim1 of 2 sources checked- cites Acosta et al., 2019 source
The Methods (Study Design; page unavailable) describe a school-randomized restorative-practices trial in Maine. The Conclusion (page unavailable) reports no significant intervention effects but positive associations between students’ restorative-practices experiences and outcomes. This supports citing Acosta et al. as one US trial with mixed findings; the paper does not establish that only two such trials had been conducted by the end of 2020. (LLM, conf=0.86)
Experimental — accuracy not yet measured - cites Augustine et al., 2018
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
May combine two claims.
PremisecitationRestorative practices are already being adopted rapidly by schools despite a lack of experimental evidence, highlighting the need for the current study.
“the proverbial “train has left the station,” given RPs are already being adopted at a fast rate by schools, despite the lack of an experimental evidence base which highlights the need for the current study (Diliberti et al., 2019; Song & Swearer, 2016).”
Quote in the paperSubstantial claim1 of 2 sources checked- cites Diliberti et al., 2019
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites Song & Swearer, 2016 source
May combine two claims.
PremisecitationPrior suspensions may damage trust and respect between students and teachers and can trigger a cycle of misconduct and further suspensions.
“Prior suspensions may lead to a breakdown of trust and respect for the teacher and can trigger a vicious cycle of misconduct and suspensions (Okonofua et al., 2016; Rosenbaum, 2020).”
Quote in the paperSubstantial claim2 of 2 sources checked- cites Okonofua et al., 2016 source
- cites Rosenbaum, 2020 source
The Hypotheses and Discussion sections say suspension may lead to school disengagement and further deviance, supporting the misconduct part of the proposed cycle. They do not describe a breakdown of trust or respect between students and teachers, or establish that the further deviance leads to further suspensions. Page numbers are unavailable in our copy. (LLM, conf=0.83)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationOther studies have also found intervention effects for students with a prior out-of-school suspension history.
“The finding of intervention effects for those students with a prior history of OSS has been shown by others as well (Glenn et al., 2020; Okonofua et al., 2022).”
Quote in the paperNot a substantial claim1 of 2 sources checked- cites Glenn et al., 2020
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites Okonofua et al., 2022 source
The Results section ('Additional student subgroups'; page unavailable) reports a significant intervention effect for students with at least one suspension the previous year. It does not distinguish prior out-of-school suspensions from prior in-school suspensions, so it does not establish the effect specifically for students with an OSS history. (LLM, conf=0.88)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationThe resource and commitment requirements of restorative practices may be similar to those of other school-wide or teacher interventions such as positive behavior intervention supports or MyTeaching Partner.
“these requirements may be similar to other school-wide or teacher interventions such as positive behavior intervention supports (PBIS; Bradshaw et al., 2010) or MyTeaching Partner (Gregory et al., 2015).”
Quote in the paperNot a substantial claim1 of 2 sources checkedProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationTeachers and administrators may be less reluctant to suspend students who already have a suspension history, particularly Black students, who are especially affected by implicit racial bias.
“Teachers and administrations may be less reluctant to issue suspensions to students who already have a track record of suspensions (Blakely & Woodward, 2000) and especially to already-suspended Black students—a group particularly impacted by implicit racial bias (Okonofua et al., 2016).”
Quote in the paperNot a substantial claim2 of 2 sources checked- cites Blakely & Woodward, 2000 source
The citation to Blakely and Woodward is attached to the claim that teachers and administrators may be less reluctant to suspend students with prior suspensions; the racial-bias clause cites a different source. The Background and bias sections discuss multilevel epidemiological research, not school discipline. Searches for suspension history and school discipline found no relevant passage in our copy. (LLM, conf=0.77)
Experimental — accuracy not yet measured - cites Okonofua et al., 2016 source
The Experiment 3 results report higher suspension rates for African American and Latino students and, separately, for students with prior suspensions (page not available in our copy). They do not establish a particular risk for previously suspended Black students or identify implicit racial bias as the cause. The broader claim about teachers and administrators issuing repeat suspensions is attached to a different citation. (LLM, conf=0.83)
Experimental — accuracy not yet measured
May combine two claims.
InterpretationcitationBecause outcomes are worse for Black students with a suspension history, the current findings suggest some promise for restorative practices in dismantling the school-to-prison pipeline.
“Given outcomes are worse for Black students with a history of suspension relative to suspended students from other racial/ethnic groups (e.g., Rosenbaum, 2020), the current study’s findings also indicate some promise for RP to aid in dismantling the school-to-prison pipeline.”
Quote in the paperSubstantial claim1 of 1 sources checked- cites Rosenbaum, 2020 source
The Discussion says Black youth “seem to have worse outcomes from school suspension,” supporting the cited premise in broad terms. The Results compare suspended Black youth with similar non-suspended Black youth, not directly with suspended students from other racial or ethnic groups. The citing sentence states that cross-group comparison more definitively than the paper does; its claim about restorative practices is the citing authors’ own conclusion. (LLM, conf=0.82)
Experimental — accuracy not yet measured
May combine two claims.
Findingown dataIn robustness checks, the intervention effect did not vary by subgroup, as all interactions were not statistically significant (all ps > 0.20).
“Results also indicated that the intervention effect did not vary by subgroup as all the interactions were not statistically significant (all ps > 0.20).”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataIn both the pre- and post-intervention periods, out-of-school suspension rates were lowest in elementary schools (1.2% to 1.3%) and highest in high schools (9.1% to 10.2%).
“In both pre- and post-intervention periods, the OSS rate was lowest in elementary schools (i.e., 1.2 to 1.3%) and highest in high schools (i.e., 9.1 to 10.2%).”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataLikelihood ratio tests indicated that including random slopes and allowing residual variances to vary by school type both improved model fit.
“LRTs indicated that the inclusion of random slopes (χ2 = 74.6, p < 0.001) and allowing residual variances to vary by school type (χ2 = 1231.4, p < 0.001) were both warranted”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataThe marginal R2 based on fixed effects increased from 0.108 in the main effects model to 0.142 in model 3, an increase of 30% from one additional predictor.
“the marginal R2, based on the model fixed effects (Nakagawa & Schielzeth, 2013), increased from 0.108 (in the main effects model) to 0.142 in model 3, which is substantial (i.e., an increase of 30% based on only one additional predictor).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Nakagawa & Schielzeth, 2013 source
May combine two claims.
Findingown dataThe intervention effect did not vary by student gender, race/ethnicity, or disability status; the likelihood ratio test was not significant (χ2(5) = 8.4, p = 0.14), and none of the interactions was statistically significant.
“An LRT, which simultaneously tested if including the interaction terms improved model fit, indicated that the effect of the intervention did not vary by the gender, race, or disability status; χ2(5) = 8.4, p = 0.14. This is also evident in model 2 (see Table 3) where none of the interactions was statistically significant (all ps > 0.05).”
- Also stated in: § Conclusion · plain
“In addition, no differential intervention effects were found based on race, gender, and disability status.”
Quote in the paperSubstantial claim- Also stated in: § Conclusion · plain
May combine two claims.
Study design and conductcitationOf those that attrited, 4.1% and 2.7% left from the control and intervention groups, respectively, which is not considered problematic based on What Works Clearinghouse guidelines.
“Of those that attrited, 4.1% and 2.7% left from the control and intervention groups, respectively, which is not considered problematic based on WWC guidelines.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites WWC, 2020
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Classified as probably not a substantial claim.
PremisecitationFronius et al. (2019) characterized the restorative practices research base as still in its infancy due to the inability of most studies to make strong causal claims about restorative practices effectiveness.
“However, despite the growth in RP studies focusing on various outcomes (e.g., school climate, student victimization), Fronius et al. (2019) characterized the RP research base as still in its infancy due to the inability of most studies to make strong causal claims about RPs effectiveness.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Fronius et al. (2019)
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductreasoningKindergarteners were excluded as they would not have records of prior suspensions.
“As preregistered, kindergarteners were excluded as they would not have records of prior suspensions.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductown dataIn all intervention schools, RP coordinators helped facilitate schoolwide planning through diverse Restorative Intervention leadership teams, which met an average of 8 times (range 4 to 13).
“in all the interventions schools, RP coordinators helped facilitate schoolwide planning through diverse Restorative Intervention (RI) leadership teams, which met an average of 8 times ranging from 4 to 13 meetings during the year.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationRestorative practices use is related to improved attendance rates.
“improved attendance rates (Gonzalez, 2012)”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Gonzalez, 2012
Classified as probably not a substantial claim. May combine two claims.
PremisecitationIn a quasi-experimental study in New Orleans, Glenn et al. (2020) reported that suspensions did not decline for the general student population that used restorative practices but out-of-school suspensions declined for students that had been suspended at least once prior to treatment.
“In a study in New Orleans using a quasi-experimental design (i.e., difference in difference models using propensity scores), Glenn et al. (2020) reported that suspensions did not decline for the general student population that used RP but OSS declined for students that had been suspended at least once prior to treatment.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Glenn et al. (2020)
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationRestorative practices use is related to higher academic achievement.
“higher academic achievement (Darling-Hammond et al., 2021)”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Darling-Hammond et al., 2021
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationThe Institute of Education Sciences recommends the dummy variable indicator method for missing baseline data in cluster randomized trials.
“the Institute of Education Sciences (IES, WWC, 2020, p. 38) suggests the use of the dummy variable indicator method.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites IES, WWC, 2020, p. 38
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductown dataNo schools dropped out of the one-year evaluation, indicating no school-level attrition.
“No schools dropped out of the 1-year evaluation (i.e., there was no school-level attrition).”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationRestorative practices use is related to fewer behavioral incidents.
“fewer behavioral incidents (Boulton & Mirsky, 2006)”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Boulton & Mirsky, 2006
Probably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductown dataRobustness checks using logistic regression with cluster robust standard errors were prespecified.
“We had also prespecified that robustness checks would be conducted using logistic regression analysis using cluster robust standard errors.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductown data208 students (3.4%) were considered attritors who completely left the school district.
“In addition, 208 students (3.4%) were considered attritors who completely left the school district for various reasons (e.g., transferred out of state, withdrew from school after repeated absences, was deceased).”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationPrior studies have found that restorative practices may not reduce suspension gaps between Black and non-Black students or between students with and without disabilities.
“Other studies have shown that RP may not necessarily lower suspension gaps between Black and non-Black students as well as students with and without disabilities (Anyon et al., 2016; Hashim et al., 2018).”
Quote in the paperNot a substantial claim0 of 2 sources checkedProbably not a substantial claim: may not pass S3 (load-bearing).
InterpretationreasoningA possible reason for the absence of main effects on suspension is that one year was too short for the intervention to fully take effect.
“One possible reason why RP main effects for suspension may not have materialized is that the time frame may have been too short (i.e., 1 year).”
Quote in the paperNot a substantial claimMay combine two claims.
Study design and conductown dataBaseline equivalence analysis showed minimal differences across all school- and student-level characteristics (all ps > 0.10), suggesting successful randomization.
“Baseline equivalence analysis, based on both school- and student-level characteristics, showed minimal differences across all characteristics (e.g., all ps > 0.10, see Table 1) suggesting successful randomization.”
Quote in the paperSubstantial claimMay combine two claims.
PremisecitationIn a two-year cluster randomized controlled trial in 44 public schools in Pittsburgh, Augustine et al. (2018) reported that students in intervention schools reported improved perceptions of school climate and fewer days suspended compared to students in control schools.
“In contrast, another 2-year CRCT was conducted in 44 public schools in Pittsburg (22 treatment and 22 control) and Augustine et al. (2018) reported that students in intervention schools reported improved perceptions of school climate and fewer days suspended compared to students in control schools.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Augustine et al. (2018) source
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationThe dummy variable method for missing baseline data is simple, yields unbiased treatment effects under randomization, and is acceptable in cluster randomized trials when only the baseline measure is missing.
“Using the dummy variable method for missing data is simple, produces unbiased treatment effects due to the randomization, and is acceptable when a CRCT is used and only if the baseline measure is missing (Puma et al., 2009).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Puma et al., 2009
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductreasoningThe final analytic sample retained 253 students who transferred to another school within the wider school system, in line with the prespecified intention-to-treat analysis which preserved the original randomized design.
“The final analytic sample consisted of 5878 students, of which 253 students had transferred to another school within the wider school system (the school system was comprised of over 30 school districts with over a million students) but were retained in the analytic sample in line with the prespecified intention-to-treat (ITT) analysis which preserved the original randomized design.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductnone givenThe linear probability model was prespecified in the preregistered research protocol and approved by the grant funding agency's technical advisor.
“the LPM was prespecified as part of the preregistered research protocol and was approved by the technical advisor of the grant funding agency”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductcitation421 late joiners were excluded as prespecified based on What Works Clearinghouse guidelines.
“Out of the 6507 students, 421 enrolled 6 weeks after the start of class and were considered “late joiners” and were excluded as was prespecified based on What Works Clearinghouse (WWC, 2020) guidelines.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites WWC, 2020
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductown dataFor the Schoolwide Staff Development component, RP coordinators in all nine schools held the 15-hour training and conducted individualized coaching, averaging 5 sessions per trained circle facilitator (range 1 to 24).
“For the Schoolwide Staff Development training component, RP coordinators, in all 9 schools, held the 15-h training for staff and conducted individualized coaching (an average of 5 sessions per trained circle facilitator with a range of 1 session to 24 sessions per person).”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductown dataAll nine intervention school principals attended the two-day RP Leadership Team training and individual coaching sessions, averaging 11 sessions (range 7 to 14).
“for the RP Leadership Team training component, all 9 principals attended the 2-day training and individual coaching sessions (they averaged 11 sessions ranging from 7 to 14 sessions).”
Quote in the paperNot a substantial claimMay combine two claims.
Study design and conductown dataMissing data were minimal (5.4%) and occurred only in the baseline measure.
“As there was little missing data (5.4%) and only coming from the baseline measure (e.g., OSS recorded in the prior year)”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductown dataTraining and support for the Family RP Opportunities component were variable: four of nine schools trained their parent coordinators in RP, and six of nine schools held RP family events.
“training and support for the Family RP Opportunities component were more variable with 4 of the 9 schools training their parent coordinators in RP and 6 of the 9 schools holding RP family events.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductown dataThe study completed a preregistered research protocol prior to randomizing schools to the intervention and control conditions.
“For the current study, and as required by the funder, we completed a preregistered research protocol (submitted to Abt Associates, the technical advisors of the US Department of Education) prior to randomizing schools to the intervention and control conditions.”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationAugustine et al. (2018) found no change in overall student behavior, student arrest rates, and daily attendance.
“However, the study noted no change in overall student behavior, student arrest rates (which were already low at 1.2%), and daily attendance.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Augustine et al. (2018) source
Probably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductown dataThe implementation fidelity measures were designed and refined during a pilot year with three schools not enrolled in the current study.
“Implementation measures were designed and refined during a pilot year of the project working with three schools (not enrolled in the current study).”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductown dataOnly three of the nine intervention schools had multiple staff members participate in the formal two-day Restorative Intervention training.
“However, only 3 of the 9 schools had multiple staff members participate in the formal 2-day RI training.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).
Premisenone givenPreregistration increases the credibility of findings based on a priori hypotheses.
“Preregistration increases the credibility of findings based on a priori hypotheses.”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationown dataAfter one year of implementation, restorative practices did not have a direct effect on reducing suspensions, and the effect did not differ by student race, disability status, or gender.
“We show, using a rigorous CRCT, that after 1 year of implementation, RP did not have a direct effect in the reduction of suspensions, and effects were consistent regardless of student race, disability status, or gender (i.e., there was no differential effect of RP).”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Study design and conductown dataImplementation fidelity of the training was monitored throughout the project through close contact with RP coordinators and Morningside Center staff.
“Implementation fidelity of the training was measured throughout the project in close contact with RP coordinators and Morningside Center staff.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductown dataBlocking by school type resulted in an equal number of elementary, middle, combined, and high schools randomized into the business-as-usual and intervention conditions.
“Blocking by school type (e.g., elementary) resulted in an equal number of elementary, middle, combined, and high schools randomized into the BAU and intervention conditions.”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationown dataAlthough no main effect was found, restorative practices may be a promising approach for reducing suspensions, especially among students already at greater risk of academic failure.
“Although main effects were not found, RP may be a promising approach for reducing suspensions in schools, especially for a subsample of students who are already at greater risk of academic failure.”
Quote in the paperSubstantial claimMay combine two claims.
InterpretationreasoningDuring remote schooling, out-of-school suspensions were not issued because students were already not in school, making the 2019-2020 data incomparable with prior years.
“during remote schooling, out-of-school suspensions were not being issued (as students were already not in school), which made 2019–2020 incomparable with prior years”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenThe Whole School RP Project combined restorative practices, a social-emotional learning curriculum, and activities to shift adult mindsets and practices to increase racial equity in discipline.
“the Whole School RP Project weaved together RP, an SEL curricula, and activities to shift adult mindsets and practices to increase racial equity in discipline.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Study design and conductnone givenThe evaluation was conducted after one year instead of the originally planned two years because of the COVID-19 pandemic, following a preregistered analytic plan.
“Based on the preregistered analytic plan, of which the evaluation was conducted after 1 year instead of the 2 years as originally planned as a result of the COVID-19 pandemic”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationRestorative practices and other interventions are being more widely adopted across the country.
“this is reflective of discipline reform efforts across the country as RPs (and other types of interventions) are more widely adopted (Diliberti et al., 2019).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Diliberti et al., 2019
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationIn the wider school system, suspension rates fell from approximately 8% to 5% between 2012 and 2019.
“suspension rates in the school system fell from approximately 8 to 5% from 2012 to 2019 (Rodriguez & Welsh, 2022).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Rodriguez & Welsh, 2022 source
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductown dataIn the third-year extension, schools used a hybrid mode where suspensions were rarely used, and many students originally in the project had progressed to the next level of schooling or graduated.
“Although we attempted to collect data in the third year (i.e., through an extension of the implementation to 2021, which was not planned), schools were using a hybrid mode where suspensions were rarely used (since students were mostly at home) and students who were originally part of the project had progressed to the next level of schooling or beyond”
Quote in the paperNot a substantial claimMay combine two claims. The quote only partly states this claim.
Interpretationown dataDespite low prevalence and some control schools using restorative approaches, the Whole School RP Project reduced suspension likelihood among students with a prior suspension.
“Yet, even when faced with this comparison and low prevalence rates, the Whole School RP Project had an impact on those with a prior history of suspension.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
InterpretationreasoningThe Whole School RP Project requires substantial resources, commitment, and effort, including RP coordinators in schools several days a week and buy-in from teachers, administrators, and students.
“an intervention such as the Whole School RP Project requires resources, commitment, and effort (e.g., RP coordinators are in schools several days a week; teacher, administrator, and student buy-in is required).”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationRestorative practices have been described as an effective but exhausting alternative to suspensions.
“RP has been previously described as “an effective but exhausting alternative” to suspensions (Dominus, 2016).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Dominus, 2016
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
May combine two claims.
Interpretationown dataAmong students with a prior suspension, the large reduction in suspension rates observed in intervention schools is practically meaningful and represents breaking the cycle of suspensions.
“In the current study, of those who had been suspended previously, the large reduction in suspension rates is practically meaningful and represents the breaking of the cycle of suspensions.”
Quote in the paperSubstantial claim
The paper relies on this without support.
The restorative practices intervention was implemented with sufficient fidelity in the intervention schools.
“The nine intervention schools were assigned to one of eight Morningside Center RP Coordinators and one of three RP/equity principal coaches.”
The paper relies on this without support.
Control schools did not implement restorative practices or similar practices during the study period.
“control/business-as-usual condition (BAU)”
The paper relies on this without support.
Out-of-school suspension records accurately reflect suspensions and are comparable across intervention and control schools.
“likelihood of receiving an OSS”
The paper relies on this without support.
The study had sufficient statistical power to detect meaningful effects on out-of-school suspensions.
“Multilevel regression results, after 1 year of the intervention, showed no differences in the likelihood of suspension between students in the intervention and control schools”
The paper relies on this without support.
One year of intervention is sufficient to produce detectable changes in out-of-school suspensions.
“outcomes were measured after a single year of the initiative, at the end of the 2018–2019 school year, given the second year was interrupted by the COVID-19 pandemic.”
The paper relies on this without support.
Students' prior suspension history is accurately recorded and available.
“for students who had previously been suspended”
The paper relies on this without support.
The district's suspension records accurately and completely capture all out-of-school suspension events for students.
“Suspension data, as well as student demographic data, were provided by the district.”
The paper relies on this without support.
The observed reduction in suspension likelihood among students with a prior suspension is not due to regression to the mean or other statistical artifacts.
“The final model (see model 3, Table 3) showed that the intervention effect varied as a function of whether the student had received a prior OSS or not (B = − 0.14, p = 0.04).”
The paper relies on this without support.
The prior out-of-school suspension variable from the prior school year accurately identifies students with a suspension history.
“In addition, we also use suspension data from the prior school year.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Two included evaluations of restorative programmes, Augustine et al. (2018) and Huang et al. (2023), could be described as underpowered cluster-RCTs.
“For example, two included evaluations of RP, Augustine et al. (2018) and Huang et al. (2023), could be described as underpowered cluster‐RCTs.”
From School‐Based Interventions for Reducing Disciplinary School Exclusion. An Updated Systematic Review (2025) · cites this paper as Huang et al. (2023)
The Methodology (Participants; Analytic Strategy, page unavailable) confirms that Huang et al. conducted a cluster-randomized trial with 18 schools and notes the limited number of clusters. The Conclusion (page unavailable) says the evaluation ended after one year rather than the planned two, but the paper does not establish that the trial was statistically underpowered. It identifies Augustine et al. (2018) as another cluster-randomized trial without establishing that trial’s power either. (LLM, conf=0.83)
Experimental — accuracy not yet measuredHuang et al. (2023) measured the impact of restorative programmes with 18 clusters (schools) and 5878 students, averaging approximately 327 students per school.
“This evaluation measured the impact of RP with 18 clusters (in this case, schools) and a total of 5878 students, which averages to approximately 327 students per school.”
From School‐Based Interventions for Reducing Disciplinary School Exclusion. An Updated Systematic Review (2025) · cites this paper as Huang et al. (2023)
The estimated statistical power of Huang et al. (2023) is only 40%, assuming a normally distributed outcome and without considering potential benefits from stratification or baseline controls.
“When we assess the statistical power of this study, and assume a normally distributed outcome, without considering potential benefits from factors like stratification or baseline controls, the estimated statistical power of the study is only 40%.”
From School‐Based Interventions for Reducing Disciplinary School Exclusion. An Updated Systematic Review (2025) · cites this paper as Huang et al. (2023)
The Methodology describes the 18-school trial, blocking by school type, and baseline covariates, but gives no 40% power estimate. Searches for statistical power, a normally distributed outcome, and a power analysis found no passage that verifies the citing authors’ calculation. (LLM, conf=0.74)
Experimental — accuracy not yet measured
Statistics recomputed
Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.
- Consistentχ²(5) = 8.4reported p = .14 · recomputed p = .136
“χ2(5) = 8.4, p = .14”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Trial-study percentage does not match the stated review size
MinorThe paper says two of the 71 reviewed studies used a cluster randomized controlled trial, but two divided by 71 is about 2.8%, not 7.7%.
Reported 7.7 · recomputed 2.816901
“In a review of 71 studies on school-based RP published in academic journals between 2000 and 2020 (i.e., the last 20 years), Zakszeski and Rutherford (2021) noted that only two peer-reviewed studies published in journals (7.7%) used a cluster randomized controlled trial (CRCT)”
Cites
R: A Language and Environment for Statistical Computing
R Core Team (2000) · doi:10.32614/r.manuals · Cited by 355,706 · see citing works
Nakagawa, S., & Schielzeth, H. (2013) · Methods in Ecology and Evolution · doi:10.1111/j.2041-210x.2012.00261.x · Cited by 10,356 · see citing works
Rao, J. N. K., & Scott, A. J. (1981) · Journal of the American Statistical Association · doi:10.1080/01621459.1981.10477633 · Cited by 1,009 · see citing works
Bradshaw, C. P., Mitchell, M. M., & Leaf, P. J. (2010) · Journal of Positive Behavior Interventions · doi:10.1177/1098300709334798 · Cited by 868 · see citing works
Brief intervention to encourage empathic discipline cuts suspension rates in half among adolescents
Okonofua, J. A., Paunesku, D., & Walton, G. M. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1523698113 · Cited by 510 · cited 3× · see citing works
“Teachers and administrations may be less reluctant to issue suspensions to students who already have a track record of suspensions (Blakely & Woodward, 2000) and especially to already-suspended Black students—a group particularly impacted by implicit racial bias (Okonofua et al., 2016).”
BackgroundRelationship Between School Suspension and Student Outcomes: A Meta-Analysis
Noltemeyer, A. L., Ward, R. M., & Mcloughlin, C. S. (2015) · School Psychology Review · doi:10.17105/spr-14-0008.1 · Cited by 455 · cited 1× · see citing works
“However, several studies have indicated that students who are suspended from school, relative to their peers, are more likely to have worse academic outcomes, to be held back a grade level or leave school, and have a greater risk of substance use, arrest, or incarceration (Anderson et al., 2019; Dong & Krohn, 2020; Hw…”
BackgroundEcological effects in multi-level studies
Blakely, T. A., & Woodward, A. J. (2000) · Journal of Epidemiology and Community Health · doi:10.1136/jech.54.5.367 · Cited by 385 · see citing works
Jones, S. M., Brown, J. L., & Lawrence Aber, J. (2011) · Child Development · doi:10.1111/j.1467-8624.2010.01560.x · Cited by 351 · see citing works
Bias reduction in standard errors for linear regression with multi-stage samples
McCaffrey, D. F., & Bell, R. M. (2003) · Quality Engineering · Cited by 318 · see citing works
Bonell, C., Allen, E., Warren, E., McGowan, J., Bevilacqua, L., Jamal, F., Legood, R., Wiggins, M., Opondo, C., Mathiot, A., et al. (2018) · The Lancet · doi:10.1016/s0140-6736(18)31782-3 · Cited by 302 · see citing works
School Discipline as a Turning Point
Mowen, T. J., & Brent, J. J. (2016) · Journal of Research in Crime and Delinquency · doi:10.1177/0022427816643135 · Cited by 186 · cited 1× · see citing works
“…likely to have worse academic outcomes, to be held back a grade level or leave school, and have a greater risk of substance use, arrest, or incarceration (Anderson et al., 2019; Dong & Krohn, 2020; Hwang, 2018; Lacoe & Steinberg, 2019; Mowen & Brent, 2016; Noltemeyer et al., 2015; Rosenbaum, 2020).”
BackgroundRestorative Interventions and School Discipline Sanctions in a Large Urban School District
Anyon, Y., Gregory, A., Stone, S., Farrar, J., Jenson, J. M., McQueen, J., Downing, B., Greer, E., & Simmons, J. (2016) · American Educational Research Journal · doi:10.3102/0002831216675719 · Cited by 181 · cited 2× · see citing works
“Quantitative (descriptive or correlational) studies on RP have generally shown that RP use is related to lower suspensions (Anyon et al., 2016; Gregory et al., 2018), fewer behavioral incidents (Boulton & Mirsky, 2006), improved attendance rates (Gonzalez, 2012), and higher academic achievement (Darling-Hammond et al.”
BackgroundResultSelf-citationAugustine, C., Engberg, J., Grimm, G., Lee, E., Wang, E., Christianson, K., & Joseph, A. (2018) · doi:10.7249/rr2840 · Cited by 176 · see citing works
Educational and Criminal Justice Outcomes 12 Years After School Suspension
Rosenbaum, J. (2020) · Youth & Society · doi:10.1177/0044118x17752208 · Cited by 174 · cited 5× · see citing works
“However, several studies have indicated that students who are suspended from school, relative to their peers, are more likely to have worse academic outcomes, to be held back a grade level or leave school, and have a greater risk of substance use, arrest, or incarceration (Anderson et al., 2019; Dong & Krohn, 2020; Hw…”
BackgroundInfluential(2012) · Scholar Commons (University of South Carolina) · Cited by 173 · see citing works
Acosta, J., Chinman, M., Ebener, P., Malone, P. S., Phillips, A., & Wilks, A. (2019) · Journal of Youth and Adolescence · doi:10.1007/s10964-019-01013-2 · Cited by 166 · cited 1× · see citing works
“, 2020), only two cluster randomized controlled (CRCTs) studies involving RPs, long considered the gold standard in providing evidence of a causal relationship in evaluation studies, have been conducted, specifically in the USA, with mixed results (Acosta et al., 2019; Augustine et al., 2018).”
BackgroundRao, J. N. K., & Scott, A. J. (1981) · Journal of the American Statistical Association · doi:10.2307/2287815 · Cited by 150 · see citing works
LaHuis, D. M., & Ferguson, M. W. (2007) · Organizational Research Methods · doi:10.1177/1094428107308984 · Cited by 145 · see citing works
Do Suspensions Affect Student Outcomes?
Lacoe, J. R., & Steinberg, M. P. (2018) · Educational Evaluation and Policy Analysis · doi:10.3102/0162373718794897 · Cited by 132 · see citing works
Restorative Justice in U.S. Schools: A Research Review.
Fronius, T. A., Persson, H., Guckenburg, S., Hurley, N., & Petrosino, A. J. (2016) · Cited by 128 · see citing works
An Examination of Restorative Interventions and Racial Equity in Out-of-School Suspensions
Gregory, A., Huang, F. L., Anyon, Y., Greer, E., & Downing, B. (2018) · School Psychology Review · doi:10.17105/spr-2017-0073.v47-2 · Cited by 127 · see citing works
Self-citationAnderson, K. P., Ritter, G. W., & Zamarro, G. (2019) · Educational Researcher · doi:10.3102/0013189x19848720 · Cited by 111 · cited 1× · see citing works
“However, several studies have indicated that students who are suspended from school, relative to their peers, are more likely to have worse academic outcomes, to be held back a grade level or leave school, and have a greater risk of substance use, arrest, or incarceration (Anderson et al., 2019; Dong & Krohn, 2020; Hw…”
Background(2019) · Cited by 108 · see citing works
Alternatives to logistic regression models in experimental studies
Huang, F. L. (2019) · The Journal of Experimental Education · doi:10.1080/00220973.2019.1699769 · Cited by 107 · see citing works
Self-citationEffectiveness of restorative justice in US K-12 schools: A review of quantitative research
Darling-Hammond, S., Fronius, T. A., Sutherland, H., Guckenburg, S., Petrosino, A., & Hurley, N. (2020) · Contemporary School Psychology · doi:10.1007/s40688-020-00290-0 · Cited by 107 · see citing works
Losen, D. J., & Martínez, P. (2020) · eScholarship (California Digital Library) · Cited by 105 · see citing works
Hashim, A. K., Strunk, K. O., & Dhaliwal, T. K. (2018) · Peabody Journal of Education · doi:10.1080/0161956x.2018.1435040 · Cited by 88 · cited 2× · see citing works
“Other studies have shown that RP may not necessarily lower suspension gaps between Black and non-Black students as well as students with and without disabilities (Anyon et al., 2016; Hashim et al., 2018).”
BackgroundResultMind the Gap: A Systematic Review of Research on Restorative Practices in Schools
Zakszeski, B., & Rutherford, L. (2021) · School Psychology Review · doi:10.1080/2372966x.2020.1852056 · Cited by 86 · cited 4× · see citing works
“…country are seeking alternatives to suspensions and are already adopting RPs at a fast rate (Diliberti et al., 2019), despite the lack of a compelling evidence base, “it is imperative to invest in well-designed experimental (i.e., cluster randomized) trials” (Zakszeski & Rutherford, 2021, p. 12).”
BackgroundInfluentialKline, D. M. S. (2016) · Multicultural Perspectives · doi:10.1080/15210960.2016.1159099 · Cited by 82 · cited 3× · see citing works
“RPs in schools include several types of programs and no one single definition exists in the literature, though RPs are often seen as both a nonpunitive approach to dealing with conflict (Fronius et al., 2016) and as a means of building community (Kline, 2016).”
BackgroundInfluentialThe Cart Before the Horse: The Challenge and Promise of Restorative Justice Consultation in Schools
Song, S. Y., & Swearer, S. M. (2016) · Journal of Educational and Psychological Consultation · doi:10.1080/10474412.2016.1246972 · Cited by 72 · cited 2× · see citing works
“S, VA 22904 Charlottesville, USA 1 3 However, despite RPs growing popularity, there is a lack of experimental studies that have investigated its efficacy (Song & Swearer, 2016; Zakszeski & Rutherford, 2021).”
BackgroundUsing cluster-robust standard errors when analyzing group-randomized trials with few clusters
Huang, F. L., & Li, X.-T. (2021) · Behavior Research Methods · doi:10.3758/s13428-021-01627-0 · Cited by 67 · see citing works
Self-citationMansfield, K. C., Fowler, B., & Rainbolt, S. (2018) · Educational Administration Quarterly · doi:10.1177/0013161x17751178 · Cited by 65 · cited 1× · see citing works
“, 3 to 5 years) to fully implement (Glenn et al., 2020; Hashim et al., 2018; Mansfield et al., 2018) and that RP coordinators at the school may still be building relationships with school staff during the first year, it is uncertain how effective the RP initiative may have been in the longer term.”
BackgroundSuspensions and Achievement: Varying Links by Type, Frequency, and Subgroup
Hwang, N. (2018) · Educational Researcher · doi:10.3102/0013189x18779579 · Cited by 59 · cited 2× · see citing works
“However, several studies have indicated that students who are suspended from school, relative to their peers, are more likely to have worse academic outcomes, to be held back a grade level or leave school, and have a greater risk of substance use, arrest, or incarceration (Anderson et al., 2019; Dong & Krohn, 2020; Hw…”
BackgroundA scalable empathic-mindset intervention reduces group disparities in school suspensions
Okonofua, J. A., Goyer, J. P., Lindsay, C. A., Haugabrook, J., & Walton, G. M. (2022) · Science Advances · doi:10.1126/sciadv.abj0691 · Cited by 57 · cited 2× · see citing works
“Another set of studies also demonstrated effects for students with prior suspensions: in evaluating two separate empathetic mindset interventions for teachers with the aim of improving student–teacher relationships, an interaction effect for the intervention condition and prior student suspension was statistically sig…”
BackgroundRodriguez, L. A., & Welsh, R. O. (2022) · AERA Open · doi:10.1177/23328584221083669 · Cited by 47 · cited 2× · see citing works
“As done by Rodriguez and Welsh (2022), we combined the latter two categories as these represent students who are not allowed to attend school, similar to what is more commonly referred to as an OSS.”
Background(2014) · RePEc: Research Papers in Economics · Cited by 46 · see citing works
Prior Problem Behaviors Do Not Account for the Racial Suspension Gap
Huang, F. L. (2020) · Educational Researcher · doi:10.3102/0013189x20932474 · Cited by 42 · cited 1× · see citing works
“In addition, the disparities in the use of exclusionary discipline have been well documented for several decades (Huang, 2020).”
BackgroundSelf-citationAccounting for Heteroskedasticity Resulting from Between-Group Differences in Multilevel Models
Huang, F. L., Wiedermann, W., & Zhang, B. (2022) · Multivariate Behavioral Research · doi:10.1080/00273171.2022.2077290 · Cited by 39 · see citing works
Self-citationImplementing Restorative Practices in Schools
Thorsborne, M., & Blood, P. (2013) · Jessica Kingsley Publishers eBooks · doi:10.5040/9781805014478 · Cited by 39 · see citing works
Gregory, A., Huang, F., & Ward-Seidel, A. R. (2022) · Journal of School Psychology · doi:10.1016/j.jsp.2022.09.003 · Cited by 31 · cited 2× · see citing works
“We found that overall, students in the RP Project schools were less likely to receive a discipline incident record as compared to students in the comparison schools (Gregory et al., 2022).”
ResultInfluentialSelf-citationHuang, F. L. (2021) · Prevention Science · doi:10.1007/s11121-021-01228-5 · Cited by 28 · cited 1× · see citing works
“LPMs are valid for experimental studies (Deke, 2014; Huang, 2021, 2022) though we had prespecified robustness checks using logistic regression models as well.”
MethodSelf-citationEliminating the Racial Disparity in Classroom Exclusionary Discipline
Gregory, A., Allen, J. P., Mikami, A. Y., Hafen, C. A., & Pianta, R. (2015) · Journal of Applied Research on Children: Informing Policy for Children at Risk · doi:10.58464/2155-5834.1212 · Cited by 26 · see citing works
Self-citationDiliberti, M. K., Kemp, J., Hansen, R., et al. (2017) · Cited by 26 · see citing works
Dong, B., & Krohn, M. D. (2019) · Justice Quarterly · doi:10.1080/07418825.2018.1561924 · Cited by 22 · see citing works
Restorative Practices in Schools
Darling-Hammond, S., & Fronius, T. A. (2022) · doi:10.4324/9781003275312-6 · Cited by 14 · see citing works
Using Robust Standard Errors for the Analysis of Binary Outcomes with a Small Number of Clusters
Huang, F. L., Zhang, B., & Li, X. (2022) · Journal of Research on Educational Effectiveness · doi:10.1080/19345747.2022.2100301 · Cited by 13 · see citing works
Self-citation(2006) · Reclaiming Children and Youth · Cited by 9 · see citing works
Investigating the value of restorative practice
Standing, V., Fearon, C., & Dee, T. (2012) · International Journal of Educational Management · doi:10.1108/09513541211227764 · Cited by 7 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- Boyes-Watson, C., & Pranis, K. (2015). Circle forward: Building a restorative school community. Living Justice Press.
- Evans, K., & Vaandering, D. (2016). The little book of restorative justice in education: Fostering responsibility, healing, and hope in schools. Simon and Schuster.
Cited by
Sorted by citations
Marcucci, O., Grant, A. A., Rice, E., Parker, A., Satchell, T. W., Alicea, M., Phillips, A., & Cruz, R. A. (2025) · Review of Educational Research · doi:10.3102/00346543251329595 · Cited by 11 · see citing works
Positive student-teacher relationships and exclusionary discipline practices
Eddy, C. L., Huang, F. L., Prewett, S. L., Herman, K. C., Hrabal, K. M., de Marchena, S. L., & Reinke, W. M. (2024) · Journal of School Psychology · doi:10.1016/j.jsp.2024.101314 · Cited by 9 · see citing works
Self-citationGregory, A., Huang, F. L., & Ward-Seidel, A. R. (2024) · AERA Open · doi:10.1177/23328584241288525 · Cited by 9 · see citing works
“This study builds directly on prior findings using data from the same experimental trial of the restorative practice project (Gregory et al., 2022; Huang et al., 2023).”
Self-citationBehavioral Threat Assessment and Equity in Exclusionary School Discipline
Cornell, D. G., Maeng, J. L., Winter, S. D., Huang, F. L., Konold, T. R., Kerere, J., Afolabi, K., & Cowley, D. (2025) · School Psychology Review · doi:10.1080/2372966x.2025.2457006 · Cited by 8 · see citing works
“Several reviews of these programs have concluded that, when properly implemented, they produce some overall reductions in exclusionary discipline, but racial disparities remain large (Huang et al., 2023; Ryberg et al., 2021; Valdebenito et al., 2018; Welsh, 2022).”
Self-citationAdvanced Categorical Data Analysis in Prevention Science
Wiedermann, W., Bonifay, W. E., & Huang, F. L. (2023) · Prevention Science · doi:10.1007/s11121-022-01485-y · Cited by 6 · see citing works
Self-citationThe Seasonality of School Climate
Huang, F. L., Zhang, B., Reinke, W. M., Herman, K. C., & Sebastian, J. L. (2023) · School Psychology Review · doi:10.1080/2372966x.2023.2176160 · Cited by 6 · see citing works
“%) of disciplinary incidents and sanctioning (e.g., suspensions, office referrals) occurred from March to the end of the school year (Huang et al., 2023 ).”
Self-citationSchool‐Based Interventions for Reducing Disciplinary School Exclusion. An Updated Systematic Review
Valdebenito, S., Gaffney, H., Arosemena‐Burbano, M. J., Hitchcock, S., Jolliffe, D., & Sutherland, A. (2025) · Campbell Systematic Reviews · doi:10.1002/cl2.70063 · Cited by 4 · see citing works
Vincent, C. G., McClure, H. H., Svanks, R., Girvan, E. J., Inglish, J., Reiley, D., & Smith, S. (2023) · Journal of Research in Innovative Teaching & Learning · doi:10.1108/jrit-03-2023-0028 · Cited by 4 · see citing works
Gregory, A., Moya, G., Jimenez, S., Zenou, J., Ward-Seidel, A. R., & Huang, F. L. (2023) · Journal of Educational and Psychological Consultation · doi:10.1080/10474412.2023.2234365 · Cited by 3 · see citing works
“The school was part of a two-year 18-school federally-funded experimental trial examining the RP Projects’ effectiveness in reducing the use of exclu-sionary discipline (Gregory et al., 2022; Huang et al., 2023; Manassah et al., 2018).”
MethodSelf-citationInterventions to Address Racism in Disciplinary Actions in K-12 Schools: A Systematic Review
Scott, B. A., Stilwell, S. M., Pearson, Z. V., Zimmerman, M. A., Hsieh, H., & Heinze, J. E. (2025) · Prevention Science · doi:10.1007/s11121-025-01855-2 · Cited by 4 · see citing works
“…not culturally relevant (Borman et al., 2022; Goyer et al., 2019; Gregory et al., 2014, 2016, 2019; Okonofua et al., 2020); and three articles had an intervention that neither reduced racial disparities nor was culturally relevant (Huang et al., 2023; McIntosh et al., 2020; Vincent et al., 2023).”
Restorative Justice Processes in K-12 Schools: A Scoping Review
Gen, B. M., Wojtowicz, O. H., & Johnson, N. L. (2025) · Journal of Prevention and Health Promotion · doi:10.1177/26320770251384671 · Cited by 1 · see citing works
“Reductions in punitive discipline were evident, though not always widespread (Davison et al., 2022; Huang et al., 2023); Huang et al. (2023) found that RJP-involvement was associated with reduced suspensions, but only for students who had received a prior suspension.”
InfluentialInformal exclusionary discipline practices in US schools: Recent evidence and policy considerations
Steyer, L., Gleit, R. D., Hawkins, C., Provençal, M., Pearman, F. A., & Obradović, J. (2025) · Child Policy Nexus · doi:10.1002/sop2.70000 · Cited by 1 · see citing works
Lind, A., Mason, S. M., & Brady, S. S. (2024) · Preventive Medicine · doi:10.1016/j.ypmed.2024.107917 · Cited by 2 · see citing works
New Frontiers in Prevention Research Models: Commentary on the Special Issue
Wood, P. K. (2023) · Prevention Science · doi:10.1007/s11121-023-01508-2 · Cited by 1 · see citing works
Restorative Justice and Relational Accountability
Gutierrez, G. “. (2026) · IGI Global eBooks · doi:10.4018/979-8-3373-8382-8.ch010
Iñiguez, J. A. S., Ramírez, J. E. G., & Gaona, A. J. A. (2026) · LÍNEA IMAGINARIA · doi:10.56219/lneaimaginaria.v25i1.5846
Sergeeva, T. V., & Samsonova, N. V. (2026) · Samara Journal of Science · doi:10.55355/snv2026152319
McCluskey, G. (2026) · Springer series on Asian criminology and criminal justice research · doi:10.1007/978-3-032-22142-1_11
Not a Neutral Space: Early Childhood Education as a Site of Exclusion and Liberation
Morris, C. T., Lumpkins, A., Fox, L., & Lansing, D. (2025) · Youth · doi:10.3390/youth5040126 · Cited by 1 · see citing works
“Restorative circles, a practice drawn from restorative justice and Indigenous traditions, emphasizes dialog and community repair (Huang et al., 2023).”
Building TRUST: Empowering Teachers to Support Students With Emotional and Behavioral Disorders
Mitchell, C. C., Gershwin, T., Robinson, J. D., & Hynes, K. C. (2025) · Intervention in School and Clinic · doi:10.1177/10534512251336158
“…and teachers better understand each other’s perspectives, support students with understanding the impact of their behavior on others, and collaboratively brainstorm socially acceptable strategies students can use to prevent future challenging behaviors (Huang et al., 2023; Lodi et al., 2021).”
- Authors
- Huang, F. L., Gregory, A., & Ward-Seidel, A. R.
- Affiliations
- University of Missouri (US)
- Rutgers, The State University of New Jersey (US)
- University of Virginia (US)
- Corresponding author
- Huang, Francis L.
- Venue
- Prevention Science
- Published
- 2023-02-28
- Volume
- 24
- Issue
- 5
- Pages
- 962-973
- ISSN
- 1389-4986 (ISSN-L)
- 1573-6695
- PMC
- PMC9972315
- OpenAlex
- W4322617060
- Semantic Scholar
- d725cbc924be356cc9f380d7cd3c8d597e8ed69c
- Cited by
- 21
- References
- 46
- Citations by year
- 2026: 5 · 2025: 8 · 2024: 3 · 2023: 5
- FWCI
- 15.49
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- Prevention Science (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- U.S. Department of Education (U411C160035)
- Topics
- Social Sciences › Education › Education Discipline and Inequality
- Social Sciences › Law › Legal Issues in Education
- Social Sciences › Psychology › Developmental and Educational Psychology › Behavioral and Psychological Studies
- Keywords
- restorative practices, out-of-school suspension, student discipline, exclusionary discipline, K-12 schools, urban schools, gender differences
- MeSH terms
- Ethnicity, Humans, Schools, Students, Suspensions
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-09-28.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-09-28.
Cited sources are checked by gpt-6-sol.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review