CoChat
Sign Up
PreprintCited by 15Criminal Justice and Corrections AnalysisCriminal Justice and PenologyCrime Patterns and Interventions

Can Restorative Justice Conferencing Reduce Recidivism? Evidence From the Make-it-Right Program

Open access
Shem-Tov, Yotam, Raphael, Steven, Skog, Alissa·2021·National Bureau of Economic Research
DOI 10.3386/w29150
Open this paper in CoChat — free account

Abstract

This paper studies the effect of a restorative justice intervention targeted at youth ages 13 to 17 facing felony charges of medium severity (e.g., burglary, assault). Eligible youths were randomly assigned to participate in the Make-it-Right (MIR) restorative justice program or a control group where they faced standard criminal prosecution. We estimate the effects of MIR on the likelihood that a youth will be rearrested in the four years following randomization. Assignment to MIR reduces the probability of a rearrest within six months by 19 percentage points, a 44 percent reduction relative to the control group. Moreover, the reduction in recidivism persists even four years after randomization. Thus, our estimates show that restorative justice conferencing can reduce recidivism among youth charged with relatively serious offenses and can be an effective alternative to traditional criminal justice practices.

AI snapshot

Written from the paper’s full text.

TL;DR

This randomized evaluation of San Francisco’s Make-it-Right program finds that assigning juveniles charged with medium-severity felonies to restorative justice conferencing instead of ordinary felony prosecution substantially lowered subsequent rearrest. The reductions appeared within six months and persisted through four years, though the trial enrolled only 143 youths.

Key findings

  • Assignment to Make-it-Right reduced rearrest within six months by 18.9 percentage points (ITT), a 44% reduction relative to the control complier mean; control-group rearrest was 43% at six months and 75% at four years. At four years, the ITT reduction was 26.7 percentage points, or 30% of the control complier mean.
  • Treatment-on-the-treated estimates for enrollment were roughly 1.3–1.4 times the ITT effects. Completion effects were larger: completing the program reduced six- and twelve-month rearrest by about 36 percentage points, corresponding to 76% and 65% reductions relative to control complier means.
  • Take-up was high: 81% of those assigned to Make-it-Right enrolled, 52.5% of those assigned completed, and 66.7% of enrollees completed. Among youths who participated in a conference, 95% fulfilled the agreement and completed the program.
  • Make-it-Right reduced felony convictions (5% vs. 20% in the control group), but effects persisted among youths whose cases did not result in a felony conviction, suggesting the conferencing component—not only diversion from conviction—drives the recidivism reductions. The control group’s rearrest curve was similar to that of the full population of San Francisco juveniles charged with felonies, and equality could not be rejected.

Methods

  • Randomized controlled trial at the case level in San Francisco. Youths aged 13–17 charged with medium-severity felonies such as theft, burglary, or assault were assigned to Make-it-Right or standard felony prosecution; 99 were assigned to treatment and 44 to control, totaling 143 youths, or 13% of juveniles charged with a felony during the period.
  • Data included program records and administrative juvenile and adult arrest records from October 2010 to November 2020. Outcomes measured rearrest within 6, 12, 24, 36, and 48 months, plus the 12-to-48-month window.
  • Analysis used intent-to-treat comparisons, Kaplan-Meier failure functions, and two-stage least squares with random assignment as an instrument for enrollment or completion. Inference used case-clustered standard errors and randomization inference with 1,000 permutations; hypothesis tests were pre-specified as one-sided.

Limitations

  • The authors note the small sample of 143 youths and discuss statistical power, though they argue the effects are large enough to support conclusions and estimate very low sign-error probabilities and a modest exaggeration ratio of about 1.2.
  • The authors state they lack exact cost estimates for Make-it-Right relative to standard prosecution and did not evaluate whether effect sizes vary by offense severity. Their pre-specified one-sided tests also mean they cannot test whether the program increases arrest likelihood.
  • Evident from the text: eligibility was not random—youths were excluded for factors such as gang affiliation, weapon use, prior 707(b) arrests, moderate or significant victim injury, and probation or detention status—so external validity depends on the similarity between the control group and the broader felony-charged population. Long-run estimates rely on earlier cohorts with longer follow-up, and later observation periods overlap with COVID-19 restrictions.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.3386/w29150

Cites

94 of 94 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

4 of 30 cited sources checked · 3 supported · 0 flagged · accuracy not yet measured

  • Findingown data

    43 percent of control group members are rearrested within six months.

    “43 percent of control group members are rearrested within six months”
    Quote in the paperSubstantial claim
  • Findingown data

    After treatment, juveniles are less likely to be convicted of a new offense.

    “Correspondingly, after treatment, juveniles are also less likely to be convicted of a new offense.”
    Quote in the paperSubstantial claim
  • Findingown data

    Among youth assigned to MIR, only five percent of cases result in a felony conviction relative to 20 percent in the control group.

    “among youth assigned to MIR, only five percent of the cases result in a felony conviction relative to 20 percent in the control group.”
    Quote in the paperSubstantial claim
  • Findingown data

    An F-test of the overall significance of a regression of a treatment dummy on the covariate list yields a p-value of 0.789.

    “While there are a few differences, an F-test of the overall significance of a regression of a treatment dummy on the covariate list yields a p-value of 0.789.”
    Quote in the paperNot a substantial claim
  • Findingown data

    The difference in the percentage ever arrested on a new offense between treatment and control reaches roughly 20 percentage points within six months.

    “The difference in the percentage ever arrested on a new offense reaches roughly 20 percentage points within six months.”
    Quote in the paperSubstantial claim
  • Classified as probably not a substantial claim. 38
  • May combine two claims. 42
  • The quote only partly states this claim. 6

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Heller et al. (2017) conducted a randomized controlled trial among youth in Chicago and showed that cognitive-behavioral therapy changes a youth's decision-making, reduces criminal behavior, and increases high school graduation.

    “Heller et al. (2017) conduct a randomized controlled trail (RCT) among youth in Chicago and show that CBT changes a youth's decision-making, reduces criminal behavior, and increases high school graduation.”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Heller et al. (2017) source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The effects of cognitive-behavioral therapy dissipate and are not significant after a year unless combined with a monetary grant.

    “However, the effects dissipate and are not significant after a year unless combined with a monetary grant.”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Blattman et al. (2017) source

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Blattman et al. (2017) conducted a randomized controlled trial in Liberia recruiting high-risk men aged 18-35 and found that cognitive-behavioral therapy reduces anti-social behavior.

    “Blattman et al. (2017) conduct an RCT in Liberia recruiting high-risk men aged 18-35.They find that CBT reduces anti-social behavior.”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Blattman et al. (2017) source

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Randomization inference yields valid p-values for two null hypotheses: finite-sample exact inference on the sharp null of no treatment effect on all individuals, and asymptotically valid inference on the null hypothesis of no average treatment effect.

    “Note that randomization inference yields valid p-values for two null hypotheses.First, it provides finite-sample exact inference on the sharp null of no treatment effect on all individuals.Second, when the sharp null hypothesis is incorrect (e.g., some units are impacted positively while others negatively), randomization inference still provides asymptotically valid inference on the null hypothesi”
    Quote in the paperNot a substantial claim1 of 3 sources checked
    • cites Li and Ding (2017) source
    • cites Wu and Ding (2020) source
    • cites Zhao and Ding (2021) source

      Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

      Experimental — accuracy not yet measured

  • May combine two claims.

    Findingown data

    Two-stage least squares estimates of the treatment-on-the-treated effect with respect to enrollment and completion are roughly 1.3 and 1.9 times larger than the intent-to-treat effects.

    “Two-stage least squares (2SLS) estimates of the TOT with respect to enrollment and completion are roughly 1. 3 and 1.9 times larger than the ITT effects.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    The experiment included 143 youth, constituting 13 percent of all juveniles charged with a felony in San Francisco during the time period.

    “The experiment included 143 youth, constituting 13 percent of all juveniles charged with a felony in San Francisco during the time period.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Nearly half of the control group was arrested within six months of randomization, and over 70 percent were rearrested by the end of the four-year period.

    “For both groups, the likelihood of rearrest is quite high, with nearly half of the control group arrested within six months of randomization and over 70 percent rearrested by the end of the four-year period.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Juveniles assigned to MIR are less likely to be subsequently arrested for any reason, for felonies, or for more severe offenses.

    “Juveniles assigned to MIR are less likely to be subsequently arrested for any reason, for felonies, or for more severe offenses.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingnone given

    Among those enrolled in MIR, 66.7 percent completed the program, and as a result, formal charges were never filed against them.

    “Among those enrolled in MIR, 66.7 percent completed the program, and as a result, formal charges were never filed against them.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Over a third of non-MIR juveniles are arrested for severe person offenses (2 percent homicide, 1.4 percent sex offense, 38.7 percent robbery) compared with roughly two percent of MIR youth arrested for robbery.

    “Specifically, over a third of non-MIR juveniles are arrested for severe person offenses (2 percent for homicide, 1.4 percent for a sex offense, and 38.7 percent for robbery) compared with roughly two percent of MIR youth arrested for robbery.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Relative to the broader population of juveniles charged with felony offenses, MIR youth are more likely to be male, somewhat less likely to be Black or Hispanic, are less likely to have a prior arrest, and have a first arrest occurring at a slightly older age.

    “Relative to the broader population of juveniles charged with felony offenses, described in Column (4), the MIR youth are more likely to be male, somewhat less likely to be Black or Hispanic, are less likely to have a prior arrest, and have a first arrest occurring at a slightly older age (approximately 15 vs. 14.5).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The study finds economically meaningful and statistically significant estimates that MIR reduces recidivism both in the short-run and in the long run after four years from randomization.

    “Lastly, we find economically meaningful and statistically significant estimates that MIR reduces recidivism both in the short-run and in the long run after four years from randomization.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given

    Take-up rates for those assigned to the MIR treatment were relatively high, especially compared to past restorative justice experiments.

    “Take-up rates for those assigned to the MIR treatment were relatively high, especially compared to past restorative justice experiments.”
    Quote in the paperNot a substantial claim

  • The quote only partly states this claim.

    Findingown data

    Assignment to MIR (ITT) reduces the likelihood of being arrested within 6 months (estimate -0.189, SE 0.084, one-sided p=0.0132).

    “| Assigned to MIR (ITT) | -0.189 | -0.184 | -0.144 | -0.147 | -0.267 | -0.270 |”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The ITT effects of MIR on the likelihood of being rearrested range from 14 to 27 percentage points, which is meaningfully larger than the effects of MIR on the case not resulting in a felony conviction.

    “Furthermore, the ITT effects of MIR on the likelihood of being rearrested range from 14 to 27 percentage points, which is meaningfully larger than the effects of MIR on the case not resulting in a felony conviction.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Large disparities in the failure functions open up soon after the initial arrest and persist throughout the observation period.

    “Large disparities in the failure functions open up soon after the initial arrest and persist throughout the observation period.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingcitation

    The probability of a sign error in the effects of enrollment to MIR is 0.00002 and 0.00003 for rearrests within one and four years, respectively.

    “Following the procedure proposed by Gelman and Carlin (2014), we estimate the probability of a sign error in the effects of enrollment to MIR to be 0.00002 and 0.00003 for rearrests within one and four years, for example.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Gelman and Carlin (2014) source

  • May combine two claims.

    Findingown data

    Even among the sub-sample whose case did not result in a felony conviction, youth assigned to MIR had a meaningfully lower likelihood of being rearrested both in the first few months and after four years from randomization.

    “Figure 4 shows that even among this sub-sample, youth participants assigned to MIR had a meaningfully lower likelihood of being rearrested both in the first few months as well as after four years from randomization.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Relative to the control-complier means, the TOT estimates range from 30 percent of the recidivism rate for the 24-month outcome to 54 percent of recidivism occurring within six months and 51 percent of recidivism occurring in years one through four.

    “Relative to the control-complier means, the TOT estimates range from 30 percent of the recidivism rate for the 24-month outcome to 54 percent of recidivism occurring within six months and 51 percent of recidivism occurring in years one through four.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Similar patterns emerge for felony rearrest, rearrests for new offenses as severe as the original offense, or rearrests that lead to a conviction.

    “Similar patterns also emerge when examining effects on more severe interactions such as felony rearrest, rearrests for new offenses that are as severe as the original offense, or rearrests that lead to a conviction.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingreasoning

    Relative to the control-complier means, the completion effects are reductions of 76 and 65 percent for six and 12 months.

    “Relative to the control-complier means, these are reductions of 76 and 65 percent.”
    Quote in the paperSubstantial claim

  • The quote only partly states this claim.

    Findingown data

    Completing MIR reduces the likelihood of being arrested within 48 months (estimate -0.525, SE 0.214, one-sided p=0.0084).

    “| Completed MIR (treated) | -0.361 | -0.351 | -0.268 | -0.286 | -0.525 | -0.531 |”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The empirical failure function of all juveniles accused of a felony is similar to that of the MIR control group, and the null hypothesis that the two curves are equal cannot be rejected.

    “Interestingly, the empirical failure function of all juveniles accused of a felony is similar to that of the control group, and we cannot reject the null hypothesis that the two curves are equal.”
    Quote in the paperSubstantial claim

  • The quote only partly states this claim.

    Findingown data

    Participation in MIR (TOT) reduces the likelihood of being arrested within 6 months (estimate -0.234, SE 0.103, one-sided p=0.012).

    “| Participated in MIR (treated) | -0.234 | -0.228 | -0.184 | -0.196 | -0.363 | -0.368 |”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The predicted future arrest index is highly correlated with the likelihood of a new arrest observed in the experimental sample, and a coefficient of one cannot be rejected.

    “Appendix Figure B.1 shows that the predicted future arrest index is highly correlated with the likelihood of a new arrest observed in the experimental sample, and we cannot reject a coefficient of one.”
    Quote in the paperNot a substantial claim

  • The quote only partly states this claim.

    Findingown data

    Participation in MIR (TOT) reduces the likelihood of being arrested within 12 months (estimate -0.228, SE 0.111, one-sided p=0.0211).

    “| Participated in MIR (treated) | -0.234 | -0.228 | -0.184 | -0.196 | -0.363 | -0.368 |”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The reduction in future arrest is observed for arrests overall and for more severe felony arrests.

    “We observe this for arrests overall and for more severe felony arrests.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingreasoning

    Relative to the control complier means, the ITT effect sizes imply a 44 percent reduction in recidivism within six months, 33 percent within one year, and 24 percent within two years.

    “Relative to the control complier means, these effect sizes imply a 44 percent, 33 percent and 24 percent reduction in recidivism, respectively.”
    Quote in the paperSubstantial claim

  • The quote only partly states this claim.

    Findingown data

    Assignment to MIR (ITT) reduces the likelihood of being arrested within 48 months (estimate -0.267, SE 0.133, one-sided p=0.0249).

    “| Assigned to MIR (ITT) | -0.189 | -0.184 | -0.144 | -0.147 | -0.267 | -0.270 |”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The overall ITT effect sizes hold up if rearrest is measured within three years (14.7 percentage points, equivalent to 20 percent of the control-complier mean) and widen for four years (26.7 percentage points, equivalent to 30 percent of the control-complier mean).

    “The overall effect sizes hold up if we measure rearrest within three years of randomization (14.7 percentage points equivalent to 20 percent of the control-complier mean) and widens for four years (26.7 percentage points equivalent to 30 percent of the control-complier mean).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Both the standard non-parametric asymptotic test and randomization inference reject the null hypothesis of equal failure functions, with p-values of 0.0071 and 0.017 respectively.

    “Both tests reject the null hypothesis of equal failure functions, with the p-value from the randomization inference (p = 0.017) roughly 2.5 times the value based on the standard non-parametric asymptotic inference (p = 0.0071).”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The estimates are largest for the 2016-2017 cohort.

    “The estimates are the largest for the 2016-2017 cohort.”
    Quote in the paperNot a substantial claim

  • The quote only partly states this claim.

    Findingown data

    Participation in MIR (TOT) reduces the likelihood of being arrested within 48 months (estimate -0.363, SE 0.165, one-sided p=0.0157).

    “| Participated in MIR (treated) | -0.234 | -0.228 | -0.184 | -0.196 | -0.363 | -0.368 |”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The take-up rate is not affected by whether or not cohort fixed effects are included in the model.

    “Moreover, the take-up rate is not affected by whether or not cohort fixed effects are included in the model.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    The take-up rate is about 80 percent.

    “The take-up rate is about 80 percent.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingreasoning

    Among individuals assigned to MIR, 19.2% were determined by Community Works as unsuitable and faced standard criminal prosecution for a felony offense.

    “19.2% will be determined by CW as unsuitable and will face a standard criminal prosecution for a felony offense.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    Among youths who enrolled in MIR, 66.7% continued to complete the program.

    “Among the youths who enrolled into MIR, 66.7% will continue to complete the program”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given

    Among individuals assigned to MIR, 80.8% were found suitable by Community Works and continued to enroll in MIR.

    “Among the individuals assigned to MIR, 80.8% will be found suitable by CW and continue to enroll in MIR”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingreasoning

    Among youths who enrolled in MIR, 33.3% did not complete the program and faced standard prosecution for their felony offense.

    “Another 33.3% will not succeed to complete the program and will face standard prosecution for their felony offense.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    McCold and Wachtel (1998) evaluate a family-group conference intervention in Pennsylvania and find no evidence of reductions in recidivism after one year.

    “McCold and Wachtel (1998) evaluate a family-group conference intervention in Pennsylvania and find no evidence of reductions in recidivism after one year.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites McCold and Wachtel (1998)

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationown data

    The analysis complements prior findings by showing that, similar to cognitive-behavioral therapy, interventions that appeal to one's sense of responsibility and specifically encourage pro-social behaviors have the potential to reduce recidivism.

    “Our analysis complements these findings by showing that similar to CBT, interventions that appeal to one's sense of responsibility (and perhaps shame) and, unlike CBT, specifically encourage pro-social behaviors have the potential to reduce recidivism.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    There is some weak evidence in sub-group analyses of increased offending, though the estimates are underpowered and appear to be exploratory.

    “However, there is some weak evidence in sub-group analyses of increased offending, though the estimates are underpowered and appear to be exploratory.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Sherman et al. (2015) source

  • Classified as probably not a substantial claim.

    Premisenone given

    The MIR program diverts youth who have been arrested for certain felony offenses to a restorative justice conferencing program.

    “The program diverts youth who have been arrested for certain felony offenses to a restorative justice conferencing program.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    Over time the effects of the Indiana restorative justice program disappear and no impacts on recidivism are detectable in the longer-run.

    “However, overtime the effects disappear and no impacts on recidivism are detectable in the longer-run (Jeong et al., 2012).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Jeong et al., 2012 source

  • May combine two claims.

    Interpretationown data

    MIR causes reductions in recidivism across a variety of different measures, reducing both the likelihood of any future arrests and the likelihood of arrests for relatively serious offenses.

    “These findings show that MIR causes reductions in recidivism across a variety of different measures, reducing both the likelihood of any future arrests as well as the likelihood of arrests for relatively serious offenses.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    The extant body of research generally finds that restorative justice programs are well received by victims and may help alleviate post-traumatic stress symptoms.

    “The extant body of research generally finds that restorative justice programs are well received by victims and may help alleviate post-traumatic stress symptoms”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites McCold and Wachtel, 1998

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites McGarrell, 2001

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Sherman et al., 2015 source

  • May combine two claims.

    Premisecitation

    Augustine et al. (2021) and Mueller-Smith and Schnepel (2020) find evidence that diversion programs reduce recidivism.

    “Augustine et al. (2021) exploit quasi-random assignment to judges and Mueller-Smith and Schnepel (2020) leverage natural experiments associated with shifts in diversion policy.Both studies find evidence that diversion programs reduce recidivism.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Augustine et al. (2021)

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Mueller-Smith and Schnepel (2020) source

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    Eligible youth are randomized to either the treatment or control groups after the juvenile prosecutor reaches the decision to file charges but before charges are formally filed.

    “Eligible youth are randomized to either the treatment or control groups after the juvenile prosecutor reaches the decision to file charges but before charges are formally filed.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    The Australian experiments reviewed by Sherman et al. (2015) find some evidence of a reduction in repeat offending, especially for offenses that involved a victim.

    “The Australian experiments reviewed by Sherman et al. (2015) find some evidence of a reduction in repeat offending, especially for offenses that involved a victim.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Sherman et al. (2015) source

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    The pre-analysis plan specified that the study would evaluate only one-sided hypothesis tests pertaining to whether the MIR program reduced the likelihood of recidivism.

    “Finally, in our pre-analysis plan we specified that we will evaluate only one-sided hypothesis tests pertaining to whether the MIR program reduced the likelihood of recidivism or not.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    The randomization process was designed to separate the actual treatment assignment from the prosecutor in charge of the program.

    “The randomization process was designed to separate the actual treatment assignment from the prosecutor in charge of the program.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductown data

    Eligible youths ages 13 to 17 facing felony charges of medium severity were randomly assigned to participate in the Make-it-Right (MIR) restorative justice program or a control group where they faced standard criminal prosecution.

    “Eligible youths were randomly assigned to participate in the Make-it-Right (MIR) restorative justice program or a control group where they faced standard criminal prosecution.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    McGarrell (2001) and McGarrell and Hipple (2007) evaluate a randomized controlled trial of a restorative justice program in Indiana focused on children and find meaningful decreases in recidivism within one year relative to a control group allocated to another diversion program.

    “McGarrell (2001) and McGarrell and Hipple (2007) evaluate an RCT of a restorative justice program in Indiana focused on children (the average age is 12.5).They find meaningful decreases in recidivism within one year relative to a control group allocated to another diversion program.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites McGarrell (2001)

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites McGarrell and Hipple (2007) source

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    Previous studies found that among adults, a felony conviction can increase recidivism and negatively impact labor market outcomes.

    “Previous studies found that among adults, a felony conviction can increase recidivism and negatively impact labor market outcomes (e.g., Mueller-Smith and Schnepel, 2020;Augustine et al., 2021).”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites Mueller-Smith and Schnepel, 2020 source
    • cites Augustine et al., 2021

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationown data

    Restorative justice conferencing can reduce recidivism among youth charged with relatively serious offenses and be an effective alternative to traditional criminal justice practices.

    “To conclude, our findings show that restorative justice conferencing can reduce recidivism among youth charged with relatively serious offenses and be an effective alternative to traditional criminal justice practices.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    As long as the program does not increase recidivism, it helps reduce the number of individuals involved in criminal proceedings with no additional costs.

    “Therefore, as long as the program does not increase recidivism, it helps to reduce the number of individuals involved in criminal proceedings with no additional costs.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The cost of allocating youth to MIR is not higher and might be even lower than the cost of juvenile criminal proceedings.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Restorative justice conferencing appears to be most effective when the offense involves an identifiable victim.

    “restorative justice conferencing appears to be most effective when the offense involves an identifiable victim (Sherman and Strang, 2007).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Sherman and Strang, 2007

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    The p-values in the population comparison hypothesis tests are two-sided because the pre-analysis plan did not specify how they would be conducted.

    “The p-values in these hypothesis tests are two-sided since we did not pre-specified in the pre-analysis plan how they will be conducted.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    Assignment to MIR causes a large reduction in the likelihood of future arrest, both in the immediate aftermath of the initial arrest and in the four years following.

    “Assignment to MIR causes a large reduction in the likelihood of future arrest, both in the immediate aftermath of the initial arrest as well as in the four years following.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    In California, employers generally cannot use juvenile arrests and proceedings in employment decisions, and this information does not appear in criminal history background checks.

    “in California, employers generally cannot use juvenile arrests and proceedings (e.g., charges, convictions) in employment decisions (Assembly Bill No. 1843) and this information does not appear in criminal history background checks.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Assembly Bill No. 1843

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The pre-analysis plan specified that only one-sided tests would be conducted, rejecting the null when MIR is preferable to the control regime in reducing recidivism.

    “We pre-specified in the pre-analysis plan that would to conduct only one-sided tests that reject the null when MIR is preferable to the control regime in reducing recidivism.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    The p-values in these hypothesis tests are two-sided because the pre-analysis plan did not pre-specify how they would be conducted.

    “The p-values in these hypothesis tests are two-sided since we did not pre-specified in the pre-analysis plan how they will be conducted.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    The pre-analysis plan was submitted before looking at the data and without knowing the exact sample size.

    “We submitted the pre-analysis plan before looking at the data and without knowing the exact sample size.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    The p-values in the never-takers hypothesis test are two-sided because the pre-analysis plan did not specify how they would be conducted.

    “The p-values in these hypothesis test are two-sided since we did not pre-specify in the pre-analysis plan how they would be conducted.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    The procedure for conducting covariate adjustment was pre-specified with an eye on parsimony.

    “we pre-specified our procedure for conducting covariate adjustment with an eye on parsimony.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    The experiment did not evaluate whether effect size varies by offense severity.

    “Given the relatively small scale of the experiment, we did not evaluate whether effect size varies by offense severity.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The pre-analysis plan pre-specified that only one-sided tests would be conducted, rejecting the null when MIR is preferable to the control regime in reducing recidivism.

    “We pre-specified in the pre-analysis plan that we would conduct only one-sided tests that reject the null when MIR is preferable to the control regime in reducing recidivism.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    The sample size and estimated effects are sufficient to draw firm conclusions on the effectiveness of MIR and rule out potential inference errors.

    “Thus, although small, our sample size and estimated effects are sufficient to draw firm conclusions on the effectiveness of MIR and rule out potential inference errors.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    The sample size does not change across columns because the authors restrict attention to shorter time horizons in which a balanced sample is observed.

    “Importantly, the sample size does not change across columns as we restrict attention to shorter time horizons in which a balanced sample is observed.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    The last randomization occurred in October 2019, and recidivism data are observed through November 2020.

    “Note, the last randomization occurred in October 2019, and we observe recidivism data through (including) November 2020.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Any observable imbalances between treatment and control groups are not systematic.

    “Note that any observable imbalances are not systematic; for example, individuals in the control group are less likely to have a prior felony arrest; however, they have more prior arrests on average.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Agan et al. (2020) emphasize the importance of having any criminal history, even for non-violent misdemeanor offenses, on the likelihood of recidivism.

    “Agan et al. (2020) emphasize the importance of having any criminal history, even for non-violent misdemeanor offenses, on the likelihood of recidivism.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Agan et al., 2020

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Using auxiliary observational data to improve the accuracy of experimental estimates has been proposed in other studies.

    “The idea of using auxiliary observational data to improve the accuracy of experimental estimates has been proposed in other studies (e.g.,Gagnon-Bartsch et al., 2020).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Gagnon-Bartsch et al., 2020

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductreasoning

    Both p-values are one-sided, rejecting the null hypothesis that MIR is not preferable to the control regime when the MIR Kaplan-Meier failure curve dominates that of the control units.

    “Both p-values are one-sided, rejecting the null hypothesis that MIR is not preferable to the control regime when the MIR Kaplan-Meier failure curve dominates that of the control units.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    The study controlled for a single index summarizing the predictive ability of all covariates to avoid choosing which covariates to include in the model.

    “To avoid needing to choose which covariates to include in the model, we opted for the option of controlling for a single index that summarizes the predictive ability of all the covariates as we describe in detail in Appendix B.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    The authors pre-specified in the pre-analysis plan that they would conduct only one-sided tests that reject the null when MIR is preferable to the control regime in reducing recidivism.

    “We pre-specified in the pre-analysis plan that we would conduct only one-sided tests that reject the null when MIR is preferable to the control regime in reducing recidivism.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductnone given

    MIR diverts youth before charging but after an arrest took place.

    “MIR diverts youth before charging but after an arrest took place.”
    Quote in the paperSubstantial claim
77 of 77

  • The paper relies on this without support.

    Little is known about whether criminal behavior can be influenced by less punitive interventions that use self-reflection and empathy to directly change a youth's decision-making.

    “However, little is known about whether or not criminal behavior can be influenced by less punitive interventions that use self reflection and empathy to directly change a youth's decision-making.”

  • The paper relies on this without support.

    In California, arrests, charges, and convictions in juvenile court do not show up in employment criminal history background checks.

    “One potential cause for these findings is that in California, arrests, charges, and convictions in juvenile court do not show up in employment criminal history background checks.”

  • The paper relies on this without support.

    The prosecutor does not receive information about a youth's participation in or completion of MIR, or the reasons for non-completion.

    “CW emphasizes that they do not share any information with the prosecutor that can potentially be used against the youth, including the circumstances that lead a juvenile not to participate or complete MIR.”

  • The paper relies on this without support.

    The cost of allocating youth to MIR is not higher and might be even lower than the cost of juvenile criminal proceedings.

    “in our interviews with employees of the district attorney and juvenile probation department, we have been told that the cost of allocating youth to MIR is not higher and might be even lower than the cost of juvenile criminal proceedings.”

  • The paper relies on this without support.

    The auxiliary observational data on all youth charged with felony offenses who did not participate in the experiment can be used to estimate the predicted likelihood of future arrest for study participants.

    “Specifically, we calculate the predicted likelihood of a future arrest using data on all the youth charged with a felony offenses who did not participate in the experiment (i.e., Column (4) of Table 2).”

  • The paper relies on this without support.

    The MIR program treated youth charged with more serious offenses that are typically not eligible for similar restorative justice or diversion programs.

    “The MIR program treated youth charged with more serious offenses that are typically not eligible for similar restorative justice or diversion programs.”

  • The authors state this as an assumption.

    The recidivism hazard function is stable across cohorts defined by randomization date.

    “implicitly assuming that the recidivism hazard function is stable across cohorts defined by randomization date.”

  • The paper relies on this without support.

    On March 16, 2020, San Francisco and five other Bay Area counties enacted strict shelter-in-place orders that greatly reduced social interactions outside the home and closed all in-person instruction in public schools throughout the region.

    “On March 16, 2020 San Francisco and five other Bay Area counties enacted a strict shelter-in-place orders that greatly reduced social interactions outside of the home and closed all in-person instruction in public schools throughout the region.”

  • The paper relies on this without support.

    March 15, 2020 is the beginning of COVID-19 restrictions in California.

    “March 15, 2020 Which Is the Beginning of COVID-19 Restrictions in California”
9 of 9

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Four-year control rearrest rate conflicts with Table 3

    Minor

    The text repeatedly gives a 75 percent control-group rearrest rate at four years, but Table 3 reports 0.750 at 36 months and 0.833 at 48 months. The stated four-year rate appears to be the three-year value.

  • Average and median enrollment waits are reversed

    Minor

    The prose assigns 15 days to the average and 21 days to the median; the case-flow figure assigns those values the other way around.

  • Smallest rearrest effect is not larger than the conviction effect

    Minor

    The claimed range of rearrest effects starts at 14 percentage points, below the paper's stated 15-percentage-point effect on felony conviction. Thus, not every rearrest effect in that range is larger than the conviction effect, contrary to the mechanism argument.

  • Control-group prior-arrest comparison reverses the reported averages

    Minor

    The main text reports fewer prior arrests on average in the control group than in the treatment group, while a note says the control group had more.

  • Experiment end date precedes the stated last randomization

    Minor

    The experiment is said to have concluded in May 2019, but a later note says its last randomization occurred in October 2019.

5 of 5

Cites

Flagged first, then by citations

2 of the 96 references couldn’t be matched to a known paper.

Research works printed but not linked: 3

  1. [12]12. Bhuller, Manudeep, Gordon B. Dahl, Katrine V. Løken, and Magne Mogstad, "Inter- generational Effects of Incarceration," AEA Papers and Proceedings, 2018, 108, 234-40.
  2. [13]13. and , "Incarceration, Recidivism, and Employment," Journal of Political Economy, 2020, 128 (4), 1269-1324.
  3. [78]78. Umbreit, Mark S. and Marilyn Peterson Armour, "Restorative Justice and Dialogues: Im- pact, Opportunities, and Challenges in the Global Community," Washington University Journal of Law Policy, 2011, 36, 65-89.

Cited by

Sorted by citations

Related

Authors
Shem-Tov, Y., Raphael, S., & Skog, A.
Affiliations
National Bureau of Economic Research (US)
ORCID
Venue
National Bureau of Economic Research
Publisher
National Bureau of Economic Research
Published
2021-08
DOI
10.3386/w29150
OpenAlex
W3193782519
Cited by
15
References
96
Citations by year
2025: 4 · 2024: 7 · 2023: 4
Type
report
Language
en
Open access
Locations
National Bureau of Economic Research (publishedVersion)
Funders
  • Arnold Ventures
  • Office of the President, University of California (M21PR3278)
Topics
  • Social Sciences › Sociology and Political Science › Criminal Justice and Corrections Analysis
  • Social Sciences › Law › Criminal Justice and Penology
  • Social Sciences › Sociology and Political Science › Crime Patterns and Interventions
Keywords
restorative justice, recidivism, juvenile offenders, juvenile justice, felony charges, rearrest, random assignment

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-11.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-11.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review