CoChat
Sign Up
Cited by 67Ethics and Social Impacts of AICriminal Justice and Corrections AnalysisArtificial Intelligence in Law

The DOI matches its Crossref record; 20 of the 20 references it cites were found in a registry or an index; none of the 26 sources its claims cite is checked for support yet.

The sexist algorithm

Open access
Hamilton, Melissa·2019·Behavioral Sciences & the Law
DOI 10.1002/bsl.2406
Open this paper in CoChat — free accountOpen PDF

Abstract

Algorithmic risk assessment tools are informed by scientific research concerning which factors are predictive of recidivism and thus support the evidence‐based practice movement in criminal justice. Automated assessments of individualized risk (low, medium, high) permit officials to make more effective management decisions. Computer‐generated algorithms appear to be objective and neutral. But are these algorithms actually fair? The focus herein is on gender equity. Studies confirm that women typically have far lower recidivism rates than men. This differential raises the question of how well algorithmic outcomes fare in terms of predictive parity by gender. This essay reports original research using a large dataset of offenders who were scored on the popular risk assessment tool COMPAS. Findings indicate that COMPAS performs reasonably well at discriminating between recidivists and non‐recidivists for men and women. Nonetheless, COMPAS algorithmic outcomes systemically overclassify women in higher risk groupings. Multiple measures of algorithmic equity and predictive accuracy are provided to support the conclusion that this algorithm is sexist.

Overview

AI snapshot

Written from the paper’s full text.

TL;DR

This study audits the COMPAS risk assessment tool using Broward County data and finds that it discriminates recidivists from non-recidivists similarly for men and women but systematically overclassifies women into higher risk categories. Using multiple algorithmic fairness metrics, including Northpointe’s preferred predictive parity measure, the paper concludes that COMPAS exhibits gender bias and is sexist.

Key findings

  • COMPAS showed comparable discrimination by gender: AUC was .711 for males and .697 for females, a difference that was not statistically significant, and higher risk bins were associated with higher recidivism for both genders.
  • Yet COMPAS overclassified women: in each risk bin, females recidivated at lower rates than males by 9–12% (p < .01); for violent recidivism, high-risk males violently reoffended at more than twice the rate of high-risk females.
  • Nested logistic regressions found gender significant in Model 3 (Female OR = .623; Model 1 OR = .588; Model 4 OR = .624), while the Female*decile interaction was not significant (.999), indicating bias in the intercept—men had higher recidivism even controlling for COMPAS decile—but not in the slope.
  • Equal calibration showed female observed recidivism below expected rates at every decile, with gaps ranging from 4% to 12% and averaging 9%; male–female differences appeared at every decile, ranging from 4% (Decile 1) to 18% (Decile 2), averaging 11%. PPVs were significantly lower for females at both cutpoints (.52 vs. .65; .65 vs. .75), so predictive parity was not met, while FPRs were .30/.30 at the lower cutpoint and .09 male vs. .07 female at the higher cutpoint, not statistically significant.

Methods

  • Original statistical audit of COMPAS using the same Broward County, Florida dataset as ProPublica and Northpointe: 6,172 individuals arrested and scored on the COMPAS general recidivism risk scale in 2013–2014, with a two-year recidivism follow-up.
  • The study followed ProPublica’s methodology for defining general/violent recidivism, excluding missing data, and excluding cases not scored on COMPAS in a timely manner.
  • Analyses included logistic regression nested models with Female, COMPAS decile score, and a Female*decile interaction predicting general recidivism; risk-bin recidivism rates by gender; equal calibration graphs; false positive rates and positive predictive values at two cutpoints; and area under the curve.

Limitations

  • Stated in the paper: the single site limits generalization; archival data may contain unobservable systemic data collection errors; official recidivism records exclude undetected crimes; the dataset lacked interrater reliability scores for COMPAS scoring; and secondary data analysis did not permit controlling for supervision or pretrial conditions.
  • Evident from the text: COMPAS’s algorithm is proprietary and protected by trade secret law, so the audit relies on outcomes rather than the algorithm itself; Broward County elected not to use COMPAS’s gender-specific norms, so findings may not generalize to jurisdictions that do use gendered scoring.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1002/bsl.2406

Cites

20 of 20 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

0 of 26 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    At the lower cutpoint (low vs medium/high), FPRs are the same for males and females.

    “at the lower cutpoint (low versus medium/high) the FPRs are the same”
    Quote in the paperNot a substantial claim
  • Findingown data

    Observed recidivism rates for females were significantly lower than expected rates at every score, ranging from 4% (Decile 3) to 12% (Decile 8), average 9%.

    “The observed recidivism rates for females were significantly lower than expected rates at every score, ranging from 4% (Decile 3) to 12% (Decile 8), with an average of 9%.”
    Quote in the paperSubstantial claim
  • Findingown data

    COMPAS algorithmic outcomes systemically overclassify women in higher risk groupings.

    “Nonetheless, COMPAS algorithmic outcomes systemically overclassify women in higher risk groupings.”
    • Also stated in: § 4.2 | The validity of risk bins · boosted
      “Figure 1, thereby, visually demonstrates that COMPAS overclassifies women as compared with men.”
    • Also stated in: § Abstract · plain
      “Additional calculations regarding algorithmic fairness likewise show that COMPAS overclassifies females.”
    Quote in the paperSubstantial claim
  • Findingown data

    Medium and high risk classifications do not have the same predictive utility across genders.

    “Medium and high risk classifications simply do not have the same predictive utility across genders.”
    Quote in the paperSubstantial claim
  • Findingown data

    At the higher cutpoint, males have a higher FPR than females (9% vs 7%).

    “at the higher cutpoint males have a higher FPR than females (9% versus 7%, respectively).”
    Quote in the paperNot a substantial claim
  • It may state two separate claims as one. 17
  • Probably not a substantial claim. 7

  • It may state two separate claims as one.

    Findingown data

    Males recidivated at higher rates than females at every decile score, with variance ranging from 4% (Decile 1) to 18% (Decile 2), average 11%.

    “males recidivated at higher rates than females at every decile score, ranging from a variance of 4% (Decile 1) to 18% (Decile 2), with an average of 11%.”
    Quote in the paperSubstantial claim

  • It may state two separate claims as one.

    Findingcitation

    In a more robust model controlling for decile, race, age, criminal history, and charge degree, the gender variable retained statistical significance and the same direction, as a negative predictor of general recidivism for women.

    “In a more robust model (not presented herein), the gender variable retained statistical significance and in the same direction (i.e., a negative predictor of general recidivism for women) when also controlling for decile, race, age, criminal history, and charge degree.73”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [73]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • It may state two separate claims as one.

    Findingreasoning

    Gender and decile score each have independent and significant effects on general recidivism.

    “In other words, these findings indicate that gender and decile score each have independent and significant effects on general recidivism.”
    Quote in the paperSubstantial claim

  • It may state two separate claims as one.

    Findingown data

    In each COMPAS risk bin, females recidivated at lower rates than males, between 9 and 12% lower, and the differences were statistically significant at p < .01.

    “In each risk bin, females recidivated at lower rates than males (between 9 and 12% lower), and the differences were statistically significant (p < .01).”
    Quote in the paperSubstantial claim

  • It may state two separate claims as one.

    Findingown data

    In Figure 1, the use of three risk bins showed good discrimination, with rates of reoffending significantly increasing from low to medium to high.

    “In Figure 1 the use of the three risk bins showed good discrimination in that the rates of reoffending in each bin significantly increase from low to medium to high.”
    Quote in the paperSubstantial claim

  • It may state two separate claims as one.

    Findingown data

    Calibration errors against women exist at every decile and in the same direction.

    “Overall, calibration errors against women exist at every decile and in the same direction.”
    Quote in the paperSubstantial claim

  • It may state two separate claims as one.

    Findingreasoning

    The nested models demonstrate that bias is in the intercept, with men having a higher recidivism rate, but not in the slope.

    “In sum, the set of nested models demonstrates that the bias is in the intercept (men have a higher recidivism rate) but not in the slope of the relationship (because a higher decile score is associated with a similar increase in recidivism rates for both genders).”
    Quote in the paperSubstantial claim

  • It may state two separate claims as one.

    Findingown data

    Gender and decile score are significantly predictive of general recidivism both individually and when entered together.

    “indicate that gender and decile score are significantly predictive of general recidivism both individually and when entered together.”
    Quote in the paperSubstantial claim

  • It may state two separate claims as one.

    Findingreasoning

    The relationship between gender and general recidivism is not moderated by decile score, and the relationship between decile score and general recidivism is not moderated by gender.

    “This latter finding indicates that the form of the relationship between (a) gender and general recidivism is not moderated by the decile score, and (b) decile score and general recidivism is not moderated by gender.”
    Quote in the paperSubstantial claim

  • It may state two separate claims as one.

    Findingreasoning

    Gender is statistically significant in Model 3, meaning the regression lines for males and females do not share the same intercepts.

    “Gender is statistically significant in Model 3, which means that the regression lines for males and females do not share the same intercepts.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Study design and conductreasoning

    The study relied upon archival data, and it is possible that there were systemic errors in data collection that are not observable.

    “This study relied upon archival data, and it is thereby possible for there to have been systemic errors in data collection that are not observable.”
    Quote in the paperNot a substantial claim

  • It may state two separate claims as one.

    Interpretationcitation

    Unless a risk tool accounts for base rate differences, overclassification of females is a foreseeable result, as is differential validity.

    “Unless a risk tool accounts for these base rate differences, overclassification of females is a foreseeable result, as is differential validity-meaning that the tool will not predict at equal levels of accuracy in both gender groups.29”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [29]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • It may state two separate claims as one.

    Interpretationreasoning

    The PPV measures indicate bias against females while the FPRs do not.

    “the PPV measures indicate bias against females while the FPRs do not.”
    Quote in the paperSubstantial claim

  • It may state two separate claims as one.

    Interpretationreasoning

    The risk tool overclassifies women and thus more of them are likely to be unfairly treated in criminal justice decisions and be subject to unnecessary levels of supervision.

    “The unfortunate consequence is that the risk tool overclassifies women and thus more of them are likely to be unfairly treated in criminal justice decisions and be subject to unnecessary levels of supervision.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Study design and conductreasoning

    It would have been preferable to control for aspects of supervision, as pretrial services/conditions may moderate reoffending rates, but secondary data analysis did not permit it.

    “it would have been preferable to control for aspects of supervision, as pretrial services/conditions may moderate reoffending rates, but secondary data analysis did not permit it.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Study design and conductnone given

    The single site limits generalization of results.

    “The single site limits generalization of results.”
    Quote in the paperNot a substantial claim

  • It may state two separate claims as one.

    Premisecitation

    The AUC has serious limitations and cannot present a holistic portrait of a tool's abilities.

    “the AUC has serious limitations, and thus cannot present a holistic portrait of a tool's abilities.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [86]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Interpretationcitation

    The inclusion of gender in a risk tool would positively serve legal and empirical interests if it favorably improves predictive accuracy.

    “Indeed, the inclusion of gender in a tool would appear to positively serve legal and empirical interests if it favorably improves predictive accuracy, thus permitting the tool to better inform management decisions.40”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [40]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim.

    Premisecitation

    Many gender-neutral factors result in higher correlations to recidivism for men.

    “Researchers have objected that these “gender-neutral” factors may not be as evenhanded as might first appear to be the case, in that many of these factors result in higher correlations to recidivism for men.32”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [32]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Premisecitation

    The Wisconsin Supreme Court noted that not including gender would lead to misclassifying both sexes.

    “The Wisconsin Supreme Court addressed the use of the COMPAS gendered scales in sentencing decisions and, in denying a due process challenge, noted that not including gender would lead to misclassifying both sexes.47”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [47]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • It may state two separate claims as one.

    Interpretationreasoning

    Bias in the intercept shows that the issue is overclassification for women and confirms test bias and disparate impact.

    “Bias in the intercept here shows that the issue is overclassification for women and confirms test bias and disparate impact.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Study design and conductreasoning

    Recidivism outcomes were from official records and thus will not include undetected crimes.

    “Recidivism outcomes were from official records and thus will not include undetected crimes.”
    Quote in the paperNot a substantial claim
22 of 22

  • The paper relies on this without support.

    The Broward County dataset is representative of COMPAS-scored populations generally.

    “This Article addresses this question with new statistical auditing of the COMPAS tool using the same Broward County dataset underlying the ProPublica and Northpointe studies.”
    “The primary dataset includes 6,172 individuals arrested in Broward County, Florida, and scored on the COMPAS general recidivism risk scale soon after their arrests in 2013 and 2014.55”

  • The paper relies on this without support.

    The agency studied declined to incorporate gender-specific scoring.

    “When agencies, such as the one studied here, decline to incorporate gendered scoring”
2 of 2

Citations

Cites

Flagged first, then by citations

Research works printed but not linked: 20

  1. Li Zhou, Is Your Software Racist?, POLITICO (Feb. 7, 2018, 5:05 AM), https://www.politico.com/agenda/story/2018/02/07/ algorithmic-bias-software-recommendations-000631.
  2. Id. 15 Melissa Hamilton, Risk-Needs Assessment: Constitutional and Ethical Challenges, 52 AM. CRIM. L. REV. 231, 232 (2015).
  3. Jessica M. Eaglin, Constructing Recidivism Risk, 67 EMORY L.J. 59, 67 (2017).
  4. Kristin Bechtel et al., A Meta-Analytic Review of Pretrial Research: Risk Assessment, Bond Type, and Interventions, 42 AM. J. CRIM. JUST. 443, 444 (2017).
  5. John Logan Koepke & David G. Robinson, Danger Ahead: Risk Assessment and the Future of Bail Reform, 93 Wash. L. Rev. 1725, 1725 (2018).
  6. United States v. Salerno, 481 U.S. 739, 751 (1987).
  7. Flavio Chierichetti et al., Fair Clustering Through Fairlets, 31st Conference on Neural Information Processing Systems 1 (2017), http://www.chierichetti.name/papers/fairlets.pdf.
  8. Julia A. Yesberg et al., Predicting Women's Recidivism: Validating a Dynamic Community-Based "Gender-Neutral" Tool, 62 PROB. J. 33, 36 (2015).
  9. Jennifer Skeem et al., Gender, Risk, Assessment, and Sanctioning: The Cost of Treating Women Like Men, 40 LAW & HUM. BEHAV. 580, 582 (2016) (discussing both sides).
  10. Frank Pasquale, Toward a Fourth Law of Robotics, 78 OHIO ST. L. J. 1243, 1247 (2017).
  11. Anupam Chander, The Racist Algorithm?, 115 MICH. L. REV. 1023, 1036 (2017).
  12. Richard G. Kopf, Federal Supervised Release and Actuarial Data (Including Age, Race, and Gender), 27 FED. SENT'G REP. 207, 211 (2015).
  13. Christopher Slobogin, Principles of Risk Assessment, 15 OHIO ST. J. CRIM. L. 583, 592 (2018).
  14. Jennifer L. Skeem & Christopher T. Lowenkamp, Race, Risk, and Recidivism: Predictive Bias and Disparate Impact, 54 CRIM- INOLOGY 680, 692 (2016).
  15. Adam W. Meade & Michael Fetzer, Test Bias, Differential Prediction, and a Revised Approach for Determining the Suitability of a Predictor in a Selection Context, 12 ORG. RES. METHODS 738, 740 (2009).
  16. Richard Berk, Accuracy and Fairness for Juvenile Justice Risks Assessments 16 J. Empirical Leg. Stud. 175, 191 (2019).
  17. Jay P. Singh et al., Measurement of Predictive Validity in Violence Risk Assessment Studies, 31 BEHAV. SCI. & L. 55, 64 (2013).
  18. Martin Rettenberger et al., Prospective Actuarial Risk Assessment: A Comparison of Five Risk Assessment Instruments in Differ- ent Sexual Offender Subtypes, 54 INT'L J. OFFENDER THERAPY & COMP. CRIMINOLOGY 169, 176 (2010).
  19. Stephane M. Shepherd & Danny Sullivan, Covert and Implicit Influences on the Interpretation of Violence Risk Instruments, 24 PSYCHIATRY, PSYCHOL. & L. 292, 294 (2017). 87
  20. Singh, supra note 81, at 16. 88 Id. at 17. How to cite this article: Hamilton M. The sexist algorithm. Behav Sci Law. 2019;37:145-157. https://doi.org/ 10.1002/bsl.2406

Cited by

Sorted by citations

Related

Metadata

Authors
Hamilton, M.
Affiliations
University of Surrey (GB)
Corresponding author
Hamilton, Melissa
ORCID
Hamilton, Melissa
Venue
Behavioral Sciences & the Law
Publisher
Wiley
Published
2019-03
Volume
37
Issue
2
Pages
145-157
ISSN
  • 0735-3936 (ISSN-L)
  • 1099-0798
DOI
10.1002/bsl.2406
OpenAlex
W2925706453
Semantic Scholar
adbad500235739a65d564631c20c6c8c9b7d5c14
Cited by
67
References
20
Citations by year
2026: 12 · 2025: 11 · 2024: 12 · 2023: 9 · 2022: 8 · 2021: 10 · 2020: 4 · 2019: 1
FWCI
2.31
Type
journal article
Language
en
Open access
Licence
onlinelibrary.wiley.com/termsAndConditions#vor (vor, 30 d)
Locations
  • Behavioral Sciences & the Law (publishedVersion)
  • PubMed (publishedVersion)
  • Surrey Research Insight Open Access (The University of Surrey) (submittedVersion)
  • SSRN Electronic Journal (submittedVersion)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Social Sciences › Sociology and Political Science › Criminal Justice and Corrections Analysis
  • Social Sciences › Law › Artificial Intelligence in Law
Keywords
COMPAS, algorithmic bias, gender discrimination, recidivism prediction, risk assessment tools, algorithmic fairness, women offenders, criminal justice, evidence-based practice
MeSH terms
Recidivism, Adult, Algorithms, Female, Humans, Male, Sex Factors, Risk Assessment, Criminals

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-11-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-11-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review