50 Years of Test (Un)fairness: Lessons for Machine Learning
Open accessThe DOI matches its OpenAlex record; 5 of the 5 references it cites were found in a registry or an index.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Fair prediction with disparate impact: a study of bias in recidivism prediction instruments
Chouldechova, A. (2017) · Big Data · doi:10.1089/big.2016.0047 · Cited by 2,159 · cited 3×
“uality of opportunity [32]. That same year, Linn [43] introduced (but did not advocate for) equal Positive Predictive Value (PPV) as a fairness criterion, recognizable as modern day predictive parity [9].4 Under Cleary and Darlington’s conceptions, bias or (un)fairness is a property of the test itself. This is contrar…”
BackgroundResultFairness in Criminal Justice Risk Assessments: The State of the Art
Berk, R., Heidari, H., Jabbari, S., Kearns, M., & Roth, A. (2021) · Sociological Methods & Research · doi:10.1177/0049124118782533 · Cited by 988 · cited 1×
“• Peterson and Novick’s “conditional probability and its con-verse” sufficiency [4], equalized odds [32], or conditional procedure accuracy [5], sometimes expressed as the conditional independence A ⊥ D | Y .”
BackgroundPreventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairness
Kearns, M., Neel, S., Roth, A., & Wu, Z. S. (2017) · arXiv · doi:10.48550/arxiv.1711.05144 · Cited by 469 · cited 1×
“, skin tone), as recent work has begun to address [27, 39].”
BackgroundOn the (im)possibility of fairness
Friedler, S. A., Scheidegger, C., & Venkatasubramanian, S. (2016) · arXiv · doi:10.48550/arxiv.1609.07236 · Cited by 247 · cited 2×
“in 2016 [28], Thorndike also pointed out the tension between individual notions of fairness and group notions of fairness: “the two definitions of fairness—one based on predicted criterion score for individuals and the other on the distribution of criterion scores in the two groups— will always be in conflict.”
BackgroundThe problem of infra-marginality in outcome tests for discrimination
Simoiu, C., Corbett-Davies, S., & Goel, S. (2017) · The Annals of Applied Statistics · doi:10.1214/17-aoas1058 · Cited by 154 · cited 1×
“t we know of. There are conceptual similarities between Jones’ criteria and the constrained ranking problem described by [8], and also between Einhorn’s criterion and concerns about infra-marginality [60]. For a binary classifier, Thorndike’s 1971 group parity criterion is equivalent to requiring that the ratio of pos…”
Background
Cited by
A Survey on Bias and Fairness in Machine Learning
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2022) · ACM Computing Surveys · doi:10.1145/3457607 · Cited by 5,812
Bias in data‐driven artificial intelligence systems—An introductory survey
Ntoutsi, E., Fafalios, P., Gadiraju, U., Iosifidis, V., Nejdl, W., Vidal, M., Ruggieri, S., Turini, F., Papadopoulos, S., Krasanakis, E., et al. (2020) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1356 · Cited by 1,043
“…have appeared thus far in the computer science literature (Verma & Rubin, 2018; Zliobaite, 2017); and some of these definitions and others were proposed and investigated in work on formalizing fairness from other disciplines, such as education, over the past 50 years (Hutchinson & Mitchell, 2019).”
Baker, R. S., & Hawn, A. (2022) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-021-00285-9 · Cited by 993
Fairness in machine learning: A survey
Caton, S., & Haas, C. (2024) · ACM Computing Surveys · doi:10.1145/3616865 · Cited by 646
Ai for social good: unlocking the opportunity for positive impact
Tomašev, N., Cornebise, J., Hutter, F., Mohamed, S., Picciariello, A., Connelly, B., Belgrave, D. C. M., Ezer, D., Haert, F. C. V. D., Mugisha, F., et al. (2020) · Nature Communications · doi:10.1038/s41467-020-15871-z · Cited by 357
Metadata
- Authors
- Hutchinson, B., & Mitchell, M.
- Venue
- arXiv (Cornell University)
- Publisher
- Cornell University
- Published
- 2018-11-25
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1811.10104
- OpenAlex
- W2901107694
- Cited by
- 118
- References
- 5
- Type
- posted content
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: 50 Years of Test (Un)fairness: Lessons for Machine Learning. https://app.cochat.ai/papers/10.48550/arxiv.1811.10104
- The paper
- Hutchinson, B., & Mitchell, M. (2018). 50 Years of Test (Un)fairness: Lessons for Machine Learning. arXiv (Cornell University). https://doi.org/10.48550/arxiv.1811.10104
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.