“Un”Fair Machine Learning Algorithms
Open accessThe DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
Abstract
Ensuring fairness in algorithmic decision making is a crucial policy issue. Current legislation ensures fairness by barring algorithm designers from using demographic information in their decision making. As a result, to be legally compliant, the algorithms need to ensure equal treatment. However, in many cases, ensuring equal treatment leads to disparate impact particularly when there are differences among groups based on demographic classes. In response, several “fair” machine learning (ML) algorithms that require impact parity (e.g., equal opportunity) at the cost of equal treatment have recently been proposed to adjust for the societal inequalities. Advocates of fair ML propose changing the law to allow the use of protected class-specific decision rules. We show that the proposed fair ML algorithms that require impact parity, while conceptually appealing, can make everyone worse off, including the very class they aim to protect. Compared with the current law, which requires treatment parity, the fair ML algorithms, which require impact parity, limit the benefits of a more accurate algorithm for a firm. As a result, profit maximizing firms could underinvest in learning, that is, improving the accuracy of their machine learning algorithms. We show that the investment in learning decreases when misclassification is costly, which is exactly the case when greater accuracy is otherwise desired. Our paper highlights the importance of considering strategic behavior of stake holders when developing and evaluating fair ML algorithms. Overall, our results indicate that fair ML algorithms that require impact parity, if turned into law, may not be able to deliver some of the anticipated benefits. This paper was accepted by Kartik Hosanagar, information systems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Barocas, S., & Selbst, A. D. (2014) · doi:10.2139/ssrn.2477899 · Cited by 3,048
Fair prediction with disparate impact: a study of bias in recidivism prediction instruments
Chouldechova, A. (2017) · Big Data · doi:10.1089/big.2016.0047 · Cited by 2,159
Algorithmic Decision Making and the Cost of Fairness
Corbett-Davies, S., Pierson, E., Feller, A., Goel, S., & Huq, A. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098095 · Cited by 1,149
Human Decisions and Machine Predictions
Kleinberg, J., Lakkaraju, H., Leskovec, J., Ludwig, J., & Mullainathan, S. (2018) · The Quarterly Journal of Economics · doi:10.1093/qje/qjx032 · Cited by 1,040
Inherent Trade-Offs in the Fair Determination of Risk Scores
Kleinberg, J., Mullainathan, S., & Raghavan, M. (2016) · arXiv · doi:10.48550/arxiv.1609.05807 · Cited by 941
Predictably Unequal? The Effects of Machine Learning on Credit Markets
Fuster, A., Goldsmith-Pinkham, P. S., Ramadorai, T., & Walther, A. (2017) · doi:10.2139/ssrn.3072038 · Cited by 769
Human Decisions and Machine Predictions
Kleinberg, J., Lakkaraju, H., Leskovec, J., Ludwig, J., & Mullainathan, S. (2017) · doi:10.3386/w23180 · Cited by 545
Building Classifiers with Independency Constraints
Calders, T., Kamiran, F., & Pechenizkiy, M. (2009) · 2009 IEEE International Conference on Data Mining Workshops · doi:10.1109/icdmw.2009.83 · Cited by 468
RISK, RACE, AND RECIDIVISM: PREDICTIVE BIAS AND DISPARATE IMPACT*
SKEEM, J. L., & LOWENKAMP, C. T. (2016) · Criminology · doi:10.1111/1745-9125.12123 · Cited by 317
Cited by
Intelligent digital twin (iDT) for supply chain stress-testing, resilience, and viability
Ivanov, D. (2023) · International Journal of Production Economics · doi:10.1016/j.ijpe.2023.108938 · Cited by 270
Digital governance: A conceptual framework and research agenda
Hanisch, M., Goldsby, C. M., Fabian, N. E., & Oehmichen, J. (2023) · Journal of Business Research · doi:10.1016/j.jbusres.2023.113777 · Cited by 267
Conceptualisation of a 7-element digital twin framework in supply chain and operations management
Ivanov, D. (2024) · International Journal of Production Research · doi:10.1080/00207543.2023.2217291 · Cited by 193
Welfare Implications of Fairness and Accountability for Insurance Pricing
Shimao, H., & Huang, F. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4225159 · Cited by 4
Are numerical or verbal explanations of AI the key to appropriate user reliance and error detection?
Papenkordt, J., Ngonga Ngomo, A.-C., & Thommes, K. (2026) · Behaviour & Information Technology · doi:10.1080/0144929x.2025.2568928 · Cited by 2
Metadata
- Authors
- Fu, R., Aseri, M., Singh, P. V., & Srinivasan, K.
- Venue
- Management Science
- Published
- 2022-06
- Volume
- 68
- Issue
- 6
- Pages
- 4173-4195
- ISSN
- 0025-1909 (ISSN-L)
- 1526-5501
- OpenAlex
- W3206540055
- Cited by
- 78
- References
- 9
- FWCI
- 5.46
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: “Un”Fair Machine Learning Algorithms. https://app.cochat.ai/papers/10.1287/mnsc.2021.4065
- The paper
- Fu, R., Aseri, M., Singh, P. V., & Srinivasan, K. (2022). “Un”Fair Machine Learning Algorithms. Management Science. https://doi.org/10.1287/mnsc.2021.4065
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.