The DOI matches its Crossref record; 46 of the 46 references it cites were found in a registry or an index.
Bias Mitigation for Machine Learning Classifiers: A Comprehensive Survey
Open accessAbstract
This article provides a comprehensive survey of bias mitigation methods for achieving fairness in Machine Learning (ML) models. We collect a total of 341 publications concerning bias mitigation for ML classifiers. These methods can be distinguished based on their intervention procedure (i.e., pre-processing, in-processing, post-processing) and the technique they apply. We investigate how existing bias mitigation methods are evaluated in the literature. In particular, we consider datasets, metrics, and benchmarking. Based on the gathered insights (e.g., What is the most popular fairness metric? How many datasets are used for evaluating bias mitigation methods?), we hope to support practitioners in making informed choices when developing and evaluating new bias mitigation methods.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
SMOTE synthetic minority over-sampling technique
Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002) · Journal of Artificial Intelligence Research · doi:10.1613/jair.953 · Cited by 32,865
Dissecting racial bias in an algorithm used to manage the health of populations
Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019) · Science · doi:10.1126/science.aax2342 · Cited by 7,137 · cited 1×
“While fairness is a multi-disciplinary field of research, and has been addressed by various communities, including law [45], health studies [272], and criminal justice [32], we focus on the algorithmic fairness and bias as exhibited by ML models.”
BackgroundFinite-time Analysis of the Multiarmed Bandit Problem
Auer, P., Cesa-Bianchi, N., & Fischer, P. (2002) · Machine Learning · doi:10.1023/a:1013689704352 · Cited by 6,033
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012) · Proceedings of the 3rd Innovations in Theoretical Computer Science Conference · doi:10.1145/2090236.2090255 · Cited by 3,501
Barocas, S., & Selbst, A. D. (2014) · doi:10.2139/ssrn.2477899 · Cited by 3,048
Bellamy, R. K. E., Dey, K., Hind, M., Hoffman, S. C., Houde, S., Kannan, K., Lohia, P., Martino, J., Mehta, S., Mojsilovic, A., et al. (2018) · arXiv · doi:10.48550/arxiv.1810.01943 · Cited by 1,581
Mitigating Unwanted Biases with Adversarial Learning
Zhang, B. H., Lemoine, B., & Mitchell, M. (2018) · Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3278721.3278779 · Cited by 1,387
Data preprocessing techniques for classification without discrimination
Kamiran, F., & Calders, T. (2012) · Knowledge and Information Systems · doi:10.1007/s10115-011-0463-8 · Cited by 1,305 · cited 9×
“…ICCCC Calders et al. [37] 2009 ICDMW Loung et al. [38] 2011 KDD Žliobaite et al. [39] 2011 ICDM Hajian et al. [40] 2012 IEEE Trans Knowl Data Eng Kamiran and Calders [41] 2012 KAIS Zhang et al. [42] 2018 IJCAI Iosifidis et al. [43] 2019 DEXA Sun et al. [44] 2022 EuroS&P Seker et al. [45] 2022 Stud.”
BackgroundMethodInfluentialAmazon Scraps Secret AI Recruiting Tool that Showed Bias against Women *
Dastin, J. (2022) · Ethics of Data and Analytics · doi:10.1201/9781003278290-44 · Cited by 1,165
Language (Technology) is Power: A Critical Survey of “Bias” in NLP
Blodgett, S. L., Barocas, S., Daumé III, H., & Wallach, H. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.485 · Cited by 1,050 · cited 2×
“Other surveys are concerned with fairness and consider the following perspectives: learningbased sequential decision algorithms [407], criminal justice [38], graph representations [406], ML testing [398], Software Engineering [68, 336], or Natural Language Processing [44, 338].”
BackgroundFairness in Criminal Justice Risk Assessments: The State of the Art
Berk, R., Heidari, H., Jabbari, S., Kearns, M., & Roth, A. (2021) · Sociological Methods & Research · doi:10.1177/0049124118782533 · Cited by 988 · cited 2×
“ML is used in a variety of critical applications such as justice risk assessments [24, 38], job recommendations [413], and autonomous driving [227].”
BackgroundMachine Learning Testing: Survey, Landscapes and Horizons
Zhang, J. M., Harman, M., Ma, L., & Liu, Y. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2019.2962027 · Cited by 897
Self-citationAI Fairness 360: An extensible toolkit for detecting and mitigating algorithmic bias
Bellamy, R. K. E., Dey, K., Hind, M., Hoffman, S. C., Houde, S., Kannan, K., Lohia, P., Martino, J., Mehta, S., Mojsilović, A., et al. (2019) · IBM Journal of Research and Development · doi:10.1147/jrd.2019.2942287 · Cited by 884
CPAR: Classification based on Predictive Association Rules
Yin, X., & Han, J. (2003) · Proceedings of the 2003 SIAM International Conference on Data Mining · doi:10.1137/1.9781611972733.40 · Cited by 833
Fairness-Aware Classifier with Prejudice Remover Regularizer
Kamishima, T., Akaho, S., Asoh, H., & Sakuma, J. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33486-3_3 · Cited by 694
Discrimination-aware data mining
Pedreshi, D., Ruggieri, S., & Turini, F. (2008) · Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1401890.1401959 · Cited by 676 · cited 3×
“ML algorithms, which are never designed to intentionally incorporate bias, run the risk of replicating or even amplifying bias present in realworld data [6], [7].”
BackgroundInfluentialBickel, P. J., Hammel, E. A., & O'Connell, J. W. (1975) · Science · doi:10.1126/science.187.4175.398 · Cited by 659
Fairness in Machine Learning: A Survey
Caton, S., & Haas, C. (2020) · arXiv · doi:10.48550/arxiv.2010.04053 · Cited by 646
Fairness in machine learning: A survey
Caton, S., & Haas, C. (2024) · ACM Computing Surveys · doi:10.1145/3616865 · Cited by 646 · cited 5×
“For this, we follow taxonomies devised by Dunkelau and Leuschel [18], as well as Caton and Haas [24].”
BackgroundMethodInfluentialZafar, M. B., Valera, I., Gomez Rodriguez, M., & Gummadi, K. P. (2017) · Proceedings of the 26th International Conference on World Wide Web · doi:10.1145/3038912.3052660 · Cited by 632 · cited 2×
“Notably, the five most-frequently used methods include each of the three types: sampling and relabelling for pre-processing [185], constraints [384, 387] and adversarial learning [393] for in-processing, and classifier modification for post-processing [144].”
MethodA comparative study of fairness-enhancing interventions in machine learning
Friedler, S. A., Scheidegger, C., Venkatasubramanian, S., Choudhary, S., Hamilton, E. P., & Roth, D. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287589 · Cited by 601
A Review on Fairness in Machine Learning
Pessach, D., & Shmueli, E. (2023) · ACM Computing Surveys · doi:10.1145/3494672 · Cited by 598
A Survey of App Store Analysis for Software Engineering
Martin, W., Sarro, F., Jia, Y., Zhang, Y., & Harman, M. (2017) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2016.2630689 · Cited by 459 · cited 1×
“To ensure that irrelevant publications are excluded from the search results, we manually check publications in three stages [251]:”
BackgroundSelf-citation50 Years of Test (Un)fairness: Lessons for Machine Learning
Hutchinson, B., & Mitchell, M. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287600 · Cited by 297
A multidisciplinary survey on discrimination analysis
Romei, A., & Ruggieri, S. (2014) · The Knowledge Engineering Review · doi:10.1017/s0269888913000039 · Cited by 288
Auditing black-box models for indirect influence
Adler, P., Falk, C., Friedler, S. A., Nix, T., Rybeck, G., Scheidegger, C., Smith, B., & Venkatasubramanian, S. (2018) · Knowledge and Information Systems · doi:10.1007/s10115-017-1116-3 · Cited by 280
k-NN as an implementation of situation testing for discrimination discovery and prevention
Luong, B. T., Ruggieri, S., & Turini, F. (2011) · Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/2020408.2020488 · Cited by 263
Classification with Fairness Constraints: A Meta-Algorithm with Provable Guarantees
Celis, L. E., Huang, L., Keswani, V., & Vishnoi, N. K. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287586 · Cited by 231
Fairness in Learning: Classic and Contextual Bandits
Joseph, M., Kearns, M., Morgenstern, J., & Roth, A. (2016) · arXiv · doi:10.48550/arxiv.1605.07139 · Cited by 199 · cited 2×
“Moreover, they [299] 2016 NeurIPS Johnson et al. [300] 2016 Stat Sci Kusner et al. [301] 2017 NeurIPS Joseph et al. [302] 2018 AIES Hashimoto et al. [303] 2018 ICML Hébert-Johnson et al. [304] 2018 ICML Chiappa and Isaac [305] 2018 IFIP Alabi et al. [306] 2018 COLT Madras et al. [307] 2018 NeurIPS…”
BackgroundMethodA Convex Framework for Fair Regression
Berk, R., Heidari, H., Jabbari, S., Joseph, M., Kearns, M., Morgenstern, J., Neel, S., & Roth, A. (2017) · arXiv · doi:10.48550/arxiv.1706.02409 · Cited by 198
Why is my classifier discriminatory?
Chen, I., Johansson, F. D., & Sontag, D. (2018) · arXiv · doi:10.48550/arxiv.1805.12002 · Cited by 185 · cited 1×
“[66] 2018 NeurIPS Iosifidis and Ntoutsi [160] 2018 report Xu et al.”
BackgroundJohndrow, J. E., & Lum, K. (2019) · The Annals of Applied Statistics · doi:10.1214/18-aoas1201 · Cited by 141 · cited 5×
“[355] 2018 NeurIPS Johndrow and Lum [173] 2019 Ann Appl Stat Wang et al.”
BackgroundInfluentialThe Landscape and Gaps in Open Source Fairness Toolkits
Lee, M. S. A., & Singh, J. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445261 · Cited by 129
Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements
Beutel, A., Chen, J., Doshi, T., Qian, H., Woodruff, A., Luu, C., Kreitmann, P., Bischof, J., & Chi, E. H. (2019) · Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3306618.3314234 · Cited by 111
In-Processing Modeling Techniques for Machine Learning Fairness: A Survey
Wan, M., Zha, D., Liu, N., & Zou, N. (2023) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3551390 · Cited by 99 · cited 1×
“[352], which provides information on 38 in-processing approaches developed for various ML tasks.”
BackgroundAdaFair: Cumulative Fairness Adaptive Boosting
Iosifidis, V., & Ntoutsi, E. (2019) · Proceedings of the 28th ACM International Conference on Information and Knowledge Management · doi:10.1145/3357384.3357974 · Cited by 75
Perrone, V., Donini, M., Zafar, M. B., Schmucker, R., Kenthapadi, K., & Archambeau, C. (2021) · Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3461702.3462629 · Cited by 66
Taking Advantage of Multitask Learning for Fair Classification
Oneto, L., Doninini, M., Elders, A., & Pontil, M. (2019) · Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3306618.3314255 · Cited by 64
Fair Regression with Wasserstein Barycenters
Chzhen, E., Denis, C., Hebiri, M., Oneto, L., & Pontil, M. (2020) · arXiv · doi:10.48550/arxiv.2006.07286 · Cited by 46 · cited 5×
“…2019 JSAI Kim et al. [180] 2020 ICML Jiang et al. [181] 2020 UAI Savani et al. [355] 2020 NeurIPS Chzhen et al. [356] 2020 NeurIPS Chzhen et al. [266] 2020 NeurIPS Awasthi et al. [357] 2020 PMLR Chzhen and Schreuder [271] 2020 arxiv Schreuder and Chzhen [358] 2021 UAI Kanamori and Arimura [359]…”
BackgroundMethodInfluentialFAIR: Fair adversarial instance re-weighting
Petrović, A., Nikolić, M., Radovanović, S., Delibašić, B., & Jovanović, M. (2022) · Neurocomputing · doi:10.1016/j.neucom.2021.12.082 · Cited by 40
Data Pre-Processing for Discrimination Prevention: Information-Theoretic Optimization and Analysis
Calmon, F. D. P., Wei, D., Vinzamuri, B., Ramamurthy, K. N., & Varshney, K. R. (2018) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2018.2865887 · Cited by 33
Promoting Fairness through Hyperparameter Optimization
F.Cruz, A., Saleiro, P., Belem, C., Soares, C., & Bizarro, P. (2021) · 2021 IEEE International Conference on Data Mining (ICDM) · doi:10.1109/icdm51629.2021.00119 · Cited by 28
Meritocratic Fairness for Infinite and Contextual Bandits
Joseph, M., Kearns, M., Morgenstern, J., Neel, S., & Roth, A. (2018) · Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3278721.3278764 · Cited by 24
Using Pareto simulated annealing to address algorithmic bias in machine learning
Blanzeisky, W., & Cunningham, P. (2022) · The Knowledge Engineering Review · doi:10.1017/s0269888922000029 · Cited by 8 · cited 3×
“[312] 2022 DS Liu and Vicente [233] 2022 CMS Blanzeisky and Cunningham [43] 2022 Knowl Eng Rev Boulitsakis-Logothetis [46] 2022 arXiv Suriyakumar et al.”
BackgroundSłowik, A., & Bottou, L. (2021) · arXiv · doi:10.48550/arxiv.2106.09467 · Cited by 7 · cited 2×
“Zhao et al. [278] 2021 KDD Zhang et al. [279] 2021 MOD Li et al. [280] 2021 LAK Du and Wu [82] 2021 CIKM Perrone et al. [281] 2021 AIES Słowik and Bottou [282] 2021 arXiv Mishler and Kennedy [193] 2021 arXiv Lawless et al. [283] 2021 arXiv Choi et al. [284] 2021 AAAI Park et al. [285] 2022 WWW Wang et al. [198] 2022 C…”
Background
Cited by
Bias in medical AI: Implications for clinical decision-making
Cross, J. L., Choma, M. A., & Onofrey, J. A. (2024) · PLOS Digital Health · doi:10.1371/journal.pdig.0000651 · Cited by 607
Survey on Machine Learning Biases and Mitigation Techniques
Siddique, S., Haque, M. A., George, R., Gupta, K. D., Gupta, D., & Faruk, M. J. H. (2023) · Digital · doi:10.3390/digital4010001 · Cited by 78
Kotsiopoulos, T., Papakostas, G., Vafeiadis, T., Dimitriadis, V., Nizamis, A., Bolzoni, A., Bellinati, D., Ioannidis, D., Votis, K., Tzovaras, D., et al. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.124839 · Cited by 19
Low, D. M., Rao, V., Randolph, G., Song, P. C., & Ghosh, S. S. (2024) · PLOS Digital Health · doi:10.1371/journal.pdig.0000516 · Cited by 16
Metadata
- Authors
- Hort, M., Chen, Z., Zhang, J. M., Harman, M., & Sarro, F.
- Venue
- ACM Journal on Responsible Computing
- Published
- 2024-06-20
- Volume
- 1
- Issue
- 2
- Pages
- 1-52
- ISSN
- 2832-0565 (ISSN-L)
- OpenAlex
- W4388229648
- Cited by
- 219
- References
- 46
- FWCI
- 52.98
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.