AI Fairness 360: An extensible toolkit for detecting and mitigating algorithmic bias
The DOI matches its Crossref record; 28 of the 31 references it cites were found in a registry or an index.
Abstract
Fairness is an increasingly important concern as machine learning models are used to support decision making in high-stakes applications such as mortgage lending, hiring, and prison sentencing. This article introduces a new open-source Python toolkit for algorithmic fairness, AI Fairness 360 (AIF360), released under an Apache v2.0 license (https://github.com/ibm/aif360). The main objectives of this toolkit are to help facilitate the transition of fairness research algorithms for use in an industrial setting and to provide a common framework for fairness researchers to share and evaluate algorithms. The package includes a comprehensive set of fairness metrics for datasets and models, explanations for these metrics, and algorithms to mitigate bias in datasets and models. It also includes an interactive Web experience that provides a gentle introduction to the concepts and capabilities for line-of-business users, researchers, and developers to extend the toolkit with their new algorithms and improvements and to use it for performance benchmarking. A built-in testing infrastructure maintains code quality.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“the Linux Foundation, which hosts several of the open source tools we found (IBM's AI Fairness 360 [15], for example).”
From Towards AI Accountability Infrastructure: Gaps and Opportunities in AI Audit Tooling (2025) · cites this paper as [15]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“An open-source toolkit, the AI Fairness 360, is available to help AI researchers examine, report, and mitigate discrimination and bias in machine learning models throughout the AI development lifecycle [52].”
From Bias in artificial intelligence algorithms and recommendations for mitigation (2023) · cites this paper as [52]
The cited source says what the claim attributes to it.
In the citing paper
“It includes a comprehensive set of metrics for datasets and models to test for biases, explanations for those metrics, and algorithms to mitigate bias in datasets and models.”
From Bias in artificial intelligence algorithms and recommendations for mitigation (2023) · cites this paper as [52]
The cited source says what the claim attributes to it.
In the citing paper
“If bias is discovered in an algorithm, a variety of solutions are now available to help mitigate and reduce the level of bias (Bellamy et al., 2019).”
From Addressing Fairness, Bias, and Appropriate Use of Artificial Intelligence and Machine Learning in Global Health (2021) · cites this paper as Bellamy et al., 2019
The cited source says what the claim attributes to it.
In the citing paper
“Various toolboxes have been released in the past years to assist the algorithm experimentation and software development to enhance AI trustworthiness in different aspects such as fairness [12, 13, 71], XAI [56, 1, 8], and adversarial robustness [55, 2].”
From Trustworthy AI: From Principles to Practices (2023) · cites this paper as [12]
4 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
UCI Machine Learning Repository
Asuncion, A. (2007) · Medical Entomology and Zoology · Cited by 25,410
Equality of Opportunity in Supervised Learning
Hardt, M., Price, E., & Srebro, N. (2016) · arXiv · doi:10.48550/arxiv.1610.02413 · Cited by 2,536
Certifying and Removing Disparate Impact
Feldman, M., Friedler, S. A., Moeller, J., Scheidegger, C., & Venkatasubramanian, S. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2783311 · Cited by 1,827
Mitigating Unwanted Biases with Adversarial Learning
Zhang, B. H., Lemoine, B., & Mitchell, M. (2018) · Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3278721.3278779 · Cited by 1,387
Data preprocessing techniques for classification without discrimination
Kamiran, F., & Calders, T. (2012) · Knowledge and Information Systems · doi:10.1007/s10115-011-0463-8 · Cited by 1,305
(2013) · Cited by 1,126
Counterfactual explanations without opening the black box: automated decisions and the GDPR
Wachter, S., Mittelstadt, B., & Russell, C. (2017) · SSRN Electronic Journal · doi:10.2139/ssrn.3063289 · Cited by 1,061
Kusner, M. J., Loftus, J. R., Russell, C., & Silva, R. (2017) · arXiv · doi:10.48550/arxiv.1703.06856 · Cited by 995
Optimized Pre-Processing for Discrimination Prevention
Calmon, F. P., Wei, D., Vinzamuri, B., Ramamurthy, K. N., & Varshney, K. R. (2017) · neural information processing systems · Cited by 978
Self-citationInherent trade-offs in the fair determination of risk scores
Kleinberg, J., Mullainathan, S., & Raghavan, M. (2017) · Schloss Dagstuhl – Leibniz-Zentrum für Informatik · doi:10.4230/lipics.itcs.2017.43 · Cited by 943
Research works printed but not linked: 4
- Kohavi, R. Scaling up the accuracy of naive-Bayes classifiers: a decision-tree hybrid. In Proceedings of the Second International Conference on Knowledge Discovery and Data Mining , 1996. URL https://archive. ics.uci.edu/ml/datasets/adult .
- Kusner, M. J., Loftus, J. R., Russell, C., and Silva, R. Counterfactual fairness. In NIPS , 2017.
- bellamy, 2018, AI Fairness 360: An extensible toolkit for detecting, understanding, and mitigating unwanted algorithmic bias, arXiv 1810 01943
- Narayanan, A. Translation tutorial: 21 fairness definitions and their politics. In Conference on Fairness, Accountability, and Transparency , February 2018.
Cited by
Items 101–200 of 1,009
Mitigating Bias in Federated Learning
Abay, A., Zhou, Y., Baracaldo, N., Rajamoni, S., Chuba, E., & Ludwig, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.02447 · Cited by 40
Fairness in AI-Driven Recruitment: Challenges, Metrics, Methods, and Future DirectionsPreprint
Mujtaba, D. F., & Mahapatra, N. R. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2405.19699 · Cited by 157
Comparison of Methods to Reduce Bias From Clinical Prediction Models of Postpartum Depression
Park, Y., Hu, J., Singh, M., Sylla, I., Dankwa-Mullan, I., Koski, E., & Das, A. K. (2021) · JAMA Network Open · doi:10.1001/jamanetworkopen.2021.3909 · Cited by 116
Self-citationArtificial intelligence for diagnostic and prognostic neuroimaging in dementia: a systematic review
Borchert, R. J., Azevedo, T., Badhwar, A., Bernal, J., Betts, M., Bruffaerts, R., Burkhart, M. C., Dewachter, I., Gellersen, H. M., Low, A., et al. (2023) · Alzheimer's & Dementia · doi:10.1002/alz.13412 · Cited by 125
“…(He et al. 2020), 178 which removes correlations between the data and specified protected features, as well as IBM’s AI Fairness 360 toolkit (Bellamy et al. 2019), 179 which provides an accessible set of fairness metrics for a model and accompanying explanations to help mitigate bias.”
Does Gender Matter? Towards Fairness in Dialogue Systems
Liu, H., Dacon, J., Fan, W., Liu, H., Liu, Z., & Tang, J. (2020) · Proceedings of the 28th International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.390 · Cited by 78
Bias in Machine Learning: A Literature Review
Mavrogiorgos, K., Kiourtis, A., Mavrogiorgou, A., Menychtas, A., & Kyriazis, D. (2024) · Applied Sciences · doi:10.3390/app14198860 · Cited by 144
Saghiri, A. M., Vahidipour, S. M., Jabbarpour, M. R., Sookhak, M., & Forestiero, A. (2022) · Applied Sciences · doi:10.3390/app12084054 · Cited by 85
Wang, A., Ramaswamy, V. V., & Russakovsky, O. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533101 · Cited by 87
Advancing algorithmic bias management capabilities in AI-driven marketing analytics research
Akter, S., Sultana, S., Mariani, M. M., Wamba, S. F., Spanaki, K., & Dwivedi, Y. K. (2023) · Industrial Marketing Management · doi:10.1016/j.indmarman.2023.08.013 · Cited by 137
In AI We Trust? Factors That Influence Trustworthiness of AI-infused Decision-Making Processes
Ashoori, M., & Weisz, J. D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.02675 · Cited by 53
Advancing AI in healthcare: A comprehensive review of best practices
Polevikov, S. (2023) · Clinica Chimica Acta · doi:10.1016/j.cca.2023.117519 · Cited by 140
González-Sendino, R., Serrano, E., & Bajo, J. (2024) · Future Generation Computer Systems · doi:10.1016/j.future.2024.02.023 · Cited by 99
The life cycle of large language models in education: A framework for understanding sources of bias
Lee, J., Hicke, Y., Yu, R., Brooks, C., & Kizilcec, R. F. (2024) · British Journal of Educational Technology · doi:10.1111/bjet.13505 · Cited by 119
Fairness and Bias in Algorithmic Hiring: A Multidisciplinary Survey
Fabris, A., Baranowska, N., Dennis, M. J., Graus, D., Hacker, P., Saldivar, J., Zuiderveen Borgesius, F., & Biega, A. J. (2025) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3696457 · Cited by 117
Zhang, D., Kou, Z., & Wang, D. (2020) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata50022.2020.9378043 · Cited by 154
Fairness in Ranking, Part I: Score-Based Ranking
Zehlike, M., Yang, K., & Stoyanovich, J. (2022) · ACM Computing Surveys · doi:10.1145/3533379 · Cited by 180
Biswas, S., & Rajan, H. (2020) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3368089.3409704 · Cited by 146
FairBatch: Batch Selection for Model FairnessPreprint
Roh, Y., Lee, K., Whang, S. E., & Suh, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.01696 · Cited by 135
“We use the same pre-processing as in IBM’s AI Fairness 360 (Bellamy et al., 2019) and use GENDER as the sensitive attribute.”
MethodData Augmentation for Discrimination Prevention and Bias Disambiguation
Sharma, S., Zhang, Y., Ríos Aliaga, J. M., Bouneffouf, D., Muthusamy, V., & Varshney, K. R. (2020) · Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3375627.3375865 · Cited by 87
“All three datasets have been studied for demonstrating bias with respect to at least one protected attribute, specifically in [4] and the associated open-source notebooks, and we consider the same versions of the datasets and logistic regression models as is available in their open-source…”
MethodSelf-citationCharting the Sociotechnical Gap in Explainable AI: A Framework to Address the Gap in XAI
Ehsan, U., Saha, K., De Choudhury, M., & Riedl, M. O. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579467 · Cited by 85
Fairness in Ranking: A SurveyPreprint
Zehlike, M., Yang, K., & Stoyanovich, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.14000 · Cited by 123
Fair Class Balancing: Enhancing Model Fairness without Observing Sensitive Attributes
Yan, S., Kao, H.-T., & Ferrara, E. (2020) · Proceedings of the 29th ACM International Conference on Information & Knowledge Management · doi:10.1145/3340531.3411980 · Cited by 93
Explaining the Principles to Practices Gap in AI
Schiff, D., Rakova, B., Ayesh, A., Fanti, A., & Lennon, M. (2021) · IEEE Technology and Society Magazine · doi:10.1109/mts.2021.3056286 · Cited by 91
How do we assess the trustworthiness of AI? Introducing the trustworthiness assessment model (TrAM)
Schlicker, N., Baum, K., Uhde, A., Sterz, S., Hirsch, M. C., & Langer, M. (2025) · Computers in Human Behavior · doi:10.1016/j.chb.2025.108671 · Cited by 76
Algorithmic Political Bias in Artificial Intelligence Systems
Peters, U. (2022) · Philosophy & Technology · doi:10.1007/s13347-022-00512-8 · Cited by 111
Andrus, M., Spitzer, E., Brown, J., & Xiang, A. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445888 · Cited by 111
Privacy and Security of Big Data in AI Systems: A Research and Standards Perspective
Dilmaghani, S., Brust, M. R., Danoy, G., Cassagnes, N., Pecero, J., & Bouvry, P. (2019) · 2019 IEEE International Conference on Big Data (Big Data) · doi:10.1109/bigdata47090.2019.9006283 · Cited by 111
Koshiyama, A., Kazim, E., Treleaven, P., Rai, P., Szpruch, L., Pavey, G., Ahamat, G., Leutner, F., Goebel, R., Knight, A., et al. (2024) · Royal Society Open Science · doi:10.1098/rsos.230859 · Cited by 110
The Causal Fairness Field Guide: Perspectives From Social and Formal Sciences
Carey, A. N., & Wu, X. (2022) · AI and Ethics · doi:10.3389/fdata.2022.892837 · Cited by 135
Designing Responsible AI: Adaptations of UX Practice to Meet Responsible AI Challenges
Wang, Q., Madaio, M., Kane, S., Kapania, S., Terry, M., & Wilcox, L. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581278 · Cited by 90
“…and development process [53, 67, 69, 70, 84]—in contrast with prior work that has studied data annotation [9, 28, 31, 71] and model evaluation [32, 56, 88], or developed resources for data and model documentation [36, 47, 83] or post-hoc interpretation of model performance [3, 6, 7, 10, 68, 107].”
BackgroundResultdalex: Responsible Machine Learning with Interactive Explainability and Fairness in PythonPreprint
Baniecki, H., Kretowicz, W., Piątyszek, P., Wisniewski, J., & Biecek, P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.14406 · Cited by 147
On the Privacy Risks of Algorithmic Fairness
Chang, H., & Shokri, R. (2021) · IEEE European Symposium on Security and Privacy · doi:10.1109/eurosp51992.2021.00028 · Cited by 174
Fairway: A Way to Build Fair ML Software
Chakraborty, J., Majumder, S., Yu, Z., & Menzies, T. (2020) · Figshare · doi:10.6084/m9.figshare.12363347 · Cited by 103
Schlicker, N., Langer, M., Ötting, S. K., Baum, K., König, C. J., & Wallach, D. (2021) · Computers in Human Behavior · doi:10.1016/j.chb.2021.106837 · Cited by 106
Algorithmic Fairness in Mortgage Lending: from Absolute Conditions to Relational Trade-offs
Lee, M. S. A., & Floridi, L. (2021) · Minds and Machines · doi:10.1007/s11023-020-09529-4 · Cited by 106
Fairness in Ranking, Part II: Learning-to-Rank and Recommender Systems
Zehlike, M., Yang, K., & Stoyanovich, J. (2022) · ACM Computing Surveys · doi:10.1145/3533380 · Cited by 154
Fairness in Large Language Models: A Taxonomic Survey
Chu, Z., Wang, Z., & Zhang, W. (2024) · ACM SIGKDD Explorations Newsletter · doi:10.1145/3682112.3682117 · Cited by 59
Deng, W. H., Guo, B., Devrio, A., Shen, H., Eslami, M., & Holstein, K. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581026 · Cited by 103
Fairness issues, current approaches, and challenges in machine learning models
Jui, T. D., & Rivas, P. (2024) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-023-02083-2 · Cited by 131
The Integration of AI and Metaverse in Education: A Systematic Literature Review
Almeman, K., Ayeb, F. E., Berrima, M., Issaoui, B., & Morsy, H. (2025) · Applied Sciences · doi:10.3390/app15020863 · Cited by 119
“Moreover, implementing discrimination-aware unit tests at each stage of the algorithmic process and using open-source toolkits like Aequitas [125] and AI Fairness 360 [126] is recommended to monitor and mitigate potential biases.”
Gender bias in AI-based decision-making systems: a systematic literature review
Nadeem, A., Marjanovic, O., & Abedin, B. (2022) · AJIS. Australasian journal of information systems/AJIS. Australian journal of information systems/Australian journal of information systems · doi:10.3127/ajis.v26i0.3835 · Cited by 120
Bias and Fairness in Chatbots: An Overview
Xue, J., Wang, Y.-C., Wei, C., Liu, X., Woo, J., & Kuo, C. ‐. J. (2024) · APSIPA Transactions on Signal and Information Processing · doi:10.1561/116.00000064 · Cited by 108
Agentic AI: A Comprehensive Survey of Technologies, Applications, and Societal Implications
Pati, A. K. (2025) · IEEE Access · doi:10.1109/access.2025.3585609 · Cited by 100
Exploring How Machine Learning Practitioners (Try To) Use Fairness Toolkits
Deng, W. H., Nagireddy, M., Lee, M. S. A., Singh, J., Wu, Z. S., Holstein, K., & Zhu, H. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533113 · Cited by 99
A survey on ethical principles of AI and implementations
Zhou, J., Chen, F., Berry, A., Reed, M., Zhang, S., & Savage, S. (2020) · 2020 IEEE Symposium Series on Computational Intelligence (SSCI) · doi:10.1109/ssci47803.2020.9308437 · Cited by 99
The Problem of Fairness in Synthetic Healthcare Data
Bhanot, K., Qi, M., Erickson, J. S., Guyon, I., & Bennett, K. P. (2021) · Entropy · doi:10.3390/e23091165 · Cited by 91
Evaluating and mitigating bias in machine learning models for cardiovascular disease prediction
Li, F., Wu, P., Ong, H. H., Peterson, J. F., Wei, W.-Q., & Zhao, J. (2023) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2023.104294 · Cited by 81
Balayn, A., Lofi, C., & Houben, G.-J. (2021) · The VLDB Journal · doi:10.1007/s00778-021-00671-8 · Cited by 98
REVISE: A Tool for Measuring and Mitigating Bias in Visual Datasets
Wang, A., Liu, A., Zhang, R., Kleiman, A., Kim, L., Zhao, D., Shirai, I., Narayanan, A., & Russakovsky, O. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-022-01625-5 · Cited by 77
Understanding and Evaluating Racial Biases in Image Captioning
Zhao, D., Wang, A., & Russakovsky, O. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01456 · Cited by 83
“Researchers have advocated for increased transparency of datasets [27, 35], including developing tools to steer researcher intervention [8, 70].”
BackgroundGoing Beyond XAI: A Systematic Survey for Explanation-Guided Learning
Gao, Y., Gu, S., Jiang, J., Hong, S.-S., Yu, D., & Zhao, L. (2024) · ACM Computing Surveys · doi:10.1145/3644073 · Cited by 121
“The most noteworthy finding in our survey for the landmark surveys is that the approaches for detecting bias in ML are more than the ways to mitigate the bias [19, 41].”
Background"Ignorance and Prejudice" in Software Fairness
Zhang, J. M., & Harman, M. (2021) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse43902.2021.00129 · Cited by 129
“Due to the instability of machine learning fairness metric values [41], we repeat each prediction process 50 times.”
MethodRashidi, H. H., Pantanowitz, J., Hanna, M. G., Tafti, A. P., Sanghani, P., Buchinsky, A., Fennell, B. D., Deebajah, M., Wheeler, S., Pearce, T. M., et al. (2025) · Modern Pathology · doi:10.1016/j.modpat.2024.100688 · Cited by 112
Fairea: a model behaviour mutation approach to benchmarking bias mitigation methods
Hort, M., Zhang, J. M., Sarro, F., & Harman, M. (2021) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3468264.3468565 · Cited by 120
Petersen, E., Potdevin, Y., Mohammadi, E., Zidowitz, S., Breyer, S., Nowotka, D., Henn, S., Pechmann, L., Leucker, M., Rostalski, P., et al. (2022) · IEEE Access · doi:10.1109/access.2022.3178382 · Cited by 112
The Fairness of Credit Scoring Models
Hurlin, C., Pérignon, C., & Saurin, S. (2021) · Management Science · doi:10.1287/mnsc.2022.03888 · Cited by 120
Training individually fair ML models with Sensitive Subspace RobustnessPreprint
Yurochkin, M., Bower, A., & Sun, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.00020 · Cited by 109
Homeyer, A., Geißler, C., Schwen, L. O., Zakrzewski, F., Evans, T., Strohmenger, K., Westphal, M., Bülow, R. D., Kargl, M., Karjauv, A., et al. (2022) · Modern Pathology · doi:10.1038/s41379-022-01147-y · Cited by 95
Palladino, N. (2022) · Telecommunications Policy · doi:10.1016/j.telpol.2022.102479 · Cited by 106
A brief review on algorithmic fairness
Wang, X., Zhang, Y., & Zhu, R. (2022) · Management System Engineering · doi:10.1007/s44176-022-00006-z · Cited by 118
Self-citationChen, Z., Wu, M., Chan, A., Li, X., & Ong, Y.-S. (2023) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2023.3245733 · Cited by 102
Busch, F., Adams, L. C., & Bressem, K. K. (2023) · Medical Science Educator · doi:10.1007/s40670-023-01815-x · Cited by 88
“However, this is often not applicable, for example, when applications are not open source or with missing knowledge about the technology used [43].”
BackgroundA Survey of Ambient Intelligence
Dunne, R., Morris, T., & Harper, S. (2021) · ACM Computing Surveys · doi:10.1145/3447242 · Cited by 102
“…many areas of computer science to find solutions to biased AI, such as ‘AI Fairness 360: An Extensible Toolkit for Detecting and Mitigating Algorithmic Bias’ from Bellamy et al. [12], that enables developers to compare various existing bias detection and mitigation algorithms in a common framework.”
BackgroundDeho, O. B., Zhan, C., Li, J., Liu, J., Liu, L., & Le, T. D. (2022) · British Journal of Educational Technology · doi:10.1111/bjet.13217 · Cited by 104
D-BIAS: A Causality-Based Human-in-the-Loop System for Tackling Algorithmic Bias
Ghai, B., & Mueller, K. (2022) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2022.3209484 · Cited by 52
Chen, Z., Zhang, J. M., Sarro, F., & Harman, M. (2022) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3540250.3549093 · Cited by 109
“They have been integrated into the IBMAIF360 toolkit [22] andwidely adopted in previous SE studies [25, 29, 47].”
MethodA Survey of Data Quality Requirements That Matter in ML Development Pipelines
Priestley, M., O’donnell, F., & Simperl, E. (2023) · Journal of Data and Information Quality · doi:10.1145/3592616 · Cited by 86
FairLens: Auditing black-box clinical decision support systems
Panigutti, C., Perotti, A., Panisson, A., Bajardi, P., & Pedreschi, D. (2021) · Information Processing & Management · doi:10.1016/j.ipm.2021.102657 · Cited by 86
A survey on artificial intelligence in histopathology image analysis
Abdelsamea, M. M., Zidan, U., Senousy, Z., Gaber, M. M., Rakha, E. A., & Ilyas, M. (2022) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.1474 · Cited by 101
Juhn, Y. J., Ryu, E., Wi, C.-I., King, K. S., Malik, M., Romero-Brufau, S., Weng, C., Sohn, S., Sharp, R. R., & Halamka, J. D. (2022) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocac052 · Cited by 83
Theorising Algorithmic Justice
Marjanovic, O., Cecez-Kecmanovic, D., & Vidgen, R. (2022) · European Journal of Information Systems · doi:10.1080/0960085x.2021.1934130 · Cited by 66
Deepfakes, Phrenology, Surveillance, and More! A Taxonomy of AI Privacy Risks
Lee, H.-P. (., Yang, Y.-J., Von Davier, T. S., Forlizzi, J., & Das, S. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642116 · Cited by 82
FlipTest: fairness testing via optimal transport
Black, E., Yeom, S., & Fredrikson, M. (2020) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3351095.3372845 · Cited by 131
NeuronFair: interpretable white-box fairness testing through biased neuron identification
Zheng, H., Chen, Z., Du, T., Zhang, X., Cheng, Y., Ji, S., Wang, J., Yu, Y., & Chen, J. (2022) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3510003.3510123 · Cited by 95
Fairness in Deep Learning: A Survey on Vision and Language Research
Parraga, O., Móre, M. D., Mattjie, C., Gavenski, N. S., Kupssinskü, L. S., Medronha, A., de Moura, L. V., Simões, G. S., & Barros, R. C. (2025) · ACM Computing Surveys · doi:10.1145/3637549 · Cited by 123
“…and present the fairness risks in neural models; (ii) how to identify who should be responsible for owning and dealing with fairness issues in neural models; (iii) how to deal with the fairness-accu-racy trade-off; and (iv) current challenges and research directions for fairer Foundation Models.”
Towards AI Accountability Infrastructure: Gaps and Opportunities in AI Audit Tooling
Ojewale, V., Steed, R., Vecchione, B., Birhane, A., & Raji, I. D. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713301 · Cited by 81
Systematic literature review on bias mitigation in generative AI
Afreen, J., Mohaghegh, M., & Doborjeh, M. (2025) · AI and Ethics · doi:10.1007/s43681-025-00721-9 · Cited by 92
Transport-based Counterfactual ModelsPreprint
de Lara, L., González-Sanz, A., Asher, N. M., Risser, L., & Loubes, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.13025 · Cited by 85
How Do We Assess the Trustworthiness of AI? Introducing the Trustworthiness Assessment Model (TrAM)
Schlicker, N., Baum, K., Uhde, A., Sterz, S., Hirsch, M. C., & Langer, M. (2022) · doi:10.31234/osf.io/qhwvx · Cited by 79
Shen, H., Deng, W. H., Chattopadhyay, A., Wu, Z. S., Wang, X., & Zhu, H. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445971 · Cited by 71
Fair-AdaBoost: Extending AdaBoost method to achieve fair classification
Huang, X., Li, Z., Jin, Y., & Zhang, W. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.117240 · Cited by 86
Situated algorithms: a sociotechnical systemic approach to bias
Draude, C., Klumbytė, G., Lücking, P., & Treusch, P. (2019) · Online Information Review · doi:10.1108/oir-10-2018-0332 · Cited by 91
Barbado, A., Corcho, Ó., & Benjamins, R. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.116100 · Cited by 77
Data Readiness for AI: A 360-Degree Survey
Hiniduma, K., Byna, S., & Bez, J. L. (2025) · ACM Computing Surveys · doi:10.1145/3722214 · Cited by 93
Ortega-Bolaños, R., Bernal-Salcedo, J., Germán Ortiz, M., Galeano Sarmiento, J., Ruz, G. A., & Tabares-Soto, R. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10740-3 · Cited by 76
Conceptualising fairness: three pillars for medical algorithms and health equity
Sikstrom, L., Maslej, M. M., Hui, K., Findlay, Z., Buchman, D. Z., & Hill, S. L. (2022) · BMJ Health & Care Informatics · doi:10.1136/bmjhci-2021-100459 · Cited by 76
Soliciting Stakeholders’ Fairness Notions in Child Maltreatment Predictive Systems
Cheng, H.-F., Stapleton, L., Wang, R., Bullock, P., Chouldechova, A., Wu, Z. S. S., & Zhu, H. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445308 · Cited by 54
Biswas, S., & Rajan, H. (2021) · Iowa State University Digital Repository (Iowa State University) · Cited by 75
A Review of Bias and Fairness in Artificial Intelligence.
González-Sendino, R., Serrano, E., Bajo, J., & Nováis, P. (2024) · International Journal of Interactive Multimedia and Artificial Intelligence · doi:10.9781/ijimai.2023.11.001 · Cited by 89
Aldhafeeri, F. M. (2025) · Diagnostics · doi:10.3390/diagnostics15182300 · Cited by 80
“One central ethical concern is the potential bias in data used to train AI algorithms, with repercussions for fairness and equity in diagnostic outcomes [46].”
Algorithmic pollution: Making the invisible visible
Marjanovic, O., Cecez-Kecmanovic, D., & Vidgen, R. (2021) · Journal of Information Technology · doi:10.1177/02683962211010356 · Cited by 51
Forgetting Practices in the Data Sciences
Muller, M., & Strohmayer, A. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517644 · Cited by 55
Tal, A. S., Kuflik, T., & Kliger, D. (2022) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2022.2095705 · Cited by 86
Information-Theoretic Testing and Debugging of Fairness Defects in Deep Neural Networks
Monjezi, V., Trivedi, A., Tan, G., & Tizpaz-Niari, S. (2023) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse48619.2023.00136 · Cited by 83
“Beyond the scope of this paper, a body of prior work [42], [43], [44], [23], [45], [46] considered testing for group fairness.”
BackgroundYücer, Ş., Akçay, S., Al-Moubayed, N., & Breckon, T. P. (2020) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00017 · Cited by 101
“IBM [39] provides an extensive toolkit to detect bias and determine the current model fairness level.”
BackgroundREVISE: A Tool for Measuring and Mitigating Bias in Visual Datasets
Wang, A., Narayanan, A., & Russakovsky, O. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58580-8_43 · Cited by 98
Leverage Generative AI for human resource management: integrated risk analysis approach
Jiang, Y., Cai, Z., & Wang, X. (2025) · The International Journal of Human Resource Management · doi:10.1080/09585192.2025.2544972 · Cited by 78
“IBM used AI Fairness 360, its own open-source toolkit, to help audit AI models for potential bias and recommend adjustments to mitigate discriminatory outcomes (Bellamy et al., 2019; Johnson et al., 2023).”
Automated Data Slicing for Model Validation: A Big Data - AI Integration Approach
Chung, Y., Kraska, T., Polyzotis, N., Tae, K. H., & Whang, S. E. (2019) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2019.2916074 · Cited by 93
Zhang, A., Walker, O., Nguyen, K., Dai, J., Chen, A., & Lee, M. K. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579601 · Cited by 72
“…and practitioners when creating fair AI systems in order to inform techniques that can help them [44, 69] such as AI fairness checklists [70], provenance for datasets [37, 46], visualizations of model behaviors [103], and toolkits for detecting and mitigating against ML unfairness [8, 10, 89].”
BackgroundBertrand, A., Viard, T., Belloum, R., Eagan, J. R., & Maxwell, W. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581314 · Cited by 71
“The Silva explanation interface [127], similarly to IBM’s AIF360 tool [14], allows the user to modify dataset attributes and sensitive inputs to see how it afects specifed fairness measures.”
Method
Metadata
- Authors
- Bellamy, R. K. E., Dey, K., Hind, M., Hoffman, S. C., Houde, S., Kannan, K., Lohia, P., Martino, J., Mehta, S., Mojsilović, A., Nagar, S., Ramamurthy, K. N., Richards, J., Saha, D., Sattigeri, P., Singh, M., Varshney, K. R., & Zhang, Y.
- ORCID
- Venue
- IBM Journal of Research and Development
- Publisher
- IBM
- Published
- 2019-07-01
- Volume
- 63
- Issue
- 4/5
- Pages
- 4:1-4:15
- ISSN
- 0018-8646 (ISSN-L)
- OpenAlex
- W2974817986
- Cited by
- 884
- References
- 31
- Citations by year
- 2026: 163 · 2025: 226 · 2024: 145 · 2023: 143 · 2022: 99 · 2021: 69 · 2020: 35 · 2019: 4
- FWCI
- 90.90
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IBM Journal of Research and Development (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Imbalanced Data Classification Techniques
- Keywords
- algorithmic fairness, algorithmic bias detection, bias mitigation, fairness metrics, open-source software toolkit, high-stakes decision making
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.