AI Fairness 360: An Extensible Toolkit for Detecting, Understanding, and Mitigating Unwanted Algorithmic Bias
Open accessAbstract
Fairness is an increasingly important concern as machine learning models are used to support decision making in high-stakes applications such as mortgage lending, hiring, and prison sentencing. This paper introduces a new open source Python toolkit for algorithmic fairness, AI Fairness 360 (AIF360), released under an Apache v2.0 license {https://github.com/ibm/aif360). The main objectives of this toolkit are to help facilitate the transition of fairness research algorithms to use in an industrial setting and to provide a common framework for fairness researchers to share and evaluate algorithms. The package includes a comprehensive set of fairness metrics for datasets and models, explanations for these metrics, and algorithms to mitigate bias in datasets and models. It also includes an interactive Web experience (https://aif360.mybluemix.net) that provides a gentle introduction to the concepts and capabilities for line-of-business users, as well as extensive documentation, usage guidance, and industry-specific tutorials to enable data scientists and practitioners to incorporate the most appropriate tool for their problem into their work products. The architecture of the package has been engineered to conform to a standard paradigm used in data science, thereby further improving usability for practitioners. Such architectural design and abstractions enable researchers and developers to extend the toolkit with their new algorithms and improvements, and to use it for performance benchmarking. A built-in testing infrastructure maintains code quality.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
AI Fairness 360 provides bias metrics and mitigation methods across the machine-learning pipeline.
“Toolkits such as AI Fairness 360 operationalize these concepts by providing bias metrics and mitigation methods across the machine-learning pipeline”
From The Impact of Structured Validation and Audit Frameworks on the Fairness and Efficiency of AI-Driven Hiring Systems (2025) · cites this paper as Bellamy et al., 2018
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Inherent trade-offs in the fair determination of risk scores
Kleinberg, J., Mullainathan, S., & Raghavan, M. (2017) · Schloss Dagstuhl – Leibniz-Zentrum für Informatik · doi:10.4230/lipics.itcs.2017.43 · Cited by 943 · cited 2× · see citing works
“Furthermore, researchers have shown that it is impossible to satisfy all definitions of fairness at the same time (Kleinberg et al., 2017).”
BackgroundA large-scale analysis of racial disparities in police stops across the United States
Pierson, E., Simoiu, C., Overgoor, J., Corbett-Davies, S., Jenson, D., Shoemaker, A., Ramachandran, V., Barghouty, P., Phillips, C., Shroff, R., et al. (2020) · Nature Human Behaviour · doi:10.1038/s41562-020-0858-1 · Cited by 552 · cited 1× · see citing works
“Figure 3 illustrates feature bias localization on the Stanford Open Policing dataset (Pierson et al., 2017) for Connecticut, with county name as the feature and race as the protected attribute.”
BackgroundFairness testing: testing software for discrimination
Galhotra, S., Brun, Y., & Meliou, A. (2017) · Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering · doi:10.1145/3106237.3106277 · Cited by 405 · see citing works
On the (im)possibility of fairness
Friedler, S. A., Scheidegger, C., & Venkatasubramanian, S. (2016) · arXiv · doi:10.48550/arxiv.1609.07236 · Cited by 247 · see citing works
FairTest: Discovering Unwarranted Associations in Data-Driven Applications
Tramèr, F., Atlidakis, V., Geambasu, R., Hsu, D., Hubaux, J.-P., Humbert, M., Juels, A., & Lin, H. (2015) · arXiv · doi:10.48550/arxiv.1510.02377 · Cited by 171 · see citing works
Flagged first, then by citations
Research works printed but not linked: 21
- Adebayo, J. A. "FairML : Toolbox for diagnosing bias in predictive modeling". Master's thesis, Massachusetts In- stitute of Technology, 2016. URL https://github. com/adebayoj/fairml. AHRQ. Medical Expenditure Panel Survey data: 2015 Full Year Consolidated Data File, 2015. URL https://meps.ahrq.gov/mepsweb/data_ stats/download_data_files_detail.jsp? cboPufNumber=HC-181. AHRQ. Medical Expenditure Panel Survey data: 2016 Full Year Consolidated Data File, 2016. URL https://meps.ahrq.gov/mepsweb/data_ stats/download_data_files_detail.jsp? cboPufNumber=HC-192.
- Bantilan, N. Themis-ml: A fairness-aware machine learn- ing interface for end-to-end discrimination discovery and mitigation. Journal of Technology in Human Services, 36(1):15-30, 2018. URL https://github.com/ cosmicBboy/themis-ml.
- Calders, T. and Verwer, S. Three naive Bayes ap- proaches for discrimination-free classification. Data Mining journal; special issue with selected papers from ECML/PKDD, 2010.
- Calmon, F. P., Wei, D., Vinzamuri, B., Natesan Ra- mamurthy, K., and Varshney, K. R. Optimized pre- processing for discrimination prevention. In Advances in Neural Information Processing Systems 30, pp. 3992- 4001, Long Beach, USA, December 2017.
- 'Alessandro, B., O'Neil, C., and LaGatta, T. Con- scientious classification: A data scientist's guide to discrimination-aware classification. Big Data, 5(2):120- 134, June 2017.
- Feldman, M., Friedler, S. A., Moeller, J., Scheidegger, C., and Venkatasubramanian, S. Certifying and removing disparate impact. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 259-268, Sydney, Australia, August 2015.
- Friedler, S. A., Scheidegger, C., Venkatasubramanian, S., Choudhary, S., Hamilton, E. P., and Roth, D. A com- parative study of fairness-enhancing interventions in machine learning. http://arxiv.org/abs/1802.04422, Feb. 2018. URL https://github.com/ algofairness/fairness-comparison. · arXiv
- Hardt, M., Price, E., and Srebro, N. Equality of opportunity in supervised learning. In Advances in Neural Informa- tion Processing Systems 29, pp. 3315-3323, Barcelona, Spain, December 2016.
- Hu, L. and Chen, Y. Welfare and distributional impacts of fair classification. In Proc. Fairness Accountability Transparency Machine Learning Workshop, July 2018.
- Kamiran, F. and Calders, T. Data preprocessing techniques for classification without discrimination. Knowledge and Information Systems, 33(1):1-33, 2012. doi: https://doi. org/10.1007/s10115-011-0463-8. · DOI · Paper page
- Kamiran, F., Karim, A., and Zhang, X. Decision theory for discrimination-aware classification. In IEEE Interna- tional Conference on Data Mining, pp. 924-929, 2012. doi: https://doi.org/10.1109/ICDM.2012. · DOI
- Kamishima, T., Akaho, S., Asoh, H., and Sakuma, J. Fairness-aware classifier with prejudice remover regu- larizer. Machine Learning and Knowledge Discovery in Databases, pp. 35-50, 2012.
- Kohavi, R. Scaling up the accuracy of naive-Bayes classi- fiers: a decision-tree hybrid. In Proceedings of the Sec- ond International Conference on Knowledge Discovery and Data Mining, 1996. URL https://archive. ics.uci.edu/ml/datasets/adult.
- Kusner, M. J., Loftus, J. R., Russell, C., and Silva, R. Counterfactual fairness. In NIPS, 2017.
- Narayanan, A. Translation tutorial: 21 fairness definitions and their politics. In Conference on Fairness, Account- ability, and Transparency, February 2018.
- Pleiss, G., Raghavan, M., Wu, F., Kleinberg, J., and Wein- berger, K. Q. On fairness and calibration. In Advances in Neural Information Processing Systems, pp. 5680-5689, 2017.
- Speicher, T., Heidari, H., Grgic-Hlaca, N., Gummadi, K. P., Singla, A., Weller, A., and Zafar, M. B. A unified ap- proach to quantifying algorithmic unfairness: Measuring individual & group unfairness via inequality indices. In Proc. ACM SIGKDD Int. Conf. Knowl. Disc. Data Min., pp. 2239-2248, London, UK, August 2018.
- Ustun, B., Spangher, A., and Liu, Y. Actionable recourse in linear classification. arXiv:1809.06514, September 2018. · arXiv
- Wachter, S., Mittelstadt, B., and Russell, C. Counterfactual explanations without opening the black box: Automated decisions and the GDPR. Harvard Journal of Law & Technology, 31(2):841-887, Spring 2018.
- Zemel, R., Wu, Y. L., Swersky, K., Pitassi, T., and Dwork, C. Learning fair representations. In Proceedings of the International Conference on Machine Learning, pp. 325-333, Atlanta, USA, June 2013.
- Zhang, B. H., Lemoine, B., and Mitchell, M. Mitigat- ing unwanted biases with adversarial learning. In Proc. AAAI/ACM Conf. Artif. Intell., Ethics, Society, New Or- leans, USA, February 2018.
Cited by
Sorted by citations
A Survey on Bias and Fairness in Machine Learning
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2022) · ACM Computing Surveys · doi:10.1145/3457607 · Cited by 5,812 · see citing works
Lekadir, K., Frangi, A. F., Porras, A. R., Glocker, B., Cintas, C., Langlotz, C. P., Weicken, E., Asselbergs, F. W., Prior, F., Collins, G. S., et al. (2025) · BMJ · doi:10.1136/bmj-2024-081554 · Cited by 957 · see citing works
Towards a standard for identifying and managing bias in artificial intelligence
Schwartz, R., Vassilev, A., Greene, K., Perine, L., Burt, A., & Hall, P. (2022) · doi:10.6028/nist.sp.1270 · Cited by 548 · see citing works
Fairness in machine learning: A survey
Caton, S., & Haas, C. (2024) · ACM Computing Surveys · doi:10.1145/3616865 · Cited by 646 · see citing works
Schiff, D. (2022) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-021-00270-2 · Cited by 489 · see citing works
“…(Mitchell et al. 2019) can help to promote transparency and address explicability concerns, while suites of tools such as IBM’s AI Fairness 360 (Bellamy et al. 2018) or the Scoping, Mapping, Artifact Collection, Testing, and Reflection (SMACTR) audit framework proposed by Raji et al. (2020)…”
MethodFactSheets: Increasing trust in AI services through supplier's declarations of conformity
Arnold, M., Bellamy, R. K. E., Hind, M., Houde, S., Mehta, S., Mojsilović, A., Nair, R., Ramamurthy, K. N., Olteanu, A., Piorkowski, D., et al. (2019) · IBM Journal of Research and Development · doi:10.1147/jrd.2019.2942288 · Cited by 434 · see citing works
Self-citationMadaio, M. A., Stark, L., Wortman Vaughan, J., & Wallach, H. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376445 · Cited by 436 · see citing works
Belenguer, L. (2022) · AI and Ethics · doi:10.1007/s43681-022-00138-8 · Cited by 289 · see citing works
Putting AI ethics to work: are the tools fit for purpose?
Ayling, J., & Chapman, A. (2022) · AI and Ethics · doi:10.1007/s43681-021-00084-x · Cited by 253 · see citing works
A high-level overview of AI ethics
Kazim, E., & Koshiyama, A. S. (2021) · Patterns · doi:10.1016/j.patter.2021.100314 · Cited by 269 · see citing works
Akinrinola, O., Okoye, C. C., Ofodile, O. C., & Ugochukwu, C. E. (2024) · GSC Advanced Research and Reviews · doi:10.30574/gscarr.2024.18.3.0088 · Cited by 206 · see citing works
An empirical characterization of fair machine learning for clinical risk prediction
Pfohl, S. R., Foryciarz, A., & Shah, N. H. (2021) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2020.103621 · Cited by 164 · see citing works
A review of predictive policing from the perspective of fairness
Alikhademi, K., Drobina, E., Prioleau, D., Richardson, B., Purves, D., & Gilbert, J. E. (2022) · Artificial Intelligence and Law · doi:10.1007/s10506-021-09286-4 · Cited by 167 · see citing works
Fairness and Explanation in AI-Informed Decision Making
Angerschmid, A., Zhou, J., Theuermann, K., Chen, F., & Holzinger, A. (2022) · Machine Learning and Knowledge Extraction · doi:10.3390/make4020026 · Cited by 161 · see citing works
AI Fairness in Data Management and Analytics: A Review on Challenges, Methodologies and Applications
Chen, P., Wu, L., & Wang, L. (2023) · Applied Sciences · doi:10.3390/app131810258 · Cited by 155 · see citing works
Considerations for AI fairness for people with disabilities
Trewin, S., Basson, S., Muller, M., Branham, S., Treviranus, J., Gruen, D., Hebert, D., Lyckowski, N., & Manser, E. (2019) · AI Matters · doi:10.1145/3362077.3362086 · Cited by 143 · see citing works
Z-Inspection <sup>®</sup> : A Process to Assess Trustworthy AI
Zicari, R. V., Brodersen, J., Brusseau, J., Dudder, B., Eichhorn, T., Ivanov, T., Kararigas, G., Kringen, P., McCullough, M., Moslein, F., et al. (2021) · IEEE Transactions on Technology and Society · doi:10.1109/tts.2021.3066209 · Cited by 134 · see citing works
Fairness in Machine Learning: A Survey
Caton, S., & Haas, C. (2020) · arXiv · doi:10.48550/arxiv.2010.04053 · Cited by 646 · see citing works
Fairness and Bias in Algorithmic Hiring: A Multidisciplinary Survey
Fabris, A., Baranowska, N., Dennis, M. J., Graus, D., Hacker, P., Saldivar, J., Zuiderveen Borgesius, F., & Biega, A. J. (2025) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3696457 · Cited by 117 · see citing works
Schlicker, N., Langer, M., Ötting, S. K., Baum, K., König, C. J., & Wallach, D. (2021) · Computers in Human Behavior · doi:10.1016/j.chb.2021.106837 · Cited by 106 · see citing works
Algorithmic Fairness in Mortgage Lending: from Absolute Conditions to Relational Trade-offs
Lee, M. S. A., & Floridi, L. (2021) · Minds and Machines · doi:10.1007/s11023-020-09529-4 · Cited by 106 · see citing works
Privacy and Security of Big Data in AI Systems: A Research and Standards Perspective
Dilmaghani, S., Brust, M. R., Danoy, G., Cassagnes, N., Pecero, J., & Bouvry, P. (2019) · 2019 IEEE International Conference on Big Data (Big Data) · doi:10.1109/bigdata47090.2019.9006283 · Cited by 111 · see citing works
FairLens: Auditing black-box clinical decision support systems
Panigutti, C., Perotti, A., Panisson, A., Bajardi, P., & Pedreschi, D. (2021) · Information Processing & Management · doi:10.1016/j.ipm.2021.102657 · Cited by 86 · see citing works
Towards AI Accountability Infrastructure: Gaps and Opportunities in AI Audit Tooling
Ojewale, V., Steed, R., Vecchione, B., Birhane, A., & Raji, I. D. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713301 · Cited by 81 · see citing works
Barbado, A., Corcho, Ó., & Benjamins, R. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.116100 · Cited by 77 · see citing works
Wang, H. E., Landers, M., Adams, R., Subbaswamy, A., Kharrazi, H., Gaskin, D. J., & Saria, S. (2022) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocac065 · Cited by 63 · see citing works
Data Cleaning for Accurate, Fair, and Robust Models: A Big Data - AI Integration Approach
Tae, K. H., Roh, Y., Oh, Y. H., Kim, H., & Whang, S. E. (2019) · Proceedings of the 3rd International Workshop on Data Management for End-to-End Machine Learning · doi:10.1145/3329486.3329493 · Cited by 62 · see citing works
“More recently, there are mitigation techniques for fixing unfairness, which can be done before (pre-processing), during (in-processing), or after (post-processing) model training [2].”
BackgroundMethodInfluentialKoshiyama, A., Kazim, E., & Treleaven, P. (2022) · Computer · doi:10.1109/mc.2021.3067225 · Cited by 55 · see citing works
Lessons and tips for designing a machine learning study using EHR data
Arbet, J., Brokamp, C., Meinzen-Derr, J., Trinkley, K. E., & Spratt, H. M. (2021) · Journal of Clinical and Translational Science · doi:10.1017/cts.2020.513 · Cited by 46 · see citing works
Personalized chatbot trustworthiness ratings
Srivastava, B., Rossi, F., Usmani, S., & Bernagozzi, M. (2020) · IEEE Transactions on Technology and Society · doi:10.1109/tts.2020.3023919 · Cited by 40 · see citing works
Marrying Fairness and Explainability in Supervised Learning
Grabowicz, P. A., Perello, N., & Mishra, A. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533236 · Cited by 37 · see citing works
Achieving a data-driven risk assessment methodology for ethical AI
Felländer, A., Rebane, J., Larsson, S., Wiggberg, M., & Heintz, F. (2022) · Digital Society · doi:10.1007/s44206-022-00016-0 · Cited by 36 · see citing works
Understanding Relations Between Perception of Fairness and Trust in Algorithmic Decision Making
Zhou, J., Verma, S., Mittal, M., & Chen, F. (2021) · 2021 8th International Conference on Behavioral and Social Computing (BESC) · doi:10.1109/besc53957.2021.9635182 · Cited by 36 · see citing works
Mitigating Dataset Harms Requires Stewardship: Lessons from 1000 Papers
Peng, K., Mathur, A., & Narayanan, A. (2021) · arXiv · doi:10.48550/arxiv.2108.02922 · Cited by 31 · see citing works
Quantifying representativeness in randomized clinical trials using machine learning fairness metrics
Qi, M., Cahan, O., Foreman, M. A., Gruen, D. M., Das, A. K., & Bennett, K. P. (2021) · JAMIA Open · doi:10.1093/jamiaopen/ooab077 · Cited by 27 · see citing works
MIMIC-IF: Interpretability and Fairness Evaluation of Deep Learning Models on MIMIC-IV Dataset
Meng, C., Trinh, L., Xu, N., & Liu, Y. (2021) · doi:10.21203/rs.3.rs-402058/v1 · Cited by 21 · see citing works
How Do We Assess the Trustworthiness of AI? Introducing the Trustworthiness Assessment Model (TrAM)
Schlicker, N., Baum, K., Uhde, A., Sterz, S., Hirsch, M. C., & Langer, M. (2022) · doi:10.31234/osf.io/qhwvx · Cited by 79 · see citing works
Question-Driven Design Process for Explainable AI User Experiences
Liao, Q. V., Pribić, M., Han, J., Miller, S., & Sow, D. (2021) · arXiv · doi:10.48550/arxiv.2104.03483 · Cited by 18 · see citing works
AI Data Readiness Inspector (AIDRIN) for Quantitative Assessment of Data Readiness for AI
Hiniduma, K., Byna, S., Bez, J. L., & Madduri, R. (2024) · Proceedings of the 36th International Conference on Scientific and Statistical Database Management · doi:10.1145/3676288.3676296 · Cited by 15 · see citing works
‘Led by Intelligence': A Scoping Review on the Experimental Evaluation of Intelligence-Led Policing
Khalfa, R., & Hardyns, W. (2024) · Evaluation Review · doi:10.1177/0193841x231204588 · Cited by 12 · see citing works
Ten simple rules for building and maintaining a responsible data science workflow
Stoudt, S., Jernite, Y., Marshall, B., Marwick, B., Sharan, M., Whitaker, K., & Danchev, V. (2024) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1012232 · Cited by 11 · see citing works
Constantin, R., Duck, M., Alexandrov, A., Matosevic, P., Keidar, D., & El-Assady, M. (2022) · 2022 IEEE Workshop on TRust and EXpertise in Visual Analytics (TREX) · doi:10.1109/trex57753.2022.00005 · Cited by 3 · see citing works
MS Industrial and Organizational Psychology from Missouri State University Department of Psychology and SHRM- CP, U., & Ahmad, S. (2025) · International Journal of Research and Applied Innovations · doi:10.15662/ijrai.2025.0806023
Auditing a Dutch Public Sector Risk Profiling Algorithm Using an Unsupervised Bias Detection Tool
Holstege, F., Jorgensen, M., Padh, K., Parie, J., Proroković, K., Persson, J., & Snoek, L. (2025) · arXiv · doi:10.48550/arxiv.2502.01713
“Legislators across various jurisdictions are implementing regulations to impose audit frameworks for these systems.”
- Authors
- Bellamy, R. K. E., Dey, K., Hind, M., Hoffman, S. C., Houde, S., Kannan, K., Lohia, P., Martino, J., Mehta, S., Mojsilovic, A., Nagar, S., Ramamurthy, K. N., Richards, J., Saha, D., Sattigeri, P., Singh, M., Varshney, K. R., & Zhang, Y.
- ORCID
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2018
- Volume
- 63
- Issue
- 4/5
- Pages
- 4:1-4:15
- ISSN
- 0018-8646 (ISSN-L)
- arXiv
- 1810.01943
- OpenAlex
- W2974817986
- Cited by
- 1,581
- References
- 5
- Citations by year
- 2026: 247 · 2025: 326 · 2024: 228 · 2023: 237 · 2022: 205 · 2021: 182 · 2020: 117 · 2019: 37 · 2018: 2
- FWCI
- 222.21
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- public-domain
- arxiv.org/licenses/nonexclusive-distrib/1.0/
- Locations
- IBM Journal of Research and Development (publishedVersion)
- arXiv (Cornell University) (submittedVersion, public-domain)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Imbalanced Data Classification Techniques
- Keywords
- algorithmic fairness, algorithmic bias detection, bias mitigation, fairness metrics, open-source software toolkit, high-stakes decision making
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
