The Landscape and Gaps in Open Source Fairness Toolkits
Open accessThe DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.
Abstract
With the surge in literature focusing on the assessment and mitigation of unfair outcomes in algorithms, several open source ‘fairness toolkits’ recently emerged to make such methods widely accessible. However, little studied are the differences in approach and capabilities of existing fairness toolkits, and their fit-for-purpose in commercial contexts. Towards this, this paper identifies the gaps between the existing open source fairness toolkit capabilities and the industry practitioners’ needs. Specifically, we undertake a comparative assessment of the strengths and weaknesses of six prominent open source fairness toolkits, and investigate the current landscape and gaps in fairness toolkits through an exploratory focus group, a semi-structured interview, and an anonymous survey of data science/machine learning (ML) practitioners. We identify several gaps between the toolkits’ capabilities and practitioner needs, highlighting areas requiring attention and future directions towards tooling that better support ‘fairness in practice.’
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“As existing work has demonstrated, selecting an appropriate fairness metric and mitigation method depending on the context of the use case is challenging even for experienced AI practitioners [91].”
From Out of Context: Investigating the Bias and Fairness Concerns of “Artificial Intelligence as a Service” (2023) · cites this paper as [91]
The cited source says what the claim attributes to it.
In the citing paper
“While there are multiple studies on the use of tools for fairness evaluation [54, 70, 119] and explainability [1, 46, 72, 82, 85, 98], fewer studies examine how practitioners evaluate other criteria such as basic functionality [117], safety, privacy, or recourse, just as fewer tools exist for this purpose.”
From Towards AI Accountability Infrastructure: Gaps and Opportunities in AI Audit Tooling (2025) · cites this paper as [119]
In the citing paper
“It also serves to illustrate how complex this landscape is, and as others have noted [7,19,98,102,104,105], this provides a barrier to those developing or purchasing AI systems as to which tool is appropriate for their purposes.”
From Putting AI ethics to work: are the tools fit for purpose? (2022) · cites this paper as [102]
In the citing paper
“However, as existing literature makes clear, fairness is often contextual [57,90,91,131,136]: a seemingly”
From Out of Context: Investigating the Bias and Fairness Concerns of “Artificial Intelligence as a Service” (2023) · cites this paper as [91]
In the citing paper
“These issues have been shown in more general ML-contexts; for example, studies exploring user (ML practitioner) attitudes and expectations regarding fairness toolkits showed a misalignment of tools with user expectations and needs, and generally being difcult to use (among other concerns), which not only makes such tooling less-efective, but can potentially lead to increased bias risks [40,91].”
From Out of Context: Investigating the Bias and Fairness Concerns of “Artificial Intelligence as a Service” (2023) · cites this paper as [91]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The global landscape of AI ethics guidelines
Jobin, A., Ienca, M., & Vayena, E. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0088-2 · Cited by 6,220 · cited 1×
“A systematic investigation of 84 such documents found “justice, fairness, and equity” is the second most prevalent principle after “transparency” [20].”
BackgroundAn Empirical Evaluation of the System Usability Scale
Bangor, A., Kortum, P. T., & Miller, J. T. (2008) · International Journal of Human–Computer Interaction · doi:10.1080/10447310802205776 · Cited by 5,181 · cited 1×
“scoring in the high 70s to upper 80s” [3], scores we see are beyond that for the fairness toolkits.”
BackgroundDwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012) · Proceedings of the 3rd Innovations in Theoretical Computer Science Conference · doi:10.1145/2090236.2090255 · Cited by 3,501 · cited 1×
“Substantial academic literature on algorithmic fairness has concerned the development of mathematical and computational defnitions of fairness (e.g. [11, 14, 26]), prompting work to explain the distinctions between them [31, 39] and the trade-ofs [24] given some of these defnitions are impossible…”
BackgroundEquality of Opportunity in Supervised Learning
Hardt, M., Price, E., & Srebro, N. (2016) · arXiv · doi:10.48550/arxiv.1610.02413 · Cited by 2,536 · cited 4×
“…methods to remove estimated bias from the data set [6, 12, 21], in-processing methods to train a model to both maximise accuracy and increase fairness [22, 46], and post-processing methods to adapt the predictions after the model build to equalise a metric of fairness between groups [14, 21, 32].”
BackgroundMethodInfluentialCertifying and Removing Disparate Impact
Feldman, M., Friedler, S. A., Moeller, J., Scheidegger, C., & Venkatasubramanian, S. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2783311 · Cited by 1,827 · cited 3×
“…has been an explosion of recent work introducing diferent mathematical formalisations of fairness (e.g. [12, 14, 26]), various technical implementations to for “debiasing” [6, 12, 21, 22, 32, 46], and the distinctions and trade-ofs among the mathematically incompatible defnitions [24, 28, 39].”
BackgroundMethodBellamy, R. K. E., Dey, K., Hind, M., Hoffman, S. C., Houde, S., Kannan, K., Lohia, P., Martino, J., Mehta, S., Mojsilovic, A., et al. (2018) · arXiv · doi:10.48550/arxiv.1810.01943 · Cited by 1,581
Mitigating Unwanted Biases with Adversarial Learning
Zhang, B. H., Lemoine, B., & Mitchell, M. (2018) · Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3278721.3278779 · Cited by 1,387 · cited 2×
“…has been an explosion of recent work introducing diferent mathematical formalisations of fairness (e.g. [12, 14, 26]), various technical implementations to for “debiasing” [6, 12, 21, 22, 32, 46], and the distinctions and trade-ofs among the mathematically incompatible defnitions [24, 28, 39].”
BackgroundMethodhttp://ljournal.ru/wp-content/uploads/2016/08/d-2016-154.pdf
(2016) · ljournal eBooks · doi:10.18411/d-2016-154 · Cited by 1,178
Amazon Scraps Secret AI Recruiting Tool that Showed Bias against Women *
Dastin, J. (2022) · Ethics of Data and Analytics · doi:10.1201/9781003278290-44 · Cited by 1,165
Fairness definitions explained
Verma, S., & Rubin, J. (2018) · Proceedings of the International Workshop on Software Fairness · doi:10.1145/3194770.3194776 · Cited by 1,124
Research works printed but not linked: 6
- Kate Crawford. 2017. Artifcial intelligence with very real biases. The Wall Street Journal. Retrieved from https://www. wsj. com/articles/artifcial-intelligencewith- very-real-biases-1508252717 (2017).
- Avi Feller, Emma Pierson, Sam Corbett-Davies, and Sharad Goel. 2016. A com- puter program used for bail and sentencing decisions was labeled biased against blacks. It's actually not that clear. The Washington Post (2016).
- Rasmus Hauch. 2020. Denmark introduces mandatory legislation for AI and data ethics. 2021.ai (2020). https://2021.ai/denmark-introduces-mandatory- legislation-ai-data-ethics/
- Kacper Sokol, Raul Santos-Rodriguez, and Peter Flach. 2019. FAT Forensics: A Python toolbox for algorithmic fairness, accountability and transparency. arXiv preprint arXiv:1909.05167 (2019). · arXiv
- Neil Vigdor. 2019. Apple card investigated after gender discrimination complaints. The New York Times (2019).
- Stephen Walli, Dave Gynn, and Bruno von Rotz. 2005. The growth of open source software in organizations. Publication Report. Optaros Inc (2005).
Cited by
Items 101–190 of 190
Precel, H., McDonald, A., Hecht, B. J., & Vincent, N. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642749 · Cited by 10
“…work in HCI could explore how disparate impacts and IP dispossession interact with perceptions of accuracy, trustworthiness, and fairness of LLMs. Similarly, HCI has also contributed to toolkits for ML practitioners to support better decision-making around fairness, bias, and transparency [70].”
BackgroundHarvey, E., Sheng, E., Blodgett, S. L., Chouldechova, A., Garcia-Gathright, J. I., Olteanu, A., & Wallach, H. (2025) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2025.findings-acl.947 · Cited by 9
An Empirical Study of Modular Bias Mitigators and EnsemblesPreprint
Feffer, M., Hirzel, M. J., Hoffman, S. C., Kate, K., Ram, P., & Shinnar, A. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2202.00751 · Cited by 9
“Mitigation almost always improved disparate impact outcomes, but ensemble learning generally incurred a slight penalty relative to the no-ensemble base-line.”
BackgroundOxonFair: A Flexible Toolkit for Algorithmic FairnessPreprint
Delaney, E. D., Fu, Z., Wachter, S., Mittelstadt, B., & Russell, C. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4894794 · Cited by 9
Shrestha, H., Cachel, K., Alkhathlan, M., Rundensteiner, E. A., & Harrison, L. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3594108 · Cited by 8
“[32] evaluated the capabilities of open-source fairness toolkits and their suitability for commercial use through practitioner interviews and surveys.”
A Taxonomy of Machine Learning Fairness Tool Specifications, Features and Workflows
Mim, S. A., Smith, J., & Johnson, B. (2023) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vl-hcc57772.2023.00036 · Cited by 8
“While it may seem promising to have an abundance of available fairness tools, research has shown that practitioners do not always have the necessary expertise to make the best use of existing tools [2].”
BackgroundMethodInfluentialFrom Expectation to Habit: Why do Software Practitioners Adopt Fairness Toolkits?
Voria, G., Lambiase, S., Schiavone, M. C., Catolino, G., & Palomba, F. (2025) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Society (ICSE-SEIS) · doi:10.1109/icse-seis66351.2025.00015 · Cited by 8
“Lee and Singh [19] examined the misalignment between current open-source fairness toolkits and practitioners’ needs through focus groups, interviews, and surveys, identifying gaps that necessitate improved support for implementing fairness.”
Detecting Risk of Biased Output with Balance Measures
Mecati, M., Vetrò, A., & Torchiano, M. (2022) · Journal of Data and Information Quality · doi:10.1145/3530787 · Cited by 8
“Our work is also complementary to the existing toolkits for bias detection and mitigation [12], since therein the proposed measures of balance are not taken into consideration yet.”
Exploring Participants’ Representations and Shifting Sensitivities in a Hackathon for Dementia
Hodge, J., Foley, S., Lambton-Howard, D., Booi, L., Montague, K., Coulter, S., Kirk, D. S., & Morrissey, K. (2023) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3571814 · Cited by 8
Gaps Between Research and Practice When Measuring Representational Harms Caused by LLM-Based SystemsPreprint
Harvey, E., Sheng, E., Blodgett, S. L., Chouldechova, A., Garcia-Gathright, J. I., Olteanu, A., & Wallach, H. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2411.15662 · Cited by 8
“Particularly salient to our participants (and in line with considerable prior work [e.g., 2, 6, 7, 19–21, 29, 37, 39, 41, 47, 49–51, 53]) were challenges related to doing measurement in practice —i.e., when working within the constraints surrounding products and services that are deployed to real…”
Hollanek, T., Pi, Y., Fiorini, C., Vignali, V., Peters, D., & Drage, E. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732078 · Cited by 7
Muli-Agent LLMs as Ethics Advocates for AI based Systems
Yamani, A. Z., Baslyman, M., & Ahmed, M. (2025) · IEEE International Requirements Engineering Conference Workshops (REW) · doi:10.1109/rew66121.2025.00078 · Cited by 7
“It also necessitates a shared understanding between stakeholders and requirements engineers on this interdisciplinary topic and its importance [4]–[6].”
Explainable Fairness in Regulatory Algorithmic AuditingPreprint
O’Neil, C., Sargeant, H., & Appel, J. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4756637 · Cited by 7
Kallina, E., & Singh, J. (2025) · Proceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706599.3719726 · Cited by 6
Self-citationTalking About the Assumption in the Room
Mothilal, R. K., Lalani, F. M., Ahmed, S. I., Guha, S., & Sultana, S. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713958 · Cited by 6
“Prior works have argued against the techno-solutionist adoption of these toolkits where the mere following of specific processes is often assumed to solve ethical or fairness issues [7, 25, 74, 137].”
Wong, R. Y. (2025) · Decennial Aarhus Conference · doi:10.1145/3744169.3744171 · Cited by 7
“…research, HCI and adjacent fields have designed many values and ethics tools to help tech workers consider and address values and ethical issues during the technology design process, which prior researchers have inventoried and analyzed across multiple dimensions (e.g., [18, 34, 82, 150]).”
The EU AI act in development practice: a pro-justice approach
Drage, E., Pi, Y., Hollanek, T., Peters, D., & Yakar, S. (2025) · AI and Ethics · doi:10.1007/s43681-025-00885-4 · Cited by 6
“Applying our Pro-Justice Lens”
Calzati, S., & Ploeger, H. D. (2024) · Big Data & Society · doi:10.1177/20539517241270687 · Cited by 6
LUCID: Exposing Algorithmic Bias through Inverse Design
Mazijn, C., Prunkl, C., Algaba, A., Danckaert, J., & Ginis, V. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i12.26683 · Cited by 6
Mental Models of Adversarial Machine LearningPreprint
Bieringer, L., Grosse, K., Backes, M., Biggio, B., & Krombholz, K. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.03726 · Cited by 6
“In line with recent work on fairness [52] and ethics [22], we consider this crucial for designing usable and accessible tools, corporate guidelines and regulations.”
BackgroundPractitioners and Bias in Machine Learning: A Study
Cinca, R., Costanza, E., & Musolesi, M. (2025) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3733838 · Cited by 5
“Similar research by Deng et al. [23] and Lee and Singh [43] support these findings, with interviews and surveys indicating that fairness tools need to provide more context-specific guidance for ML practitioners throughout the entire development process.”
Bias Delayed is Bias Denied? Assessing the Effect of Reporting Delays on Disparity Assessments
Gosciak, J., Balagopalan, A., Ouyang, D., Koenecke, A., Ghassemi, M., & Ho, D. E. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732123 · Cited by 5
Sharma, S., Kaur, N., Kaur, G., & Sood, P. (2025) · Wiley eBooks · doi:10.1002/9781394287383.ch9 · Cited by 5
Fairness set and forgotten: Mining fairness toolkit usage in open-source machine learning projects
Cannavale, A., Voria, G., Scognamiglio, A., Giordano, G., Catolino, G., & Palomba, F. (2025) · Information and Software Technology · doi:10.1016/j.infsof.2025.107957 · Cited by 5
FAIR and Beyond: Evolving Principles for Modern Data Ecosystems
Bashir, M. A. (2025) · Lecture notes in networks and systems · doi:10.1007/978-3-031-89274-5_12 · Cited by 5
Exploring Visualization for Fairness in AI Education
Yan, X., Zhou, Y., Mishra, A., Mishra, H., & Wang, B. (2024) · IEEE Pacific Visualization Conference (PacificVis) · doi:10.1109/pacificvis60374.2024.00010 · Cited by 5
“However, the IBM’s tool conceals all technical details and is perceived to be overly simplified by Lee and Singh [32], ranking as the least effective by Mashhadi et al. [34].”
BackgroundMethodInfluentialFerreira, C. G. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2308.02042 · Cited by 5
“…such as sci-kit-fairness/sci-kit-lego, IBM Fairness 360 (AIF360), Aequitas Tool, Google What-if tool, and Fair-learn are available aiming to assess and mitigate unfair outcomes in algorithms, despite the existence of gaps between their capabilities and the real needs of practitioners [46].”
BackgroundFair Enough? A map of the current limitations of the requirements to have fair algorithmsPreprint
Regoli, D., Castelnovo, A., Inverardi, N., Nanino, G., & Penco, I. G. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2311.12435 · Cited by 5
“Unfortunately, the level of maturity and consistency of such toolkits appears to be still very low, and their effectiveness in addressing the overall challenge of “fairness” is debated (Lee and Singh, 2021).”
Blodgett, S. L., Curry, A. C., Dev, S., Madaio, M., Nenkova, A., Yang, D., & Xiao, Z. (2024) · doi:10.18653/v1/2024.hcinlp-1 · Cited by 5
The problem of fairness in tools for algorithmic fairness
Sun, X.-Y., Ye, B., & Xia, B.-H. (2024) · AI and Ethics · doi:10.1007/s43681-024-00533-3 · Cited by 4
“However, some scholars argue that fairness tools oversimplify the definition of fairness and may not be applicable or beneficial in addressing algorithmic fairness issues [43].”
An Exploratory Analysis of Available Fairness Interventions in Open Source
Mim, S. A., Zohra, F. T., Smith, J., & Johnson, B. (2025) · IEEE International Conference on Software Analysis, Evolution and Reengineering - Companion (SANER-C) · doi:10.1109/saner-c66551.2025.00010 · Cited by 4
“This focus likely stems from the widespread use of classification algorithms in critical decision-making areas [7].”
InfluentialGuidelines for designing visualization tools for group fairness analysis in binary classification
Cruz, A., Salazar, T., Carvalho, M., Maç�ãs, C., Machado, P., & Abreu, P. H. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11179-w · Cited by 5
“Lee and Singh ( 2021 ) compare fairness toolkits and identify gaps between their capabilities and practitioner needs.”
Navigating Ensemble Configurations for Algorithmic FairnessPreprint
Feffer, M., Hirzel, M. J., Hoffman, S. C., Kate, K., Ram, P., & Shinnar, A. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2210.05594 · Cited by 4
“Our work takes inspiration from earlier empirical studies of fairness techniques (Biswas and Rajan 2021; Friedler et al. 2019; Holstein et al. 2019; Lee and Singh 2021; Singh et al. 2021; Valentim, Lourenc¸o, and Antunes 2019; Yang et al. 2020), which help practitioners and researchers better…”
Method“It’s Just a Wild, Wild West”: Harnessing Public Procurement as an AI Governance Mechanism
Hudig, A. I., Kallina, E., & Singh, J. (2026) · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791968 · Cited by 3
Self-citationYan, J. N., Harvey, E., Wang, J., Rzeszotarski, J. M., & Koenecke, A. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791347 · Cited by 4
“Practitioners struggle to reconcile incompatible fairness metrics [71, 83, 91, 116], work with messy real-world data [34, 54, 59, 97, 99, 106, 108], and adapt general-purpose academic tools and metrics to their specific real-world needs [11, 32, 37, 55, 56, 66, 76, 84, 93, 96, 105].”
Inside Fairness Tools: What Academic Practitioners Really Experience
Mim, S. A., & Johnson, B. (2025) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vl-hcc65237.2025.00067 · Cited by 3
“Some studies have conducted human-centric evaluations to compare these tools [1], while others have examined how comprehensively these tools establish the broader concept of ethics [5].”
From Reflection to Repair: A Scoping Review of Dataset Documentation Tools
Reynolds-Cuéllar, P., Wong-Villacrés, M., Garcia, A. A., & Precel, H. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791344 · Cited by 3
“We argue that increasing focus on producing empirical evidence of how documentation meaning-fully contributes to increasing accountability and transparency, can help practitioners better judge the value of tools for their particular applications.”
Toward Third-Party Assurance of AI Systems: Design Requirements, Prototype, and Early Testing
Kim, R., Kuehnert, B., Lai, A., Holstein, K., Heidari, H., & Ghani, R. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3806759 · Cited by 4
Fairness under uncertainty in sequential decisions
Lee, M. S. A., Padh, K. S., Watson, D., Kilbertus, N., & Singh, J. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3812272 · Cited by 3
“Similar visualizations of the ML lifecycle have been used in previous work [38].”
Self-citationLee, M. S. A., Floridi, L., & Singh, J. (2022) · Digital ethics lab yearbook · doi:10.1007/978-3-031-09846-8_11 · Cited by 3
Self-citationLenders, D., & Calders, T. (2023) · AI and Ethics · doi:10.1007/s43681-023-00342-0 · Cited by 3
“Hence, interviewees in the studies of [12] and [15] expressed their interest in tools that let them explore the statistical significance of subgroup biases, by reporting the confidence interval of bias measures.”
InfluentialMANILA: A Low-Code Application to Benchmark Machine Learning Models and Fairness-Enhancing Methods
d’Aloisio, G. (2025) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3696630.3728600 · Cited by 3
Quintessence: An Intersectional Reflexivity Tool for Data-Centric Research & Development
Boyd, A. E., Jakpor, J., & Johnson, B. (2022) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vl/hcc53370.2022.9833001 · Cited by 3
Evaluating Behavior Change Interventions for Responsible Data Science
Zhang, D., Wu, K., Battle, L., & Wall, E. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791566 · Cited by 2
“Lee and Singh[24] conducted a comprehensive landscape analysis of open source fairness toolkits, providing a comparative assessment of existing tools and identifying key gaps in the fairness toolkit ecosystem.”
Coverage for Detecting Bias in Facial Recognition Datasets
Mhatre, S., Lee, B., Lanus, E., & Freeman, L. (2025) · IEEE Conference on Artificial Intelligence (CAI) · doi:10.1109/cai64502.2025.00047 · Cited by 3
“An extensive review found disparities between the toolkits used for evaluating and mitigating bias [12].”
Identifying Imbalance Thresholds in Input Data to Achieve Desired Levels of Algorithmic Fairness
Mecati, M., Adrignola, A., Vetrò, A., & Torchiano, M. (2022) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata55660.2022.10021078 · Cited by 2
“In addition, our work is complementary to the existing toolkits for bias detection and mitigation [23] since the balance measures proposed herein have not been taken into account yet.”
MethodA Proposal of a Fair Voting Ensemble Classifier Using Multi-objective Optimization
Monteiro, W. R., & Reynoso-Meza, G. (2024) · Lecture notes in networks and systems · doi:10.1007/978-3-031-51982-6_5 · Cited by 2
ShortKit-ML : an open toolkit for measuring shortcut potential in medical imaging embeddingsPreprint
Cajas, S. A., Marzullo, A., Kapadia, S., Santos, F., Osorio, F. O., Kong, Q., Quarta, A., Kuo, P., Patel, M., Sillery, R. I. R., et al. (2026) · medRxiv · doi:10.64898/2026.04.29.26352053 · Cited by 2
“Moreover, in the absence of a proper characterization of representational bias, strategies such as oversampling, often used to “guarantee” balanced data, may fail to mitigate disparities and may even amplify pre-existing inequities [35].”
Kilhoffer, Z. (2025) · Illinois Digital Environment for Access to Learning and Scholarship (University of Illinois at Urbana-Champaign) · Cited by 2
Interactive Human-Centric Bias Mitigation
Vejsbjerg, I., Daly, E., Nair, R., & Nizhnichenkov, S. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i21.30582 · Cited by 1
“However, a recent study of data scientists using these toolk-its stated that one of the most important features was the importance of the “ability to adapt to a context-specific use case and data” (Lee and Singh 2021).”
BackgroundKumar, S., Malviya, R., Sridhar, S. B., Shareef, J., & Wadhwa, T. (2025) · Anti-Inflammatory & Anti-Allergy Agents in Medicinal Chemistry · doi:10.2174/0118715230389406250906142050 · Cited by 1
Participatory Data Governance in Practice
Powar, J., Kallina, E., Hudig, A. I., Singh, J., Kyi, L., Janssen, H., Smith, G. M., & Zhang, R. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3778746 · Cited by 1
“…if based on the needs of communities to who that data relates, belongs, or affects—operate on the intersection of several traditional HCI domains, i.e. community needs [13, 16, 24, 36], transparency [4, 39], accountability [4, 32, 47, 48], and AI governance [23, 25, 26, 47, 48].”
Self-citationJin, A., Asemota, A., Krane, D. E., Adams, N., & Abebe, R. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791552 · Cited by 1
To Build or Not to Build? Factors that Lead to Non-Development or Abandonment of AI Systems
Chappidi, S., & Singh, J. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3806736 · Cited by 2
“Many academic works interrogate and propose artifacts to improve development practices, including auditing datasets and their creation [56], toolkits for fairness evaluations [94], and approaches to improve model transparency [107].”
Self-citationLambiase, S., Voria, G., Schiavone, M. C., Catolino, G., & Palomba, F. (2026) · Empirical Software Engineering · doi:10.1007/s10664-026-10902-7 · Cited by 1
“Lee and Singh (2021) examined the misalignment between current open-source fairness toolkits and practitioners’ needs through focus groups, interviews, and surveys, identifying gaps that necessitate improved support for implementing fairness.”
Data Preparation for Fairness-Performance Trade-Offs: A Practitioner-Friendly Alternative?Preprint
Voria, G., Di Matteo, R., Giordano, G., Catolino, G., & Palomba, F. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2412.15920 · Cited by 1
“These costs include both the additional computation demands they place on model development and the extra effort required for practitioners to apply them as they diverge from standard workflows [14], [15].”
Rocha, M. D. S., Pelegrina, G. D., & Duarte, L. T. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5472615 · Cited by 1
BiMi Sheets: Infosheets for bias mitigation methodsPreprint
Defrance, M., Bied, G., Buyl, M., Lijffijt, J., & Bie, T. D. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2505.22114 · Cited by 1
Context-conscious fairness throughout the machine learning lifecycle
Lee, S. A. (2022) · Apollo (University of Cambridge) · doi:10.17863/cam.104351 · Cited by 1
Hollanek, T., Pi, Y., Fiorini, C., Vignali, V., Peters, D., & Drage, E. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2505.17165 · Cited by 1
Organizational factors that influence fairness in algorithmic decision-support
Adelakun, D. (2024) · Document Server@UHasselt (UHasselt) · Cited by 1
Taka, E., Nakao, Y., Sonoda, R., Yokota, T., Luo, L., & Stumpf, S. (2023) · arXiv · doi:10.48550/arxiv.2312.08064
Trentz, C., Bell, T., Canahuate, G. M., & Casavant, T. L. (2022) · doi:10.17077/etd.006394
Functional Requirements for Interactive Bias-Audit Tools
Lenders, D., & Calders, T. (2023) · Frontiers in artificial intelligence and applications · doi:10.3233/faia230118
“Different forms of bias can be detected [3, 6, 7, 9, 11]”
BackgroundMethodWhat Makes a Fairness Tool Project Sustainable in Open Source?Preprint
Mim, S. A., Vares, F., Meneely, A., & Johnson, B. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2505.09802
“Lee et al. built a taxonomy of machine learning fairness tools based on a mixed methods study with industry practitioners that illuminated gaps in existing tools that impact their ability to meet practitioner needs [53].”
InfluentialAI Ethics Advisory and Principlism: Bridging Theory and Practice
Carnevale, A. (2025) · Springer eBooks · doi:10.1007/978-3-032-05526-2_2
Exploring the Landscape of Fairness Interventions in Software EngineeringPreprint
Mim, S. A. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2507.18726
“To address these challenges and ensure ethical outcomes in technological advancements, researchers have explored various approaches [71].”
InfluentialNavigating the Complexities of Algorithmic Auditing: Challenges and Considerations
Vecchione, B. (2023) · eCommons (Cornell University) · doi:10.7298/9826-gh79
Beattie, A. (2023) · SHAREOK (University of Oklahoma; Oklahoma State University; Central Oklahoma University)
Amin, S. (. (2024) · Research Repository (Delft University of Technology)
Fairness regularization in machine learning : methods and limitations
Buyl, M. (2023) · Ghent University Academic Bibliography (Ghent University)
김병필 (2023) · Seoul National University Open Repository (Seoul National University)
Kandul, S., Hertweck, C., & Leicht-Deobald, U. (2025) · AI and Ethics · doi:10.1007/s43681-025-00854-x
“However, interviews and surveys with practitioners revealed that practitioners are often overwhelmed by the sheer number of fairness metrics presented to them and lack guidance on how to choose among them [19–21 ].”
Robust Evaluation of Fairness and Explainability in AI Systems
de Jesus, S. G. P. (2025) · Portuguese National Funding Agency for Science, Research and Technology (RCAAP Project by FCT)
Towards Practical Software Discrimination Testing
Mim, S. A. (2025) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vl-hcc65237.2025.00061
“I encountered several studies that mapped the landscape of open-source fairness tools [7], [8], offering a comprehensive view of what was currently available.”
Statistical and Structural Approaches to Algorithmic FairnessPreprint
Ferrara, A. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.26200
Shukla, N. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3806457
What We Know about Responsible AI Practices in Industry: A Half Decade of Empirical ResearchPreprint
Deng, W. H., Balayn, A., Selbst, A., Hong, J. I., Eslami, M., Holstein, K., Wallach, H., Vaughan, J., & Barocas, S. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2608.10431
“Studies published between 2019 and 2022 often found that many developers lacked RAI awareness, as they had not yet recognized concepts such as AI fairness, explainability, or trustworthiness as relevant to their work prior to participating in the studies [83, 109, 121, 139, 146, 151, 190, 197, 198].”
InfluentialMartin, M. A., Copado, P. J., Vellido, A., & König, C. (2026) · Computational and Structural Biotechnology Journal · doi:10.34133/csbj.0215
“The appropriate approach is therefore to avoid unnecessary use of sensitive variables for prediction while still using them for audit, interpretation, and governance [ 75 – 79 ].”
Kulker, E. J. (2026) · TU/e Research Portal
Balayn, A., Rancourt, F., Casati, F., & Gadiraju, U. (2026) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3816909
“More broadly, AI researchers and policymakers propose structuring principles [64, 106, 154], frameworks and guidelines [15, 109], toolkits [16, 85], checklists [93, 123], and other documentation tools [8, 57, 105], to support responsible AI development within and across organizations.”
Integrating Ethical Algorithm Design into Data Science Education: An Accessible, Practical Framework
Ellison, V. (2026) · Journal of Statistics and Data Science Education · doi:10.1080/26939169.2026.2740033
A Critical Pragmatism Approach for AI Ethics: Lessons from Urban Planning Theory
Gosciak, J., Levy, K., & Koenecke, A. (2026) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1609/aies.v9i2.43210
“Deng et al. (2022) and Lee and Singh (2021) both explore how industry practitioners use fairness toolk-its, observing a need for more contextualization, integration with existing workflows, and accessible communication with non-technical audiences.”
Constructing the Artificial Intelligence research field: studies of infrastructures and practices
Berman, G. (2026) · ANU Open Research (Australian National University) · doi:10.25911/qjzb-as30
Enhancing fairness: A cross-platform application for mitigating bias in data ecosystems
Mavrogiorgos, K., Kiourtis, A., Mavrogiorgou, A., & Kyriazis, D. (2026) · Array · doi:10.1016/j.array.2026.101431
Li, Y. (2026) · Journal of King Saud University - Computer and Information Sciences · doi:10.1007/s44443-026-01469-w
Understanding Gaps in ML Fairness Tool Design: An Experience Report
Mim, S. A., Zohra, F. T., & Johnson, B. (2026) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vl/hcc71707.2026.11772861
Towards Onboarding Newcomers to Software Discrimination Testing
Mim, S. A. (2026) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vl/hcc71707.2026.11772795
Joseph, G. G. (2026) · Digital Policy Regulation and Governance · doi:10.1108/dprg-08-2026-0623
Metadata
- Authors
- Lee, M. S. A., & Singh, J.
- Affiliations
- University of Cambridge (GB)
- Venue
- Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2021-05-06
- Pages
- 1-13
- OpenAlex
- W3162866579
- Cited by
- 129
- References
- 38
- Citations by year
- 2026: 10 · 2025: 36 · 2024: 28 · 2023: 25 · 2022: 18 · 2021: 11
- FWCI
- 18.53
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- SSRN Electronic Journal (acceptedVersion)
- Funders
- Engineering and Physical Sciences Research Council (EP/P024394/1)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Business, Management and Accounting › Organizational Behavior and Human Resource Management › AI and HR Technologies
- Keywords
- algorithmic fairness, FAIRness assessment, fairness mitigation, practitioner needs, commercial deployment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.