Exploring How Machine Learning Practitioners (Try To) Use Fairness Toolkits
Open accessThe DOI matches its Crossref record; 120 of the 120 references it cites were found in a registry or an index.
Abstract
Recent years have seen the development of many open-source ML fairness toolkits aimed at helping ML practitioners assess and address unfairness in their systems. However, there has been little research investigating how ML practitioners actually use these toolkits in practice. In this paper, we conducted the first in-depth empirical exploration of how industry practitioners (try to) work with existing fairness toolkits. In particular, we conducted think-aloud interviews to understand how participants learn about and use fairness toolkits, and explored the generality of our findings through an anonymous online survey. We identified several opportunities for fairness toolkits to better address practitioner needs and scaffold them in using toolkits effectively and responsibly. Based on these findings, we highlight implications for the design of future open-source fairness toolkits that can support practitioners in better contextualizing, communicating, and collaborating around ML fairness efforts.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“While there are multiple studies on the use of tools for fairness evaluation [54, 70, 119] and explainability [1, 46, 72, 82, 85, 98], fewer studies examine how practitioners evaluate other criteria such as basic functionality [117], safety, privacy, or recourse, just as fewer tools exist for this purpose.”
From Towards AI Accountability Infrastructure: Gaps and Opportunities in AI Audit Tooling (2025) · cites this paper as [54]
In the citing paper
“AI ethics frameworks are only just starting to appear to provide designers with the language they need to begin discussing these important issues [36, 66, 180].”
From Design Principles for Generative AI Applications (2024) · cites this paper as [36]
In the citing paper
“These issues have been shown in more general ML-contexts; for example, studies exploring user (ML practitioner) attitudes and expectations regarding fairness toolkits showed a misalignment of tools with user expectations and needs, and generally being difcult to use (among other concerns), which not only makes such tooling less-efective, but can potentially lead to increased bias risks [40,91].”
From Out of Context: Investigating the Bias and Fairness Concerns of “Artificial Intelligence as a Service” (2023) · cites this paper as [40]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
R: A Language and Environment for Statistical Computing
R Core Team (2025) · doi:10.32614/r.manuals · Cited by 352,181
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016
Dermatologist-level classification of skin cancer with deep neural networks
Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017) · Nature · doi:10.1038/nature21056 · Cited by 14,362
(2017) · Cited by 11,625
Scikit-learn: Machine Learning in Python
(2011) · Journal of Machine Learning Research · doi:10.5555/1953048.2078195 · Cited by 8,539
Dissecting racial bias in an algorithm used to manage the health of populations
Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019) · Science · doi:10.1126/science.aax2342 · Cited by 7,137 · cited 1×
“A growing body of research has drawn attention to the ways these systems can, whether inadvertently or intentionally, serve to amplify existing social inequities or create new ones [12, 15, 17, 22, 48, 59, 78].”
BackgroundOn the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988
(2018) · Cited by 3,510
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012) · Proceedings of the 3rd Innovations in Theoretical Computer Science Conference · doi:10.1145/2090236.2090255 · Cited by 3,501 · cited 3×
“In our study, a large proportion of participants committed to the “fairness through unawareness” trap [34].”
BackgroundResultInfluential
Research works printed but not linked: 6
- Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. 2016. Machine bias. ProPublica, May 23 (2016), 2016.
- Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. Advances in neural information processing systems 29 (2016), 4349-4357.
- Avi Feller, Emma Pierson, Sam Corbett-Davies, and Sharad Goel. 2016. A computer program used for bail and sentencing decisions was labeled biased against blacks. It's actually not that clear. The Washington Post (2016).
- Min Kyung Lee, Daniel Kusbit, Anson Kahng, Ji Tae Kim, Xinran Yuan, Allissa Chan, Daniel See, Ritesh Noothigattu, Siheon Lee, Alexandros Psomas, et al. 2019. WeBuildAI: Participatory framework for algorithmic governance. Proceedings of the ACM on Human-Computer Interaction 3, CSCW (2019), 1-35.
- Bogdana Rakova, Jingying Yang, Henriette Cramer, and Rumman Chowdhury. 2020. Where Responsible AI meets Reality: Practitioner Perspectives on Enablers for shifting Organizational Practices. arXiv preprint arXiv:2006.12358 (2020). · arXiv
- Neil Vigdor. 2019. Apple card investigated after gender discrimination com- plaints. The New York Times (2019).
Cited by
Design Principles for Generative AI Applications
Weisz, J. D., He, J., Muller, M., Hoefer, G., Miles, R., & Geyer, W. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642466 · Cited by 232
“Various attempts have been made to assist design practitioners in the process of operationalizing guidelines for AI systems, including guide-books [133], toolkits [36], and checklists [110].”
BackgroundSeeing Like a Toolkit: How Toolkits Envision the Work of AI Ethics
Wong, R. Y., Madaio, M. A., & Merrill, N. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579621 · Cited by 135
“Several papers have performed systemic meta-reviews and empirical analyses of AI ethics toolkits [5, 19, 43, 55, 67].”
BackgroundInvestigating How Practitioners Use Human-AI Guidelines: A Case Study on the People + AI Guidebook
Yildirim, N., Pushkarna, M., Goyal, N., Wattenberg, M., & Viégas, F. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580900 · Cited by 104
“Recently, a few researchers investigated how specic resources and guidelines are used by industry practitioners, such as fairness toolkits [22] and conversational UI guidelines [51].”
BackgroundResultLiao, Q. V., Subramonyam, H., Wang, J., & Wortman Vaughan, J. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580652 · Cited by 86
“Recent years have seen a growing interest in studying practitioners’ practices, challenges, and gaps in dealing with RAI issues, such as fairness [35, 55, 74, 75, 82], transparency [18, 57, 66], and accountability [21, 81].”
BackgroundDesigning Responsible AI: Adaptations of UX Practice to Meet Responsible AI Challenges
Wang, Q., Madaio, M., Kane, S., Kapania, S., Terry, M., & Wilcox, L. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581278 · Cited by 90
“However, AI practitioners fnd it challenging to represent diverse demographic groups in datasets and in fairness assessments, given that practitioners tend to draw on their (often homogenous [106]) personal experiences and perspectives when it comes to RAI [30, 53, 69].”
BackgroundInfluentialDeng, W. H., Guo, B., Devrio, A., Shen, H., Eslami, M., & Holstein, K. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581026 · Cited by 103
“, [24, 42, 54]), recruitment for such studies can be highly challenging.”
BackgroundMethodInfluentialSelf-citationSeamful XAI: Operationalizing Seamful Design in Explainable AI
Ehsan, U., Liao, Q. V., Passi, S., Riedl, M. O., & Daumé, H. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3637396 · Cited by 105
“However, recent work studying how practitioners deal with RAI issues on the ground, including AI fairness [17, 38, 55, 63], transparency [35, 40, 50], accountability [70], and overall harms mitigation [4, 71], report that practitioners grapple with tremendous challenges [67].”
BackgroundTowards AI Accountability Infrastructure: Gaps and Opportunities in AI Audit Tooling
Ojewale, V., Steed, R., Vecchione, B., Birhane, A., & Raji, I. D. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713301 · Cited by 81
Deng, W. H., Yildirim, N., Chang, M., Eslami, M., Holstein, K., & Madaio, M. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594037 · Cited by 57
“ily on supporting more technical roles in conducting quantitative analyses [9, 27, 93], despite rhetoric about enabling socio-technical work [93].”
BackgroundResultInfluentialSelf-citationA.I. Robustness: a Human-Centered Perspective on Technological Challenges and Opportunities
Tocchetti, A., Corti, L., Balayn, A., Yurrita, M., Lippmann, P., Brambilla, M., & Yang, J. (2025) · ACM Computing Surveys · doi:10.1145/3665926 · Cited by 98
Real Risks of Fake Data: Synthetic Data, Diversity-Washing and Consent Circumvention
Whitney, C. D., & Norman, J. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3659002 · Cited by 92
Harnessing human and machine intelligence for planetary-level climate action
Debnath, R., Creutzig, F., Sovacool, B. K., & Shuckburgh, E. F. (2023) · npj Climate Action · doi:10.1038/s44168-023-00056-3 · Cited by 76
“FAccT scholars have proposed various solutions to establish a subset of fairness notions 41,42,47 .”
BackgroundLLM-Driven Robots Risk Enacting Discrimination, Violence, and Unlawful Actions
Hundt, A., Azeem, R., Mansouri, M. I., & Brandão, M. (2025) · International Journal of Social Robotics · doi:10.1007/s12369-025-01301-x · Cited by 66
“…and their limitations via comprehensive risk assessment [71, 94], more extensive red teaming, broader operational context, mechanisms for governance of robot operations, participatory [176, 194] input, governance of projects [94] and “go or no-go” decisions and fairness toolkits [195] for robotics.”
Ethics in AI through the practitioner’s view: a grounded theory literature review
Pant, A., Hoda, R., Tantithamthavorn, C. K., & Turhan, B. (2024) · Empirical Software Engineering · doi:10.1007/s10664-024-10465-5 · Cited by 66
Zeno: An Interactive Framework for Behavioral Evaluation of Machine Learning
Cabrera, Á. A., Fu, E., Bertucci, D., Holstein, K., Talwalkar, A., Hong, J. I., & Perer, A. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581268 · Cited by 51
“Furthermore, many behavioral evaluation tools, such as fairness toolkits, often do not support the types of models, data, and behaviors that practitioners work with in the real world [17].”
BackgroundSelf-citationA hunt for the Snark: Annotator Diversity in Data Practices
Kapania, S., Taylor, A. S., & Wang, D. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580645 · Cited by 50
“, [22, 53]) through research with practitioners.”
van Berkel, N., Sarsenbayeva, Z., & Gonçalves, J. (2022) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2022.102954 · Cited by 61
Sketching AI Concepts with Capabilities and Examples: AI Innovation in the Intensive Care Unit
Yildirim, N., Zlotnikov, S., Sayar, D., Kahn, J. M., Bukowski, L. A., Amin, S. S., Riman, K. A., Davis, B. S., Minturn, J. S., King, A. J., et al. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3641896 · Cited by 55
“In recent years, resources in the form of guidelines and toolkits became available to address some of AI’s design challenges (e.g., human-AI guidelines [2, 3, 94], fairness toolkits [30]).”
Brintrup, A. M., Baryannis, G., Tiwari, A., Ratchev, S. M., Martínez-Arellano, G., & Singh, J. (2025) · Data-Centric Engineering · doi:10.1017/dce.2025.10032 · Cited by 49
Self-citationConstantinides, M., Bogucka, E. P., Quercia, D., Kallio, S., & Tahaei, M. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3686927 · Cited by 59
“Similarly, Deng et al. [21] stressed the lack of standardized guidelines in toolkits like AIF360 for introducing fairness issues to non-technical collaborators.”
Learning about Responsible AI On-The-Job: Learning Pathways, Orientations, and Aspirations
Madaio, M., Kapania, S., Qadri, R., Wang, D., Zaldivar, A., Denton, E., & Wilcox, L. G. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3658988 · Cited by 56
Can Fairness be Automated? Guidelines and Opportunities for Fairness-aware AutoML
Weerts, H., Pfisterer, F., Feurer, M., Eggensperger, K., Bergman, E., Awad, N. H., Vanschoren, J., Pechenizkiy, M., Bischl, B., & Hutter, F. (2024) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.14747 · Cited by 53
“Practitioners often lack knowledge on how to integrate fairness toolkits into their work-flow (Holstein et al., 2019; Deng et al., 2022) and may face difficulties in proper evaluation of fairness metrics (Agrawal et al., 2020).”
BackgroundKuehnert, B., Kim, R., Forlizzi, J., & Heidari, H. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732191 · Cited by 48
From Paper to Card: Transforming Design Implications with Generative AI
Shin, D.-H., Wang, L. L., & Hsieh, G. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642266 · Cited by 39
Navigating fairness measures and trade-offs
Buijsman, S. (2023) · AI and Ethics · doi:10.1007/s43681-023-00318-0 · Cited by 40
“…access to financing is one that makes sense, and that with only the decision tree at hand it can be easy to end up with a fairness measure that does more harm than good, especially considering the somewhat ad hoc way in which fairness toolkits are sometimes used in practice (Deng et al., 2022).”
BackgroundLewicki, K., Lee, M. S. A., Cobbe, J., & Singh, J. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581463 · Cited by 30
“[40], which considered the perceptions and use of so-called “fairness toolkits” that aim to support ML practitioners with fairness concerns, fnding signifcant disconnects”
BackgroundSelf-citationA novel approach for assessing fairness in deployed machine learning algorithms
Uddin, S., Lu, H., Rahman, A., & Gao, J. (2024) · Scientific Reports · doi:10.1038/s41598-024-68651-w · Cited by 34
Smith, J. J., Beattie, L., & Cramer, H. S. M. (2023) · International World Wide Web Conference (WWW) · doi:10.1145/3543507.3583204 · Cited by 34
“Skew at K [9], Top-K Fairness [2], Normalized Discounted Diference [20], Viable-Lambda Test [16] Multi-group Item Exposure Provider groups should have an equal item exposure distribution.”
BackgroundConstantinides, M., Tahaei, M., Quercia, D., Stumpf, S., Madaio, M., Kennedy, S., Wilcox, L. G., Vitak, J., Cramer, H. S. M., Bogucka, E. P., et al. (2024) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3613905.3643979 · Cited by 32
Human-Centered Evaluation and Auditing of Language Models
Xiao, Z., Deng, W. H., Lam, M. S., Eslami, M., Kim, J., Lee, M., & Liao, Q. V. (2024) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3613905.3636302 · Cited by 40
Self-citationThe Fall of an Algorithm: Characterizing the Dynamics Toward Abandonment
Johnson, N., Moharana, S., Harrington, C., Andalibi, N., Heidari, H., & Eslami, M. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3658910 · Cited by 28
Trust and transparency in AI: industry voices on data, ethics, and compliance
McCormack, L., Bendechache, M., Lewis, D., & Huyskes, D. (2025) · AI & Society · doi:10.1007/s00146-025-02654-7 · Cited by 32
“…also found a lack of suitable industry tools, highlighting the impractical level of human effort required to make existing toolkits work to mitigate bias effectively ([11]), and the difficulties in translating real-world use cases into the quantifiable metrics required by these toolkits ([12]).”
WeAudit: Scaffolding User Auditors and AI Practitioners in Auditing Generative AI
Deng, W. H., Claire, W., Han, H. Z., Hong, J., Holstein, K., & Eslami, M. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757702 · Cited by 28
Self-citationAshktorab, Z., Hoover, B., Agarwal, M., Dugan, C., Geyer, W., Yang, H., & Yurochkin, M. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581227 · Cited by 24
“Other research suggests that fairness toolkits should broaden their scope, be designed for interdisciplinary research, and include anti-patterns in addition to patterns for using toolkits [17].”
BackgroundNavigating fairness: practitioners’ understanding, challenges, and strategies in AI/ML development
Pant, A., Hoda, R., Tantithamthavorn, C. K., & Turhan, B. (2025) · Empirical Software Engineering · doi:10.1007/s10664-025-10650-0 · Cited by 24
“Several qualitative studies, such as those by Deng et al. [12], Richard-son et al. [45], and Balayn et al. [4], have explored AI practitioners’ experiences and perspectives on specific fairness toolkits.”
Examining the Expanding Role of Synthetic Data Throughout the AI Development Pipeline
Kapania, S., Ballard, S. A., Kessler, A., & Vaughan, J. W. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732005 · Cited by 31
“This line of work has explored practices around responsible technology development [18, 19, 40, 61, 62, 75, 79, 100, 107] and examined data practices specifically [15, 34, 37, 50, 67, 82, 103, 109], exposing the challenges faced by practitioners throughout data curation [ 34], exploratory data…”
Kaur, H., Conrad, M. R., Rule, D., Lampe, C. A. C., & Gilbert, É. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3637354 · Cited by 33
WeAudit: Scaffolding User Auditors and AI Practitioners in Auditing Generative AIPreprint
Deng, W. H., Wang, C., Han, H. Z., Hong, J., Holstein, K., & Eslami, M. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2501.01397 · Cited by 20
Self-citationLin, H., Karusala, N., Okolo, C. T., D’Ignazio, C., & Gajos, K. Z. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3687009 · Cited by 24
“…to help AI developers responsibly build tools, but studies have found that factors such as conflicting values within teams [115], profit motives [32], lack of context-specific guidance [33, 51, 52, 124], or lack of prior experience [7] shape whether developers can successfully implement them.”
Fairness-aware practices from developers’ perspective: A survey
Voria, G., Sellitto, G., Ferrara, C., Abate, F., Lucia, A. D., Ferrucci, F., Catolino, G., & Palomba, F. (2025) · Information and Software Technology · doi:10.1016/j.infsof.2025.107710 · Cited by 23
Supporting User Engagement in Testing, Auditing, and Contesting AI
Deng, W. H., Lam, M. S., Cabrera, Á. A., Metaxa, D., Eslami, M., & Holstein, K. (2023) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/3584931.3611279 · Cited by 23
“However, we still lack understanding on the design requirements and considerations to effectively engage users [9, 10, 19, 21, 34].”
BackgroundSelf-citationDeng, W. H., Barocas, S., & Vaughan, J. W. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3711076 · Cited by 23
Self-citationA Catalog of Fairness-Aware Practices in Machine Learning Engineering
Voria, G., Sellitto, G., Ferrara, C., Abate, F., Lucia, A. D., Ferrucci, F., Catolino, G., & Palomba, F. (2025) · International Conference on Evaluation and Assessment in Software Engineering (EASE) · doi:10.1145/3727967.3756824 · Cited by 18
Wang, Z. J., Munechika, D., Lee, S., & Chau, D. H. (2024) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3613905.3650848 · Cited by 18
“Finally, we observe that users request developers to provide modular APIs so that they can apply data transformation and analysis functionalities themselves in notebook code before instantiating the VA tools.”
Towards a Non-Ideal Methodological Framework for Responsible ML
Mothilal, R. K., Guha, S., & Ahmed, S. I. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642501 · Cited by 16
“Similar to how data is engaged, Deng et al. [26] fnds that practitioners’ analysis choices with fairness toolkits are heavily infuenced by their personal experiences, knowledge, and beliefs and discusses how practitioners even reformulate the actual ML problem in a format supported by these…”
MethodBalayn, A., Rikalo, N., Yang, J., & Bozzon, A. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581555 · Cited by 18
“Our methodology inspired from previous works [11, 30, 44] already provided us with main challenges and limitations.”
BackgroundMethodResultInfluentialAn Emerging Design Space of How Tools Support Collaborations in AI Design and Development
Muralikumar, M. D., & McDonald, D. W. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3701181 · Cited by 11
“…how toolkits shape work practices related to ethical AI development [104], constructed a design space of how tools support different stages in the sensemaking framework of model behavior [24], and analyzed limitations of open-source fairness toolkits in promoting reflexive practice [13, 31].”
Access Denied: Meaningful Data Access for Quantitative Algorithm Audits
Zaccour, J., Binns, R., & Rocher, L. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713963 · Cited by 14
Yurrita, M., Verma, H., Balayn, A., Gadiraju, U., Pont, S. C., & Bozzon, A. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713145 · Cited by 13
Gorichanaz, T. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3711008 · Cited by 11
“Moreover, existing structures such as toolkits may not adequately address the ethical issues they intend to address [19, 109].”
Towards a Responsible AI Organizational Maturity Model
Heger, A., Passi, S., Dhanorkar, S., Kahn, Z., Wang, R., & Vorvoreanu, M. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757514 · Cited by 19
Yurrita, M., Balayn, A., & Gadiraju, U. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.00739 · Cited by 17
“, data visualizations and reports tailored for non-engineers [9, 24]) to enable discursive negotiation [20].”
BackgroundInfluentialUsing Model Cards for ethical reflection on machine learning models: an interview-based study
Nunes, J. L., Barbosa, G. D. J., de Souza, C. S., & Barbosa, S. D. J. (2024) · Journal on Interactive Systems · doi:10.5753/jis.2024.3444 · Cited by 16
Employing Hybrid AI Systems to Trace and Document Bias in ML Pipelines
Russo, M., Chudasama, Y., Purohit, D., Sawischa, S. F., & Vidal, M.-E. (2024) · IEEE Access · doi:10.1109/access.2024.3427388 · Cited by 11
“…metrics to use, how to identify sensitive attributes, how to interpret algorithmic output or lack of access to domain knowledge [26], [44], [45], [46], in our work, we propose the implementation of existing bias measures to comprehend the factors influencing the manifestation of bias patterns in…”
BackgroundNavigating the Audit Landscape: A Framework for Developing Transparent and Auditable XR
Norval, C., Cloete, R., & Singh, J. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3594090 · Cited by 13
“There is potential for work exploring how such a framework can best integrate into, and augment, existing developer processes for realising auditable systems (in line with work exploring how developers can be better assisted in making decisions [67], finding information [21], and using fairness assessment tools [29, 6…”
BackgroundSelf-citationArticulation Work and Tinkering for Fairness in Machine Learning
Fahimi, M., Russo, M., Scott, K. M., Vidal, M.-E., Berendt, B., & Kinder-Kurlanda, K. E. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3686973 · Cited by 12
““So you were sitting together with the configuration of people that is somehow given because they are there, and they are supposed to work now on something.”
BackgroundMei, K. X., Choi, A. S. G., Schellmann, H., Sloane, M., & Koenecke, A. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3812320 · Cited by 13
“Our sample size of seven participants, while small, follows the practices of recent papers similarly exploring fairness in technology with only 10-13 interviewees [13, 25, 26, 41], and is further restricted given the low incidence rate of aphasia (estimated at 0.06% [18]).”
InfluentialHarrison, G., Bryson, K., Bamba, A. E. B., Dovichi, L., Binion, A. H., Borem, A., & Ur, B. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642755 · Cited by 11
Martino, V. D., Voria, G., Troiano, C., Catolino, G., & Palomba, F. (2025) · Journal of Systems and Software · doi:10.1016/j.jss.2025.112458 · Cited by 10
“By leveraging AIF360, which has been validated in both academic research and industry practice [57, 32], we minimized such risks and ensured the reliability and reproducibility of our findings.”
Toward Patient-Centered AI Fact Labels: Leveraging Extrinsic Trust Cues
Yoo, D. W., Stroud, A. M., Zhu, X., Miller, J. E., & Barry, B. A. (2025) · Designing Interactive Systems Conference · doi:10.1145/3715336.3735758 · Cited by 9
“AI documentation, an emerging effort in providing relevant information about AI models such as model cards [49], has been proposed as one of the potential solutions to these challenges [3, 7, 17, 65].”
Gaps Between Research and Practice When Measuring Representational Harms Caused by LLM-Based SystemsPreprint
Harvey, E., Sheng, E., Blodgett, S. L., Chouldechova, A., Garcia-Gathright, J. I., Olteanu, A., & Wallach, H. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2411.15662 · Cited by 8
“Particularly salient to our participants (and in line with considerable prior work [e.g., 2, 6, 7, 19–21, 29, 37, 39, 41, 47, 49–51, 53]) were challenges related to doing measurement in practice —i.e., when working within the constraints surrounding products and services that are deployed to real…”
Human-Centered Evaluation and Auditing of Language Models
Liu, Y. L., Deng, W. H., Lam, M. S., Eslami, M., Kim, J., Liao, Q. V., Xu, W., Novikova, J., & Xiao, Z. (2025) · Proceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706599.3706729 · Cited by 7
Self-citationSolyst, J., Wilcox, L. G., & Madaio, M. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714144 · Cited by 9
“Within the human-computer interaction (HCI) research community, there has been growing interest in industry practices for learning about and creating responsible AI [e.g., 27, 35, 100].”
From Expectation to Habit: Why do Software Practitioners Adopt Fairness Toolkits?
Voria, G., Lambiase, S., Schiavone, M. C., Catolino, G., & Palomba, F. (2025) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Society (ICSE-SEIS) · doi:10.1109/icse-seis66351.2025.00015 · Cited by 8
“Deng et al. [18] conducted an empirical investigation into industry practitioners’ engagement with fairness toolkits, identifying usability and effectiveness improvements through think-aloud interviews and surveys.”
InfluentialOkolo, C. T., & Lin, H. (2024) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3613905.3651080 · Cited by 12
“While the flexibility might allow practitioners to adapt general guidelines to local contexts, it might also cause practitioners to feel unsupported and take on additional tasks on top of their existing workloads [11, 33, 36].”
BackgroundGörücü, S., Ren, Y., SAMUEL, G. G., & Panagiotidou, G. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732088 · Cited by 6
Ren, B., Cheon, E., & Li, J. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757641 · Cited by 7
Sarkar, P., & Liem, C. C. S. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3637360 · Cited by 7
Collaboratively Designing and Evaluating Responsible AI Interventions
Deng, W. H., Berman, G., Kaur, H., Schwartz, R., Ibrahim, M. A., Eslami, M., Holstein, K., & Madaio, M. (2024) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/3678884.3687136 · Cited by 5
“For example, addressing fairness issues in training data may require data scientists to work alongside social scientists and legal scholars to better anticipate and identify representativeness factors, and define and implement a mitigation strategy [6].”
Self-citationVipera: Towards systematic auditing of generative text-to-image models at scale
Huang, Y., Deng, W. H., Xiao, S., Eslami, M., Hong, J., & Perer, A. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3719757 · Cited by 5
“We believe Vipera will contribute significantly to the advancement of end-user auditing and foster more responsible human-AI collaboration in creative applications.”
Self-citationWu, M., Zhao, Y., Han, S., Liu, M. X., & Shen, H. Y. (2026) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3816897 · Cited by 6
Guerdan, L., Saxena, D., Chancellor, S., Wu, Z. S., & Holstein, K. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757628 · Cited by 7
Self-citationThe Enterprising and Elusive Prospects of Human-AI Collaboration
Gadiraju, U., & Balayn, A. (2025) · Springer eBooks · doi:10.1007/978-3-032-01940-0_7 · Cited by 5
Shrestha, H., Cachel, K., Alkhathlan, M., Rundensteiner, E. A., & Harrison, L. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3594108 · Cited by 8
RecSys Fairness Metrics: Many to Use But Which One To Choose?Preprint
Smith, J. J., & Beattie, L. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2209.04011 · Cited by 7
The Generative AI Ethics PlaybookPreprint
Smith, J. J., Deng, W. H., Smith, W. H., Sap, M., DeCario, N., & Dodge, J. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2501.10383 · Cited by 6
Self-citationPractitioner Insights on Fairness Requirements in the AI Development Life Cycle: An Interview Study
Boufaied, C., Nguyen, T. H., & Santos, R. D. S. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3821435 · Cited by 6
“Several definitions of fairness exist in AI/ML systems [6, 9, 13, 14, 17, 19, 29, 33, 35, 38, 40, 47, 48], including statistical/demographic parity (similar outcomes across groups), equalized odds (i.e., achieving similar true positive and false positive rates, TPR and FPR, respectively), fairness…”
InfluentialDull, Dirty, Dangerous: Understanding the Past, Present, and Future of a Key Motivation for Robotics
Nakajima, N., Reynolds-Cuéllar, P., Lynch, C., & Darling, K. (2026) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1145/3757279.3785574 · Cited by 4
A Design Space of Behavior Change Interventions for Responsible Data Science
Zhang, D., Barrett, T., Patil, A., Shoda, Y., Battle, L., & Wall, E. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708359.3712140 · Cited by 6
“However, it is still difficult for data scientists to navigate complex ethical considerations even with interventions [16].”
Rajanandhini, V. M., Alruwais, N., Mazroa, A. A., & Alkharashi, A. (2024) · Matéria (Rio de Janeiro) · doi:10.1590/1517-7076-rmat-2024-0569 · Cited by 4
Towards "Anytime, Anywhere" Community Learning and Engagement around the Design of Public Sector AI
Deng, W. H., Eslami, M., & Holstein, K. (2023) · arXiv · doi:10.48550/arxiv.2304.00167 · Cited by 3
“How can we design to empower diverse community members, spanning a broad range of backgrounds and relevant literacies, to articulate their desires for new technologies that can better address their actual needs, as well as their concerns about technologies currently in use? How can we equip children and young adults w…”
BackgroundSelf-citationPractitioners and Bias in Machine Learning: A Study
Cinca, R., Costanza, E., & Musolesi, M. (2025) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3733838 · Cited by 5
“Similar research by Deng et al. [23] and Lee and Singh [43] support these findings, with interviews and surveys indicating that fairness tools need to provide more context-specific guidance for ML practitioners throughout the entire development process.”
InfluentialHuang, Y., Deng, W. H., Xiao, S., Eslami, M., Hong, J., Narechania, A. A., & Perer, A. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791942 · Cited by 4
Self-citationOkolo, C. T., & Lin, H. (2024) · ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO) · doi:10.1145/3689904.3694707 · Cited by 5
Guidelines for designing visualization tools for group fairness analysis in binary classification
Cruz, A., Salazar, T., Carvalho, M., Maç�ãs, C., Machado, P., & Abreu, P. H. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11179-w · Cited by 5
“Similarly, Deng et al. ( 2022 ) and Richardson et al. ( 2021 ) both explore how industry practitioners interact with fairness toolkits.”
WeAudit: Scaffolding User Auditors and AI Practitioners in Auditing Generative AIPreprint
Deng, W. H., Wang, C., Han, H. Z., Hong, J., Holstein, K., & Eslami, M. (2025) · Qeios · doi:10.32388/b1lxnv · Cited by 4
Self-citationTalking About the Assumption in the Room
Mothilal, R. K., Lalani, F. M., Ahmed, S. I., Guha, S., & Sultana, S. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713958 · Cited by 6
“…prior works refer to practitioners’ assumptions as one of the factors hampering their collaborative efforts with stakeholders of different technical backgrounds [127, 134, 139]; a few others discuss how assumptions distort practitioners’ understanding of ethical or fairness issues [4, 14, 25, 55].”
Yan, J. N., Harvey, E., Wang, J., Rzeszotarski, J. M., & Koenecke, A. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791347 · Cited by 4
Practical Perspectives on Fairness In Technology-Aided Personnel Selection
Müller, I., Dill, K., & Mihaljević, H. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757647 · Cited by 6
Nahar, N., Yang, C., Chen, Y., Deng, W. H., Holstein, K., Eslami, M., & Kästner, C. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791285 · Cited by 3
“Due to the sensitive nature of responsible AI and non-championship, we intentionally did not collect details about user study participants’ demographics, following prior work on responsible AI practices in industry [25, 46, 64, 65].”
Self-citationFrom Reflection to Repair: A Scoping Review of Dataset Documentation Tools
Reynolds-Cuéllar, P., Wong-Villacrés, M., Garcia, A. A., & Precel, H. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791344 · Cited by 3
“For example, HCI-related research has already suggested that a productive integration pathway is to design for challenging linear views of data lifecycles [47, 48, 81].”
InfluentialBitacora: A Toolkit for Supporting NonProfits to Critically Reflect on Social Media Data Use
Garcia, A. A., Wong-Villacrés, M., Hernández, B., & Dantec, C. A. L. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642673 · Cited by 4
“…design of toolkits for guiding technology-savvy users as they apply novel technologies such as Machine learning (ML), Artificial Intelligence (AI), and cybersecurity systems have stressed, however, that using toolkits to support responsible technology adoption is not free of problems [32, 97, 119].”
BackgroundInfluentialImpact Charts: A Tool for Identifying Systematic Bias in Social Systems and Data
Vengroff, D. E. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3658965 · Cited by 3
“In studying this we intend to build on existing work on how practitioners work with anti-bias tools and how tools can be constructed to be more useful to practitioners [3, 7, 10, 16, 19, 32, 38].”
BackgroundWise Fusion : Group Fairness Enhanced Rank Fusion
Cachel, K., & Rundensteiner, E. A. (2024) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3627673.3679649 · Cited by 2
Improving forecasting of concrete strength using advanced machine learning methods
Ellappan, P., Keshav, L., Raja, K. C. P., & Sanijya, G. (2025) · Matéria (Rio de Janeiro) · doi:10.1590/1517-7076-rmat-2024-0789 · Cited by 2
Designing Data Science Software for Social Care Organisations
Reinmund, T. (2023) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3544549.3577062 · Cited by 1
“Few studies evaluate how practitioners use these interventions collaboratively and within their day-to-day work [19].”
BackgroundXiong, M., Wang, Y., & Olya, H. (2025) · Emerald Group Publishing Limited eBooks · doi:10.1108/978-1-80262-963-720251009 · Cited by 1
Adaptive Machine Learning for Resource- Constrained Environments: A Path toward Sustainable AI
Shuvra, S. M. K., Gony, M. N., & Fatema, K. (2024) · International Journal of Research and Applied Innovations · doi:10.15662/ijrai.2024.0706012 · Cited by 1
Exploring demographic influences on fairness perceptions across the ML development lifecycle
Mishra, A., Khazanchi, D., & Tripathi, A. (2026) · AI and Ethics · doi:10.1007/s43681-026-01034-1 · Cited by 1
Jin, A., Asemota, A., Krane, D. E., Adams, N., & Abebe, R. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791552 · Cited by 1
Metadata
- Authors
- Deng, W. H., Nagireddy, M., Lee, M. S. A., Singh, J., Wu, Z. S., Holstein, K., & Zhu, H.
- Affiliations
- Carnegie Mellon University (US)
- University of Cambridge (GB)
- ORCID
- Venue
- 2022 ACM Conference on Fairness Accountability and Transparency
- Publisher
- ACM
- Published
- 2022-06-20
- Pages
- 473-484
- OpenAlex
- W4280609065
- Cited by
- 99
- References
- 118
- Citations by year
- 2026: 21 · 2025: 34 · 2024: 28 · 2023: 14 · 2022: 2
- FWCI
- 19.38
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- Apollo (University of Cambridge) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- Carnegie Mellon University Block Center for Technology and Society Award (No. 53680.1.5007718)
- Aviva and the UK Engineering and Physical Science Research Council (EP/R033501/1)
- Jacob Foundation for CERES network
- National Science Foundation (IIS2001851)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Keywords
- algorithmic fairness, open source software, tool design, responsible AI, FAIRness assessment, think-aloud interviews, industry professionals
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.