What We Can't Measure, We Can't Understand: Challenges to Demographic Data Procurement in the Pursuit of Fairness
Abstract
As calls for fair and unbiased algorithmic systems increase, so too does the number of individuals working on algorithmic fairness in industry. However, these practitioners often do not have access to the demographic data they feel they need to detect bias in practice. Even with the growing variety of toolkits and strategies for working towards algorithmic fairness, they almost invariably require access to demographic attributes or proxies. We investigated this dilemma through semi-structured interviews with 38 practitioners and professionals either working in or adjacent to algorithmic fairness. Participants painted a complex picture of what demographic data availability and use look like on the ground, ranging from not having access to personal data of any kind to being legally required to collect and use demographic data for discrimination assessments. In many domains, demographic data collection raises a host of difficult questions, including how to balance privacy and fairness, how to define relevant social categories, how to ensure meaningful consent, and whether it is appropriate for private companies to infer someone's demographics. Our research suggests challenges that must be considered by businesses, regulators, researchers, and community groups in order to enable practitioners to address algorithmic bias in practice. Critically, we do not propose that the overall goal of future work should be to simply lower the barriers to collecting demographic data. Rather, our study surfaces a swath of normative questions about how, when, and whether this data should be procured, and, in cases where it is not, what should still be done to mitigate bias.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Data on demographic groups may be unavailable to organizations or external auditors due to privacy legislation.
“However, data on these groups may be unavailable to organizations or external auditors due to privacy legislation.”
From Auditing a Dutch Public Sector Risk Profiling Algorithm Using an Unsupervised Bias Detection Tool (2025) · cites this paper as Andrus et al., 2021
Cites
Situated Knowledges: The Science Question in Feminism and the Privilege of Partial Perspective
Haraway, D. (1988) · Feminist Studies · doi:10.2307/3178066 · Cited by 18,364 · see citing works
Crenshaw, K. (2018) · Feminist Legal Theory · doi:10.4324/9780429500480-5 · Cited by 9,510 · see citing works
Crossing the Quality Chasm: A New Health System for the 21st Century
Committee on Quality of Health Care in America (2002) · Journal for Healthcare Quality · doi:10.1111/j.1945-1474.2002.tb00463.x · Cited by 8,508 · see citing works
Situated Knowledges: The Science Question in Feminism and the Privilege of Partial Perspective
Haraway, D. (1988) · PhilPapers (PhilPapers Foundation) · Cited by 6,992 · see citing works
Thematic networks: an analytic tool for qualitative research
Attride-Stirling, J. (2001) · Qualitative Research · doi:10.1177/146879410100100307 · Cited by 5,506 · see citing works
Barocas, S., & Selbst, A. D. (2014) · doi:10.2139/ssrn.2477899 · Cited by 3,048 · see citing works
Fairness and Abstraction in Sociotechnical Systems
Selbst, A. D., Boyd, D., Friedler, S. A., Venkatasubramanian, S., & Vertesi, J. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287598 · Cited by 1,631 · see citing works
What is data justice? The case for connecting digital rights and freedoms globally
Taylor, L. (2017) · Big Data & Society · doi:10.1177/2053951717736335 · Cited by 802 · see citing works
Improving Fairness in Machine Learning Systems: What Do Industry Practitioners Need?
Holstein, K., Wortman Vaughan, J., Daumé, H., Dudik, M., & Wallach, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300830 · Cited by 777 · see citing works
Fairness Constraints: Mechanisms for Fair Classification
Zafar, M. B., Valera, I., Rodriguez, M. G., & Gummadi, K. P. (2015) · arXiv · doi:10.48550/arxiv.1507.05259 · Cited by 645 · see citing works
Where fairness fails: data, algorithms, and the limits of antidiscrimination discourse
Hoffmann, A. L. (2019) · Information, Communication & Society · doi:10.1080/1369118x.2019.1573912 · Cited by 535 · see citing works
Veale, M., & Binns, R. (2017) · Big Data & Society · doi:10.1177/2053951717743530 · Cited by 512 · see citing works
The What-If Tool: Interactive Probing of Machine Learning Models
Wexler, J., Pushkarna, M., Bolukbasi, T., WATTENBERG, M. P., Viégas, F. B., & Wilson, J. (2019) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2019.2934619 · Cited by 512 · see citing works
Big Data from the South(s): Beyond Data Universalism
Milan, S., & Treré, E. (2019) · Television & New Media · doi:10.1177/1527476419837739 · Cited by 511 · see citing works
Veale, M., Van Kleek, M., & Binns, R. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174014 · Cited by 490 · cited 1× · see citing works
“There is a growing literature of work studying fairness practitioners themselves that has focused on the needs of public sector practitioners [82], the needs of private sector practitioners [41] and the organizational patterns they fit within [67].”
BackgroundFairness-aware Learning through Regularization Approach
Kamishima, T., Akaho, S., & Sakuma, J. (2011) · 2011 IEEE 11th International Conference on Data Mining Workshops · doi:10.1109/icdmw.2011.83 · Cited by 475 · see citing works
Consumer Financial Protection Bureau
Camacho, L. (2014) · Choice Reviews Online · doi:10.5860/choice.51-3000 · Cited by 333 · see citing works
Towards a critical race methodology in algorithmic fairness
Hanna, A., Denton, R., Smart, A., & Smith-Loud, J. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372826 · Cited by 295 · cited 3× · see citing works
“Work in this area has addressed race [13, 35], gender [34, 43, 72], and disability [11], calling into question what it means to even rely on these categories as a basis for assessing unfairness, and what harms are reproduced by relying on these infrastructures of categorization.”
BackgroundElliott, M. N., Morrison, P. A., Fremont, A., McCaffrey, D. F., Pantoja, P., & Lurie, N. (2009) · Health Services and Outcomes Research Methodology · doi:10.1007/s10742-009-0047-1 · Cited by 224 · see citing works
Gender Recognition or Gender Reductionism?
Hamidi, F., Scheuerman, M. K., & Branham, S. M. (2018) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173582 · Cited by 222 · see citing works
How Algorithms Discriminate Based on Data They Lack: Challenges, Solutions, and Policy Implications
Williams, B. A., Brooks, C., & Shmargad, Y. (2018) · Journal of Information Policy · doi:10.5325/jinfopoli.8.2018.0078 · Cited by 203 · see citing works
Scheuerman, M. K., Wade, K., Lustig, C., & Brubaker, J. R. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3392866 · Cited by 193 · see citing works
Democratizing Algorithmic Fairness
Wong, P.-H. (2020) · Philosophy & Technology · doi:10.1007/s13347-019-00355-w · Cited by 182 · see citing works
30th Conference on Neural Information Processing Systems (NIPS 2016)
Papamakarios, G., & Murray, I. (2016) · neural information processing systems · Cited by 164 · see citing works
Algorithmic realism: expanding the boundaries of algorithmic thought
Green, B., & Viljoen, S. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372840 · Cited by 161 · see citing works
D’Amour, A., Srinivasan, H., Atwood, J. R., Baljekar, P. N., Sculley, D., & Halpern, Y. (2020) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3351095.3372878 · Cited by 156 · see citing works
Toward situated interventions for algorithmic equity: lessons from the field
Katell, M., Young, M., Dailey, D., Herman, B., Guetler, V., Tam, A., Bintz, C., Raz, D., & Krafft, P. M. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372874 · Cited by 153 · see citing works
Fairness GAN: Generating datasets with fairness properties using a generative adversarial network
Sattigeri, P., Hoffman, S. C., Chenthamarakshan, V., & Varshney, K. R. (2019) · IBM Journal of Research and Development · doi:10.1147/jrd.2019.2945519 · Cited by 134 · see citing works
Regulating Artificial Intelligence: Proposal for a Global Solution
Erdélyi, O. J., & Goldsmith, J. (2018) · Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3278721.3278731 · Cited by 133 · see citing works
Žliobaitė, I., & Custers, B. (2016) · Artificial Intelligence and Law · doi:10.1007/s10506-016-9182-5 · Cited by 125 · see citing works
This Thing Called Fairness: Disciplinary Confusion Realizing a Value in Technology
Mulligan, D. K., Kroll, J. A., Kohli, N., & Wong, R. Y. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359221 · Cited by 111 · see citing works
Racial categories in machine learning
Benthall, S., & Haynes, B. D. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287575 · Cited by 110 · see citing works
A Relational Theory of Data Governance
Viljoen, S. (2020) · doi:10.2139/ssrn.3727562 · Cited by 109 · see citing works
(2020) · doi:10.1145/3386296.3386301 · Cited by 107 · see citing works
Elliott, M. N., Morrison, P. A., Fremont, A. M., McCaffrey, D. F., Pantoja, P. M., & Lurie, N. (2009) · Health Services and Outcomes Research Methodology · doi:10.1007/s10742-009-0055-1 · Cited by 95 · see citing works
Discrimination- and privacy-aware patterns
Hajian, S., Domingo-Ferrer, J., Monreale, A., Pedreschi, D., & Giannotti, F. (2015) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-014-0393-7 · Cited by 93 · see citing works
Interventions over Predictions: Reframing the Ethical Debate for Actuarial Risk Assessment
Barabas, C., Dinakar, K., Ito, J., Virza, M., & Zittrain, J. (2017) · arXiv · doi:10.48550/arxiv.1712.08238 · Cited by 87 · see citing works
Understanding Discrimination in the Scored SocietyPreprint
Zarsky, T. (2014) · SSRN Electronic Journal · Cited by 81 · see citing works
What's sex got to do with machine learning?
Hu, L., & Kohler-Hausmann, I. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3375674 · Cited by 74 · see citing works
Assessing Fair Lending Risks Using Race/Ethnicity Proxies
Zhang, Y. (2018) · Management Science · doi:10.1287/mnsc.2016.2579 · Cited by 71 · see citing works
Vasudevan, S., & Kenthapadi, K. (2020) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3340531.3412705 · Cited by 60 · see citing works
Is Algorithmic Affirmative Action Legal
Bent, J. R. (2020) · University of North Carolina School of Law Scholarship Repository (University of North Carolina Hospitals) · Cited by 58 · see citing works
Bogen, M., Rieke, A., & Ahmed, S. (2020) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3351095.3372877 · Cited by 49 · see citing works
Howell, J., & Emerson, M. O. (2017) · Sociology of Race and Ethnicity · doi:10.1177/2332649216648465 · Cited by 41 · see citing works
Sokol, K., Hepburn, A., Poyiadzi, R., Clifford, M., Santos‐Rodríguez, R., & Flach, P. A. (2020) · The Journal of Open Source Software · doi:10.21105/joss.01904 · Cited by 36 · see citing works
(2021) · eYLS (Yale Law School) · Cited by 27 · see citing works
Self-citationIssues in Open Data: Indigenous data sovereignty
Rainie, S., Kukutai, T., Walter, M., Figueroa-Rodriguez, O., Walker, J. D., & Axelsson, P. (2019) · DiVA at Umeå University (Umeå University) · doi:10.5281/zenodo.2677801 · Cited by 26 · see citing works
Maramwidze-Merrison, E. (2016) · The Electronic Journal of Business Research Methods · Cited by 24 · see citing works
Beyond Bias: Re-Imagining the Terms of ‘Ethical AI’ in Criminal LawPreprint
Barabas, C. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3377921 · Cited by 24 · see citing works
This Thing Called Fairness: Disciplinary Confusion Realizing a Value in TechnologyPreprint
Mulligan, D. K., Kroll, J. A., Kohli, N., & Wong, R. Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.11869 · Cited by 14 · see citing works
TransVis: Integrated Distant and Close Reading of Othello Translations
Alharbi, M., Laramee, R. S., & Cheesman, T. (2020) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2020.3012778 · Cited by 12 · see citing works
Sokol, K., Hepburn, A., Poyiadzi, R., Clifford, M., Santos‐Rodríguez, R., & Flach, P. A. (2020) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.3833199 · Cited by 11 · see citing works
Eliminating Latent Discrimination: Train Then Mask
Ghili, S., Kazemi, E. M., & Karbasi, A. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33013672 · Cited by 10 · see citing works
Stretching Human Laws to Apply to Machines: The Dangers of a ‘Colorblind’ Computer
Harned, Z., & Wallach, H. (2019) · Scholorship Repository - Florida State University College of Law (Florida State University) · Cited by 6 · see citing works
Towards a Just Theory of Measurement
Andrus, M., & Gilbert, T. K. (2019) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3306618.3314275 · Cited by 4 · see citing works
Self-citation
Flagged first, then by citations
3 of the 58 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Advances, challenges and opportunities in creating data for trustworthy AI
Liang, W., Tadesse, G. A., Ho, D. E., Fei-Fei, L., Zaharia, M., Zhang, C., & Zou, J. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00516-1 · Cited by 582 · see citing works
Moving beyond “algorithmic bias is a data problem”
Hooker, S. (2021) · Patterns · doi:10.1016/j.patter.2021.100241 · Cited by 181 · see citing works
Assessing the Fairness of AI Systems: AI Practitioners' Processes, Challenges, and Needs for Support
Madaio, M., Egede, L., Subramonyam, H., Wortman Vaughan, J., & Wallach, H. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3512899 · Cited by 156 · see citing works
Building Human Values into Recommender Systems: An Interdisciplinary Synthesis
Stray, J., Halevy, A. Y., Assar, P., Hadfield-Menell, D., Boutilier, C. E., Ashar, A., Bakalar, C., Beattie, L., Ekstrand, M. D., Leibowicz, C., et al. (2023) · ACM Transactions on Recommender Systems · doi:10.1145/3632297 · Cited by 145 · see citing works
Self-citationChallenges in Detoxifying Language Models
Welbl, J., Glaese, A., Uesato, J., Dathathri, S., Mellor, J., Hendricks, L. A., Anderson, K., Kohli, P., Coppin, B., & Huang, P.-S. (2021) · Findings of the Association for Computational Linguistics: EMNLP 2021 · doi:10.18653/v1/2021.findings-emnlp.210 · Cited by 112 · see citing works
Auditing for Discrimination in Algorithms Delivering Job Ads
Imana, B., Korolova, A., & Heidemann, J. (2021) · International World Wide Web Conference (WWW) · doi:10.1145/3442381.3450077 · Cited by 109 · see citing works
Exploring How Machine Learning Practitioners (Try To) Use Fairness Toolkits
Deng, W. H., Nagireddy, M., Lee, M. S. A., Singh, J., Wu, Z. S., Holstein, K., & Zhu, H. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533113 · Cited by 99 · see citing works
Exploring gender biases in ML and AI academic research through systematic literature review
Shrestha, S., & Das, S. (2022) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2022.976838 · Cited by 105 · see citing works
Conceptualising fairness: three pillars for medical algorithms and health equity
Sikstrom, L., Maslej, M. M., Hui, K., Findlay, Z., Buchman, D. Z., & Hill, S. L. (2022) · BMJ Health & Care Informatics · doi:10.1136/bmjhci-2021-100459 · Cited by 76 · see citing works
Designing Disaggregated Evaluations of AI Systems: Choices, Considerations, and Tradeoffs
Barocas, S., Guo, A., Kamar, E., Krones, J., Morris, M. R., Vaughan, J. W., Wadsworth, W. D., & Wallach, H. (2021) · Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3461702.3462610 · Cited by 72 · see citing works
van Bekkum, M., & Zuiderveen Borgesius, F. (2023) · Computer Law & Security Review · doi:10.1016/j.clsr.2022.105770 · Cited by 69 · see citing works
Estimating Example Difficulty using Variance of Gradients
Agarwal, C., D’souza, D., & Hooker, S. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01012 · Cited by 75 · see citing works
Evaluation Gaps in Machine Learning Practice
Hutchinson, B., Rostamzadeh, N., Greer, C., Heller, K., & Prabhakaran, V. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533233 · Cited by 75 · see citing works
Yee, K., Tantipongpipat, U., & Mishra, S. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479594 · Cited by 59 · see citing works
Fair ranking: a critical review, challenges, and future directions
Patro, G. K., Porcaro, L., Mitchell, L., Zhang, Q., Zehlike, M., & Garg, N. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533238 · Cited by 61 · see citing works
Can Workers Meaningfully Consent to Workplace Wellbeing Technologies?
Chowdhary, S., Kawakami, A., Gray, M. L., Suh, J., Olteanu, A., & Saha, K. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3594023 · Cited by 54 · see citing works
Measuring and Mitigating Racial Disparities in Tax Audits
Elzayn, H., Smith, E., Hertz, T., Guage, C., Ramesh, A., Fisher, R., Ho, D. E., & Goldin, J. S. (2024) · The Quarterly Journal of Economics · doi:10.1093/qje/qjae027 · Cited by 52 · see citing works
Model Positionality and Computational Reflexivity: Promoting Reflexivity in Data Science
Cambo, S. A., & Gergle, D. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501998 · Cited by 49 · see citing works
Envisioning Equitable Speech Technologies for Black Older Adults
Brewer, R. N., Harrington, C., & Heldreth, C. M. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3594005 · Cited by 43 · see citing works
Inherent Limitations of AI Fairness
Buyl, M., & Bie, T. D. (2022) · Communications of the ACM · doi:10.1145/3624700 · Cited by 44 · see citing works
Rismani, S., Shelby, R. M., Smart, A., Jatho, E. W., Kroll, J. A., Moon, A., & Rostamzadeh, N. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581407 · Cited by 38 · see citing works
Asthana, S., Im, J., Chen, Z., & Banovic, N. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642180 · Cited by 34 · see citing works
When Fair Ranking Meets Uncertain Inference
Ghosh, A., Dutt, R., & Wilson, C. (2021) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3404835.3462850 · Cited by 32 · see citing works
Learning about Responsible AI On-The-Job: Learning Pathways, Orientations, and Aspirations
Madaio, M., Kapania, S., Qadri, R., Wang, D., Zaldivar, A., Denton, E., & Wilcox, L. G. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3658988 · Cited by 31 · see citing works
What-is and How-to for Fairness in Machine Learning: A Survey, Reflection, and Perspective
Tang, Z., Zhang, J., & Zhang, K. (2023) · ACM Computing Surveys · doi:10.1145/3597199 · Cited by 33 · see citing works
Adomavicius, G., & Yang, M. (2022) · ACM Transactions on Management Information Systems · doi:10.1145/3519420 · Cited by 20 · see citing works
Beyond Skin Tone: A Multidimensional Measure of Apparent Skin Color
Thong, W. E., Joniak, P., & Xiang, A. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00452 · Cited by 26 · see citing works
Self-citationHow Might We Create Better Benchmarks for Speech Recognition?
Aksënova, A., van Esch, D., Flynn, J., & Golik, P. (2021) · Proceedings of the 1st Workshop on Benchmarking: Past, Present and Future · doi:10.18653/v1/2021.bppf-1.4 · Cited by 25 · see citing works
Privacy-Preserving Fair Learning of Support Vector Machine with Homomorphic Encryption
Park, S., Byun, J., & Lee, J. (2022) · International World Wide Web Conference (WWW) · doi:10.1145/3485447.3512252 · Cited by 25 · see citing works
Multi-dimensional Discrimination in Law and Machine Learning - A Comparative Overview
Roy, A. G., Horstmann, J., & Ntoutsi, E. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3593979 · Cited by 26 · see citing works
Overwriting Pretrained Bias with Finetuning Data
Wang, A., & Russakovsky, O. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00366 · Cited by 24 · see citing works
Fairness Without Demographic Data: A Survey of Approaches
Ashurst, C., & Weller, A. (2023) · Equity and Access in Algorithms, Mechanisms, and Optimization · doi:10.1145/3617694.3623234 · Cited by 21 · see citing works
Fairness Audit of Machine Learning Models with Confidential Computing
Park, S., Kim, S., & Lim, Y.-S. (2022) · International World Wide Web Conference (WWW) · doi:10.1145/3485447.3512244 · Cited by 21 · see citing works
Why, when, and from whom: considerations for collecting and reporting race and ethnicity data in HCI
Chen, Y. T., Smith, A. D. R., Reinecke, K., & To, A. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581122 · Cited by 22 · see citing works
Fair human-centric image dataset for ethical AI benchmarking
Xiang, A., Andrews, J. T. A., Bourke, R. L., Thong, W. E., LaChance, J., Georgievski, T., Modas, A., Rahmattalabbi, A., Ba, Y., Nagpal, S., et al. (2025) · Nature · doi:10.1038/s41586-025-09716-2 · Cited by 21 · see citing works
Self-citationImana, B., Korolova, A., & Heidemann, J. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579610 · Cited by 19 · see citing works
MinMax fairness: from Rawlsian Theory of Justice to solution for algorithmic bias
Barsotti, F., & Koçer, R. G. (2022) · AI & Society · doi:10.1007/s00146-022-01577-x · Cited by 18 · see citing works
The Impact of Artificial Intelligence on Human Resource Management: Opportunities and Challenges
Madanchian, M., & Taherdoost, H. (2024) · Lecture notes in networks and systems · doi:10.1007/978-3-031-54671-6_30 · Cited by 19 · see citing works
Demographic Bias in Low-Resolution Deep Face Recognition in the Wild
Atzori, A., Fenu, G., & Marras, M. (2023) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2023.3249485 · Cited by 17 · see citing works
We are the Data: Challenges and Opportunities for Creating Demographically Diverse Anthropographics
Dhawka, P., He, H. A., & Willett, W. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581086 · Cited by 17 · see citing works
The Privacy-Bias Tradeoff: Data Minimization and Racial Disparity Assessments in U.S. Government
King, J., Ho, D. E., Gupta, A., Wu, V. Y., & Webley-Brown, H. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3594015 · Cited by 17 · see citing works
Long-term Dynamics of Fairness Intervention in Connection Recommender Systems
Akpinar, N.-J., DiCiccio, C. J., Nandy, P., & Basu, K. (2022) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3514094.3534173 · Cited by 16 · see citing works
The AI field needs translational Ethical AI research
Borg, J. S. (2022) · AI Magazine · doi:10.1002/aaai.12062 · Cited by 15 · see citing works
Bennett, S., Claisse, C., Luger, E. A., & Durrant, A. C. (2023) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3600211.3604684 · Cited by 16 · see citing works
Madaio, M., Chen, J., Wallach, H., & Vaughan, J. W. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3653705 · Cited by 13 · see citing works
Lazy Data Practices Harm Fairness Research
Simson, J., Fabris, A., & Kern, C. A. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3658931 · Cited by 13 · see citing works
Auditing the Audits: Lessons for Algorithmic Accountability from Local Law 144's Bias Audits
Gerchick, M. K., Encarnación, R., Tanigawa-Lau, C., Armstrong, L., Gutiérrez, A., & Metaxa, D. (2025) · Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3715275.3732004 · Cited by 10 · see citing works
Disentangling and Operationalizing AI Fairness at LinkedIn
Candela, J. Q., Wu, Y., Hsu, B., Jain, S., Ramos, J., Adams, J., Hallman, R., & Basu, K. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3594075 · Cited by 11 · see citing works
Fair CRISP-DM: Embedding Fairness in Machine Learning (ML) Development Life Cycle
Singh, V. K., Singh, A., & Joshi, K. (2022) · Hawaii International Conference on System Sciences (HICSS) · doi:10.24251/hicss.2022.190 · Cited by 10 · see citing works
Subverting Fair Image Search with Generative Adversarial Perturbations
Ghosh, A., Jagielski, M., & Wilson, C. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533128 · Cited by 10 · see citing works
SoK: Technical Implementation and Human Impact of Internet Privacy Regulations
Birrell, E., Rodolitz, J., Ding, A. A., Lee, J., McReynolds, E., Hutson, J. A., & Lerner, A. (2024) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp54263.2024.00206 · Cited by 9 · see citing works
Mind the data gap(s): Investigating power in speech and language datasets
Markl, N. (2022) · Workshop on Language Technology for Equality, Diversity, Inclusion (LT-EDI) · doi:10.18653/v1/2022.ltedi-1.1 · Cited by 8 · see citing works
Fairness, explainability, privacy, and robustness for trustworthy algorithmic decision-making
Majumdar, S. (2023) · Elsevier eBooks · doi:10.1016/b978-0-323-85713-0.00017-7 · Cited by 8 · see citing works
Jarke, J., Prietl, B., Egbert, S., Boeva, Y., Heuer, H., & Arnold, M. (2024) · Amsterdam University Press eBooks · doi:10.1515/9789048556908 · Cited by 15 · see citing works
Algorithmic fairness datasets: the story so far
Fabris, A., Messina, S., Silvello, G., & Susto, G. A. (2022) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-022-00854-z · Cited by 128 · see citing works
Gebru, T., & Denton, E. (2024) · Foundations and Trends® in Computer Graphics and Vision · doi:10.1561/0600000102 · Cited by 7 · see citing works
Yew, R.-J., & Xiang, A. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533163 · Cited by 7 · see citing works
Self-citationMAG-BERT-ARL for Fair Automated Video Interview Assessment
Putra, B., Azizah, K., Mawalim, C. O., Hanif, I. A., Sakti, S., Leong, C. W., & Okada, S. (2024) · IEEE Access · doi:10.1109/access.2024.3473314 · Cited by 6 · see citing works
Pragmatic Fairness: Evaluating ML Fairness Within the Constraints of Industry
Smith, J. J., Madaio, M., Burke, R., & Fiesler, C. L. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732040 · Cited by 6 · see citing works
Estimating and Implementing Conventional Fairness Metrics With Probabilistic Protected Features
Elzayn, H., Black, E., Vossler, P., Jo, N., Goldin, J. S., & Ho, D. E. (2024) · IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) · doi:10.1109/satml59370.2024.00016 · Cited by 6 · see citing works
Addy, E., Adabor, E. S., & Badu, Y. A. (2025) · Industry and Higher Education · doi:10.1177/09504222251336297 · Cited by 5 · see citing works
Is there consistency in ethical sensitivity in artificial intelligence? A review of language models
Tutar, H., Bilan, S., & Şentürk, Ü. (2025) · Human Technology · doi:10.14254/1795-6889.2025.21-2.4 · Cited by 5 · see citing works
Group fairness without demographics using social networks
Liu, D., Do, V., Usunier, N., & Nickel, M. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3594091 · Cited by 6 · see citing works
When Fair Classification Meets Noisy Protected Attributes
Ghosh, A., Kvitca, P., & Wilson, C. (2023) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3600211.3604707 · Cited by 5 · see citing works
Evaluating the Social Impact of Generative AI Systems
Solaiman, I., Talat, Z., Agnew, W. S., Ahmad, L., Baker, D., Blodgett, S. L., Chen, C., Daumé, H., Dodge, J., Duan, I., et al. (2025) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780198940272.013.0025 · Cited by 5 · see citing works
Mirror, Mirror, on the Wall, Who's the Fairest of Them All?
Xiang, A. (2024) · Daedalus · doi:10.1162/daed_a_02058 · Cited by 4 · see citing works
Self-citationAuditing for Bias in Ad Delivery Using Inferred Demographic Attributes
Imana, B., Korolova, A., & Heidemann, J. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732172 · Cited by 4 · see citing works
Surrogate Membership for Inferred Metrics in Fairness Evaluation
Thielbar, M., Kadıoğlu, S., Zhang, C., Pack, R., & Dannull, L. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-44505-7_29 · Cited by 4 · see citing works
Requirements on and Procurement of Explainable Algorithms—A Systematic Review of the Literature
Franke, U., Hallström, C. H., Artman, H., & Dexe, J. (2024) · Advances in intelligent systems and computing · doi:10.1007/978-3-031-66635-3_4 · Cited by 4 · see citing works
Goldshtein, M., Chiou, E. K., & Roscoe, R. D. (2024) · Societies · doi:10.3390/soc14070105 · Cited by 5 · see citing works
Surrogate Modeling to Address the Absence of Protected Membership Attributes in Fairness Evaluation
Kadıoğlu, S., & Thielbar, M. (2024) · ACM Transactions on Evolutionary Learning and Optimization · doi:10.1145/3700145 · Cited by 3 · see citing works
Fairness analysis of machine learning predictions of aggression in acute psychiatric care
Wang, Y., Sikstrom, L., Xiao, R., Findlay, Z., Zaheer, J., Hill, S., & Maslej, M. M. (2026) · npj Mental Health Research · doi:10.1038/s44184-026-00194-6 · Cited by 4 · see citing works
Dhawka, P., & Dasgupta, S. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713243 · Cited by 3 · see citing works
Harnessing digital health data for suicide prevention and care: A rapid review
Bennett‐Poynter, L., Kundurthi, S., Besa, R., Joyce, D. W., Kormilitzin, A., Shen, N., Sunwoo, J., Szkudlarek, P., Sequiera, L., & Sikstrom, L. (2025) · Digital Health · doi:10.1177/20552076241308615 · Cited by 3 · see citing works
Simplicity Bias Leads to Amplified Performance Disparities
Bell, S. J., & Sagun, L. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3594003 · Cited by 3 · see citing works
An Anti-Subordination Approach to Fair ClassificationPreprint
Keswani, V., & Celis, L. E. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4116835 · Cited by 3 · see citing works
Lin, Q., Ouyang, D., Guage, C., Gallegos, I. O., Goldin, J. S., & Ho, D. E. (2025) · Scientific Data · doi:10.1038/s41597-025-04753-y · Cited by 3 · see citing works
Improving Group Fairness Assessments with Proxies
Harvey, E., Lee, M. S. A., & Singh, J. P. (2024) · ACM Journal on Responsible Computing · doi:10.1145/3677175 · Cited by 2 · see citing works
Positive act of reporting negative results in large language model research: a call for transparency
Tripathi, S., Alkhulaifat, D., & Cook, T. S. (2026) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaf221 · Cited by 2 · see citing works
Methods for Learning to Quantify
Esuli, A., Fabris, A., Moreo, A., & Sebastiani, F. (2023) · The information retrieval series · doi:10.1007/978-3-031-20467-8_4 · Cited by 2 · see citing works
DomainDemo: a dataset of domain-sharing activities among different demographic groups on Twitter
Yang, K.-C., Goel, P., Quintana, A., Horgan, L., McCabe, S., Grinberg, N., Joseph, K., & Lazer, D. M. J. (2025) · Scientific Data · doi:10.1038/s41597-025-05604-6 · Cited by 2 · see citing works
Talking About the Assumption in the Room
Mothilal, R. K., Lalani, F. M., Ahmed, S. I., Guha, S., & Sultana, S. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713958 · Cited by 1 · see citing works
A Comprehensive Review of Real-Time Multi-View Multi-Person Markerless Motion Capture
Nagorny, P., Kevelham, B., Chagué, S., & Charbonnier, C. (2025) · ACM Computing Surveys · doi:10.1145/3757733 · Cited by 1 · see citing works
Human, Algorithm, or Both? Gender Bias in Human-Augmented Recruiting
Kaya, M., & Bogers, T. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3806756 · Cited by 3 · see citing works
LaVan, H., & Martin, M. (2023) · Employee Responsibilities and Rights Journal · doi:10.1007/s10672-023-09437-6 · Cited by 1 · see citing works
Esuli, A., Fabris, A., Moreo, A., & Sebastiani, F. (2023) · The information retrieval series · doi:10.1007/978-3-031-20467-8_1 · Cited by 1 · see citing works
A Method to Assess and Explain Disparate Impact in Online Retailing
Arreola, R. B. (2023) · International World Wide Web Conference (WWW) · doi:10.1145/3543507.3583270 · Cited by 1 · see citing works
15. Making Algorithms Fair : Ethnographic Insights from Machine Learning Interventions
Kinder-Kurlanda, K. E., & Fahimi, M. (2024) · Amsterdam University Press eBooks · doi:10.1515/9789048556908-015 · Cited by 1 · see citing works
Nicho, M., Hamed, A., Gaber, T., & Arimi, J. H. A. (2025) · Cogent Social Sciences · doi:10.1080/23311886.2025.2527392 · Cited by 1 · see citing works
Lenhart, A. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4178631 · Cited by 1 · see citing works
Assessing Algorithmic Fairness without Sensitive Information
Noiret, S. (2021) · International Conference on Information Technology for Social Good (GoodIT) · doi:10.1145/3462203.3475894
Evaluation of Quantification Algorithms
Esuli, A., Fabris, A., Moreo, A., & Sebastiani, F. (2023) · The information retrieval series · doi:10.1007/978-3-031-20467-8_3
Esuli, A., Fabris, A., Moreo, A., & Sebastiani, F. (2023) · The information retrieval series · doi:10.1007/978-3-031-20467-8_5
Esuli, A., Fabris, A., Moreo, A., & Sebastiani, F. (2023) · The information retrieval series · doi:10.1007/978-3-031-20467-8_7
Esuli, A., Fabris, A., Moreo, A., & Sebastiani, F. (2023) · The information retrieval series · doi:10.1007/978-3-031-20467-8_6
Applications of Quantification
Esuli, A., Fabris, A., Moreo, A., & Sebastiani, F. (2023) · The information retrieval series · doi:10.1007/978-3-031-20467-8_2
Technical and Socio-Legal Aspects of Smart Governance in the Context of Indian Bureaucracy
Sharma, S., Bali, S., Verma, V., & Bali, P. (2025) · IGI Global eBooks · doi:10.4018/979-8-3373-0817-3.ch020
Bias Delayed is Bias Denied? Assessing the Effect of Reporting Delays on Disparity Assessments
Gosciak, J., Balagopalan, A., Ouyang, D., Koenecke, A., Ghassemi, M., & Ho, D. E. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732123
The Collaborative Interplay Between Data Science and UX Research
Sankar, K. P. N., Arjunwadkar, H., Gajaralwar, T., & Sinha, A. K. (2025) · Lecture notes in mechanical engineering · doi:10.1007/978-981-96-5503-8_12
Open‐source AI at scale: Establishing an enterprise AI strategy through modular frameworks
Kadıoğlu, S. (2025) · AI Magazine · doi:10.1002/aaai.70032
- Authors
- Andrus, M., Spitzer, E., Brown, J., & Xiang, A.
- Affiliations
- Partnership on AI
- Minnesota State University, Mankato (US)
- Sony AI, Partnership on AI
- Venue
- Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency
- Publisher
- ACM
- Published
- 2021-03
- Pages
- 249-260
- OpenAlex
- W3135773605
- Cited by
- 111
- References
- 58
- Citations by year
- 2026: 13 · 2025: 19 · 2024: 18 · 2023: 30 · 2022: 23 · 2021: 8
- FWCI
- 17.86
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion)
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Business, Management and Accounting › Organizational Behavior and Human Resource Management › AI and HR Technologies
- Keywords
- algorithmic fairness, algorithmic bias, demographic data, data availability, privacy, consent, social categories, bias mitigation, practitioner perspectives, data collection ethics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
