50 Years of Test (Un)fairness: Lessons for Machine Learning
The DOI matches its Crossref record; 71 of the 71 references it cites were found in a registry or an index.
Abstract
Quantitative definitions of what is unfair and what is fair have been introduced in multiple disciplines for well over 50 years, including in education, hiring, and machine learning. We trace how the notion of fairness has been defined within the testing communities of education and hiring over the past half century, exploring the cultural and social context in which different fairness definitions have emerged. In some cases, earlier definitions of fairness are similar or identical to definitions of fairness in current machine learning research, and foreshadow current formal work. In other cases, insights into what fairness means and how to measure it have largely gone overlooked. We compare past and current notions of fairness along several dimensions, including the fairness criteria, the focus of the criteria (e.g., a test, a model, or its use), the relationship of fairness to individuals, groups, and subgroups, and the mathematical method for measuring fairness (e.g., classification, regression). This work points the way towards future research and measurement of (un)fairness that builds from our modern understanding of fairness while incorporating insights from the past.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“These toolkits have been criticised for offering a 'reductionist understanding of fairness as mathematical conditions' [71, p. 1], and reflect a longer history of attempts to reduce (un)fairness to a metric [72].”
From Putting AI ethics to work: are the tools fit for purpose? (2022) · cites this paper as [72]
In the citing paper
“Speciically, there is a rich set of fairness-related work in a variety of disciplines, often with concepts that are similar or equal to current ML fairness research [147].”
From Fairness in machine learning: A survey (2024) · cites this paper as [147]
Citations
Cites
Advances in Neural Information Processing Systems 19
(2007) · The MIT Press eBooks · doi:10.52202/085713 · Cited by 15,539
Proceedings of the 25th international conference on Machine learning - ICML '08
(2008) · doi:10.1145/1390156 · Cited by 6,680
(2018) · Cited by 3,510
Semi-supervised learning using Gaussian fields and harmonic functions
Zhu, X., Zoubin Ghahramani, A., & Lafferty, J. (2003) · Cited by 3,509
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012) · Proceedings of the 3rd Innovations in Theoretical Computer Science Conference · doi:10.1145/2090236.2090255 · Cited by 3,501 · cited 1×
“…A ⊥ R ; it is equivalent when A and R have a bivariate Gaussian • Guion’s definition “people with equal probabilities of success is a special case of Dwork’s [23] individual fairness with the relationship between Y and R (since an ability being measured affects the for the optimal score R ∗”
BackgroundDoyle, T. (2017) · The Information Society · doi:10.1080/01972243.2017.1354593 · Cited by 3,071
Equality of Opportunity in Supervised Learning
Hardt, M., Price, E., & Srebro, N. (2016) · arXiv · doi:10.48550/arxiv.1610.02413 · Cited by 2,536 · cited 5×
“bility of success on the job” is a construct that can be meaningfully reasoned about. The fairness literature in both the fields of ML and in testing have also been motivated by causal considerations [32, 41]. Darlington [19] motivate his definition (3) on the basis of a causal relationship between Y and R (since an a…”
BackgroundResultInfluentialFair prediction with disparate impact: a study of bias in recidivism prediction instruments
Chouldechova, A. (2017) · Big Data · doi:10.1089/big.2016.0047 · Cited by 2,159 · cited 3×
“uality of opportunity [32]. That same year, Linn [43] introduced (but did not advocate for) equal Positive Predictive Value (PPV) as a fairness criterion, recognizable as modern day predictive parity [9].4 Under Cleary and Darlington’s conceptions, bias or (un)fairness is a property of the test itself. This is contrar…”
BackgroundResultMitigating Unwanted Biases with Adversarial Learning
Zhang, B. H., Lemoine, B., & Mitchell, M. (2018) · Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3278721.3278779 · Cited by 1,387 · cited 1×
“or bias in model features. Instead, one approach adopted in ML fairness has been the use of adversarial methods to mitigate the effects of features with undesirable correlations with subgroups, e.g., [6, 69]. 3.4 Model vs. Model Use Section 2 described how the test literature had competing notions of whether fairness…”
MethodInfluentialSelf-citationAlgorithmic Decision Making and the Cost of Fairness
Corbett-Davies, S., Pierson, E., Feller, A., Goel, S., & Huq, A. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098095 · Cited by 1,149
Research works printed but not linked: 11
- Beutel Alex, 2017, Chi
- Buolamwini Joy, 2018, Conference on Fairness, Accountability and Transparency. 77--91
- James, 2018, Foulds and Shimei Pan
- Friedler Sorelle A, 2016, On the (im) possibility of fairness. arXiv preprint arXiv: 1609.07236 · arXiv
- Hoffmann Anna Lauren, Spaces for the Future
- Jencks Christopher, 1998, Racial bias in testing. The Black-White test score gap 55
- 10.3102/00346543043002139 · doi:10.3102/00346543043002139 · Paper page
- National Council on Measurement in Education NCME (Ed.). 1976. Journal of Education Measurement. 13 1 (1976). National Council on Measurement in Education NCME (Ed.). 1976. Journal of Education Measurement. 13 1 (1976).
- Phillips S E, Fairness in Educational Assessment and Measurement
- Rice Mitchell F, 1994, Race Norming, Validity Generalization, and Employment Testing, Handbook of Public Personnel Administration, 58, 451
- Zwick Rebecca, Fairness in Educational Assessment and Measurement
Cited by
A Survey on Bias and Fairness in Machine Learning
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2022) · ACM Computing Surveys · doi:10.1145/3457607 · Cited by 5,812
“rvey on Bias and Fairness in Machine Learning 11 4.2 Definitions of Fairness In [16], authors studied fairness definitions in political philosophy and tried to tie them to machinelearning. Authors in [60] studied the 50-year history of fairness definitions in the areas of education and machine-learning. In [122], auth…”
BackgroundInfluentialA Survey on Bias and Fairness in Machine Learning
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2019) · arXiv · doi:10.48550/arxiv.1908.09635 · Cited by 5,812
Mitigating bias in algorithmic hiring: evaluating claims and practices
Raghavan, M., Barocas, S., Kleinberg, J., & Levy, K. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372828 · Cited by 845
“This has led to a decades-long line of literature seeking to measure and mitigate the observed disparities (see [50] for a survey).”
BackgroundBaker, R. S., & Hawn, A. (2022) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-021-00285-9 · Cited by 993
“In historical perspective, Hutchinson and Mitchell (2019) point out that many formalized definitions of fairness have precedents from the 1960s and 1970s as researchers worked to minimize bias in standardized testing.”
BackgroundAlgorithmic Fairness: Choices, Assumptions, and Definitions
Mitchell, S., Potash, E., Barocas, S., D'Amour, A., & Lum, K. (2021) · Annual Review of Statistics and Its Application · doi:10.1146/annurev-statistics-042720-125902 · Cited by 687
Prediction-Based Decisions and Fairness: A Catalogue of Choices, Assumptions, and Definitions
Mitchell, S., Potash, E., Barocas, S., D'Amour, A., & Lum, K. (2018) · arXiv · doi:10.48550/arxiv.1811.07867 · Cited by 687
Fairness in machine learning: A survey
Caton, S., & Haas, C. (2024) · ACM Computing Surveys · doi:10.1145/3616865 · Cited by 646
Ai for social good: unlocking the opportunity for positive impact
Tomašev, N., Cornebise, J., Hutter, F., Mohamed, S., Picciariello, A., Connelly, B., Belgrave, D. C. M., Ezer, D., Haert, F. C. V. D., Mugisha, F., et al. (2020) · Nature Communications · doi:10.1038/s41467-020-15871-z · Cited by 357
“Fairness in particular should be explicitly accounted for, to avoid reinforcing existing societal biases re fl ected in the data used for model development 45 – 47 .”
MethodFairness perceptions of algorithmic decision-making: A systematic review of the empirical literature
Starke, C., Baleis, J., Keller, B., & Marcinkowski, F. (2022) · Big Data & Society · doi:10.1177/20539517221115189 · Cited by 316
“By reviewing more than 20 different formal definitions, Verma and Rubin (2018) found that formal concepts can largely be clustered into three categories (for an elaborate historical discussion of fairness definitions, see Hutchinson and Mitchell, 2019).”
BackgroundOn the apparent conflict between individual and group fairness
Binns, R. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372864 · Cited by 312
“p fairness measure. 1Formalised definitions of fairness which foreshadow those in Fair-ML did appear in areas other than ML, such as testing in education and hiring. See Hutchinson & Mitchell 2019 [30]. On the Apparent Conflict Between Individual and Group Fairness FAT* ’20, January 27–30, 2020, Barcelona, Spain 2.…”
BackgroundInfluentialInherent Tradeoffs in the Fair Determination of Risk Scores
(2023) · The Societal Impacts of Algorithmic Decision-Making · doi:10.1145/3603195.3603197 · Cited by 310
Algorithmic fairness in education
Kizilcec, R. F., & Lee, H. (2022) · The Ethics of Artificial Intelligence in Education · doi:10.4324/9780429329067-10 · Cited by 193
Evaluating Recommender Systems: Survey and Framework
Zangerle, E., & Bauer, C. (2023) · ACM Computing Surveys · doi:10.1145/3556536 · Cited by 283
“One way to address group fairness from the user perspective is to disaggregate the user-oriented metrics to measure and compare to which extent user groups are provided with lower-quality recommendations (e.g., References [69, 73, 74, 108, 142, 158, 196]).”
BackgroundAI in education: learner choice and fundamental rights
Berendt, B., Littlejohn, A., & Blakemore, M. (2020) · Learning, Media and Technology · doi:10.1080/17439884.2020.1786399 · Cited by 281
Algorithmic bias: Senses, sources, solutions
Fazelpour, S., & Danks, D. (2021) · Philosophy Compass · doi:10.1111/phc3.12760 · Cited by 272
Can HR adapt to the paradoxes of artificial intelligence?
Charlwood, A., & Guenole, N. (2022) · Human Resource Management Journal · doi:10.1111/1748-8583.12433 · Cited by 271
“Until recently, the computer science community has not had a method for differentiating these two possibilities, although recent work has started link the social science and computer science fields on this issue (Hutchinson & Mitchell, 2019).”
MethodDemographic Bias in Biometrics: A Survey on an Emerging Challenge
Drozdowski, P., Rathgeb, C., Dantcheva, A., Damer, N., & Busch, C. (2020) · IEEE Transactions on Technology and Society · doi:10.1109/tts.2020.2992344 · Cited by 263
“In fact, dozens of formal definitions (see [164], [165]) have been proposed to address different situations and possible criteria of fairness.”
BackgroundTrustworthy ai: A computational perspective
Liu, H., Wang, Y., Fan, W., Liu, X., Li, Y., Jain, S., Liu, Y., Jain, A., & Tang, J. (2023) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3546872 · Cited by 261
“In one survey [183], the authors trace the evolution of the notions and measurements of fairness in diferent ields, such as education and hiring, over the past 50 years.”
BackgroundYang, K., Qinami, K., Fei-Fei, L., Deng, J., & Russakovsky, O. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3375709 · Cited by 254
“A great overview of the history of machine learning fairness can be found in Hutchinson and Mitchell [38].”
BackgroundPutting AI ethics to work: are the tools fit for purpose?
Ayling, J., & Chapman, A. (2022) · AI and Ethics · doi:10.1007/s43681-021-00084-x · Cited by 253
“1], and reflect a longer history of attempts to reduce (un)fairness to a metric [72].”
BackgroundBias Mitigation for Machine Learning Classifiers: A Comprehensive Survey
Hort, M., Chen, Z., Zhang, J. M., Harman, M., & Sarro, F. (2024) · ACM Journal on Responsible Computing · doi:10.1145/3631326 · Cited by 219
Bias Mitigation for Machine Learning Classifiers: A Comprehensive Survey
Hort, M., Chen, Z., Zhang, J. M., Harman, M., & Sarro, F. (2022) · arXiv · doi:10.48550/arxiv.2207.07068 · Cited by 222
Re-imagining Algorithmic Fairness in India and Beyond
Sambasivan, N., Arnesen, E., Hutchinson, B., Doshi, T., & Prabhakaran, V. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445896 · Cited by 228
Self-citationED-AI Lit: An Interdisciplinary Framework for AI Literacy in Education
Allen, L. K., & Kendeou, P. (2024) · Policy Insights from the Behavioral and Brain Sciences · doi:10.1177/23727322231220339 · Cited by 168
“These perspectives underscore the critical nature of AI ethics education, guiding students to comprehend the ethical intricacies of AI technology and recognize the consequences of biased algorithms on diverse social groups (Bender & Friedman, 2018; Hutchinson & Mitchell, 2019).”
BackgroundMethodAn empirical characterization of fair machine learning for clinical risk prediction
Pfohl, S. R., Foryciarz, A., & Shah, N. H. (2021) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2020.103621 · Cited by 164
“Shown, for each group, is the number of patients extracted and the incidence of hospital mortality, prolonged length of stay, and 30-day readmission Outcome Incidence Group Count Hospital Mortality Prolonged Length of Stay 30-Day Readmission [18-30) 23,042 0.”
BackgroundInfluentialThe ethical implications of using generative chatbots in higher education
Williams, R. T. (2024) · Frontiers in Education · doi:10.3389/feduc.2023.1331607 · Cited by 183
Marcinkowski, F., Kieslich, K., Starke, C., & Lünich, M. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372867 · Cited by 159
“While technical definitions and formal proof of fairness are widely acknowledged within the machine learning community, it is time to focus on individual perceptions [22].”
BackgroundFairness, Equality, and Power in Algorithmic Decision-Making
Kasy, M., & Abebe, R. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445919 · Cited by 144
“Fairness Definitions Numerous definitions of fairness have been proposed in the literature [23, 37].”
BackgroundFeng, S., Park, C. Y., Liu, Y., & Tsvetkov, Y. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.656 · Cited by 149
Measuring algorithmically infused societies
Wagner, C., Strohmaier, M., Olteanu, A., Kıcıman, E., Contractor, N., & Eliassi-Rad, T. (2021) · Nature · doi:10.1038/s41586-021-03666-1 · Cited by 144
Financial Risk Management and Explainable, Trustworthy, Responsible AI
Fritz-Morgenthal, S., Hein, B., & Papenbrock, J. (2022) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2022.779799 · Cited by 147
“An example for using interpretable machine learning techniques is a use case developed by Munich Re Markets, published in Jaeger et al. (2021).”
MethodThe Landscape and Gaps in Open Source Fairness Toolkits
Lee, M. S. A., & Singh, J. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445261 · Cited by 129
“The debates on fairness in systematic and algorithmic decisionmaking has long predated the prevalence of ML [19].”
BackgroundFairness in Machine Learning: A Survey
Caton, S., & Haas, C. (2020) · arXiv · doi:10.48550/arxiv.2010.04053 · Cited by 646
Human Judgment in algorithmic loops: Individual justice and automated decision‐making
Binns, R. (2022) · Regulation & Governance · doi:10.1111/rego.12358 · Cited by 134
Rethinking Fairness: An Interdisciplinary Survey of Critiques of Hegemonic ML Fairness Approaches
Weinberg, L. (2022) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.13196 · Cited by 127
“Hutchinson and Mitchell (2019) describe how 1960s and 1970s debates from the use of assessment tests in public and private industry “stimulated a wealth of research into how to mathematically measure unfair bias and discrimination within the educational and employment testing communities, often…”
BackgroundInfluentialUnderstanding and Mitigating Annotation Bias in Facial Expression Recognition
Chen, Y., & Joo, J. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01471 · Cited by 160
“Fairness generally means that the model is not discriminatory with respect to some protected attribute, such as race, color, religion, sex, or national origin [33].”
BackgroundFairness and Bias in Algorithmic Hiring: A Multidisciplinary Survey
Fabris, A., Baranowska, N., Dennis, M. J., Graus, D., Hacker, P., Saldivar, J., Zuiderveen Borgesius, F., & Biega, A. J. (2025) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3696457 · Cited by 117
Policy advice and best practices on bias and fairness in AI
Álvarez, J. M., Colmenarejo, A. B., Elobaid, A., Fabbrizzi, S., Fahimi, M., Ferrara, A., Ghodsi, S., Mougan, C., Papageorgiou, I. T., Lobo, P. R., et al. (2024) · Ethics and Information Technology · doi:10.1007/s10676-024-09746-w · Cited by 163
“…frameworks for the elaboration of documentation include: Datasheets for Datasets (Gebru et al., 2021), Dataset Nutrition Labels (Chmielinski et al., 2022), Data Statements (Bender & Friedman, 2018), Data Readiness Report (Afzal et al., 2021), and Model Cards for Models (Mitchell et al., 2019).”
InfluentialVaccaro, K., Sandvig, C., & Karahalios, K. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3415238 · Cited by 102
Fairness Testing: A Comprehensive Survey and Analysis of Trends
Chen, Z., Zhang, J. M., Hort, M., Harman, M., & Sarro, F. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3652155 · Cited by 204
Balayn, A., Lofi, C., & Houben, G.-J. (2021) · The VLDB Journal · doi:10.1007/s00778-021-00671-8 · Cited by 98
The Problem of Fairness in Synthetic Healthcare Data
Bhanot, K., Qi, M., Erickson, J. S., Guyon, I., & Bennett, K. P. (2021) · Entropy · doi:10.3390/e23091165 · Cited by 91
Break the Loop: Gender Imbalance in Music Recommenders
Ferraro, A., Serra, X., & Bauer, C. (2021) · Proceedings of the 2021 Conference on Human Information Interaction and Retrieval · doi:10.1145/3406522.3446033 · Cited by 92
“There is a wealth of research on what is fair or unfair (for an overview, see Hutchinson and Mitchell [25]).”
BackgroundUnderstanding and Evaluating Racial Biases in Image Captioning
Zhao, D., Wang, A., & Russakovsky, O. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01456 · Cited by 83
A Human-Centered Systematic Literature Review of Cyberbullying Detection Algorithms
Kim, S., Razi, A., Stringhini, G., Wisniewski, P. J., & De Choudhury, M. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3476066 · Cited by 79
“Potential bias may originate in the dataset – whether its source, filtering and curation strategy used by the researchers, or the manner in which ground truth data is annotated [75].”
BackgroundInforming Age-Appropriate AI: Examining Principles and Practices of AI for Children
Wang, G., Zhao, J., Van Kleek, M., & Shadbolt, N. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3502057 · Cited by 76
Crowdsourcing Perceptions of Fair Predictors for Machine Learning: A Recidivism Case Study
van Berkel, N., Goncalves, J., Hettiachchi, D., Wijenayake, S., Kelly, R. M., & Kostakos, V. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359130 · Cited by 60
Czarnowska, P., Vyas, Y., & Shah, K. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00425 · Cited by 72
“Surveys of Mehrabi et al. (2019), Hutchinson and Mitchell (2019) and Chouldechova and Roth (2020) cover a broad scope of literature on algorithmic fairness.”
BackgroundLee, M. S. A., Floridi, L., & Singh, J. (2021) · AI and Ethics · doi:10.1007/s43681-021-00067-y · Cited by 71
“Hutchinson and Mitchell [30] warn of the gap between the unambiguous formalisation of fairness metrics and the contextual and practical needs of society, politics, and law.”
Trustworthy Distributed AI Systems: Robustness, Privacy, and Governance
Wei, W., & Liu, L. (2025) · ACM Computing Surveys · doi:10.1145/3645102 · Cited by 128
Towards responsible media recommendation
Elahi, M., Jannach, D., Skjærven, L., Knudsen, E., Sjøvaag, H., Tolonen, K., Holmstad, Ø., Pipkin, I., Throndsen, E., Stenbom, A., et al. (2022) · AI and Ethics · doi:10.1007/s43681-021-00107-7 · Cited by 60
“Research on fairness and unfairness can be traced back to well over 50 years [44], and it has received renewed attention in the most recent years, in particular also in the areas of machine learning or artificial intelligence in general.”
BackgroundBias and Fairness in Multimodal Machine Learning: A Case Study of Automated Video Interviews
Booth, B. M., Hickman, L., Subburaj, S. K., Tay, L., Woo, S. E., & D'Mello, S. K. (2021) · Proceedings of the 2021 International Conference on Multimodal Interaction · doi:10.1145/3462244.3479897 · Cited by 63
“Readersarereferredto[29,36,50]formoreinformationonother biasandfairnessmetrics.”
BackgroundInfluentialPan, C. A., Yakhmi, S., Iyer, T. P., Strasnick, E., Zhang, A. X., & Bernstein, M. S. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3512929 · Cited by 66
Evaluation Gaps in Machine Learning Practice
Hutchinson, B., Rostamzadeh, N., Greer, C., Heller, K., & Prabhakaran, V. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533233 · Cited by 62
Self-citationMitigating Bias in Algorithmic Hiring: Evaluating Claims and Practices
(2023) · The Societal Impacts of Algorithmic Decision-Making · doi:10.1145/3603195.3603203 · Cited by 65
Epistemic injustice and data science technologies
Symons, J., & Alvarado, R. (2022) · Synthese · doi:10.1007/s11229-022-03631-z · Cited by 65
“…the distribution of false negatives across subgroups in the data set did not achieve group parity, a measure of cultural fairness that seeks to equalize accuracy metrics across groups considering features such as race (Corbett-Davies, 2018; Hutchinson & Mitchell, 2019; Saxena et al., 2019).”
BackgroundFairness in Music Recommender Systems: A Stakeholder-Centered Mini Review
Dinnissen, K., & Bauer, C. (2022) · Frontiers in Big Data · doi:10.3389/fdata.2022.913608 · Cited by 69
“Overall, there is an increasing interest in research on fairness in ML in general (Hutchinson and Mitchell, 2019), and in RSs in particular (Ekstrand et al., 2019).”
BackgroundDeep Fair Clustering for Visual Learning
Li, P., Zhao, H., & Liu, H. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00909 · Cited by 84
“With the prevalence of machine learning based decision support systems in high-stakes domains, including college admission, loan approval, bail/parole judgments, and hiring processes, it has already become an imperative object of study to guarantee that individuals are treated equally in such automated decision proces…”
BackgroundKasinidou, M., Kleanthous, S., Barlas, P., & Otterbacher, J. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445931 · Cited by 57
“With regards to algorithmic fairness in decision-making, recent work in HCI and FAccT has looked into perceived fairness, only to find that people understand fairness differently and according to the context [26] where the system is operating.”
BackgroundBias, Fairness and Accountability with Artificial Intelligence and Machine Learning Algorithms
Zhou, N., Zhang, Z., Nair, V. N., Singhal, H., & Chen, J. (2022) · International Statistical Review · doi:10.1111/insr.12492 · Cited by 99
Yalçın, E., & Bilge, A. (2022) · Information Processing & Management · doi:10.1016/j.ipm.2022.103100 · Cited by 68
Algorithmic Fairness in AI: An Interdisciplinary View
Pfeiffer, J., Gutschow, J., Haas, C. S., Möslein, F., Maspfuhl, O., Borgers, F., & Alpsancar, S. (2023) · Business & Information Systems Engineering · doi:10.1007/s12599-023-00787-x · Cited by 97
“Specifically, the years after the introduction of Title VI and Title VII laws (prohibiting discrimination in employment) saw the emergence of fundamental concepts and definitions of fairness, many of which are still used today (Hutchinson and Mitchell 2019).”
BackgroundOlteanu, A., Garcia-Gathright, J. I., de Rijke, M., Ekstrand, M. D., Roegiest, A., Lipani, A., Beutel, A., Olteanu, A., Lučić, A., Stoica, A.-A., et al. (2019) · ACM SIGIR Forum · doi:10.1145/3458553.3458556 · Cited by 97
“As with other AI-driven technologies, IR systems are also under the influence of those that design, build, maintain or use them, embedding and amplifying their biases (Barocas and Selbst, 2016; Olteanu et al., 2019a; Baeza-Yates, 2018; Hutchinson and Mitchell, 2019).”
BackgroundUnderstanding Inequalities in Ride-Hailing Services Through Simulations
Bokányi, E., & Hannák, A. (2020) · Scientific Reports · doi:10.1038/s41598-020-63171-9 · Cited by 85
Towards Explainability for AI Fairness
Zhou, J., Chen, F., & Holzinger, A. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-04083-2_18 · Cited by 43
Mitigating Bias in Algorithmic Employment Screening: Evaluating Claims and Practices
Raghavan, M., Barocas, S., Kleinberg, J., & Levy, K. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3408010 · Cited by 51
Laufer, B., Jain, S., Cooper, A. F., Kleinberg, J. M., & Heidari, H. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533107 · Cited by 89
Agarwal, A., Agarwal, H., & Agarwal, N. (2023) · AI and Ethics · doi:10.1007/s43681-022-00147-7 · Cited by 48
“Some of these definitions were proposed for formalizing fairness from other disciplines [Hutchinson and Mitchell, 2019].”
Seven HCI Grand Challenges Revisited: Five-Year Progress
Stephanidis, C., Salvendy, G., Antona, M., Duffy, V. G., Gao, Q., Karwowski, W., Konomi, S., Nah, F. F., Ntoa, S., Rau, P. P., et al. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2450411 · Cited by 70
“Furthermore, previous studies typically used computational notions of fairness and equality of datasets and models (Hutchinson & Mitchell, 2019; Raghavan et al., 2020), but computational justice is not enough to capture the complex side of human perception of justice.”
Awareness in practice: tensions in access to sensitive attribute data for antidiscrimination
Bogen, M., Rieke, A., & Ahmed, S. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372877 · Cited by 45
“ork The fair ML research community has long reflected on the social and policy contexts of its work, recognizing legal tensions [8], historical parallels in prior debates over definitions of fairness [37], and the limitations of data-dependent problem formulation [64, 70]. However, when emphasizing the importance of a…”
BackgroundUsing fair AI to predict students’ math learning outcomes in an online platform
Li, C., Xing, W., & Leite, W. (2024) · Interactive Learning Environments · doi:10.1080/10494820.2022.2115076 · Cited by 47
“For example, Hutchinson and Mitchell (2019) described algorithmic bias as unjusti fi able performance on predictions against disadvantaged groups in social settings (e.g. race, gender, social-economic status).”
BackgroundConsiderations for a more ethical approach to data in AI: On data representation and infrastructure
Baird, A., & Schuller, B. (2020) · Frontiers in Big Data · doi:10.3389/fdata.2020.00025 · Cited by 46
“Furthermore, the evaluation of fairness in machine learning is another prominent topic, highlighted as a machine learning consideration in Hutchinson and Mitchell (2019).”
BackgroundAuditing and instructing text-to-image generation models on fairness
Friedrich, F., Brack, M., Struppek, L., Hintersdorf, D., Schramowski, P., Luccioni, A. S., & Kersting, K. (2024) · AI and Ethics · doi:10.1007/s43681-024-00531-5 · Cited by 82
(2023) · The Societal Impacts of Algorithmic Decision-Making · doi:10.1145/3603195.3603207 · Cited by 45
Algorithmic fairness in credit scoring
Bono, T., Croxson, K., & Giles, A. (2021) · Oxford Review of Economic Policy · doi:10.1093/oxrep/grab020 · Cited by 42
Saha, K., Grover, T., Mattingly, S. M., swain, V. D., Gupta, P., Martinez, G. J., Robles-Granda, P., Mark, G., Striegel, A., & De Choudhury, M. (2021) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3448117 · Cited by 43
“Demographic attribute-based stratified modeling has been identified by the Fairness, Accountability, Transparency literature to reinforce stereotypes and existing societal biases, and even exacerbate them [53, 58, 96].”
BackgroundKassir, S., Baker, L. J., Dolphin, J., & Polli, F. E. (2022) · AI and Ethics · doi:10.1007/s43681-022-00208-x · Cited by 81
Explainable and Fair AI: Balancing Performance in Financial and Real Estate Machine Learning Models
Acharya, D. B., Divya, B., & Kuppan, K. (2024) · IEEE Access · doi:10.1109/access.2024.3484409 · Cited by 78
“In addition, [11] and [18] has provided a comprehensive review of XAI techniques in the financial domain, illustrating how bias in credit scoring and fraud detection models can be mitigated through fairness-aware learning frameworks.”
BackgroundFairness in Design: A Framework for Facilitating Ethical Artificial Intelligence Designs
Zhang, J., Shu, Y., & Yu, H. (2023) · International Journal of Crowd Science · doi:10.26599/ijcs.2022.9100033 · Cited by 52
Antidiscrimination Insurance Pricing: Regulations, Fairness Criteria, and Models
Xin, X., & Huang, F. (2024) · North American Actuarial Journal · doi:10.1080/10920277.2023.2190528 · Cited by 43
Dang, V. N., Cascarano, A., Mulder, R. H., Cecil, C. A. M., Zuluaga, M. A., Hernández-González, J., & Lekadir, K. (2024) · Scientific Reports · doi:10.1038/s41598-024-58427-7 · Cited by 57
Mitigating Unfairness via Evolutionary Multiobjective Ensemble Learning
Zhang, Q., Liu, J., Zhang, Z., Wen, J., Mao, B., & Yao, X. (2022) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2022.3209544 · Cited by 53
“So far, no one fairness measure is accepted as a universal notion of fairness quantification since different perspectives of fairness can be interpreted into different quantitative definitions [1], [2], [4], [5], [17].”
BackgroundMethodInfluentialA Geometric Solution to Fair Representations
He, Y., Burghardt, K., & Lerman, K. (2020) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3375627.3375864 · Cited by 54
Demographic Fairness in Biometric Systems: What Do the Experts Say?
Rathgeb, C., Drozdowski, P., Frings, D., Damer, N., & Busch, C. (2022) · IEEE Technology and Society Magazine · doi:10.1109/mts.2022.3217700 · Cited by 47
“In fact, dozens of formal definitions (see [25] and [26]) have been proposed to address different situations, which are additionally entangled with different criteria of fairness.”
BackgroundHuman or AI robot? Who is fairer on the service organizational frontline
Wu, X., Li, S., Guo, Y., & Fang, S. (2024) · Journal of Business Research · doi:10.1016/j.jbusres.2024.114730 · Cited by 43
What Is Fair? Exploring the Artists’ Perspective on the Fairness of Music Streaming Platforms
Ferraro, A., Serra, X., & Bauer, C. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-85616-8_33 · Cited by 70
“Generally, the topic of fairness received particular attention in machine learning and artificial intelligence [35], focusing on the quantification of fairness and the algorithmic perspective.”
BackgroundInfluentialEnter the metrics: critical theory and organizational operationalization of AI ethics
Krijger, J. (2021) · AI & Society · doi:10.1007/s00146-021-01256-3 · Cited by 54
“This tension mainly applies to the normative principle of ‘fair-ness’, which has multiple quantitative definitions [discussed by a.o. Hutchinson and Mitchell (2019); Verma and Rubin (2018)] but is also relevant for operationalizing explainabil-ity, privacy and accountability measures.”
BackgroundFairness in Recommender Systems
Ekstrand, M. D., Das, A., Burke, R., & Díaz, F. (2012) · Springer eBooks · doi:10.1007/978-1-0716-2197-4_18 · Cited by 60
Evaluating Proposed Fairness Models for Face Recognition Algorithms
Howard, J. J., Laird, E., Rubin, R. E., Sirotin, Y. B., Tipton, J. L., & Vemury, A. R. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-37660-3_31 · Cited by 56
“Others, have made this observation as well [51], [52], [53], [48].”
BackgroundIntrinsic bias metrics do not correlate with application bias
Goldfarb-Tarrant, S., Marchant, R., Muñoz Sánchez, R., Pandya, M., & Lopez, A. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.150 · Cited by 33
“The coreference task (Figure 3, rows 1-3) doesn’t display a clear correlation in all cases, and yet it has the clearest relationship of all three tasks, with a significant moderate positive correlation for both Predictive Parity (precision) and Equality of Opportunity (re-call) for word2vec (columns 3 & 4).”
BackgroundInfluentialSahlgren, O. (2023) · Learning, Media and Technology · doi:10.1080/17439884.2021.1986065 · Cited by 30
Dense neural networks in knee osteoarthritis classification: a study on accuracy and fairness
Moustakidis, S., Papandrianos, N. I., Christodolou, E., Papageorgiou, E. I., & Tsaopoulos, D. E. (2020) · Neural Computing and Applications · doi:10.1007/s00521-020-05459-5 · Cited by 33
Pan, Y. (2019) · Frontiers of Information Technology & Electronic Engineering · doi:10.1631/fitee.1910001 · Cited by 30
Das, S. R., Stanton, R., & Wallace, N. E. (2023) · Annual Review of Financial Economics · doi:10.1146/annurev-financial-110921-125930 · Cited by 61
Critical Perspectives on Governance Mechanisms for AI/ML Systems
Stark, L., Greene, D., & Hoffmann, A. L. (2020) · Springer eBooks · doi:10.1007/978-3-030-56286-1_9 · Cited by 44
“These efforts have historical parallels in the social sciences, particularly around quantitative educational, vocational (Hutchinson & Mitchell, 2019), and psychometric testing (Lussier, 2018).”
BackgroundDesigning Up with Value-Sensitive Design: Building a Field Guide for Ethical ML Development
Boyd, K. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3534626 · Cited by 45
Intersectional analysis for science and technology
Nielsen, M. W., Gissi, E., Heidari, S., Horton, R. C., Nadeau, K. C., Ngila, D., Noble, S., Paik, H. Y., Tadesse, G. A., Zeng, E. Y., et al. (2025) · Nature · doi:10.1038/s41586-025-08774-w · Cited by 43
“These competing theories encompass notions of individual fairness, which focus on ensuring equal treat-ment for individuals with similar qualifications, and notions of group and subgroup fairness, which aim to achieve equal outcomes across different demographic subgroups 157 .”
Quantifying representativeness in randomized clinical trials using machine learning fairness metrics
Qi, M., Cahan, O., Foreman, M. A., Gruen, D. M., Das, A. K., & Bennett, K. P. (2021) · JAMIA Open · doi:10.1093/jamiaopen/ooab077 · Cited by 27
An information theoretic approach to reducing algorithmic bias for machine learning
Kim, J.-Y., & Cho, S. (2022) · Neurocomputing · doi:10.1016/j.neucom.2021.09.081 · Cited by 38
Algorithmic Fairness and Vertical Equity: Income Fairness with IRS Tax Audit Models
Black, E., Elzayn, H., Chouldechova, A., Goldin, J., & Ho, D. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533204 · Cited by 24
Metadata
- Authors
- Hutchinson, B., & Mitchell, M.
- ORCID
- Mitchell, Margaret
- Venue
- Proceedings of the Conference on Fairness, Accountability, and Transparency
- Publisher
- ACM
- Published
- 2019-01-29
- Pages
- 49-58
- OpenAlex
- W4288617757
- Cited by
- 297
- References
- 68
- Citations by year
- 2026: 23 · 2025: 45 · 2024: 38 · 2023: 61 · 2022: 49 · 2021: 45 · 2020: 25 · 2019: 8
- FWCI
- 33.19
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Law › Discrimination and Equality Law
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Keywords
- test fairness, educational testing, algorithmic fairness, fairness criteria, fairness measurement, subgroup fairness, history of testing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.