50 Years of Test (Un)fairness: Lessons for Machine Learning
The DOI matches its Crossref record; 71 of the 71 references it cites were found in a registry or an index.
Abstract
Quantitative definitions of what is unfair and what is fair have been introduced in multiple disciplines for well over 50 years, including in education, hiring, and machine learning. We trace how the notion of fairness has been defined within the testing communities of education and hiring over the past half century, exploring the cultural and social context in which different fairness definitions have emerged. In some cases, earlier definitions of fairness are similar or identical to definitions of fairness in current machine learning research, and foreshadow current formal work. In other cases, insights into what fairness means and how to measure it have largely gone overlooked. We compare past and current notions of fairness along several dimensions, including the fairness criteria, the focus of the criteria (e.g., a test, a model, or its use), the relationship of fairness to individuals, groups, and subgroups, and the mathematical method for measuring fairness (e.g., classification, regression). This work points the way towards future research and measurement of (un)fairness that builds from our modern understanding of fairness while incorporating insights from the past.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“These toolkits have been criticised for offering a 'reductionist understanding of fairness as mathematical conditions' [71, p. 1], and reflect a longer history of attempts to reduce (un)fairness to a metric [72].”
From Putting AI ethics to work: are the tools fit for purpose? (2022) · cites this paper as [72]
In the citing paper
“Speciically, there is a rich set of fairness-related work in a variety of disciplines, often with concepts that are similar or equal to current ML fairness research [147].”
From Fairness in machine learning: A survey (2024) · cites this paper as [147]
Citations
Cites
Advances in Neural Information Processing Systems 19
(2007) · The MIT Press eBooks · doi:10.52202/085713 · Cited by 15,539
Proceedings of the 25th international conference on Machine learning - ICML '08
(2008) · doi:10.1145/1390156 · Cited by 6,680
(2018) · Cited by 3,510
Semi-supervised learning using Gaussian fields and harmonic functions
Zhu, X., Zoubin Ghahramani, A., & Lafferty, J. (2003) · Cited by 3,509
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012) · Proceedings of the 3rd Innovations in Theoretical Computer Science Conference · doi:10.1145/2090236.2090255 · Cited by 3,501 · cited 1×
“…A ⊥ R ; it is equivalent when A and R have a bivariate Gaussian • Guion’s definition “people with equal probabilities of success is a special case of Dwork’s [23] individual fairness with the relationship between Y and R (since an ability being measured affects the for the optimal score R ∗”
BackgroundDoyle, T. (2017) · The Information Society · doi:10.1080/01972243.2017.1354593 · Cited by 3,071
Equality of Opportunity in Supervised Learning
Hardt, M., Price, E., & Srebro, N. (2016) · arXiv · doi:10.48550/arxiv.1610.02413 · Cited by 2,536 · cited 5×
“bility of success on the job” is a construct that can be meaningfully reasoned about. The fairness literature in both the fields of ML and in testing have also been motivated by causal considerations [32, 41]. Darlington [19] motivate his definition (3) on the basis of a causal relationship between Y and R (since an a…”
BackgroundResultInfluentialFair prediction with disparate impact: a study of bias in recidivism prediction instruments
Chouldechova, A. (2017) · Big Data · doi:10.1089/big.2016.0047 · Cited by 2,159 · cited 3×
“uality of opportunity [32]. That same year, Linn [43] introduced (but did not advocate for) equal Positive Predictive Value (PPV) as a fairness criterion, recognizable as modern day predictive parity [9].4 Under Cleary and Darlington’s conceptions, bias or (un)fairness is a property of the test itself. This is contrar…”
BackgroundResultMitigating Unwanted Biases with Adversarial Learning
Zhang, B. H., Lemoine, B., & Mitchell, M. (2018) · Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3278721.3278779 · Cited by 1,387 · cited 1×
“or bias in model features. Instead, one approach adopted in ML fairness has been the use of adversarial methods to mitigate the effects of features with undesirable correlations with subgroups, e.g., [6, 69]. 3.4 Model vs. Model Use Section 2 described how the test literature had competing notions of whether fairness…”
MethodInfluentialSelf-citationAlgorithmic Decision Making and the Cost of Fairness
Corbett-Davies, S., Pierson, E., Feller, A., Goel, S., & Huq, A. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098095 · Cited by 1,149
Research works printed but not linked: 11
- Beutel Alex, 2017, Chi
- Buolamwini Joy, 2018, Conference on Fairness, Accountability and Transparency. 77--91
- James, 2018, Foulds and Shimei Pan
- Friedler Sorelle A, 2016, On the (im) possibility of fairness. arXiv preprint arXiv: 1609.07236 · arXiv
- Hoffmann Anna Lauren, Spaces for the Future
- Jencks Christopher, 1998, Racial bias in testing. The Black-White test score gap 55
- 10.3102/00346543043002139 · doi:10.3102/00346543043002139 · Paper page
- National Council on Measurement in Education NCME (Ed.). 1976. Journal of Education Measurement. 13 1 (1976). National Council on Measurement in Education NCME (Ed.). 1976. Journal of Education Measurement. 13 1 (1976).
- Phillips S E, Fairness in Educational Assessment and Measurement
- Rice Mitchell F, 1994, Race Norming, Validity Generalization, and Employment Testing, Handbook of Public Personnel Administration, 58, 451
- Zwick Rebecca, Fairness in Educational Assessment and Measurement
Cited by
Items 201–300 of 304
Item Response Theory in Sample Reweighting to Build Fairer Classifiers
Minatel, D., dos Santos, N. R., da Silva, V. F., Cúri, M., & Lopes, A. D. A. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-63616-5_14 · Cited by 4
Olteanu, A., Garcia-Gathright, J. I., de Rijke, M., & Ekstrand, M. D. (2019) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3331184.3331644 · Cited by 4
“As with other AI-driven technologies, IR systems are also under the influence of those that design, build, maintain or use them, embedding and amplifying their biases [2, 3, 7, 11].”
BackgroundHow to Reduce Bias in the Life Cycle of a Data Science Project
Banks, G. C., Tonidandel, S., Dou, W., Gerson, M. J., Xu, D., & Yavorsky, J. E. (2025) · Journal of Business and Psychology · doi:10.1007/s10869-025-10022-x · Cited by 6
Johnson, D. K. N. (2025) · The Review of Black Political Economy · doi:10.1177/00346446251331615 · Cited by 3
“To do this well, reviewers should understand the reasons for deviation, the causal pathways of bias (Hutchinson & Mitchell, 2019).”
Mota, V. F., Laranjeira, C., Steinmüller, T., & Duarte, E. F. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3806733 · Cited by 3
“However, the definition of fairness varies significantly across different fields, with each discipline applying its own lens to interpret what fairness means in specific contexts, from legal frameworks [14] to Machine Learning [29].”
Surrogate Modeling to Address the Absence of Protected Membership Attributes in Fairness Evaluation
Kadıoğlu, S., & Thielbar, M. (2024) · ACM Transactions on Evolutionary Learning and Optimization · doi:10.1145/3700145 · Cited by 3
“This has led to a rich literature on the meaning of “fair” models [Hutchinson and Mitchell 2019], how model fairness can be measured [Caton and Haas 2024], and a collection of benchmark data sets that can be used to test various fairness scenarios [Le Quy et al. 2022].”
de Lima, L. F. F. P., Ricarte, D. R. D., & Siebra, C. (2024) · Computational Intelligence · doi:10.1111/coin.70003 · Cited by 3
Preference eigensystems for fair ranking
Melucci, M. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.126324 · Cited by 3
Demographic-Adapted ROC Curve for Assessing Automated Matching of Latent Fingerprints
Marasco, E., He, M., Tang, L. L., & Sriram, S. (2022) · SN Computer Science · doi:10.1007/s42979-022-01080-6 · Cited by 3
“Scientists are currently confronting complex research questions about what it means to make an algo-rithm fair and they are debating about how to best measure algorithmic fairness [1, 2].”
BackgroundModel-Agnostic Bias Measurement in Link Prediction
Schwertmann, L., Ravi, M. P. K., & de Melo, G. (2023) · Findings of the Association for Computational Linguistics: EACL · doi:10.18653/v1/2023.findings-eacl.121 · Cited by 3
“The gap is: We choose this and the following metric because they are well-established in the algorithmic fairness community (Verma and Rubin, 2018; Hutchinson and Mitchell, 2019; Barocas et al., 2019), and each focuses on different capabilities of a classifier.”
BackgroundDIF-SR: A Differential Item Functioning-Based Sample Reweighting Method
Minatel, D., Parmezan, A. R. S., Cúri, M., & Lopes, A. D. A. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-49018-7_45 · Cited by 3
Enhancing Business Process Models with Ethical Considerations
Amico, B., Combi, C., Vecchia, A. D., Migliorini, S., Oliboni, B., & Quintarelli, E. (2025) · Lecture notes in business information processing · doi:10.1007/978-3-031-79059-1_1 · Cited by 3
Hohma, E., Burnell, R., Corrigan, C. C., & Luetge, C. (2022) · IEEE Transactions on Technology and Society · doi:10.1109/tts.2022.3211073 · Cited by 3
“Trying to solve the issue on how to incorporate fairness in ML algorithms, researchers have already gathered and developed numerous definitions of fairness, e.g., [6], [44], [45] and translated them into several distinct fairness models, e.g., [5], [46], [47], [48], [49].”
BackgroundA Responsible and Inclusive Technology Framework for Attending to Business-to-Business Contexts
Sandoval, . C. B., de Santana, V. F., Berger, S., Quigley, L. T., & Hobson, S. F. (2025) · ACM Journal on Responsible Computing · doi:10.1145/3712310 · Cited by 14
Ponmalar, S., Kumar, K. N., Balaji, R. V., Rohith, T., & Kaarthick, A. K. N. (2025) · International Conference on Augmented Intelligence and Sustainable Systems (ICAISS) · doi:10.1109/icaiss61471.2025.11042179 · Cited by 3
Towards Systematic Specification and Verification of Fairness Requirements: A Position Paper
Ramadan, Q., Ruohonen, J., Tiwari, A., Alami, A., & Boukhers, Z. (2025) · IEEE International Requirements Engineering Conference Workshops (REW) · doi:10.1109/rew66121.2025.00059 · Cited by 6
The Multifaced Nature of Bias in AI: Impact on Model Generalization, Robustness, and Fairness
Ntoutsi, E. (2025) · Springer eBooks · doi:10.1007/978-3-658-48522-1_11 · Cited by 2
Exploring Societal Biases in Generative AI using Social Science Constructs and Theories
Bano, M., Hoda, R., & Zowghi, D. (2026) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3786581.3786924 · Cited by 3
“Despite ongoing de-biasing efforts [31, 39], such biases persist due to the inherent challenges of removing historical and social prejudices from large datasets [33, 62].”
Fairness of linear regression in decision making
Cohen-Addad, V., Gavva, S. T., Karthik, C. S., Mathieu, C., & Namrata, N. (2023) · International Journal of Data Science and Analytics · doi:10.1007/s41060-023-00423-7 · Cited by 5
The Emerging Artifacts of Centralized Open-Code
Choksi, M. Z., Mandel, I., Widder, D. G., & Shvartzshnaider, Y. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3659019 · Cited by 5
“We use a variety of primary and secondary sources to build on prior work that traces the history of other political concepts such as accountability [33], fairness [72], and AI ethics [12].”
BackgroundChowdhury, S., & Klautzer, L. (2025) · Cambridge Forum on AI Law and Governance · doi:10.1017/cfl.2025.7 · Cited by 4
Three methods for fair ranking of multiple protected items
Melucci, M. (2025) · Scientific Reports · doi:10.1038/s41598-025-12735-8 · Cited by 2
“4 There are two types of fairness, as explained in 5 : group fairness and individual fairness, which influence ranking effectiveness.”
Lee, M. S. A., Floridi, L., & Singh, J. (2022) · Digital ethics lab yearbook · doi:10.1007/978-3-031-09846-8_11 · Cited by 3
Best vs. All: Equity and Accuracy of Standardized Test Score Reporting
Niu, M., Kannan, S., Roth, A., & Vohra, R. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533121 · Cited by 13
Automated Scoring of Open-Ended Written Responses: Possibilities and Challenges
Flor, M., & Cahill, A. (2025) · Methodology of educational measurement and assessment · doi:10.1007/978-3-031-90951-1_11 · Cited by 6
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603198 · Cited by 2
Algorithmic Monoculture and Social Welfare
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603211 · Cited by 2
Should Bank Stress Tests Be Fair?Preprint
Glasserman, P., & Li, M. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4158738 · Cited by 3
The Fairness Impact Assessment: Conceptualizing Problems of Fairness in Technological Design
Shelley, C. (2022) · International Journal of Technoethics · doi:10.4018/ijt.291554 · Cited by 2
Identifying Imbalance Thresholds in Input Data to Achieve Desired Levels of Algorithmic Fairness
Mecati, M., Adrignola, A., Vetrò, A., & Torchiano, M. (2022) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata55660.2022.10021078 · Cited by 2
“Our contribution can be located in the main area of research on algorithmic bias and fairness, with a specific focus on inputs and processes, as suggested by several recent studies (e.g., [19], [20] and [21]).”
BackgroundSuk, Y., & Han, K. T. (2024) · Journal of Educational and Behavioral Statistics · doi:10.3102/10769986241269820 · Cited by 5
“Definitions of modern-day algorithmic fairness are often similar or identical to earlier definitions of test fairness (Hutchinson & Mitchell, 2019).”
Algorithmic Fairness as an Inconsistent Concept
Hummel, P. (2025) · American Philosophical Quarterly · doi:10.5406/21521123.62.1.04 · Cited by 1
“COMPAS is of course not an isolated case: as already anticipated early discourses on fairness in classification and prediction in the 1970s (Hutchinson and Mitchell 2019, 50–51) and proven more recently by several authors (Chouldechova 2017; Kleinberg, Mullainathan, and Raghavan 2017; Berk et al.…”
InfluentialWhy causal inference is necessary for algorithmic fairness
Tolbert, A. W. (2025) · Synthese · doi:10.1007/s11229-025-05044-0 · Cited by 3
“Hutchinson and Mitchell (2019) argue that this shift came at a cost, potentially leading to a loss of focus on the core questions that require causal knowledge.”
InfluentialFrom rights to runtime: Privacy engineering for agentic AI
Navaie, K. (2025) · AI Magazine · doi:10.1002/aaai.70036 · Cited by 7
The Anatomy of AI-Generated Disinformation from Deepfakes to Voice Cloning
Ghai, A. S., Jasola, S., Sharma, R. C., Machado, A. A., Sousa, M. J., & Ghai, K. (2026) · Springer eBooks · doi:10.1007/978-981-95-4871-2_7 · Cited by 2
Algorithmic ableism and the normative body: Disability in the age of technocapitalist AI
İnce, E. (2026) · Technology and Disability · doi:10.1177/10554181261433364 · Cited by 2
Software Unfairness Detection in Machine Learning-Based Systems: A Systematic Mapping Study
Alharbi, R., & Abbadeni, N. (2026) · Software · doi:10.3390/software5020018 · Cited by 3
Generative AI for the power sector: Methodology, safety, trustworthiness, and policy
Alshehri, K. A., & Sayed‐Mouchaweh, M. (2026) · Next Energy · doi:10.1016/j.nxener.2026.100636 · Cited by 3
Machine Learning Bias in Computer Vision: Why do I have to care?
Laranjeira, C., Mota, V. F., & dos Santos, J. A. (2021) · SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI) · doi:10.1109/sibgrapi54419.2021.00010 · Cited by 3
Mitigating Bias in Algorithmic Decision-making: Evaluating Claims and Practices
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603212 · Cited by 1
MBFair: a model-based verification methodology for detecting violations of individual fairness
Ramadan, Q., Konersmann, M., Ahmadian, A. S., Jürjens, J., & Staab, S. (2024) · Software & Systems Modeling · doi:10.1007/s10270-024-01184-y · Cited by 6
Human Perceptions of Fairness: A Survey Experiment
Sengewald, J., Schlichter, A., Siepermann, M., & Lackes, R. (2025) · Lecture notes in information systems and organisation · doi:10.1007/978-3-031-80119-8_4 · Cited by 1
Lussier, K. (2021) · IEEE Annals of the History of Computing · doi:10.1109/mahc.2021.3055417 · Cited by 1
Marasco, E., He, M., Tang, L. L., & Sriram, S. (2021) · Communications in computer and information science · doi:10.1007/978-981-16-1086-8_30 · Cited by 3
The Coded Gaze: algorithmic bias, facial recognition and beyond
Hooberman, L. (2021) · ACM Web Science Conference (WebSci) · doi:10.1145/3447535.3468455 · Cited by 1
The Hidden Assumptions Behind Counterfactual Explanations and Principal Reasons
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603204 · Cited by 1
The Cost of Fairness: Evaluating Economic Implications of Fairness-Aware Machine LearningPreprint
Shimao, H., Komiyama, J., & Khern-am-nuai, W. (2018) · SSRN Electronic Journal · doi:10.2139/ssrn.3314373 · Cited by 1
Rzepka, N., Fernsel, L., Müller, H.-G., Simbeck, K., & Pinkwart, N. (2023) · EdArXiv (OSF Preprints) · doi:10.35542/osf.io/qa9vz · Cited by 4
On Prediction-Modelers and Decision-Makers: Why Fairness Requires More Than a Fair Prediction ModelPreprint
Scantamburlo, T., Baumann, J., & Heitz, C. (2023) · Zürcher Hochschule für Angewandte Wissenschaften digital collection (Zurich University of Applied Sciences) · doi:10.48550/arxiv.2310.05598 · Cited by 2
“Therefore, our general framework does not foresee the provision of training data by the decision-maker as a minimum deliverable. the model [more] than of its use since it does not depend on decision thresholds” (Hutchinson and Mitchell 2019, p. 55).”
Kursuncu, U., Baird, A. M., & Xia, Y. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.5035722 · Cited by 4
Romanova, A. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5371922 · Cited by 2
Tang, K., Jiang, X., Li, C., & Zhang, K. (2022) · doi:10.2196/preprints.40794
Penteado, B. E. (2022) · Anais do ... Simpósio Brasileiro de Informática na Educação/Anais do Simpósio Brasileiro de Informática na Educação · doi:10.5753/sbie.2022.225710
“Education large-scale testing has faced bias since the 1960s and laid out many aspects of the modern literature on algorithmic bias and fairness (Hutchinson & Mitchell, 2019).”
BackgroundFairness von Biometrischen Systemen: Ein Überblick
Kolberg, J. (2023) · Datenschutz und Datensicherheit - DuD · doi:10.1007/s11623-022-1709-1
“Vielmehr wurden Dutzende formale Definitionen vorgeschlagen [68], [69], um verschiedene Situationen und mögliche Kriterien der Fairness zu berücksichtigen.”
BackgroundWhat is a related work? A typology of relationships in research literature
Doroudi, S. (2023) · Synthese · doi:10.1007/s11229-022-03976-5 · Cited by 1
“To formalize this, Hutchinson and Mitchell (2019) explicitly construct an analogy between the two literatures: Test items (questions) are analogous to model features, and item responses analogous to specific activations of those features.”
BackgroundExploring Out-of-Distribution in Image Classification for Neural Networks Via Concepts
Holmberg, L. (2023) · Lecture notes in networks and systems · doi:10.1007/978-981-99-3243-6_13 · Cited by 1
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603213
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603205
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603196
How Do Classifiers Induce Agents to Behave Strategically?
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603201
How Do Classifiers Induce Agents to Behave Strategically?
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603210
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603214
A Case Study of Accurate and Fair Classification
Liu, X., & Chao, Z. (2024) · Lecture notes in electrical engineering · doi:10.1007/978-981-99-9243-0_40
Hoeppner, S., & Samek, M. (2024) · AUC IURIDICA · doi:10.14712/23366478.2024.24 · Cited by 2
Fairness and Transparency in Music Recommender Systems: Improvements for Artists
Dinnissen, K. (2024) · ACM Conference on Recommender Systems (RecSys) · doi:10.1145/3640457.3688024 · Cited by 1
Integrating AI Into Early Childhood Education: Theory to Practice
Μεταξούδης, Ε. Θ. (2024) · Advances in educational technologies and instructional design book series · doi:10.4018/979-8-3693-6210-5.ch002 · Cited by 1
Kamp, S., Nkeng, T., Riquelme, V., & Fish, B. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-74630-7_18
“These works have focused on equalizing accuracy, error, or outcomes between groups defined by sensitive attributes such as race or gender [25, 27, 43, 48].”
BackgroundBaker, B. (2020) · University of North Texas Digital Library (University of North Texas) · doi:10.12794/metadc1707237 · Cited by 2
“This case study is significant because there have been numerous attempts to examine the BWTSG (Barton & Coley, 2010; Hutchinson & Mitchell, 2019; Jencks & Phillips, 2011; Lewis & Hunt, 2019; Reardon et al., 2019; Rodriguez & McGuire, 2019; Vigdor & Ludwig, 2007) to 8 find an answer to the question…”
A Comprehensive Review of Fake News Detection Approaches Using Machine Learning
Okorie, K. M., Okebanama, U. F., & Ellam, I. V. (2025) · European Journal of Theoretical and Applied Sciences · doi:10.59324/ejtas.2025.3(2).41
Ethical Considerations in AI-Driven Software Development Life Cycle
Sushir, R. D., & Bhutada, S. S. (2025) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3693-8407-7.ch001 · Cited by 1
Social Influence and the Perceived Fairness of Algorithmic Decisions: An Exploratory Study
Kleanthous, S., Ioannou, A., & Papandreou, F. (2025) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3708319.3733679 · Cited by 1
“Research shows that people’s views on fairness can differ widely based on the context, a person’s experiences and cultural background [6, 15], and there is no clear consensus on which aspects of algorithmic decisions are universally considered fair [10, 12, 23].”
Fair for Whom? Investigating School Identity, Algorithmic Fairness, and Educational Technologies
Belitz, C. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-99261-2_26
An Empirical Evaluation of Machine Learning Classifiers for Credit Risk in Peer-to-Peer LendingPreprint
Broby, D., HAMARAT, Ç., & Samak, Z. A. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5387370
Rocha, M. D. S., Pelegrina, G. D., & Duarte, L. T. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5472615 · Cited by 1
Practical Perspectives on Fairness In Technology-Aided Personnel Selection
Müller, I., Dill, K., & Mihaljević, H. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757647 · Cited by 6
More, A. K., Lorde, N., Kalaria, T., & Gama, R. M. (2025) · Diagnostics · doi:10.3390/diagnostics15222919 · Cited by 1
Eren, S. U. N. A., & Özer, M. (2025) · Chinese/English Journal of Educational Measurement and Evaluation · doi:10.59863/cmge2179
Inherent Tradeoffs in the Fair Determination of Risk Scores
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603206 · Cited by 3
The Externalities of Exploration and How Data Diversity Helps Exploitation
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603199
Algorithmic monoculture and social welfare
(2023) · The Societal Impacts of Algorithmic Decision-Making · doi:10.1145/3603195.3603202
Selection Problems in the Presence of Implicit Bias
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603200
Bias Evaluation in Contextual Machine Learning
Vecchia, A. D., & Stefanidis, K. (2025) · Communications in computer and information science · doi:10.1007/978-3-032-05727-3_45 · Cited by 2
Fairness-aware Machine Learning
Kamishima, T. (2025) · Ouyou toukeigaku · doi:10.5023/jappstat.54.81 · Cited by 1
Khushf, G., & Iltis, A. S. (2025) · The Journal of Medicine and Philosophy A Forum for Bioethics and Philosophy of Medicine · doi:10.1093/jmp/jhaf019 · Cited by 1
Fair GANs through model rebalancing for extremely imbalanced class distributions
Jain, A., Memon, N. D., & Togelius, J. (2025) · IEEE International Joint Conference on Biometrics (IJCB) · doi:10.1109/ijcb65343.2025.11410860 · Cited by 3
“Fairness in generative models is defined as equal representation with respect to one or more attributes (e.g. ethnicity or gender) [14].”
Generating the language of AI harms: mapping guardrails using critical code studies
Ciston, S. (2026) · AI & Society · doi:10.1007/s00146-026-02922-0 · Cited by 1
FairDIF: debiasing classifiers with item response theory and differential item functioning
Minatel, D., Parmezan, A. R. S., dos Santos, N. R., Curi, M. L. D. L. C., Prudêncio, R. B. C., & Lopes, A. D. A. (2026) · AI and Ethics · doi:10.1007/s43681-026-01055-w · Cited by 1
Hemamou, L. (2021) · theses.fr (ABES) · doi:10.70675/6bf948faza1c0z4e9fza95cze35f40a39ce3 · Cited by 1
Semi-supervised learning in insurance : fairness and active learning
Hu, F. (2022) · theses.fr (ABES) · doi:10.70675/f9cec72az9beaz4c9bz9436z152694ee9162 · Cited by 3
Gangrade, M., & Sivasamy, S. (2026) · International Conference on Smart Multidomain Integrated Learning Environments (ICSMILE) · doi:10.1109/icsmile69273.2026.11519239
Human Factors in Human-AI Interaction
Zhang, J., Zhang, L., & Sun, X. (2026) · Springer eBooks · doi:10.1007/978-981-95-5585-7_18
Evaluating AI in Education: Balancing Fairness, Explainability, and Performance
Gunasekara, S., & Saarela, M. (2026) · International Congress on Human-Computer Interaction, Optimization and Robotic Applications (ICHORA) · doi:10.1109/ichora69329.2026.11537052
From Post-Hoc Regulation to Runtime Governance: Ethics and Explainability in AI-Driven Education
Saarela, M., & Shenoy, P. (2026) · International Congress on Human-Computer Interaction, Optimization and Robotic Applications (ICHORA) · doi:10.1109/ichora69329.2026.11536988 · Cited by 1
Algorithmic Fairness Criteria as Evidence
Fleisher, W. (2026) · Ergo an Open Access Journal of Philosophy · doi:10.3998/ergo.9869 · Cited by 2
Praharenka, J. (2026) · AI and Ethics · doi:10.1007/s43681-026-01209-w
Lee, Y.-C., Li, J., Zhang, R., & Liu, N. (2026) · Cambridge University Press eBooks · doi:10.1017/9781009625180.002
Same System, Opposite Verdicts: Metric Discretion in AI Ethics Audits and the Limits of DisclosurePreprint
Tsaban, S. (2026) · Research Square · doi:10.21203/rs.3.rs-10576395/v1
Building Fairness-Aware Ensembles with AutoFairGE
Chen, Z., Connor, M., & O’Neill, M. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-36217-9_24
BODUROĞLU, E., AYHANÖZ, G. S., & ALAGÖZ, T. (2026) · Türk Eğitim Bilimleri Dergisi · doi:10.37217/tebd.1817291
Acharya, N., & Kar, P. (2026) · Frontiers in Future Transportation · doi:10.3389/ffutr.2026.1951756
Metadata
- Authors
- Hutchinson, B., & Mitchell, M.
- ORCID
- Mitchell, Margaret
- Venue
- Proceedings of the Conference on Fairness, Accountability, and Transparency
- Publisher
- ACM
- Published
- 2019-01-29
- Pages
- 49-58
- OpenAlex
- W4288617757
- Cited by
- 297
- References
- 68
- Citations by year
- 2026: 23 · 2025: 45 · 2024: 38 · 2023: 61 · 2022: 49 · 2021: 45 · 2020: 25 · 2019: 8
- FWCI
- 33.19
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Law › Discrimination and Equality Law
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Keywords
- test fairness, educational testing, algorithmic fairness, fairness criteria, fairness measurement, subgroup fairness, history of testing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.