How Do Algorithmic Fairness Metrics Align with Human Judgement? A Mixed-Initiative System for Contextualized Fairness Assessment
Abstract
Fairness evaluation presents a challenging problem in machine learning, and is usually restricted to the exploration of various metrics that attempt to quantify algorithmic fairness. However, due to cultural and perceptual biases, such metrics are often not powerful enough to accurately capture what people perceive as fair or unfair. To close the gap between human judgement and automated fairness evaluation, we develop a mixed-initiative system named FairAlign, where laypeople assess the fairness of different classification models by analyzing expressive and interactive visualizations of data. Using the aggregated qualitative feedback, data scientists and machine learning experts can examine the similarities and the differences between predefined fairness metrics and human judgement in a contextualized setting. To validate the utility of our system, we conducted a small study on a socially relevant classification task, where six people were asked to assess the fairness of multiple prediction models using the provided visualizations. The results show that our platform is able to give valuable guidance for model evaluation in case of otherwise contradicting and indecisive metrics for algorithmic fairness.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Algorithmic fairness dashboards have complexity in designing comprehensive dashboards.
“Complexity in designing comprehensive dashboards”
From AI Fairness in Data Management and Analytics: A Review on Challenges, Methodologies and Applications (2023) · cites this paper as [264]
Algorithmic fairness dashboards provide real-time visualizations and metrics for monitoring.
“Real-time visualizations and metrics for monitoring”
From AI Fairness in Data Management and Analytics: A Review on Challenges, Methodologies and Applications (2023) · cites this paper as [264]
Algorithmic fairness dashboards enable continuous fairness monitoring.
“Enables continuous fairness monitoring”
From AI Fairness in Data Management and Analytics: A Review on Challenges, Methodologies and Applications (2023) · cites this paper as [264]
Cites
UCI Machine Learning Repository
Asuncion, A. (2007) · Medical Entomology and Zoology · Cited by 24,190 · see citing works
Smoothing and Differentiation of Data by Simplified Least Squares Procedures.
Savitzky, A., & Golay, M. J. E. (1964) · Analytical Chemistry · doi:10.1021/ac60214a047 · Cited by 21,904 · see citing works
A NEW MEASURE OF RANK CORRELATION
Kendall, M. G. (1938) · Biometrika · doi:10.1093/biomet/30.1-2.81 · Cited by 5,988 · see citing works
Equality of Opportunity in Supervised Learning
Hardt, M., Price, E., & Srebro, N. (2016) · arXiv · doi:10.48550/arxiv.1610.02413 · Cited by 2,536 · see citing works
A New Measure of Rank Correlation
Kendall, M. G. (1938) · Biometrika · doi:10.2307/2332226 · Cited by 2,346 · see citing works
Fair prediction with disparate impact: a study of bias in recidivism prediction instruments
Chouldechova, A. (2017) · Big Data · doi:10.1089/big.2016.0047 · Cited by 2,159 · see citing works
Bellamy, R. K. E., Dey, K., Hind, M., Hoffman, S. C., Houde, S., Kannan, K., Lohia, P., Martino, J., Mehta, S., Mojsilovic, A., et al. (2018) · arXiv · doi:10.48550/arxiv.1810.01943 · Cited by 1,581 · see citing works
Algorithmic Decision Making and the Cost of Fairness
Corbett-Davies, S., Pierson, E., Feller, A., Goel, S., & Huq, A. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098095 · Cited by 1,149 · see citing works
Inherent trade-offs in the fair determination of risk scores
Kleinberg, J., Mullainathan, S., & Raghavan, M. (2017) · Schloss Dagstuhl – Leibniz-Zentrum für Informatik · doi:10.4230/lipics.itcs.2017.43 · Cited by 943 · cited 1× · see citing works
“However, these definitions may contradict each other [18], and it is challenging to determine which fairness definition is the appropriate one for a given context.”
BackgroundInherent Trade-Offs in the Fair Determination of Risk Scores
Kleinberg, J., Mullainathan, S., & Raghavan, M. (2016) · arXiv · doi:10.48550/arxiv.1609.05807 · Cited by 941 · see citing works
Fairness in Machine Learning: A Survey
Caton, S., & Haas, C. (2020) · arXiv · doi:10.48550/arxiv.2010.04053 · Cited by 646 · see citing works
Fairness in machine learning: A survey
Caton, S., & Haas, C. (2024) · ACM Computing Surveys · doi:10.1145/3616865 · Cited by 646 · see citing works
Zafar, M. B., Valera, I., Gomez Rodriguez, M., & Gummadi, K. P. (2017) · Proceedings of the 26th International Conference on World Wide Web · doi:10.1145/3038912.3052660 · Cited by 632 · see citing works
A Review on Fairness in Machine Learning
Pessach, D., & Shmueli, E. (2023) · ACM Computing Surveys · doi:10.1145/3494672 · Cited by 598 · see citing works
Keinan, G. (1987) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.52.3.639 · Cited by 387 · see citing works
Keinan, G. (1987) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.52.3.639 · Cited by 359 · see citing works
On the apparent conflict between individual and group fairness
Binns, R. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372864 · Cited by 312 · see citing works
Ranking Visualizations of Correlation Using Weber's Law
Harrison, L., Yang, F., Franconeri, S., & Chang, R. (2014) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2014.2346979 · Cited by 212 · see citing works
Fairness in Decision-Making — The Causal Explanation Formula
Zhang, J., & Bareinboim, E. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11564 · Cited by 202 · see citing works
Automated directed fairness testing
Udeshi, S., Arora, P., & Chattopadhyay, S. (2018) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3238147.3238165 · Cited by 200 · see citing works
Stasser, G. L. (1992) · Organizational Behavior and Human Decision Processes · doi:10.1016/0749-5978(92)90049-d · Cited by 189 · see citing works
Srivastava, M., Heidari, H., & Krause, A. (2019) · Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3292500.3330664 · Cited by 179 · see citing works
An Approach for Prediction of Loan Approval using Machine Learning Algorithm
Sheikh, M. A., Goel, A. K., & Kumar, T. (2020) · International Conference on Electronics and Sustainable Communication Systems (ICESC) · doi:10.1109/icesc48915.2020.9155614 · Cited by 150 · see citing works
FairSight: Visual Analytics for Fairness in Decision Making
Ahn, Y., & Lin, Y.-R. (2019) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2019.2934262 · Cited by 126 · see citing works
An empirical study on the perceived fairness of realistic, imperfect machine learning models
Harrison, G., Hanson, J., Jacinto, C., Ramirez, J., & Ur, B. (2020) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3351095.3372831 · Cited by 114 · see citing works
Themis: automatically testing software for discrimination
Angell, R., Johnson, B., Brun, Y., & Meliou, A. (2018) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3236024.3264590 · Cited by 94 · see citing works
Vaidya, A. (2017) · International Conference on Computing, Communication and Networking Technologies (ICCCNT) · doi:10.1109/icccnt.2017.8203946 · Cited by 58 · see citing works
Automated Essay Grading using Machine Learning Algorithm
Ramalingam, V., Pandian, A., Chetry, P., & Nigam, H. (2018) · Journal of Physics Conference Series · doi:10.1088/1742-6596/1000/1/012030 · Cited by 58 · see citing works
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2011) · arXiv · doi:10.48550/arxiv.1104.3913 · Cited by 55 · see citing works
Towards Explainability for AI Fairness
Zhou, J. J., Chen, F., & Holzinger, A. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-04083-2_18 · Cited by 51 · see citing works
Fairness in human judgement in assessment: a hermeneutic literature review and conceptual framework
Valentine, N. A., Durning, S. J., Shanahan, E. M., & Schuwirth, L. (2020) · Advances in Health Sciences Education · doi:10.1007/s10459-020-10002-1 · Cited by 50 · see citing works
Matthews, K. D., Janicki, T. N., He, L. N., & Patterson, L. (2012) · Journal of the Association for Information Systems · Cited by 40 · see citing works
Fair Enough: Searching for Sufficient Measures of Fairness
Majumder, S., Chakraborty, J., Bai, G. R., Stolee, K. T., & Menzies, T. (2023) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3585006 · Cited by 36 · see citing works
Fairness in Criminal Justice Risk Assessments: The State of the ArtPreprint
Berk, R. A., Heidari, H., Jabbari, S., Kearns, M. J., & Roth, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.09207 · Cited by 34 · see citing works
Human Comprehension of Fairness in Machine Learning
Saha, D., Schumann, C., McElfresh, D. C., Dickerson, J. P., Mazurek, M. L., & Tschantz, M. C. (2020) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3375627.3375819 · Cited by 18 · see citing works
Decision biases in the context of ethics: Initial scale development and validation
Watts, L. L., Medeiros, K. E., McIntosh, T. J., & Mulhearn, T. J. (2020) · Personality and Individual Differences · doi:10.1016/j.paid.2019.109609 · Cited by 15 · see citing works
Kazim, E., Barnett, J., & Koshiyama, A. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3698404 · Cited by 6 · see citing works
Schoeffer, J., Machowski, Y., & Kuehl, N. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.05792 · Cited by 4 · see citing works
Fair Enough: Searching for Sufficient Measures of FairnessPreprint
Majumder, S., Chakraborty, J., Bai, G. R., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.13029 · Cited by 3 · see citing works
Aligning AI With Shared Human Values
Hendrycks, D., Burns, C., Basart, S., Critch, A., Li, J., Song, D., & Steinhardt, J. (2020) · arXiv · doi:10.48550/arxiv.2008.02275
Flagged first, then by citations
Cited by
Sorted by citations
AI Fairness in Data Management and Analytics: A Review on Challenges, Methodologies and Applications
Chen, P., Wu, L., & Wang, L. (2023) · Applied Sciences · doi:10.3390/app131810258 · Cited by 155 · see citing works
The value alignment problem in advisory AI: a systematic literature review
Triantafyllopoulos, L., Paxinou, E., Tzanoulinou, D., Verykios, V. S., & Kalles, D. (2026) · AI and Ethics · doi:10.1007/s43681-026-01015-4 · Cited by 6 · see citing works
Shrestha, H., Cachel, K., Alkhathlan, M., Rundensteiner, E. A., & Harrison, L. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3594108 · Cited by 4 · see citing works
- Authors
- Constantin, R., Duck, M., Alexandrov, A., Matosevic, P., Keidar, D., & El-Assady, M.
- Affiliations
- ETH Zurich (CH)
- ETH Zürich,Switzerland
- Venue
- 2022 IEEE Workshop on TRust and EXpertise in Visual Analytics (TREX)
- Publisher
- IEEE
- Published
- 2022-10
- Pages
- 1-7
- OpenAlex
- W4312800493
- Cited by
- 3
- References
- 35
- Citations by year
- 2026: 1 · 2023: 2
- FWCI
- 0.50
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE Workshop on TRust and EXpertise in Visual Analytics (TREX) (publishedVersion)
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Imbalanced Data Classification Techniques
- Keywords
- algorithmic fairness, fairness metrics, human judgment, mixed-initiative systems, interactive visualization, classification models, model evaluation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
