How Do Fairness Definitions Fare?: Examining Public Attitudes Towards Algorithmic Definitions of Fairness
The DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
Abstract
What is the best way to define algorithmic fairness? While many definitions of fairness have been proposed in the computer science literature, there is no clear agreement over a particular definition. In this work, we investigate ordinary people's perceptions of three of these fairness definitions. Across two online experiments, we test which definitions people perceive to be the fairest in the context of loan decisions, and whether fairness perceptions change with the addition of sensitive information (i.e., race of the loan applicants). Overall, one definition (calibrated fairness) tends to be more pre- ferred than the others, and the results also provide support for the principle of affirmative action.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adams, J. S. (1965) · Advances in Experimental Social Psychology · doi:10.1016/s0065-2601(08)60108-2 · Cited by 10,818 · cited 3×
“Broadly, we investigate a concept of fairness known as distributive justice, or fairness regarding the outcomes [1, 2].”
BackgroundMethodInfluentialTowards an understanding of inequity
Adams, J. S. (1963) · The Journal of Abnormal and Social Psychology · doi:10.1037/h0040968 · Cited by 5,146 · cited 1×
“re sensitive to the prevailing notions of fairness in society. Definitions of Fairness Broadly, we investigate a concept of fairness known as distributive justice, or fairness regarding the outcomes (Adams, 1963; 1965). However, which characteristics regarding the individual should be relevant and which should be irre…”
MethodDwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012) · Proceedings of the 3rd Innovations in Theoretical Computer Science Conference · doi:10.1145/2090236.2090255 · Cited by 3,501
Equality of Opportunity in Supervised Learning
Hardt, M., Price, E., & Srebro, N. (2016) · arXiv · doi:10.48550/arxiv.1610.02413 · Cited by 2,536
Fair prediction with disparate impact: a study of bias in recidivism prediction instruments
Chouldechova, A. (2017) · Big Data · doi:10.1089/big.2016.0047 · Cited by 2,159 · cited 1×
“In [1]Note that Kleinberg et al. (2016), Chouldechova (2017) define ‘calibration’ in a different way, that includes the notion of a sensitive attribute.”
BackgroundLee, M. K. (2018) · Big Data & Society · doi:10.1177/2053951718756684 · Cited by 1,338
Inherent Trade-Offs in the Fair Determination of Risk Scores
Kleinberg, J., Mullainathan, S., & Raghavan, M. (2016) · arXiv · doi:10.48550/arxiv.1609.05807 · Cited by 941
'It's Reducing a Human Being to a Percentage': Perceptions of Justice in Algorithmic Decisions
Binns, R., Van Kleek, M., Veale, M., Lyngs, U., Zhao, J., & Shadbolt, N. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173951 · Cited by 691 · cited 2×
“…in targeted online advertising, Grgić-Hlača et al. (2018a) and Grgić-Hlača et al. (2018b) studied human perceptions of features used in algorithmic decision making, and Binns et al. (2018) examined people’s perception of justice in algorithmic decision making under different explanation styles.”
BackgroundA Qualitative Exploration of Perceptions of Algorithmic Fairness
Woodruff, A., Fox, S. E., Rousso-Schindler, S., & Warshaw, J. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174230 · Cited by 256
Grgic-Hlaca, N., Redmiles, E. M., Gummadi, K. P., & Weller, A. (2018) · Proceedings of the 2018 World Wide Web Conference on World Wide Web - WWW '18 · doi:10.1145/3178876.3186138 · Cited by 207
Research works printed but not linked: 14
- J Angwin , J Larson , S Mattu , and L Kirchner . 2016. Machine bias risk assessments in criminal sentencing. https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing. ProPublica https://www. propublica. org ( 2016 ). Accessed : 2018-03--27. J Angwin, J Larson, S Mattu, and L Kirchner. 2016. Machine bias risk assessments in criminal sentencing. https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing. ProPublica https://www. propublica. org (2016). Accessed: 2018-03--27.
- Edmond Awad , Sohan Dsouza , Richard Kim , Jonathan Schulz , Joseph Henrich , Azim Shariff , Jean-Francc ois Bonnefon, and Iyad Rahwan . 2018 . The moral machine experiment. Nature , Vol. 563 , 7729 (2018), 59. Edmond Awad, Sohan Dsouza, Richard Kim, Jonathan Schulz, Joseph Henrich, Azim Shariff, Jean-Francc ois Bonnefon, and Iyad Rahwan. 2018. The moral machine experiment. Nature, Vol. 563, 7729 (2018), 59.
- 10.1111/1467-8721.ep10770996 · DOI
- Jean-Francc ois Bonnefon , Azim Shariff , and Iyad Rahwan . 2016 . The social dilemma of autonomous vehicles . Science , Vol. 352 , 6293 (2016), 1573--1576. Jean-Francc ois Bonnefon, Azim Shariff, and Iyad Rahwan. 2016. The social dilemma of autonomous vehicles. Science, Vol. 352, 6293 (2016), 1573--1576.
- Evelyn Ellis and Philippa Watson . 2012. EU anti-discrimination law . Oxford University Press . Evelyn Ellis and Philippa Watson. 2012. EU anti-discrimination law .Oxford University Press.
- Pratik Gajane and Mykola Pechenizkiy . 2017. On formalizing fairness in prediction with machine learning. arXiv preprint arXiv:1710.03184 ( 2017 ). Pratik Gajane and Mykola Pechenizkiy. 2017. On formalizing fairness in prediction with machine learning. arXiv preprint arXiv:1710.03184 (2017). · arXiv
- Joshua David Greene . 2014. Moral tribes: Emotion, reason, and the gap between us and them . Penguin . Joshua David Greene. 2014. Moral tribes: Emotion, reason, and the gap between us and them .Penguin.
- Nina Grgić-Hlavc a, Muhammad Bilal Zafar , Krishna P Gummadi , and Adrian Weller . 2018 b. Beyond distributive fairness in algorithmic decision making: Feature selection for procedurally fair learning . In Thirty-Second AAAI Conference on Artificial Intelligence . Nina Grgić-Hlavc a, Muhammad Bilal Zafar, Krishna P Gummadi, and Adrian Weller. 2018b. Beyond distributive fairness in algorithmic decision making: Feature selection for procedurally fair learning. In Thirty-Second AAAI Conference on Artificial Intelligence .
- Karen Huang , Joshua David Greene, and M Bazerman . 2019 . Veil-of-ignorance reasoning favors the greater good. (2019). Working paper. Karen Huang, Joshua David Greene, and M Bazerman. 2019. Veil-of-ignorance reasoning favors the greater good. (2019). Working paper.
- 10.1145/3025453.3025884 · DOI · Paper page
- Yang Liu , Goran Radanovic , Christos Dimitrakakis , Debmalya Mandal , and David C Parkes . 2017. Calibrated fairness in bandits. arXiv preprint arXiv:1707.01875 ( 2017 ). Yang Liu, Goran Radanovic, Christos Dimitrakakis, Debmalya Mandal, and David C Parkes. 2017. Calibrated fairness in bandits. arXiv preprint arXiv:1707.01875 (2017). · arXiv
- Emma Pierson . 2017. Gender differences in beliefs about algorithmic fairness. arXiv preprint arXiv:1712.09124 ( 2017 ). Emma Pierson. 2017. Gender differences in beliefs about algorithmic fairness. arXiv preprint arXiv:1712.09124 (2017). · arXiv
- Angelisa C Plane Elissa M Redmiles Michelle L Mazurek and Michael Carl Tschantz. 2017. Exploring user perceptions of discrimination in online targeted advertising. In USENIX Security . Angelisa C Plane Elissa M Redmiles Michelle L Mazurek and Michael Carl Tschantz. 2017. Exploring user perceptions of discrimination in online targeted advertising. In USENIX Security .
- Menahem E Yaari and Maya Bar-Hillel . 1984. On dividing justly. Social choice and welfare , Vol. 1 , 1 ( 1984 ), 1--24. Menahem E Yaari and Maya Bar-Hillel. 1984. On dividing justly. Social choice and welfare, Vol. 1, 1 (1984), 1--24.
Cited by
A Survey on Bias and Fairness in Machine Learning
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2019) · arXiv · doi:10.48550/arxiv.1908.09635 · Cited by 5,812
Self-citationA Survey on Bias and Fairness in Machine Learning
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2022) · ACM Computing Surveys · doi:10.1145/3457607 · Cited by 5,812
Self-citationMachine Learning Testing: Survey, Landscapes and Horizons
Zhang, J. M., Harman, M., Ma, L., & Liu, Y. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2019.2962027 · Cited by 897
Self-citationAlgorithmic bias: review, synthesis, and future research directions
Kordzadeh, N., & Ghasemaghaei, M. (2022) · European Journal of Information Systems · doi:10.1080/0960085x.2021.1927212 · Cited by 770
“In the context of loan decisions, Saxena et al. (2019) reported that people viewed ratio-based lending (i.e., splitting the money between the two candidates in proportion to their abilities to pay back their loans) as a fair method for giving loans to people.”
InfluentialTrustworthy Artificial Intelligence: A Review
Kaur, D., Uslu, S., Rittichier, K. J., & Durresi, A. (2023) · ACM Computing Surveys · doi:10.1145/3491209 · Cited by 709
“• Fairness: Fairness of the system ensures that there is an absence of any discrimination or favoritism toward an individual or a group [136] based on any inherent or acquired characteristics that are irrelevant in the context of decision making [177].”
Background
Metadata
- Authors
- Saxena, N. A., Huang, K., DeFilippis, E., Radanovic, G., Parkes, D. C., & Liu, Y.
- Venue
- Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society
- Publisher
- ACM
- Published
- 2019-01-27
- Pages
- 99-106
- ISSN
- 3065-8365 (ISSN-L)
- OpenAlex
- W2963588812
- Cited by
- 155
- References
- 13
- FWCI
- 22.10
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: How Do Fairness Definitions Fare?: Examining Public Attitudes Towards Algorithmic Definitions of Fairness. https://app.cochat.ai/papers/10.1145/3306618.3314248
- The paper
- Saxena, N. A., Huang, K., DeFilippis, E., Radanovic, G., Parkes, D. C., & Liu, Y. (2019). How Do Fairness Definitions Fare?: Examining Public Attitudes Towards Algorithmic Definitions of Fairness. Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society. https://doi.org/10.1145/3306618.3314248
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.