k-NN as an implementation of situation testing for discrimination discovery and prevention
Abstract
With the support of the legally-grounded methodology of situation testing, we tackle the problems of discrimination discovery and prevention from a dataset of historical decisions by adopting a variant of k-NN classification. A tuple is labeled as discriminated if we can observe a significant difference of treatment among its neighbors belonging to a protected-by-law group and its neighbors not belonging to it. Discrimination discovery boils down to extracting a classification model from the labeled tuples. Discrimination prevention is tackled by changing the decision value for tuples labeled as discriminated before training a classifier. The approach of this paper overcomes legal weaknesses and technical limitations of existing proposals.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
[204] relabel the dependent variable if the data instance is determined as being discriminated against with respect to the observed outcome.
“For example, [204] relabel the dependent variable (lip it from positive to negative or vice versa) if the data instance is determined as being discriminated against with respect to the observed outcome.”
From Fairness in machine learning: A survey (2024) · cites this paper as [204]
Changes to the data should be minimised.
“As for transformation, modiication of the data via relabelling and perturbation is not always legally permissible [17] and changes to the data should be minimised [136,204].”
From Fairness in machine learning: A survey (2024) · cites this paper as [204]
Several prior efforts in algorithmic fairness have reported gender discrimination in Adult data based on a strong statistical dependency between income and gender in favor of males.
“Using this dataset, several prior efforts in algorithmic fairness have reported gender discrimination based on a strong statistical dependency between income and gender in favor of males [31,48,60].”
From Interventional Fairness: Causal Database Repair for Algorithmic Fairness (2019) · cites this paper as [31]
Cites
UCI Machine Learning Repository
Asuncion, A. (2007) · Medical Entomology and Zoology · Cited by 24,190 · see citing works
(1992) · Cited by 23,431 · see citing works
Statistical Methods for Rates and Proportions
Fleiss, J. L., Levin, B., & Paik, M. C. (2003) · Wiley Series in Probability and Statistics · doi:10.1002/0471445428 · Cited by 18,646 · see citing works
Hapeman, D. E., & Agresti, A. (1991) · Technometrics · doi:10.1080/00401706.1991.10484817 · Cited by 9,406 · see citing works
Agresti, A. (2002) · Wiley Series in Probability and Statistics · doi:10.1002/0471249688 · Cited by 6,746 · see citing works
Cohen, W. W. (1995) · Elsevier eBooks · doi:10.1016/b978-1-55860-377-6.50023-2 · Cited by 3,890 · see citing works
The Economics of Discrimination.
Collard, D. A., & Becker, G. S. (1972) · The Economic Journal · doi:10.2307/2230052 · Cited by 3,642 · see citing works
The Economics of Discrimination
Becker, G. S. (1971) · doi:10.7208/chicago/9780226041049.001.0001 · Cited by 1,794 · see citing works
Three naive Bayes approaches for discrimination-free classification.
Calders, T., & Verwer, S. (2010) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-010-0190-x · Cited by 764 · see citing works
Data mining for discrimination discovery
Ruggieri, S., Pedreschi, D., & Turini, F. (2010) · TKDD · doi:10.1145/1754428.1754432 · Cited by 250 · see citing works
Self-citationEllis, E., & Watson, P. (2012) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780199698462.001.0001 · Cited by 200 · see citing works
Affirmative Action Around the World
Sowell, T. (2017) · Yale University Press eBooks · doi:10.12987/9780300128352 · Cited by 177 · see citing works
Theories of Statistical Discrimination and Affirmative Action: A Survey
Fang, H., & Moro, A. (2010) · National Bureau of Economic Research · doi:10.3386/w15860 · Cited by 161 · see citing works
Group Rights and Discrimination in International Law
Lerner, N. (1990) · Brill | Nijhoff eBooks · doi:10.1163/9789004481633 · Cited by 147 · see citing works
Handbook on the Economics of Discrimination
Rodgers, W. M., III (2006) · Edward Elgar Publishing eBooks · doi:10.4337/9781847200150 · Cited by 82 · see citing works
Situation Testing for Employment Discrimination in the United States of America
Bendick, M. (2007) · Horizons stratégiques · doi:10.3917/hori.005.0017 · Cited by 55 · see citing works
Ruggieri, S., Pedreschi, D., & Turini, F. (2010) · Proceedings - ACM-SIGMOD International Conference on Management of Data · doi:10.1145/1807167.1807298 · Cited by 52 · see citing works
Self-citationStatistical Reasoning in the Legal Setting
Gastwirth, J. L. (1992) · The American Statistician · doi:10.1080/00031305.1992.10475851 · Cited by 37 · see citing works
Howard, E. (2018) · International Journal of Discrimination and the Law · doi:10.1177/1358229118788454 · Cited by 31 · see citing works
Rohner, R. J. (1979) · Scholarship at Catholic Law (Catholic University of America) · Cited by 27 · see citing works
Group Rights and Discrimination in International Law
Lerner, N. (1991) · Verfassung in Recht und Übersee · doi:10.5771/0506-7286-1991-4-461 · Cited by 23 · see citing works
Theories of Statistical Discrimination and Armative Action: A Survey
Fang, H., & Moro, A. (2010) · Cited by 20 · see citing works
Approaches for Dealing with Small Sample Sizes in Employment Discrimination Litigation
Piette, M. J., & White, P. F. (1999) · Journal of Forensic Economics · doi:10.5085/0898-5510-12.1.43 · Cited by 10 · see citing works
Proving Discrimination Cases: The Role of Situation Testing
Rorive, I. (2009) · Dépôt institutionnel de l'Université libre de Bruxelles (Université Libre de Bruxelles) · Cited by 7 · see citing works
Detecting and describing heterogeneity in meta-analysis
Hardy, R. J., & Thompson, S. G. (1998) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19980430)17:8
Flagged first, then by citations
Cited by
Sorted by citations
Certifying and Removing Disparate Impact
Feldman, M., Friedler, S. A., Moeller, J., Scheidegger, C., & Venkatasubramanian, S. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2783311 · Cited by 1,827 · see citing works
Data preprocessing techniques for classification without discrimination
Kamiran, F., & Calders, T. (2012) · Knowledge and Information Systems · doi:10.1007/s10115-011-0463-8 · Cited by 1,305 · see citing works
Bias in data‐driven artificial intelligence systems—An introductory survey
Ntoutsi, E., Fafalios, P., Gadiraju, U., Iosifidis, V., Nejdl, W., Vidal, M., Ruggieri, S., Turini, F., Papadopoulos, S., Krasanakis, E., et al. (2020) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1356 · Cited by 1,043 · see citing works
Self-citationFairness-Aware Classifier with Prejudice Remover Regularizer
Kamishima, T., Akaho, S., Asoh, H., & Sakuma, J. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33486-3_3 · Cited by 694 · see citing works
Zafar, M. B., Valera, I., Gomez Rodriguez, M., & Gummadi, K. P. (2017) · Proceedings of the 26th International Conference on World Wide Web · doi:10.1145/3038912.3052660 · Cited by 632 · see citing works
A Review on Fairness in Machine Learning
Pessach, D., & Shmueli, E. (2023) · ACM Computing Surveys · doi:10.1145/3494672 · Cited by 598 · see citing works
Fairness in machine learning: A survey
Caton, S., & Haas, C. (2024) · ACM Computing Surveys · doi:10.1145/3616865 · Cited by 646 · see citing works
Algorithmic Bias: From Discrimination Discovery to Fairness-aware Data Mining
Hajian, S., Bonchi, F., & Castillo, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2945386 · Cited by 334 · see citing works
A multidisciplinary survey on discrimination analysis
Romei, A., & Ruggieri, S. (2014) · The Knowledge Engineering Review · doi:10.1017/s0269888913000039 · Cited by 288 · see citing works
Self-citationThe relationship between trust in AI and trustworthy machine learning technologies
Toreini, E., Aitken, M., Coopamootoo, K., Elliott, K., Zelaya, C. G., & van Moorsel, A. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372834 · Cited by 296 · see citing works
Decision Theory for Discrimination-Aware Classification
Kamiran, F., Karim, A., & Zhang, X. (2012) · 2012 IEEE 12th International Conference on Data Mining · doi:10.1109/icdm.2012.45 · Cited by 300 · see citing works
Measuring discrimination in algorithmic decision making
Žliobaitė, I. (2017) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-017-0506-1 · Cited by 272 · see citing works
Timmons, A. C., Duong, J. B., Simo Fiallo, N., Lee, T., Vo, H. P. Q., Ahle, M. W., Comer, J. S., Brewer, L. C., Frazier, S. L., & Chaspari, T. (2023) · Perspectives on Psychological Science · doi:10.1177/17456916221134490 · Cited by 263 · see citing works
Classification with Fairness Constraints: A Meta-Algorithm with Provable Guarantees
Celis, L. E., Huang, L., Keswani, V., & Vishnoi, N. K. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287586 · Cited by 231 · see citing works
Grgic-Hlaca, N., Redmiles, E. M., Gummadi, K. P., & Weller, A. (2018) · Proceedings of the 2018 World Wide Web Conference on World Wide Web - WWW '18 · doi:10.1145/3178876.3186138 · Cited by 207 · see citing works
Fairness in Learning: Classic and Contextual Bandits
Joseph, M., Kearns, M., Morgenstern, J., & Roth, A. (2016) · arXiv · doi:10.48550/arxiv.1605.07139 · Cited by 199 · see citing works
“…otherwise known as equality of outcomes , statistical parity – or algorithms that avoid disparate impact (see e.g. Calders and Verwer [2010], Luong et al. [2011], Kamishima et al. [2011], Feldman et al. [2015], Fish et al. [2016] and Adler et al. [2016] for a study of auditing existing…”
BackgroundConscientious Classification: A Data Scientist's Guide to Discrimination-Aware Classification
d'Alessandro, B., O'Neil, C., & LaGatta, T. (2017) · Big Data · doi:10.1089/big.2016.0048 · Cited by 195 · see citing works
FairTest: Discovering Unwarranted Associations in Data-Driven Applications
Tramèr, F., Atlidakis, V., Geambasu, R., Hsu, D., Hubaux, J.-P., Humbert, M., Juels, A., & Lin, H. (2015) · arXiv · doi:10.48550/arxiv.1510.02377 · Cited by 171 · see citing works
FairTest: Discovering Unwarranted Associations in Data-Driven Applications
Tramer, F., Atlidakis, V., Geambasu, R., Hsu, D., Hubaux, J.-P., Humbert, M., Juels, A., & Lin, H. (2017) · 2017 IEEE European Symposium on Security and Privacy (EuroS&P) · doi:10.1109/eurosp.2017.29 · Cited by 171 · see citing works
Investigating the Impact of Gender on Rank in Resume Search Engines
Chen, L., Ma, R., Hannák, A., & Wilson, C. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174225 · Cited by 155 · see citing works
Interventional Fairness: Causal Database Repair for Algorithmic Fairness
Salimi, B., Rodriguez, L., Howe, B., & Suciu, D. (2019) · Proceedings of the 2019 International Conference on Management of Data · doi:10.1145/3299869.3319901 · Cited by 145 · see citing works
Towards trustworthy LLMs: a review on debiasing and dehallucinating in large language models
Lin, Z., Guan, S., Zhang, W., Zhang, H., Li, Y., & Zhang, H. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10896-y · Cited by 141 · see citing works
Fairness in Machine Learning: A Survey
Caton, S., & Haas, C. (2020) · arXiv · doi:10.48550/arxiv.2010.04053 · Cited by 646 · see citing works
Žliobaitė, I., & Custers, B. (2016) · Artificial Intelligence and Law · doi:10.1007/s10506-016-9182-5 · Cited by 125 · see citing works
Learning Optimal and Fair Decision Trees for Non-Discriminative Decision-Making
Aghaei, S., Azizi, M. J., & Vayanos, P. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33011418 · Cited by 135 · see citing works
Human Decision Making with Machine Assistance: An Experiment on Bailing and Jailing
Grgić-Hlača, N., Engel, C., & Gummadi, K. P. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359280 · Cited by 113 · see citing works
Kamiran, F., Žliobaitė, I., & Calders, T. (2013) · Knowledge and Information Systems · doi:10.1007/s10115-012-0584-8 · Cited by 99 · see citing works
Discrimination- and privacy-aware patterns
Hajian, S., Domingo-Ferrer, J., Monreale, A., Pedreschi, D., & Giannotti, F. (2015) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-014-0393-7 · Cited by 93 · see citing works
Quantifying the Impact of User Attentionon Fair Group Representation in Ranked Lists
Sapiezynski, P., Zeng, W., E Robertson, R., Mislove, A., & Wilson, C. (2019) · Companion Proceedings of The 2019 World Wide Web Conference · doi:10.1145/3308560.3317595 · Cited by 92 · see citing works
A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle
Suresh, H., & Guttag, J. (2021) · Equity and Access in Algorithms, Mechanisms, and Optimization · doi:10.1145/3465416.3483305 · Cited by 82 · see citing works
FairLens: Auditing black-box clinical decision support systems
Panigutti, C., Perotti, A., Panisson, A., Bajardi, P., & Pedreschi, D. (2021) · Information Processing & Management · doi:10.1016/j.ipm.2021.102657 · Cited by 86 · see citing works
A survey on datasets for fairness‐aware machine learning
Le Quy, T., Roy, A., Iosifidis, V., Zhang, W., & Ntoutsi, E. (2022) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1452 · Cited by 261 · see citing works
Anti-discrimination learning: a causal modeling-based framework
Zhang, L., & Wu, X. (2017) · International Journal of Data Science and Analytics · doi:10.1007/s41060-017-0058-x · Cited by 48 · see citing works
Berendt, B., & Preibusch, S. (2014) · Artificial Intelligence and Law · doi:10.1007/s10506-013-9152-0 · Cited by 44 · see citing works
Exploring Discrimination: A User-centric Evaluation of Discrimination-Aware Data Mining
Berendt, B., & Preibusch, S. (2012) · 2012 IEEE 12th International Conference on Data Mining Workshops · doi:10.1109/icdmw.2012.109 · Cited by 24 · see citing works
Learning Fair Naive Bayes Classifiers by Discovering and Eliminating Discrimination Patterns
Choi, Y., Farnadi, G., Babaki, B., & Van den Broeck, G. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i06.6565 · Cited by 5 · see citing works
- Authors
- Luong, B. T., Ruggieri, S., & Turini, F.
- Affiliations
- IMT School for Advanced Studies Lucca (IT)
- University of Pisa (IT)
- Venue
- Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining
- Publisher
- ACM
- Published
- 2011-08-21
- Pages
- 502-510
- OpenAlex
- W2059141064
- Cited by
- 263
- References
- 25
- Citations by year
- 2026: 2 · 2025: 18 · 2024: 20 · 2023: 28 · 2022: 20 · 2021: 23 · 2020: 29 · 2019: 28 · 2018: 16 · 2017: 24 · 2016: 17 · 2015: 9 · 2014: 7 · 2013: 7 · 2012: 13
- FWCI
- 61.86
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) (publishedVersion)
- UnipiEprints Open Archive (Università di Pisa) (acceptedVersion)
- CINECA IRIS Institutial research information system (University of Pisa) (submittedVersion)
- ISTI Open Portal (submittedVersion)
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Social Sciences › Law › Discrimination and Equality Law
- Social Sciences › Political Science and International Relations › Artificial Intelligence in Law
- Keywords
- k-nearest neighbors, algorithmic discrimination, discrimination prevention, protected groups, algorithmic fairness, legal methodology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
