Integrating induction and deduction for finding evidence of discrimination

Pedreschi, Dino; Ruggieri, Salvatore; Turini, Franco

doi:10.1145/1568234.1568252

Cited by 55Ethics and Social Impacts of AIArtificial Intelligence in LawExplainable Artificial Intelligence (XAI)

Integrating induction and deduction for finding evidence of discrimination

Pedreschi, Dino, Ruggieri, Salvatore, Turini, Franco·2009·Proceedings of the 12th International Conference on Artificial Intelligence and Law

DOI 10.1145/1568234.1568252

Open this paper in CoChat — free account

Abstract

Automatic Decision Support Systems (DSS) are widely adopted for screening purposes in socially sensitive tasks, including access to credit, mortgage, insurance, labor market and other benefits. While less arbitrary decisions can potentially be guaranteed, automatic DSS can still be discriminating in the socially negative sense of resulting in unfair or unequal treatment of people. We present a reference model for finding (prima facie) evidence of discrimination in automatic DSS which is driven by a few key legal concepts. First, frequent classification rules are extracted from the set of decisions taken by the DSS over an input pool dataset. Key legal concepts are then used to drive the analysis of the set of classification rules, with the aim of discovering patterns of discrimination. We present an implementation, called LP2DD, of the overall reference model integrating induction, through data mining classification rule extraction, and deduction, through a computational logic implementation of the analytical tools.

Integrity signals

Registry

Crossref · 10.1145/1568234.1568252

Cites

66 of 66 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Measures such as quotas do not mandate perfect parity, as distributions of qualifications across groups can be unbalanced for legitimate, explainable reasons.
“In general these measures do not mandate perfect parity, as distributions of qualifications across groups can be unbalanced for legitimate, explainable reasons [29, 39].”
From FA*IR: A Fair Top-k Ranking Algorithm (2017) · cites this paper as [29]

Statistical Methods for Rates and Proportions
Fleiss, J. L., Levin, B., & Paik, M. C. (2003) · Wiley Series in Probability and Statistics · doi:10.1002/0471445428 · Cited by 18,646 · see citing works
UCI Repository of machine learning databases
Blake, C. L. (1998) · Medical Entomology and Zoology · Cited by 10,491 · see citing works
Fast Algorithms for Mining Association Rules in Large Databases
Agrawal, R., & Srikant, R. (1994) · Very Large Data Bases · Cited by 9,454 · see citing works
Introduction to Data Mining
Chiu, S., & Tavella, D. (2008) · Routledge eBooks · doi:10.4324/9780080878096-12 · Cited by 6,987 · see citing works
Categorical Data Analysis
Agresti, A. (2002) · Wiley Series in Probability and Statistics · doi:10.1002/0471249688 · Cited by 6,746 · see citing works
The Economics of Discrimination.
Collard, D. A., & Becker, G. S. (1972) · The Economic Journal · doi:10.2307/2230052 · Cited by 3,642 · see citing works
ACHIEVING k-ANONYMITY PRIVACY PROTECTION USING GENERALIZATION AND SUPPRESSION
Sweeney, L. (2002) · International Journal of Uncertainty Fuzziness and Knowledge-Based Systems · doi:10.1142/s021848850200165x · Cited by 1,991 · see citing works
Frequent pattern mining: current status and future directions
Han, J., Cheng, H., Xin, D., & Yan, X. (2007) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-006-0059-1 · Cited by 1,383 · see citing works
Introduction to Data Mining, (First Edition)
Tan, P., Steinbach, M. S., & Kumar, V. (2005) · Addison-Wesley Longman Publishing Co., Inc. eBooks · Cited by 1,305 · see citing works
Interestingness measures for data mining
Geng, L., & Hamilton, H. J. (2006) · ACM Computing Surveys · doi:10.1145/1132960.1132963 · Cited by 1,151 · see citing works
The Art of Prolog
Sterling, L., Shapiro, E., & Garrett, R. (1987) · IEEE Expert · doi:10.1109/mex.1987.4307074 · Cited by 1,101 · see citing works
Statistical Classification Methods in Consumer Credit Scoring: A Review
Hand, D. J., & Henley, W. E. (1997) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.1997.00078.x · Cited by 1,042 · see citing works
Field Experiments of Discrimination in the Market Place
Riach, P. A., & Rich, J. (2002) · The Economic Journal · doi:10.1111/1468-0297.00080 · Cited by 911 · see citing works
Benchmarking state-of-the-art classification algorithms for credit scoring
Baesens, B., Van Gestel, T., Viaene, S., Stepanova, M., Suykens, J., & Vanthienen, J. (2003) · Journal of the Operational Research Society · doi:10.1057/palgrave.jors.2601545 · Cited by 904 · see citing works
CPAR: Classification based on Predictive Association Rules
Yin, X., & Han, J. (2003) · Proceedings of the 2003 SIAM International Conference on Data Mining · doi:10.1137/1.9781611972733.40 · Cited by 833 · see citing works
A survey of credit and behavioural scoring: forecasting financial risk of lending to consumers
Thomas, L. C. (2000) · International Journal of Forecasting · doi:10.1016/s0169-2070(00)00034-0 · Cited by 745 · see citing works
Discrimination-aware data mining
Pedreshi, D., Ruggieri, S., & Turini, F. (2008) · Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1401890.1401959 · Cited by 676 · see citing works
Self-citation
Test statistics and sample size formulae for comparative binomial trials with null hypothesis of non‐zero risk difference or non‐unity relative risk
Farrington, P., & Manning, G. (1990) · Statistics in Medicine · doi:10.1002/sim.4780091208 · Cited by 658 · see citing works
The Oxford University Press, 1468–192
(1922) · Notes and Queries · doi:10.1093/nq/s12-x.208.280b · Cited by 619 · see citing works
Simple and Effective Confidence Intervals for Proportions and Differences of Proportions Result from Adding Two Successes and Two Failures
Agresti, A., & Caffo, B. S. (2000) · The American Statistician · doi:10.1080/00031305.2000.10474560 · Cited by 571 · see citing works
Association rule hiding
Verykios, V. S., Elmagarmid, A. K., Bertino, E., Saygın, Y., & Dasseni, E. (2004) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2004.1269668 · Cited by 519 · see citing works
Data mining to improve personnel selection and enhance human capital: A case study in high-technology industry
Chien, C., & Chen, L. (2006) · Expert Systems with Applications · doi:10.1016/j.eswa.2006.09.003 · Cited by 458 · see citing works
Classifying without discriminating
Kamiran, F., & Calders, T. (2009) · 2009 2nd International Conference on Computer, Control and Communication · doi:10.1109/ic4.2009.4909197 · Cited by 409 · see citing works
Logic Programming with Prolog
Bramer, M. (2005) · Springer eBooks · doi:10.1007/1-84628-212-8 · Cited by 267 · see citing works
Evidence on Discrimination in Consumer Markets
Ýinger, J. (1998) · The Journal of Economic Perspectives · doi:10.1257/jep.12.2.23 · Cited by 257 · see citing works
The Neighborhood Distribution of Subprime Mortgage Lending
Calem, P. S., Gillen, K. C., & Wächter, S. M. (2004) · The Journal of Real Estate Finance and Economics · doi:10.1023/b:real.0000044020.67401.51 · Cited by 210 · see citing works
EU Anti-Discrimination Law
Ellis, E., & Watson, P. (2012) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780199698462.001.0001 · Cited by 200 · see citing works
Affirmative Action Around the World
Sowell, T. (2017) · Yale University Press eBooks · doi:10.12987/9780300128352 · Cited by 177 · see citing works
Measuring Discrimination in Socially-Sensitive Decision Records
Pedreschi, D., Ruggieri, S., & Turini, F. (2009) · SIAM International Conference on Data Mining · doi:10.1137/1.9781611972795.50 · Cited by 175 · see citing works
Self-citation
Affirmative action: What do we know?
Holzer, H. J., & Neumark, D. (2006) · Journal of Policy Analysis and Management · doi:10.1002/pam.20181 · Cited by 166 · see citing works
Efficient search for association rules
Webb, G. I. (2000) · ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/347090.347112 · Cited by 165 · see citing works
Group Rights and Discrimination in International Law
Lerner, N. (1990) · Brill | Nijhoff eBooks · doi:10.1163/9789004481633 · Cited by 147 · see citing works
Cases, Materials and Text on National, Supranational and International Non-Discrimination Law
Schiek, D., Bell, M., & Waddington, L. (2007) · Bloomsbury Publishing Plc eBooks · doi:10.5040/9781509955541 · Cited by 140 · see citing works
A hybrid rule – neural approach for the automation of legal reasoning in the discretionary domain of family law in Australia
Stranieri, A., Zeleznikow, J., Gawler, M., & Lewis, B. (1999) · Artificial Intelligence and Law · doi:10.1023/a:1008325826599 · Cited by 123 · see citing works
Template-Based Privacy Preservation in Classification Problems
Wang, K., Fung, B. C. M., & Yu, P. S. (2006) · Proceedings · doi:10.1109/icdm.2005.142 · Cited by 115 · see citing works
Racial Profiling, Insurance Style: Insurance Redlining and the Uneven Development of Metropolitan Areas
Squires, G. D. (2003) · Journal of Urban Affairs · doi:10.1111/1467-9906.t01-1-00168 · Cited by 102 · see citing works
Discrimination in the Credit and Housing Markets: Findings and Challenges
Dymski, G. A. (2006) · Edward Elgar Publishing eBooks · doi:10.4337/9781847200150.00016 · Cited by 96 · see citing works
Sex Discrimination in Labor Markets: The Role of Statistical Evidence: Reply
Kuhn, P. (2016) · American Economic Review · Cited by 93 · see citing works
The Role of Logic in Computational Models of Legal Argument: A Critical Survey
Prakken, H., & Giovanni Sartor, B. (2002) · Lecture notes in computer science · doi:10.1007/3-540-45632-5_14 · Cited by 91 · see citing works
An Alternative Approach to Mining Association Rules.
Rauch, J., & Šimůnek, M. (2005) · Cited by 81 · see citing works
An exact confidence set for two binomial proportions and exact unconditional confidence intervals for the difference and ratio of proportions
Reiczigel, J., Abonyi‐Tóth, Z., & Singer, J. (2008) · Computational Statistics & Data Analysis · doi:10.1016/j.csda.2008.04.032 · Cited by 71 · see citing works
The Economics of Discrimination: Economists Enter the Courtroom
Ashenfelter, O., & Oaxaca, R. L. (1997) · Edward Elgar Publishing eBooks · doi:10.4337/9781035377541.00014 · Cited by 55 · see citing works
Comparisons of Confidence Intervals for Attributable Risk
Leung, H. M., & Kupper, L. L. (1981) · Biometrics · doi:10.2307/2530418 · Cited by 54 · see citing works
Indirect Discrimination in the Workplace
Hunter, R. (1992) · Kent Academic Repository (University of Kent) · Cited by 47 · see citing works
Sex Discrimination in Labor Markets: The Role of Statistical Evidenc e.
Kuhn, P. J. (n.d.) · RePEc: Research Papers in Economics · Cited by 45 · see citing works
Untitled work
(2008) · IB · Cited by 38 · see citing works
Statistical Reasoning in the Legal Setting
Gastwirth, J. L. (1992) · The American Statistician · doi:10.1080/00031305.1992.10475851 · Cited by 37 · see citing works
Classification Without Discrimination
Kamiran, F., & Calders, T. (2009) · Dépôt institutionnel de l'Université libre de Bruxelles (Université Libre de Bruxelles) · Cited by 36 · see citing works
The IKBALS project: Multi-modal reasoning in legal knowledge based systems
Zeleznikow, J., Vossos, G., & Hunter, D. (1994) · Artificial Intelligence and Law · doi:10.1007/bf00871889 · Cited by 33 · see citing works
Statistical Methods for Analyzing Claims of Employment Discrimination
Gastwirth, J. L. (1984) · Industrial and Labor Relations Review · doi:10.1177/001979398403800108 · Cited by 31 · see citing works
Induction of defeasible logic theories in the legal domain
Johnston, B., & Governatori, G. (2003) · International Conference on Artificial Intelligence and Law · doi:10.1145/1047788.1047834 · Cited by 31 · see citing works
The evaluation of legal knowledge based systems
Stranieri, A., & Zeleznikow, J. (1999) · International Conference on Artificial Intelligence and Law · doi:10.1145/323706.323711 · Cited by 30 · see citing works
The Logic of Life
Harford, T. (2008) · Medical Entomology and Zoology · Cited by 25 · see citing works
Measuring discrimination : data collection and EU equality law
Makkonen, T. (2007) · IB · Cited by 25 · see citing works
Group Rights and Discrimination in International Law
Lerner, N. (1991) · Verfassung in Recht und Übersee · doi:10.5771/0506-7286-1991-4-461 · Cited by 23 · see citing works
Confidence intervals for the risk ratio under inverse sampling
Tian, M., Tang, M., Ng, H. K. T., et al. (2007) · Statistics in Medicine · doi:10.1002/sim.3158 · Cited by 21 · see citing works
Developing anti-discrimination law in Europe : the 27 EU member states compared
Chopin, I., & Gounari, E.-M. (2010) · Medical Entomology and Zoology · Cited by 19 · see citing works
Favoritism and reverse discrimination
Kim, K. (2005) · European Economic Review · doi:10.1016/j.euroecorev.2005.09.003 · Cited by 19 · see citing works
Statistical Methods in Discrimination Litigation.
Conway, D. A., Kaye, D. H., & Aickin, M. G. (1989) · Journal of the American Statistical Association · doi:10.2307/2290094 · Cited by 17 · see citing works
On Proving Discrimination: Statistical Methods and Unfolding Policy Logics
Knopff, R. (1986) · Canadian Public Policy · doi:10.2307/3550667 · Cited by 14 · see citing works
Suppressing Data Sets to Prevent Discovery of Association Rules
Hintoğlu, A. A., İnan, A., Saygın, Y., et al. (2006) · Proceedings · doi:10.1109/icdm.2005.140 · Cited by 12 · see citing works
Statistical Methods in Discrimination Litigation.
Marks, P., Kaye, D. H., & Aickin, M. G. (1988) · Journal of the Royal Statistical Society Series D (The Statistician) · doi:10.2307/2348388 · Cited by 10 · see citing works
Approaches for Dealing with Small Sample Sizes in Employment Discrimination Litigation
Piette, M. J., & White, P. F. (1999) · Journal of Forensic Economics · doi:10.5085/0898-5510-12.1.43 · Cited by 10 · see citing works
Reverse discrimination by the numbers
Lerner, R. E., & Nagai, A. K. (2000) · Academic Questions · doi:10.1007/s12129-000-1039-3 · Cited by 8 · see citing works
The economics of affirmative action
Nugent, R. A. (2002) · Routledge eBooks · doi:10.4324/9780203021149-13 · Cited by 5 · see citing works
Detecting and describing heterogeneity in meta-analysis
Hardy, R. J., & Thompson, S. G. (1998) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19980430)17:8

Flagged first, then by citations

Sorted by citations

Certifying and Removing Disparate Impact
Feldman, M., Friedler, S. A., Moeller, J., Scheidegger, C., & Venkatasubramanian, S. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2783311 · Cited by 1,827 · see citing works
FA*IR: A Fair Top-k Ranking Algorithm
Zehlike, M., Bonchi, F., Castillo, C., Hajian, S., Megahed, M., & Baeza-Yates, R. (2017) · Proceedings of the 2017 ACM on Conference on Information and Knowledge Management · doi:10.1145/3132847.3132938 · Cited by 496 · see citing works
A Methodology for Direct and Indirect Discrimination Prevention in Data Mining
Hajian, S., & Domingo-Ferrer, J. (2013) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2012.72 · Cited by 312 · see citing works
Educating software and ai stakeholders about algorithmic fairness, accountability, transparency and ethics
Bogina, V., Hartman, A., Kuflik, T., & Shulner-Tal, A. (2022) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-021-00248-0 · Cited by 142 · see citing works
Discrimination- and privacy-aware patterns
Hajian, S., Domingo-Ferrer, J., Monreale, A., Pedreschi, D., & Giannotti, F. (2015) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-014-0393-7 · Cited by 93 · see citing works
Self-citation
Better decision support through exploratory discrimination-aware data mining: foundations and empirical evidence
Berendt, B., & Preibusch, S. (2014) · Artificial Intelligence and Law · doi:10.1007/s10506-013-9152-0 · Cited by 44 · see citing works
Modeling Epistemological Principles for Bias Mitigation in AI Systems: An Illustration in Hiring Decisions
Vasconcelos, M., Cardonha, C., & Gonçalves, B. (2018) · Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3278721.3278751 · Cited by 38 · see citing works
Exploring Discrimination: A User-centric Evaluation of Discrimination-Aware Data Mining
Berendt, B., & Preibusch, S. (2012) · 2012 IEEE 12th International Conference on Data Mining Workshops · doi:10.1109/icdmw.2012.109 · Cited by 24 · see citing works

Authors: Pedreschi, D., Ruggieri, S., & Turini, F.
Affiliations: University of Pisa (IT)
ORCID: Pedreschi, Dino
Ruggieri, Salvatore
Turini, Franco
Venue: Proceedings of the 12th International Conference on Artificial Intelligence and Law
Publisher: ACM
Published: 2009-06-08
Pages: 157-166
DOI: 10.1145/1568234.1568252
OpenAlex: W2040620802
Cited by: 55
References: 66
Citations by year: 2025: 1 · 2024: 2 · 2023: 1 · 2022: 5 · 2021: 4 · 2020: 2 · 2019: 4 · 2018: 2 · 2017: 4 · 2016: 2 · 2015: 7 · 2014: 13 · 2013: 1 · 2012: 3
FWCI: 11.88
Type: conference paper
Language: en
Open access: closed
Licence: www.acm.org/publications/policies/copyright_policy#Background (vor)
Locations: International Conference on Artificial Intelligence and Law (publishedVersion)
CINECA IRIS Institutial research information system (University of Pisa) (submittedVersion)
Topics: Social Sciences › Safety Research › Ethics and Social Impacts of AI
Social Sciences › Political Science and International Relations › Artificial Intelligence in Law
Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
Keywords: algorithmic discrimination, classification rule extraction, legal concepts, computational logic

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Integrating induction and deduction for finding evidence of discrimination

Abstract

Integrity signals

Claims the paper makes

Claims other papers make about this one

Cites

Cited by

Related

Record

About these checks