Protected Grounds and the System of Non-Discrimination Law in the Context of Algorithmic Decision-Making and Artificial Intelligence
Open accessThe DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index.
Abstract
Algorithmic decision-making and similar types of artificial intelligence (AI) may lead to improvements in all sectors of society, but can also have discriminatory effects. While current non-discrimination law offers people some protection, algorithmic decision-making presents the law with several challenges. For instance, algorithms can generate new categories of people based on seemingly innocuous characteristics, such as web browser preference or apartment number, or more complicated categories combining many data points. Such new types of differentiation could evade non-discrimination law, as browser type and house number are not protected characteristics, but such differentiation could still be unfair, for instance if it reinforces social inequality. This paper explores which system of non-discrimination law can best be applied to algorithmic decision-making, considering that algorithms can differentiate on the basis of characteristics that do not correlate with protected grounds of discrimination such as ethnicity or gender. The paper analyses the current loopholes in the protection offered by non-discrimination law and explores the best way for lawmakers to approach algorithmic differentiation. While we focus on Europe, the conceptual and theoretical focus of the paper can make it useful for scholars and policymakers from other regions too, as they encounter similar problems with algorithmic decision-making.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Issues of discrimination and equality before the law occupy a growing place in discussions of algorithmic administration, as data-driven systems have been shown to reproduce or intensify unequal treatment across protected groups (Gerards and Zuiderveen Borgesius, 2020; Zuiderveen Borgesius, 2020).”
From Judicial review of algorithmic administrative systems legality evidence and remedies in the smart city state (2026) · cites this paper as Gerards and Zuiderveen Borgesius, 2020
In the citing paper
“Addressing upstream legality also responds to equality concerns, since proxy discrimination often originates in feature selection or threshold design rather than in explicit decision rules (Gerards and Zuiderveen Borgesius, 2020; Zuiderveen Borgesius, 2020).”
From Judicial review of algorithmic administrative systems legality evidence and remedies in the smart city state (2026) · cites this paper as Gerards and Zuiderveen Borgesius, 2020
In the citing paper
“Non-discrimination law has become a major focus because algorithmic systems can generate biased outcomes through data patterns, proxy variables, and unequal error rates across protected groups (Zuiderveen Borgesius, 2020; Gerards and Zuiderveen Borgesius, 2020).”
From Judicial review of algorithmic administrative systems legality evidence and remedies in the smart city state (2026) · cites this paper as Gerards and Zuiderveen Borgesius, 2020
In the citing paper
“System-focused remedies also address equality concerns, since group-level metrics may reveal disparate error patterns that single-case review cannot detect (Gerards and Zuiderveen Borgesius, 2020; Weerts et al., 2023).”
From Judicial review of algorithmic administrative systems legality evidence and remedies in the smart city state (2026) · cites this paper as Gerards and Zuiderveen Borgesius, 2020
In the citing paper
“Security-related uses trigger heightened scrutiny, particularly where systems affect mobility, identity, or access to essential services (Gerards and Zuiderveen Borgesius, 2020; Czarnocki and Pałka, 2024).”
From Judicial review of algorithmic administrative systems legality evidence and remedies in the smart city state (2026) · cites this paper as Gerards and Zuiderveen Borgesius, 2020
Citations
Cites
Barocas, S., & Selbst, A. D. (2014) · doi:10.2139/ssrn.2477899 · Cited by 3,048
Stanford Encyclopedia of Philosophy
Oberhelman, D. D. (2013) · Reference Reviews · doi:10.1108/rr.2001.15.6.9.311 · Cited by 1,477
Puhl, R. M., Andreyeva, T., & Brownell, K. D. (2008) · International Journal of Obesity · doi:10.1038/ijo.2008.22 · Cited by 900
Democracy and Distrust: A Theory of Judicial Review
Cox, A., & Ely, J. H. (1981) · Harvard Law Review · doi:10.2307/1340684 · Cited by 801
Vulnerable groups: The promise of an emerging concept in European Human Rights Convention law
Peroni, L., & Timmer, A. (2013) · International Journal of Constitutional Law · doi:10.1093/icon/mot042 · Cited by 354
Freedom of Religion under the European Convention on Human Rights
Evans, C. (2001) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780199243648.001.0001 · Cited by 305
Westen, P. (1982) · Harvard Law Review · doi:10.2307/1340593 · Cited by 288
Jolls, C., & Sunstein, C. R. (2006) · California Law Review · doi:10.2307/20439057 · Cited by 177
Toward an Anti-Stereotyping Approach for the European Court of Human Rights
Timmer, A. (2011) · Human Rights Law Review · doi:10.1093/hrlr/ngr036 · Cited by 149
Cinderella comes to the Ball: Art 14 and the right to non-discrimination in the ECHR
O’Connell, R. (2009) · Legal Studies · doi:10.1111/j.1748-121x.2009.00119.x · Cited by 74
Research works printed but not linked: 95
- Latanya Sweeney, 2013, Discrimination in OnlineAd Delivery, CoMM. ACM
- Latanya Sweeney, 2013, Discriminationin Online Ad Delivery, CoMM. ACM
- Latanya Sweeney, 2013, Discriminationin Online Ad Delivery, CoMM. ACM
- Latanya Sweeney, 2013, Discrimination in OnlineAd Delivery, CoMM. ACM
- Jennifer Valentino-Devries, 2012, Trust but Verify: A Guide to Algorithms and the Law, 31 HARV, J. L. & TECH
- We leave aside, for now, the question whether the bank's behaviour is socially desirable. We merely want to illustrate that algorithmic price differentiation can reinforce financial inequality
- David Caplovitz, 1963, CONSUMER PRACTICES OF LOW-INCOME FAMILIES
- Virginia Eubanks, 2016, See also Tal Zarsky, The Trouble with Algorithmic Decisions: An Analytic Road Map to Examine Efficiency and Fairness in Automated and Opaque Decision Making, 41 SCI, TECH. & HUM. VALUES, 118, 124
- 2013, The ECtHR's caselaw on the topic is not consistent, however; see Janneke H. Gerards, The Discrimination Grounds of Article 14 of the European Convention on Human Rights, 13 HUM, RTS. L. REV
- See Also Aalt Willem, 1999, Standards of Review for Discrimination: The Scope of Review by the Courts, NON-DISCRIMINATION LAW: COMPARATIVE PERSPECTIVES, 25
- As discussed supra
- Christa See Generally, A CASE STUDY INTO THE DEVELOPMENT OF THE LEGAL CONCEPT OF INDIRECT DISCRIMINATION UNDER EC LAW
- 2005, SOCIAL EUROPE SERIES
- Grounds of Discrimination -The Desirability of Open and Closed Lists of Grounds, 129
- Elisabeth Holzleithner, 2004, Dis/Entangling Grounds of Discrimination, 927
- 2016, Amending or Retaining Article 1 of the Netherlands Constitution: In Case of Doubt, Do Not Pass, Titia Loenen, Wijzigen of handhaven artikel 1 Grondwet: Bij twijfel niet inhalen
- Titia Loenen, 2016, Amending or Retaining Article 1 of the Netherlands Constitution: In Case of Doubt, Do Not Pass, Wijzigen of handhaven artikel 1 Grondwet: Bij twijfel niet inhalen
- Titia Loenen, 2016, Amending or Retaining Article 1 of the Netherlands Constitution: In Case of Doubt, Do Not Pass, Wijzigen of handhaven artikel 1 Grondwet: Bij twijfel niet inhalen
- Titia Loenen, 2016, Amending or Retaining Article 1 of the Netherlands Constitution: In Case of Doubt, Do Not Pass, Wijzigen of handhaven artikel 1 Grondwet: Bij twijfel niet inhalen
- On the importance of this symbolic value for protecting against discrimination
- See Janneke, 2008, Gronden van discriminatie -de wenselijkheid van open en gesloten opsommingen [Grounds of Discrimination -The Desirability of Open and Closed Lists of Grounds, 135
- H Janneke, 2016, Artikel 1 Grondwet -goede gronden voor wijziging? [Article 1 of the Netherlands Constitution -Good Grounds for Change?
- 2012, Jakob Cornides, Three Case Studies on 'Anti-Discrimination', 23 EUR, J. INT'L L, 517
- 1969, For an example of easy acceptance of 'biological' inequalities between men and women, see e, THE CONCEPT OF EQUALITY, 54, 64
- 2012, In more detail, see Jakob Cornides, Three Case Studies on 'Anti-Discrimination', 23 EUR
- 2005, there is a debate on whether appearance or weight should perhaps be considered protected grounds of discrimination; see e.g. Deborah Carr & Michael A. Friedman, Is Obesity Stigmatizing? Body Weight, Perceived Discrimination, and Psychological Well-Being in the United States, J. HEALTH AND SOC. BEHAV, 46
- On 'suspect' grounds, see infra
- Comparing the semi-closed lists of grounds of the two provisions shows that 'ethnic origin' (next to 'race' and 'national origin'), 'belief' (next to 'religion'), 'sexual orientation', 'disability', 'age' and 'genetic features' have been added
- 2020, Andrew Altman, Discrimination, THE STANFORD ENCYCLOPEDIA OF PHILOSOPHY � 7
- Andrew Altman, 2020, Discrimination, THE STANFORD ENCYCLOPEDIA OF PHILOSOPHY
- 2007, This is even more true in the EU Non-Discrimination Directives, many of which are limited to a number of particular grounds, such as either gender or race/ethnicity; see e.g. Janneke H. Gerards, Discrimination Grounds, CASES, MATERIALS AND TEXT ON NATIONAL, SUPRANATIONAL AND INTERNATIONAL NON-DISCRIMINATION LAW, 33
- 2007, This is clearly true for the EU Non-Discrimination Directives with their closed system of exemptions, but problematic examples also can be found in national law; see Janneke H. Gerards, Discrimination Grounds, CASES, MATERIALS AND TEXT ON NATIONAL, SUPRANATIONAL AND INTERNATIONAL NON-DISCRIMINATION LAW, 33
- 2008, Approaching the Intersection of Sex, Religion and Race under the European Convention on Human Rights and EC Equality Law, in EUROPEAN UNION NON-DISCRIMINATION LAW, Titia Loenen, 313
- 2018, EUROPEAN FUNDAMENTAL RIGHTS AGENCY & EUROPEAN COURT OF HUMAN RIGHTS
- Timo Makkonen, 2002, BRINGING THE EXPERIENCES OF THE MOST MARGINALIZED TO THE FORE, 36
- H Janneke, 2007, Discrimination Grounds, CASES, MATERIALS AND TEXT ON NATIONAL, SUPRANATIONAL AND INTERNATIONAL NON-DISCRIMINATION LAW, 33
- H Janneke, 2008, Gronden van discriminatie -de wenselijkheid van open en gesloten opsommingen [Grounds of Discrimination -The Desirability of Open and Closed Lists of Grounds, 141
- 2017, For further criticism of the indirect discrimination approach taken in the Achbita v. G4S Secure Solutions NV case, see e.g. Eva Brems, European Court of Justice Allows Bans on Religious Dress in the Workplace, Springer 2020), 135
- Eleanor Spaventa, 2017, The approach was received more favorably by Mark Bell, Leaving Religion at the Door? The European Court of Justice and Religious Symbols in the Workplace, What is the Point of Minimum Harmonization of Fundamental Rights? Some Further Reflections on the Achbita Case
- See Also Aalt Willem, 1999, Standards of Review for Discrimination: The Scope of Review by the Courts, NON-DISCRIMINATION LAW: COMPARATIVE PERSPECTIVES, 25
- H Janneke, 2008, Gronden van discriminatie -de wenselijkheid van open en gesloten opsommingen [Grounds of Discrimination -The Desirability of Open and Closed Lists of Grounds, 138
- 1998, See e.g. Janneke H. Gerards, Nieuwe ronde, nieuwe kansen: naar een 'semi-open' systeem van gelijkebehandelingswetgeving?, -85 of the (then) Dutch Equal Treatment Commission (since 2012 the Netherlands Human Rights Institute
- 2015, Geoffrey L�ger v. Ministre des Affaires sociales, de la Sant� et des Droits des femmes
- 2020, See Decisions 1998-137 and 2007-85 of the (then) Dutch Equal Treatment Commission (since 2012 the Netherlands Human Rights Institute
- Janneke H Gerards, 2011, Nieuwe ronde, nieuwe kansen: naar een 'semi-open' systeem van gelijkebehandelingswetgeving? [New Round, New Chances: Toward a 'Semi-Open' System of Equal Treatment Law?] 36 NEDERLANDS TIJDSCHRIFT VOOR DE MENSENRECHTEN, NETH. J. HUM. RTS
- 2020, See Decisions 1998-137 and 2007-85 of the (then) Dutch Equal Treatment Commission (since 2012 the Netherlands Human Rights Institute
- 2020, Decisions 1998-137 and 2007-85 of the (then) Dutch Equal Treatment Commission (since 2012 the Netherlands Human Rights Institute
- 2020, Decisions 1998-137 and 2007-85 of the (then) Dutch Equal Treatment Commission (since 2012 the Netherlands Human Rights Institute
- See Janneke, 2011, Nieuwe ronde, nieuwe kansen: naar een 'semi-open' systeem van gelijkebehandelingswetgeving? [New Round, New Chances: Toward a 'Semi-Open' System of Equal Treatment Law?] 36 NEDERLANDS TIJDSCHRIFT VOOR DE MENSENRECHTEN, NETH. J. HUM. RTS
- 2000, See also e.g. Dagmar Schiek, A New Framework on Equal Treatment of Persons in EC Law? Directives, 290
- 2000, ); and see Dagmar Schiek, A New Framework on Equal Treatment of Persons in EC Law? Directives, RTS. L. REV, 99
- 2011, App. No. 56328/07, � 45, 1
- 2020, Andrew Altman, Discrimination, THE STANFORD ENCYCLOPEDIA OF PHILOSOPHY �, 4
- 2013, As is witnessed by the ECtHR deeming this ground "suspect"; see e, 36
- Peter Petkoff, 1997, For the application of these notions to the context of choice/nonchoice grounds, Forum Internum and Forum Externum in Canon Law and Public International Law with a Particular Reference to the Jurisprudence of the European Court of Human Rights, 7 RELIGION & HUM. RTS, 183
- Pilvi R�m�, 2019, Towards the Core of Conscience. A Study on Conscientious Objection in the Workplace Under the Freedom of Thought, Conscience and Religion
- 1993, ) and Dagmar Schiek, A New Framework on Equal Treatment of Persons in EC Law? Directives, 36516
- E G Pilvi R�m�, 2019, Towards the Core of Conscience. A Study on Conscientious Objection in the Workplace Under the Freedom of Thought, Conscience and Religion
- Carolyn Evans, 2001, HUMAN RIGHTS, 76
- See Pilvi R�m�, 2019, Towards the Core of Conscience. A Study on Conscientious Objection in the Workplace Under the Freedom of Thought, Conscience and Religion
- H Janneke, 2008, For a further discussion of different types of religious expression, see e.g. Javier Mart�nez-Torr�n, Manifestations of Religion or Belief in the Case Law of the European Court of Human Rights, 12 RELIGION & HUM, RTS, 147, 120
- 2013, On headscarves, 10
- C- Case, 2017, Achbita v. G4S Secure Solutions NV
- 2017, On blasphemous speech, 38450, 1
- See Janneke, 2008, Gronden van discriminatie -de wenselijkheid van open en gesloten opsommingen [Grounds of Discrimination -The Desirability of Open and Closed Lists of Grounds, 147
- H Janneke, 2008, Gronden van discriminatie -de wenselijkheid van open en gesloten opsommingen [Grounds of Discrimination -The Desirability of Open and Closed Lists of Grounds, 147
- 2020, Andrew Altman, Discrimination, THE STANFORD ENCYCLOPEDIA OF PHILOSOPHY �, 4
- 2007, Christopher McCrudden & Haris Kountouros, Human Rights and European Equality Law, EQUALITY LAW IN AN ENLARGED EUROPEAN UNION. UNDERSTANDING THE ARTICLE 13 DIRECTIVES, 4
- 2002, See also TIMO MAKKONEN, MULTIPLE, COMPOUND AND INTERSECTIONAL DISCRIMINATION: BRINGING THE EXPERIENCES OF THE MOST MARGINALIZED TO THE FORE 2 (Institute For Human Rights
- 2015, PROTECTING VULNERABLE GROUPS: THE EUROPEAN HUMAN RIGHTS FRAMEWORK
- Valeska David, 2018, Crossing Divides and Seeking the Whole: An Integrated View of Cultural Difference and Economic Disadvantage in Regional, Human Rights Courts
- For a different, more individualised type of conceptualisation, see Martha A. Fineman, Equality, Autonomy, and the Vulnerable Subject in Law and Politics, VULNERABILITY: REFLECTIONS ON A NEW ETHICAL FOUNDATION FOR LAW AND POLITICS, 20
- 2020, See also the overview of different rationales in Andrew Altman, Discrimination, 4
- However, some recurring problems in open systems are the risk of judicial bias, and the lack of legal certainty
- See supra
- See Janneke, 2008, Gronden van discriminatie -de wenselijkheid van open en gesloten opsommingen [Grounds of Discrimination -The Desirability of Open and Closed Lists of Grounds, 153
- H Janneke, 2008, Gronden van discriminatie -de wenselijkheid van open en gesloten opsommingen [Grounds of Discrimination -The Desirability of Open and Closed Lists of Grounds, 153
- See Janneke, 2013, The Discrimination Grounds of Article 14 of the European Convention on Human Rights, 13 HUM, RTS. L. REV, 99
- 2007, ) and Christopher McCrudden & Haris Kountouros, Human Rights and European Equality Law, EQUALITY LAW IN AN ENLARGED EUROPEAN UNION. UNDERSTANDING THE ARTICLE 13 DIRECTIVES, 73
- See Janneke, 2013, The Discrimination Grounds of Article 14 of the European Convention on Human Rights, 13 HUM, RTS. L. REV, 99
- Samantha Besson, 2008, Never Shall the Twain Meet?, 8 HUM, 647, 653
- H Janneke, 2013, The Discrimination Grounds of Article 14 of the European Convention on Human Rights, 13 HUM, RTS. L. REV, 99
- H Janneke, 2013, The Discrimination Grounds of Article 14 of the European Convention on Human Rights, 13 HUM, RTS. L. REV, 99
- H Janneke, 2013, The ground of discrimination may certainly be relevant to the intensity of review and the applicability of the very weighty reasons test, The Discrimination Grounds of Article 14 of the European Convention on Human Rights, 13 HUM. RTS. L. REV. 99, 117
- Erwin Chemerinsky, 1983, Defense of Equality: A Reply to Professor Westen
- H Janneke, 2008, Gronden van discriminatie -de wenselijkheid van open en gesloten opsommingen [Grounds of Discrimination -The Desirability of Open and Closed Lists of Grounds, 154
- H Janneke, 2008, Gronden van discriminatie -de wenselijkheid van open en gesloten opsommingen [Grounds of Discrimination -The Desirability of Open and Closed Lists of Grounds, 154
- See Janneke, 2008, Gronden van discriminatie -de wenselijkheid van open en gesloten opsommingen [Grounds of Discrimination -The Desirability of Open and Closed Lists of Grounds, 155
- H See Also Janneke, 2008, Gronden van discriminatie -de wenselijkheid van open en gesloten opsommingen [Grounds of Discrimination -The Desirability of Open and Closed Lists of Grounds, 129
- H Janneke, 2008, Gronden van discriminatie -de wenselijkheid van open en gesloten opsommingen [Grounds of Discrimination -The Desirability of Open and Closed Lists of Grounds, 166
- H See Janneke, Nieuwe ronde, nieuwe kansen: naar een 'semi-open' systeem van gelijkebehandelingswetgeving?, New Round, New Chances: Toward a 'Semi-Open' System of Equal Treatment Law?] 36 NEDERLANDS TIJDSCHRIFT VOOR DE MENSENRECHTEN
- See supra, Section 5.1
- H Janneke, 2008, Gronden van discriminatie -de wenselijkheid van open en gesloten opsommingen [Grounds of Discrimination -The Desirability of Open and Closed Lists of Grounds, 166
- H Janneke, Gronden van discriminatie -de wenselijkheid van open en gesloten opsommingen [Grounds of Discrimination -The Desirability of Open and Closed Lists of Grounds, 129
- 2008, Critically on the current set-up of the EU non-discrimination directives, see also e.g. Dagmar Schiek, Broadening the Scope and the Norms of
Cited by
Policy advice and best practices on bias and fairness in AI
Álvarez, J. M., Colmenarejo, A. B., Elobaid, A., Fabbrizzi, S., Fahimi, M., Ferrara, A., Ghodsi, S., Mougan, C., Papageorgiou, I. T., Lobo, P. R., et al. (2024) · Ethics and Information Technology · doi:10.1007/s10676-024-09746-w · Cited by 160
Considering a Unified Model of Artificial Intelligence Enhanced Social Work: A Systematic Review
Garkisch, M., & Goldkind, L. (2024) · Journal of Human Rights and Social Work · doi:10.1007/s41134-024-00326-y · Cited by 89
“Especially in the datified society, discrimination can be amplified and codified in automated systems (Gerards & Borgesius, 2022; Heinrichs, 2022).”
Strycharz, J., & Duivenvoorde, B. B. (2021) · Internet Policy Review · doi:10.14763/2021.4.1585 · Cited by 59
Mapping the Road of the Ethical Dilemmas Behind Artificial Intelligence
Vousinas, G. L., Simitsi, I., Livieri, G., Gkouva, G. C., & Efthymiou, I.-P. (2022) · Journal of Politics and Ethics in New Technologies and AI · doi:10.12681/jpentai.31238 · Cited by 25
Ma, W. (2022) · Mobile Information Systems · doi:10.1155/2022/4636485 · Cited by 18
Metadata
- Authors
- Gerards, J., & Zuiderveen Borgesius, F.
- Affiliations
- Utrecht University (NL)
- ORCID
- Gerards, Janneke
- Venue
- SSRN Electronic Journal
- Publisher
- Elsevier BV
- Published
- 2021
- ISSN
- 1556-5068 (ISSN-L)
- OpenAlex
- W3144953237
- Cited by
- 72
- References
- 32
- Citations by year
- 2026: 2 · 2025: 6 · 2024: 10 · 2023: 48 · 2022: 3 · 2021: 3
- Type
- posted content
- Language
- en
- Open access
- green
- doi.org
- Locations
- SSRN Electronic Journal (acceptedVersion)
- Colorado Law Scholarly Commons (University of Colorado Colorado Springs) (submittedVersion)
- Utrecht University Repository (Utrecht University) (submittedVersion, other-oa)
- Radboud Repository (Radboud University) (submittedVersion)
- Radboud Repository (Radboud University) (submittedVersion, other-oa)
- Topics
- Social Sciences › Law › Discrimination and Equality Law
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Law › Digitalization, Law, and Regulation
- Keywords
- algorithmic decision-making, artificial intelligence, anti-discrimination law, protected grounds, algorithmic discrimination, automatic differentiation, social inequality, legal loopholes, European Union law, algorithmic fairness
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.