What does it mean to 'solve' the problem of discrimination in hiring?: social, technical and legal perspectives from the UK on automated hiring systems

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The group definitions used by the three AHSs are binary gender, ethnicity (using US demographics), and age interval.

“Examples of group definitions are binary gender, ethnicity (using definitions informed specifically by US demographics) and age interval.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Pymetrics additionally uses statistical tests to compare group scores and passing rates.

“Additionally, Pymetrics uses statistical tests to compare group scores and passing rates which are a common way of comparing groups represented by samples.”

Classified as probably not a substantial claim.

HireVue's patent proposes to create a model that mitigates discrimination by modifying a standard objective function to equally account for prediction errors of all groups and penalize solutions that violate classification parity.

“HireVue, meanwhile, proposes in its patent to create a model that mitigates discrimination by modifying a standard objective function to equally account for the prediction errors of all the groups and also to penalize solutions that violate the classification parity constraint.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

HireVue's proposal is abstract and has no validation reports, making it difficult to analyse.

“However, this proposal is abstract and with no validation reports making it difficult to analyse.”

May combine two claims.

Social class and disabilities are not found in the examples or available validation tests of the three AHSs.

“Social class or disabilities are not found in either the examples nor in the available validation tests.”

May combine two claims.

Pymetrics validates fairness of each assessment test by performing multiple group comparisons across each attribute, but does not directly compare multiple axes of group identification.

“Pymetrics validates fairness of each assessment test by performing multiple group comparisons across each attribute, e.g.comparing its determined racial groups scorings.Multiple axes of group identification are not directly compared, e.g.hispanic women with white men.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Another UK system used widely, Thrive Map, makes no claims as to bias mitigation at all and is therefore not included in the paper.

“Interestingly, also, another UK system used widely, Thrive Map makes no claims as to bias mitigation at all and is therefore not included in this paper.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Other UK systems were considered but either made no claims as to debiasing, or did not make available sufficient public material to analyse.

“Other UK systems were considered but either made no claims as to debiasing, or did not make available sufficient public material to analyse.”

May combine two claims.

The two US-originated systems (Pymetrics and HireVue) make use of the 4/5ths rule, while the UK system discussed (Applied) does not.

“It is noticeable that the two US-originated systems make use of the 4/5ths rule while the UK system discussed (Applied) does not.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

HireVue performs bias detection by measuring demographic parity as defined by the US EEOC.

“Bias detection is performed by measuring demographic parity as defined by the US EEOC.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Applied provides aggregated analytics to evaluate the whole process to visually detect biases at different stages, rather than performing formal statistical tests.

“Rather than performing formal statistical tests, the platform provides aggregated analytics to evaluate the whole process to visually detect biases at different stages (see Figure 3).”

Classified as probably not a substantial claim. May combine two claims.

The three AHSs were selected because they are known to be used in the UK and, unlike many competitors, have publicly available information to inform an evaluation of their approach.

“These AHSs were selected as they are known to be used in the UK, and, unlike many of their competitors, there is publicly available information to inform an evaluation of their approach.”

May combine two claims.

HireVue's bias mitigation evaluates adverse impact on protected groups by detecting violation of the 4/5ths rule, and features that cause biased results are removed and models re-trained until bias is not detected.

“The bias mitigation consists of evaluating the adverse impact on protected groups by detecting violation of the 4/5ths rule.Features that cause biased results are removed and the models are re-trained and re-evaluated until bias is not detected.”

May combine two claims.

Applied performs bias discovery and mitigation by providing a de-biasing guide and demographic analytics reports for the hiring pipeline, and does not perform automatic candidate assessment but semi-automates discrimination monitoring.

“In contrast to Pymetrics and HireVue, it performs bias discovery and mitigation by providing a de-biasing guide and demographic analytics reports for the hiring pipeline [11].Note this tool does not perform automatic candidate assessmet but it semi-automates the task of discrimination monitoring.”

• cites [11]Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

The basis upon which claims of bias mitigation by AHSs are made is rarely scrutinised and evaluated.

“Yet the basis upon which such claims are made is rarely scrutinised and evaluated.”

May combine two claims.

Literature on bias mitigation in AHSs is at an early stage and so far primarily focused on the US context of employment both socially and legally.

“While algorithmic bias generally and in employment law specifically [14] has had extensive investigation in the FAT* community, literature on bias mitigation in AHSs is at an early stage and so far primarily focused on the US context of employment both socially and legally [7,15,40].”

• cites [14]Not checked yet

  Not checked in the free run.

• cites [7] sourceNot checked yet

  Not checked in the free run.

• cites [15]Not checked yet

  Not checked in the free run.

• cites [40] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S4 (based).

This is the first scholarly attempt to consider the question of how satisfactorily bias and discrimination might be mitigated in AHSs within the UK context.

“This is the first scholarly attempt to consider the question of how satisfactorily bias and discrimination might be mitigated in AHSs within the UK context.2”

Classified as probably not a substantial claim. May combine two claims.

The three AHSs explicitly claim to address issues of discrimination in hiring and, unlike many competitors, provide some information about how their systems work that can inform an analysis.

“These systems have been chosen as they explicitly claim to address issues of discrimination in hiring and, unlike many of their competitors, provide some information about how their systems work that can inform an analysis.”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

A number of AHSs specifically claim to address bias and discrimination in hiring, and seek to do so across organisational and national contexts.

“This is especially pertinent as a number of AHSs specifically claim to address bias and discrimination in hiring, and seek to do so across organisational and national contexts.”

Classified as probably not a substantial claim.

None of the discussions of computational limitations of bias mitigation in AHSs take account of the different national contexts in which these systems are deployed and how different legal frameworks might apply.

“However, none of these discussions take account of the different national contexts in which these systems are being deployed and how different legal frameworks might apply, despite the prominence of legal definitions in the outline of the system design.”

Classified as probably not a substantial claim.

All the fairness categories depend on clear definitions of groups.

“Notice that all the categories depend on clear definitions of groups.”

Probably not a substantial claim: may not pass S4 (based).

Adding many terms to objective functions decreases the influence of each term, causing convergence problems in the learning algorithm.

“The problem arises in the fact that adding many terms to these functions will decrease the influence of each term, causing convergence problems in the learning algorithm.”

Probably not a substantial claim: may not pass S2 (checkable).

If the mismatch between US-centric bias mitigation and UK contexts is not explicitly disclosed and analysed, there is a patent danger that inappropriate US-centric values and laws relating to bias in hiring may be exported to UK workplaces.

“If this mismatch is not explicitly disclosed and analysed, there is a patent danger that inappropriate US-centric values and laws relating to bias in hiring (and more generally in society) may be exported to UK workplaces.”

May combine two claims.

While hiring goals and values may be universal (itself questionable), legal regulation of data-driven automated systems is decidedly not universal.

“While hiring goals and values may be universal (itself questionable), legal regulation of data-driven automated systems is decidedly not.”

Probably not a substantial claim: may not pass S2 (checkable).

In terms of bias mitigation, where the system is developed is crucial.

“Yet in terms of bias mitigation, where the system is developed is crucial.”

May combine two claims.

Pymetrics features bias mitigation in candidate assessment based on game design intended to avoid score correlation with protected groups and be agnostic with respect to non-verbal communication and culture.

“Pymetrics features bias mitigation in candidate assessment based on the game design that is intended to avoid score correlation with protected groups and be agnostic with respect to non-verbal communication and culture [58].”

• cites [58]Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.

DP rights may prove in the UK/EU context to be as or more important in uncovering or combating bias than sectoral employment or equality rights; yet US systems may not be optimised to meet these rights.

“DP rights may prove in the UK/EU context to be as or more important in uncovering or combating bias than sectoral employment or equality rights; yet US systems may (unsurprisingly) not be optimised to meet these rights.”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

Data protection rights, such as the alleged 'right to an explanation', may not be implemented in US systems, potentially rendering useless the rights of candidates.

“Data protection rights, such as the alleged”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

HireVue promises to eliminate human biases in candidate assessment while finding the subset of candidates most likely to be successful by comparing them to employees already performing that job.

“HireVue promises to eliminate human biases in the assessment of candidates whilst simultaneously finding the subset of candidates that are most likely to be successful in a job by comparing them to employees already performing that job.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

EU data protection law, now codified and reformed in the GDPR, provides a suite of rights to data subjects that are unknown to Federal US law.

“Furthermore, EU DP law, now codified and reformed in the GDPR, provides a suite of rights to”

Probably not a substantial claim: may not pass S3 (load-bearing).

Pymetrics's patent reports that when comparing the final fit-to-role score, multivariate statistical analyses concluded there were no differences (statistical bias) between sample groups by age (p>0.05), gender (p>0.05) and race (p>>0.1).

“When comparing the final score that assesses the suitability of a person for a position, the multivariate statistical analyses concluded that there were no differences, statistical bias, between samples groups by age (p>0.05),gender (p>0.05) and race (p>>0.1).”

• cites [39] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

UK law on equality and discrimination in employment matters is considerably different from US law; most obviously, in the UK there is no 4/5ths rule and hard statistical goals to (dis)prove bias do not seem formally to exist in either statute or case law.

“Yet UK law (itself heavily permeated by EU law) on equality and discrimination in employment matters is considerably different; most obviously in the UK there is no 4/5ths rule and hard statistical goals to (dis)prove bias do not seem formally to exist, in either statute or case law.”

May combine two claims.

Biases buried within 'black box' AHSs may not be manifestly obvious and so remain unchallenged, unlike biased workplace hiring practices in traditional modes of hiring which may at least to some extent be evident and combatted by unions, strategic litigation and regulators.

“What is more, while biased workplace hiring practices in traditional modes of hiring may at least to some extent be evident, and combatted in traditional ways by unions, strategic litigation and regulators, there are severe dangers that such biases buried within”

Classified as probably not a substantial claim.

As AHSs become ever more popular, especially in precarious sectors such as retail and the gig economy where prospective employees may have little economic power or knowledge of rights, this is a serious problem.

“As AHSs become ever more popular, especially in sectors which are precarious, such as retail and the gig economy where prospective employees (or contractors) may have little economic power or knowledge of rights [7,15], this is, we argue, a serious problem.”

• cites [7] sourceNot checked yet

  Not checked in the free run.

• cites [15]Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable).

By not recognising the broader functions of automation in shaping power dynamics when considering bias mitigation, we run the risk of neutralising challenges in a way that actively facilitates discrimination and inequality under a banner of fairness.

“Of course, asking providers of AHSs to attend to the dynamics of power in labour relations and society more broadly might seem unnecessarily burdensome, but by not recognising the broader functions of automation in shaping those dynamics when considering forms of bias mitigation, we run the risk of neutralising challenges in a way that actively facilitates discrimination and inequality under a b”

• cites [20] sourceNot checked yet

  Not checked in the free run.

• cites [23] sourceNot checked yet

  Not checked in the free run.

• cites [28]Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Pymetrics's US patent reports that multivariate ANOVA and Hotelling's T-squared tests concluded that for each game none of the tasks showed significant differences by ethnic group and a subset of tasks showed different scores based on age and gender.

“The multivariate ANOVA and Hotelling's T-squared tests concluded that for each game none of the tasks showed significant differences by ethnic group and a subset of the tasks showed different scores based on age and gender.”

• cites [39] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

Common practices emerging to mitigate bias in AHSs built in the US are inevitably likely to reflect US legal and societal conceptions of bias and discrimination, and are exported wholesale as products to UK and other markets beyond the US.

“We conclude by arguing that, while common practices may be emerging to mitigate bias in AHSs built in the US, these are inevitably likely to reflect US legal and societal conceptions of bias and discrimination, and yet are exported wholesale as products to UK and other markets beyond the US.”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

The UK has a very different legal background to the US in terms not only of hiring and equality law, but also in terms of data protection law.

“The UK has a very different legal background to the US in terms not only of hiring and equality law, but also in terms of data protection (DP) law.”

May combine two claims.

The turn to AHSs is significant for the potential scale of impact, difficulty interrogating decision-making, increasing information asymmetry between labour and management, standardisation of techniques, obfuscation of accountability, and the veneer of objectivity that such technologies afford employers, despite evidence of discrimination at each stage of the data-driven hiring funnel.

“At the same time, the turn to AHSs is significant also for the potential scale of impact, the difficulty with interrogating decisionmaking, increasing information asymmetry between labour and management, the standardisation of techniques, the obfuscation of accountability, and the veneer of objectivity that such technologies often afford employers, despite evidence of discrimination at each stage”

Classified as probably not a substantial claim. May combine two claims.

Ongoing analysis points out issues such as the unsuitability of maths to capture the full meaning of social concepts such as fairness, especially in a general sense, or the risk of technological solutionism.

“Ongoing analysis point out issues such as the unsuitability of maths to capture the full meaning of social concepts such as fairness, especially in a general sense, or the risk of technological solutionism [43].”

• cites [43]Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim. May combine two claims.

In the EU, all natural persons whose personal data is processed are given data rights, including rights to transparency and protection against power imbalance, which may be as useful in combating bias in AHSs as employment equality rights.

“Importantly, we raise the point that in the EU, all natural persons whose personal data is processed are given data rights, including rights to transparency and protection against power imbalance, which may be as useful in combating bias in AHSs as employment equality rights; these data protection (DP) rights, ensconced in the EU General Data Protection Regulation (GDPR), are not found in omnibus”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

These companies are part of standardising not only managerial techniques but also how we should both understand and 'solve' the problem of discrimination in hiring, within a potentially global market.

“This is particularly important as these companies are part of standardising not only managerial techniques [7], but also how we should both understand and 'solve' the problem of discrimination in hiring, within a potentially global market.”

• cites [7] sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim. May combine two claims.

Fairness can be procedural (equal treatment of individuals) or substantive (equal impact on groups), and these do not necessarily align and may actually be contradictory.

“Moreover, fairness can be procedural, as in the equal treatment of individuals, or substantive, as in the equal impact on groups [14], what is also referred to as opportunity-based vs. outcome-based notions of bias.These do not necessarily align and may actually be contradictory [33].”

• cites [14]Not checked yet

  Not checked in the free run.

• cites [33]Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

This difference might be important for addressing concerns about transparency and could mean a challenge to building bias mitigation into AHSs definitively.

“We argue that this might be important for addressing concerns about transparency and could mean a challenge to building bias mitigation into AHSs definitively”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

The UK's Equality Act 2010 attempts to replace a number of piecemeal prior laws relating to different types of inequality with a coherent statute covering inter alia sex, race and disability discrimination.

“The UK's Equality Act 2010 attempts to replace a number of piecemeal prior laws relating to different types of inequality with a coherent statute covering inter alia sex, race and disability discrimination.”

May combine two claims.

Technologies are the product of human action and 'histories of discrimination live on in digital platforms and become part of the logic of everyday algorithmic systems.'

“Rather, in line with Ajunwa [5], we see value in addressing issues that such tools have made salient with the understanding that technologies are the product of human action and that, as Wajcman puts it,”

• cites [5]Not checked yet

  Not checked in the free run.

• cites [57]Not checked yet

  Not checked in the free run.

May combine two claims.

Using 'cultural fit' as part of the selection criterion has been shown to lead to exclusive hiring practices, and is now outlawed by some companies as part of an effort to decrease unconscious bias.

“Moreover, using 'cultural fit' as part of the selection criterion has been shown to lead to exclusive hiring practices, and is now outlawed by some companies as part of an effort to decrease unconscious bias [5].”

• cites [5]Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Under UK law, discrimination can be direct or indirect.

“Rather as with US law, discrimination can be direct or indirect.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

Deciding on such a criteria has long been imbued with the potential for discriminatory practices.

“Deciding on such a criteria has long been imbued with the potential for discriminatory practices, and the aim here is not to suggest that those practices have only emerged with the introduction of data-driven tools nor is it to further the construct of a (false) binary between human vs. automated decision-making.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Moore argues that the on-going quantification of the workplace comes to discipline workers as they continuously seek to adapt to the needs of the technologies in place to assess them in a process of 'self-quantification'.

“Similarly, Moore [34] argues that the on-going quantification of the workplace comes to discipline workers as they continuously seek to adapt to the needs of the technologies in place to assess them in a process of 'self-quantification'.”

• cites [34]Not checked yet

  Not checked in the free run.

May combine two claims.

The goal of hiring for 'fit' with an organisation leverages the employer significant discretion and may only be useful if an employer can make an accurate determination of such a fit.

“Central to the hiring process, only made more salient with automation, is the goal of hiring for 'fit' with an organisation, a criterion that leverages the employer significant discretion and may only be useful if an employer can make an accurate determination of such a fit [5].”

• cites [5]Not checked yet

  Not checked in the free run.

May combine two claims.

The role that AHSs come to play in entrenching and furthering information asymmetry between labour and management inherent in the hiring process, the increased lack of accountability in decision-making, and the advancement of a global standard of management techniques changes the terms of control in the workplace; what Ajunwa refers to as 'platform authoritarianism'.

“The role that AHSs come to play in entrenching and furthering the information asymmetry between labour and management inherent in the hiring process, the increased lack of accountability in decision-making, and the advancement of a global standard of management techniques changes the terms of control in the workplace; what Ajunwa [4] refers to as 'platform authoritarianism' in which employers gain”

• cites [4]Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

Reliance on candidate profiles based on incomplete data, proxies and inferences may further this sense of authoritarianism on the basis of incomplete or inaccurate profiles.

“What is more, the reliance on what are often candidate profiles based on incomplete data, proxies and inferences, may further this sense of authoritarianism on the basis of incomplete or inaccurate profiles.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

The data-driven hiring funnel demands attention and scrutiny, particularly in relation to issues of discrimination and rights.

“The data-driven hiring funnel therefore demands attention and scrutiny, particularly in relation to issues of discrimination and rights.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Direct discrimination in employment is nowadays regarded as rare.

“Direct discrimination in employment is nowadays regarded as rare [42].”

• cites [42]Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim. May combine two claims.

The Code of Practice states that the aim of the discriminatory provision, criterion or practice should be legal, should not be discriminatory in itself, and must represent a real, objective consideration.

“which states that the aim of the discriminatory provision, criterion or practice”

• cites [1]Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

Under s 19(2), bias can be justified if it is shown to be a proportionate means of achieving a legitimate aim.

“Furthermore according to s 19(2), bias can be justified if it is shown to be a”

Probably not a substantial claim: may not pass S3 (load-bearing).

The guidance does not always require a formal comparative exercise using statistical evidence.

“Importantly, the guidance does not always require a formal comparative exercise using statistical evidence.”

• cites [1]Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Indirect discrimination is defined in s 19(1) of the Equality Act 2010 as a provision, criterion or practice which is discriminatory in relation to a protected characteristic.

“Indirect discrimination is defined in s 19 (1) as”

Classified as probably not a substantial claim. May combine two claims.

All data subjects have rights in relation to the processing of personal data, including rights to transparency and access to data held about them and to object to decisions which have legal or similarly significant effects being made about them using their personal data and by solely automated means under GDPR art 22.

“all data subjects have a number of rights in relation to the processing of personal data (itself defined in GDPR, art 4(1)), including rights to transparency and access to data held about them (”

• cites GDPR, arts 12-15Not checked yet

  Not checked in the free run.

• cites GDPR, art 22Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Whether a difference is significant depends on the context, such as the size of the pool and the numbers behind the proportions.

“Whether a difference is significant will depend on the context, such as the size of the pool and the numbers behind the proportions.”

• cites [1]Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Indirect discrimination effectively occurs when a policy that applies in the same way to everybody has an effect that particularly disadvantages people with a protected characteristic.

“Effectively it occurs when a policy that applies in the same way to everybody has an effect that particularly disadvantages people with a protected characteristic.”

Probably not a substantial claim: may not pass S2 (checkable).

Given the language of proportionality and legitimacy in UK equality law, it is hard to imagine how either proportionality or legitimacy could be coded into a hiring tool.

“Given this kind of language, it is hard to imagine how either proportionality or legitimacy could be coded into a hiring tool.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The pool for comparison consists of the group which the provision, criterion or practice affects or would affect either positively or negatively, while excluding workers who are not affected by it either positively or negatively.

“This”

• cites [1]Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The UK idea of indirect discrimination is similar to the US idea of disparate impact.

“This is similar to the US idea of disparate impact.”

Classified as probably not a substantial claim. May combine two claims.

Legitimacy is not defined in the Equality Act 2010, though guidance can be drawn from the CJEU and the Code of Practice.

“Legitimacy is not even defined in the 2010 Act though guidance can be drawn from the CJEU and the Code of Practice”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

For a comparison under the Code, the circumstances of the two groups must be sufficiently similar and there must be no material differences in circumstances.

“The circumstances of the two groups must be sufficiently similar for a comparison to be made and there must be no material differences in circumstances (para 4.15).”

• cites [1]Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

In legal discourse, proportionality is an open textured concept: it is impossible to predict what factors will come relevantly into play in advance, lacking a sufficiently large dataset of case law, nor how factors would be ranked.

“In legal discourse, proportionality is what is known as an”

Classified as probably not a substantial claim.

The Code of Practice provides only that a comparison must be made between workers with the protected characteristic and those without it.

“Instead, reflecting s 19(2), the Code provides only that a comparison must be made between workers with the protected characteristic and those without it.”

• cites [1]Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The Code of Practice accompanying the Equality Act does not lay down a statistical rule of thumb akin to the 4/5ths rule to prove bias.

“However the Code of Practice which accompanies the Equality Act [1] does not lay down a statistical rule of thumb akin to the 4/5ths rule to prove bias.”

• cites [1]Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

The Code adds that although reasonable business needs and economic efficiency may be legitimate aims, an employer solely aiming to reduce costs cannot expect to satisfy the test.

“It adds:”

• cites [1]Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

It can be argued that the use of the phrase at least means that semi-automated decision making may not exclude the right to meaningful information.

“It can be argued that the use of the phrase”

• cites [53]Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The A29 Working Party recommend that the data subject should be provided with general information, notably on factors taken into account for the decision-making process and on their respective weight on an aggregate level, which is also useful for him or her to challenge the decision.

“The A29 Working Party recommend that the data subject should be provided with”

• cites [17]Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

It seems prima facie unlikely that submission to an automated hiring system could be necessary for entering the contract of employment.

“It seems prima facie unlikely but an employer might argue eg triage when many 1000s of applications are received does not just benefit from but actually requires solely automated systems.”

May combine two claims.

It could be argued that any use of a fully automated automated hiring system, regardless of whether bias can be proved or mitigated, can be refused by the prospective employee on the ground that it is a solely automated decision under art 22, requiring explicit consent, and the candidate could ask for a human to make or reconsider the decision.

“Thus it could be argued that any use of a fully automated AHS, regardless of whether bias can be proved or is, indeed, said to have been mitigated, can be refused by the prospective employee on the ground that it is a solely automated decision under art 22, requiring explicit consent (art 22(2)(c)) and instead, the candidate could ask for a human to make that decision instead, or reconsider it.”

• cites art 22(2)(c)Not checked yet

  Not checked in the free run.

May combine two claims.

What constitutes enough interaction by a human such that art 22 is not triggered remains an unclear issue in the GDPR, and may vary from member state to member state.

“What constitutes”

• cites [2]Not checked yet

  Not checked in the free run.

• cites [53]Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

If the right to object under art 22 is excluded by the lack of a solely automated decision, then any right to an explanation read from art 22 may also fall.

“If the right to object under art 22 is excluded by the lack of a”

May combine two claims.

Later evidence emerged that no automatic sackings by Amazon in fact occurred and a human supervisor was always there to reverse the sacking.

“However later evidence emerged that no”

• cites [32]Not checked yet

  Not checked in the free run.

May combine two claims.

A right to meaningful information need not be chained to proof of bias or failure to mitigate bias, and could prove highly effective in exposing discriminatory or arbitrary or erroneous practices, at an individual and possibly group level, given the possibility for collective redress actions within the GDPR.

“Again such a right need not be chained to proof of bias, or failure to mitigate bias, and yet could prove highly effective in exposing discriminatory or even simply arbitrary or erroneous practices, at an individual and possibly even at a group level, given the possibility for collective redress actions within the GDPR (see arts 80 and 82).”

• cites arts 80 and 82Not checked yet

  Not checked in the free run.

May combine two claims.

Uber's global driver terms and conditions state that automated firings can take place but then add that in the EU a right to object to a human is available.

“Uber's global driver terms and conditions state that automated firings can take place but then add that in the EU a right to object to a human is available [51].”

• cites [51] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Selbst and Powles argue that for a right to meaningful information about the logic involved to be meaningful, it must be more than a simple regurgitation of source code.

“Selbst and Powles argue, for example, that for a right to”

• cites [44] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Any machine learning system is likely to be regarded as high risk processing requiring a prior Data Protection Impact Assessment, which should show inter alia that potential for unfairness and bias had been considered and steps taken to avert.

“It is also worth noting here that any machine learning system is likely to be regarded as”

• cites art 35(3)(a)Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

The lack of information about how automated hiring systems work, the approach they take to tackling discriminatory hiring practices, and where and how they are used around the world is a significant problem.

“In conclusion, the lack of information about how AHSs work, the approach they take to tackling discriminatory hiring practices, and crucially, where and how they are used around the world is therefore a significant problem.”

Probably not a substantial claim: may not pass S2 (checkable).

In seeking to explicitly tackle issues of discrimination in hiring practices, the automated hiring systems examined provide some insights into how such issues are understood and approached by automated hiring systems.

“In particular, in seeking to explicitly tackle issues of discrimination in hiring practices, these systems provide some insights into how such issues are understood and approached by AHSs.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

It is not clear how relevant stakeholders, not least job seekers, are able to access and understand information about how decisions about their eligibility might have been reached through automated hiring systems.

“Moreover, it is not clear how relevant stakeholders, not least job seekers, are able to access and understand information about how decisions about their eligibility might have been reached through AHSs.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

What approach to transparency is required by EU law remains itself vague.

“On the other hand, what approach to transparency is required by EU law, remains itself vague.”

Probably not a substantial claim: may not pass S4 (based).

GDPR transparency rights may be ignored or ill-implemented in systems not built within the EU.

“Yet these rights may be ignored or ill-implemented in systems not built within the EU.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Bias mitigation in hiring is one of the key selling points for several automated hiring tools.

“This is particularly pertinent as bias mitigation in hiring is one of the key selling points for several of these tools.”

May combine two claims.

The lack of access to and understanding of information about automated hiring decisions makes it difficult to assess if and how discriminatory practices might have been part of the hiring process, and leaves little room for anyone to challenge the decision made.

“This makes it difficult to assess if and how discriminatory practices might have been part of the hiring process, and leaves little room for anyone to challenge the decision made.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

To date there has been no relevant case law in the UK and the provision is not expanded on in the Data Protection Act 2018 which implements the GDPR in the UK.

“To date there has been no relevant case law in the UK and the provision is not expanded on in the Data Protection Act (DPA) 2018 which implements the GDPR in the UK (in comparison to some other member states such as France).”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

Claims and validation about automated hiring systems are often vague and abstract, if they are provided at all.

“Claims and validation are often vague and abstract, if they are provided at all.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

GDPR transparency rights in arts 15 and 22 may provide avenues to overturn aspects of the candidate-employer information asymmetry and might even outright prohibit the use of automated hiring systems for wholly automated decision-making in hiring.

“GDPR transparency rights in arts 15 and 22 may provide avenues to overturn aspects of the candidateemployer information asymmetry and might even outright prohibit the use of AHSs for wholly automated decision-making in hiring.”

• cites arts 15 and 22Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

The automated hiring systems looked at in the paper are relatively unique in providing some information about their workings, even if the exact data sources and model remain obscure and will vary according to different clients.

“The AHSs we have looked at in this paper are relatively unique in providing some information about their workings, even if the exact data sources and model remain obscure and will vary according to different clients.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Levels of discrimination against ethnic minorities in accessing jobs in the UK have remained relatively unchanged since the 1960s.

“not least in the context of the UK where levels of discrimination against ethnic minorities in accessing jobs have remained relatively unchanged since the 1960s [46]”

• cites [46]Not checked yet

  Not checked in the free run.

May combine two claims.

Whilst a few companies developing automated hiring systems provide some documentation to evidence bias-mitigation claims, access to relevant information remains a key problem for conducting any thorough analysis.

“Whilst a few of the companies developing AHSs provide some documentation to evidence such claims, access to relevant information remains a key problem for conducting any thorough analysis.”

Classified as probably not a substantial claim. May combine two claims.

Computational fairness concerns relate to inherent problems with data-driven predictions, relying on quantification for determining criteria of good or appropriate fit, the necessary reductionist nature of group identification in computational systems, and the neglect of intersectionality.

“These relate not only to the inherent problems with data-driven predictions and with relying on quantification for determining criteria of 'good' or appropriate 'fit', but also to the necessary reductionist nature of group identification in computational systems and the neglect of intersectionality that have also been the subject of significant criticism in legal understandings of discrimination.”

Probably not a substantial claim: may not pass S4 (based).

In Europe and in the UK specifically, a quick survey of Employment Appeal tribunal cases shows most revolve around firing or issues of in-work conditions.

“In Europe and in the UK specifically, things are very different and a quick survey of Employment Appeal tribunal cases shows most revolve around firing or issues of in-work conditions.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Automated hiring systems are part of a growing diversity, equity, and inclusion industry that has boomed in the last couple of years.

“They are part of a growing 'diversity, equity, and inclusion' (D.E.I) industry that has boomed in the last couple of years [60].”

• cites [60]Not checked yet

  Not checked in the free run.