AI bias: exploring discriminatory algorithmic decision-making models and the application of possible machine-centric solutions adapted from the pharmaceutical industry

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

Accuracy, fairness, and privacy must always be balanced against one another.

“There is always a trade-off between accuracy, fairness and privacy that needs to be taken into account as Corbett-Davies et al. [31] examined in their study.”

• cites [31] Corbett-Davies, S., Pierson, E., Feller, A., Goel, S., Huq, A.: Algorithmic decision making and the cost of fairness. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 797–806 (2017) sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Machines take on human prejudices, acting as human proxies and shifting responsibility away.

“The machines incorporate those prejudices, becoming a proxy of humans and delegating responsibility.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

Human bias can arise because a small elite at the top of the social hierarchy shapes and sustains society.

“Humans can be biased as it is the way society is constructed and maintained by a minority elite at the top of the hierarchy.”

• cites [17] Bonilla-Silva, E.: White Supremacy and Racism in the Post-Civil Rights Era. Lynne Rienner Publishers, Boulder (2001) sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Machine-learning models may change and adapt what they produce as they receive new training data.

“Machine-learning models can have the capacity to evolve, develop and adapt their production in accordance with training information streams [3].”

• cites [3] Alpaydin, E. (2020). Introduction to machine learning. MIT press.Not checked yet

  Not checked in the free run.

May combine two claims.

Two especially relevant causes of incomplete or biased data are historical human biases and incomplete or unrepresentative data.

“While there are many reasons for incomplete or biased data, two are particularly relevant: historical human biases and incomplete or unrepresentative data [51].”

• cites [51] Lee, N.T., Resnick, P., Barton, G.: Algorithmic bias detection and mitigation: best practices and policies to reduce consumer harms. Brookings Institute, Washington, DC (2019)Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

A bias impact assessment can help identify key stakeholders, their interests, their power to block or permit changes, and short- and long-term effects.

“The bias impact assessment can be very helpful in clearly identifying the main stakeholders, their interests and their position of power when blocking or allowing necessary changes and the short- and long-term impacts.”

Classified as probably not a substantial claim.

Data scientists and programmers treat AI biases as bugs, i.e., technical issues like security to be fixed.

“On the other hand, it is the approach by data scientists and programmers that characterise AI biases as bugs implying that it is just a technical issue like security that needs to be fixed [Tramer et al. 2016, 53, 54].”

• cites Tramer et al. 2016Not checked yet

  Not checked in the free run.

• cites [53] McDuff, D., Cheng, R., Kapoor, A.: Identifying bias in AI using simulation. arXiv:1810.00471 (2018)Not checked yet

  Not checked in the free run.

• cites [54] Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., Galstyan, A.: A survey on bias and fairness in machine learning. ACM Comput. Surv. 54(6), 1–35 (2021) sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

Kearns and Roth argue that before an algorithm can be fair or private, people must agree on precise meanings that a machine can understand.

“Of course, the first challenge in asking an algorithm to be fair or private is agreeing on what those words should mean in the first place—and not in the way a lawyer or philosopher might describe them, but in so precise a manner that they can be “explained” to a machine”

• cites [49, p. 21]Not checked yet

  Not checked in the free run.

May combine two claims.

Biased training data can cause discrimination through prejudicial examples or through unfair valuation of under-represented groups.

“A set of prejudicial examples from which the model learns or in the case of under-represented groups which receives an incorrect or unfair valuation.”

• cites [84] Thelisson, E., Padh, K., Celis, L.E.: Regulatory mechanisms and algorithms towards trust in AI/ML. In: Proceedings of the IJCAI 2017 Workshop on Explainable Artificial Intelligence (XAI), Melbourne, Australia (2017)Not checked yet

  Not checked in the free run.

May combine two claims.

ImageNet lacks geographical diversity and better represents pale-skinned populations in Western countries.

“For example, a lack geographical diversity in datasets like ImageNet (a large visual database designed for use in visual object recognition software research such as facial recognition) is an example for this type of bias [81]. This demonstrates a better representation of the pale skin population in the Western countries.”

• cites [81] Suresh, H., Guttag, J.V.: A framework for understanding unintended consequences of machine learning. arXiv:1901.10002 (2019)Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

An independent regulatory body is a necessity that repeatedly appears in AI ethics discussions.

“In addition, they have adopted an independent regulatory body (US FDA, UK MHRA or EU MDA)—a necessity that keeps coming up in many AI Ethics discussions [36].”

• cites [36] Erdélyi, O.J., Goldsmith, J.: Regulating Artificial Intelligence: Proposal for a Global Solution. In: Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society (2018) sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Data about certain groups can be absent or incorrect, as was initially true for female car drivers, who were a minority.

“Some data from specific groups’ databases can be either non-existent or simply incorrect, as was initially the case with female car drivers who were a minority.”

May combine two claims.

When early car safety belts and airbags were designed, they fitted tall males, and other body types, especially shorter people, were ignored, leading to a higher death rate in crashes.

“When the first safety belts and airbags for cars were designed, they suited tall males (the average engineer). Any other humans with other physical characteristics, especially shorter stature, were not considered, ending in a higher fatality rate in a car crash [18].”

• cites [18] Bose, D., Segui-Gomez, S.C.D.M., Crandall, J.R.: Vulnerability of female drivers involved in motor vehicle crashes: an analysis of US population at risk. Am. J. Public Health 101(12), 2368–2373 (2011) sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Societies are structured by an elite at the top of the hierarchy that controls power and top decision-making roles, and whose biases can become a societal standard and produce historical human biases.

“Societies, as described by Bonilla-Silva [17], are structured by an elite at the high end of the hierarchy, controlling power and the top stages in the decision-making processes (e.g. judges, senior civil servants and politicians), whose biases have the ability to be adopted as a standard across society which then lead to historical human biases.”

• cites [17] Bonilla-Silva, E.: White Supremacy and Racism in the Post-Civil Rights Era. Lynne Rienner Publishers, Boulder (2001) sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Prisons and police enforcement appear to be tools for perpetuating classism and maintaining a rigid social structure, with racism as a by-product.

“Prisons and police enforcement seems to be a tool to perpetuate classism and maintain a rigid social structure, of which racism is a by-product [17, 22].”

• cites [17] Bonilla-Silva, E.: White Supremacy and Racism in the Post-Civil Rights Era. Lynne Rienner Publishers, Boulder (2001) sourceNot checked yet

  Not checked in the free run.

• cites [22] Brewer, R.M., Heitzeg, N.A.: The racialization of crime and punishment: criminal justice, color-blind racism, and the political economy of the prison industrial complex. Am. Behav. Sci. 51(5), 625–644 (2008) sourceNot checked yet

  Not checked in the free run.

May combine two claims.

The statistics indicate that this is a socio-economic issue in which race plays a major role.

“However, according to the statistics, it is evident that it is a socio-economic issue in which race plays a big part.”

May combine two claims.

Black adults are 33% of the US prison population but only 12% of the US adult population.

“Black adults make up 33% of US prison population while just making 12% of the US adult population [102].”

• cites [102] Zeng, Z.: Jail Inmates in 2018, US Census Bureau, Bureau of Justice Statistics. https://bjs.ojp.gov/library/publications/jail-inmates-2018. Accessed 22 June 2021 (2020)Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Women working as programmers in IT make up 19.8% of that workforce.

“Female programmers in IT constitute only 19.8% of the total workforce [79, p. 5]”

• cites [79, p. 5]Not checked yet

  Not checked in the free run.

May combine two claims.

Amazon removed the biased CV algorithm once bias was detected, and the company says it was never used.

“Although it was removed as soon as the bias was detected, and the company states that it was never in use, it is a good illustration of gender discriminatory outcomes as the case study will demonstrate [33, 91].”

• cites [33] Dastin, J.: Amazon scraps secret AI recruiting tool that showed bias against women. Reuters. https://www.reuters.com/article/us-amazon-com-jobs-automation-insight/amazon-scraps-secret-ai-recruiting-tool-that-showed-bias-against-women-idUSKCN1MK08G (2018). Accessed 24 Apr 2021 sourceNot checked yet

  Not checked in the free run.

• cites [91] Vincent, J.: Amazon reportedly scraps internal AI recruiting tool that was biased against women. The Verge. https://www.theverge.com/2018/10/10/17958784/ai-recruiting-tool-bias-amazon-report (2018). Accessed 28 Mar 2021Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

A COMPAS score of 8 corresponds to twice the re-offending rate of a score of 4.

“For example, an individual who scores 8 has twice the re-offending rate of those who have 4.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Amazon has a long history of algorithmic decision-making, with users long receiving product recommendations based on earlier searches and purchases.

“It is a problematic finding because Amazon has a long history in the use of algorithmic decision-making, as users have long been recommended products based on previous searches and purchases [52].”

• cites [52] Linden, G., Smith, B., York, J.: Amazon.com recommendations: Item-to-item collaborative filtering. IEEE Internet Comput. 7(1), 76–80 (2003) sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

On December 31, 2017, private prison facilities-non-secure community corrections centres and home confinement included-held 15% of the federal prison population.

“Prison institutions in the US—private prison facilities, including non-secure community corrections centres and home confinement, held 15% of the federal prison population on December 31, 2017 [21, p. 16].”

• cites [21, p. 16]Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

There is tension between fairness and the tendency to preserve the status quo, which may rest on generational prejudices against other groups.

“There is a tension between fairness, one of the key values, and the tendency to maintain the status quo, which might be based on generational held prejudices against other groups, according to Bell [8].”

• cites [8] Bell, D. Faces at the Bottom of the Well: The Permanence of Racism. Hachette, UK (2018)Not checked yet

  Not checked in the free run.

May combine two claims.

Amazon's CV selection model penalised the word woman and favoured male applicants.

“The model ended in penalising the word woman and favouring male applicants.”

• cites [33] Dastin, J.: Amazon scraps secret AI recruiting tool that showed bias against women. Reuters. https://www.reuters.com/article/us-amazon-com-jobs-automation-insight/amazon-scraps-secret-ai-recruiting-tool-that-showed-bias-against-women-idUSKCN1MK08G (2018). Accessed 24 Apr 2021 sourceNot checked yet

  Not checked in the free run.

• cites [91] Vincent, J.: Amazon reportedly scraps internal AI recruiting tool that was biased against women. The Verge. https://www.theverge.com/2018/10/10/17958784/ai-recruiting-tool-bias-amazon-report (2018). Accessed 28 Mar 2021Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

The assessment can help reduce bias in automated decision AI systems and support accountability.

“It can then help with how to mitigate them in the assessment of automated decision AI systems and facilitates accountability.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

The bias impact assessment accomplishes that awareness, which is why it is relevant.

“The bias impact assessment does that and hence its relevance.”

May combine two claims.

COMPAS gathers historical data from earlier discriminatory court sentences and amplifies those prejudices, while also acting as a human proxy and shifting moral responsibility.

“The ML model COMPAS collects historical data from previous discriminatory court sentences and enhances those prejudices, with the added characteristic of being a proxy of a human and delegating moral responsibility.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

The assessment can also improve understanding of complex social and historical contexts and support a practical and ethical framework for detecting possible bias.

“The assessment can also provide a better understanding of complex social and historical contexts as well as supporting a practical and ethical framework for spotting possible bias.”

May combine two claims.

There appears to be a power imbalance favouring a privileged white population that holds most high-ranking Justice and Police positions, which could support institutionalised racism against black people, as shown by their over-representation in prison.

“There seems to be an imbalance of power between a privileged white population that holds a majority of high-ranking positions in the Justice and Police departments, which could favour institutionalised racism over the black population,8 as figures seem to demonstrate in the case of the black population being over-represented in the prison population [22].”

• cites [22] Brewer, R.M., Heitzeg, N.A.: The racialization of crime and punishment: criminal justice, color-blind racism, and the political economy of the prison industrial complex. Am. Behav. Sci. 51(5), 625–644 (2008) sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

The public sector running state prisons appears willing to preserve the status quo according to the figures given.

“The public sector, operating state prisons, seems to be willing to maintain the status quo by the figures provided in point 5.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

COMPAS is a risk-assessment tool that supports criminal justice organisations and extends other judicial information systems.

“COMPAS is a risk-assessment tool that aids the operations of criminal justice organisations and is an extension of other judicial information systems [2].”

• cites [2] Angwin, J., et al.: (2016) Machine bias: there’s software used across the country to predict future criminals. And it’s biased against blacks. ProPublica. https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing (2016). Accessed 28 Mar 2021Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim. May combine two claims.

Spiekermann's model follows clear steps for identifying key elements such as stakeholders, benefits, and harms, and is complemented by the EU paper's four-value approach.

“The main reason is that Spiekermann’s model follows clear steps in identifying key elements such as stakeholders, benefits and harms while being complemented by the four value approach from the High-Level Expert EU paper.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Reisman et al. show that algorithmic impact assessment is needed as standard practice in AI.

“Reisman et al. [70, pp. 5–6] demonstrate the necessity of algorithmic impact assessment in AI as standard practice”

• cites [70, pp. 5–6]Not checked yet

  Not checked in the free run.

• cites [25] Calvo, R.A., Peters, D., Cave, S.: Advancing impact assessment for intelligent systems. Nat. Mach. Intell. 2(2), 89–91 (2020) sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

According to the World Prison Brief, the US has one of the world's highest incarceration rates; in 2018 the figure was 23.4% of the total population.

“According to the World Prison Brief [100], the US has one of the highest incarceration rates in the world. In 2018, the figure was 23.4% of the total population.”

• cites [100] World Prison Brief.: https://prisonstudies.org/country/united-states-america (2018). Accessed 22 Apr 2021Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Women account for only one quarter of technology industry employees.

“and make up only a quarter of employees in the technology industry [82], p. 1]”

• cites [82], p. 1]Not checked yet

  Not checked in the free run.

May combine two claims.

The elite gain by reinforcing their privileges, while others have limited access to well-paid jobs and colleges.

“The elite have the benefit of reinforcing privileges, and the rest of the population have limited access to progress to well-paid jobs and colleges [8].”

• cites [8] Bell, D. Faces at the Bottom of the Well: The Permanence of Racism. Hachette, UK (2018)Not checked yet

  Not checked in the free run.

May combine two claims.

A wrong COMPAS assessment can lead to a person who would not re-offend being jailed before trial while a more dangerous person likely to offend goes free.

“Someone can be wrongly imprisoned while awaiting trial who would not re-offend while a more dangerous individual more likely to offend would be let free (Fig. 3).”

May combine two claims.

White defendants' COMPAS scores clustered in lower-risk categories, unlike black defendants' scores.

“These charts show that scores for white defendants tended toward lower-risk categories. Scores for black defendants did not.”

• cites [2] Angwin, J., et al.: (2016) Machine bias: there’s software used across the country to predict future criminals. And it’s biased against blacks. ProPublica. https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing (2016). Accessed 28 Mar 2021Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Fairness through awareness holds that the first step to reduce or remove bias in an algorithmic model is awareness of the biases and their causes.

“The concept fairness through awareness was introduced by Dwork et al. [35], and it states that if we wish to mitigate or remove bias in an algorithmic model, the first step is to be aware of the biases and why they occur.”

• cites [35] Dwork, C., Hardt, M., Pitassi, T., Reingold, O., Zemel, R.: Fairness through awareness. In Proceedings of the 3rd Innovations in Theoretical Computer Science Conference, pp. 214–226 (2012) sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Because eliminating bias and ensuring equal opportunity proved difficult, Amazon abandoned the system.

“It proved challenging to remove bias and guarantee equal opportunities, so Amazon decided to scrap it altogether [33].”

• cites [33] Dastin, J.: Amazon scraps secret AI recruiting tool that showed bias against women. Reuters. https://www.reuters.com/article/us-amazon-com-jobs-automation-insight/amazon-scraps-secret-ai-recruiting-tool-that-showed-bias-against-women-idUSKCN1MK08G (2018). Accessed 24 Apr 2021 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Tramer et al. report using FairTest to investigate and sometimes address disparate impact, offensive labelling, and uneven algorithmic error rates in four data-driven applications.

“Tramer et al. [86, pp. 1 and 2] describe their Test themselves and its use as: “We report on the use of FairTest to investigate and in some cases address disparate impact, offensive labelling, and uneven rates of algorithmic error in four data-driven applications”.”

• cites [86, pp. 1 and 2]Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

DuPont's Teflon caused environmental damage that cost the company about one billion dollars.

“For example, DuPont, a highly respected company that produced the popular material Teflon used in cooking tools, caused environmental damage that ended up costing the company around a billion dollars [78, p. 1].”

• cites [78, p. 1]Not checked yet

  Not checked in the free run.

May combine two claims.

In the Amazon case, women were the disadvantaged group, prevented from accessing jobs, which restricted their human autonomy.

“In this specific example, the disadvantaged group are the women,9 as they are blocked or impeded from accessing those jobs and limiting respect for human autonomy (financial independence).”

May combine two claims.

Training data for machine-learning models is inherently biased, like society, and tends to replicate human behaviour.

“The data used in training is intrinsically biased, as society is, and it tends to simply replicate human behaviour—one of the core aims of Artificial Intelligence.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

Failing to act ethically can harm a business.

“Not doing the right thing can end up harming a business.”

May combine two claims.

The AI industry resembles the pharmaceutical industry in being multidisciplinary, needing diverse voices and expertise, and having applied ethics play a central role.

“The AI industry is similar to the pharmaceutical industry in its multidisciplinary environment, its need for diverse voices and expertise, and the pivotal role that applied Ethics has played in its development.”

• cites [74] Santoro, M.A., Gorrie, T.M.: Ethics and the Pharmaceutical Industry. Cambridge University Press, Cambridge (2005) sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The recruitment selection process may therefore have been influenced by the system's recommendations, whether knowingly or not.

“Thus consciously, or unconsciously, affecting the selection process.”

May combine two claims.

Amazon stated the system was never deployed but did not confirm that recruiters could not see its recommendations.

“Although Amazon said it was never implemented, it did not confirm that recruiters had no access to the machine’s recommendations.”

• cites [33] Dastin, J.: Amazon scraps secret AI recruiting tool that showed bias against women. Reuters. https://www.reuters.com/article/us-amazon-com-jobs-automation-insight/amazon-scraps-secret-ai-recruiting-tool-that-showed-bias-against-women-idUSKCN1MK08G (2018). Accessed 24 Apr 2021 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

A diverse team benefits a company by representing all voices and tailoring to their needs.

“A diverse team benefits the company in the sense that all voices are represented and their needs tailored [59].”

• cites [59] Nielsen, M.W., Alegria, S., Börjeson, L., Etzkowitz, H., Falk-Krzesinski, H.J., Joshi, A., Leahey, E., Smith-Doerr, L., Woolley, A.W., Schiebinger, L.: Opinion: gender diversity leads to better science. Proc. Natl. Acad. Sci. 114(8), 1740–1742 (2017) sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Vioxx, an anti-inflammatory drug made by Merck, was associated with heart attacks and strokes in long-term use and was withdrawn from the market in 2004.

“That does not mean that there are no public safety issues like the anti-inflammatory drug Vioxx produced by Merck, which was linked to heart attacks and strokes in long-term use and withdrawn from the market in 2004 [10, 74, p. 13].”

• cites [10, 74, p. 13]Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Wang et al. introduced the concept of noisy protected groups, meaning data that are corrupted, missing or unreliable because of social pressures.

“A more effective use of this concept is tested by Wang et al. [94] in their paper when introducing the idea of noisy protected groups. By “noisy protected groups”, the authors mean when data are corrupted, missing or unreliable due to social pressures.”

• cites [94] Wang, S., Guo, W., Narasimhan, H., Cotter, A., Gupta, M., Jordan, M.I.: Robust optimization for fairness with noisy protected groups. arXiv:2002.09343 (2020)Not checked yet

  Not checked in the free run.

May combine two claims.

Efforts were made to reduce the problem, but gender discrimination in Amazon's system was deeply entrenched.

“There were some attempts to ameliorate the problem, but the issue of gender discrimination was deep-rooted.”

• cites [33] Dastin, J.: Amazon scraps secret AI recruiting tool that showed bias against women. Reuters. https://www.reuters.com/article/us-amazon-com-jobs-automation-insight/amazon-scraps-secret-ai-recruiting-tool-that-showed-bias-against-women-idUSKCN1MK08G (2018). Accessed 24 Apr 2021 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Drugs found unsafe or ineffective will not pass through all four phases.

“As Sedgwick [77, p. 1] assures: “Drugs under development that are found to be unsafe or ineffective will not progress through all four phases”.”

• cites [77, p. 1]Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

The AI industry appears to be following a similar path of unregulated growth that may be followed by regulation, greater awareness and mainstream acceptance, but over a shorter period.

“The AI industry seems to enjoy a similar path of unregulated growth followed, hopefully, by regulations, better awareness in the industry and by the mainstream market—although it is vying to achieve this in a shorter period.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The technology industry is male-dominated, with men making up almost 75% of its workforce.

“The tech industry is male dominated, comprising almost 75% of the workforce [79, 82].”

• cites [79] Shields, M.: Women's participation in Seattle's high-tech economy. https://smartech.gatech.edu/bitstream/handle/1853/53790/madelyn_shields_womens_participation_in_seattles_hightech_economy.pdf (2015). Accessed 15 Aug 2021Not checked yet

  Not checked in the free run.

• cites [82] Swift, S.: Gender Disparities in the Tech Industry: The Effects of Gender and Stereotypicability on Perceived Environmental Fit. In: 2015 NCUR (2015)Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In the Amazon case, fairness was lacking when a qualified candidate was denied an interview because of gender.

“Thirdly, a lack of fairness, if a candidate matching those requisites is not selected for a job interview because of their being the wrong gender.”

Probably not a substantial claim: may not pass S2 (checkable).

There appears to be no reason why technology companies cannot operate similarly to reduce bias.

“There does not appear to be any reasons why tech companies cannot conduct their businesses in a similar manner to mitigate bias.”

Classified as probably not a substantial claim.

The pharmaceutical industry uses data collection and comparison strategies such as randomised controlled trials, which can improve bias impact assessments by revealing unexpected outcomes.

“It also has a variety of collecting and contrasting data strategies like randomised control trials, which can help improve bias impact assessments by unearthing unexpected outcomes.”

Probably not a substantial claim: may not pass S3 (load-bearing).

A narrower range of ideas and backgrounds may lead to lower creativity and innovation in new products.

“A less diverse source of ideas and backgrounds might result in poorer creativity and overall innovation in producing new products.”

• cites [59] Nielsen, M.W., Alegria, S., Börjeson, L., Etzkowitz, H., Falk-Krzesinski, H.J., Joshi, A., Leahey, E., Smith-Doerr, L., Woolley, A.W., Schiebinger, L.: Opinion: gender diversity leads to better science. Proc. Natl. Acad. Sci. 114(8), 1740–1742 (2017) sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim. May combine two claims.

The main purpose of blind testing is to detect differing outcomes across groups and determine whether the reason is fair or discriminatory, as in the COMPAS case.

“The main intention, at this stage, is to detect different outcomes from different groups and determine whether there is a fair reasoning behind this or whether the outcome is discriminatory, as seen in the COMPAS case study.”

May combine two claims.

Side effects are typically detected and recorded through surveillance systems such as Pharmacovigilance in phase IV, and a regulatory process exists to respond.

“However, side effects are usually noticed and recorded with the assistance of surveillance and monitoring systems like Pharmacovigilance in phase IV (Hauben et al. 2009), and there is a regulatory procedure to act upon it.”

• cites (Hauben et al. 2009)Not checked yet

  Not checked in the free run.

May combine two claims.

Before 1962, pharmaceutical companies were not required to run trials proving their products' safety and accuracy; the US Congress passed the Kefauver-Harris Amendments that year and Europe soon followed.

“Pharmaceutical companies were not expected to conduct trials to demonstrate the safety and accuracy of their medical products until 1962. That year, the US Congress passed the Kefauver-Harris Amendments to the Food, Drug and Cosmetics act of 1938, and Europe followed suit soon afterwards [74, p. 12].”

• cites [74, p. 12]Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In the Amazon case, there was no prevention of harm to self-esteem or self-value when women applied and their career development was blocked.

“In addition, there is no prevention of harm (self-esteem/self-value), if women apply for those positions and it stops their career development.”

May combine two claims.

Pharmaceutical companies test different age, gender and other groups at different stages and compare results, helping to develop a standard method that maximises benefits and reduces harm.

“In addition, they conduct their trials on different age, gender and other characteristics groups at different stages and compare results. This process helps to develop a standard methodology to maximise benefits and remove, or minimise, harm.”

Probably not a substantial claim: may not pass S3 (load-bearing).

In 2015, Google image recognition misidentified black people as gorillas.

“For example, in 2015, it was found that Google image misidentified black persons as gorillas [41]”

• cites [41] Guynn, J.: Google photos labelled black people 'gorillas'. USA today. http://www.usatoday.com/story/tech/2015/07/01/google-apologizes-after-photos-identify-black-people-as-gorillas/29567465/ (2015). Accessed 15 Mar 2021Not checked yet

  Not checked in the free run.

May combine two claims.

Regulation need not hinder innovation; pharmaceutical companies developed an mRNA COVID-19 vaccine in record time.

“On the other hand, regulation does not have to affect innovation, for example, when pharmaceutical companies developed a COVID-19 vaccine utilising a cutting-edge technology, mRNA, in a record time (Kim 2020).”

• cites (Kim 2020)Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Some male tech workers prefer the status quo and may be prejudiced against women, which can make women feel unwelcome.

“Some of the male workers would prefer business as usual.10 They might be prejudiced against women and prefer the perpetuation of those values, making some women feel unwelcome in tech companies.”

• cites [92] Wajcman, J.: Feminism Confronts Technology. Penn State Press, Pennsylvania (1991)Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Other male workers and female workers prefer a more gender-balanced company where everyone feels welcome and management reflects gender diversity.

“Other male workers, and the rest of the female workers, would prefer a more gender-balanced company where everybody feels welcome and the management board reflects that gender diversity [59, 101].”

• cites [59] Nielsen, M.W., Alegria, S., Börjeson, L., Etzkowitz, H., Falk-Krzesinski, H.J., Joshi, A., Leahey, E., Smith-Doerr, L., Woolley, A.W., Schiebinger, L.: Opinion: gender diversity leads to better science. Proc. Natl. Acad. Sci. 114(8), 1740–1742 (2017) sourceNot checked yet

  Not checked in the free run.

• cites [101] Yasser, Q.R., Al Mamun, A., Ahmed, I.: Corporate social responsibility and gender diversity: insights from Asia Pacific. Corp. Soc. Responsib. Environ. Manag. 24(3), 210–221 (2017) sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim. May combine two claims.

Both case studies conclude that a bias impact assessment is needed to raise awareness of bias and its causes before deployment, and that this is urgent.

“Both case studies conclude with the necessity of applying a bias impact assessment to raise awareness of any bias and its possible reasons before being implemented as demonstrated by Dwork et al. [35]—and its urgency.”

• cites [35] Dwork, C., Hardt, M., Pitassi, T., Reingold, O., Zemel, R.: Fairness through awareness. In Proceedings of the 3rd Innovations in Theoretical Computer Science Conference, pp. 214–226 (2012) sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Clinical trials are standard; drugs are tested on humans only after laboratory testing, through successive phases I to IV.

“Today, clinical trials are the norm. The drugs are tested on humans only after they have undergone laboratory testing. This takes the form of a series of successive clinical trials known as phase I, phase II, phase III, and phase IV trials.”

• cites [77] Sedgwick, P.: Phases of clinical trials. BMJ 343, d6068 (2011) sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

FairTest investigates associations between application outcomes and sensitive user attributes and provides debugging capabilities to fix detected unfair effects.

“It is designed to investigate associations between application outcomes (such as prices or premiums) and sensitive user attributes (such as race or gender). Once detected, it provides debugging capabilities that help programmers solve the detected unfair effects.”

• cites [86] Tramer, F., Atlidakis, V., Geambasu, R., Hsu, D., Hubaux, J.P., Humbert, M., Juels, A., Lin, H.: FairTest: discovering unwarranted associations in data-driven applications. In: 2017 IEEE European Symposium on Security and Privacy (EuroS&P), pp. 401–416. IEEE (2017) sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Phase I determines safety, phase II tests effectiveness, phase III compares effectiveness with standard treatment, and phase IV monitors risks and benefits after market release.

“Phase I trials are conducted to determine the safety of the product, phase II trials test whether the drug is effective, phase III trials will compare its effectiveness with the standard treatment available, and phase IV trials monitor any risks and benefits once the product is in the market.”

• cites [77] Sedgwick, P.: Phases of clinical trials. BMJ 343, d6068 (2011) sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

AI has life-saving potential, such as in medical imaging for cancer detection.

“AI can save lives, for example, in the case of cancer detection technologies in medical imaging [12].”

• cites [12] Bi, W.L., Hosny, A., Schabath, M.B., Giger, M.L., Birkbak, N.J., Mehrtash, A., Allison, T., Arnaout, O., Abbosh, C., Dunn, I.F., Mak, R.H.: Artificial intelligence in cancer imaging: clinical challenges and applications. CA Cancer J Clin 69(2), 127–157 (2019) sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim. May combine two claims.

AI is no longer confined to laboratories and is now trialled on millions of people, which makes the spread of harmful ideas especially risky.

“As Crawford [32, pp. 117–118] warns us: “The reproduction of harmful ideas is particularly dangerous now that AI has moved from being an experimental discipline used only in laboratories to being tested at scale on millions of people”.”

• cites [32] Crawford, K.: The Atlas of AI. Yale University Press (2021) sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Effective regulation reduces harm and costs less than paying for damage and penalties later, as the DuPont example indicates.

“Finally, good regulation mitigates harm, and it is more cost-effective than trying to cover the costs of damages and fines, as the previous example of DuPont shows.”

May combine two claims.

Prior work cited as [8, 55, 67] shows how bias is deeply embedded in society and hard to eliminate, a pattern mirrored in AI.

“Studies by [8, 55, 67], are potent illustrators of embedded bias in society and the difficulty of removing them which is reflected in the AI industry.”

• cites [8] Bell, D. Faces at the Bottom of the Well: The Permanence of Racism. Hachette, UK (2018)Not checked yet

  Not checked in the free run.

• cites [55] Mills, C.W.: The Racial Contract. Cornell University Press, Ithaca (2014)Not checked yet

  Not checked in the free run.

• cites [67] Pateman, C.: The Sexual Contract. Wiley, Weinheim (2018)Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Resource-poor countries can use a rich country's regulatory approval as a basis for letting a medicine enter their market.

“Third world countries poor in resources can rely on the seal of approval from rich countries to allow a medicine to reach their market.”

• AssumptionRests on an assumption: Regulatory approval from the FDA makes approval by other national agencies highly probable.

May combine two claims.

Elderly people with pre-existing health conditions were incorrectly discriminated against, with predictions of more hospital visits and higher premiums.

“Elderly people with some pre-existing health conditions were discriminated against incorrectly, predicting more visits to the hospital, in some cases, and being charged a higher premium.”

• cites [86, p. 4]Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Gmail once deployed an algorithm for filtering spam.

“An example can be when Gmail introduced an algorithm to remove spam emails [11]—the same principle could be used to reduce fake news.”

• cites [11] Bhuiyan, H., Ashiquzzaman, A., Juthi, T.I., Biswas, S., Ara, J.: A survey of existing e-mail spam filtering methods considering machine learning techniques. Glob. J. Comput. Sci. Technol. 18(2-c)(2018)Not checked yet

  Not checked in the free run.

May combine two claims.

Multinational firms use intricate accounting arrangements to reduce their tax payments, a practice that is spreading in the EU and US.

“as international companies devise complex accounting mechanisms to avoid paying much tax (as becoming more common in the EU and US, Bageri 2013).”

• cites Bageri 2013 sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

AI is inherently transnational because it is device-based.

“However, by its device-based nature, AI is a transnational technology, and it needs solutions applicable beyond borders.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The association-guided tree construction algorithm further investigates initial findings, uses visual presentation to aid interpretation, and identifies subsets affected by algorithmic bias.

“The association-guided tree construction algorithm further investigates the findings in the first step. It introduces a visual presentation to facilitate the interpretation and identify which subsets are affected by algorithmic bias [86].”

• cites [86] Tramer, F., Atlidakis, V., Geambasu, R., Hsu, D., Hubaux, J.P., Humbert, M., Juels, A., Lin, H.: FairTest: discovering unwarranted associations in data-driven applications. In: 2017 IEEE European Symposium on Security and Privacy (EuroS&P), pp. 401–416. IEEE (2017) sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The initial AIF360 Python package implements inputs from eight published papers, more than 71 bias detection metrics, and nine bias mitigation algorithms.

“As they detailed in the paper: the initial AIF360 Python package implements inputs from 8 published papers, over 71 bias detection metrics, and 9 bias mitigation algorithms.”

• cites [9] Bellamy, R.K., Dey, K., Hind, M., Hoffman, S.C., Houde, S., Kannan, K., Lohia, P., Martino, J., Mehta, S., Mojsilovic, A., Nagar, S. AI fairness 360: an extensible toolkit for detecting, understanding, and mitigating unwanted algorithmic bias. arXiv:1810.01943 (2018)Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Woolley et al. give examples of manipulation, including political bots in Brazil and the roots of digital misinformation in Russia.

“Woolley et al. [99] provides case studies as examples such as: political bot intervention during pivotal events in Brazil or the origins of digital misinformation in Russia in which manipulation took place.”

• cites [99] Woolley, S.C., Howard, P.N. (eds.) Computational Propaganda: Political Parties, Politicians, and Political Manipulation on Social Media. Oxford University Press, Oxford (2018)Not checked yet

  Not checked in the free run.

May combine two claims.

Large technology companies such as Google and Facebook operate across national borders, and their failures have transnational consequences.

“In addition, tech companies like Google or Facebook operate beyond borders and its implications when things go wrong are transnational [99].”

• cites [99] Woolley, S.C., Howard, P.N. (eds.) Computational Propaganda: Political Parties, Politicians, and Political Manipulation on Social Media. Oxford University Press, Oxford (2018)Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim. May combine two claims.

Evidence alone about past threats such as man-made climate change from the 1980s has not been enough to alter corporate and government conduct, so more proactive approaches are required.

“When scholars look back at discovering previous threats to humankind like climate change due to man-made pollution, as early as the 80s [90], they realise that providing the evidence is insufficient to modifying behaviour by companies and governments and they surmise that more pro-active strategies are needed.”

• cites [90] Verdin, J., Funk, C., Senay, G., Choularton, R.: Climate science and famine early warning. Philos. Trans. R. Soc. B Biol. Sci. 360(1463), 2155–2168 (2005) sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

A fairness problem in one group may not appear in another, as with Simpson's Paradox; for instance, lengthy waits for cancer care are seen as unfair generally but not by people with private insurance.

“Third, instances of unfairness in one group might not translate into another group, as it is in the previously explained Simpson’s Paradox. For example, long waiting lists for a cancer treatment is considered unfair in the general population, but does not affect those with a private health insurance.”