Thinking responsibly about responsible AI and ‘the dark side’ of AI

May combine two claims.

A review of IS literature found more papers on dark side issues but few that directly examine the dark side concept itself.

“we reviewed key IS literature that refers to the dark side and found an increasing number of papers dealing with dark side issues, but not so much that specifically address the concept of the dark side itself.”

May combine two claims.

In Kordzadeh and Ghasemaghaei's (2021) comprehensive literature analysis, most papers on algorithmic bias conceptually discussed ethical, legal, and design implications, while few provided an empirical examination of their claims.

“In the comprehensive literature analysis of Kordzadeh and Ghasemaghaei (2021), most papers on algorithmic bias conceptually discussed the ethical, legal, and design implications, while few provided an empirical examination of their claims.”

• cites Kordzadeh and Ghasemaghaei, 2021Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Transparency is a concept that regularly appears in contemporary conversations around AI.

“Transparency is a concept that regularly appears in contemporary conversations around AI, particularly about enabling more lucidity on AI outcomes and the procedures around responsibility and accountability (Barredo Arrieta et al., 2020;Dignum, 2019).”

• cites Barredo Arrieta et al., 2020Not checked yet

  Not checked in the free run.

• cites Dignum, 2019Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

The question of accountability is inextricably tied to the types of testing performed to ensure that AI applications do not cause harm.

“Finally, the question of accountability is also inextricably tied with the types of testing performed to ensure that AI applications do not cause harm.”

Classified as probably not a substantial claim.

Adverse outcomes of AI use and the threat of losing control over superior AI entities have sparked an ongoing debate about establishing principles to govern AI.

“Such adverse outcomes of AI use, along with the threat of loss of control or autonomy over superior AI entities, has sparked an ongoing debate about the need to establish a set of principles to effectively govern AI (Barredo Arrieta et al., 2020;Fjeld et al., 2020).”

• cites Barredo Arrieta et al., 2020Not checked yet

  Not checked in the free run.

• cites Fjeld et al., 2020Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

Examining how negative effects emerge and unfold during the design and deployment of AI applications can help research and practice understand how value from such investments emanates.

“Examining how negative effects emerge and how they unfold during the design and deployment of AI applications can aid research and practice in understanding how the value from such investments emanates.”

Probably not a substantial claim: may not pass S3 (load-bearing).

In recent years, every organisation connected to technology policy has proposed a set of guiding principles for AI.

“In the past several years, every organisation connected to technology policy has proposed a set of guiding principles around AI (Google, 2021;IBM, 2021;OECD, 2021).”

• cites Google, 2021Not checked yet

  Not checked in the free run.

• cites IBM, 2021Not checked yet

  Not checked in the free run.

• cites OECD, 2021Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim. May combine two claims.

One key issue when developing AI applications concerns the decisional dilemmas AI applications need to address and how they align with the ethical principles society adheres to.

“One such key issue when developing AI applications concerns the decisional dilemmas AI applications need to address and how they align with the ethical principles we adhere to as a society (Daly et al., 2021).”

• cites Daly et al., 2021Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Responsible AI principles are proposed on the logic that they will prevent or minimise the intended and unintended negative effects of AI.

“These principles that jointly comprise responsible AI are suggested under the logic that they will prevent or minimise the intended and unintended negative effects that AI will introduce to everyday life (Wang et al., 2020).”

• cites Wang et al., 2020Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Responsible AI captures a diverse set of requirements that need to be met throughout the system's entire life cycle.

“In this regard, the notion of responsible AI captures a diverse set of requirements that need to be met throughout the system's entire life cycle (European Commission, 2019).”

• cites European Commission, 2019Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Rana et al. (2021) encourage managers to avoid inappropriate outcomes of AI-based business analysis systems and call for beneficial, explicable, and transparent systems.

“In this special issue, the study of Rana et al. (2021) encourages managers to avoid inappropriate outcomes of AI-BA systems.They call for beneficial, explicable, and transparent systems.”

• cites Rana et al., 2021Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Kordzadeh and Ghasemaghaei (2021) and Marjanovic et al. (2021) cite examples of algorithmic bias and injustice and discuss implications and challenges in designing AI to increase fairness and reduce bias.

“In this special issue alone, for example, Kordzadeh and Ghasemaghaei (2021) and Marjanovic et al. (2021) cite examples of algorithmic bias and injustice and the implications of both and of the challenges and steps needed in the design of AI to increase fairness and reduce bias.”

• cites Kordzadeh and Ghasemaghaei, 2021Not checked yet

  Not checked in the free run.

• cites Marjanovic et al., 2021Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

There is an underlying agreement that responsible AI constitutes a set of principles ensuring ethical, transparent, and accountable use of AI technologies consistent with user expectations, organisational values, and societal laws and norms.

“Based on the growing consensus on responsible AI, there is an underlying agreement that it constitutes a set of principles that ensure ethical, transparent, and accountable use of AI technologies consistent with user expectations, organisational values, and societal laws and norms (Accenture, 2021).”

• cites Accenture, 2021Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The growing interest in AI has been accompanied by a plethora of cases where negative or unintended consequences have emerged.

“The growing interest in AI has also been accompanied by a plethora of cases where negative or unintended consequences have emerged (Fuchs, 2018).”

• cites Fuchs, 2018Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

Using a dark-side lens pre-emptively can help researchers and practitioners anticipate potential negative effects of AI and develop policies and safeguards.

“However, utilising such a dark-side lens pre-emptively can enable researchers and practitioners to envision the potential negative or unintended effects of AI before they happen and develop appropriate policies and safeguards (Tarafdar et al., 2013).”

• cites Tarafdar et al., 2013Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Giermindl et al. (2021) highlight that one risk of AI is that it can impair transparency and accountability as decisions become more opaque and automated, distanced from a human agent.

“In this special issue, the study of Giermindl et al. (2021) highlights that one of the risks around AI is that it can impair transparency and accountability as decisions become more opaque and automated, thus being distanced from a human agent.”

• cites Giermindl et al., 2021Not checked yet

  Not checked in the free run.

May combine two claims.

Explainable AI comes with challenges, including the difficulty of translating in a structured way how complex algorithmic networks derive outcomes and the challenge of continuously changing algorithms based on updated data.

“Nevertheless, explainable AI comes with a whole set of challenges, including the difficulty in translating in a structured way the approach through which complex algorithmic networks derive outcomes and the challenge of continuously changing algorithms based on updated data (Barredo Arrieta et al., 2020).”

• cites Barredo Arrieta et al., 2020Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Fairness has recently emerged as a key concern for managers and data scientists.

“As a result, fairness has recently emerged as a key concern for managers and data scientists (Osoba & Welser, 2017).”

• cites Osoba & Welser, 2017Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

AI has drawn attention for the negative and unintended consequences it can potentially produce.

“In particular, artificial intelligence (AI) has attracted attention regarding the dark side implications it can potentially result in.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

For a new AI technology or application to be persuasive, it must discuss and ideally test for limitations in terms of fairness.

“However, we argue that before any new AI technology or new application of technology is to be persuasive, it must discuss and ideally test for any limitations in terms of fairness.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Such phenomena have created a strong requirement for explainable AI, where the outcomes and data used to derive a decision from AI must be documentable in a format interpretable by different stakeholders.

“Such phenomena have placed a strong requirement for explainable AI (Barredo Arrieta et al., 2020), whereby the outcomes and data used to derive a decision from AI must be documentable in a format that can be interpreted by different stakeholders (Gunning et al., 2019).”

• cites Barredo Arrieta et al., 2020Not checked yet

  Not checked in the free run.

• cites Gunning et al., 2019Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

Minimising harmful or unintended consequences of AI requires a comprehensive understanding of the role of responsible principles during the design, implementation, and maintenance of AI applications.

“Ensuring that harmful or unintended consequences are minimised or do not occur during the lifespan of AI projects requires a comprehensive understanding of the role of responsible principles during the design, implementation and maintenance of AI applications.”

May combine two claims.

AI introduces new ethical, legal, and governance challenges, including unintended discrimination, biased outcomes, and issues related to customers' awareness and knowledge about AI involvement in decision outcomes.

“AI, therefore, introduces new ethical, legal, and governance challenges -including but not limited tounintended discrimination, biased outcomes, and issues related to the customers' awareness and knowledge about how AI is involved in decision outcomes (Singapore_Government, 2021).”

• cites Singapore_Government, 2021Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

It is important to understand how key stakeholders throughout developing AI systems are responsible and accountable for adverse outcomes.

“For instance, it is important to understand how key stakeholders throughout developing AI systems are responsible and accountable for adverse outcomes.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Responsible AI has gained traction at the policy-making level, with several countries defining fundamental principles that organisations should follow.

“As a testament to its importance, responsible AI has even gained traction at the policy-making level, with several countries now defining the fundamental principles that underlie responsible AI and that organisations, private and public, should need to follow (Jobin et al., 2019).”

• cites Jobin et al., 2019Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

The importance of developing robust and safe AI applications has been heavily discussed over the last years.

“Extending on the previous point of accountability, the importance of developing robust and safe AI applications has been one that has been heavily discussed over the last years (Gehr et al., 2018;Russell et al., 2015;Seshia et al., 2016).”

• cites Gehr et al., 2018Not checked yet

  Not checked in the free run.

• cites Russell et al., 2015Not checked yet

  Not checked in the free run.

• cites Seshia et al., 2016Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Earlier studies examined isolated factors of responsible principles, while recent years have seen a move towards a more holistic understanding of responsible AI.

“While earlier studies have looked at isolated factors relating to responsible principles, such as elimination of bias (Brighton & Gigerenzer, 2015), explainability of AI outcomes (Gunning et al., 2019), or safety and security (Srivastava et al., 2017), the last few years have seen a move towards a more holistic understanding of what constitutes responsible AI (Dignum, 2019).”

• cites Dignum, 2019Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Enabling documentation throughout the process facilitates the requirement of auditability.

“Enabling such ways of documentation throughout the process also facilitates the requirement of auditability.”

Probably not a substantial claim: may not pass S2 (checkable).

Such cases set a precedent for paying careful attention to how data is tagged and how training of AI algorithms might influence outcomes.

“Such cases set a precedent for paying careful attention to how data is tagged and how this is done to ensure non-bias, as well as how the training of AI algorithms might also influence outcomes.”

May combine two claims.

Prominent cases of AI adverse effects include Facebook mis-tagging individuals as primates, Google's biased hiring AI disadvantaging women and Asian candidates, and Amazon's hiring AI with biases against women.

“Some prominent cases include the mishap of Facebook mis-tagging individuals as primates (Wiggers, 2021), Google's biased hiring AI application which disadvantaged women and Asian candidates (Weston, 2021), as well as Amazon's hiring AI application that also ingrained biases against women (Jeffrey, 2018).”

• cites Wiggers, 2021Not checked yet

  Not checked in the free run.

• cites Weston, 2021Not checked yet

  Not checked in the free run.

• cites Jeffrey, 2018Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Accountability has been extensively discussed in the literature on AI, particularly in high-critical cases such as healthcare and autonomous vehicles.

“Accountability has been extensively discussed in the literature on AI, particularly in cases where the outcome is high-critical; e.g., healthcare (Lebovitz et al., 2021) and the unclear accountability of autonomous vehicles by referring to the justification of intentions, motives, and rationalities.”

• cites Lebovitz et al., 2021Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

The seconds or minutes a surgeon spends evaluating an AI recommendation or spot-checking AI decisions affect how ethical or dark that use is.

“However, the number of seconds or minutes the surgeon spends assessing the AI recommendation or spot-checking the various AI decisions must in some way affect the ethicality or darkness of its use.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.

What counts as dark may change over time, or its negative impacts may appear at different times and have different effects when they occur.

“What constitutes dark may change over time, or its negative impacts may materialise at different points in time and have other effects when they do occur.”

Probably not a substantial claim: may not pass S3 (load-bearing).

In organisational behaviour, the first criterion for classifying something as dark was that it was abnormal or deviant.

“when the field of organisational behaviour (OB) first began to look at the dark side in their context systematically (e.g., bullying, theft, etc.), the first qualifying criterion for something to be considered dark was that it was abnormal or deviant (Linstead et al., 2014).”

• cites Linstead et al., 2014Not checked yet

  Not checked in the free run.

May combine two claims.

Technology's dark, dystopian side includes its capacity to generate deviancy and exceptions and to enforce Kafka-esque compliance and conformity.

“we argue that a dark, dystopian side of technology lies in its ability to create deviancy and exceptions and its ability to force Kafka-esque compliance and conformity (e.g., McCabe, 2014).”

• cites McCabe, 2014Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

The dark side may be unknown.

“The dark side may be unknown.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Aaen et al. (2021) highlight the need for governance practices for functional, stakeholder, and data subsystems dynamics to maintain balance and support project success.

“The authors highlight the need for governance practices for the functional, stakeholder, and data subsystems dynamics to maintain balance and support project success.”

• cites Aaen et al., 2021Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable).

The practical instantiation of ethical values and principles will require trade-offs that require further understanding.

“Thus, the practical instantiation of ethical values and principles will require trade-offs that require further understanding.”

May combine two claims.

When legal rules are overly vague, organizations may violate them or avoid AI adoption or use because they fear legal repercussions.

“In cases where there is too much vagueness, private and public bodies might breach corresponding laws or regulations, or they may even result in non-adoption or use of AI in fear of legal consequences.”

Classified as probably not a substantial claim.

Laws and regulations create obstacles when restrictions are set without adequate mechanisms for compliance.

“they also impose hurdles when defining restrictions without appropriate ways to deal with such requirements.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Whether a surgeon is at the beginning or end of a 24-hour shift when screening or calibrating an algorithm has some influence on the algorithm's ethicality or darkness.

“Similarly, if the surgeon is at the start or end of a 24-hour work shift when screening or calibrating, the algorithm must also have some bearing.”

Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

Human oversight should cover known problems such as bias or specified technical errors and also scan for unknown, unprecedented AI decisions and actions that may have unknown negative effects.

“it is logical that this needs to include oversight of known issues such as bias or specified technical errors and scanning for unknown and unprecedented AI decisions and actions that may have unknown negative impacts.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

Responsible AI principles are high-level and abstract and offer practitioners little guidance on deploying AI in practice.

“they are also quite high-level and abstract and do not provide much guidance for practitioners regarding the realities of deploying AI in practice.”

• AssumptionRests on an assumption: Responsible AI principles are important for developing trustworthy and safe AI applications.

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

Abnormal, deviant, and outlier cases may be positive or offer positive opportunities rather than being dark and needing reduction or elimination.

“The abnormal, the deviant, and the outliers may be positive or present positive opportunities rather than something dark to be reduced or eliminated.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Once something is labelled dark, it is unlikely to be reclassified.

“It is unlikely to be reclassified once something is labelled as dark, e.g., stress, dysfunctional behaviour, or inequality.”

Probably not a substantial claim: may not pass S3 (load-bearing).

In analytics and AI, the dark side label refers to bias, inequality, and similar issues.

“This is also true in the context of analytics and AI, where the dark side term refers to bias, inequality, etc.”

Probably not a substantial claim: may not pass S4 (based).

Research on the dark side generally presumes that the dark side being investigated is already known.

“dark side research usually assumes that the dark side being sought is known.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Rinta-Kahila et al. (2021) underscore the importance of establishing appropriate governance practices in AI projects to avoid unintended consequences.

“In this special issue, the study of Rinta-Kahila et al. (2021) underscores the importance of establishing appropriate governance practices in AI projects to avoid unintended consequences.”

• cites Rinta-Kahila et al., 2021Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

Many dark aspects of AI and analytics may be overlooked or understudied because they do not fit the labels contemporary researchers use for the dark side.

“Many dark aspects of AI and analytics may be ignored or understudied simply because they do not neatly fit the labels that contemporary researchers associate with the dark side concept.”

Probably not a substantial claim: may not pass S2 (checkable).

Something dark in one context may suddenly be reclassified.

“However, logic would dictate that something dark in one instance may suddenly be reclassified.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Marjanovic et al. (2021) argue that society, particularly automated welfare decisions, is ingrained with systemic algorithmic injustices.

“They argue that society, particularly automated welfare decisions, is ingrained with systemic algorithmic injustices.”

• cites Marjanovic et al., 2021Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

Nearly all AI and analytics topics can in principle give rise to dark side issues.

“However, we argue that almost all topics in AI and analytics can, in principle, be the source of dark side issues.”