CoChat
Sign Up
Cited by 229Ethics and Social Impacts of AIExplainable Artificial Intelligence (XAI)Digitalization, Law, and Regulation

Ethical principles in machine learning and artificial intelligence: cases from the field and possible ways forward

Open access
Lo Piano, Samuele·2020·Humanities and Social Sciences Communications
DOI 10.1057/s41599-020-0501-9
Open this paper in CoChat — free accountOpen PDF

Abstract

Decision-making on numerous aspects of our daily lives is being outsourced to machine-learning (ML) algorithms and artificial intelligence (AI), motivated by speed and efficiency in the decision process. ML approaches—one of the typologies of algorithms underpinning artificial intelligence—are typically developed as black boxes. The implication is that ML code scripts are rarely scrutinised; interpretability is usually sacrificed in favour of usability and effectiveness. Room for improvement in practices associated with programme development have also been flagged along other dimensions, including inter alia fairness, accuracy, accountability, and transparency. In this contribution, the production of guidelines and dedicated documents around these themes is discussed. The following applications of AI-driven decision-making are outlined: (a) risk assessment in the criminal justice system, and (b) autonomous vehicles, highlighting points of friction across ethical principles. Possible ways forward towards the implementation of governance on AI are finally examined.

AI snapshot

Written from the paper’s full text.

TL;DR

The paper reviews ethical principles in machine learning and AI, maps the proliferation of guideline documents, and examines two applied cases—criminal justice recidivism-risk assessment and autonomous vehicles—to show trade-offs among fairness, accuracy, transparency, accountability, and privacy. It argues that AI ethics remains nascent, guidelines are mostly voluntary, and governance should include scrutiny tools, stakeholder inclusion, and possibly super partes standards.

Key findings

  • Guideline production has skyrocketed since 2016; Floridi and Cowls identified 47 principles mapping onto five dimensions (beneficence, non-maleficence, autonomy, justice, explicability), while Jobin et al. reviewed 84 documents and classified values around transparency, justice and fairness, non-maleficence, responsibility, and privacy.
  • In the COMPAS case, an investigative report examined a pool of cases with recidivism scores for >18,000 criminal defendants in Broward County, Florida and flagged potential racial bias; the company refused to disclose full proprietary code, and while about 140 variables exist, only core variables were disclosed, with race not among them. Interpretable alternatives reportedly achieved similar accuracy using only two variables—age and total number of previous convictions.
  • Autonomous vehicles pose continuous decisions and a “statistical trolley dilemma”; reported failures include inability to perceive a pedestrian as such and acceleration when braking was required due to conflicting instructions from different algorithms, with millions of lines of code making causality practically impossible to scrutinise.
  • The paper highlights points of friction across ethical dimensions, including transparency versus accountability/privacy/property, fairness versus accuracy, group versus individual fairness, and ex-ante assumptions versus ex-post consequences.

Methods

  • Conceptual/review paper: it discusses the production of AI ethics guidelines and secondary literature, then examines two applied cases—recidivism-risk assessment in the criminal justice system and autonomous vehicles.
  • Draws on cited secondary literature and reports, including Floridi and Cowls, Jobin et al., COMPAS/ProPublica, and the MIT Moral Machine; no primary empirical data collection or systematic review protocol is stated.
  • Main analysis is a narrative synthesis of ethical dimensions, trade-offs, and possible governance ways forward, such as global sensitivity analysis, NUSAP, sensitivity auditing, extended peer communities, and a super partes authority for standards.

Limitations

  • The paper states that AI ethics is at its infancy and that the development process of algorithms cannot be perfect; one must be open to negotiation and work with imperfections and clumsiness.
  • It notes that guidelines are largely voluntary with no sanctionary scheme for non-compliance, accountability is mainly theoretical, transparency may not suffice, and proprietary code is often undisclosed.
  • Evident from the text: there is no formal limitations section; the work is a narrative synthesis with two selected case studies, not a systematic review or empirical evaluation.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1057/s41599-020-0501-9

Cites

71 of 71 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 81 cited sources checked · 0 supported · 0 flagged

  • Premisecitation

    AI lacks a full understanding of the consequences of the actions enacted by algorithms.

    “which also lacks a full understanding of the consequences of the actions enacted by algorithms (Wallach and Allen, 2008).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Wallach and Allen, 2008

      Not checked in the free run.

  • Interpretationnone given

    Loss in accuracy may be produced by the exclusion of politically critical features, such as gender, race, and age, from the pool of training predictive variables.

    “Loss in accuracy may be produced by the exclusion of politically critical features (such as gender, race, age, etc.) from the pool of training predictive variables.”
    Quote in the paperNot a substantial claim
  • Premisecitation

    Issues of garbage-in-garbage-out may be prone to emerge in contexts when external control is entirely removed.

    “Furthermore, issues of garbage-in-garbage-out (Saltelli and Funtowicz, 2014) may be prone to emerge in contexts when external control is entirely removed.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Saltelli and Funtowicz, 2014

      Not checked in the free run.

    • Rests on an assumption: Decision-making on numerous aspects of daily lives is being outsourced to ML algorithms and AI, motivated by speed and efficiency in the decision process.
  • Premisecitation

    Publishing scripts exposes their developers to the public scrutiny of professional programmers, who may find shortcomings in the development of the code.

    “However, publishing scripts expose their developers to the public scrutiny of professional programmers, who may find shortcomings in the development of the code (Sonnenburg, 2007).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Sonnenburg, 2007

      Not checked in the free run.

  • Interpretationreasoning

    There is a possible political trade-off between fairness, demanded by society, and algorithmic accuracy, demanded by a private actor.

    “This aspect points again to possible political issues of a trade-off between fairness, demanded by society, and algorithmic accuracy, demanded by, e.g., a private actor.”
    Quote in the paperSubstantial claim
  • Classified as probably not a substantial claim. 72
  • May combine two claims. 28

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    Full transparency has four main objections: leaking of privacy sensitive data into the open; backfiring into an implicit invitation to game the system; harming company property rights with negative consequences on competitiveness; and inherent opacity of algorithms.

    “Hence, he argues against full transparency along four main lines of reasoning: (i) leaking of privacy sensitive data into the open; (ii) backfiring into an implicit invitation to game the system; (iii) harming of the company property rights with negative consequences on their competitiveness (and on the developers reputation as discussed above); (iv) inherent opacity of algorithms, whose interpret”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites de Laat (2018) source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Articles 21 and 22 of the European Union General Data Protection Regulation include passages functional to AI governance, although further action has been demanded from the European Parliament.

    “Even articles 21 and 22 of the recent European Union General Data Protection Regulation include passages functional to AI governance, although further action has been recently demanded from the European Parliament (De Sutter, 2019).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites De Sutter, 2019

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Watson and Floridi defined a formal framework for interpretable ML, where explanatory accuracy can be assessed against algorithmic simplicity and relevance.

    “To this end, Watson and Floridi (2019) defined a formal framework for interpretable ML, where explanatory accuracy can be assessed against algorithmic simplicity and relevance.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Watson and Floridi (2019)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The European Union High-Level Expert Group produced a deliverable on the required criteria for AI trustworthiness.

    “The group produced a deliverable on the required criteria for AI trustworthiness (Daly, 2019).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Daly, 2019

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Jobin et al. reviewed 84 documents produced by several actors of the field, almost half of which from private companies or governmental agencies.

    “Jobin et al. (2019) reviewed 84 documents, which were produced by several actors of the field, almost half of which from private companies or governmental agencies.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Jobin et al. (2019)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Errors on the Amazon website, such as the quotation of plain items up to 10,000 dollars, have been reported.

    “These have been documented, for instance, in the case of the Amazon website, for which errors, such as the quotation of plain items (often books) up to 10,000 dollars (Smith, 2018) have been reported.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Smith, 2018

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    A process of algorithmic auditing within the software-development company could help in tackling some of the ethical issues raised.

    “Raji et al. (2020) suggest that a process of algorithmic auditing within the software-development company could help in tackling some of the ethical issues raised.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Raji et al. (2020)

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    Resorting to full algorithmic transparency may not be an adequate means to address ethical dimensions; opening up the black-box would not suffice to disclose the algorithm's modus operandi.

    “Ananny and Crawford (2018) comment that resorting to full algorithmic transparency may not be an adequate means to address their ethical dimensions; opening up the black-box would not suffice to disclose their modus operandi.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Ananny and Crawford (2018) source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given

    The European Union High-Level Expert Group on Artificial Intelligence is composed of 52 experts from academia, civil society, and industry.

    “An international European initiative is the multi-stakeholder European Union High-Level Expert Group on Artificial Intelligence, which is composed by 52 experts from academia, civil society, and industry.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Jobin et al.'s classification is around a set of values: transparency, justice and fairness, non-maleficience, responsibility and privacy.

    “The classification proposed by Jobin et al. (2019) is around a slightly different set of values: transparency, justice and fairness, non-maleficience, responsibility and privacy.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Jobin et al. (2019)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    ML algorithms are autonomous and self-sufficient when performing their learning function.

    “ML algorithms are autonomous and self-sufficient when performing their learning function.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Samuel, 1959 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Floridi and Cowls identified a set of 47 principles from statements/declarations elaborated since 2016, which mapped onto five overarching dimensions: beneficence, non-maleficence, autonomy, justice, and explicability.

    “A set of 47 principles has been identified, which mapped onto five overarching dimensions (Floridi and Cowls, 2019): beneficence, non-maleficence, autonomy, justice and, explicability.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Floridi and Cowls, 2019

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Attributing responsibilities in complicated projects where many parties and developers may be involved, known as the problem of many hands, may be very difficult.

    “Attributing responsibilities in complicated projects where many parties and developers may be involved, an issue known as the problem of many hands (Nissenbaum, 1996), may indeed be very difficult.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Nissenbaum, 1996 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationnone given

    Larger interpretability could in principle be achieved by using simpler algorithms, although this may come at the expense of accuracy.

    “Larger interpretability could be in principle achieved by using simpler algorithms, although this may come at the expenses of accuracy.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisecitation

    The increasing use of AI in many fields raises new important questions about its suitability.

    “the increasing use of AI in many fields raises new important questions about its suitability (Yu et al., 2018).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Yu et al., 2018

      Not checked in the free run.

    • Rests on an assumption: Decision-making on numerous aspects of daily lives is being outsourced to ML algorithms and AI, motivated by speed and efficiency in the decision process.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    France's Digital Republic Act gives the right to an explanation as regards decisions on an individual made through the use of administrative algorithms.

    “The France's Digital Republic Act gives the right to an explanation as regards decisions on an individual made through the use of administrative algorithms (Edwards and Veale, 2018).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Edwards and Veale, 2018 source

      Not checked in the free run.

  • May combine two claims.

    Interpretationcitation

    A formal profession with a set of clearly defined and governed good-behaviour practices exists in medicine, but this is not the case for AI.

    “Secondly, a formal profession with a set of clearly defined and governed good-behaviour practices exists in medicine.This is not the case for AI”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    China has been allocating efforts on privacy and data protection.

    “In this context, China has also been allocating efforts on privacy and data protection (Roberts, 2019).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Roberts, 2019

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    In many fields, an increasing amount of functions are being ceded to algorithms to the detriment of human control, raising concern for loss of fairness and equitability.

    “In all these fields, an increasing amount of functions are being ceded to algorithms to the detriment of human control, raising concern for loss of fairness and equitability (Sareen et al., 2020).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Sareen et al., 2020 source

      Not checked in the free run.

    • Rests on an assumption: Decision-making on numerous aspects of daily lives is being outsourced to ML algorithms and AI, motivated by speed and efficiency in the decision process.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The production of dedicated documents on AI ethics has been skyrocketing from 2016.

    “The production of dedicated documents has been skyrocketing from 2016 (Jobin et al., 2019).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Jobin et al., 2019

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Accountability and responsibility were rarely defined in the studies reviewed by Jobin et al.

    “Other potentially relevant dimensions, such as accountability and responsibility, were rarely defined in the studies reviewed by these authors.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Jobin et al. (2019)

      Not checked in the free run.

  • May combine two claims.

    Interpretationnone given

    Medical professionals rest on common aims and fiduciary duties, which AI developers lack.

    “Firstly, the medical professional rests on common aims and fiduciary duties, which AI developers lack.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    According to Floridi and Cowls, a prominent role is played by the autonomy dimension: the possibility of refraining from ceding decision power to AI for overriding reasons.

    “According to Floridi and Cowls (2019) a prominent role is also played by the autonomy dimension; the possibility of refraining from ceding decision power to AI for overriding reasons (e.g., the gain of efficacy is not deemed fit to justify the loss of control over decision-making).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Floridi and Cowls (2019)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

    Interpretationreasoning

    While mistakes about monetary values may be easy to spot, the situation may become more complex and less intelligible when incommensurable dimensions come to play.

    “While mistakes about monetary values may be easy to spot, the situation may become more complex and less intelligible when incommensurable dimensions come to play.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Explicability is a new dimension specifically acknowledged in the case of AI, while the other dimensions were already identified in bioethics.

    “The latter is a new dimension specifically acknowledged in the case of AI, while the others were already identified in the controversial domain of bioethics.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Floridi and Cowls, 2019

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Mittelstadt noted that the dimensions addressed in AI ethics are converging towards those of medical ethics.

    “Mittelstadt (2019) critically analysed the current debate and actions in the field of AI ethics and noted that the dimensions addressed in AI ethics are converging towards those of medical ethics.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Mittelstadt (2019) source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Seven of the most prominent value statements from the AI/ML fields were examined in Greene et al. (2019).

    “Seven of the most prominent value statements from the AI/ML fields were examined in Greene et al. (2019)”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Greene et al. (2019)

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation

    Transparency and accountability may be at odds in the case of algorithms.

    “de Laat (2018) argues that transparency and accountability may even be at odds in the case of algorithms.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites de Laat (2018) source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Contrasting dimensions in the theoretical framing of AI ethics emerged from Jobin et al.'s review, regarding interpretation of ethical principles, reasons for their importance, and ownership and responsibility of their implementation.

    “Contrasting dimensions in terms of the theoretical framing of the issue also emerged from the review of Jobin et al. (2019), as regards interpretation of ethical principles, reasons for their importance, ownership and responsibility of their implementation.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Jobin et al. (2019)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

    Interpretationcitation

    These correlations may be discriminatory, if not illegal.

    “However, these correlations may be discriminatory, if not illegal.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    A main shortcoming of ML approaches is that they resort to proxies for driving trends, such as a person's ZIP code or language, in relation to the capacity of an individual to pay back a loan or handle a job.

    “according to O'Neil (2016), a main shortcoming of ML approaches is the fact these resort to proxies for driving trends, such as person's ZIP code or language in relation with the capacity of an individual to pay back a loan or handle a job, respectively.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites O'Neil (2016)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Machines' autonomy could be reduced in favour of human autonomy according to the metaautonomy dimension.

    “In other words, machines autonomy could be reduced in favour of human autonomy according to this metaautonomy dimension.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Floridi and Cowls (2019)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    Conflicts may emerge between the requirements to overcome potential algorithm deficits in accuracy associated with large data bases and the individual rights to privacy and autonomy of decision.

    “Conflicts may also emerge between the requirements to overcome potential algorithm deficits in accuracy associated with large data bases and the individual rights to privacy and autonomy of decision.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationreasoning

    This also applies to different ethical principles, resulting in trade-offs, difficulties in setting prioritisation strategies, operationalisation and actual compliance with the guidelines.

    “This also applies to different ethical principles, resulting in the trade-offs previously discussed, difficulties in setting prioritisation strategies, operationalisation and actual compliance with the guidelines.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given

    Sensitive governmental areas, such as national security and defence, and the private sector are excluded from France's Digital Republic Act.

    “Sensitive governmental areas, such as national security and defence, and the private sector (the largest user and producer of ML algorithms by far) are excluded from this document.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    The algorithm may also face a dilemma between a low probability of a serious harm and a higher probability of a mild harm.

    “The algorithm may also face a dilemma between low probability of a serious harm and higher probability of a mild harm.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    A modelling of the modelling process would assist in the model transparency and in addressing questions such as: Are the results from a particular model more sensitive to changes in the model and the methods used to estimate its parameters, or to changes in the data?

    “Additionally, a modelling of the modelling process would assist in the model transparency and in addressing questions such as: Are the results from a particular model more sensitive to changes in the model and the methods used to estimate its parameters, or to changes in the data?(Majone, 1989).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Majone, 1989

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given

    Accuracy in the context of self-autonomous vehicles rests on their capacity to correctly simulate the course of events.

    “Accuracy in the context of self-autonomous vehicles rests on their capacity to correctly simulate the course of the events.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationnone given

    Autonomous vehicles pose different challenges because a continuity of decisions is to be enacted while the vehicle is moving, unlike a one-off decision such as the assessment of recidivism risk.

    “The case of autonomous vehicles, also known as self-driving vehicles, poses different challenges as a continuity of decisions is to be enacted while the vehicle is moving.It is not a one-off decision as in the case of the assessment of recidivism risk.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    The total number of variables in COMPAS amounts to about 140, only the core variables were disclosed, and the race of the subject was not one of those.

    “While the total number of variables amounts to about 140, only the core variables were disclosed (Northpointe, 2012).The race of the subject was not one of those.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Northpointe, 2012

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationnone given

    The set of decision rules underpinning the AI algorithm derives from human-made assumptions, such as where to define the boundary between action and no action, between different possible choices.

    “Eventually, the set of decision rules underpinning the AI algorithm derives from human-made assumptions, such as, where to define the boundary between action and no action, between different possible choices.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Interpretationreasoning

    It poses further tension between the accuracy a vehicle manufacturer seeks and the capability to keep up the agreed fairness standards upstream from the algorithm development process.

    “It also poses further tension between the accuracy a vehicle manufacturer seeks and the capability to keep up the agreed fairness standards upstream from the algorithm development process.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisenone given

    The analysis was debunked by the company, which refused to disclose the full details of its proprietary code.

    “This analysis was debunked by the company, which, however, refused to disclose the full details of its proprietary code.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Premisecitation

    This corresponds to a risk management situation with a number of nuances and inherent complexity.

    “This corresponds to a risk management situation by all means, with a number of nuances and inherent complexity (Goodall, 2016).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Goodall, 2016 source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Even the data on which an algorithm is trained on are not an objective truth, they are dependent upon the context in which they have been produced.

    “Even the data on which an algorithm is trained on are not an objective truth, they are dependent upon the context in which they have been produced (Neff et al., 2017).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Neff et al., 2017 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given

    Value statements and guidelines have been elaborated by political and multi-stakeholder organisations.

    “As to address these dimensions, value statements and guidelines have been elaborated by political and multi-stakeholder organisations.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Premisecitation

    Tools of post-normal-science inspiration for knowledge and modelling quality assessment could be adapted to the analysis of algorithms, such as the NUSAP notation system and sensitivity auditing.

    “Tools of post-normal-science inspiration for knowledge and modelling quality assessment could be adapted to the analysis of algorithms, such as the NUSAP (Numeral Unit Spread Assessment Pedigree) notation system for the management and communication of uncertainty (Funtowicz and Ravetz, 1990;Van Der Sluijs, 2005) and sensitivity auditing (Saltelli and Funtowicz, 2014), respectively.”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites Funtowicz and Ravetz, 1990 source

      Not checked in the free run.

    • cites Van Der Sluijs, 2005 source

      Not checked in the free run.

    • cites Saltelli and Funtowicz, 2014

      Not checked in the free run.

  • May combine two claims.

    Premisenone given

    The COMPAS algorithm, developed by Northpointe, attributes a 2-year recidivism-risk score to arrested people and evaluates the risk of violent recidivism as a score.

    “The COMPAS algorithm, developed by the private company Northpointe, attributes a 2-year recidivism-risk score to arrested people.It also evaluates the risk of violent recidivism as a score.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The USA faces the issue of the world's highest incarcerated population, both in absolute and per-capita terms.

    “This country faces the issue of the world's highest incarcerated population, both in absolute and per-capita terms (Brief, 2020).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Brief, 2020

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationreasoning

    Coding algorithms that assure fairness in autonomous vehicles can be a very challenging issue.

    “Coding algorithms that assure fairness in autonomous vehicles can be a very challenging issue.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    No corrective action in the algorithm code may be possible at this stage, with no room for improvement in accuracy.

    “Hence, no corrective action in the algorithm code may be possible at this stage, with no room for improvement in accuracy.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    Tools for technically scrutinising the potential behaviour of an algorithm and its uncertainty already exist and could be included in the workflow of algorithm development.

    “Tools for technically scrutinising the potential behaviour of an algorithm and its uncertainty already exist and could be included in the workflow of algorithm development.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Global sensitivity analysis may help in exploring how the uncertainty in the input parameters and modelling assumptions would affect the output.

    “For instance, global sensitivity analysis (Saltelli, 2008) may help in exploring how the uncertainty in the input parameters and modelling assumptions would affect the output.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Saltelli, 2008

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Rudin found that the COMPAS algorithm seemed to behave differently from the intentions of its creators, with a non-linear dependence on age and a weak correlation with one's criminal history.

    “Rudin (2019) found that the algorithm seemed to behave differently from the intentions of their creators (Northpointe, 2012) with a non-linear dependence on age and a weak correlation with one's criminal history.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites Rudin (2019)

      Not checked in the free run.

    • cites Northpointe, 2012

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The moral-machine experiment is a serious game where users are requested to fulfil the function of an autonomous-vehicle decision-making algorithm in a situation of danger, entailing choices that would prioritise the safety of some categories of users over others.

    “An exercise to appreciate the value-ladenness of these decisions is the moral-machine experiment (Massachussets Institute of Technology 2019)-a serious game where users are requested to fulfil the function of an autonomous-vehicle decision-making algorithm in a situation of danger.This experiment entails performing choices that would prioritise the safety of some categories of users over others.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Massachussets Institute of Technology 2019

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Unforeseen events can still play a prominent role, and profoundly affect the vehicles behaviour and reactions.

    “While this is based on physics and can be informed by the numerous sensors these vehicles are equipped with, unforeseen events can still play a prominent role, and profoundly affect the vehicles behaviour and reactions (Yurtsever et al., 2020).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Yurtsever et al., 2020

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationnone given

    The field of AI ethics is just at its infancy and it is still to be conceptualised how AI developments that encompass ethical dimensions could be attained.

    “However, the field of AI ethics is just at its infancy and it is still to be conceptualised how AI developments that encompass ethical dimensions could be attained.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    According to the report, the recidivism-risk was systematically overestimated for black people: the decile distribution of white defendants was skewed towards the lower end, while the decile distribution of black defendants was only slightly decreasing towards the higher end; the risk of violent recidivism within 2 years followed a similar trend.

    “According to the authors of the report, the recidivism-risk was systematically overestimated for black people: the decile distribution of white defendants was skewed towards the lower end.Conversely, the decile distribution of black defendants was only slightly decreasing towards the higher end.The risk of violent recidivism within 2 years followed a similar trend.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Angwin and Larson, 2016

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationreasoning

    Minimising the overall physical harm may be achieved by implementing an algorithm that, in the circumstance of an unavoidable collision, would target the vehicles with the highest safety standards.

    “Minimising the overall physical harm may be achieved by implementing an algorithm that, in the circumstance of an unavoidable collision, would target the vehicles with the highest safety standards.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The Alan Turing Institute released a guide for responsible design and implementation of AI that covers the whole life-cycle of design, use, and monitoring.

    “For instance, The Alan Turing Institute released a guide for responsible design and implementation of AI (Leslie, 2019) that covers the whole life-cycle of design, use, and monitoring.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Leslie, 2019

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    Algorithms are learning by direct experience and they may still end up conflicting with the initial set of ethical rules around which they have been conceived.

    “One should also not forget that these algorithms are learning by direct experience and they may still end up conflicting with the initial set of ethical rules around which they have been conceived.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    Odds may emerge between the interest of the vehicle owner and passengers, on one side, and the collective interest of minimising the overall harm, on the other.

    “Odds may emerge between the interest of the vehicle owner and passengers, on one side, and the collective interest of minimising the overall harm, on the other.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationreasoning

    Such conflicts may exacerbate tensions, further complicating agreeing on standards and practices.

    “Such conflicts may exacerbate tensions, further complicating agreeing on standards and practices.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    An overarching meta-framework for the governance of AI in experimental technologies (i.e., robot use) has also been proposed.

    “An overarching meta-framework for the governance of AI in experimental technologies (i.e., robot use) has also been proposed (Rego de Almeida et al., 2020).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Rego de Almeida et al., 2020

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    If not locked down, transparency today may not be helpful in understanding what the system does tomorrow.

    “If not, transparency today may not be helpful in understanding what the system does tomorrow.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Trying to revert the current state of affairs may expose the first movers in the AI field to a competitive disadvantage.

    “Trying to revert the current state of affairs may expose the first movers in the AI field to a competitive disadvantage (Morley et al., 2019).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Morley et al., 2019

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given

    Attempts to make the process more inclusive, with a higher participation from all the stakeholders, are scant.

    “As are the attempt to make the process more inclusive, with a higher participation from all the stakeholders.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

    Interpretationreasoning

    This initiative stems from the attempt to include all the forms of governance put forth and would rest on an integrated set of feedback and interactions across dimensions and actors.

    “This initiative stems from the attempt to include all the forms of governance put forth and would rest on an integrated set of feedback and interactions across dimensions and actors.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Berk (2019) asked for the intervention of super partes authorities to define standards of transparency, accuracy and fairness for algorithm developers in line with the role of the Food and Drug administration in the US and other regulation bodies.

    “An interesting proposal comes from Berk (2019), who asked for the intervention of super partes authorities to define standards of transparency, accuracy and fairness for algorithm developers in line with the role of the Food and Drug administration in the US and other regulation bodies.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Berk, 2019

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationreasoning

    If a ML system with real world consequences is 'locked down' to increase transparency, the algorithm could become defective.

    “If yes, the algorithm could become defective.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given

    Attempts to expose to public scrutiny the algorithms developed are yet scant.

    “Attempts to expose to public scrutiny the algorithms developed are yet scant.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Premisecitation

    The post-normal-science concept of extended peer communities could assist also in this endeavour.

    “The post-normal-science concept of extended peer communities could assist also in this endeavour (Funtowicz and Ravetz, 1997).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Funtowicz and Ravetz, 1997

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Interpretationreasoning

    The development pace of new algorithms would be necessarily reduced so as to comply with the standards defined and the required clearance processes.

    “The development pace of new algorithms would be necessarily reduced so as to comply with the standards defined and the required clearance processes.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationnone given

    The development of decision-making algorithms remains quite obscure in spite of the concerns raised and the intentions manifested to address them.

    “The development of decision-making algorithms remains quite obscure in spite of the concerns raised and the intentions manifested to address them.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Premisecitation

    Example-based explanations may also contribute to an effective engagement of all the parties by helping in bridging technical divides across developers, experts in other fields, and lay-people.

    “Example-based explanations (Molnar, 2020) may also contribute to an effective engagement of all the parties by helping in bridging technical divides across developers, experts in other fields, and lay-people.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Molnar, 2020

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Interpretationreasoning

    A shared regulation could help in tackling the potential competitive disadvantage a first mover may suffer.

    “A shared regulation could help in tackling the potential competitive disadvantage a first mover may suffer.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S1 (asserted), S2 (checkable), S4 (based). May combine two claims.

    Premisenone given

    ML systems that keep learning are dangerous and hard to understand because they can quickly change.

    “As noted by a perceptive reviewer, ML systems that keep learning are dangerous and hard to understand because they can quickly change.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    Greene et al. (2019) found seven common core elements across these documents: (i) design's moral background (universal concerns, objectively measured); (ii) expert oversight; (iii) valuesdriven determinism; (iv) design as locus of ethical scrutiny; (v) better building; (vi) stakeholder-driven legitimacy; and, (vii) machine translation.

    “Greene et al. (2019) found seven common core elements across these documents: (i) design's moral background (universal concerns, objectively measured); (ii) expert oversight; (iii) valuesdriven determinism; (iv) design as locus of ethical scrutiny; (v) better building; (vi) stakeholder-driven legitimacy; and, (vii) machine translation.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Greene et al., 2019

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationcitation

    The development process of the algorithm cannot be perfect in this setting.

    “Hence, the development process of the algorithm cannot be perfect in this setting, one has to be open to negotiation and unavoidably work with imperfections and clumsiness (Ravetz, 1987).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Ravetz, 1987 source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    Points of friction across ethical dimensions may emerge, e.g., between transparency and accountability, or accuracy and fairness as highlighted in the case studies.

    “One should also not forget that points of friction across ethical dimensions may emerge, e.g., between transparency and accountability, or accuracy and fairness as highlighted in the case studies.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationreasoning

    Hard-coding the set of rules on the behaviour of the algorithm, once these are agreed upon among the involved stakeholders, would prevent the algorithm learning process from conflicting with the standards agreed.

    “This would prevent the algorithmlearning process from conflicting with the standards agreed.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Interpretationnone given

    Identifying a relevant pool of social actors may require an important effort in terms of stakeholders' mapping so as to assure a complete, but also effective, governance in terms of number of participants and simplicity of working procedures.

    “Identifying a relevant pool of social actors may require an important effort in terms of stakeholders' mapping so as to assure a complete, but also effective, governance in terms of number of participants and simplicity of working procedures.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    In this setting, seeking algorithm transparency would not be harmful for their developers as scrutiny would be delegated to entrusted intermediate parties, to take place behind closed doors.

    “In this setting, seeking algorithm transparency would not be harmful for their developers as scrutiny would be delegated to entrusted intermediate parties, to take place behind closed doors (de Laat, 2018).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites de Laat, 2018 source

      Not checked in the free run.

83 of 83

  • The paper relies on this without support.

    Decision-making on numerous aspects of daily lives is being outsourced to ML algorithms and AI, motivated by speed and efficiency in the decision process.

    “Decision-making on numerous aspects of our daily lives is being outsourced to machinelearning (ML) algorithms and artificial intelligence (AI), motivated by speed and efficiency in the decision process.”

  • The paper relies on this without support.

    ML approaches are typically developed as black boxes, so their code scripts are rarely scrutinised and interpretability is usually sacrificed in favour of usability and effectiveness.

    “ML approaches-one of the typologies of algorithms underpinning artificial intelligence-are typically developed as black boxes.The implication is that ML code scripts are rarely scrutinised; interpretability is usually sacrificed in favour of usability and effectiveness.”

  • The paper relies on this without support.

    Room for improvement in practices associated with programme development has been flagged along dimensions including fairness, accuracy, accountability, and transparency.

    “Room for improvement in practices associated with programme development have also been flagged along other dimensions, including inter alia fairness, accuracy, accountability, and transparency.”

  • The paper relies on this without support.

    Developers of algorithms may not be capable of explaining in plain language how a given tool works and what functional elements it is based on.

    “Moreover, developers of algorithm may not be capable of explaining in plain language how a given tool works and what functional elements it is based on.”
4 of 4

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Algorithm-driven predictive systems threaten the autonomy of students and teachers and their capacity to direct their own lives.

    “Predictive systems which are powered by algorithms jeopardize students and teachers’ autonomy and their ability to govern their own life [46, 47].”

    From Artificial intelligence in education: Addressing ethical challenges in K-12 settings (2022) · cites this paper as [46]

    The Introduction (page unavailable) says that ceding functions to algorithms can diminish human control, and “Points of friction between ethical dimensions” discusses retaining human decision-making autonomy. The paper lists education as an application, but these passages do not establish that predictive systems threaten students’ and teachers’ autonomy or their ability to govern their lives. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • AI ethics is at a nascent stage.

    “Given the nascent stage of AI ethics, there is a need to devise frameworks for AI advancements to incorporate ethical considerations (Lo Piano, 2020).”

    From Navigating the perils of artificial intelligence: a focused review on ChatGPT and responsible research and innovation (2024) · cites this paper as Lo Piano, 2020

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Cowls citation year differs from reference

    Minor

    The text cites Cowls as a 2019 source, but the standalone Cowls reference is dated 2020.

  • Jobin surname differs from reference

    Minor

    The text attributes the review of AI ethics documents to Jobin et al. (2019), while the corresponding reference lists the lead author as Jobi.

  • De Sutter citation year differs from reference

    Minor

    The text cites De Sutter as a 2019 source, but the corresponding reference is dated 2020.

3 of 3

Cites

Flagged first, then by citations

Research works printed but not linked: 2

  1. Funtowicz S, Ravetz J (1997) Environmental problems, post-normal science, and extended peer communities. Études et Recherches sur les Systémes Agraires et le Développement. INRA Editions. pp. 169–175
  2. McCarthy J, Minsky ML, Rochester N, Shannon CE (2006) A proposal for the dartmouth summer research project on artificial intelligence, August 31, 1955. AI Mag 27:12–12

Cited by

Sorted by citations

Related

Authors
Lo Piano, S.
ORCID
Lo Piano, Samuele
Venue
Humanities and Social Sciences Communications
Publisher
Springer Science and Business Media LLC
Published
2020-06-17
Volume
7
Issue
1
ISSN
  • 2055-1045 (ISSN-L)
  • 2662-9992
DOI
10.1057/s41599-020-0501-9
OpenAlex
W3029725182
Semantic Scholar
a5bd811e5358e477c079278ee2a0066dc81a386f
Cited by
229
References
69
Citations by year
2026: 18 · 2025: 39 · 2024: 44 · 2023: 45 · 2022: 36 · 2021: 40 · 2020: 7
FWCI
43.61
Type
journal article
Language
en
Open access
Licence
Locations
  • Humanities and Social Sciences Communications (publishedVersion, cc-by)
  • CentAUR (University of Reading) (submittedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
Universitetet i Bergen
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Social Sciences › Law › Digitalization, Law, and Regulation
Keywords
ethical principles, machine learning, artificial intelligence, algorithmic decision-making, black-box algorithms, interpretability, algorithmic fairness, accountability, transparency, AI governance, autonomous vehicles, AI guidelines

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-06.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-06.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review