CoChat

From What to How: An Initial Review of Publicly Available AI Ethics Tools, Methods and Research to Translate Principles into Practices

Open access
Morley, Jessica, Floridi, Luciano, Kinsey, Libby, Elhalal, Anat · 2020 · Science and Engineering Ethics

The DOI matches its Crossref record; 103 of the 104 references it cites were found in a registry or an index; 12 of the 50 sources its claims cite were checked for support.

Abstract

The debate about the ethical implications of Artificial Intelligence dates from the 1960s (Samuel in Science, 132(3429):741–742, 1960. 10.1126/science.132.3429.741 ; Wiener in Cybernetics: or control and communication in the animal and the machine, MIT Press, New York, 1961). However, in recent years symbolic AI has been complemented and sometimes replaced by (Deep) Neural Networks and Machine Learning (ML) techniques. This has vastly increased its potential utility and impact on society, with the consequence that the ethical debate has gone mainstream. Such a debate has primarily focused on principles—the ‘what’ of AI ethics (beneficence, non-maleficence, autonomy, justice and explicability)—rather than on practices, the ‘how.’ Awareness of the potential issues is increasing at a fast rate, but the AI community’s ability to take action to mitigate the associated risks is still at its infancy. Our intention in presenting this research is to contribute to closing the gap between principles and practices by constructing a typology that may help practically-minded developers apply ethics at each stage of the Machine Learning development pipeline, and to signal to researchers where further work is needed. The focus is exclusively on Machine Learning, but it is hoped that the results of this research may be easily applicable to other branches of AI. The article outlines the research method for creating this typology, the initial findings, and provides a summary of future research needs.

Overview

CoChat Summary

TL;DR

This paper reviews publicly available AI ethics tools and methods and constructs an “Applied AI Ethics” typology mapping five ethical principles to seven stages of ML development. It finds the tools are unevenly distributed, skewed toward explicability and post hoc explanation, and often not yet usable or production-ready.

Key findings

  • The review identified 425 sources from over 1000 initial results and built a typology combining beneficence, non-maleficence, autonomy, justice, and explicability with seven development stages: business/use-case development, design, training/test data procurement, building, testing, deployment, and monitoring.
  • Tool availability is uneven: non-maleficence has tools across stages, beneficence tools are mostly for early planning, and the greatest concentration is on explicability during testing, including statistical methods such as LIME, SHAP, and sensitivity analysis, which the authors call “at most necessary but not sufficient.”
  • Few tools assess individual or societal impact; the deployment column is sparsely populated, and many tools lack usability, practical guidance, or low skill requirements. The paper cites that 79% of tech workers report wanting practical resources for ethical considerations.
  • The authors argue the field needs coordinated, multidisciplinary work to fill typology gaps, evaluate existing tools, and create a “how” for every “what” across the ML pipeline.

Methods

  • Literature review searching Scopus, arXiv, PhilPapers, and Google, using terms combined with “machine learning OR Artificial Intelligence”; original searches ran in February 2019, with weekly alerts reviewed until mid-July 2019.
  • Every result was checked for relevance, actionability by ML developers, and generalisability across industry sectors; 425 sources were reviewed from over 1000 results.
  • The typology was created by combining five ethical principles with seven stages from the ICO auditing framework, then translating principles into tangible system requirements adapted from Chapter II of the European Commission’s “Ethics Guidelines for Trustworthy AI.”

Limitations

  • The paper states the research question was very broad, and compromises—such as database choices and excluding industry-sector-specific tools—likely missed many publicly available tools and proprietary tools, including PWC’s “Responsible AI Toolkit.”
  • The paper states the typology’s neat stage distinctions may not exist in practice and could reduce usability; it also states it is unclear how the identified tools will improve the governability of algorithmic systems.
  • Evident in the text: the categorisations may appear “somewhat ad-hoc” (noted by an anonymous reviewer), and many tools are immature, not production-ready, poorly documented, or require high skill levels; there is no clear problem statement or business case for investment.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1007/s11948-019-00165-5

Cites

103 of 103 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

8 Valid · 4 Not valid · 33 Not checked · 47 Own results and interpretations

42 more claims · 1 with their cited sources checked

Create a free account to see all 92 claims and to verify the 38 references still open.

See all claims — free account
  • It may state two separate claims as one. 22
  • Probably not a substantial claim. 27
  • The quote only partly states this claim. 1

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

3 more claims about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Citations

Cites

Show all 103 references

Research works printed but not linked: 2

  1. Lighthill, J. (1973). ‘Artificial Intelligence: A general survey’ in Artificial Intelligence: A paper symposium. Retrieved from UK Science Research Council website: http://www.chilton-computing.org.uk/inf/literature/reports/lighthill_report/p001.htm.
  2. Lipton, Z. C. (2016). The mythos of model interpretability. arXiv:1606.03490 [Cs, Stat]. · arXiv

Cited by

See all 883

Metadata

Authors
Morley, J., Floridi, L., Kinsey, L., & Elhalal, A.
Affiliations
  • University of Oxford (GB)
  • Digital Catapult (GB)
Corresponding author
Morley, Jessica
Venue
Science and Engineering Ethics
Published
2019-12-11
Volume
26
Issue
4
Pages
2141-2168
ISSN
  • 1353-3452 (ISSN-L)
  • 1471-5546
OpenAlex
W2996541402
Cited by
882
References
104
Citations by year
2026: 253 · 2025: 206 · 2024: 128 · 2023: 90 · 2022: 93 · 2021: 84 · 2020: 25 · 2019: 3
FWCI
89.66
Type
journal article
Language
en
Open access
Licence
Locations
  • Science and Engineering Ethics (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
Funders
Digital Catapult
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Safety Systems Engineering in Autonomy
Keywords
AI ethics, machine learning ethics, beneficence, non-maleficence, autonomy, justice, explainability
MeSH terms
Machine Learning, Animals, Artificial Intelligence, Humans, Research Personnel, Social Justice, Beneficence

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: From What to How: An Initial Review of Publicly Available AI Ethics Tools, Methods and Research to Translate Principles into Practices. https://app.cochat.ai/papers/10.1007/s11948-019-00165-5
The paper
Morley, J., Floridi, L., Kinsey, L., & Elhalal, A. (2020). From What to How: An Initial Review of Publicly Available AI Ethics Tools, Methods and Research to Translate Principles into Practices. Science and Engineering Ethics. https://doi.org/10.1007/s11948-019-00165-5

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.