The Ethics of AI Ethics: An Evaluation of Guidelines
Open accessThe DOI matches its Crossref record; 98 of the 98 references it cites were found in a registry or an index; 5 of the 88 sources its claims cite were checked for support.
Abstract
Abstract Current advances in research, development and application of artificial intelligence (AI) systems have yielded a far-reaching discourse on AI ethics. In consequence, a number of ethics guidelines have been released in recent years. These guidelines comprise normative principles and recommendations aimed to harness the “disruptive” potentials of new AI technologies. Designed as a semi-systematic evaluation, this paper analyzes and compares 22 guidelines, highlighting overlaps but also omissions. As a result, I give a detailed overview of the field of AI ethics. Finally, I also examine to what extent the respective ethical principles and values are implemented in the practice of research, development and application of AI systems—and how the effectiveness in the demands of AI ethics can be improved.
Overview
CoChat Summary
## TL;DR
This paper semi-systematically reviews 22 AI ethics guidelines, finding strong overlap on a few technicalizable topics but many important omissions. It argues that AI ethics is often ineffective in practice and should be supplemented by more concrete technical guidance and a shift toward virtue ethics.
## Key findings
- The paper analyzes and compares 22 AI ethics guidelines and finds that accountability, privacy, and fairness appear in about 80% of them, making them the closest thing to a common minimum standard.
- The author reports that guidelines from industrial contexts name on average 9.1 distinct ethical aspects, while science-based ethics codes average 10.8.
- Only two guidelines contain genuinely technical explanations, and these are described as very sparse; the most extensive technical commentary is in “The Malicious Use of AI”.
- The paper says a controlled study found the effectiveness of ethics guidelines or codes to be “almost zero”: among 63 software engineering students and 105 professional software developers, there was no statistically significant difference in responses between those who did and did not see the ACM code of ethics.
- The paper highlights major omissions across guidelines, including political abuse of AI, hidden social and ecological costs, lack of diversity, robot ethics, trolley problems, and public-private partnerships/industry-funded research.
## Methods
- Semi-systematic literature analysis of AI ethics guidelines, with a deliberate focus on comprehensive guideline documents rather than all AI-ethics-related writings.
- The final sample consisted of 22 major ethical guidelines, identified through searches in Google, Google Scholar, Web of Science, ACM Digital Library, arXiv, and SSRN, plus manual reference screening and the Algorithm Watch AI Ethics Guidelines Global Inventory.
- The paper compares which ethical issues are present or absent in the guidelines, and then relates those principles to practice using cited empirical studies and examples from AI development and deployment.
## Limitations
- The author states the sample is biased toward western/northern documents and excludes guidelines not written in English.
- The selection deliberately excludes guidelines focused on robot ethics, big data, algorithms, or robotics, so omissions in those areas are partly a result of the sampling method.
- The paper notes that its list of omissions is non-exhaustive.
- Evident from the text: the paper is primarily a theoretical/semi-systematic overview, not a direct empirical test of all AI ethics guideline implementations.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
4 Valid · 3 Not valid · 63 Not checked · 64 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“So far, some papers have been published on the subject of teaching ethics to data scientists (Garzcarek and Steuer 2019; Burton etal. 2017; Goldsmith and Burton 2017; Johnson 2017) but by and large very little to nothing has been written about the tangible implementation of ethical goals and values.”
What the sources say
- Supportedcites Garzcarek and Steuer 2019 source
The citation is attached to the statement that papers have discussed teaching ethics to data scientists, not to the citing authors’ subsequent assessment of writing on tangible implementation. Section 5.2 (p. 14) explicitly argues that ethics should be part of the data-science curriculum. (LLM, conf=0.90)
- Not matchedcites Burton etal. 2017
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Supportedcites Goldsmith and Burton 2017 source
The citation appears in a list of papers about teaching ethics to data scientists, not as support for the sentence’s later claim about how little has been written on implementation. The Introduction (p. 1) presents the paper as an argument for ethical training for AI students and practitioners, and the Conclusions (p. 4) advocate teaching ethics in AI classes. (LLM, conf=0.91)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“In the light of these two approaches, the traditional type of AI ethics can be assigned to the deontological concept (Mittelstadt 2019).”
What the sources say
- Partly supportedcites Mittelstadt 2019 sourceExperimental — accuracy not yet measured
The paper describes a prominent, principle-based approach to AI ethics and says AI ethics initiatives have converged around principles resembling those of medical ethics (opening section; page unavailable). It does not classify that approach as deontological, so the claimed assignment to the deontological concept is not supported by our copy. (LLM, conf=0.78)
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“My analysis of the distribution of female and male authors of the guidelines, as far as authors were indicated in the documents, showed that the proportion of women was 41.7%.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Finally, yet importantly, it is noteworthy that almost all guidelines suggest that technical solutions exist for many of the problems described.”
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“What is also largely absent from most guidelines is the issue of a lack in diversity within the AI community.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
54 more claims
Create a free account to see all 134 claims and every source verdict.
- Probably not a substantial claim. 59
- It may state two separate claims as one. 29
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Ethical concerns arising from the design and deployment of AI technology have motivated a number of initiatives [47]—such as the Ethical Guidelines for Trustworthy AI by the European Commission expert group on AI, and Microsoft’s FATE: Fairness, Accountability, Transparency, and Ethics in AI—addressing issues including mitigating bias and discrimination in AI systems and fairness in the use of AI”
From Future directions for chatbot research: An interdisciplinary research agenda (2021) · cites this paper as [47]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Additional recommendations on AI ethics teaching content include “explainability,” “liability,” and “accountability,” which are also considered important by available guidelines [30,40].”
From Proposing a Principle-Based Approach for Teaching AI Ethics in Medical Education (2024) · cites this paper as [30]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This emphasis is also echoed by existing guidelines and frameworks regarding AI ethics [30].”
From Proposing a Principle-Based Approach for Teaching AI Ethics in Medical Education (2024) · cites this paper as [30]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“En dépit de nombreuses chartes, discussions et nombreux documents sur les bonnes pratiques éthiques, l’espoir est faible que l’IA puisse éventuellement se montrer généralement conforme (Hagendorff, 2020) :”
From Gouvernance algorithmique, intelligence artificielle, enjeux, éthique : esquisse d’une analyse critique (2021) · cites this paper as Hagendorff, 2020
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The experience of mixed emotional responses and ethnical navigation was supported by literature (Hagendorff, 2020;Mamo et al., 2024;Mittelstadt, 2019).”
From Into the Unknown: AI Adoption and Job Satisfaction of Christian Faculty (2026) · cites this paper as Hagendorff, 2020
Experimental — accuracy not yet measured
13 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
The spread of true and false news online
Vosoughi, S., Roy, D., & Aral, S. (2018) · Science · doi:10.1126/science.aap9559 · Cited by 8,847
The future of employment: How susceptible are jobs to computerisation?
Frey, C. B., & Osborne, M. A. (2017) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2016.08.019 · Cited by 8,727
The global landscape of AI ethics guidelines
Jobin, A., Ienca, M., & Vayena, E. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0088-2 · Cited by 6,220
Floridi, L., Cowls, J., Beltrametti, M., Chatila, R., Chazerand, P., Dignum, V., Luetge, C., Madelin, R., Pagallo, U., Rossi, F., et al. (2018) · Minds and Machines · doi:10.1007/s11023-018-9482-5 · Cited by 4,774 · cited 2×
“They include the adoption of legal framework conditions, the establishment of mechanisms for an independent auditing of technologies, the establishment of institutions for complaints, which also compensate for harms caused by AI systems, and the expansion of university curricula in particular through content from ethi…”
Background(2011) · Cited by 4,430
Lazer, D. M. J., Baum, M. A., Benkler, Y., Berinsky, A. J., Greenhill, K. M., Menczer, F., Metzger, M. J., Nyhan, B., Pennycook, G., Rothschild, D., et al. (2018) · Science · doi:10.1126/science.aao2998 · Cited by 4,160
The Black Box Society: The Secret Algorithms That Control Money and Information
Pasquale, F. (2015) · doi:10.4159/harvard.9780674736061 · Cited by 3,796
Experimental evidence of massive-scale emotional contagion through social networks
Kramer, A. D. I., Guillory, J. E., & Hancock, J. T. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1320040111 · Cited by 3,687
(2018) · Cited by 3,510
Private traits and attributes are predictable from digital records of human behavior
Kosinski, M., Stillwell, D., & Graepel, T. (2013) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1218772110 · Cited by 2,829
Research works printed but not linked: 24
- Abrassart, C., Bengio, Y., Chicoisne, G., de Marcellis-Warin, N., Dilhac, M.-A., Gambs, S., Gautrais, V., et al. (2018). Montréal declaration for responsible development of artificial intelligence (pp. 1–21).
- Asimov, I. (2004). I, Robot. New York: Random House LLC.
- Bakewell, J. D., Clement-Jones, T. F., Giddens, A., Grender, R. M., Hollick, C. R., Holmes, C., Levene, P. K. et al. (2018). AI in the UK: Ready, willing and able?. Select committee on artificial intelligence (pp. 1–183).
- Baron, B., Musolesi, M. (2017). Interpretable machine learning for privacy-preserving pervasive systems. arXiv (pp. 1–10).
- Bourdieu, P. (1984). Distinction: A social critique of the judgement of taste. Cambridge: Harvard University Press.
- Brahnam, S. (2006). Gendered bots and bot abuse. In Antonella de Angeli, Sheryl Brahnam, Peter Wallis, & Peter Dix (Eds.), Misuse and abuse of interactive technologies (pp. 1–4). Montreal: ACM.
- Derrida, J. (1997). Of grammatology. Baltimore: Johns Hopkins Univ. Press.
- Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Daumeé, III, H., Crawford, K. (2018). Datasheets for datasets. arXiv (pp. 1–17).
- Google. (2019). Perspectives on issues in AI governance (pp. 1–34). Retrieved February 11, 2019. https://ai.google/static/documents/perspectives-on-issues-in-ai-governance.pdf.
- Helbing, D. (Ed.). (2019). Towards digital enlightment: Essays on the darf and light sides of the digital revolution. Cham: Springer.
- Holdren, J. P., Bruce, A., Felten, E., Lyons, T., & Garris, M. (2016). Preparing for the future of artificial intelligence (pp. 1–58). Washington, D.C: Springer.
- Johnson, D. G. (2017). Can engineering ethics be taught? The Bridge,47(1), 59–64.
- Luhmann, N. (1988). Die Wirtschaft der Gesellschaft. Frankfurt A.M: Suhrkamp.
- Morley, J., Floridi, L., Kinsey, L., Elhalal, A. (2019). From what to how. An overview of AI ethics tools, methods and research to translate principles into practices. arXiv (pp. 1–21).
- Pekka, A.-P., Bauer, W., Bergmann, U., Bieliková, M., Bonefeld-Dahl, C., Bonnet, Y., Bouarfa, L. et al. (2018). The European Commission’s high-level expert group on artificial intelligence: Ethics guidelines for trustworthy ai. Working Document for stakeholders’ consultation. Brussels (pp. 1–37).
- Pistono, F., Yampolskiy, R. (2016). Unethical research: How to create a malevolent artificial intelligence. arXiv (pp. 1–6).
- Podgaiska, I., Shklovski, I. Nordic engineers’ stand on artificial intelligence and ethics: Policy recommendations and guidelines (pp. 1–40).
- Rawls, J. (1975). Eine Theorie Der Gerechtigkeit. Frankfurt am Main: Suhrkamp.
- Shoham, Y., Perrault, R., Brynjolfsson, E., Clark, J., Manyika, J., Niebles, J. C., Lyons, T., Etchemendy, J., Grosz, B., Bauer, Z. (2018). The AI index 2018 annual report. Stanford, Kalifornien (pp. 1–94).
- Sitawarin, C., Bhagoji, A. N., Mosenia, A., Chiang, M., Mittal, P. (2018). DARTS: Deceiving autonomous cars with toxic signs. arXiv (pp. 1–27).
- Tegmark, A. (2017). Life 3.0: Being human in the age of artificial intelligence. New York: Alfred A. Knopf.
- The IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems. (2016). Ethically aligned design: A vision for prioritizing human well-being with artificial intelligence and autonomous systems (pp. 1–138).
- Whittaker, M., Crawford, K., Dobbe, R., Fried, G., Kaziunas, E., Mathur, V., West, S. M., Richardson, R., Schultz, J., Schwartz, O. (2018). AI now report 2018 (pp. 1–62).
- Zeng, Y., Lu, E., Huangfu, C. (2018). Linking artificial intelligence principles. arXiv (pp. 1–4).
Cited by
Conceptualizing AI literacy: An exploratory review
Ng, D. T. K., Leung, J. K. L., Chu, S. K. W., & Qiao, M. S. (2021) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2021.100041 · Cited by 2,205
Principles alone cannot guarantee ethical AI
Mittelstadt, B. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0114-4 · Cited by 1,886
A Unified Framework of Five Principles for AI in Society
Floridi, L., & Cowls, J. (2019) · Harvard Data Science Review · doi:10.1162/99608f92.8cd550d1 · Cited by 1,631
Ethical principles for artificial intelligence in education
Nguyen, A., Ngo, H. N., Hong, Y., Dang, B., & Nguyen, B.-P. T. (2023) · Education and Information Technologies · doi:10.1007/s10639-022-11316-w · Cited by 1,228
“…and open communica - tions with beneficiaries: educators, students, parents, AI developers, and policymak - ers (Berendt et al., 2020 ; Nigam et al., 2021 ; Hagendorff, 2020 ) stated that more emphasis is necessary to enforce ethical guidelines for AI systems to better align with societal values.”
BackgroundTrust in AI and Its Role in the Acceptance of AI Technologies
Choung, H., David, P., & Ross, A. (2023) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2022.2050543 · Cited by 963
“Their guidelines commonly recommend that trustworthy AI should be built into the system by design (Hagendorff, 2020; Jobin et al., 2019).”
Background
Metadata
- Authors
- Hagendorff, T.
- Affiliations
- University of Tübingen (DE)
- Corresponding author
- Hagendorff, Thilo
- ORCID
- Hagendorff, Thilo
- Venue
- Minds and Machines
- Published
- 2020-02-01
- Volume
- 30
- Issue
- 1
- Pages
- 99-120
- ISSN
- 0924-6495 (ISSN-L)
- 1572-8641
- OpenAlex
- W3003385079
- Cited by
- 1,911
- References
- 86
- Citations by year
- 2026: 370 · 2025: 446 · 2024: 361 · 2023: 290 · 2022: 239 · 2021: 160 · 2020: 41
- FWCI
- 248.38
- Type
- journal article
- Language
- en
- Update notices
- correction (2020-07-28)
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Minds and Machines (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Deutsche Forschungsgemeinschaft (390727645)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Law › Digitalization, Law, and Regulation
- Social Sciences › Business, Management and Accounting › Organizational Behavior and Human Resource Management › Ethics in Business and Education
- Keywords
- AI ethics, ethics guidelines, normative principles, moral values
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-12, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-12 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The Ethics of AI Ethics: An Evaluation of Guidelines. https://app.cochat.ai/papers/10.1007/s11023-020-09517-8
- The paper
- Hagendorff, T. (2020). The Ethics of AI Ethics: An Evaluation of Guidelines. Minds and Machines. https://doi.org/10.1007/s11023-020-09517-8
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.