The DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.
Ethical risk for AI
Abstract
The term ‘ethical risk’ often appears in discussions about the responsible development and deployment of artificial intelligence (AI). However, ethical risk remains inconsistently defined in this context, obscuring what distinguishes it from other forms of risk, such as social, reputational or legal risk, for example. In this paper we present a definition of ethical risk for AI as being any risk associated with an AI that may cause stakeholders to fail one or more of their ethical responsibilities towards other stakeholders. To support our definition, we describe how stakeholders have role responsibilities that follow from their relationship with the AI, and that these responsibilities are towards other stakeholders associated with the AI. We discuss how stakeholders may differ in their ability to make decisions about an AI, their exposure to risk, and whether they or others may benefit from these risks. Stakeholders without the ability to make decisions about the risks associated with an AI and how it is used are dependent on other stakeholders with this ability. This relationship places those who depend on decision-making stakeholders at ethical risk of being dominated by them. The decision-making stakeholder is ethically responsible for the risks their decisions about the AI impose on those affected by them. We illustrate our account of ethical risk for AI with two examples: AI-designed attachments for surgical robots that are optimised for treating specific patients, and self-driving ‘robotaxis’ that carry passengers on public roads.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The ethical paradox is stark. Emotional AI can alleviate suffering—but it can also entrench it. It can empower—but it can also disempower. It's very strengths—availability, responsiveness, non-judgment—can become liabilities when they prevent users from seeking or sustaining real relationships (Douglas et al., 2025).”
From Emotional AI and the rise of pseudo-intimacy: are we trading authenticity for algorithmic affection? (2025) · cites this paper as Douglas et al., 2025
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988 · cited 1×
“Recognising the hardware necessary to develop and use an AI system ensures that the material characteristics of AI (such as the environmental impact and financial cost of operating it) are not overlooked [ 48].”
How the machine “thinks”: Understanding opacity in machine learning algorithms
Burrell, J. (2016) · Big Data & Society · doi:10.1177/2053951715622512 · Cited by 3,196 · cited 1×
“…have several characteristics that contribute to these risks, such as: ● Many forms of machine learning (ML) are opaque (or ‘black boxes’), meaning that it is difficult (if not impos - sible) for developers and users to understand exactly how the system made a particular decision [9, 10].”
ECHO CHAMBERS AND EPISTEMIC BUBBLES
Nguyen, C. T. (2020) · Episteme · doi:10.1017/epi.2018.32 · Cited by 1,115
The responsibility gap: Ascribing responsibility for the actions of learning automata
Matthias, A. (2004) · Ethics and Information Technology · doi:10.1007/s10676-004-3422-1 · Cited by 1,101 · cited 2×
“● Using AI systems for decision-making may lead to ‘re-sponsibility gaps’, where there is uncertainty about responsibility for the decisions recommended (or actions taken) by AI systems [13, 14].”
Four Responsibility Gaps with Artificial Intelligence: Why they Matter and How to Address them
Santoni de Sio, F., & Mecacci, G. (2021) · Philosophy & Technology · doi:10.1007/s13347-021-00450-x · Cited by 490 · cited 3×
“● Using AI systems for decision-making may lead to ‘re-sponsibility gaps’, where there is uncertainty about responsibility for the decisions recommended (or actions taken) by AI systems [13, 14].”
Solving the Black Box Problem: A Normative Framework for Explainable Artificial Intelligence
Zednik, C. (2021) · Philosophy & Technology · doi:10.1007/s13347-019-00382-7 · Cited by 419
Tubaro, P., Casilli, A. A., & Coville, M. (2020) · Big Data & Society · doi:10.1177/2053951720919776 · Cited by 262
Embedding Values in Artificial Intelligence (AI) Systems
van de Poel, I. (2020) · Minds and Machines · doi:10.1007/s11023-020-09537-4 · Cited by 255
Technology with No Human Responsibility?
Johnson, D. G. (2015) · Journal of Business Ethics · doi:10.1007/s10551-014-2180-1 · Cited by 195
Mind the gap: responsible robotics and the problem of responsibility
Gunkel, D. J. (2020) · Ethics and Information Technology · doi:10.1007/s10676-017-9428-2 · Cited by 174
Moral Responsibility and the Problem of Many Hands
van de Poel, I., Royakkers, L., & Zwart, S. D. (2015) · doi:10.4324/9781315734217 · Cited by 152
Artificial intelligence explainability: the technical and ethical dimensions
McDermid, J. A., Jia, Y., Porter, Z., & Habli, I. (2021) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2020.0363 · Cited by 136 · cited 8×
“While this is not intended to be an exhaustive list, possible classes of AI stakeholders include: ● Accident and Incident Investigators : those who investigate failures and accidents involving AI systems to determine whether the AI system was casually responsible [54].”
InfluentialAI, agency and responsibility: the VW fraud case and beyond
Johnson, D. G., & Verdicchio, M. (2019) · AI & SOCIETY · doi:10.1007/s00146-017-0781-9 · Cited by 79 · cited 1×
“In this paper we will assume that moral agency (and thus, ethical responsibility) is only possessed by human persons, and that AI systems as artificial agents do not possess moral agency [40].”
Boden, M. A. (2014) · The Cambridge Handbook of Artificial Intelligence · doi:10.1017/cbo9781139046855.007 · Cited by 29
Research works printed but not linked: 52
- Eubanks, V.: Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor. Picador, New York (2019)
- Benjamin, R.: Race after Technology. Polity, Cambridge (2019)
- Gebru, T.: Race and gender. in In: The Oxford Handbook of Ethics of AI, pp. 253–269. Oxford University Press, New York (2020)
- DeArman, A.: The Wild, Wild West: A case study of self-driving vehicle testing in Arizona. Arizona Law Rev. 61, 983–1012 (2019)
- Koopman, P.: Anatomy of a Robotaxi Crash: Lessons from the Cruise Pedestrian Dragging Mishap, 8 February 2024. [Online]. Available: https://arxiv.org/abs/2402.06046. [Accessed 17 April 2024] · arXiv
- Hansson, S.O.: Ethical risk analysis. In: Hansson, S.O. (ed.) in The Ethics of Technology: Methods and Approaches, pp. 157–171. Rowman & Littlefield, London (2017)
- Petrozzino, C.: Who pays for ethical debt in AI? AI Ethics. 1(3), 205–208 (2021) · DOI
- Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., Galstyan, A.: A Survey on Bias and Fairness in Machine Learning. ACM Comput. Surveys. 54(6), 115 (July 2021) · DOI · Paper page
- Srininvasan, R., Chander, A.: Biases in AI systems. Commun. ACM. 64(8), 44–49 (2021) · DOI
- Tigard, D.W.: There is no techno-responsibility gap. Philos. Technol. 34(3), 589–607 (2021) · DOI · Paper page
- Vredenburgh, K.: The right to explanation. J. Political Philos. 30(2), 209–229 (2022) · DOI
- Coeckelbergh, M.: The Political Philosophy of AI. Polity, Cambridge (2022)
- Tricker, B., Tricker, G.: Business Ethics: A Stakeholder, Governance and Risk Approach. Routledge (2014) · doi:10.4324/9780203066522
- Rotta, C.P.: A Short Guide to Ethical Risk. Routledge, London and New York (2017) · doi:10.4324/9781315263670
- High-Level Expert Group on AI (AI HLEG): Ethics Guidelines for Trustworthy AI. European Commission, Brussels (2019)
- Society, I.E.E.E.C.: IEEE Standard Model process for addressing ethical concerns during System Design. IEEE, (2021)
- Winfield, A.F.T., Winkle, K.: RoboTed: A Case Study in Ethical Risk Assessment, in ICRES 2020: 5th International Conference on Robot Ethics and Standards, Taipei, Taiwan, (2020)
- Beauchamp, T.L., Childress, J.F.: Principles of Biomedical Ethics, 7th edn. Oxford University Press, Oxford (2013)
- Maheshwari, K.: On the harm of imposing risk of harm. Ethical Theory Moral. Pract. pp. 965–980, (2021) · doi:10.1007/s10677-021-10227-y
- Vallor, S.: Moral Deskilling and Upskilling in a New Machine Age: Reflections on the ambiguous future of Character. Philos. Technol. 28(1), 107–124 (2015) · DOI · Paper page
- Giddens, A.: Risk and Responsibility, The Modern Law Review, vol. 62, no. 1, pp. 1–10, January (1999) · DOI · Paper page
- Kermisch, C.: Risk and responsibility: A complex and evolving relationship. Sci Eng. Ethics. 18(1), 91–102 (2012) · doi:10.1007/s11948-010-9246-y
- Nihlén Fahlquist, J. Moral Responsibility and Risk in Society. Routledge, London and New York (2019)
- Nihlén Fahlquist, J: Responsibility analysis. In: Hansson, S.O. (ed.) in The Ethics of Technology: Methods and Approaches, pp. 129–142. Rowman & Littlefield, London (2017)
- Hart, H.L.A.: Punishment and Responsibility: Essays in the Philosophy of Law, 2nd edn. Oxford University Press, Oxford (2008) · DOI · Paper page
- van de Poel, I.: The Relation Between Forward-Looking and Backward-Looking Responsibility, in Moral Responsibility, N. A. Vincent, I. van de Poel and J. van den Hoven, Eds., Dordrecht, Springer, pp. 37–52. (2011) · DOI · Paper page
- van de Poel, I., Sand, M.: Varieties of Responsibility: Two Problems of Responsible Innovation, Synthese, vol. 198, no. Supplment 19, pp. S4769-S4787, (2021) · DOI
- Mahler, T.: Defining Legal Risk, in Proceedings of the Conference Commercial Contracting for Strategic Advantage - Potentials and Prospects, Turku, Finland, (2007)
- Coeckelbergh, M.: AI Ethics., A.I.: Cambridge, Massachusetts: The MIT Press, (2020) · DOI · Paper page
- Santoro, M., Marino, D., Tamburrini, G.: Learning Robots interacting with humans: From epistemic risk to responsibility. AI Soc. 22(3), 301–314 (2008) · DOI
- Alpaydin, E.: Machine Learning, Revised and Updated Edition. The MIT Press, Cambridge, Massachusetts (2021)
- Razjigaev, A., Pandey, A.K., Howard, D., Roberts, J., Wo, L.: End-to-end design of Bespoke, Dexterous Snake-Like Surgical Robots: A Case Study with the RAVEN II. IEEE Trans. Robot. 38(5), 2827–2840 (2022) · doi:10.1109/tro.2022.3164841
- Powers, T.M., Ganascia, J.-G.: The Ethics of the Ethics of AI. in In: The Oxford Handbook of Ethics of AI, pp. 27–51. Oxford University Press, Oxford (2020)
- Franssen, M., Kroes, P.: Sociotechnical Systems. In: Friis, J.K.B.O., Pedersen, S.A., Hendricks, V.F. (eds.) in A Companion to the Philosophy of Technology, pp. 223–226. Wiley-Blackwell, Malden, MA (2013)
- Shneiderman, B., Rose, A.: Social Impact Statements: Engaging Public Participation in Information Technology Design, in Proceedings of the Symposium on Computers and the Quality of Life, (1996) · DOI
- Friedman, B., Hendry, D.G.: Value Sensitive Design, Cambridge, Massaschusetts: The MIT Press, (2019) · DOI · Paper page
- Douglas, D.M., Lacey, J., Howard, D.: Ethical responsibility and computational design: Bespoke surgical tools as an instructive case study. Ethics Inf. Technol. 24(1), 11 (2022) · DOI
- Tomsett, R., Braines, D., Harborne, D., Preece, A., Chakraborty, S.: Interpretable to Whom? A Role-based Model for Analyzing Interpretable Machine Learning Systems, in 2018 ICML Workshop on Human Interpretability in Machine Learning (WHI 2018), Stockholm, (2018)
- Douglas, H.E.: Science, Policy, and the Value-Free Ideal. University of Pittsburgh, Pittsburgh, PA (2009) · DOI · Paper page
- Alexandra, A., Miller, S.: Ethics in Practice: Moral Theory and the Professions. University of New South Wales, Sydney (2009)
- Domingos, P.: The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake Our World. Penguin Books (2015)
- Hermansson, H., Hansson, S.O.: A three-Party Model Tool for ethical risk analysis. Risk Manage. 9, 129–144 (2007) · doi:10.1057/palgrave.rm.8250028
- van de Poel, I., Nihlén, J., Fahlquist: Risk and responsibility. In: Roeser, S., Hillerbrand, R., Sandin, P., Peterson, M. (eds.) in Essentials of Risk Theory, pp. 107–143. Springer, Dordrecht (2013) · DOI
- Klinke, A., Renn, O.: The coming of age of risk governance. Risk Anal. 41(3), 544–557 (2019) · DOI
- Malakar, Y., Lacey, J., Bertsch, P.M.: Towards responsible science and technology: How Nanotechnology Research and Development is shaping Risk Governance practices in Australia. Humanit. Social Sci. Commun. 9, 17 (2022) · DOI
- Douglas, D.M., Lacey, J., Howard, D.: Ethical risks of AI-Designed products: Bespoke Surgical Tools as a case study. AI Ethics. 3(4), 1117–1133 (2023) · DOI
- Wolff, J.: Five types of Risky Situation. Law Innov. Technol. 2(2), 151–163 (2010) · DOI
- Lovett, F.: A General Theory of Domination and Justice. Oxford University Press, Oxford (2010) · DOI · Paper page
- Maheshwari, K., Nyholm, S.: Dominating risk impositions. J. Ethics. 26(4), 613–637 (2022) · doi:10.1007/s10892-022-09407-4
- Hansson, S.O.: The Ethics of Risk: Ethical Analysis in an Uncertain World. Palgrave Macmillan, Basingstoke, Hampshire (2013) · DOI
- Lewens, T. (ed.): Risk: Philosophical Perspectives, London and New York: Routledge, (2007) · DOI
- Guntzbunger, Y., Johnson, K.J., Martineau, J.T., Pauchant, T.C.: Professional ethnocentrism and ethical risk management efficacy: How engineer’s emotional openness mediates this complex relationship. Saf. Sci. 109, 27–35 (2018) · DOI
Cited by
Emotional AI and the rise of pseudo-intimacy: are we trading authenticity for algorithmic affection?
Babu, J., Joseph, D., Kumar, R. M., Alexander, E., Sasi, R., & Joseph, J. (2025) · Frontiers in Psychology · doi:10.3389/fpsyg.2025.1679324 · Cited by 76
Metadata
- Authors
- Douglas, D. M., Lacey, J., & Howard, D.
- Venue
- AI and Ethics
- Published
- 2024-08-08
- Volume
- 5
- Issue
- 3
- Pages
- 2189-2203
- ISSN
- 2730-5953
- 2730-5961
- Cited by
- 18
- References
- 14
- Language
- en
- Licence
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Funders
- Commonwealth Scientific and Industrial Research Organisation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.