The DOI matches its DataCite record.
Responsible Autonomy
Open accessAbstract
As intelligent systems are increasingly making decisions that directly affect society, perhaps the most important upcoming research direction in AI is to rethink the ethical implications of their actions. Means are needed to integrate moral, societal and legal values with technological developments in AI, both during the design process as well as part of the deliberation algorithms employed by these systems. In this paper, we describe leading ethics theories and propose alternative ways to ensure ethical behavior by artificial systems. Given that ethics are dependent on the socio-cultural context and are often only implicit in deliberation processes, methodologies are needed to elicit the values held by designers and stakeholders, and to make these explicit leading to better understanding and trust on artificial autonomous systems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
HITL ensures proper oversight of system decisions, while creating a clear party to hold accountable by making a human ultimately responsible for decisions.
“HITL ensures proper oversight of system decisions, while creating a clear party to hold accountable by making a human ultimately responsible for decisions (Dignum, 2017).”
From Liability for harm caused by AI in healthcare: an overview of the core legal concepts (2023) · cites this paper as Dignum, 2017
Section 4.1 describes human-in-the-loop control as a design in which a human supervises or intervenes and a person or group is responsible for the decision. It says this design requires shared awareness so the decision-maker has enough information to intervene; it does not say HITL automatically ensures proper oversight. The Introduction also describes a chain of responsibility involving multiple stakeholders, rather than necessarily one clear party to hold accountable. (LLM, conf=0.86)
Experimental — accuracy not yet measuredThe principal-agent approach would ensure that there is always an identifiable human part of the decision-making process and would be in line with an AI design philosophy called "human-in-the-loop" systems (HITL).
“This approach would ensure that there is always an identifiable human part of the decision-making process and would be in line with an AI design philosophy called “human-in-the-loop” systems (HITL) (Dignum, 2017).”
From Liability for harm caused by AI in healthcare: an overview of the core legal concepts (2023) · cites this paper as Dignum, 2017
Section 4.1, “Who takes the decision?” (page unavailable), describes human-in-the-loop control systems in which a person or group is responsible for the decision. It does not discuss a principal–agent approach or establish that such an approach would always leave an identifiable human part in the decision-making process. (LLM, conf=0.84)
Experimental — accuracy not yet measuredAs these systems become more autonomous and powerful, moral assessment may become an essential component of their decision-making, especially in a field such as healthcare.
“This would also ensure that incorporated morals are interpreted in an acceptable way, meaning that, as these systems become more autonomous and powerful, moral assessment may become an essential component of their decision-making, especially in a field such as healthcare (Dignum, 2017).”
From Liability for harm caused by AI in healthcare: an overview of the core legal concepts (2023) · cites this paper as Dignum, 2017
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
No references could be resolved yet.
Research works printed but not linked: 23
- Huib Aldewereld, S. Álvarez- Napagao, F. Dignum, and J. Vázquez-Salceda. Making norms concrete. In AAMAS'10, pages 807-814, 2010.
- C. Allen, I. Smit, and W. Wallach. Ar- tificial morality: Top-down, bottom-up, and hybrid ap- proaches. Ethics and Information Technology, 7(3):149- 155, 2005.
- Ken Binmore. Natural justice. Oxford Uni- versity Press, 2005.
- Björklund, 2003] Fredrik Björklund. Differences in the jus- tification of choices in moral dilemmas: Effects of gen- der, time pressure and dilemma seriousness. Scandinavian Journal of Psychology, 44(5):459-466, 2003.
- Guido Boella, L. Van Der Torre, and H. Verhagen. Introduction to normative multiagent sys- tems. Computational & Mathematical Organization The- ory, 12(2-3):71-79, 2006.
- Bonnefon et al., 2016] Jean-Franc ¸ois Bonnefon, Azim Shariff, and Iyad Rahwan. The social dilemma of autonomous vehicles. Science, 352(6293):1573-1576, 2016.
- Joanna Bryson and Philip Kime. Just an artifact: why machines are perceived as moral agents. In IJCAI'11, pages 1641-1646, 2011.
- Rosaria Conte, C. Castelfranchi, and F. Dignum. Autonomous norm acceptance. In Interna- tional Workshop on Agent Theories, Architectures, and Languages, pages 99-112. Springer, 1998.
- Luciano Floridi and J. Sanders. On the morality of artificial agents. Minds and machines, 14(3):349-379, 2004.
- Gigerenzer, 2010] Gerd Gigerenzer. Moral satisficing: Re- thinking moral behavior as bounded rationality. Topics in cognitive science, 2(3):528-554, 2010.
- Jesse Graham, Brian Nosek, Jonathan Haidt, Ravi Iyer, Spassena Koleva, and Peter Ditto. Map- ping the moral domain. Journal of personality and social psychology, 101(2):366, 2011.
- Ken Lay, O. C. Ferrell, and L. Ferrell. The responsibility and accountability of ceos: The last inter- view with ken lay. Journal of Business Ethics, 100(2):209- 219, 2011.
- Wenchao Li, Dorsa Sadigh, Shankar Sastry, and Sanjit Seshia. Synthesis for Human-in-the-Loop Con- trol Systems, pages 470-484. Springer, 2014.
- Bertram Malle, Matthias Scheutz, Thomas Arnold, John Voiklis, and Corey Cusimano. Sac- rifice one for the good of many?: People apply different moral norms to human and robot agents. In Proc. 10th ACM/IEEE Int. Conf. on Human-Robot Interaction, pages 117-124. ACM, 2015.
- James Misener and S. Shladover. Path investigations in vehicle-roadside cooperation and safety: A foundation for safety and vehicle-infrastructure integration research. In Intelligent Transportation Systems Conference, 2006, pages 9-16. IEEE, 2006.
- Stuart Russell and P Norvig. Ar- tificial Intelligence: A Modern Approach. Pearson Educa- tion, 3rd edition, 2009.
- Schwartz, 2006b] Shalom Schwartz. A theory of cultural value orientations: Explication and applications. Compar- ative sociology, 5(2):137-182, 2006.
- Searle, 2010] John Searle. Making the social world: The structure of human civilization. Oxford UP, 2010.
- Sen, 2004] Amartya Sen. Elements of a theory of human rights. Philosophy & Public Affairs, 32(4):315-356, 2004.
- van de Poel, 2013] Ibo van de Poel. Translating Values into Design Requirements, pages 253-266. Springer Nether- lands, Dordrecht, 2013.
- van den Hoven, 2005] Jeroen van den Hoven. Design for values and values for design. Information Age +, Journal of the Australian Computer Society, 7(2):4-7, 2005.
- Ilse Verdiesen, Martijn Cligge, Jan Timmermans, Lennart Segers, Virginia Dignum, and Jeroen van den Hoven. MOOD: Massive open online de- liberation platform a practical application. In 1st Workshop on Ethics in the Design of Intelligent Agents, pages 6-11, 2016.
- Wallach and Allen, 2008] Wendall Wallach and Collin Allen. Moral machines: Teaching robots right from wrong. Oxford University Press, 2008.
Cited by
Sorted by citations
Liability for harm caused by AI in healthcare: an overview of the core legal concepts
Bottomley, D., & Thaldar, D. (2023) · Frontiers in Pharmacology · doi:10.3389/fphar.2023.1297353 · Cited by 80
Metadata
- Authors
- Dignum, V.
- Affiliations
- Delft University of Technology (NL)
- Corresponding author
- Dignum, Virginia
- ORCID
- Dignum, Virginia
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2017
- Pages
- 4698-4704
- arXiv
- 1706.02513
- OpenAlex
- W2921766120
- Cited by
- 103
- References
- 0
- Citations by year
- 2026: 9 · 2025: 14 · 2024: 12 · 2023: 15 · 2022: 13 · 2021: 11 · 2020: 11 · 2019: 10 · 2018: 7
- FWCI
- 19.10
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- public-domain
- arxiv.org/licenses/nonexclusive-distrib/1.0/
- Locations
- International Joint Conference on Artificial Intelligence (publishedVersion, public-domain)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- International Joint Conference on Artificial Intelligence
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Business, Management and Accounting › Organizational Behavior and Human Resource Management › Ethics in Business and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Keywords
- AI ethics, autonomous systems, ethical decision-making, ethics theory, value elicitation, stakeholder value, trust in AI, responsible AI
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
