Perspectives about artificial moral agents
Open accessAbstract
The pursuit of AMAs is complicated. Disputes about the development, design, moral agency, and future projections for these systems have been reported in the literature. This empirical study explores these controversial matters by surveying (AI) Ethics scholars with the aim of establishing a more coherent and informed debate. Using Q-methodology, we show the wide breadth of viewpoints and approaches to artificial morality. Five main perspectives about AMAs emerged from our data and were subsequently interpreted and discussed: (i) Machine Ethics: The Way Forward; (ii) Ethical Verification: Safe and Sufficient; (iii) Morally Uncertain Machines: Human Values to Avoid Moral Dystopia; (iv) Human Exceptionalism: Machines Cannot Moralize; and (v) Machine Objectivism: Machines as Superior Moral Agents. A potential source of these differing perspectives is the failure of Machine Ethics to be widely observed or explored as an applied ethic and more than a futuristic end. Our study helps improve the foundations for an informed debate about AMAs, where contrasting views and agreements are disclosed and appreciated. Such debate is crucial to realize an interdisciplinary approach to artificial morality, which allows us to gain insights into morality while also engaging practitioners.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Narrow AI systems, such as LLMs, lack the capacities regarded as essential for moral agency, such as consciousness, self-concepts, personal memory, life experiences, goals, and emotions.
“narrow AI systems, such as LLMs, lack the capacities regarded as essential for moral agency, such as consciousness, self-concepts, personal memory, life experiences, goals, and emotions [18, 139, 151].”
From The ethics of using artificial intelligence in scientific research: new guidance needed for a new tool (2025) · cites this paper as [139]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Awad, E., Dsouza, S., Kim, R., Schulz, J., Henrich, J., Shariff, A., Bonnefon, J.-F., & Rahwan, I. (2018) · Nature · doi:10.1038/s41586-018-0637-6 · Cited by 1,859 · cited 2× · see citing works
“Design approaches based on societal preferences, which derive an ought from an is, as seen in the Moral Machine Experiment [13], also divided the perspectives.”
BackgroundThe social dilemma of autonomous vehicles
Bonnefon, J.-F., Shariff, A., & Rahwan, I. (2016) · Science · doi:10.1126/science.aaf2654 · Cited by 1,527 · see citing works
Doing Q Methodological Research: Theory, Method and Interpretation
Watts, S., & Stenner, P. H. D. (2012) · SAGE Publications Ltd eBooks · doi:10.4135/9781446251911 · Cited by 1,332 · see citing works
Doing Q ethodology: theory, method and interpretation
Watts, S., & Stenner, P. H. D. (2005) · Qualitative Research in Psychology · doi:10.1191/1478088705qp022oa · Cited by 1,256 · see citing works
The ethics of AI in health care: A mapping review
Morley, J., Machado, C. C. V., Burr, C., Cowls, J., Joshi, I., Taddeo, M., & Floridi, L. (2020) · Social Science & Medicine · doi:10.1016/j.socscimed.2020.113172 · Cited by 1,105 · cited 1× · see citing works
“Consequently, the practical discussions about the ethics of particular AI Systems are realized primarily in relation to the field affected, such as AI in healthcare or AI in transportation [46, 51, 59].”
BackgroundThe ethics of artificial intelligence
Bostrom, N., & Yudkowsky, E. (2014) · The Cambridge Handbook of Artificial Intelligence · doi:10.1017/cbo9781139046855.020 · Cited by 918 · see citing works
The Study of Behavior: Q-Technique and Its Methodology.
Solomon, H., & Stephenson, W. (1955) · Journal of the American Statistical Association · doi:10.2307/2281274 · Cited by 887 · see citing works
Moral machines: teaching robots right from wrong
(2009) · Choice Reviews Online · doi:10.5860/choice.46-4493 · Cited by 793 · see citing works
Wallach, W., & Allen, C. (2009) · Oxford University Press · doi:10.1093/acprof:oso/9780195374049.001.0001 · Cited by 711 · see citing works
The Nature, Importance, and Difficulty of Machine Ethics
Moor, J. H. (2006) · IEEE Intelligent Systems · doi:10.1109/mis.2006.80 · Cited by 704 · see citing works
Doing Q Methodological Research: Theory, Method & Interpretation
Watts, S., & Stenner, P. H. D. (2012) · Cited by 700 · see citing works
Future Progress in Artificial Intelligence: A Survey of Expert Opinion
Müller, V. C., & Bostrom, N. (2016) · Synthese Library · doi:10.1007/978-3-319-26485-1_33 · Cited by 655 · see citing works
McKeown, B., & Thomas, D. B. (2013) · Sage eBooks · doi:10.4135/9781483384412 · Cited by 544 · see citing works
Stephenson, W. (1935) · Nature · doi:10.1038/136297b0 · Cited by 492 · see citing works
When and how to use Q methodology to understand perspectives in conservation research
Zabala, A., Sandbrook, C. G., & Mukherjee, N. (2018) · Conservation Biology · doi:10.1111/cobi.13123 · Cited by 420 · see citing works
(2007) · Cited by 366 · see citing works
Ethical dimensions of using artificial intelligence in health care
(2019) · AMA Journal of Ethics · doi:10.1001/amajethics.2019.121 · Cited by 328 · see citing works
(2011) · Cambridge University Press eBooks · doi:10.1017/cbo9780511978036 · Cited by 317 · see citing works
Himma, K. E. (2009) · Ethics and Information Technology · doi:10.1007/s10676-008-9167-5 · Cited by 276 · see citing works
Allen, C., Wallach, W., & Smit, I. (2006) · IEEE Intelligent Systems · doi:10.1109/mis.2006.83 · Cited by 239 · see citing works
Anderson, M. R., & Anderson, S. (2011) · Cited by 212 · see citing works
Integrating robot ethics and machine morality: the study and design of moral competence in robots
Malle, B. F. (2015) · Ethics and Information Technology · doi:10.1007/s10676-015-9367-8 · Cited by 202 · see citing works
Prospects for a Kantian Machine
Powers, T. M. (2006) · IEEE Intelligent Systems · doi:10.1109/mis.2006.77 · Cited by 165 · see citing works
Critiquing the Reasons for Making Artificial Moral Agents
van Wynsberghe, A., & Robbins, S. (2019) · Science and Engineering Ethics · doi:10.1007/s11948-018-0030-8 · Cited by 161 · see citing works
An architecture for ethical robots inspired by the simulation theory of cognition
Vanderelst, D. F., & Winfield, A. F. T. (2017) · Cognitive Systems Research · doi:10.1016/j.cogsys.2017.04.002 · Cited by 160 · see citing works
Ethical issues in focus by the autonomous vehicles industry
Martinho, A. M., Herber, N., Kroesen, M., & Chorus, C. G. (2021) · Transport Reviews · doi:10.1080/01441647.2020.1862355 · Cited by 160 · see citing works
Self-citationMachine morality: bottom-up and top-down approaches for modelling human moral faculties
Wallach, W., Allen, C., & Smit, I. (2008) · AI & SOCIETY · doi:10.1007/s00146-007-0099-0 · Cited by 152 · see citing works
Artificial Moral Agents: A Survey of the Current Status
Cervantes, J.-A., López, S., Rodríguez, L.-F., Cervantes, S., Cervantes, F., & Ramos, F. (2020) · Science and Engineering Ethics · doi:10.1007/s11948-019-00151-x · Cited by 151 · see citing works
Implementations in Machine Ethics
Tolmeijer, S., Kneer, M., Sarasua, C., Christen, M., & Bernstein, A. (2020) · ACM Computing Surveys · doi:10.1145/3419633 · Cited by 149 · see citing works
A Voting-Based System for Ethical Decision Making
Noothigattu, R., Gaikwad, S. S., Awad, E., Dsouza, S., Rahwan, I., Ravikumar, P., & Procaccia, A. D. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11512 · Cited by 146 · see citing works
(1998) · Cited by 143 · see citing works
Artificial Intelligence Safety Engineering: Why Machine Ethics Is a Wrong Approach
Yampolskiy, R. V. (2012) · Studies in applied philosophy, epistemology and rational ethics · doi:10.1007/978-3-642-31674-6_29 · Cited by 126 · see citing works
Responses to catastrophic AGI risk: a survey
Sotala, K., & Yampolskiy, R. V. (2015) · Physica Scripta · doi:10.1088/0031-8949/90/1/018001 · Cited by 121 · see citing works
Incorporating Ethical Considerations Into Automated Vehicle Control
Thornton, S. M., Pan, S., Erlien, S. M., & Gerdes, J. C. (2016) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2016.2609339 · Cited by 111 · see citing works
The rise of the robots and the crisis of moral patiency
Danaher, J. (2017) · AI & Society · doi:10.1007/s00146-017-0773-9 · Cited by 103 · see citing works
A Challenge for Machine Ethics
Tonkens, R. (2009) · Minds and Machines · doi:10.1007/s11023-009-9159-1 · Cited by 102 · see citing works
Implementations in Machine Ethics A Survey
Tolmeijer, S., Kneer, M., Sarasua, C., Christen, M., & Bernstein, A. (2020) · Zurich Open Repository and Archive (University of Zurich) · doi:10.5167/uzh-197237 · Cited by 102 · see citing works
Limitations and risks of machine ethics
Brundage, M. (2014) · Journal of Experimental & Theoretical Artificial Intelligence · doi:10.1080/0952813x.2014.895108 · Cited by 97 · see citing works
Bohannon, J. (2015) · Science · doi:10.1126/science.349.6245.252 · Cited by 91 · see citing works
Ethics of Artificial Intelligence
(2020) · doi:10.1093/oso/9780190905033.001.0001 · Cited by 88 · see citing works
On Proactive, Transparent, and Verifiable Ethical Reasoning for Robots
Bremner, P., Dennis, L. A., Fisher, M., & Winfield, A. F. T. (2019) · Proceedings of the IEEE · doi:10.1109/jproc.2019.2898267 · Cited by 85 · see citing works
A Comparison between Major Factor Extraction and Factor Rotation Techniques in Q-Methodology
Akhtar‐Danesh, N. (2017) · Open Journal of Applied Sciences · doi:10.4236/ojapps.2017.74013 · Cited by 83 · see citing works
Verdiesen, I., de Sio, F. S., & Dignum, V. (2020) · Minds and Machines · doi:10.1007/s11023-020-09532-9 · Cited by 79 · see citing works
Artificial Intelligence and Robot Responsibilities: Innovating Beyond Rights
Ashrafian, H. (2014) · Science and Engineering Ethics · doi:10.1007/s11948-014-9541-0 · Cited by 78 · see citing works
A Normative Approach to Artificial Moral Agency
Behdadi, D., & Munthe, C. (2020) · Minds and Machines · doi:10.1007/s11023-020-09525-8 · Cited by 76 · see citing works
Anderson, M., & Anderson, S. (2010) · Scientific American · doi:10.1038/scientificamerican1010-72 · Cited by 74 · see citing works
The Need for Moral Competency in Autonomous Agent Architectures
Scheutz, M. J. (2016) · Synthese Library/Synthese library · doi:10.1007/978-3-319-26485-1_30 · Cited by 72 · see citing works
Artificial Morality. Concepts, Issues and Challenges
Misselhorn, C. (2018) · Society · doi:10.1007/s12115-018-0229-y · Cited by 72 · see citing works
Making moral machines: why we need artificial moral agents
Formosa, P., & Ryan, M. (2020) · AI & Society · doi:10.1007/s00146-020-01089-6 · Cited by 71 · see citing works
The ethics of designing artificial agents
Grodzinsky, F. S., Miller, K., & Wolf, M. J. (2008) · Ethics and Information Technology · doi:10.1007/s10676-008-9163-9 · Cited by 69 · see citing works
The Dark Side of Ethical Robots
Vanderelst, D. F., & Winfield, A. F. T. (2018) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3278721.3278726 · Cited by 68 · see citing works
Gamez, P., Shank, D. B., Arnold, C., & North, M. (2020) · Ai & Society · doi:10.1007/s00146-020-00977-1 · Cited by 68 · see citing works
GenEth: a general ethical dilemma analyzer
Anderson, M., & Anderson, S. (2018) · Paladyn Journal of Behavioral Robotics · doi:10.1515/pjbr-2018-0024 · Cited by 66 · see citing works
Ethics of Artificial Intelligence
(2019) · Research Library Issues · doi:10.29242/rli.299 · Cited by 59 · see citing works
Overcoming carer shortages with care robots: Dynamic value trade-offs in run-time
Poulsen, A., & Burmeister, O. K. (2019) · AJIS. Australasian journal of information systems/AJIS. Australian journal of information systems/Australian journal of information systems · doi:10.3127/ajis.v23i0.1688 · Cited by 54 · see citing works
Self-citationImplementation of Moral Uncertainty in Intelligent Machines
Bogosian, K. (2017) · Minds and Machines · doi:10.1007/s11023-017-9448-z · Cited by 53 · see citing works
Out of character: on the creation of virtuous machines
Tonkens, R. (2012) · Ethics and Information Technology · doi:10.1007/s10676-012-9290-1 · Cited by 47 · see citing works
Quantifying Human Subjectivity Using Q Method: When Quality Meets Quantity
Kamal, S., Kocór, M., & Grodzińska‐Jurczak, M. (2014) · Qualitative Sociology Review · doi:10.18778/1733-8077.10.3.03 · Cited by 46 · see citing works
Toward Non-Intuition-Based Machine and Artificial Intelligence Ethics
Hooker, J., & Kim, T. W. (2018) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3278721.3278753 · Cited by 45 · see citing works
Virtuous vs. utilitarian artificial moral agents
Bauer, W. A. (2018) · AI & Society · doi:10.1007/s00146-018-0871-3 · Cited by 42 · see citing works
Artificial Moral Cognition: Moral Functionalism and Autonomous Moral Agency
Howard, D. A., & Muntean, I. (2017) · Philosophical studies series · doi:10.1007/978-3-319-61043-6_7 · Cited by 40 · see citing works
Artificial moral agents: moral mentors or sensible tools?
Fossa, F. (2018) · Ethics and Information Technology · doi:10.1007/s10676-018-9451-y · Cited by 40 · see citing works
Landscape of Machine Implemented Ethics
Nallur, V. (2020) · Science and Engineering Ethics · doi:10.1007/s11948-020-00236-y · Cited by 40 · see citing works
Fritz, A., Brandt, W., Gimpel, H., & Bayer, S. (2020) · De Ethica A Journal of Philosophical Theological and Applied Ethics · doi:10.3384/de-ethica.2001-8819.20613 · Cited by 37 · see citing works
Landscape of Machine Implemented Ethics
Nallur, V. (2021) · Arrow@dit (Dublin Institute of Technology) · Cited by 35 · see citing works
Toward the Engineering of Virtuous Machines
Govindarajulu, N. S., Bringsjord, S., Ghosh, R., et al. (2019) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3306618.3314256 · Cited by 32 · see citing works
Misselhorn, C. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2019.103179 · Cited by 31 · see citing works
A Prima Facie Duty Approach to Machine Ethics
Anderson, S., & Anderson, M. (2011) · Cambridge University Press eBooks · doi:10.1017/cbo9780511978036.032 · Cited by 30 · see citing works
An approach for combining ethical principles with public opinion to guide public policy
Awad, E., Anderson, M., Anderson, S. L., & Liao, B. (2020) · Artificial Intelligence · doi:10.1016/j.artint.2020.103349 · Cited by 28 · see citing works
Anderson, S., & Anderson, M. (2020) · AI and Ethics · doi:10.1007/s43681-020-00003-6 · Cited by 25 · see citing works
Handbook of research on technoethics
(2009) · Choice Reviews Online · doi:10.5860/choice.46-2996 · Cited by 25 · see citing works
How AI Systems Challenge the Conditions of Moral Agency?
Hallamaa, J., & Kalliokoski, T. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-50267-6_5 · Cited by 21 · see citing works
Köse, U. (2018) · BRAIN BROAD RESEARCH IN ARTIFICIAL INTELLIGENCE AND NEUROSCIENCE · Cited by 20 · see citing works
Augmented Utilitarianism for AGI Safety
Aliman, N.-M., & Kester, L. J. H. M. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-27005-6_2 · Cited by 20 · see citing works
Anderson, S., & Anderson, M. (2011) · Cited by 18 · see citing works
Artificial Moral Agency in Technoethics
Sullins, J. P. (2009) · IGI Global eBooks · doi:10.4018/978-1-60566-022-6.ch014 · Cited by 18 · see citing works
Towards Provably Moral AI Agents in Bottom-up Learning Frameworks
Shaw, N. P., Stöckel, A., Orr, R. W., et al. (2018) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3278721.3278728 · Cited by 18 · see citing works
Liao, B., Anderson, M., & Anderson, S. L. (2021) · AI and Ethics · doi:10.1007/s43681-020-00001-8 · Cited by 18 · see citing works
Computationally rational agents can be moral agents
Mabaso, B. A. (2020) · Ethics and Information Technology · doi:10.1007/s10676-020-09527-1 · Cited by 17 · see citing works
Building blocks for artificial moral agents
Wiegel, V. (2006) · Cited by 16 · see citing works
Ethics in the Design of Automated Vehicles: The AVEthics project.
Doğan, E., Chatila, R., Chauvier, S., Evans, K., Hadjixenophontos, P., & Perrin, J. (2016) · Cited by 15 · see citing works
Corrigendum: Responses to catastrophic AGI risk: a survey (2015 Phys. Scr. 90 018001)
Sotala, K., & Yampolskiy, R. V. (2015) · Physica Scripta · doi:10.1088/0031-8949/90/6/069501 · Cited by 12 · see citing works
Waser, M. R. (2015) · Procedia Computer Science · doi:10.1016/j.procs.2015.12.213 · Cited by 11 · see citing works
Responses to a Critique of Artificial Moral AgentsPreprint
Poulsen, A., Anderson, M., Anderson, S., Byford, B., Fossa, F., Neely, E. L., Rosas, A., & Winfield, A. F. T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.07021 · Cited by 7 · see citing works
Self-citationSexbots: a case for artificial ethical agents
Headleand, C. J., Teahan, W. J., & Cenydd, L. A. (2019) · Connection Science · doi:10.1080/09540091.2019.1640185 · Cited by 7 · see citing works
Russell, S. (2019) · IEEE Spectrum · doi:10.1109/mspec.2019.8847590 · Cited by 7 · see citing works
Artificial Moral Agents. Really?
Hunyadi, M. (2019) · Springer tracts in advanced robotics · doi:10.1007/978-3-030-17974-8_5 · Cited by 6 · see citing works
Köse, U. (2018) · DOAJ (DOAJ: Directory of Open Access Journals) · Cited by 5 · see citing works
MoralDM: A Computational Model of Moral Decision-Making
Dehghani, M., Tomai, E., Forbus, K., et al. (2008) · PsycEXTRA Dataset · doi:10.1037/e722352011-114 · Cited by 5 · see citing works
Powers, T. M. (2009) · Philosophy now · Cited by 4 · see citing works
Interactive Learning and Analogical Chaining for Moral and Commonsense Reasoning
Blass, J. A. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.9817 · Cited by 4 · see citing works
Flagged first, then by citations
Research works printed but not linked: 4
- Anderson, M., Anderson, S.L., Armen, C.: Towards machine eth- ics. In: AAAI-04 workshop on agent organizations: theory and practice, San Jose, CA (2004)
- Anderson, S.L.: Machine Metaethics, pp. 21-27. Cambridge Uni- versity Press, Cambridge (2011)
- Dehghani, M., Tomai, E., Forbus, K., Iliev, R., Klenk, M.: Mor- aldm: A computational modal of moral decision-making. In: Proceedings of the 30th Annual Conference of the Cognitive Sci- ence Society (CogSci). Citeseer (2008)
- Shulman, C., Jonsson, H., Tarleton, N.: Which consequentialism? machine ethics and moral divergence. In: Asia-Pacific Conference on Computing and Philosophy (APCAP 2009), Tokyo, Japan. Cit- eseer (2009)
Cited by
Sorted by citations
Resnik, D. B., & Hosseini, M. (2025) · AI and Ethics · doi:10.1007/s43681-024-00493-8 · Cited by 228 · see citing works
“2, narrow AI systems, such as LLMs, lack the capacities regarded as essential for moral agency, such as consciousness, self-concepts, personal memory, life experiences, goals, and emotions [18, 139, 151].”
Artificial moral agents: should machines take ethical responsibility?
Reiter, P., Norman, U., Weinberger, N., & Bruno, B. (2025) · 2025 IEEE International Conference on Advanced Robotics and its Social Impacts (ARSO) · doi:10.1109/arso64737.2025.11124921 · Cited by 2 · see citing works
- Authors
- Martinho, A., Poulsen, A., Kroesen, M., & Chorus, C.
- Affiliations
- Delft University of Technology (NL)
- Charles Sturt University (AU)
- Corresponding author
- Martinho, Andreia
- Venue
- AI and Ethics
- Published
- 2021-05-11
- Volume
- 1
- Issue
- 4
- Pages
- 477-490
- ISSN
- 2730-5953 (ISSN-L)
- 2730-5961
- OpenAlex
- W3160766790
- Cited by
- 27
- References
- 91
- Citations by year
- 2026: 5 · 2025: 4 · 2024: 8 · 2023: 2 · 2022: 8
- FWCI
- 5.63
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- AI and Ethics (publishedVersion, cc-by)
- Charles Sturt University Research Output (CRO) (submittedVersion, cc-by)
- Funders
- H2020 European Research Council (ERC Consolidator grant BEHAVE –724431)
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Social Sciences › Arts and Humanities › Philosophy › Philosophical Ethics and Theory
- Social Sciences › Decision Sciences › Information Systems and Management › Ethics in Business and Education
- Keywords
- artificial moral agents, machine ethics, artificial morality, moral agency, human exceptionalism, human values
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
