Robot Autonomy vs. Human Autonomy: Social Robots, Artificial Intelligence (AI), and the Nature of Autonomy
Open accessThe DOI matches its Crossref record; 104 of the 105 references it cites were found in a registry or an index.
Abstract
Social robots are robots that can interact socially with humans. As social robots and the artificial intelligence (AI) that powers them becomes more advanced, they will likely take on more social and work roles. This has many important ethical implications. In this paper, we focus on one of the most central of these, the impacts that social robots can have on human autonomy. We argue that, due to their physical presence and social capacities, there is a strong potential for social robots to enhance human autonomy as well as several ways they can inhibit and disrespect it. We argue that social robots could improve human autonomy by helping us to achieve more valuable ends, make more authentic choices, and improve our autonomy competencies. We also argue that social robots have the potential to harm human autonomy by instead leading us to achieve fewer valuable ends ourselves, make less authentic choices, decrease our autonomy competencies, make our autonomy more vulnerable, and disrespect our autonomy. Whether the impacts of social robots on human autonomy are positive or negative overall will depend on the design, regulation, and use we make of social robots in the future.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Autonomy enables scalable deployment in applications where persistent oversight (human-in-the-loop) is impractical, such as customer support bots or scheduling assistants [52,64].”
From AI Agents vs. Agentic AI: A Conceptual taxonomy, applications and challenges (2026) · cites this paper as [64]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Autonomous actions, such as auto-tagging or summarizing threads, enhance efficiency, while embedded feedback loops enable personalization through incremental learning [64].”
From AI Agents vs. Agentic AI: A Conceptual taxonomy, applications and challenges (2026) · cites this paper as [64]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Autonomous actions, such as autotagging or summarizing threads, enhance efficiency, while embedded feedback loops enable personalization through incremental learning [66].”
From AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challenges (2025) · cites this paper as [66]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“AI Agents automate email triage through content classification and prioritization.”
From AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challenges (2025) · cites this paper as [66]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(2011) · Cited by 14,317
Principles of Biomedical Ethics
(2012) · Thieves of Virtue · doi:10.7551/mitpress/9079.003.0009 · Cited by 13,453
The global landscape of AI ethics guidelines
Jobin, A., Ienca, M., & Vayena, E. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0088-2 · Cited by 6,220 · cited 3×
“The issue of machine (or artificial) autonomy is of central importance to much of the recent literature on ethical AI, as demonstrated by three recent reviews by Floridi and Cowls (2019), Hagendorff (2020), and Jobin et al. (2019).”
Dual-Processing Accounts of Reasoning, Judgment, and Social Cognition
Evans, J. S. B. T. (2008) · Annual Review of Psychology · doi:10.1146/annurev.psych.59.103006.093629 · Cited by 5,137 · cited 1×
“Drawing on dual process theory (Evans, 2008), Thaler and Sunstein describe two types of nudges, those that impact on our “Automatic System”, such as placing the item we wish to nudge someone towards at eye level, and those that impact on our “Reflec-tive System”, such as nudges that encourage us to…”
Method(1997) · Choice Reviews Online · doi:10.5860/choice.34-3702 · Cited by 4,887
Floridi, L., Cowls, J., Beltrametti, M., Chatila, R., Chazerand, P., Dignum, V., Luetge, C., Madelin, R., Pagallo, U., Rossi, F., et al. (2018) · Minds and Machines · doi:10.1007/s11023-018-9482-5 · Cited by 4,774 · cited 1×
“…Report on the ethical treatment of research subjects (via the “Respect for Persons” principle), the highly influential four principles of Beauchamp and Childress (2001) and its more recent extension by the AI4People framework (Floridi et al., 2018), all list autonomy as a basic ethical principle.”
BackgroundFreedom of the Will and the Concept of a Person
Frankfurt, H. G. (1971) · The Journal of Philosophy · doi:10.2307/2024717 · Cited by 4,663
Principles of biomedical ethics
Horton, J. (2002) · Transactions of the Royal Society of Tropical Medicine and Hygiene · doi:10.1016/s0035-9203(02)90265-8 · Cited by 4,377
On seeing human: A three-factor theory of anthropomorphism
Epley, N., Waytz, A., & Cacioppo, J. T. (2007) · Psychological Review · doi:10.1037/0033-295x.114.4.864 · Cited by 4,069 · cited 1×
“This helps to explain the human tendency to anthropomorphise technology by attributing human qualities and characteristics, such as motivations, intentions, and emotions, to non-human entities and inanimate objects (Epley et al., 2007; Fossa, 2018; Turkle, 2012).”
BackgroundKorsgaard, C. M., & O'Neill, O. (1996) · Cambridge University Press eBooks · doi:10.1017/cbo9780511554476 · Cited by 4,013 · cited 1×
“For example, on some Kantian views, persons are autonomous when they act from the endorsement of the absolute value of the dignity of all rational agents (Formosa, 2013) or from the practical identity of themselves as an equal lawgiving member of the kingdom of ends (Korsgaard, 1996).”
BackgroundThe motivational pull of video games: a self-determination theory approach
Ryan, R. M., Rigby, C. S., & Przybylski, A. (2006) · Motivation and Emotion · doi:10.1007/s11031-006-9051-8 · Cited by 3,332 · cited 1×
“Further, having some degree of control over how our choices are realised, and not being subject to excessive oversight or control in their pursuit, is also important for our sense of autonomy (Ryan et al., 2006; Ryan & Deci, 2017).”
BackgroundGroundwork of The Metaphysic of Morals
Kant, I. (2020) · Immanuel Kant: Groundwork of the Metaphysic of Morals in focus · doi:10.4324/9780203714805-2 · Cited by 3,280
Raz, J. (1988) · doi:10.1093/0198248075.001.0001 · Cited by 2,895
The Ethics of AI Ethics: An Evaluation of Guidelines
Hagendorff, T. (2020) · Minds and Machines · doi:10.1007/s11023-020-09517-8 · Cited by 1,911
The Ethics of AI Ethics -- An Evaluation of Guidelines
Hagendorff, T. (2019) · arXiv · doi:10.48550/arxiv.1903.03425 · Cited by 1,907
Social robots for education: a review.
Belpaeme, T., Kennedy, J., Ramachandran, A., Scassellati, B., & Tanaka, F. (2018) · Science Robotics · doi:10.1126/scirobotics.aat5954 · Cited by 1,558
Relational Autonomy: Feminist Perspectives on Autonomy, Agency, and the Social Self
Mackenzie, C., & Stoljar, N. (2000) · Cited by 1,414
Autonomy and Trust in Bioethics
O'Neill, O. (2002) · Cambridge University Press eBooks · doi:10.1017/cbo9780511606250 · Cited by 1,202
Automation bias: a systematic review of frequency, effect mediators, and mitigators
Goddard, K., Roudsari, A., & Wyatt, J. C. (2012) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000089 · Cited by 1,162
Breazeal, C. (2003) · Robotics and Autonomous Systems · doi:10.1016/s0921-8890(02)00373-1 · Cited by 1,032 · cited 1×
“Breazeal (2003, p. 167) defines social robots as the “class of robots that people anthropomorphise in order to interact with them”.”
BackgroundGranny and the robots: ethical issues in robot care for the elderly
Sharkey, A., & Sharkey, N. (2012) · Ethics and Information Technology · doi:10.1007/s10676-010-9234-6 · Cited by 985 · cited 1×
“…difficult to achieve, and given that social robots will likely harvest large amounts of very intimate data about us, the potential this has for expressing disrespect and limiting our ability to have autonomous control over our personal data is concerning (Hagendorff, 2020; Sharkey & Sharkey, 2012).”
BackgroundAutonomy and trust in bioethics
(2003) · Choice Reviews Online · doi:10.5860/choice.40-4540 · Cited by 900
Adams, N., Little, T. D., & Ryan, R. M. (2017) · Development of Self-Determination Through the Life-Course · doi:10.1007/978-94-024-1042-6_4 · Cited by 886
Wolf, S. (2005) · Personal Autonomy · doi:10.1017/cbo9780511614194.012 · Cited by 834
Watson, G. (1975) · The Journal of Philosophy · doi:10.2307/2024703 · Cited by 812
Hansen, P. G., & Jespersen, A. M. (2013) · European Journal of Risk Regulation · doi:10.1017/s1867299x00002762 · Cited by 773 · cited 1×
“This can involve “nudging” people through careful design of the “choice architecture” or context within which choices are made (for further discussions of this extensive literature see, for example, Thaler & Sunstein, 2008; Cohen, 2013; Quigley, 2013; Hansen & Jespersen, 2013).”
BackgroundOn Defining Artificial Intelligence
Wang, P. (2019) · Journal of Artificial General Intelligence · doi:10.2478/jagi-2019-0002 · Cited by 740 · cited 1×
“…shall understand it here to be creating information-processing systems that can do things which we would typically classify as intelligent were a human to do them, such as reason, plan, solve problems, categorise, adapt to its environment, and learn from experience (for discussion see Wang, 2019).”
BackgroundThe evolution and psychology of self-deception
von Hippel, W., & Trivers, R. (2011) · Behavioral and Brain Sciences · doi:10.1017/s0140525x10001354 · Cited by 736 · cited 1×
“…biases that impact human thinking (Kahneman, 2011), and evidence that people are more open to critical reflection after positive self-affirmation (von Hippel & Trivers, 2011), to first bolster a person’s sense of self-worth before alerting them to potential biases it has identified in their…”
BackgroundRealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models
Gehman, S., Gururangan, S., Sap, M., Choi, Y., & Smith, N. A. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.301 · Cited by 693
A Contextual Approach to Privacy Online
Nissenbaum, H. (2011) · Daedalus · doi:10.1162/daed_a_00113 · Cited by 649 · cited 1×
“This points to the problem of what Nissenbaum (2011) calls the “transparency paradox”.”
BackgroundThe Invention of Autonomy: A History of Modern Moral Philosophy
Schneewind, J. B. (1997) · Cambridge University Press eBooks · doi:10.1017/cbo9780511818288 · Cited by 640
Building a Stronger CASA: Extending the Computers Are Social Actors Paradigm
Gambino, A., Fox, J., & Ratan, R. (2020) · Human-Machine Communication · doi:10.30658/hmc.1.5 · Cited by 626 · cited 1×
“This leads humans to interact with technology by following the same social scripts, schemas, and rules, such as norms of politeness and reciprocity, that are used in human–human interactions (Reeves & Nass, 1996; for an updated review of CASA see Gambino et al., 2020).”
BackgroundSociety-in-the-loop: programming the algorithmic social contract
Rahwan, I. (2018) · Ethics and Information Technology · doi:10.1007/s10676-017-9430-8 · Cited by 553 · cited 1×
“…has been used in a number of ways across several fields, from human–computer interaction (HCI), human–robot interaction (HRI), machine learning (ML) (Rahwan, 2018), and in the military context to discuss autonomous weapons systems (Schmitt & Thurnher, 2013; Sparrow, 2016; Walsh et al., 2019).”
BackgroundThe Politics of Persons: Individual Autonomy and Socio-historical Selves
Christman, J. (2009) · Cambridge University Press eBooks · doi:10.1017/cbo9780511635571 · Cited by 537 · cited 1×
“Both substantive and procedural theories of personal autonomy typically differentiate between competency and authenticity conditions (Christman, 2009; Susser et al., 2019).”
BackgroundMeyers, D. T. (2018) · Routledge eBooks · doi:10.4324/9780429500466 · Cited by 523
Pu, L., Moyle, W., Jones, C., & Todorovic, M. (2019) · The Gerontologist · doi:10.1093/geront/gny046 · Cited by 516 · cited 1×
“For example, a systematic review of the use of social robots among older adults found a lack of high-quality studies but some indications that social robots can reduce agitation, anxiety, and loneliness (Pu et al., 2019), which could in turn boost relevant autonomy competencies such as self-esteem.”
BackgroundRelational Autonomy, Liberal Individualism, and the Social Constitution of Selves
Christman, J. (2004) · Philosophical Studies · doi:10.1023/b:phil.0000014532.56866.5c · Cited by 510
Technology, autonomy, and manipulation
Susser, D., Roessler, B., & Nissenbaum, H. (2019) · Internet Policy Review · doi:10.14763/2019.2.1410 · Cited by 474
Ethics in artificial intelligence: introduction to the special issue
Dignum, V. (2018) · Ethics and Information Technology · doi:10.1007/s10676-018-9450-z · Cited by 459
Anthropomorphism and human likeness in the design of robots and human-robot interaction
Fink, J. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-34103-8_20 · Cited by 455 · cited 1×
“However, while the tendency to anthropomorphise technology applies beyond social robots, it has been shown that the more socially interactive and human-like the robot is, the stronger is the tendency to anthropomorphise it (Fink, 2012).”
BackgroundA Contextual Approach to Privacy Online
Helen, N. (2012) · Digital Enlightenment Yearbook 2012 · doi:10.3233/978-1-61499-057-4-219 · Cited by 445
Relational Autonomy, Normative Authority and Perfectionism
Mackenzie, C. (2008) · Journal of Social Philosophy · doi:10.1111/j.1467-9833.2008.00440.x · Cited by 397 · cited 4×
“Several self-attitudes, such as self-respect, self-love, self-esteem, and self-trust, are also seen as important autonomy competencies (Benson, 1994; Mackenzie, 2008).”
BackgroundInfluentialThe Ethics of AI Ethics: An Evaluation of Guidelines
Hagendorff, T. (2020) · PhilPapers (PhilPapers Foundation) · Cited by 394
The global landscape of AI ethics guidelines
Jobin, A., Ienca, M., & Vayena, E. (2019) · Repository for Publications and Research Data (ETH Zurich) · doi:10.3929/ethz-b-000375277 · Cited by 360
Autonomy and the Challenges to Liberalism: New Essays
Christman, J., & Anderson, J. (2005) · Autonomy and the Challenges to Liberalism · doi:10.1017/cbo9780511610325.002 · Cited by 340
Vallor, S. (2015) · Philosophy & Technology · doi:10.1007/s13347-014-0156-9 · Cited by 330 · cited 1×
“In terms of skill maintenance and cultivation in adults, Vallor (2015) raises the related worry of “moral deskilling”.”
BackgroundThe ethics of nudging: An overview
Schmidt, A. T., & Engelen, B. (2020) · Philosophy Compass · doi:10.1111/phc3.12658 · Cited by 328
Relational artifacts with children and elders: the complexities of cybercompanionship
Turkle, S., Taggart, W., Kidd, C. D., & Dasté, O. (2006) · Connection Science · doi:10.1080/09540090600868912 · Cited by 286 · cited 3×
“Many of these imaginary interventions by a social robot constitute examples of “nudging” a human to be more autonomous (Thaler & Sunstein, 2008).”
BackgroundBenson, P., & Journal of Philosophy Inc. (1994) · Journal of Philosophy · doi:10.2307/2940760 · Cited by 271 · cited 1×
“Several self-attitudes, such as self-respect, self-love, self-esteem, and self-trust, are also seen as important autonomy competencies (Benson, 1994; Mackenzie, 2008).”
BackgroundI'd blush if I could: closing gender divides in digital skills through education
UNESCO, & Coalition, E. S. (2019) · doi:10.54675/rapc9356 · Cited by 262 · cited 3×
“A concrete example of this is that at one point Apple’s Siri responded to “You’re a slut” with “I’d blush if I could” (UNESCO, 2019, p. 107).”
BackgroundInfluentialThe Politics of Persons: Individual Autonomy and Socio-historical Selves
Christman, J. P. (2009) · Cited by 246
Cohen, S. (2013) · The American Journal of Bioethics · doi:10.1080/15265161.2013.781704 · Cited by 218 · cited 1×
“This can involve “nudging” people through careful design of the “choice architecture” or context within which choices are made (for further discussions of this extensive literature see, for example, Thaler & Sunstein, 2008; Cohen, 2013; Quigley, 2013; Hansen & Jespersen, 2013).”
Background(1997) · The Invention of Autonomy · doi:10.1017/cbo9780511818288.023 · Cited by 206
The privacy implications of social robots: Scoping review and expert interviews
Lutz, C., Schöttler, M., & Hoffmann, C. P. (2019) · Mobile Media & Communication · doi:10.1177/2050157919843961 · Cited by 204
Autonomy and Oppressive Socialization
Benson, P., & Department of Philosophy, F. S. U. (1991) · Social Theory and Practice · doi:10.5840/soctheorpract199117319 · Cited by 197 · cited 3×
“This is seen in the much-discussed case of the 1950s housewife who endorses the sexist and heteronomous norms that a woman should be subservient and under the control of her husband because her oppressive socialisation has left her unable to assess the falsity of such norms (Benson, 1991).”
BackgroundAUTONOMY AND THE SPLIT‐LEVEL SELF
Friedman, M. A. (1986) · The Southern Journal of Philosophy · doi:10.1111/j.2041-6962.1986.tb00434.x · Cited by 196 · cited 1×
“Oppression can also undermine the authenticity of our choices by leading us to hold values and norms that are the result of undue external social pressures and are thus not really our “own” (Friedman, 1986).”
BackgroundRobots, Rape, and Representation
Sparrow, R. (2017) · International Journal of Social Robotics · doi:10.1007/s12369-017-0413-z · Cited by 195 · cited 3×
“…include undertaking care work for children, the elderly and the sick, becoming our teachers and work colleagues and, eventually, our social companions, friends, and even sexual partners (Darling, 2018; Ferreira et al., 2017; Lin et al., 2012; Mackenzie, 2018; Pirhonen et al., 2020; Sparrow, 2017).”
Background“Who’s Johnny?” Anthropomorphic Framing in Human–Robot Interaction, Integration, and Policy
Darling, K. (2017) · Oxford Scholarship Online · doi:10.1093/oso/9780190652951.003.0012 · Cited by 194
A Unified Framework of Five Principles for AI in Society
Floridi, L., & Cowls, J. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3831321 · Cited by 182
Stifling artificial intelligence: Human perils
Gürkaynak, G., Yılmaz, İ., & Haksever, G. (2016) · Computer law & security review · doi:10.1016/j.clsr.2016.05.003 · Cited by 180
Mind the gap: responsible robotics and the problem of responsibility
Gunkel, D. J. (2020) · Ethics and Information Technology · doi:10.1007/s10676-017-9428-2 · Cited by 174 · cited 1×
“Whether social robots or AIs could ever become persons or moral agents are further questions beyond our scope (but for discussion see, for example, Gunkel, 2020; Sparrow, 2012; Fosch-Villaronga et al., 2020).”
BackgroundSocial Robots for Early Language Learning: Current Evidence and Future Directions
Kanero, J., Geçkin, V., Oranç, C., Mamus, E., Küntay, A. C., & Göksun, T. (2018) · Child Development Perspectives · doi:10.1111/cdep.12277 · Cited by 173
Robots and Respect: Assessing the Case Against Autonomous Weapon Systems
Sparrow, R. (2016) · Ethics & International Affairs · doi:10.1017/s0892679415000647 · Cited by 167 · cited 3×
“…has been used in a number of ways across several fields, from human–computer interaction (HCI), human–robot interaction (HRI), machine learning (ML) (Rahwan, 2018), and in the military context to discuss autonomous weapons systems (Schmitt & Thurnher, 2013; Sparrow, 2016; Walsh et al., 2019).”
BackgroundPersonal Autonomy and the Paradox of Feminine Socialization
Meyers, D. T., & Journal of Philosophy Inc. (1987) · Journal of Philosophy · doi:10.5840/jphil198784117 · Cited by 164 · cited 1×
“Competency conditions point to the fact that to be autonomous you must be able to do various things and have certain skills and self-attitudes (Meyers, 1987), such as being able to critically reflect on your values, adopt ends, imagine yourself being otherwise, and regard yourself as the bearer of…”
BackgroundWhat Should We Want From a Robot Ethic?
Asaro, P. M. (2006) · The International Review of Information Ethics · doi:10.29173/irie134 · Cited by 164
The Value of Autonomy and Autonomy of the Will
Darwall, S. (2006) · Ethics · doi:10.1086/498461 · Cited by 163 · cited 2×
“However, over time the focus of autonomy has shifted (Darwall, 2006).”
BackgroundCritiquing the Reasons for Making Artificial Moral Agents
van Wynsberghe, A., & Robbins, S. (2019) · Science and Engineering Ethics · doi:10.1007/s11948-018-0030-8 · Cited by 161 · cited 1×
“Consider the example of Corti, an AI-powered machine that informs emergency call responders whether the caller is at risk of a heart attack through using machine learning to analyse breathing and speech patterns (van Wynsberghe & Robbins, 2019).”
MethodI'd blush if I could : closing gender divides in digital skills through education
(2019) · Singapore Management University Institutional Knowledge (InK) (Singapore Management University) · Cited by 158
A Unified Framework of Five Principles for AI in Society
Floridi, L., & Cowls, J. (2021) · Philosophical Studies Series · doi:10.1007/978-3-030-81907-1_2 · Cited by 134
'Who's Johnny?' Anthropomorphic Framing in Human-Robot Interaction, Integration, and PolicyPreprint
Darling, K. F. (2015) · SSRN Electronic Journal · doi:10.2139/ssrn.2588669 · Cited by 131
NUDGING FOR HEALTH: ON PUBLIC POLICY AND DESIGNING CHOICE ARCHITECTURE
Quigley, M. (2013) · Medical Law Review · doi:10.1093/medlaw/fwt022 · Cited by 124 · cited 1×
“This can involve “nudging” people through careful design of the “choice architecture” or context within which choices are made (for further discussions of this extensive literature see, for example, Thaler & Sunstein, 2008; Cohen, 2013; Quigley, 2013; Hansen & Jespersen, 2013).”
BackgroundA Social Robot to Mitigate Stress, Anxiety, and Pain in Hospital Pediatric Care
Jeong, S., Logan, D. E., Goodwin, M. S., Graca, S., O'Connell, B., Goodenough, H., Anderson, L., Stenquist, N., Fitzpatrick, K., Zisook, M., et al. (2015) · Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction Extended Abstracts · doi:10.1145/2701973.2702028 · Cited by 115 · cited 1×
“Similar positive impacts have been found in other populations (Jeong et al., 2015).”
ResultBankins, S., & Formosa, P. (2020) · European Journal of Work and Organizational Psychology · doi:10.1080/1359432x.2019.1620328 · Cited by 109 · cited 3×
“…versions of the [social] robots that we can build today” (Sparrow, 2017, p. 468), that have advanced motor, social and emotional skills, and can draw on “empathetic technology” and “extensive knowledge of our preferences” to “tailor their behaviours” toward us (Bankins & Formosa, 2020, p. 3).”
BackgroundSelf-citationFosch‐Villaronga, E., Lutz, C., & Tamò‐Larrieux, A. (2019) · International Journal of Social Robotics · doi:10.1007/s12369-019-00605-z · Cited by 104 · cited 5×
“At a society level, beyond dealing with the issues already raised above and the broader existing regulatory frameworks around privacy, AI, and robotic safety (Fosch-Villaronga et al., 2020; Hagendorff, 2020), there are two further areas of focus worth mentioning here.”
BackgroundInfluentialMaking moral machines: why we need artificial moral agents
Formosa, P., & Ryan, M. (2020) · AI & Society · doi:10.1007/s00146-020-01089-6 · Cited by 96 · cited 3×
“…versions of the [social] robots that we can build today” (Sparrow, 2017, p. 468), that have advanced motor, social and emotional skills, and can draw on “empathetic technology” and “extensive knowledge of our preferences” to “tailor their behaviours” toward us (Bankins & Formosa, 2020, p. 3).”
BackgroundSelf-citationLyell, D., Coiera, E., Chen, J., Shah, P., & Magrabi, F. (2021) · BMJ Health & Care Informatics · doi:10.1136/bmjhci-2020-100301 · Cited by 95 · cited 2×
“While there may be more complex ways to make this distinction (such as differentiating between automating information provision, information analysis, and decision options; for discussion, see Lyell et. al., 2021), this simple tripartite model will suffice for our purposes.”
BackgroundCan communication with social robots influence how children develop empathy? Best-evidence synthesis
Pashevich, E. (2021) · AI & Society · doi:10.1007/s00146-021-01214-z · Cited by 89 · cited 1×
“…(Kanero et al., 2018), and if social robots take on more education and caring roles, then this could lead to children developing lower levels of autonomy competencies than they would through skilled human teaching (although there is a general lack of evidence in this regard; see Pashevich, 2021).”
BackgroundRobotic Nudges: The Ethics of Engineering a More Socially Just Human Being
Borenstein, J., & Arkin, R. (2015) · Science and Engineering Ethics · doi:10.1007/s11948-015-9636-2 · Cited by 84 · cited 5×
“With this background in place, we then justify our focus on social robots on the grounds that, given their physical presence and social abilities, they have the potential to have very significant impacts on our autonomy (Borenstein & Arkin, 2016).”
BackgroundInfluentialEtzioni, A., & Etzioni, O. (2016) · Ethics and Information Technology · doi:10.1007/s10676-016-9400-6 · Cited by 84 · cited 1×
“Machine autonomy can be understood as “the ability of a computer to follow a complex algorithm in response to environmental inputs, independently of real-time human input” (Etzioni & Etzioni, 2016, p. 149).”
BackgroundSunstein, C. R. (2015) · bepress Legal Repository · doi:10.17176/20181005-151230-0 · Cited by 76
Rogers, W. A., Draper, H., & Carter, S. M. (2021) · Bioethics · doi:10.1111/bioe.12885 · Cited by 76 · cited 1×
“This transforms Corti from what is known in the medical AI literature as a “decision support” into an “autonomous decision” technology (Lyell et al., 2021; Rogers et al., 2021).”
BackgroundRobbins, S. (2020) · AI & SOCIETY · doi:10.1007/s00146-019-00891-1 · Cited by 70 · cited 2×
“Consider the example of Corti, an AI-powered machine that informs emergency call responders whether the caller is at risk of a heart attack through using machine learning to analyse breathing and speech patterns (van Wynsberghe & Robbins, 2019).”
BackgroundMethodWhat are Adaptive Preferences? Exclusion and Disability in the Capability Approach
Begon, J. (2014) · Journal of Applied Philosophy · doi:10.1111/japp.12102 · Cited by 69 · cited 1×
“This problem is related to the issue of adaptive preferences, whereby people can adapt their preferences to suit poor or unjust circumstances (Begon, 2015).”
BackgroundPetit, N. (2017) · SSRN Electronic Journal · doi:10.2139/ssrn.2931339 · Cited by 68 · cited 2×
“…range of possible responses to the ethical issues raised here include improved user education and public awareness, design considerations, ethical guidelines, industry standards, government agencies, and regulatory or legal frameworks (for discussion see Fosch-Villaronga et al., 2020; Petit, 2017).”
BackgroundPirhonen, J., Melkas, H., Laitinen, A., & Pekkarinen, S. (2020) · Ethics and Information Technology · doi:10.1007/s10676-019-09524-z · Cited by 60 · cited 5×
“(Clearly, this example extends to many other cases of robots helping people to overcome restricted functional abilities—see Pirhonen et al. (2020).”
BackgroundInfluentialTAME: Time-Varying Affective Response for Humanoid Robots
Moshkina, L. V., Park, S., Arkin, R. C., Lee, J. K., & Jung, H. (2011) · International Journal of Social Robotics · doi:10.1007/s12369-011-0090-2 · Cited by 59 · cited 1×
“These positive social outcomes could be strengthened through the social robot’s use of gestures, tone of voice, eye contact, expression of (what appears to be) emotions such as sympathy, and physically embodied presence (Borenstein & Arkin, 2016; Li, 2013; Moshkina et al., 2011).”
BackgroundArtificial moral agents: moral mentors or sensible tools?
Fossa, F. (2018) · Ethics and Information Technology · doi:10.1007/s10676-018-9451-y · Cited by 58 · cited 4×
“This helps to explain the human tendency to anthropomorphise technology by attributing human qualities and characteristics, such as motivations, intentions, and emotions, to non-human entities and inanimate objects (Epley et al., 2007; Fossa, 2018; Turkle, 2012).”
BackgroundInfluentialThe bionic blues: Robot rejection lowers self-esteem
Nash, K. A., Lea, J. M., Davies, T., & Yogeeswaran, K. (2017) · Computers in Human Behavior · doi:10.1016/j.chb.2017.09.018 · Cited by 51
Paternalism and Respect for Autonomy
Scoccia, D. (1990) · Ethics · doi:10.1086/293179 · Cited by 49 · cited 1×
“Autonomy as a moral principle also speaks to the dangers of paternalism on the grounds that it disrespects autonomy through bypassing the consent of others (Scoccia, 1990).”
BackgroundBeauchamp, T. L., & DeGrazia, D. (2004) · Springer eBooks · doi:10.1007/1-4020-2127-5_3 · Cited by 41
Social robots: beyond tools to partners
Breazeal, C., Gray, J., Hoffman, G., & Berlin, M. (2005) · IEEE RO-MAN · doi:10.1109/roman.2004.1374820 · Cited by 34 · cited 4×
“…will likely be due, in part, to the human tendency to anthropomorphise social robots by regarding them as social agents who have “states of mind”, including attitudes toward us, that develop through our intuitive social interactions with them (Breazeal et al., 2004; Fossa, 2018; Turkle, 2012).”
BackgroundMethodInfluentialPersonalized clothing recommendation by a social robot
Woiceshyn, L., Wang, Y., Nejat, G., & Benhabib, B. (2017) · IEEE International Symposium on Robotics and Intelligent Sensors (IRIS) · doi:10.1109/iris.2017.8250118 · Cited by 32 · cited 1×
“…proactively selects an appropriate shirt and tie for a business meeting that Sam has that morning (for an example of this sort of social robot see Woiceshyn et al., 2017) and brings the clothes to Sam at the exact moment it calculates that he will need them to get dressed to make his meeting on…”
BackgroundForty Years of the Four Principles: Enduring Themes from Beauchamp and Childress
Shea, M. (2020) · The Journal of Medicine and Philosophy A Forum for Bioethics and Philosophy of Medicine · doi:10.1093/jmp/jhaa020 · Cited by 31 · cited 1×
“This is clearest in the Kantian tradition (Kant, 1996), but is also present in various forms of principlism (Shea, 2020).”
BackgroundCompanion Robots: the Hallucinatory Danger of Human-Robot Interactions
Bisconti, P., & Nardi, D. (2018) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3278721.3278741 · Cited by 25
A World with Robots: International Conference on Robot Ethics: ICRE 2015
Ferreira, M. I. A., Sequeira, J., Tokhi, M. O., Kadar, E. E., & VIRK, G. S. (2017) · International series on intelligent systems, control and automation: science and engineering · doi:10.1007/978-3-319-46667-5 · Cited by 23
Neurotechnologies, Relational Autonomy, and Authenticity
Walker, M. J., & Mackenzie, C. (2020) · International Journal of Feminist Approaches to Bioethics · doi:10.3138/ijfab.13.1.06 · Cited by 23 · cited 1×
“A choice is authentic if one acts “on motives, desires, preferences and other reasons” that are “one’s own”, and they count as “one’s own” when, on reflection, one endorses or acknowledges them (Walker & Mackenzie, 2020, p. 8).”
BackgroundK ant's Conception of Personal Autonomy
Formosa, P. (2013) · Journal of Social Philosophy · doi:10.1111/josp.12028 · Cited by 22 · cited 4×
“However, critics of such theories worry that they can struggle to justify which substantive values or norms are the right ones (Formosa, 2013).”
BackgroundMethodInfluentialSelf-citationSeïler, N. R., & Craig, P. (2016) · Elsevier eBooks · doi:10.1016/b978-0-12-801872-9.00004-1 · Cited by 18
Sexbots: Sex Slaves, Vulnerable Others or Perfect Partners?
Mackenzie, R. (2018) · International Journal of Technoethics · doi:10.4018/ijt.2018010101 · Cited by 14 · cited 1×
“…include undertaking care work for children, the elderly and the sick, becoming our teachers and work colleagues and, eventually, our social companions, friends, and even sexual partners (Darling, 2018; Ferreira et al., 2017; Lin et al., 2012; Mackenzie, 2018; Pirhonen et al., 2020; Sparrow, 2017).”
BackgroundSupporting human autonomy in AI systems
Calvo, R. A., Peters, D., Vold, K., & Ryan, R. M. (2020) · PhilPapers (PhilPapers Foundation) · Cited by 12
Net Privacy: How We Can Be Free in an Age of Surveillance
Molitorisz, S. (2020) · McGill-Queen's University Press eBooks · doi:10.1515/9780228002888 · Cited by 9
Li, J. J. (2013) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/2522848.2532193 · Cited by 8 · cited 1×
“These positive social outcomes could be strengthened through the social robot’s use of gestures, tone of voice, eye contact, expression of (what appears to be) emotions such as sympathy, and physically embodied presence (Borenstein & Arkin, 2016; Li, 2013; Moshkina et al., 2011).”
BackgroundA World with Robots: International Conference on Robot Ethics: ICRE 2015
Ferreira, M., Sequeira, J. S., Tokhi, M. O., Kadar, E. E., & VIRK, G. S. (2017) · Cited by 8
Research works printed but not linked: 10
- Anderson, J., Christman, J., & Anderson, J. (2005). Autonomy and the challenges to liberalism. Cam- bridge University Press.
- Bostrom, N. (2014). Superintelligence. Oxford University Press.
- Darling, K. (2016). Extending legal protection to social robots. In R. Calo, A. Froomkin, & I. Kerr (Eds.), Robot law. Edward Elgar.
- Formosa, P. (2017). Kantian ethics. Cambridge University Press.
- Lin, P., Abney, K., & Bekey, G. A. (Eds.). (2012). Robot ethics. MIT Press.
- Schmitt, M. N., & Thurnher, J. S. (2013). "Out of the loop": Autonomous weapon systems and the law of armed conflict. Harvard National Security Journal, 4, 231-281.
- Sparrow, R. (2012). Can machines be people? In P. Lin, K. Abney, & G. Bekey (Eds.), Robot ethics (pp. 301-316). MIT Press.
- Stoljar, N. (2000). Autonomy and the FEMINIST INTUITion. In C. Mackenzie & N. Stoljar (Eds.), Rela- tional autonomy. Oxford University Press.
- Thaler, R. H., & Sunstein, C. R. (2008). Nudge. Yale University Press.
- Turkle, S. (2012). Alone together. Basic Books.
Cited by
AI Agents vs. Agentic AI: A Conceptual taxonomy, applications and challenges
Sapkota, R., Roumeliotis, K. I., & Karkee, M. (2026) · Information Fusion · doi:10.1016/j.inffus.2025.103599 · Cited by 671
“Auton-omy enables scalable deployment in applications where persistent oversight (human-in-the-loop) is impractical, such as customer support bots or scheduling assistants [52,64].”
The Ethical Implications of Artificial Intelligence (AI) For Meaningful Work
Bankins, S., & Formosa, P. (2023) · Journal of Business Ethics · doi:10.1007/s10551-023-05339-7 · Cited by 448
“We focus only on narrow AI as it is already used across many diverse sectors, including in healthcare, judicial, educational, manufacturing, and military contexts, among many others (see Bankins & Formosa, 2021; Bekey, 2012; Walsh et al., 2019).”
BackgroundSelf-citationAI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challenges
Sapkota, R., Roumeliotis, K. I., & Karkee, M. (2025) · arXiv · doi:10.48550/arxiv.2505.10468 · Cited by 295
“Auton-omy enables scalable deployment in applications where persistent oversight (human-in-the-loop) is impractical, such as customer support bots or scheduling assistants [52,64].”
AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challenges
Sapkota, R., Roumeliotis, K. I., & Karkee, M. (2025) · SuperIntelligence - Robotics - Safety & Alignment · doi:10.70777/si.v2i3.15161 · Cited by 186
Human-machine Collaborative Decision-making: An Evolutionary Roadmap Based on Cognitive Intelligence
Ren, M., Chen, N., & Qiu, H. (2023) · International Journal of Social Robotics · doi:10.1007/s12369-023-01020-1 · Cited by 178
Metadata
- Authors
- Formosa, P.
- Affiliations
- Macquarie University (AU)
- Corresponding author
- Formosa, Paul
- ORCID
- Formosa, Paul
- Venue
- Minds and Machines
- Published
- 2021-10-25
- Volume
- 31
- Issue
- 4
- Pages
- 595-616
- ISSN
- 0924-6495 (ISSN-L)
- 1572-8641
- OpenAlex
- W3210139507
- Cited by
- 144
- References
- 105
- Citations by year
- 2026: 49 · 2025: 48 · 2024: 28 · 2023: 12 · 2022: 7
- FWCI
- 11.51
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Minds and Machines (publishedVersion, cc-by)
- PhilPapers (PhilPapers Foundation) (submittedVersion, cc-by)
- PhilPapers (PhilPapers Foundation) (submittedVersion)
- Topics
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Law › Literary and Philosophical Studies
- Keywords
- social robots, human autonomy, robot autonomy, AI ethics, human-robot interaction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.