The DOI matches its Crossref record; 84 of the 84 references it cites were found in a registry or an index.
Patiency is not a virtue: the design of intelligent systems and systems of ethics
Open accessAbstract
The question of whether AI systems such as robots can or should be afforded moral agency or patiency is not one amenable either to discovery or simple reasoning, because we as societies constantly reconstruct our artefacts, including our ethical systems. Consequently, the place of AI systems in society is a matter of normative, not descriptive ethics. Here I start from a functionalist assumption, that ethics is the set of behaviour that maintains a society. This assumption allows me to exploit the theoretical biology of sociality and autonomy to explain our moral intuitions. From this grounding I extend to consider possible ethics for maintaining either human- or of artefact-centred societies. I conclude that while constructing AI systems as either moral agents or patients is possible, neither is desirable. In particular, I argue that we are unlikely to construct a coherent ethics in which it it is ethical to afford AI moral subjectivity. We are therefore obliged not to build AI we are obliged to.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The genetical evolution of social behaviour. I
Hamilton, W. D. (1964) · Journal of Theoretical Biology · doi:10.1016/0022-5193(64)90038-4 · Cited by 15,725 · cited 3×
“Returning to Hamilton’s Law, the answer might be “yes” if we could assume that our technology was likely to survive our species or even our civilisation, but to date digital technology formats tend to survive less than 1 16 as long as individual humans.”
BackgroundInfluentialThe Evolution of Reciprocal Altruism
Trivers, R. L. (1971) · The Quarterly Review of Biology · doi:10.1086/406755 · Cited by 11,010
A treatise of human nature (1739-40)
Hume, D. (1996) · The Sublime · doi:10.1017/cbo9780511620409.040 · Cited by 9,842
Cooperation and Punishment in Public Goods Experiments
Fehr, E., & Gächter, S. (2000) · American Economic Review · doi:10.1257/aer.90.4.980 · Cited by 4,547
(1987) · Medical Entomology and Zoology · Cited by 4,269
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012) · Proceedings of the 3rd Innovations in Theoretical Computer Science Conference · doi:10.1145/2090236.2090255 · Cited by 3,501
Semantics derived automatically from language corpora contain human-like biases
Caliskan, A., Bryson, J. J., & Narayanan, A. (2017) · Science · doi:10.1126/science.aal4230 · Cited by 2,887
Self-citationKitano, H. (2004) · Nature Reviews Genetics · doi:10.1038/nrg1471 · Cited by 2,278 · cited 1×
“This is genetically determined by their strain, but the two sorts of behaviour are accessible from each other during reproduction (the construction of new individuals) via common mutations 1 3 (Kitano 2004; Youk and Lim 2014).”
BackgroundCertifying and Removing Disparate Impact
Feldman, M., Friedler, S. A., Moeller, J., Scheidegger, C., & Venkatasubramanian, S. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2783311 · Cited by 1,827
Atkinson, A. B. (2015) · Harvard University Press eBooks · doi:10.4159/9780674287013 · Cited by 1,684
Byrne, R. W., & Whiten, A. (1988) · Medical Entomology and Zoology · Cited by 1,683
(1982) · Medical Entomology and Zoology · Cited by 1,329
Good Natured: The Origins of Right and Wrong in Humans and Other Animals
de Waal, F. Β. Μ. (1996) · Dominican Scholar (Dominican University of California) · Cited by 1,075
DE WAAL, F. B. M., & Waal, F. B. M. D. (1996) · Harvard University Press eBooks · doi:10.4159/9780674033177 · Cited by 952
The better angels of our nature : the decline of violence in history and its causes
Pinker, S. A. (2011) · Cited by 942
Grundlegung zur Metaphysik der Sitten
Immanuel, I. (2016) · Felix Meiner Verlag eBooks · doi:10.28937/978-3-7873-2878-9 · Cited by 900
The Better Angels in Our Nature: The Decline of Violence in History and Its Causes
Bulloch, D. (2012) · Global Policy · doi:10.1111/j.1758-5899.2012.00186.x · Cited by 636
Atkinson, A. B. (2025) · London School of Economics and Political Science Research Online (London School of Economics and Political Science) · doi:10.21953/lse.q3wqhy5t36kn · Cited by 590
The Common Law and Economic Growth: Hayek Might Be Right
Mahoney, P. G. (2001) · The Journal of Legal Studies · doi:10.1086/322053 · Cited by 553
Moral typecasting: Divergent perceptions of moral agents and moral patients.
Gray, K., & Wegner, D. M. (2009) · Journal of Personality and Social Psychology · doi:10.1037/a0013748 · Cited by 535
Carlson, N. R. (1978) · Cited by 534
The evolution of animal ‘cultures’ and social intelligence
Whiten, A., & van Schaik, C. P. (2007) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2006.1998 · Cited by 482
Evolution and Learning: The Baldwin Effect Reconsidered
(2003) · The MIT Press eBooks · doi:10.7551/mitpress/2871.001.0001 · Cited by 482
Bryson, J. J. (2010) · Natural Language Processing · doi:10.1075/nlp.8.11bry · Cited by 455
Self-citationThe Machine Question: Ethics, Alterity, and Technology
Gunkel, D. J. (2007) · Series in Philosophy/Communication · doi:10.5840/philcomm200745 · Cited by 440
Grundlegung zur Metaphysik der Sitten
Kant, I. (n.d.) · Amtliche Mitteilungen (Universitätsbibliothek Paderborn) · Cited by 414
A Companion to Linguistic Anthropology
(2005) · A Companion to Linguistic Anthropology · doi:10.1111/b.9781405144308.2005.00015.x · Cited by 408
Flesh and Machines: How Robots Will Change Us
Brooks, R. A. (2002) · DigitalGeorgetown (Georgetown University Library) · Cited by 395
Recent Work on Moral Responsibility
Fischer, J. M. (1999) · Ethics · doi:10.1086/233206 · Cited by 395
Robot rights? Towards a social-relational justification of moral consideration
Coeckelbergh, M. (2010) · Ethics and Information Technology · doi:10.1007/s10676-010-9235-5 · Cited by 391
Of, for, and by the people: the legal lacuna of synthetic persons
Bryson, J. J., Diamantis, M. E., & Grant, T. D. (2017) · Artificial Intelligence and Law · doi:10.1007/s10506-017-9214-9 · Cited by 368
Self-citationWhat traits are carried on mobile genetic elements, and why?
Rankin, D. J., Rocha, E. P. C., & Brown, S. P. (2011) · Heredity · doi:10.1038/hdy.2010.24 · Cited by 346
Standardizing Ethical Design for Artificial Intelligence and Autonomous Systems
Bryson, J., & Winfield, A. (2017) · Computer · doi:10.1109/mc.2017.154 · Cited by 293
Self-citationComputer systems: Moral entities but not moral agents
Johnson, D. G. (2006) · Ethics and Information Technology · doi:10.1007/s10676-006-9111-5 · Cited by 277
Winston, P. H. (1977) · Cited by 265
The machine question: critical perspectives on AI, robots, and ethics
(2013) · Choice Reviews Online · doi:10.5860/choice.50-4929 · Cited by 240
Social Responsibility and Ethics: Clarifying the Concepts
Fischer, J. (2004) · Journal of Business Ethics · doi:10.1007/s10551-004-2545-y · Cited by 234
Costs of deception: cheaters are punished in rhesus monkeys (Macaca mulatta).
Hauser, M. D. (1992) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.89.24.12137 · Cited by 221
Interpretable classification models for recidivism prediction
Zeng, J., Ustun, B., & Rudin, C. (2017) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/rssa.12227 · Cited by 220
The crying shame of robot nannies: An ethical appraisal
Sharkey, N., & Sharkey, A. (2010) · Interaction Studies. Social Behaviour and Communication in Biological and Artificial Systems · doi:10.1075/is.11.2.01sha · Cited by 200
Action Selection Methods using Reinforcement Learning
Humphrys, M. (1996) · From Animals to Animats 4 · doi:10.7551/mitpress/3118.003.0018 · Cited by 195
de Waal, F. (2007) · Johns Hopkins University Press eBooks · doi:10.56021/9780801886560 · Cited by 194
Secreting and Sensing the Same Molecule Allows Cells to Achieve Versatile Social Behaviors
Youk, H., & Lim, W. A. (2014) · Science · doi:10.1126/science.1242782 · Cited by 186
Mind the gap: responsible robotics and the problem of responsibility
Gunkel, D. J. (2020) · Ethics and Information Technology · doi:10.1007/s10676-017-9428-2 · Cited by 174
Reproductive Social Behavior: Cooperative Games to Replace Sexual Selection
Roughgarden, J., Oishi, M., & Akçay, E. (2006) · Science · doi:10.1126/science.1110105 · Cited by 160
Does ‘Ought’ Imply ‘Can’? And Did Kant Think It Does?
Stern, R. (2004) · Utilitas · doi:10.1017/s0953820803001055 · Cited by 159
Designing Interactions for Robot Active Learners
Çakmak, M., Chao, C., & Thomaz, A. L. (2010) · IEEE Transactions on Autonomous Mental Development · doi:10.1109/tamd.2010.2051030 · Cited by 159
Designing and implementing transparency for real time inspection of autonomous robots
Theodorou, A. A., Wortham, R. H., & Bryson, J. J. (2017) · Connection Science · doi:10.1080/09540091.2017.1310182 · Cited by 154
Self-citationThe crying shame of robot nannies: An ethical appraisal
SHARKEY, N. E., & SHARKEY, A. J. C. (2010) · PhilPapers (PhilPapers Foundation) · Cited by 146
Robot transparency, trust and utility
Wortham, R. H., & Theodorou, A. A. (2017) · Connection Science · doi:10.1080/09540091.2017.1313816 · Cited by 122
A Challenge for Machine Ethics
Tonkens, R. (2009) · Minds and Machines · doi:10.1007/s11023-009-9159-1 · Cited by 120
A Mixture of “Cheats” and “Co-Operators” Can Enable Maximal Group Benefit
MacLean, R. C., Fuentes-Hernández, A., Greig, D., Hurst, L. D., & Gudelj, I. (2010) · PLoS Biology · doi:10.1371/journal.pbio.1000486 · Cited by 119
A Vindication of the Rights of Machines
Gunkel, D. J. (2013) · Philosophy & Technology · doi:10.1007/s13347-013-0121-z · Cited by 118
Computer systems: Moral entities but not moral agents
Johnson, D. G. (2020) · Routledge eBooks · doi:10.4324/9781003075011-6 · Cited by 112
THE CONCURRENT EVOLUTION OF COOPERATION AND THE POPULATION STRUCTURES THAT SUPPORT IT
Powers, S. T., Penn, A. S., & Watson, R. A. R. (2011) · Evolution · doi:10.1111/j.1558-5646.2011.01250.x · Cited by 77
Intelligence in the flesh: why your mind needs your body much more than it thinks
(2016) · Choice Reviews Online · doi:10.5860/choice.194455 · Cited by 77
Ullán, A. M., Belver, M. H., Fernández, E., Lorente, F., Badía, M., & Fernández, B. (2012) · Pain Management Nursing · doi:10.1016/j.pmn.2012.10.004 · Cited by 74
Inclusive fitness: 50 years on
Gardner, A., & West, S. A. (2014) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2013.0356 · Cited by 62
Coman, A., Stone, C. B., Castano, E., & Hirst, W. C. (2014) · Psychological Science · doi:10.1177/0956797614531024 · Cited by 60
There is no fitness but fitness, and the lineage is its bearer
Akçay, E., & Cleve, J. V. (2016) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2015.0085 · Cited by 59
THE POPULAR APPEAL OF APOCALYPTIC AI
Geraci, R. M. (2010) · Zygon® · doi:10.1111/j.1467-9744.2010.01146.x · Cited by 55
Using Stories to Teach Human Values to Artificial Agents
Riedl, M. O., & Harrison, B. E. (2016) · National Conference on Artificial Intelligence · Cited by 52
Just an Artifact: Why Machines are Perceived as Moral Agents ∗
Bryson, J. J., & Kime, P. P. (2013) · Cited by 50
Self-citationPrescott, T. J. (2017) · Connection Science · doi:10.1080/09540091.2017.1279125 · Cited by 50
Granting Automata Human Rights: Challenge to a Basis of Full-Rights Privilege
Miller, L. F. (2015) · Human Rights Review · doi:10.1007/s12142-015-0387-x · Cited by 47
Female Economic Dependence and the Morality of Promiscuity
Price, M. E., Pound, N., & Scott, I. M. (2014) · Archives of Sexual Behavior · doi:10.1007/s10508-014-0320-4 · Cited by 46
DECONSTRUCTING CORPORATE SOCIAL RESPONSIBILITY: INSIGHTS FROM LEGAL AND ECONOMIC THEORY
Ostas, D. T. (2001) · American Business Law Journal · doi:10.1111/j.1744-1714.2001.tb00290.x · Cited by 43
Introduction to the Special Issue on Machine Morality: The Machine as Moral Agent and Patient
Gunkel, D. J., & Bryson, J. J. (2014) · Philosophy & Technology · doi:10.1007/s13347-014-0151-1 · Cited by 35
Self-citationA ROLE FOR CONSCIOUSNESS IN ACTION SELECTION
Bryson, J. J. (2012) · International Journal of Machine Consciousness · doi:10.1142/s1793843012400276 · Cited by 31
Self-citationBelief in The Singularity is Fideistic
Bringsjord, S., Bringsjord, A., & Bello, P. (2012) · The frontiers collection · doi:10.1007/978-3-642-32560-1_19 · Cited by 31
A Pragmatic Evaluation of the Theory of Information Ethics
Siponen, M. T. (2004) · Ethics and Information Technology · doi:10.1007/s10676-005-6710-5 · Cited by 30
Artificial Intelligence and Pro-Social Behaviour
Bryson, J. J. (2015) · Springer eBooks · doi:10.1007/978-3-319-15515-9_15 · Cited by 29
Self-citationThe meaning of the EPSRC principles of robotics
Bryson, J. J. (2017) · Connection Science · doi:10.1080/09540091.2017.1313817 · Cited by 26
Self-citationParthemore, J., & Whitby, B. (2014) · International Journal of Machine Consciousness · doi:10.1142/s1793843014400162 · Cited by 24
Bryson, J. J. (2008) · Mind & Society · doi:10.1007/s11299-007-0044-4 · Cited by 22
Self-citationIntelligence in the flesh: why your mind needs your body much more than it thinks
Mullan, K. J. (2017) · Body Movement and Dance in Psychotherapy · doi:10.1080/17432979.2017.1365095 · Cited by 22
TIME, CONSCIOUSNESS, AND MIND UPLOADING
Choe, Y., Kwon, J., & Chung, J. R. (2012) · International Journal of Machine Consciousness · doi:10.1142/s179384301240015x · Cited by 19
Non-genetic transmission of information: Candidate cognitive processes and the evolution of culture
Plotkin, H. C. (1995) · Behavioural Processes · doi:10.1016/0376-6357(95)00056-9 · Cited by 17
Rohlfshagen, P., & Bryson, J. (2010) · Cognitive Computation · doi:10.1007/s12559-010-9057-0 · Cited by 16
Self-citationA Proposal for the Humanoid Agent-builders League (HAL)
Bryson, J. J. (2000) · Cited by 12
Self-citationAgency and Patiency: Back to Nature?
Karlsson, M. M. (2002) · Philosophical Explorations · doi:10.1080/10002002018538722 · Cited by 12
The spiritual in artificial life: Recombining science and religion in a computational culture medium
Helmreich, S. (1997) · Science as Culture · doi:10.1080/09505439709526474 · Cited by 11
Understanding and Addressing Cultural Variation in Costly Antisocial Punishment
Bryson, J. J., Mitchell, J., Powers, S. T., & Sylwester, K. (2014) · Springer eBooks · doi:10.1007/978-1-4939-0280-4_9 · Cited by 8
Self-citationBehind the mask: machine morality
Miller, K., Wolf, M. J., & Grodzinsky, F. S. (2014) · Journal of Experimental & Theoretical Artificial Intelligence · doi:10.1080/0952813x.2014.948315 · Cited by 7
Cited by
In AI We Trust: Ethics, Artificial Intelligence, and Reliability
Ryan, M. (2020) · Science and Engineering Ethics · doi:10.1007/s11948-020-00228-y · Cited by 600
“…inappropriately elevate AI, while disavowing the responsibility of those developing and implementing it: ‘Assigning responsibility to the artefact for actions we designed it to execute would be to deliberately disavow our responsibility for that design’ (Bryson 2018b, p. 21; see also Andras et al.”
BackgroundMethodAli, O., Abdelbaki, W., Shrestha, A., Elbasi, E., Alryalat, M. A. A., & Dwivedi, Y. K. (2023) · Journal of Innovation & Knowledge · doi:10.1016/j.jik.2023.100333 · Cited by 568
Artificial intelligence and knowledge sharing: Contributing factors to organizational performance
Olan, F., Ogiemwonyi Arakpogun, E., Suklan, J., Nakpodia, F., Damij, N., & Jayawickrama, U. (2022) · Journal of Business Research · doi:10.1016/j.jbusres.2022.03.008 · Cited by 542
Artificial Intelligence Regulation: A framework for governance
de Almeida, P. G. R., dos Santos, C. D., & Farias, J. S. (2021) · Ethics and Information Technology · doi:10.1007/s10676-021-09593-z · Cited by 395
Hu, Q., Lu, Y., Pan, Z., Gong, Y., & Yang, Z. (2021) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2020.102250 · Cited by 307
Metadata
- Authors
- Bryson, J. J.
- Venue
- Ethics and Information Technology
- Published
- 2018-02-16
- Volume
- 20
- Issue
- 1
- Pages
- 15-26
- ISSN
- 1388-1957 (ISSN-L)
- 1572-8439
- OpenAlex
- W2786522954
- Cited by
- 285
- References
- 84
- FWCI
- 35.62
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.