CoChat
Sign Up
Cited by 255Ethics and Social Impacts of AIEthics in Business and EducationExplainable Artificial Intelligence (XAI)

Embedding Values in Artificial Intelligence (AI) Systems

Open access
van de Poel, Ibo·2020·Minds and Machines
DOI 10.1007/s11023-020-09537-4
Open this paper in CoChat — free accountOpen PDF

Abstract

Organizations such as the EU High-Level Expert Group on AI and the IEEE have recently formulated ethical principles and (moral) values that should be adhered to in the design and deployment of artificial intelligence (AI). These include respect for autonomy, non-maleficence, fairness, transparency, explainability, and accountability. But how can we ensure and verify that an AI system actually respects these values? To help answer this question, I propose an account for determining when an AI system can be said to embody certain values. This account understands embodied values as the result of design activities intended to embed those values in such systems. AI systems are here understood as a special kind of sociotechnical system that, like traditional sociotechnical systems, are composed of technical artifacts, human agents, and institutions but—in addition—contain artificial agents and certain technical norms that regulate interactions between artificial agents and other elements of the system. The specific challenges and opportunities of embedding values in AI systems are discussed, and some lessons for better embedding values in AI systems are drawn.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1007/s11023-020-09537-4

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • AI systems contain a particularly high degree of uncertainty and unpredictability.

    “As a result, AI systems contain a particularly high degree of uncertainty and unpredictability.”

    From Contestable AI by Design: Towards a Framework (2022) · cites this paper as van de Poel, 2020

    The passage labeled “1 3” (page unavailable) says unintended consequences are “probably more endemic” in AI systems because of their learning capabilities. The Conclusion likewise discusses unintended and unforeseen consequences. These passages support heightened unpredictability, but they hedge the comparison and do not establish the claim’s unqualified “particularly high degree” of uncertainty in general. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 106 · Cited by 255

Research works printed but not linked: 73

  1. Akrich, M. (1992). The Description of Technical Objects. In W. Bijker & J. Law (Eds.), Shaping technol- ogy/building society: Studies in sociotechnical change (pp. 205-224). Cambridge: MIT Press.
  2. Aldewereld, H., & Sichman, J. S. (2013). Coordination, organizations, institutions, and norms in agent systems VIII: 14th International Workshop, COIN 2012 Lecture notes in artificial intelligence (Vol. 7756). New York: Springer.
  3. Anderson, E. (1993). Value in ethics and economics. Cambridge: Harvard University Press.
  4. Anderson, M., & Anderson, S. L. (2011). Machine ethics. New York: Cambridge University Press.
  5. Bauer, J. M., & Herder, P. M. (2009). Designing Socio-Technical Systems. In A. Meijers (Ed.), Philoso- phy of Technology and Engineering Sciences (pp. 601-630). Amsterdam: Elsevier.
  6. Baxter, G., & Sommerville, I. (2011). Socio-technical systems: From design methods to systems engi- neering. Interacting with Computers, 23(1), 4-17. https ://doi.org/10.1016/j.intco m.2010.07.003. · DOI · Paper page
  7. Becker, H. S. (1963). Outsiders. Studies in the Sociology of Deviance. New York: The Free Press of Glencoe.
  8. Behymer, K. J., & Flach, J. M. (2016). From Autonomous Systems to Sociotechnical Systems: Designing Effective Collaborations. She Ji: The Journal of Design, Economics, and Innovation, 2(2), 105-114. https ://doi.org/10.1016/j.sheji .2016.09.001. · DOI
  9. Bicchieri, C. (2006). The grammar of society: the nature and dynamics of social norms. New York: Cam- bridge University Press.
  10. Boddington, P. (2017). Towards a code of ethics for artificial intelligence research. New York: Springer.
  11. Boissier, O. (2006). Coordination, organizations, institutions, and norms in multi-agent systems: AAMAS 2005 International Workshops on Agents, Norms and Institutions for Regulated Multi-Agent Sys- tems, ANIREM 2005, and From Organizations to Organization-Oriented Programming in Multi- Agent Systems, OOOP 2005 (Lecture notes in computer science,, Vol. 3913). Berlin; New York: Springer.
  12. Borenstein, J., Herkert, J. R., & Miller, K. W. (2019). Self-driving cars and engineering ethics: The need for a system level analysis. Science and Engineering Ethics, 25(2), 383-398. https ://doi. org/10.1007/s1194 8-017-0006-0. · DOI
  13. Bowker, G. C., Baker, K., Millerand, F., & Ribes, D. (2010). Toward information infrastructure studies: Ways of knowing in a networked environment. In J. Hunsinger, L. Klastrup, & M. Allen (Eds.), International handbook of internet research (pp. 97-117). Dordrecht: Springer.
  14. Brosch, T., Sander, D., Clément, F., Deonna, J. A., Fehr, E., & Vuilleumier, P. (2016). Handbook of value: perspectives from economics, neuroscience, philosophy, psychology and sociology. Oxford: Oxford University Press.
  15. Bruijn, H., & Herder, P. M. (2009). System and actor perspectives on sociotechnical systems. IEEE Transactions on Systems, Man, and Cybernetics -Part A: Systems and Humans, 39(5), 981-992. https ://doi.org/10.1109/TSMCA .2009.20254 · DOI
  16. Calvert, R. L. (1995). The rational choice theory of social institutions: cooperation, coordination, and communication. In E. A. Hanushek & J. S. Banks (Eds.), Modern political economy: Old topics, new directions (Political economy of institutions and decisions) (pp. 216-268). Cambridge: Cam- bridge University Press.
  17. Cave, S., Nyrup, R., Vold, K., & Weller, A. (2019). Motivations and risks of machine ethics. Proceedings of the IEEE, 107(3), 562-574. https ://doi.org/10.1109/JPROC .2018.28659 · DOI
  18. Coeckelbergh, M. (2020). AI ethics. Cambridge: The MIT Press.
  19. Dam, K. H., Nikolic, I., & Lukszo, Z. (2013). Agent-based modelling of socio-technical systems (Vol. 9). New York: Springer.
  20. Dennett, D. C. (1987). The intentional stance. Cambridge: MIT Press. 1
  21. Dybalova, D., Testerink, B., Dastani, M., & Logan, B. A Framework for Programming Norm-Aware Multi-agent Systems. In Cham, 2014 (pp. 364-380, Coordination, Organizations, Institutions, and Norms in Agent Systems IX): Springer International Publishing
  22. Flanagan, M., Howe, D. C., & Nissenbaum, H. (2008). Embodying values in technology. Theory and practise. In J. Van den Hoven & J. Weckert (Eds.), Information technology and moral philosophy (pp. 322-353). Cambridge: Cambridge University Press.
  23. Floridi, L., & Sanders, J. W. (2004). On the morality of artificial agents. Minds and Machines, 14(3), 349-379.
  24. Fogg, B. J. (2003). Persuasive technology: using computers to change what we think and do (The Morgan Kaufmann series in interactive technologies). Amsterdam: Morgan Kaufmann Publishers.
  25. Franssen, M. (2015). Design for values and operator roles in sociotechnical systems. In J. van den Hoven, P. E. Vermaas, & I. van de Poel (Eds.), Handbook of ethics, values, and technological design: sources, theory, values and application domains (pp. 117-149). Dordrecht: Springer.
  26. Franssen, M. Modelling Systems in Technology as Instrumental Systems. In Berlin, Heidelberg, 2014 (pp. 543-562, Model-Based Reasoning in Science and Technology): Springer Berlin Heidelberg
  27. Ghorbani, A., Aldewereld, H., Dignum, V., & Noriega, P. Shared Strategies in Artificial Agent Soci- eties. In Berlin, Heidelberg, 2013 (pp. 71-86, Coordination, Organizations, Institutions, and Norms in Agent Systems VIII): Springer Berlin Heidelberg
  28. Grodzinsky, F. S., Miller, K. W., & Wolf, M. J. (2008). The ethics of designing artificial agents. Ethics and Information Technology, 10(2), 115-121. https ://doi.org/10.1007/s1067 6-008-9163-9. · DOI
  29. Hirose, I., & Olson, J. (2015). The Oxford handbook of value theory. New York: Oxford University Press.
  30. Ho, P. (2016). Empty institutions, non-credibility and pastoralism: China's grazing ban, mining and ethnicity. The Journal of Peasant Studies, 43(6), 1145-1176. https ://doi.org/10.1080/03066 150.2016.12396 · DOI
  31. Hollander, C. D., & Wu, A. S. (2011). The current state of normative agent-based systems. Journal of Artificial Societies and Social Simulation, 14(2), 6. https ://doi.org/10.18564 /jasss .1750. · DOI
  32. Houkes, W., & Vermaas, P. E. (2010). Technical functions: on the use and design of artifactartifacts (Philosophy of engineering and technology (Vol. 1). Dordrecht: Springer.
  33. IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems (2019). Ethically Aligned Design: A Vision for Prioritizing Human Well-being with Autonomous and Intelligent Systems. (1st ed.): IEEE.
  34. Ihde, D. (2012). Experimental phenomenology: multistabilities (2nd ed.). Albany: State University of New York Press.
  35. Illies, C., & Meijers, A. (2009). Artifact artifacts without agency. The Monist, 92(3), 420-440.
  36. Jacobson, D. (2011). Fitting Attitude Theories of Value. In E. N. Zalta (Ed.), The Stanford Encyclope- dia of Philosophy (Spring 2011 Edition).
  37. Johnson, D. G., & Miller, K. W. (2008). Un-making artificial moral agents. Ethics and Information Technology, 10(2), 123-133. https ://doi.org/10.1007/s1067 6-008-9174-6. · DOI
  38. Jones, A. J. I., Artikis, A., & Pitt, J. (2013). The design of intelligent socio-technical systems. Artifi- cial Intelligence Review, 39(1), 5-20. https ://doi.org/10.1007/s1046 2-012-9387-2. · DOI
  39. Klenk, M. (2020). How do technological artifact artifacts embody moral values? Philosophy & Tech- nology. https ://doi.org/10.1007/s1334 7-020-00401 -y. · DOI
  40. Kroes, P. (2010). Engineering and the dual nature of technical artifactartifacts. Cambridge Journal of Economics, 34(1), 51-62. https ://doi.org/10.1093/cje/bep01 9. · DOI
  41. Kroes, P., Franssen, M., Van de Poel, I., & Ottens, M. (2006). Treating socio-technical systems as engineering systems: some conceptual problems. Systems Research and Behavioral Science, 23(6), 803-814. https ://doi.org/10.1002/sres.703. · DOI
  42. Kroes, P., & Meijers, A. (2006). The dual nature of technical artifactartifacts. Studies In History and Philosophy of Science Part A, 37(1), 1-4.
  43. Kroes, P., & Verbeek, P.-P. (Eds.). (2014). The moral status of technical artifactartifacts. Dordrecht: Springer.
  44. Latour, B. (1992). Where are the missing masses? In W. Bijker & J. Law (Eds.), Shaping Technology/ Building Society; Studies in Sociotechnical change (pp. 225-258). Cambridge: MIT Press.
  45. Leenes, R., & Lucivero, F. (2014). Laws on Robots, Laws by Robots, Laws in Robots: Regulat- ing Robot Behaviour by Design. Law, Innovation and Technology, 6(2), 193-220. https ://doi. org/10.5235/17579 961.6.2.193. · DOI
  46. Mahmoud, M. A., Ahmad, M. S., Mohd Yusoff, M. Z., & Mustapha, A. (2014). A review of norms and normative multiagent systems. The Scientific World Journal, 2014, 684587. https ://doi. org/10.1155/2014/68458 · DOI
  47. Miller, B. (2020). Is Technology Value-Neutral? Science, Technology and Human Values. https ://doi. org/10.1177/01622 43919 90096 · DOI
  48. Moor, J. H. (2006). The nature, importance, and difficulty of machine ethics. IEEE Intelligent Systems, 21(4), 18-21. https ://doi.org/10.1109/MIS.2006. · DOI
  49. Müller, V. C. (2020). Ethics of Artificial Intelligence and Robotics. In E. N. Zalta (Ed.), The Stanford Encyclopedia of Philosophy (Fall 2020 Edition). · DOI
  50. Nickel, P. J. (2013). Trust in Technological Systems. In M. J. de Vries, S. O. Hansson, & A. W. M. Mei- jers (Eds.), Norms in technology (pp. 223-237). Dordrecht: Springer.
  51. Norman, D. A. (2000). The design of everyday things. Cambridg: MIT Press.
  52. North, D. C. (1990). Institutions, institutional change and economic performance. Cambridge: Cam- bridge University Press.
  53. Ostrom, E. (2005). Understanding institutional diversity (Princeton paperbacks). Princeton: Princeton University Press.
  54. Ostrom, E., Gardner, R., & Walker, J. (1994). Rules, games, and common-pool resources. Ann Arbor: University of Michigan Press.
  55. Ottens, M., Franssen, M., Kroes, P., & Van de Poel, I. (2006). Modeling engineering systems as socio- technical systems. International Journal of Critical Infrastructures, 2, 133-145.
  56. Panagiotidi, S., Vázquez-Salceda, J., & Dignum, F. Reasoning over Norm Compliance via Planning. In Berlin, Heidelberg, 2013 (pp. 35-52, Coordination, Organizations, Institutions, and Norms in Agent Systems VIII): Springer Berlin Heidelberg
  57. Peterson, M., & Spahn, A. (2011). Can technological artifactArtifacts be moral agents? Science and Engi- neering Ethics, 17(3), 411-424.
  58. Pitt, J. C. (2014). Guns don't kill, people kill"; values in and/or around technologies. In P. Kroes & P.-P. Verbeek (Eds.), The moral status of technical artifactartifacts (pp. 89-101). Dordrecht: Springer.
  59. Raz, J. (1999). Engaging reason. On the theory of value and action. Oxford: Oxford University Press.
  60. Scanlon, T. M. (1998). What we owe to each other. Cambridge: Harvard University Press.
  61. Searle, J. R. (1984). Minds, brains, and science (The 1984 Reith lectures). Cambridge: Harvard Univer- sity Press.
  62. Singh, M. P. (2014). Norms as a basis for governing sociotechnical systems. ACM Trans. Intell. Syst. Technol., 5(1), 1-23. https ://doi.org/10.1145/25421 82.25422 · DOI
  63. Stevenson, C. L. (1944). Ethics and language. New Haven: Yale University Press. · DOI
  64. Sullins, J. P. (2006). When is a robot a moral agent. International Review of Information Ethics, 6(12), 23-30.
  65. Thaler, R. H., & Sunstein, C. R. (2009). Nudge: improving decisions about health, wealth, and happiness (Rev. and expanded ed.. New York: Penguin Books.
  66. Ullmann-Margalit, E. (1977). The emergence of norms (Clarendon library of logic and philosophy). Oxford Eng: Clarendon Press.
  67. Van de Poel, I., & Kroes, P. (2014). Can technology embody values? In P. Kroes & P.-P. Verbeek (Eds.), The moral status of technical artifacts (pp. 103-124). Dordrecht: Springer.
  68. Vanderelst, D., & Winfield, A. (2018). The Dark Side of Ethical Robots. Paper presented at the Proceed- ings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, New Orleans, LA, USA,
  69. Verbeek, P.-P. (2011). Moralizing technology: understanding and designing the morality of things. Chi- cago, London: The University of Chicago Press.
  70. Vermaas, P. E., & Houkes, W. (2006). Technical functions: a drawbridge between the intentional and structural natures of technical artifacts. Studies In History and Philosophy of Science Part A, 37(1), 5-18.
  71. Winfield, A. F. (2019). Machine ethics: The design and governance of ethical AI and autonomous sys- tems [scanning the issue]. Proceedings of the IEEE, 107(3), 509-517.
  72. Winner, L. (1977). Autonomous Technology. Technics-out-of-Control as a Theme in Political Thought. Cambridge: MIT Press.
  73. Zimmerman, M. J. (2015). Value and Normativity. In I. Hirose & J. Olson (Eds.), The Oxford handbook of value theory. Oxford: Oxford University Press.

Authors
van de Poel, I.
ORCID
van de Poel, Ibo
Venue
Minds and Machines
Publisher
Springer Science and Business Media LLC
Published
2020-09
Volume
30
Issue
3
Pages
385-409
ISSN
  • 0924-6495 (ISSN-L)
  • 1572-8641
DOI
10.1007/s11023-020-09537-4
OpenAlex
W3081716527
Cited by
255
References
106
Citations by year
2026: 48 · 2025: 48 · 2024: 65 · 2023: 36 · 2022: 39 · 2021: 18 · 2020: 1
FWCI
32.01
Type
journal article
Language
en
Open access
Licence
Locations
  • Minds and Machines (publishedVersion, cc-by)
  • Research Repository (Delft University of Technology) (submittedVersion, cc-by)
  • Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
Funders
H2020 European Research Council (788321)
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Social Sciences › Decision Sciences › Information Systems and Management › Ethics in Business and Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
Keywords
AI ethics, embodied values, AI design, socio-technical systems, artificial agents, technical standards, autonomy, non-maleficence, fairness, transparency, explainability, accountability

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review