The DOI matches its Crossref record; 57 of the 57 references it cites were found in a registry or an index; 4 of the 21 sources its claims cite were checked for support.
Instrumental Robots.
Open accessAbstract
Advances in artificial intelligence research allow us to build fairly sophisticated agents: robots and computer programs capable of acting and deciding on their own (in some sense). These systems raise questions about who is responsible when something goes wrong-when such systems harm or kill humans. In a recent paper, Sven Nyholm has suggested that, because current AI will likely possess what we might call "supervised agency", the theory of responsibility for individual agency is the wrong place to look for an answer to the question of responsibility. Instead, or so argues Nyholm, because supervised agency is a form of collaborative agency-of acting together-the right place to look is the theory of collaborative responsibility-responsibility in cases of acting together. This paper concedes that current AI will possess supervised agency, but argues that it is nevertheless wrong to think of the relevant human-AI interactions as a form of collaborative agency and, hence, that responsibility in cases of collaborative agency is not the right place to look for the responsibility-grounding relation in human-AI interactions. It also suggests that the right place to look for this responsibility-grounding relation in human-AI interactions is the use of certain sorts of agents as instruments.
Overview
CoChat Summary
## TL;DR
The paper argues that even if current AI has “supervised agency,” human–AI interactions are not best understood as collaborative agency. Instead, responsibility in cases like autonomous cars or weapons should be grounded in the familiar relation of using the AI as an instrument.
## Key findings
- The paper concedes Nyholm’s premise that current AI plausibly has supervised agency, but rejects his conclusion that this makes human–AI interaction a case of collaborative agency.
- It argues that many standard accounts of joint/shared agency already rule out current AI as collaborators; for example, on Gilbert’s account, shared intention requires normative commitments, and on Searle’s account it requires “we-intentions,” both of which current AI cannot have.
- Using Bratman’s framework, the paper says supervised agency is still not enough for genuine “doing something together,” because cases like Driver and Company do not require the human and AI to share an intention to drive together.
- It proposes an alternative: responsibility should be explained by the relation of using minimal agents as instruments, with analogies to dogs, horses, and other non-human animals used instrumentally.
## Methods
- Conceptual/philosophical argument, centered on evaluating Nyholm’s account of responsibility in human–AI interactions.
- Uses comparative case analysis, especially Driver, Company, and Adult/Child, plus examples of autonomous vehicles, autonomous weapons, AlphaGoZero, and animal instruments.
- Main analysis is normative and theoretical: it tests whether supervised agency satisfies conditions for shared/joint agency, then contrasts that with responsibility norms for instrument use.
## Limitations
- The paper states that it does not offer “a fully fleshed out account” of the relevant norms governing responsibility for AI used as instruments, and says these require further interdisciplinary research.
- It explicitly takes no stance on what capacities are required for moral agency and assumes, rather than argues, that current AI are not moral agents.
- Evident from the text: the argument is largely philosophical and illustrative, so it does not provide empirical evidence about real-world AI systems or responsibility practices.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
3 Valid · 1 Not valid · 10 Not checked · 32 Own results and interpretations
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“However, this objection can be dealt with within the framework on offer, by drawing on the kinds of considerations that are relevant when responsibility is grounded in instrument use.”
What the sources say
- Could not verifycites [21] sourceExperimental — accuracy not yet measured
Section 8, “Implications of the Account” (page unavailable), discusses obligations arising from joint commitments, not responsibility grounded in instrument use. Searches for instrument use, AI agency, and foreseeability found no passage addressing the attributed objection or its proposed response. (LLM, conf=0.75)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
None of the sources cited for this claim have been checked yet.
In the paper
“It is common ground amongst all accounts of shared agency that to genuinely do something together, the agents involved need a shared intention: they need to intend that they ϕ.”
§ Instrumental Robots > Human and AI: Not Working Together
What the sources say
- Not matchedcites Bratman 2013
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not checked — no full text of the sourcecites Gilbert 1996
- Not matchedcites Searle 1995
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
None of the sources cited for this claim have been checked yet.
In the paper
“However, different accounts disagree on what it takes to have such a shared intention (see e.g. Bratman 2013; Gilbert 1996; Searle 1995; or Toumela 2007).”
§ Instrumental Robots > Human and AI: Not Working Together
What the sources say
- Not matchedcites Bratman 2013
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not checked — no full text of the sourcecites Gilbert 1996
- Not matchedcites Searle 1995
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“there would be no in principle difference for questions about responsibility between AI and other supervised minimal agents, such as non-human animals, that are being used as instruments”
§ Instrumental Robots > Introduction Advances in artificial intelligence research—mostly advances in machine learning techniques combined with increased computing power—, allow us to build and employ fairly sophisticated automated systems: robots and computer programs like Google cars, military drones, or AlphaGoZero. It is natural to describe these systems—call them “current AI”—as agents who are capable of acting and deciding on their own, i.e. who are to some relevant degree autonomous (in some sense).1 One important question these systems raise is who is morally responsible when such AI perform actions with harmful outcomes (see e.g. Coeckelbergh 2016; Danaher 2016; Gunkel 2017; Hellström 2013; Hevelke and Nida-Rümelin 2015; Himmelreich 2019; Matthias 2004; Purves et al. 2015; Robillard 2018; Roff 2013; or Sparrow 2007).2 For those who think that current AI cannot themselves be responsible, this raises a challenge: one must identify a relation in which agents that are capable of being responsible stand to the outcomes of the AI’s action such that those agents are responsible for those outcomes. In a recent paper, Sven Nyholm (2018a) has argued that, because current AI will possess what one might call “supervised agency,” one should not consult the theory of responsibility for individual agency to find such a relation. Instead, because supervised agency is a form of collaborative agency—of acting together—one should consult the theory of collaborative responsibility—responsibility in cases of acting together. This paper concedes that AI possess supervised agency, but argues that the relevant human-AI interactions are nevertheless not forms of collaborative agency and, hence, that the responsibility grounding relation in human-AI interactions is not to be found in the theory of collaborative responsibility. It also argues that if AI possess supervised agency, the place to find the relation that grounds responsibility in human-AI interactions is simply responsibility for the use of instruments. This is so, because if AI are supervised agents, this shows that the agency of the AI is not of any kind that would disallow us from subsuming responsibility in human-AI interactions under this explanation. Rather, there would be no in principle difference for questions about responsibility between AI and other supervised minimal agents, such as non-human animals, that are being used as instruments—for whom we know that responsibility explanations appealing to this relation are fitting. The paper proceeds as follows: The first section (“Current AI, Agency, Responsibility Gaps”) states some of the assumptions that will be made in this paper and offers a more detailed presentation of the challenge current AI pose for questions about responsibility. The second section (“Nyholm’s Account: Human-AI interactions as Collaborative Agency”) presents in greater detail Nyholm’s argument for the thesis that to ground responsibility in human-AI interactions we should look at the theory of collaborative responsibility. The third section (“Human and AI: Not Working Together”) argues that supervised agency is not sufficient for collaborative agency and that, indeed, current AI can be expected not to possess the capacities required for collaborative agency, even if they possess supervised agency. Hence, Nyholm’s thesis that the theory of collaborative responsibility provides an account of the relation relevant for responsibility attributions in human-AI interactions, is false. The last section (“Instrumental Robots”) suggests an alternative of how responsibility can be grounded, if AI possess supervised agency. It argues that in this case, responsibility in human/AI interactions can just be subsumed under responsibility in using something an instrument.
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Hence, Nyholm’s thesis that the theory of collaborative responsibility provides an account of the relation relevant for responsibility attributions in human-AI interactions, is false.”
§ Instrumental Robots > Introduction Advances in artificial intelligence research—mostly advances in machine learning techniques combined with increased computing power—, allow us to build and employ fairly sophisticated automated systems: robots and computer programs like Google cars, military drones, or AlphaGoZero. It is natural to describe these systems—call them “current AI”—as agents who are capable of acting and deciding on their own, i.e. who are to some relevant degree autonomous (in some sense).1 One important question these systems raise is who is morally responsible when such AI perform actions with harmful outcomes (see e.g. Coeckelbergh 2016; Danaher 2016; Gunkel 2017; Hellström 2013; Hevelke and Nida-Rümelin 2015; Himmelreich 2019; Matthias 2004; Purves et al. 2015; Robillard 2018; Roff 2013; or Sparrow 2007).2 For those who think that current AI cannot themselves be responsible, this raises a challenge: one must identify a relation in which agents that are capable of being responsible stand to the outcomes of the AI’s action such that those agents are responsible for those outcomes. In a recent paper, Sven Nyholm (2018a) has argued that, because current AI will possess what one might call “supervised agency,” one should not consult the theory of responsibility for individual agency to find such a relation. Instead, because supervised agency is a form of collaborative agency—of acting together—one should consult the theory of collaborative responsibility—responsibility in cases of acting together. This paper concedes that AI possess supervised agency, but argues that the relevant human-AI interactions are nevertheless not forms of collaborative agency and, hence, that the responsibility grounding relation in human-AI interactions is not to be found in the theory of collaborative responsibility. It also argues that if AI possess supervised agency, the place to find the relation that grounds responsibility in human-AI interactions is simply responsibility for the use of instruments. This is so, because if AI are supervised agents, this shows that the agency of the AI is not of any kind that would disallow us from subsuming responsibility in human-AI interactions under this explanation. Rather, there would be no in principle difference for questions about responsibility between AI and other supervised minimal agents, such as non-human animals, that are being used as instruments—for whom we know that responsibility explanations appealing to this relation are fitting. The paper proceeds as follows: The first section (“Current AI, Agency, Responsibility Gaps”) states some of the assumptions that will be made in this paper and offers a more detailed presentation of the challenge current AI pose for questions about responsibility. The second section (“Nyholm’s Account: Human-AI interactions as Collaborative Agency”) presents in greater detail Nyholm’s argument for the thesis that to ground responsibility in human-AI interactions we should look at the theory of collaborative responsibility. The third section (“Human and AI: Not Working Together”) argues that supervised agency is not sufficient for collaborative agency and that, indeed, current AI can be expected not to possess the capacities required for collaborative agency, even if they possess supervised agency. Hence, Nyholm’s thesis that the theory of collaborative responsibility provides an account of the relation relevant for responsibility attributions in human-AI interactions, is false. The last section (“Instrumental Robots”) suggests an alternative of how responsibility can be grounded, if AI possess supervised agency. It argues that in this case, responsibility in human/AI interactions can just be subsumed under responsibility in using something an instrument.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
19 more claims
Create a free account to see all 46 claims and every source verdict.
- Probably not a substantial claim. 16
- It may state two separate claims as one. 10
Citations
Cites
(2016) · Cited by 11,679
The construction of social reality
(1995) · Choice Reviews Online · doi:10.5860/choice.33-0869 · Cited by 4,674
Freedom of the Will and the Concept of a Person
Frankfurt, H. G. (1971) · The Journal of Philosophy · doi:10.2307/2024717 · Cited by 4,663
Some Studies in Machine Learning Using the Game of Checkers
Samuel, A. L. (1959) · IBM Journal of Research and Development · doi:10.1147/rd.33.0210 · Cited by 4,443
(1987) · Medical Entomology and Zoology · Cited by 4,269
Strawson, P. F. (1962) · PhilPapers (PhilPapers Foundation) · Cited by 3,223
Davidson, D. (1963) · The Journal of Philosophy · doi:10.2307/2023177 · Cited by 2,938 · cited 1×
“Broadly speaking, agents are systems whose behavior can correctly be described as intentional , i.e. as appropriately connected to the system’s intentions (Davidson 1963).”
BackgroundResponsibility and Control: A Theory of Moral Responsibility
Fischer, J. M., & Ravizza, M. (1998) · doi:10.1017/cbo9780511814594 · Cited by 2,249
Psychophysical and theoretical identifications
Lewis, D. (1972) · Australasian Journal of Philosophy · doi:10.1080/00048407212341301 · Cited by 1,580 · cited 1×
“According to functionalist theories (e.g. Block 1980; Lewis 1972 or Putnam 1975; see Braddon-Mitchell and Jackson 1996 or Levin 2013 for excellent introductions into functionalist views about the mind), mental states are characterized by their causal-functional role in the system of which they are…”
List, C., & Pettit, P. (2011) · doi:10.1093/acprof:oso/9780199591565.001.0001 · Cited by 1,344
Research works printed but not linked: 5
- Burri, S. (2017). What’s the moral problem with killer robots? In R. Jenkins, M. Robillard, & B. J. Strawser (Eds.), Who should die?. Oxford: Oxford University Press.
- Calo, R. (2015). Robots and the lessons of cyberlaw. California Law Review, 103, 513–563.
- Calo, R. (2016). Robots in American Law (February 24, 2016). University of Washington School of Law Research Paper No. 2016-04. Available at SSRN: https://ssrn.com/abstract=2737598.
- Darwall, S. (2006). The second person standpoint. Cambridge, MA: Harvard University Press.
- Levin, J. (2013). Functionalism. In E. N. Zalta (Ed.), The Stanford encyclopedia of philosophy (Fall 2018 ed.). Stanford, CA: The Metaphysics Research Lab, Center for the Study of Language and Information, Stanford University.
Cited by
Artificial intelligence and responsibility gaps: what is the problem?
Königs, P. (2022) · Ethics and Information Technology · doi:10.1007/s10676-022-09643-0 · Cited by 157
“18 The observation that the negligent use of autonomous systems is blameworthy has also been eloquently made by Burri, 2018, pp. 175–177; Himmelreich, 2019; Köhler et al., 2018; Köhler, 2020, pp. 3133–3139; Robillard, 2018, p. 709; Simpson & Müller, 2016, pp. 306–307.”
BackgroundThe risks of autonomous machines: from responsibility gaps to control gaps
Hindriks, F., & Veluwenkamp, H. (2023) · Synthese · doi:10.1007/s11229-022-04001-5 · Cited by 71
“…responsibility gaps, see Coeckelbergh (2016), Danaher (2016), Gunkel (2017), Hellström (2013), Hevelke and Nida-Rümelin (2015), Himmelreich (2019), Köhler (2020), Köhler et al. (2017), Nyholm (2018a), Purves et al. (2015), Robillard (2018), Roff (2013), Santoni de Sio and Mecacci (2021), Sparrow…”
BackgroundResponsible AI Through Conceptual Engineering
Himmelreich, J., & Köhler, S. (2022) · Philosophy & Technology · doi:10.1007/s13347-022-00542-2 · Cited by 48
“Another option assumes that responsibility requires some other causal-like relation—such as supervision or collaboration— instead of control and contends that humans stand in this relation (see, e.g., Köhler, 2020; Nyholm, 2018; Robillard, 2018).”
BackgroundSelf-citationThe Responsibility Gap and LAWS: a Critical Mapping of the Debate
Oimann, A. (2023) · Philosophy & Technology · doi:10.1007/s13347-022-00602-7 · Cited by 47
“Sebastian Köhler argues that responsibility in human-AI interactions should be sought in the responsibility for the use of an instrument and treats it analogously to cases where we use and train non-human animals as instruments such as police dogs and racehorses (Köhler, 2020, 3134).”
BackgroundReasons for Meaningful Human Control
Veluwenkamp, H. (2022) · Ethics and Information Technology · doi:10.1007/s10676-022-09673-8 · Cited by 34
Metadata
- Authors
- Köhler, S.
- Affiliations
- Frankfurt School of Finance & Management (DE)
- Corresponding author
- Köhler, Sebastian
- ORCID
- Köhler, Sebastian
- Venue
- Science and engineering ethics
- Published
- 2020-08-19
- Volume
- 26
- Issue
- 6
- Pages
- 3121-3141
- ISSN
- 1353-3452 (ISSN-L)
- 1471-5546
- PMC
- PMC7755622
- OpenAlex
- W4237346123
- Cited by
- 18
- References
- 57
- Citations by year
- 2026: 1 · 2025: 3 · 2024: 5 · 2023: 2 · 2022: 6 · 2020: 1
- FWCI
- 3.06
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Science and Engineering Ethics (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Funders
- Frankfurt School of Finance & Management gGmbH
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Law › Literary and Philosophical Studies
- Social Sciences › Arts and Humanities › Philosophy › Philosophical Ethics and Theory
- Keywords
- artificial intelligence, robot agency, collaborative agency, individual agency, responsibility for harm, human-AI interaction, moral responsibility
- MeSH terms
- Artificial Intelligence, Humans, Robotics, Social Behavior, Software
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-06, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-06 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Instrumental Robots. https://app.cochat.ai/papers/10.1007/s11948-020-00259-5
- The paper
- Köhler, S. (2020). Instrumental Robots. Science and engineering ethics. https://doi.org/10.1007/s11948-020-00259-5
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.