The DOI matches its Crossref record; 151 of the 151 references it cites were found in a registry or an index; none of the 117 sources its claims cite is checked for support yet.
The Moral Psychology of Artificial Intelligence
Open accessPublished version of a preprint
Abstract
Moral psychology was shaped around three categories of agents and patients: humans, other animals, and supernatural beings. Rapid progress in artificial intelligence has introduced a fourth category for our moral psychology to deal with: intelligent machines. Machines can perform as moral agents, making decisions that affect the outcomes of human patients or solving moral dilemmas without human supervision. Machines can be perceived as moral patients, whose outcomes can be affected by human decisions, with important consequences for human–machine cooperation. Machines can be moral proxies that human agents and patients send as their delegates to moral interactions or use as a disguise in these interactions. Here we review the experimental literature on machines as moral agents, moral patients, and moral proxies, with a focus on recent findings and the open questions that they suggest.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
This review organizes moral psychology research on AI into machines as moral agents, moral patients, and moral proxies, focusing on experimental findings and open questions. It argues that people apply distinct moral expectations and reactions to machines, including a persistent “machine penalty” in cooperation and unresolved challenges for value alignment.
Key findings
- As implicit moral agents, people demand high machine performance: a representative German sample believed human experts would have a 20–30% mistake rate for credit default or recidivism but required machines to have a mistake rate below 10%; in a US driver sample, the median respondent believed they were in the top 25% of drivers and estimated two-thirds of crashes would be avoided if everyone drove like them, then required very high safety from self-driving cars.
- Blame patterns are asymmetric: people judge crashes caused by self-driving cars as more severe and place more blame on the car than on a human causing a comparable crash, but when human and machine jointly produce a crash, people typically blame the human most; algorithmic discrimination also causes less moral outrage than human discrimination.
- Cooperation shows a machine penalty in incentivized games: in a one-shot trust game, 34% reciprocated a machine’s trust versus 75% for a human; in a one-shot prisoners’ dilemma, 36% cooperated with machines versus 49% with humans; in a dictator game, people allocated 16% to a machine versus 39% to a human; in a public good game, contributions were 40% with machines versus 55% with humans. Repeated games suggest the machine penalty carries over unchanged.
- Value alignment and proxy use remain unsettled: the Moral Machine Experiment considered nine possible priorities for AVs, leading to millions of scenarios; in a survey of 160 experiments, 82% of instructions did not explain what happened to machine earnings; machine masquerade/AI-mediated communication is described as a vast but understudied field.
Methods
- Design: narrative review of experimental literature on machines as moral agents, moral patients, and moral proxies; no new empirical data are reported.
- Data/sample: synthesizes cited experiments including representative population samples, incentivized economic games, autonomous-vehicle dilemma studies, and studies of online bot interactions.
- Analysis: organizes findings by moral role and discusses open questions; no formal meta-analysis or systematic search protocol is described.
Limitations
- The paper states that any review will only be a snapshot given the pace of AI change, and that it has not addressed every issue at the intersection of AI and moral psychology, such as AI plagiarism, sexbots, or AI effects on trust between human groups.
- The paper notes that evidence is often concentrated in automated driving and it is not yet clear whether patterns such as the blame reversal generalize to other domains; it also states that value-alignment normative issues fall beyond moral psychology’s purview and that the machine masquerade section is speculative.
- The text makes evident that, as a narrative review, it does not report a systematic search or quantitative synthesis, and some cited findings come from single experiments or specific samples.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
82 Not checked yet · 15 No basis given · 6 Own interpretation
None of the sources cited for this claim have been checked yet.
In the paper
“Providing objective evidence that a machine performs better than humans is not trivial to begin with, though (Kalra & Paddock 2016, Kleinberg et al. 2018, Noy et al. 2018), and psychological biases may complicate things even further.”
§ The Moral Psychology of Artificial Intelligence > 2.1.1. Performance.
What the sources say
- Not checked yetcites Kalra & Paddock 2016
- Not checked yetcites Kleinberg et al. 2018
- Not checked yetcites Noy et al. 2018
plain- Asserted: passes — The authors put it forward as true.
- Checkable: unclear — It is unclear whether it could be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“If we wait for skin cancer detection algorithms to be perfectly accurate, we sacrifice the thousands of lives that could have been saved by an earlier detection (Esteva et al. 2017).”
§ The Moral Psychology of Artificial Intelligence > 2.1.1. Performance.
What the sources say
- Not checked yetcites Esteva et al. 2017
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“If we require self-driving cars to be perfectly safe before we allow them on the road, we sacrifice the thousands of lives they could have saved by being allowed on the road just a little sooner (Kalra & Groves 2017).”
§ The Moral Psychology of Artificial Intelligence > 2.1.1. Performance.
What the sources say
- Not checked yetcites Kalra & Groves 2017
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“First, there is ample discussion in the media about the danger that machines will learn, amplify and legitimize the biases embedded in the human decisions they are trained from (O'neil 2017).”
§ The Moral Psychology of Artificial Intelligence > 2.1.2. Bias.
What the sources say
- Not checked yetcites O'neil 2017
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“For example, people seem to prefer simple demographic parity when considering university admission algorithms, that is, to require similar admission rates for all demographic groups of applicants (Srivastava et al. 2019);”
§ The Moral Psychology of Artificial Intelligence > 2.1.2. Bias.
What the sources say
- Not checked yetcites Srivastava et al. 2019
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Comparable results were later found for white versus hispanic defendants (Hamilton 2019).”
§ The Moral Psychology of Artificial Intelligence > 2.1.2. Bias.
What the sources say
- Not checked yetcites Hamilton 2019
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“In a landmark investigation (Angwin et al. 2016), the news organization ProPublica published evidence of a racial bias in the results of the COMPAS algorithm, which is used in some US courts to predict (among other outcomes) the risk that a defendant be rearrested in the next two years.”
§ The Moral Psychology of Artificial Intelligence > 2.1.2. Bias.
What the sources say
- Not checked yetcites Angwin et al. 2016
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Second, people may consider that machines are more homogeneous than humans; that is, that a machine being biased is a sign that all machines are comparably biased (Longoni et al. 2022).”
§ The Moral Psychology of Artificial Intelligence > 2.1.2. Bias.
What the sources say
- Not checked yetcites Longoni et al. 2022
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“This concern translates into resistance to medical AI: because patients think their unique characteristics and circumstances will be poorly understood by AI, they prefer to turn to human doctors (Longoni et al. 2019)—leaving unexamined the actual ability of human doctors to take into account these unique characteristics and circumstances.”
§ The Moral Psychology of Artificial Intelligence > 2.1.1. Performance.
What the sources say
- Not checked yetcites Longoni et al. 2019
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“when algorithms decide whether to grant bail to defendants, people prefer that they equalize false positive rates across groups, rather than accuracy across groups (Harrison et al. 2020);”
§ The Moral Psychology of Artificial Intelligence > 2.1.2. Bias.
What the sources say
- Not checked yetcites Harrison et al. 2020
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- It may state two separate claims as one. 30
- Probably not a substantial claim. 14
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Indeed, it appears that people may have extreme performance requirements for implicit moral machines, because they expect a substantial increase over baseline human performance, while overestimating this baseline human performance.”
§ The Moral Psychology of Artificial Intelligence > 2.1.1. Performance.
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Third, people may expect machines to not only inherit biases from humans, but also the difficulty of fixing them—perhaps underestimating our ability to reprogram machines, given our relative inability to reprogram humans (Mullainathan 2019).”
§ The Moral Psychology of Artificial Intelligence > 2.1.2. Bias.
What the sources say
- Not checked yetcites Mullainathan 2019
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Much of this literature on algorithmic fairness is grounded in computer science and impossibility theorems, dealing with the problem that there are many possible mathematical definitions of fairness, whose requirements are sometimes impossible to achieve simultaneously (for entry points, see Chouldechova 2017, Kleinberg et al. 2017, Pleiss et al. 2017).”
§ The Moral Psychology of Artificial Intelligence > 2.1.2. Bias.
What the sources say
- Not checked yetcites Chouldechova 2017
- Not checked yetcites Kleinberg et al. 2017
- Not checked yetcites Pleiss et al. 2017
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“In a representative sample of US drivers, the median respondent believed to be in the top 25 percent of drivers, and estimated that two-thirds of car crashes would be avoided if everyone drove like them.”
§ The Moral Psychology of Artificial Intelligence > 2.1.1. Performance.
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Experimental results, however, suggest that people do not feel especially outraged when machines discriminate, or at least not as outraged as they would feel if humans discriminated (Bigman et al. 2022, Hidalgo et al. 2021).”
§ The Moral Psychology of Artificial Intelligence > 2.1.2. Bias.
What the sources say
- Not checked yetcites Bigman et al. 2022
- Not checked yetcites Hidalgo et al. 2021
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“The key result of the analysis was that while the algorithm made the same number of mistakes for black defendants and for white defendants, it did not make the same mistakes—mistakes which were favorable to the defendant were more likely when the defendant was white, and mistakes which were unfavorable to the defendant were more likely when the defendant was black.”
§ The Moral Psychology of Artificial Intelligence > 2.1.2. Bias.
What the sources say
- Not checked yetcites Angwin et al. 2016
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“There is also a growing body of evidence suggesting that the very groups that feel at risk of biased human decisions may be the least averse to letting machines make decisions—seemingly because they are worried enough about the current decisions of humans to be willing to take a chance with machines (Bigman et al. 2021, Fumagalli et al. 2022, Jago & Laurin 2022, Pammer et al. 2021, 2023).”
§ The Moral Psychology of Artificial Intelligence > 2.1.2. Bias.
What the sources say
- Not checked yetcites Bigman et al. 2021
- Not checked yetcites Fumagalli et al. 2022
- Not checked yetcites Jago & Laurin 2022
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“For example, one of the main concerns that Americans have about implicit moral machines is that they do not understand nuance and complexity as well as humans do (Smith 2019).”
§ The Moral Psychology of Artificial Intelligence > 2.1.1. Performance.
What the sources say
- Not checked yetcites Smith 2019
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“This experimental work is mostly disconnected from moral psychology (for a review, see Starke et al. 2022), and its results seem to be highly dependent on the application domain considered in each article.”
§ The Moral Psychology of Artificial Intelligence > 2.1.2. Bias.
What the sources say
- Not checked yetcites Starke et al. 2022
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Agency and patiency are two continuous dimensions of mind perception (Wegner & Gray 2016), and people can perceive machines as occupying various positions in that two-dimensional space.”
§ The Moral Psychology of Artificial Intelligence > THE MORAL TYPECASTING OF MACHINES
What the sources say
- Not checked yetcites Wegner & Gray 2016
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“This is the approach taken in several policy reports about autonomous driving, which suggest that the minimal requirement before deploying self-driving cars on our roads is that they are provably safer than the average human driver (Bonnefon et al. 2020b, Luetge 2017, Santoni de Sio 2021).”
§ The Moral Psychology of Artificial Intelligence > 2.1.1. Performance.
What the sources say
- Not checked yetcites Bonnefon et al. 2020b
- Not checked yetcites Luetge 2017
- Not checked yetcites Santoni de Sio 2021
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“An even stronger bias exists in the domain of autonomous driving (Liu et al. 2019b, Shariff et al. 2021), where people require their self-driving car to be significantly safer than they themselves are, while substantially overestimating the safety of their own driving.”
§ The Moral Psychology of Artificial Intelligence > 2.1.1. Performance.
What the sources say
- Not checked yetcites Liu et al. 2019b
- Not checked yetcites Shariff et al. 2021
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“In like vein, people express concerns about the transparency or intelligibility of medical AI recommendations, compared to that of human doctors, without realizing that they overestimate their ability to understand human doctors in the first place (Cadario et al. 2021).”
§ The Moral Psychology of Artificial Intelligence > 2.1.1. Performance.
What the sources say
- Not checked yetcites Cadario et al. 2021
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“For example, a representative sample of the German population believed that human experts would have a 20-30 percent mistake rate when predicting credit default or recidivism, which is probably an underestimation—and working from this baseline, the same sample required that machines should have a mistake rate lower than 10 percent (Rebitschek et al. 2021).”
§ The Moral Psychology of Artificial Intelligence > 2.1.1. Performance.
What the sources say
- Not checked yetcites Rebitschek et al. 2021
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“For example, in classic dilemmas where an agent must decide whether to save several lives by sacrificing one, humans are blamed more when they choose to sacrifice one, but this pattern is eliminated or even reversed when a machine makes the decision (Malle et al. 2015, Komatsu et al. 2021).”
§ The Moral Psychology of Artificial Intelligence > BLAMING MACHINES FOR SOLVING DILEMMAS
What the sources say
- Not checked yetcites Malle et al. 2015
- Not checked yetcites Komatsu et al. 2021
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“It is very common in AI ethics to provide lists of moral values or ethical principles that AI should simultaneously pursue (e.g., beneficence, privacy, dignity, transparency) but it is far less common to provide guidance on what machines should do when these values are in conflict (Mittelstadt 2019, Morley et al. 2020).”
§ The Moral Psychology of Artificial Intelligence > 2.2. Explicit Moral Machines
What the sources say
- Not checked yetcites Mittelstadt 2019
- Not checked yetcites Morley et al. 2020
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“This implies that delegating difficult moral decisions to machines may not merely remove a psychological burden from humans, but also change social expectations about which decision should be made (Gill 2020).”
§ The Moral Psychology of Artificial Intelligence > BLAMING MACHINES FOR SOLVING DILEMMAS
What the sources say
- Not checked yetcites Gill 2020
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“For example, the Moral Machine Experiment purposefully used stylized scenarios when a collision is unavoidable (Awad et al. 2020a)”
§ The Moral Psychology of Artificial Intelligence > BLAMING MACHINES FOR SOLVING DILEMMAS
What the sources say
- Not checked yetcites Awad et al. 2020a
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“As implicit moral agents, AVs are expected to lead to an absolute increase in road safety; but AVs are also explicit moral agents in the sense that every action they take can redistribute relative levels of safety between the road users that surround them (Goodall 2016, Bonnefon et al. 2019).”
§ The Moral Psychology of Artificial Intelligence > 2.2. Explicit Moral Machines
What the sources say
- Not checked yetcites Goodall 2016
- Not checked yetcites Bonnefon et al. 2019
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Ethicists, however, are not immune to biases (Schwitzgebel & Cushman 2015), and they do not always come to an agreementfor example, a German national ethics committee could not reach a consensus on what an autonomous vehicle should do when deciding whether to save its passengers or to save other road users (Luetge 2017).”
§ The Moral Psychology of Artificial Intelligence > BLAMING MACHINES FOR SOLVING DILEMMAS
What the sources say
- Not checked yetcites Schwitzgebel & Cushman 2015
- Not checked yetcites Luetge 2017
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“The main promise of AVs is that they can reduce the number of road casualties, by being safer than human drivers; but these lives will not be saved if consumers opt out of the technology because they are unsatisfied or even outraged with the way AVs solve moral dilemmas (Bonnefon et al. 2020a, De Freitas & Cikara 2021).”
§ The Moral Psychology of Artificial Intelligence > BLAMING MACHINES FOR SOLVING DILEMMAS
What the sources say
- Not checked yetcites Bonnefon et al. 2020a
- Not checked yetcites De Freitas & Cikara 2021
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“but more realistic scenarios would have manipulated the probability of the collisions (Krügel & Uhl 2022).”
§ The Moral Psychology of Artificial Intelligence > BLAMING MACHINES FOR SOLVING DILEMMAS
What the sources say
- Not checked yetcites Krügel & Uhl 2022
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“they hesitate when asked to strike a robot (Darling et al. 2015), or to topple a block tower that a robot built and pretends to care about (Briggs & Scheutz 2014)”
§ The Moral Psychology of Artificial Intelligence > EMPATHY FOR THE MACHINE
What the sources say
- Not checked yetcites Darling et al. 2015
- Not checked yetcites Briggs & Scheutz 2014
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Experts from the AV industry may also feel a duty to protect their customers, which could explain why they have a stronger preference to save passengers, compared to the general population (Zhu et al. 2022).”
§ The Moral Psychology of Artificial Intelligence > BLAMING MACHINES FOR SOLVING DILEMMAS
What the sources say
- Not checked yetcites Zhu et al. 2022
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“The Moral Machine Experiment (Awad et al. 2018) considered nine possible priorities for AVs to decide which group of road users to save or to sacrifice, which led to millions of possible scenarios.”
§ The Moral Psychology of Artificial Intelligence > BLAMING MACHINES FOR SOLVING DILEMMAS
What the sources say
- Not checked yetcites Awad et al. 2018
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In like vein, we know that humans will inevitably be blamed for the way they solved a moral dilemma, since by definition a moral dilemma has no universally accepted solution—hence, we may want to relieve human decision-makers from unavoidable blame, by delegating the decision to a machine (see Textbox Blaming machines for solving dilemmas for further discussion).”
§ The Moral Psychology of Artificial Intelligence > Figure 1
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
In the paper
“One problem is that the industry is very reluctant to engage in this debate (Martinho et al. 2021), since any position they take may alienate either their consumer base or the general population.”
§ The Moral Psychology of Artificial Intelligence > BLAMING MACHINES FOR SOLVING DILEMMAS
What the sources say
- Not checked yetcites Martinho et al. 2021
plainconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“In other terms, and as we will consider in more detail in the rest of this section, people sometimes assume that machines have preferences-which can turn machines into moral patients, who experience preferred or dis-preferred outcomes as a result of the actions taken by other agents (Pauketat & Anthis 2022).”
§ The Moral Psychology of Artificial Intelligence > 3. MACHINES AS MORAL PATIENTS
What the sources say
- Not checked yetcites Pauketat & Anthis 2022
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“But from a psychological perspective, people do seem to experience toward machines the same manifold of negative reactions they experience toward humans, perhaps because the machines are perceived as autonomous enough to warrant these reactions (Bigman et al. 2019, Epstein et al. 2020, Franklin et al. 2022).”
§ The Moral Psychology of Artificial Intelligence > 2.1.3. Blame and other reactions to harm.
What the sources say
- Not checked yetcites Bigman et al. 2019
- Not checked yetcites Epstein et al. 2020
- Not checked yetcites Franklin et al. 2022
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“For example, research participants show physiological signs of discomfort when watching a baby dinosaur robot being punched and choked (Rosenthal-von der Pütten et al. 2013), or a robot hand being cut by a knife (Suzuki et al. 2015)”
§ The Moral Psychology of Artificial Intelligence > EMPATHY FOR THE MACHINE
What the sources say
- Not checked yetcites Rosenthal-von der Pütten et al. 2013
- Not checked yetcites Suzuki et al. 2015
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“If we agree that making such decisions inflict an emotional cost on the decision-maker, both in the short and the long term, we may agree that it is good to delegate this burden to machines, who do not experience psychological suffering (Danaher 2022).”
§ The Moral Psychology of Artificial Intelligence > Figure 1
What the sources say
- Not checked yetcites Danaher 2022
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“All other things being equal, people judge crashes as more severe and less acceptable when they are caused by self-driving cars, and place more blame and responsibility on a self-driving car causing a crash, than on a human causing a comparable crash.”
§ The Moral Psychology of Artificial Intelligence > 2.1.3. Blame and other reactions to harm.
What the sources say
- Not checked yetcites Franklin et al. 2021
- Not checked yetcites Hidalgo et al. 2021
- Not checked yetcites Hong et al. 2020
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In sum, humanization strategies usually fail to reduce the machine penalty, and the ones that succeed (partially or totally) fall short of current ethical standards.”
§ The Moral Psychology of Artificial Intelligence > 3.2. Overcoming the Machine Penalty
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Climbing up the humanization gradient does not improve cooperation much, and can even make things worse, due to the uncanny valley effect-that is, the feeling of strangeness and discomfort elicited by a machine that is largely but not quite human-like.”
§ The Moral Psychology of Artificial Intelligence > 3.2. Overcoming the Machine Penalty
What the sources say
- Not checked yetcites Mathur & Reichling 2016
- Not checked yetcites Zlotowski et al. 2016
- Not checked yetcites Cominelli et al. 2021
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Written text, profile pictures, as well as voice and facial dynamics in live online interactions can already be altered to achieve various presentation goals.”
§ The Moral Psychology of Artificial Intelligence > 4.2. Machine Masquerade
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Incentivized games provide a controlled, stylized environment to measure other-regarding preferences and prosocial behavior, that allows for easy comparison of studies and experimental treatments.”
§ The Moral Psychology of Artificial Intelligence > 3.1. Machine-Regarding Preferences
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Within weeks of its public launch, ChatGPT attracted more than a hundred million of users, affording them seemingly endless possibilities.”
§ The Moral Psychology of Artificial Intelligence > 4.2. Machine Masquerade
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“AI-powered deepfakes can create fake identities, which allows phishing attacks to become more personalized and effective.”
§ The Moral Psychology of Artificial Intelligence > 4.1. Delegation to Machines
What the sources say
- Not checked yetcites Seymour & Tully 2016
- Not checked yetcites Jagatic et al. 2007
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“but they can also raise new ethical issues for moral psychology to investigate”
§ The Moral Psychology of Artificial Intelligence > 4.2. Machine Masquerade
plainconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
In the paper
“Reflecting on this emerging worry, a panel of experts has nominated deepfakes as the most dangerous tool for AI-enabled crime (Caldwell et al. 2020).”
§ The Moral Psychology of Artificial Intelligence > 4.1. Delegation to Machines
What the sources say
- Not checked yetcites Caldwell et al. 2020
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Moral psychology has given little attention so far to AI-mediated communication, but this is likely to change given the incoming availability of machine masquerade tools, the way they will transform moral interactions, and the ethical challenges they raise.”
§ The Moral Psychology of Artificial Intelligence > 4.2. Machine Masquerade
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
The paper relies on this without support.
There will almost always be a human in the same loop as the machine, so mistakes will most often be the result of a joint failure of human and machine.
“Since there will almost always be a human in the same loop as the machine, mistakes will most often be the result of a joint failure of human and machine—so, how do people allocate responsibility and blame between humans and machines, when both contributed to a harmful mistake?”
The authors state this as an assumption.
It is possible to capture the manifold of human cooperation by using lab-based games with financial incentives.
“experimental economics has popularized the assumption that it is possible to capture the manifold of human cooperation by using lab-based games with financial incentives, such as dictator games, prisoners' dilemmas, ultimatum games, or public good games.”
The authors state this as an assumption.
Money can be used as a proxy for the many currencies of human-machine cooperation.
“These studies carry over the assumption that just as money can be used as a proxy for the many currencies of human-human cooperation, it can be used as a proxy for the many currencies of human-machine cooperation.”
The paper relies on this without support.
Machines have no affects, needs, or desires for anything.
“This may sound strange, since machines have no affects, nor needs or desires for anything.”
The paper relies on this without support.
AI enables machines to make autonomous decisions on behalf of human stakeholders.
“By design, AI enables machines to make autonomous decisions on behalf of human stakeholders.”
The paper relies on this without support.
AI can act independently.
“because AI can act independently and has the potential to cause harm with unparalleled efficiency and at scale.”
The paper relies on this without support.
People can already experiment with generative AI to create their profile pictures.
“People can already experiment with generative AI to create their profile pictures”
The paper relies on this without support.
The very recent and rapid emergence of AI-driven technology is colliding with moral intuitions forged by culture and evolution over the span of millennia.
“the very recent and rapid emergence of AI-driven technology is colliding with moral intuitions forged by culture and evolution over the span of millennia.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Simulations [162] are used to model ethical dilemmas, especially in areas like autonomous vehicles and healthcare, allowing people to evaluate AI decisions in realistic and interactive scenarios.”
From The Rise of Agentic AI: A Review of Definitions, Frameworks, Architectures, Applications, Evaluation Metrics, and Challenges (2025) · cites this paper as [162]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Systems currently struggle to properly interpret and align with the subtlety of human intention, leading to misalignment of action, including on even simple tasks and moral misalignment on work meant to help others [162].”
From The Rise of Agentic AI: A Review of Definitions, Frameworks, Architectures, Applications, Evaluation Metrics, and Challenges (2025) · cites this paper as [162]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Lack of relevant oversight processes as a safeguard may also make it difficult for humans to detect or intervene in harmful behavior when it occurs [162].”
From The Rise of Agentic AI: A Review of Definitions, Frameworks, Architectures, Applications, Evaluation Metrics, and Challenges (2025) · cites this paper as [162]
Experimental — accuracy not yet measured
Citations
Cites
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Dermatologist-level classification of skin cancer with deep neural networks
Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017) · Nature · doi:10.1038/nature21056 · Cited by 14,362
Explanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302
Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy
Doyle, T. (2017) · The Information Society · doi:10.1080/01972243.2017.1354593 · Cited by 3,111
(2016) · Cited by 2,334
Fair prediction with disparate impact: a study of bias in recidivism prediction instruments
Chouldechova, A. (2017) · Big Data · doi:10.1089/big.2016.0047 · Cited by 2,159
Resistance to Medical Artificial Intelligence
Longoni, C., Bonezzi, A., & Morewedge, C. K. (2019) · Journal of Consumer Research · doi:10.1093/jcr/ucz013 · Cited by 2,015
Principles alone cannot guarantee ethical AI
Mittelstadt, B. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0114-4 · Cited by 1,886
Awad, E., Dsouza, S., Kim, R., Schulz, J., Henrich, J., Shariff, A., Bonnefon, J.-F., & Rahwan, I. (2018) · Nature · doi:10.1038/s41586-018-0637-6 · Cited by 1,859
Self-citationThe social dilemma of autonomous vehicles
Bonnefon, J.-F., Shariff, A., & Rahwan, I. (2016) · Science · doi:10.1126/science.aaf2654 · Cited by 1,527
Self-citation
Research works printed but not linked: 12
- Angwin J, 2016, ProPublica
- Buolamwini J, 2018, PMLR, 81, 77
- Hamilton M., 2019, Am. Crim. Law Rev., 56, 1553
- Hao K, 2019, MIT Technology Review
- Kleinberg J, 2018, Q. J. Econ., 133, 237
- McAllister A., 2016, Minn. Law Rev., 101, 2527
- McCallum S, 2022, BBC News
- Mullainathan S., 2019, New York Times
- Pleiss G, 2017, Adv. Neural Inform. Process. Syst., 30, 5684
- Samuel S, 2020, Transp. Res. Interdiscip. Perspect., 5, 100147
- Smith A., 2019, Public attitudes toward computer algorithms
- Złotowski J, 2016, Paladyn J. Behav. Robot., 7, 55
Cited by
Sorted by citations
Bandi, A., Kongari, B., Naguru, R., Pasnoor, S., & Vilipala, S. V. (2025) · Future Internet · doi:10.3390/fi17090404 · Cited by 207
Artificial Intelligence in Audiology: A Scoping Review of Current Applications and Future Directions
Frosolini, A., Franz, L., Caragli, V., Genovese, E., de Filippis, C., & Marioni, G. (2024) · Sensors · doi:10.3390/s24227126 · Cited by 68
Delegation to artificial intelligence can increase dishonest behaviour
Köbis, N., Rahwan, Z., Rilla, R., Supriyatno, B. I., Bersch, C., Ajaj, T., Bonnefon, J., & Rahwan, I. (2025) · Nature · doi:10.1038/s41586-025-09505-x · Cited by 80
“One potential risk, however, is that it will lead to an increase in ethical transgressions, such as lying and cheating for profit 2,19,20 .”
Self-citationFrom robots to chatbots: unveiling the dynamics of human-AI interaction
Łukasik, A., & Gut, A. (2025) · Frontiers in Psychology · doi:10.3389/fpsyg.2025.1569277 · Cited by 60
Ding, X., Shang, B., Xie, C., Xin, J., & Yu, F. (2025) · Humanities and Social Sciences Communications · doi:10.1057/s41599-025-04564-x · Cited by 44
Metadata
- Authors
- Bonnefon, J.-F., Rahwan, I., & Shariff, A.
- Affiliations
- Centre National de la Recherche Scientifique (FR)
- Max Planck Institute for Human Development (DE)
- University of British Columbia (CA)
- Venue
- Annual Review of Psychology
- Publisher
- Annual Reviews
- Published
- 2024-01-18
- Volume
- 75
- Issue
- 1
- Pages
- 653-675
- ISSN
- 0066-4308 (ISSN-L)
- 1545-2085
- OpenAlex
- W4386820363
- Semantic Scholar
- 97217d457944ee4267d9e439646fa75e5d5cd156
- Cited by
- 151
- References
- 148
- Citations by year
- 2026: 61 · 2025: 58 · 2024: 26
- FWCI
- 5.67
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Annual Review of Psychology (publishedVersion, cc-by)
- PubMed (publishedVersion)
- HAL (submittedVersion, other-oa)
- HAL (submittedVersion)
- Toulouse 1 Capitole Publications (Université Toulouse I Capitole) (submittedVersion)
- MPG.PuRe (Max Planck Society) (submittedVersion, other-oa)
- PsyArXiv (OSF Preprints) (submittedVersion, cc-by)
- Funders
- Agence Nationale de la Recherche (ANR-17-EURE-0010, ANR-19-P3IA-0004, ANR-17-EURE-, ANR‐19, PI3A-0004, 19-PI3A-0004, 17-EURE-00-10, ANR-19-PI3A-0004, ANR-17)
- Social Sciences and Humanities Research Council of Canada
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Psychology of Moral and Emotional Judgment
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Keywords
- moral psychology, artificial intelligence, moral agents, moral patients, human-machine collaboration, ethical dilemmas, anthropomorphism
- MeSH terms
- Animals, Artificial Intelligence, Humans, Intelligence, Morals
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
