Capable but Amoral? Comparing AI and Human Expert Collaboration in Ethical Decision Making
Open accessThe DOI matches its Crossref record; 115 of the 116 references it cites were found in a registry or an index, and 1 are not checked yet.
Abstract
While artificial intelligence (AI) is increasingly applied for decision-making processes, ethical decisions pose challenges for AI applications. Given that humans cannot always agree on the right thing to do, how would ethical decision-making by AI systems be perceived and how would responsibility be ascribed in human-AI collaboration? In this study, we investigate how the expert type (human vs. AI) and level of expert autonomy (adviser vs. decider) influence trust, perceived responsibility, and reliance. We find that participants consider humans to be more morally trustworthy but less capable than their AI equivalent. This shows in participants’ reliance on AI: AI recommendations and decisions are accepted more often than the human expert’s. However, AI team experts are perceived to be less responsible than humans, while programmers and sellers of AI systems are deemed partially responsible instead.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures



Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“People also expect AIs to be more competent and to act less immorally than humans [1,31,50,59], which could make the negative action more surprising.”
From The AI Double Standard: Humans Judge All AIs for the Actions of One (2025) · cites this paper as [59]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Advances in prospect theory: Cumulative representation of uncertainty
Tversky, A., & Kahneman, D. (1992) · Journal of Risk and Uncertainty · doi:10.1007/bf00122574 · Cited by 15,467 · cited 1×
“…the decision: they can either maximize probability of a positive outcome (i.e., the participants would choose the option with the highest probability when maximizing lives saved) or they can maximize utility of a positive outcome (the product of probability and people involved) [32, 56, 134].”
BackgroundThe global landscape of AI ethics guidelines
Jobin, A., Ienca, M., & Vayena, E. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0088-2 · Cited by 6,220 · cited 1×
“Jobin et al. [63] have summarized the most common ethical principles present in those guidelines: trans-parency, justice and fairness, non-maleficence, responsibility, and privacy.”
Risk Aversion and Incentive Effects
Holt, C. A., & Laury, S. K. (2002) · American Economic Review · doi:10.1257/000282802762024700 · Cited by 5,939
A Survey on Bias and Fairness in Machine Learning
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2022) · ACM Computing Surveys · doi:10.1145/3457607 · Cited by 5,812 · cited 1×
“…into something implementable for real-word cases [133], second, there can be trade-offs between these values [115] but the guidelines do not deal with the question how to address those, and third, even within each principle, it depends on the concrete definition whether a principle is met [93].”
BackgroundA Survey on Bias and Fairness in Machine Learning
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2019) · arXiv · doi:10.48550/arxiv.1908.09635 · Cited by 5,812 · cited 1×
“…into something implementable for real-word cases [133], second, there can be trade-offs between these values [115] but the guidelines do not deal with the question how to address those, and third, even within each principle, it depends on the concrete definition whether a principle is met [93].”
BackgroundExplanation in Artificial Intelligence: Insights from the Social Sciences
Miller, T. (2017) · arXiv · doi:10.48550/arxiv.1706.07269 · Cited by 5,338 · cited 1×
“Trust is often researched in HCI, as it is said to increase acceptance and appropriate use of AI [97].”
BackgroundEthics: Inventing Right and Wrong.
Feldman, F., & Mackie, J. L. (1979) · The Philosophical Review · doi:10.2307/2184791 · Cited by 5,079 · cited 2×
“Philoso-phers are divided over the question whether objective truth exists in ethics [47, 52, 88].”
BackgroundTrust in Automation: Integrating Empirical Evidence on Factors That Influence Trust
Hoff, K. A., & Bashir, M. (2015) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720814547570 · Cited by 2,817
Artificial intelligence and the future of work: Human-AI symbiosis in organizational decision making
Jarrahi, M. H. (2018) · Business Horizons · doi:10.1016/j.bushor.2018.03.007 · Cited by 2,772 · cited 1×
“Rather than aiming for AI to take over tasks completely, human-AI collaboration could be a fruitful alternative to combine strengths and produce new possibilities for the future of work [62].”
BackgroundAlgorithm appreciation: People prefer algorithmic to human judgment
Logg, J. M., Minson, J. A., & Moore, D. A. (2019) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2018.12.005 · Cited by 2,061
Research works printed but not linked: 33
- Nadia Adnan, Shahrina Md Nordin, Mohamad Ariff bin Bahruddin, and Murad Ali. 2018. How trust can drive forward the user acceptance to the technology? In-vehicle technology for autonomous vehicle. Transportation research part A: policy and practice 118 (2018), 819ś836.
- Michael Anderson, Susan Leigh Anderson, and Chris Armen. 2004. Towards machine ethics. In AAAI-04 workshop on agent publishers: theory and practice, San Jose, CA. AAAI Press, Phoenix, Arizona, USA, 2ś7.
- Edmond Awad, Sohan Dsouza, Richard Kim, Jonathan Schulz, Joseph Henrich, Azim Shariff, Jean-François Bonnefon, and Iyad Rahwan. 2018. The moral machine experiment. Nature 563, 7729 (2018), 59ś64.
- Annette Baier. 1986. Trust and antitrust. ethics 96, 2 (1986), 231ś260.
- Judith Baker. 1987. Trust and rationality. Pacific philosophical quarterly 68, 1 (1987), 1ś13.
- Gagan Bansal, Besmira Nushi, Ece Kamar, Walter S Lasecki, Daniel S Weld, and Eric Horvitz. 2019. Beyond accuracy: The role of mental models in human-AI team performance. In Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, Vol. 7. The AAAI Press, Palo Alto, California USA, 2ś11.
- Danton S Char, Nigam H Shah, and David Magnus. 2018. Implementing machine learning in health careÐaddressing ethical challenges. The New England journal of medicine 378, 11 (2018), 981.
- Mary L Cummings. 2006. Integrating ethics in design through the value-sensitive design approach. Science and engineering ethics 12, 4 (2006), 701ś715.
- Berkeley J Dietvorst, Joseph P Simmons, and Cade Massey. 2015. Algorithm aversion: People erroneously avoid algorithms after seeing them err. Journal of Experimental Psychology: General 144, 1 (2015), 114.
- Ziv Epstein, Sydney Levine, David G Rand, and Iyad Rahwan. 2020. Who gets credit for AI-generated art? Iscience 23, 9 (2020), 101515.
- Mike Farjam. 2019. On whom would I want to depend; humans or computers? Journal of Economic Psychology 72 (2019), 219ś228.
- Ethan Fast and Eric Horvitz. 2017. Long-term trends in the public perception of artificial intelligence. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 31. The AAAI Press, Palo Alto, California USA, 963ś969.
- Russell Hardin. 1993. The street-level epistemology of trust. Politics & society 21, 4 (1993), 505ś529.
- Maria Hedlund and Erik Persson. 2021. Expert responsibility in AI development. Proceedings of the International Conference of Public Policy (ICPP5) 5 (2021), 1ś24.
- Richard Holton. 1994. Deciding to trust, coming to believe. Australasian journal of philosophy 72, 1 (1994), 63ś76.
- Ekaterina Jussupow, Izak Benbasat, and Armin Heinzl. 2020. Why are we Averse towards Algorithms? A Comprehensive Literature Review on Algorithmic Aver- sion. In Proceedings of the 28th European Conference on Information Systems (ECIS). ECIS, Marrakech, Morocco, 1ś16.
- Guy Kahane, Jim AC Everett, Brian D Earp, Lucius Caviola, Nadira S Faber, Molly J Crockett, and Julian Savulescu. 2018. Beyond sacrificial harm: A two- dimensional model of utilitarian psychology. Psychological Review 125, 2 (2018), 131.
- Serhiy Kandul and Oliver Kirchkamp. 2018. Do I care if others lie? Current and future effects when lies can be delegated. Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics) 74, C (2018), 70ś78.
- Jon Kleinberg, Sendhil Mullainathan, and Manish Raghavan. 2017. Inherent Trade-Offs in the Fair Determination of Risk Scores. In 8th Innovations in Theoret- ical Computer Science Conference (ITCS 2017) (Leibniz International Proceedings in Informatics (LIPIcs), Vol. 67, Christos H. Papadimitriou (Ed.). Schloss Dagstuhlś Leibniz-Zentrum fuer Informatik, Dagstuhl, Germany, 43:1ś43:23.
- Markus Kneer and Michael T Stuart. 2021. Playing the Blame Game with Robots. In Companion of the 2021 ACM/IEEE International Conference on Human-Robot Interaction. Association for Computing Machinery, New York, NY, United States, 407ś411.
- John D Lee and Katrina A See. 2004. Trust in automation: Designing for appro- priate reliance. Human factors 46, 1 (2004), 50ś80.
- Stefanie M. Faas, Johannes Kraus, Alexander Schoenhals, and Martin Baumann. 2021. Calibrating Pedestrians' Trust in Automated Vehicles: Does an Intent Display in an External HMI Support Trust Calibration and Safe Crossing Behav- ior?. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, 1ś17.
- Andreas Matthias. 2004. The responsibility gap: Ascribing responsibility for the actions of learning automata. Ethics and information technology 6, 3 (2004), 175ś183.
- Stephanie M Merritt, Heather Heimbaugh, Jennifer LaChapell, and Deborah Lee. 2013. I trust it, but I don't know why: Effects of implicit attitudes toward automation on trust in an automated system. Human factors 55, 3 (2013), 520ś 534.
- John Stuart Mill. 1861. 1998. Utilitarianism, edited with an introduction by Roger Crisp. Oxford University Press, New York, NY, USA.
- Mahsan Nourani, Chiradeep Roy, Jeremy E Block, Donald R Honeycutt, Tahrima Rahman, Eric Ragan, and Vibhav Gogate. 2021. Anchoring Bias Affects Mental Model Formation and User Reliance in Explainable AI Systems. In 26th Inter- national Conference on Intelligent User Interfaces. Association for Computing Machinery, New York, NY, USA, 340ś350.
- International Review of the Red Cross. 2019. Artificial intelligence and machine learning in armed conflict: A human-centred approach. Technical Report 102. ICRC.
- Marianne Promberger and Jonathan Baron. 2006. Do patients trust computers? Journal of Behavioral Decision Making 19, 5 (2006), 455ś468.
- Anuschka Schmitt, Thiemo Wambsganss, Matthias Söllner, and Andreas Janson. 2021. Towards a Trust Reliance Paradox? Exploring the Gap Between Perceived Trust in and Reliance on Algorithmic Advice. In International Conference on Information Systems (ICIS), Vol. 1. ICIS, Austin, Texas, 1ś17.
- S Shyam Sundar. 2008. The MAIN model: A heuristic approach to understanding technology effects on credibility. MacArthur Foundation Digital Media and Learning Initiative, Chicago, IL, USA.
- the Committee of Ministers of the Council of Europe. 2020. Recommendation CM/Rec(2020)1 of the Committee of Ministers to member States on the human rights impacts of algorithmic systems. Technical Report. Council of Europe.
- Peter M Todd and Gerd Gigerenzer. 2000. Précis of simple heuristics that make us smart. Behavioral and brain sciences 23, 5 (2000), 727ś741.
- Dakuo Wang, Elizabeth Churchill, Pattie Maes, Xiangmin Fan, Ben Shneiderman, Yuanchun Shi, and Qianying Wang. 2020. From human-human collaboration to Human-AI collaboration: Designing AI systems that can work together with people. In Extended abstracts of the 2020 CHI conference on human factors in computing systems. Association for Computing Machinery, New York, NY, USA, 1ś6.
Cited by
Responsible Artificial Intelligence Governance: A Review and Research Framework
Papagiannidis, E., Mikalef, P., & Conboy, K. (2025) · The Journal of Strategic Information Systems · doi:10.1016/j.jsis.2024.101885 · Cited by 565
Harnessing the power of AI in qualitative research: Exploring, using and redesigning ChatGPT
Zhang, H., Wu, C., Xie, J., Lyu, Y., Cai, J., & Carroll, J. M. (2025) · Computers in Human Behavior Artificial Humans · doi:10.1016/j.chbah.2025.100144 · Cited by 402
“The ethical quandaries presented by the confluence of human intelligence and artificial intelligence have long been a focus of both scholarly deliberation and real-world application [83, 96].”
Mehrotra, S., Degachi, C., Vereschak, O., Jonker, C. M., & Tielman, M. L. (2024) · ACM Journal on Responsible Computing · doi:10.1145/3696449 · Cited by 144
Gomez, C., Cho, S. M., Ke, S., Huang, C.-M., & Unberath, M. (2025) · Frontiers in Computer Science · doi:10.3389/fcomp.2024.1521066 · Cited by 209
Position: Towards Bidirectional Human-AI Alignment
Shen, H., Knearem, T., Ghosh, R., Alkiek, K., Krishna, K., Yachuan, L., Petridis, S., Peng, Y.-H., Qiwei, L., Si, C., et al. (2025) · neural information processing systems · doi:10.52202/085713-4186 · Cited by 211
“…146, 147, 148, 149, 150, 151, 152, 86, 117, 126, 153, 154, 155, 156, 87, 157, 158, 159, 160, 161, 162, 104, 163, 164, 165, 166, 167, 168, 169, 32, 170, 171, 172, 173, 174, 175, 176, 3, 177, 178, 179, 180, 181, 89, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 122, 195, 196,…”
Metadata
- Authors
- Tolmeijer, S., Christen, M., Kandul, S., Kneer, M., & Bernstein, A.
- Affiliations
- University of Zurich (CH)
- Venue
- CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2022-04-27
- Pages
- 1-17
- OpenAlex
- W4224990314
- Cited by
- 70
- References
- 107
- Citations by year
- 2026: 12 · 2025: 25 · 2024: 23 · 2023: 7 · 2022: 3
- FWCI
- 14.41
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion, cc-by)
- Zurich Open Repository and Archive (University of Zurich) (acceptedVersion, cc-by)
- PhilPapers (PhilPapers Foundation) (submittedVersion)
- Zurich Open Repository and Archive (University of Zurich) (submittedVersion, cc-by)
- Zurich Open Repository and Archive (University of Zurich)
- Funders
- Swiss National Science Foundation (07740_187494)
- Armasuisse Science and Technology (S+T)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Business, Management and Accounting › Organizational Behavior and Human Resource Management › Ethics in Business and Education
- Life Sciences › Neuroscience › Cognitive Neuroscience › Psychology of Moral and Emotional Judgment
- Keywords
- artificial intelligence, human-AI collaboration, ethical decision-making, trust in AI, perceived responsibility, AI reliance, moral trust, AI decision-making
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Capable but Amoral? Comparing AI and Human Expert Collaboration in Ethical Decision Making. https://app.cochat.ai/papers/10.1145/3491102.3517732
- The paper
- Tolmeijer, S., Christen, M., Kandul, S., Kneer, M., & Bernstein, A. (2022). Capable but Amoral? Comparing AI and Human Expert Collaboration in Ethical Decision Making. CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3491102.3517732
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.