When something goes wrong: Who is responsible for errors in ML decision-making?
Open accessThe DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Abstract
Because of its practical advantages, machine learning (ML) is increasingly used for decision-making in numerous sectors. This paper demonstrates that the integral characteristics of ML, such as semi-autonomy, complexity, and non-deterministic modeling have important ethical implications. In particular, these characteristics lead to a lack of insight and lack of comprehensibility, and ultimately to the loss of human control over decision-making. Errors, which are bound to occur in any decision-making process, may lead to great harm and human rights violations. It is important to have a principled way of assigning responsibility for such errors. The integral characteristics of ML, however, pose serious difficulties in defining responsibility and regulating ML decision-making. First, we elaborate on these characteristics and their epistemic and ethical implications. We then analyze possible general strategies for resolving the assignment of moral responsibility and show that, due to the specific way in which ML functions, each potential solution is problematic, whether we assign responsibility to humans, machines, or using hybrid models. Then, we shift focus on an alternative approach that bypasses moral responsibility and attempts to define legal liability independently through solutions such as informed consent and the no-fault compensation system. Both of these solutions prove unsatisfactory because they leave too much room for potential abuses of ML decision-making. We conclude that both ethical and legal solutions are fraught with serious difficulties. These difficulties prompt us to re-weigh the costs and benefits of using ML for high-stake decisions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Mythos of Model Interpretability
Lipton, Z. C. (2016) · arXiv · doi:10.48550/arxiv.1606.03490 · Cited by 5,422 · cited 3×
“This makes the paths of information in ML models too numerous for any human to be able to ‘walk through’ in a normal life span (Boge and Grünke 2019; Lipton 2016).”
Explanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302
Floridi, L., Cowls, J., Beltrametti, M., Chatila, R., Chazerand, P., Dignum, V., Luetge, C., Madelin, R., Pagallo, U., Rossi, F., et al. (2018) · Minds and Machines · doi:10.1007/s11023-018-9482-5 · Cited by 4,774
How the machine “thinks”: Understanding opacity in machine learning algorithms
Burrell, J. (2016) · Big Data & Society · doi:10.1177/2053951715622512 · Cited by 3,196 · cited 1×
“Burrell (2016) calls this a mismatch between the mathematical procedures in ML and human styles of semantic interpretation, in the sense that what the NN represents is not suitable for human comprehension.”
Explaining Explanations: An Overview of Interpretability of Machine Learning
Gilpin, L. H., Bau, D., Yuan, B. Z., Bajwa, A., Specter, M., & Kagal, L. (2018) · 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA) · doi:10.1109/dsaa.2018.00018 · Cited by 2,507 · cited 2×
“The pressure from both the academic literature and the various governmental institutions and NGOs to provide such explanations resulted in the project of Explainable AI (UNI Global Union 2018; Floridi et al. 2018; Hoffman et al. 2018; Gilpin et al. 2018; Ribera and Lapedriza 2019).”
Responsibility and Control: A Theory of Moral Responsibility
Fischer, J. M., & Ravizza, M. (1998) · doi:10.1017/cbo9780511814594 · Cited by 2,249
Principles alone cannot guarantee ethical AI
Mittelstadt, B. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0114-4 · Cited by 1,886 · cited 2×
“For a comparative analysis of different models of responsibility, see Yeung (2019). ensure that the responsible person really bears the consequences (Mittelstadt 2019).”
Ananny, M., & Crawford, K. (2018) · New Media & Society · doi:10.1177/1461444816676645 · Cited by 1,836
Wachter, S., Mittelstadt, B., & Floridi, L. (2017) · International Data Privacy Law · doi:10.1093/idpl/ipx005 · Cited by 1,308
The responsibility gap: Ascribing responsibility for the actions of learning automata
Matthias, A. (2004) · Ethics and Information Technology · doi:10.1007/s10676-004-3422-1 · Cited by 1,101 · cited 3×
“However, as Matthias points out, “[w]ere it possible to supply every machine with a controlling human expert, nobody would need the machine in the first place” (2004, p. 177).”
Influential
Cited by
Extended human agency: towards a teleological account of AI
Noller, J. (2024) · Humanities and Social Sciences Communications · doi:10.1057/s41599-024-03849-x · Cited by 30
Transforming Disaster Risk Reduction With AI and Big Data: Legal and Interdisciplinary Perspectives
Chun, K. P., Octavianti, T., Dogulu, N., Tyralis, H., Papacharalampous, G., Rowberry, R., Fan, P., Everard, M., Francesch‐Huidobro, M., Migliari, W., et al. (2025) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.70011 · Cited by 19
Connectionism about human agency: responsible AI and the social lifeworld
Noller, J. (2025) · AI & SOCIETY · doi:10.1007/s00146-024-02133-5 · Cited by 6
Metadata
- Authors
- Berber, A., & Srećković, S.
- Venue
- AI & SOCIETY
- Published
- 2023-02-13
- Volume
- 39
- Issue
- 4
- Pages
- 1891-1903
- ISSN
- 0951-5666 (ISSN-L)
- 1435-5655
- OpenAlex
- W4320484574
- Cited by
- 37
- References
- 21
- FWCI
- 9.03
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: When something goes wrong: Who is responsible for errors in ML decision-making? https://app.cochat.ai/papers/10.1007/s00146-023-01640-1
- The paper
- Berber, A., & Srećković, S. (2024). When something goes wrong: Who is responsible for errors in ML decision-making? AI & SOCIETY. https://doi.org/10.1007/s00146-023-01640-1
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.