The DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Mind the gaps: Assuring the safety of autonomous systems from an engineering, ethical, and legal perspective
Open accessAbstract
This paper brings together a multi-disciplinary perspective from systems engineering, ethics, and law to articulate a common language in which to reason about the multi-faceted problem of assuring the safety of autonomous systems. The paper's focus is on the “gaps” that arise across the development process: the semantic gap, where normal conditions for a complete specification of intended functionality are not present; the responsibility gap, where normal conditions for holding human actors morally responsible for harm are not present; and the liability gap, where normal conditions for securing compensation to victims of harm are not present. By categorising these “gaps” we can expose with greater precision key sources of uncertainty and risk with autonomous systems. This can inform the development of more detailed models of safety assurance and contribute to more effective risk control.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The third international consensus definitions for sepsis and septic shock (Sepsis-3).
Singer, M., Deutschman, C. S., Seymour, C. W., Shankar-Hari, M., Annane, D., Bauer, M., Bellomo, R., Bernard, G. R., Chiche, J.-D., Coopersmith, C. M., et al. (2016) · JAMA · doi:10.1001/jama.2016.0287 · Cited by 28,250
Fleischmann, C., Scherag, A., Adhikari, N. K. J., Hartog, C. S., Tsaganos, T., Schlattmann, P., Angus, D. C., & Reinhart, K. (2016) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.201504-0781oc · Cited by 3,548 · cited 1×
“Sepsis is associated with at least 1 in 20 deaths in England [45] and is a leading cause of mortality worldwide [19] [63].”
BackgroundAleatory or epistemic? Does it matter?
Kiureghian, A. D., & Ditlevsen, O. (2009) · Structural Safety · doi:10.1016/j.strusafe.2008.06.020 · Cited by 2,454 · cited 1×
“based on the capability of the system architecture itself to minimise the risk given the complexity and unpredictability of the domain, sensing errors, and component insufficiencies [16].”
BackgroundResponsibility and Control: A Theory of Moral Responsibility
Fischer, J. M., & Ravizza, M. (1998) · doi:10.1017/cbo9780511814594 · Cited by 2,249 · cited 2×
“The Control and Epistemic conditions are sometimes called the ‘Aristotelian conditions’: they can be traced back to Aristotle’s treatment of moral responsibility [3][21][12].”
BackgroundAwad, E., Dsouza, S., Kim, R., Schulz, J., Henrich, J., Shariff, A., Bonnefon, J.-F., & Rahwan, I. (2018) · Nature · doi:10.1038/s41586-018-0637-6 · Cited by 1,859 · cited 2×
“Some of the ethical concerns fielded in respect of highly autonomous vehicles are an indirect result of the difficulties raised by the semantic gap [6].”
BackgroundThe surviving sepsis campaign bundle: 2018 update
Levy, M. M., Evans, L. E., & Rhodes, A. (2018) · Intensive Care Medicine · doi:10.1007/s00134-018-5085-0 · Cited by 1,445 · cited 1×
“Best practice for treatment is currently rapid administration of antibiotics to treat the infection, intravenous fluids to correct hypotension (low blood pressure), and vasopressor medications if the patient’s blood pressure remains low [40] [26].”
BackgroundConcrete Problems in AI Safety
Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., & Mané, D. (2016) · arXiv · doi:10.48550/arxiv.1606.06565 · Cited by 1,314
Komorowski, M., Celi, L. A., Badawi, O., Gordon, A. C., & Faisal, A. A. (2018) · Nature Medicine · doi:10.1038/s41591-018-0213-5 · Cited by 1,290
The responsibility gap: Ascribing responsibility for the actions of learning automata
Matthias, A. (2004) · Ethics and Information Technology · doi:10.1007/s10676-004-3422-1 · Cited by 1,101
SPARROW, R. (2007) · Journal of Applied Philosophy · doi:10.1111/j.1468-5930.2007.00346.x · Cited by 593 · cited 1×
“Because of the severity of risk and the importance of accountability in the military domain, the notion is often discussed with respect to autonomous weapons and meaningful human control [65] [61][51].”
Cited by
Bibri, S. E., Krogstie, J., Kaboli, A., & Alahi, A. (2024) · Environmental Science and Ecotechnology · doi:10.1016/j.ese.2023.100330 · Cited by 536
Yigitcanlar, T., & Cugurullo, F. (2020) · Sustainability · doi:10.3390/su12208548 · Cited by 337
Atakishiyev, S., Salameh, M., Yao, H., & Goebel, R. (2021) · arXiv · doi:10.48550/arxiv.2112.11561 · Cited by 336
Atakishiyev, S., Salameh, M., Yao, H., & Goebel, R. (2024) · IEEE Access · doi:10.1109/access.2024.3431437 · Cited by 323
Bibri, S. E., Alexandre, A., Sharifi, A., & Krogstie, J. (2023) · Energy Informatics · doi:10.1186/s42162-023-00259-2 · Cited by 299
Perez-Cerrolaza, J., Abella, J., Borg, M., Donzella, C., Cerquides, J., Cazorla, F. J., Englund, C., Tauber, M., Nikolakopoulos, G., & Flores, J. L. (2024) · ACM Computing Surveys · doi:10.1145/3626314 · Cited by 185
Meaningful human control: actionable properties for AI system development
Cavalcante Siebert, L., Lupetti, M. L., Aizenberg, E., Beckers, N., Zgonnikov, A., Veluwenkamp, H., Abbink, D., Giaccardi, E., Houben, G.-J., Jonker, C. M., et al. (2023) · AI and Ethics · doi:10.1007/s43681-022-00167-3 · Cited by 138
Artificial intelligence explainability: the technical and ethical dimensions
McDermid, J. A., Jia, Y., Porter, Z., & Habli, I. (2021) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2020.0363 · Cited by 136
“These stakeholders may be legally liable for the behaviour of the ML-based system once deployed [13].”
BackgroundSelf-citationEffective human–AI work design for collaborative decision-making
Jain, R., Garg, N., & Khera, S. N. (2023) · Kybernetes · doi:10.1108/k-04-2022-0548 · Cited by 85
Clinicians risk becoming “liability sinks” for artificial intelligence
Lawton, T., Morgan, P., Porter, Z., Hickey, S., Cunningham, A., Hughes, N., Iacovides, I., Jia, Y., Sharma, V., & Habli, I. (2024) · Future Healthcare Journal · doi:10.1016/j.fhj.2024.100007 · Cited by 60
“20 ‘Vicarious liability’ is when an employer is held strictly liable for the negligence or wrongdoing of an employee, which is closely connected to their employment.”
BackgroundSelf-citationReframing deception for human-centered AI
Umbrello, S., & Natale, S. (2024) · International Journal of Social Robotics · doi:10.1007/s12369-024-01184-4 · Cited by 23
Trusilo, D. (2023) · Journal of Military Ethics · doi:10.1080/15027570.2023.2213985 · Cited by 20
Analysing the Safety of Decision-Making in Autonomous Systems
Osborne, M., Hawkins, R., & McDermid, J. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-14835-4_1 · Cited by 6
“This can be a particular challenge since AS typically operate in highly complex environments that often cannot be fully specified at design time [3].”
BackgroundSelf-citationTowards Explainable and Trustworthy Autonomous Physical Systems
Omeiza, D., Anjomshoae, S., Kollnig, K., Camburu, O.-M., Främling, K., & Kunze, L. (2021) · Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411763.3441338 · Cited by 2
Metadata
- Authors
- Burton, S., Habli, I., Lawton, T., McDermid, J., Morgan, P., & Porter, Z.
- Venue
- Artificial Intelligence
- Publisher
- Elsevier BV
- Published
- 2020-02
- Volume
- 279
- Pages
- 103201
- ISSN
- 0004-3702 (ISSN-L)
- OpenAlex
- W2987375645
- Cited by
- 173
- References
- 15
- FWCI
- 14.80
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.