Regulating Autonomous Systems: Beyond Standards

Danks, David; London, Alex John

doi:10.1109/mis.2017.1

Cited by 61Digitalization, Law, and RegulationSafety Systems Engineering in AutonomyEthics and Social Impacts of AI

The DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.

Regulating Autonomous Systems: Beyond Standards

Danks, David, London, Alex John·2017·IEEE Intelligent Systems

DOI 10.1109/mis.2017.1

Open this paper in CoChat — free account

Abstract

Autonomous systems are on the verge of widespread use, but there are significant challenges to ensuring their safe and appropriate development, deployment, and application. The authors argue that autonomous systems cannot, in general, be regulated using traditional performance standards. They contend that these systems should instead be evaluated by a staged, iterative regulatory approach similar to that for pharmaceuticals and medical devices. This article outlines some features of such a system.

Integrity signals

Registry

Crossref · 10.1109/mis.2017.1

Cites

5 of 5 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

The social dilemma of autonomous vehicles
Bonnefon, J.-F., Shariff, A., & Rahwan, I. (2016) · Science · doi:10.1126/science.aaf2654 · Cited by 1,527
The Out-of-the-Loop Performance Problem and Level of Control in Automation
Endsley, M. R., & Kiris, E. O. (1995) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872095779064555 · Cited by 1,406
Killer Robots
SPARROW, R. (2007) · Journal of Applied Philosophy · doi:10.1111/j.1468-5930.2007.00346.x · Cited by 593
Performance-Based Regulation: Prospects and Limitations in Health, Safety and Environmental ProtectionPreprint
Coglianese, C., Nash, J., & Olmstead, T. A. (2003) · SSRN Electronic Journal · doi:10.2139/ssrn.392400 · Cited by 128
The Structure of Clinical Translation: Efficiency, Information, and Ethics
Kimmelman, J., & London, A. J. (2015) · Hastings Center Report · doi:10.1002/hast.433 · Cited by 85
Self-citation

Flagged first, then by citations

Research works printed but not linked: 5

2016, Federal Automated Vehicles Policy Accelerating the Next Revolution in Roadway Safety
maurer, 2015, Autonomous Driving Technical Legal and Social Aspects
scharre, 2016, Autonomous Weapons and Operational Risk
davies, 2016, Google's Self-Driving Car Caused Its First Crash, Wired
vlasic, 2016, ‘Self-Driving Tesla Was Involved in Fatal Crash ’ U S Says

Sorted by citations

Explainable Artificial Intelligence (XAI): What we know and what is left to attain Trustworthy Artificial Intelligence
Ali, S., Abuhmed, T., El-Sappagh, S., Muhammad, K., Alonso-Moral, J. M., Confalonieri, R., Guidotti, R., Del Ser, J., Díaz-Rodríguez, N., & Herrera, F. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101805 · Cited by 1,762
Governing autonomous vehicles: emerging responses for safety, liability, privacy, cybersecurity, and industry risks
Taeihagh, A., & Lim, H. S. M. (2019) · Transport Reviews · doi:10.1080/01441647.2018.1494640 · Cited by 601
Auditing large language models: a three-layered approach
Mökander, J., Schuett, J., Kirk, H. R., & Floridi, L. (2024) · AI and Ethics · doi:10.1007/s43681-023-00289-2 · Cited by 289
Meaningful Explanations of Black Box AI Decision Systems
Pedreschi, D., Giannotti, F., Guidotti, R., Monreale, A., Ruggieri, S., & Turini, F. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33019780 · Cited by 234
A Survey of Methods for Explaining Black Box Models
Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., & Pedreschi, D. (2019) · ACM Computing Surveys · doi:10.1145/3236009 · Cited by 629
“This impacts not only information on ethics but also on accountability [59], on safety [23], and on industrial liability [53].”
Background
Ethical issues in focus by the autonomous vehicles industry
Martinho, A. M., Herber, N., Kroesen, M., & Chorus, C. G. (2021) · Transport Reviews · doi:10.1080/01441647.2020.1862355 · Cited by 178
Some pitfalls in the promises of automated and autonomous vehicles
Hancock, P. A. (2018) · Ergonomics · doi:10.1080/00140139.2018.1498136 · Cited by 122
Impacts on Trust of Healthcare AI
LaRosa, E., & Danks, D. J. (2018) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3278721.3278771 · Cited by 92
Self-citation
Ethics-based auditing of automated decision-making systems: intervention points and policy implications
Mökander, J., & Axente, M. (2021) · AI & Society · doi:10.1007/s00146-021-01286-x · Cited by 87
Robots, Standards and the Law: Rivalries between Private Standards and Public Policymaking for Robot Governance
Fosch‐Villaronga, E., & Golia, A. (2019) · Computer law & security review · doi:10.1016/j.clsr.2018.12.009 · Cited by 60
“Trust but Verify”: The Difficulty of Trusting Autonomous Weapons Systems
Roff, H. M., & Danks, D. J. (2018) · Journal of Military Ethics · doi:10.1080/15027570.2018.1481907 · Cited by 57
Self-citation
Regulatory governance in emerging technologies: The case of autonomous vehicles in Sweden and Norway
Hansson, L. (2020) · Research in Transportation Economics · doi:10.1016/j.retrec.2020.100967 · Cited by 56
User‐Centered Evaluation of Explainable Artificial Intelligence (XAI): A Systematic Literature Review
Al-Ansari, N., Al‐Thani, D., & Al-Mansoori, R. S. (2024) · Human Behavior and Emerging Technologies · doi:10.1155/2024/4628855 · Cited by 58
Towards common ethical and safe ‘behaviour’ standards for automated vehicles
Papadimitriou, E., Farah, H., van de Kaa, G., de Sio, F. S., Hagenzieker, M., & van Gelder, P. H. A. J. M. (2022) · Accident Analysis & Prevention · doi:10.1016/j.aap.2022.106724 · Cited by 47
Explainable AI for Deep Learning Based Disease Detection
Kinger, S., & Kulkarni, V. Y. (2021) · International Conference on Contemporary Computing · doi:10.1145/3474124.3474154 · Cited by 49
ILIME: Local and Global Interpretable Model-Agnostic Explainer of Black-Box Decision
Elshawi, R., Mohamed, Y., Al-Mallah, M., & Sakr, S. (2019) · Symposium on Advances in Databases and Information Systems · doi:10.1007/978-3-030-28730-6_4 · Cited by 45
Generative causal explanations of black-box classifiersPreprint
O’Shaughnessy, M. R., Canal, G., Connor, M., Davenport, M. A., & Rozell, C. J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.13913 · Cited by 38
A governance perspective on user acceptance of autonomous systems in Singapore
Pande, D., & Taeihagh, A. (2024) · Technology in Society · doi:10.1016/j.techsoc.2024.102580 · Cited by 34
Open the Black Box Data-Driven Explanation of Black Box Decision SystemsPreprint
Pedreschi, D., Giannotti, F., Guidotti, R., Monreale, A., Pappalardo, L., Ruggieri, S., & Turini, F. (2018) · arXiv.org · doi:10.48550/arxiv.1806.09936 · Cited by 39
Navigating the governance challenges of disruptive technologies: insights from regulation of autonomous systems in Singapore
Pande, D., & Taeihagh, A. (2023) · Journal of Economic Policy Reform · doi:10.1080/17487870.2023.2197599 · Cited by 26
Science and Technology Parks: A Futuristic Approach
Makhdoom, I., Lipman, J., Abolhasan, M., & Challen, D. (2022) · IEEE Access · doi:10.1109/access.2022.3159798 · Cited by 23
From algorithmic accountability to digital governance
Mökander, J., & Floridi, L. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00504-5 · Cited by 28
Regulating Autonomous Vehicles
London, A. J., & Danks, D. J. (2018) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3278721.3278763 · Cited by 19
Self-citation
Balancing the Benefits of Autonomous Vehicles
Geary, T., & Danks, D. J. (2019) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3306618.3314237 · Cited by 19
Self-citation
The Race Between Cognitive and Artificial Intelligence
Arora, A. S., & Arora, A. (2019) · International Journal of Intelligent Information Technologies · doi:10.4018/ijiit.2020010101 · Cited by 13
Estimation and Interpretation of Machine Learning Models with Customized Surrogate Model
Ali, M., Khattak, A. M., Ali, Z., Hayat, B., Idrees, M., Pervez, Z., Rizwan, K., Sung, T.-E., & Kim, K. (2021) · Electronics · doi:10.3390/electronics10233045 · Cited by 14
Comprehension is a double-edged sword: Over-interpreting unspecified information in intelligible machine learning explanations
Xuan, Y., Small, E. A., Sokol, K., Hettiachchi, D., & Sanderson, M. (2024) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2024.103376 · Cited by 17
The Role of Cybersecurity in Medical Devices Regulation: Future Considerations and Solutions
Ludvigsen, K. R. (2023) · Law Technology and Humans · doi:10.5204/lthj.3080 · Cited by 14
Demystifying the black box: an overview of explainability methods in machine learning
Kinger, S., & Kulkarni, V. Y. (2023) · International Journal of Computers and Applications · doi:10.1080/1206212x.2023.2285533 · Cited by 9
Managing social-educational robotics for students with autism spectrum disorder through business model canvas and customer discovery
Arora, A. S., Arora, A., Sivakumar, K., & McIntyre, J. R. (2024) · Frontiers in Robotics and AI · doi:10.3389/frobt.2024.1328467 · Cited by 8
The problem of atypicality in LLM-powered psychiatry
Garcia, B., Chua, E. Y. S., & Brah, H. S. (2025) · Journal of Medical Ethics · doi:10.1136/jme-2025-110972 · Cited by 10
A model to discipline autonomy in cyber‐physical systems‐of‐systems and its application
Gharib, M., da Silva, L. D., & Ceccarelli, A. (2020) · Journal of Software Evolution and Process · doi:10.1002/smr.2328 · Cited by 8
Algorithmic war and the dangers of in-visibility, anonymity, and fragmentation
Baggiarini, B. (2024) · Australian Journal Of International Affairs · doi:10.1080/10357718.2024.2333824 · Cited by 10
A Model-Based Approach for Analyzing the Autonomy Levels for Cyber-Physical Systems-of-Systems
Gharib, M., da Silva, L. D., Kavalionak, H., & Ceccarelli, A. (2018) · Latin-American Symposium on Dependable Computing · doi:10.1109/ladc.2018.00024 · Cited by 7
A Review on Software/Systems Architecture Description for Autonomous Systems
Souza, L. S., Rocha, F. G., & Soares, M. S. (2022) · Recent Advances in Computer Science and Communications · doi:10.2174/2666255815666220513101350 · Cited by 6
Towards Autonomous Vehicles in Smart Cities: Risks and Risk Governance
Taeihagh, A., & Lim, H. S. M. (2021) · EAI/Springer Innovations in Communication and Computing · doi:10.1007/978-3-030-66042-0_7 · Cited by 7
Establishing Rigorous Certification Standards: A Systematic Methodology for AI Safety-Critical Systems in Military Aviation
Jurado, R. D.-A., Plaza, V. O., Ye, X., María, Z. S., Moreno, F. P., & César, G. A. (2024) · IEEE Access · doi:10.1109/access.2024.3487591 · Cited by 4
ALERA
Banerjee, S., & Chatterjee, A. (2019) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3338123 · Cited by 3
An autoencoder and vision transformer based interpretability analysis on the performance differences in automated staging of second and third molars
Büyükçakır, B., Tobel, J. D., Thevissen, P. W., Vandermeulen, D., & Claes, P. D. (2025) · Scientific Reports · doi:10.1038/s41598-025-26121-x · Cited by 2
Work, Life, and Artificial Intelligence (AI): Emerging Findings from Asia
Yan, H., & Gai, S. (2024) · Palgrave studies in equity, diversity, inclusion, and indigenization in business · doi:10.1007/978-3-031-52795-1_4 · Cited by 3
AI-Driven Cybersecurity in Industrial Automation
Tyagi, H., Goel, P. K., Tyagi, P., & Tyagi, H. (2025) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-3241-3.ch016 · Cited by 3
System Safety Engineering Approach for Autonomous Mobile Machinery
Tiusanen, R., Heikkilä, E., & Malm, T. (2020) · Lecture notes in mechanical engineering · doi:10.1007/978-3-030-64228-0_21 · Cited by 2
Homologation of Autonomous Machines from a Legal Perspective.
Žolnerčíková, V. (2018) · Cited by 2
Assured Autonomy Survey
Rouff, C., & Watkins, L. A. (2022) · Foundations and Trends® in Privacy and Security · doi:10.1561/3300000027 · Cited by 2
The Influence of Social Networking Usage Experience and Activity on Preferences of Explainable Artificial Intelligence (XAI) Representation Methods in a Hate Speech Detection System
Al-Ansari, N., Al‐Thani, D., & Bahameish, M. (2024) · Lecture notes in computer science · doi:10.1007/978-981-96-0567-5_6 · Cited by 1
AI Systems for Physicians: A Review from Socio-technical and Human-Computer Interaction Perspectives
Young, J. W., & Nah, F. F. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-13167-6_12 · Cited by 1
Effects Of Artificial Intelligence Used In Healthcare On Confidence On Patient And Physician
Maltaş, E., GÜMÜŞ, M., KIZILKAYA, E., & ORHAN, S. (2022) · Gevher Nesibe Journal IESDR · doi:10.46648/gnj.301 · Cited by 1
Ethics, Culture, and Values Impacted by Human–Robot Interactions
Barfield, W., Weng, Y.-H., & Pagallo, U. (2024) · Cambridge University Press eBooks · doi:10.1017/9781009386708.024 · Cited by 1
Navigating the sociotechnical labyrinth: dynamic certification for responsible embodied AI
Bakirtzis, G., Tubella, A. A., Theodorou, A. A., Danks, D., & Topcu, U. (2025) · Elsevier eBooks · doi:10.1016/b978-0-44-340553-2.00019-8 · Cited by 1
Self-citation
Towards an unanimous international regulatory body for responsible use of Artificial Intelligence [UIRB-AI]Preprint
Chidambaram, R. (2017) · arXiv.org · doi:10.48550/arxiv.1712.07752
Challenges in Monitoring Machine Ethics
Davison, C. B., & Davison, S. S. G. (2019) · Journal of Computer Science and Information Technology · doi:10.15640/jcsit.v7n2a1
From Algorithmic Accountability to Digital GovernancePreprint
Mökander, J., & Floridi, L. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4142849
Eight Recommendations for Ethical and Legal Assessments of Robotic Systems Interacting with Humans
Burri, T., Christen, M., Beck, J., & Trusilo, D. (2023) · Cambridge University Press eBooks · doi:10.1017/9781009386708.030
Beyond Disciplines: Applying Design Thinking Across Areas to Enhance Mental Health
Muppala, C., & Bikkina, N. (2025) · Signals and communication technology · doi:10.1007/978-3-031-68952-9_69
Enhancing Safety in Military Aviation: A Systematic Approach to the Development of AI Certification Standards
Jurado, R. D.-A., Plaza, V. O., Ye, X., Suárez, M. Z., Moreno, F. P., & Arnaldo, C. G. (2025) · doi:10.3390/engproc2025090083
Explaining in context: Perceived informativeness of Explainable Artificial Intelligence (XAI) in Arabic hate speech detection
Al-Ansari, N., Al‐Thani, D., & Bahameish, M. (2026) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2026.101083
Les activités maritimes face aux nouveaux risques : l'assurance entre défis et innovation
Edorh-Komahe, A. P. (2022) · doi:10.70675/42284fe7z04fbz47d9zaac2z18ad2634b460
XAI-Based Evaluation Model for Navigating Academic Ambiguity
Pathak, D. R. K., & Upadhyay, P. (2026) · International Journal of Innovations in Science Engineering and Management. · doi:10.69968/ijisem.2026v5si153-58
A ‘journey to trust?’ Machine learning, faith, and the discourse of ‘trusted’ autonomous systems
Baggiarini, B., Jevlgevskaja, N., & Burken, C. B.-V. (2026) · Critical Military Studies · doi:10.1080/23337486.2026.2678621
Next-generation medical nanorobotics: biohybrid systems, AI-guided navigation, and clinical translation challenges
Singh, A. P., Gupta, A., Soni, V., Gajbhiye, A., & Jain, D. (2026) · Next Nanotechnology · doi:10.1016/j.nxnano.2026.100824

Authors: Danks, D., & London, A. J.
Affiliations: Carnegie Mellon University (US)
ORCID: Danks, David
London, Alex John
Venue: IEEE Intelligent Systems
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Published: 2017-01
Volume: 32
Issue: 1
Pages: 88-91
ISSN: 1541-1672 (ISSN-L)
DOI: 10.1109/mis.2017.1
OpenAlex: W2588936615
Cited by: 61
References: 5
Citations by year: 2026: 5 · 2025: 6 · 2024: 10 · 2023: 5 · 2022: 8 · 2021: 5 · 2020: 4 · 2019: 8 · 2018: 9 · 2017: 1
FWCI: 61.95
Type: journal article
Language: en
Open access: closed
Licence: ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html (vor)
Locations: IEEE Intelligent Systems (publishedVersion)
Topics: Social Sciences › Law › Digitalization, Law, and Regulation
Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Safety Systems Engineering in Autonomy
Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
Keywords: autonomous systems, regulation, performance standards, iterative evaluation, safety, medical device regulation, drug regulation, technology governance

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Regulating Autonomous Systems: Beyond Standards

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Citations

Cites

Cited by

Related

Metadata

Record

About these checks