Artificial Intelligence for Safety-Critical Systems in Industrial and Transportation Domains: A Survey
Open accessAbstract
Artificial Intelligence (AI) can enable the development of next-generation autonomous safety-critical systems in which Machine Learning (ML) algorithms learn optimized and safe solutions. AI can also support and assist human safety engineers in developing safety-critical systems. However, reconciling both cutting-edge and state-of-the-art AI technology with safety engineering processes and safety standards is an open challenge that must be addressed before AI can be fully embraced in safety-critical systems. Many works already address this challenge, resulting in a vast and fragmented literature. Focusing on the industrial and transportation domains, this survey structures and analyzes challenges, techniques, and methods for developing AI-based safety-critical systems, from traditional functional safety systems to autonomous systems. AI trustworthiness spans several dimensions, such as engineering, ethics and legal, and this survey focuses on the safety engineering dimension.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
AI is often not used directly in a safety function, but rather as a diagnostic function.
“AI is often not used directly in a safety function, but rather as a diagnostic function.”
From A requirements model for AI algorithms in functional safety-critical systems with an explainable self-enforcing network from a developer perspective (2024) · cites this paper as [19]
Cites
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423 · see citing works
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294 · see citing works
Membership Inference Attacks Against Machine Learning Models
Shokri, R., Stronati, M., Song, C., & Shmatikov, V. (2017) · 2017 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2017.41 · Cited by 4,754 · see citing works
Awad, E., Dsouza, S., Kim, R., Schulz, J., Henrich, J., Shariff, A., Bonnefon, J.-F., & Rahwan, I. (2018) · Nature · doi:10.1038/s41586-018-0637-6 · Cited by 1,859 · see citing works
Can we open the black box of AI?
Castelvecchi, D. (2016) · Nature · doi:10.1038/538020a · Cited by 1,773 · see citing works
Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks
Katz, G., Barrett, C., Dill, D. L., Julian, K., & Kochenderfer, M. J. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-63387-9_5 · Cited by 1,706 · see citing works
Applications of artificial intelligence in transport: An overview
Abduljabbar, R., Dia, H., Liyanage, S., & Bagloee, S. A. (2019) · Sustainability · doi:10.3390/su11010189 · Cited by 769 · see citing works
Modelling uncertainty in deep learning for camera relocalization
Kendall, A., & Cipolla, R. (2016) · 2016 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2016.7487679 · Cited by 548 · see citing works
Ethical Decision Making during Automated Vehicle Crashes
Goodall, N. J. (2014) · Transportation Research Record: Journal of the Transportation Research Board · doi:10.3141/2424-07 · Cited by 292 · see citing works
Chatila, R., Firth-Butterflied, K., Havens, J. C., & Karachalios, K. (2017) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2017.2670225 · Cited by 72 · see citing works
Developing artificial neural networks for safety critical systems
Kurd, Z., Kelly, T., & Austin, J. (2006) · Neural Computing and Applications · doi:10.1007/s00521-006-0039-9 · Cited by 55 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Yigit, Y., Ferrag, M. A., Ghanem, M. C., Sarker, I. H., Maglaras, L. A., Chrysoulas, C., Moradpoor, N., Tihanyi, N., & Janicke, H. (2025) · Sensors · doi:10.3390/s25061666 · Cited by 69 · see citing works
Klüver, C., Greisbach, A., Kindermann, M., & Püttmann, B. (2024) · Security and Safety · doi:10.1051/sands/2024020 · Cited by 8 · see citing works
“The framework consists of a “world model” that provides a mathematical description of how the AI system affects the outside world, a mathematical description of what effects are acceptable in a “safety specification”, and a “verifier” that should provide a proof that the AI satisfies the safety…”
- Authors
- Perez-Cerrolaza, J., Abella, J., Borg, M., Donzella, C., Cerquides, J., Cazorla, F. J., Englund, C., Tauber, M., Nikolakopoulos, G., & Flores, J. L.
- Affiliations
- Ikerlan Technology Research Centre, Basque Research and Technology Alliance (BRTA), Mendaro, Guipuzcoa, Spain
- Barcelona Supercomputing Center (ES)
- RISE Research Institutes of Sweden (SE)
- Exida, Rovereto, Italy
- Consejo Superior de Investigaciones Científicas (ES)
- BSC and Maspatechnologies S.L., Barcelona, Spain
- Research Studios Austria (AT)
- Luleå University of Technology (SE)
- Ikerlan Technology Research Centre, BRTA, Arrasate/Mondragon, Spain
- ORCID
- Venue
- ACM Computing Surveys
- Published
- 2024-04-09
- Volume
- 56
- Issue
- 7
- Pages
- 1-40
- ISSN
- 0360-0300 (ISSN-L)
- 1557-7341
- OpenAlex
- W4387527822
- Cited by
- 185
- References
- 11
- Citations by year
- 2026: 87 · 2025: 71 · 2024: 27
- FWCI
- 31.81
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM Computing Surveys (publishedVersion, cc-by)
- DIGITAL.CSIC (Spanish National Research Council (CSIC)) (acceptedVersion)
- UPCommons institutional repository (Universitat Politècnica de Catalunya) (submittedVersion, cc-by-nc-nd)
- KTH Publication Database DiVA (KTH Royal Institute of Technology) (submittedVersion, other-oa)
- Diva portal (Dalarna University Library) (submittedVersion, cc-by)
- Topics
- Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Safety Systems Engineering in Autonomy
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Hardware and Architecture › Real-Time Systems Scheduling
- Keywords
- artificial intelligence, safety-critical systems, safety engineering, machine learning safety, autonomous systems, functional safety, trustworthy AI, safety standards, industrial safety, transport safety, human-AI collaboration
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
