CoChat

A requirements model for AI algorithms in functional safety-critical systems with an explainable self-enforcing network from a developer perspective

Open access

The DOI matches its Crossref record; 81 of the 81 references it cites were found in a registry or an index; 11 of the 71 sources its claims cite were checked for support.

Abstract

The requirements for ensuring functional safety have always been very high. Modern safety-related systems are becoming increasingly complex, making also the safety integrity assessment more complex and time-consuming. This trend is further intensified by the fact that AI-based algorithms are finding their way into safety-related systems or will do so in the future. However, existing and expected standards and regulations for the use of AI methods pose significant challenges for the development of embedded AI software in functional safety-related systems. The consideration of essential requirements from various perspectives necessitates an intensive examination of the subject matter, especially as different standards have to be taken into account depending on the final application. There are also different targets for the “safe behavior” of a system depending on the target application. While stopping all movements of a machine in industrial production plants is likely to be considered a “safe state”, the same condition might not be considered as safe in flying aircraft, driving cars or medicine equipment like heart pacemaker. This overall complexity is operationalized in our approach in such a way that it is straightforward to monitor conformity with the requirements. To support safety integrity assessments and reduce the required effort, a Self-Enforcing Network (SEN) model is presented in which developers or safety experts can indicate the degree of fulfillment of certain requirements with possible impact on the safety integrity of a safety-related system. The result evaluated by the SEN model indicates the achievable safety integrity level of the assessed system, which is additionally provided by an explanatory component.

Overview

CoChat Summary

TL;DR

The paper proposes an explainable Self-Enforcing Network (SEN) requirements model that maps developers’ assessed fulfillment of 44 AI/functional-safety requirements to achievable Safety Integrity Levels (SIL 1–3) or an unsuitable classification. It uses expert-defined reference types and Shapley-value feature importance to show which requirements drive the result.

Key findings

  • SEN validation of the reference types produced ranking/distance values of SIL 1: 1.26/0.00, SIL 2: 0.82/0.00, SIL 3: 2.68/0.00, and “Not suitable for SCS”: 2.84/0.00 (Table 1).
  • For a deterministic NN with an explanatory component, SEN classified the input as SIL 2 but with lower activation than the validated vector and at a greater distance; feature importance showed “hazard and risk analysis” was not met at all and “coding standards” only partially satisfied.
  • DL without xAI was classified as not suitable for SCS, with feature importance showing most requirements not met. DL with beginnings of xAI gave no clear SIL assignment: distance 0.85 assigned SIL 1, while the strongest ranking activation of 1.22 argued it was still unsuitable, lower than 2.84 for “Not suitable for SCS” and near the SIL 1 final activation.
  • A rule-based algorithm in a 2-channel architecture produced activation 2.17 and distance 0.61, conforming to SIL 3 requirements; feature importance showed deviations because a 3-channel architecture was expected and the base model was designed primarily for AI methods used redundantly for monitoring or diagnostics.

Methods

  • The requirements model was built from standards including EN 61508:2010, EN ISO 13849-1:2023, EN IEC 62061:2021, IEC ISO TR 5469:2024, and the AI Act, with 44 selected requirements and expert-assigned reference types for SIL 1–3 and an unsuitable method; values were normalized between -1 and 1.
  • SEN is a two-layer feed-forward self-organized learning network: semantic-matrix values are transformed by the Self-Enforcing Rule \(w_{ao}=c * v_{sm} * cvf_a\), with an Enforcing Activation Function; parameters included learning rate 0.1, one learning iteration, and cue validity factors of 0.5 for general requirements and 1.0 for explainability and system-decisive attributes.
  • Analysis used validation by presenting reference types, classification by highest activation ranking and smallest Euclidean distance, and explainability via Shapley-value feature importance, simplified in SEN as \(\Phi_i(v)=v(i)\).

Limitations

  • The paper states the basic model was developed primarily for NN-based methods, so requirements and possible applications for other algorithms need to be extended.
  • The paper states the selection and evaluation of requirements were not determined by a broad group of experts, so subjective influences cannot be ruled out; more experts should be involved to generalize the model.
  • The paper notes the DL-with-beginnings-of-xAI case produced non-equal first-place results and that its feature-importance visualization could be misinterpreted; it also states the model considers only general terms, with concrete requirements still needing identification for specific cases, and no data are associated with the article.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1051/sands/2024020

Cites

81 of 81 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

6 Valid · 9 Not valid · 50 Not checked · 72 Own results and interpretations

31 more claims

Create a free account to see all 137 claims and to verify the 28 references still open.

See all claims — free account
  • Probably not a substantial claim. 87
  • It may state two separate claims as one. 34
  • The quote only partly states this claim. 2

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

Show all 81 references

Research works printed but not linked: 6

  1. Oveisi, 2024, Adv. Stand Appl Sci, 2, 4
  2. Gaur, 2024, AI Mag, 45, 139
  3. Sanneman, 2022, Int J Human–Comput Inter, 38, 1772
  4. Weaver R, McDermid J and Kelly T. Software safety arguments: Towards a systematic categorisation of evidence. In: International System Safety Conference, Denver, CO 2002.
  5. Zinkhan, 2023, Eng Proc, 39, 41
  6. Klüver C, Werner C, Nowara P, Castel B and Israel R. Self-enforcing networks for monitoring safety-critical systems: A prototype development. In: Klüver C, & Klüver J (eds.) New algorithms for practical problems: Springer Vieweg, 2025 (in German).

Cited by

See all 8

Metadata

Authors
Klüver, C., Greisbach, A., Kindermann, M., & Püttmann, B.
Affiliations
  • Fuchs (Germany) (DE)
  • TÜV Nord (Germany) (DE)
Venue
Security and Safety
Publisher
EDP Sciences
Published
2024
Volume
3
Pages
2024020
ISSN
2826-1275 (ISSN-L)
OpenAlex
W4403935496
Cited by
8
References
68
Citations by year
2026: 4 · 2025: 4
FWCI
2.08
Type
journal article
Language
en
Open access
Locations
  • Security and Safety (publishedVersion, cc-by)
  • Springer Link (Chiba Institute of Technology) (submittedVersion)
  • Universitätsbibliographie, Universität Duisburg-Essen (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Safety Systems Engineering in Autonomy
  • Physical Sciences › Computer Science › Hardware and Architecture › Real-Time Systems Scheduling
  • Physical Sciences › Computer Science › Software › Software Reliability and Analysis Research
Keywords
functional safety, safety-critical systems, artificial intelligence algorithms, explainable AI, requirements models, safety integrity level, developer perspectives, safety standards and regulations

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: A requirements model for AI algorithms in functional safety-critical systems with an explainable self-enforcing network from a developer perspective. https://app.cochat.ai/papers/10.1051/sands/2024020
The paper
Klüver, C., Greisbach, A., Kindermann, M., & Püttmann, B. (2024). A requirements model for AI algorithms in functional safety-critical systems with an explainable self-enforcing network from a developer perspective. Security and Safety. https://doi.org/10.1051/sands/2024020

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.