CoChat

Challenges and efforts in managing AI trustworthiness risks: a state of knowledge

Open access
Polemi, Nineta, Praça, Isabel, Kioskli, Kitty, Bécue, Adrien · 2024 · Frontiers in Big Data

The DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.

Abstract

This paper addresses the critical gaps in existing AI risk management frameworks, emphasizing the neglect of human factors and the absence of metrics for socially related or human threats. Drawing from insights provided by NIST AI RFM and ENISA, the research underscores the need for understanding the limitations of human-AI interaction and the development of ethical and social measurements. The paper explores various dimensions of trustworthiness, covering legislation, AI cyber threat intelligence, and characteristics of AI adversaries. It delves into technical threats and vulnerabilities, including data access, poisoning, and backdoors, highlighting the importance of collaboration between cybersecurity engineers, AI experts, and social-psychology-behavior-ethics professionals. Furthermore, the socio-psychological threats associated with AI integration into society are examined, addressing issues such as bias, misinformation, and privacy erosion. The manuscript proposes a comprehensive approach to AI trustworthiness, combining technical and social mitigation measures, standards, and ongoing research initiatives. Additionally, it introduces innovative defense strategies, such as cyber-social exercises, digital clones, and conversational agents, to enhance understanding of adversary profiles and fortify AI security. The paper concludes with a call for interdisciplinary collaboration, awareness campaigns, and continuous research efforts to create a robust and resilient AI ecosystem aligned with ethical standards and societal expectations.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3389/fdata.2024.1381163

Cites

26 of 26 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 Attacks on AI.
Figure 1 Attacks on AI.CC BY · PubMed Central
Figure 2 AI security threats and controls navigator (OWASP Top Ten, 2021).
Figure 2 AI security threats and controls navigator (OWASP Top Ten, 2021).CC BY · PubMed Central
Figure 3 Attacks on LLMs.
Figure 3 Attacks on LLMs.CC BY · PubMed Central
Figure 4 NIST AI RMF AI actors across AI lifecycle stages (NIST AI RMF URL).
Figure 4 NIST AI RMF AI actors across AI lifecycle stages (NIST AI RMF URL).CC BY · PubMed Central
Figure 5 ENISA multilayer framework for AI-related cybersecurity good practices (ENISA URL).
Figure 5 ENISA multilayer framework for AI-related cybersecurity good practices (ENISA URL).CC BY · PubMed Central
Figure 6 MITRE adversarial threat landscape for AI systems (MITRE ATLAS™ URL).
Figure 6 MITRE adversarial threat landscape for AI systems (MITRE ATLAS™ URL).CC BY · PubMed Central
Figure 7 LLM security with OWASP and MITRE resources (OWASP URL).
Figure 7 LLM security with OWASP and MITRE resources (OWASP URL).CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 14

  1. Aufrant, 2020, Guide de recommandations pour la spécification et la qualification de systèmes intégrant de l'intelligence artificielle
  2. 2018, Supervisory Guidance on Model Risk Management
  3. Cavoukian, 2017, Privacy by Design in the Age of Big Data
  4. 2019, Ethics Guidelines for Trustworthy AI. High-Level Expert Group on AI (HLEG)
  5. Hoffman, 2018, “Evaluating human-robot interaction with the interactive causal attribution model,”
  6. Huang, 2020, Towards dependable artificial intelligence: a survey, J. Comput. Sci. Technol., 35, 487
  7. IEEE P2976 Standards2021
  8. IEEE P3119 Standards.2021
  9. Kioskli, 2020, “Measuring psychosocial and behavioral factors improves attack potential estimates,”, 216
  10. Newman, 2023, A Taxonomy of Trustworthiness for Artificial Intelligence Connecting Properties of Trustworthiness With Risk Management and the AI Lifecycle. UC Berkeley Center for Long-Term Cybersecurity Series, CLTC White Papers Series, January 2023
  11. 2024, Adversarial Machine Learning—A Taxonomy and Terminology of Attacks and Mitigations
  12. Pennycook, 2018, The Implied Truth Effect: Attaching Warnings to a Subset of Fake News Stories Increases Perceived Accuracy of Stories Without Warnings. Management.
  13. Raj, 2020, Priming for performance: the impact of learning management systems on firm performance, Manag. Sci., 66, 1489
  14. Smith, 2021, The S.A.F.E. approach to AI safety, Artif. Intell. J., 59, 58

Cited by

See all 66

Metadata

Authors
Polemi, N., Praça, I., Kioskli, K., & Bécue, A.
Affiliations
  • University of Piraeus (GR)
  • Grupo de Investigação em Engenharia e Computação Inteligente para a Inovação e o Desenvolvimento (PT)
  • University of Essex (GB)
  • Thales (France) (FR)
Corresponding author
Polemi, Nineta
Venue
Frontiers in Big Data
Published
2024-05-09
Volume
7
Pages
1381163-1381163
ISSN
2624-909X (ISSN-L)
OpenAlex
W4396775672
Cited by
46
References
24
Citations by year
2026: 22 · 2025: 21 · 2024: 3
FWCI
19.07
Type
journal article
Language
en
Open access
Locations
  • Frontiers in Big Data (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
Funders
European Commission (101121042, 101135932)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
  • Physical Sciences › Computer Science › Information Systems › Information and Cyber Security
Keywords
trustworthy AI, AI risk management, human-AI interaction, data poisoning, backdoors, algorithmic bias, misinformation, privacy erosion, AI security, ENISA, AI legislation, chatbots, AI ethics

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.