Trustworthy AI: From Principles to Practices
Open accessThe DOI matches its Crossref record; 398 of the 399 references it cites were found in a registry or an index; 74 of the 251 sources its claims cite were checked for support.
Abstract
The rapid development of Artificial Intelligence (AI) technology has enabled the deployment of various systems based on it. However, many current AI systems are found vulnerable to imperceptible attacks, biased against underrepresented groups, lacking in user privacy protection. These shortcomings degrade user experience and erode people’s trust in all AI systems. In this review, we provide AI practitioners with a comprehensive guide for building trustworthy AI systems. We first introduce the theoretical framework of important aspects of AI trustworthiness, including robustness, generalization, explainability, transparency, reproducibility, fairness, privacy preservation, and accountability. To unify currently available but fragmented approaches toward trustworthy AI, we organize them in a systematic approach that considers the entire lifecycle of AI systems, ranging from data acquisition to model development, to system development and deployment, finally to continuous monitoring and governance.
Overview
CoChat Summary
TL;DR
This review provides a comprehensive guide to trustworthy AI, organizing robustness, generalization, explainability, transparency, reproducibility, fairness, privacy preservation, and accountability across the AI system lifecycle. It proposes TrustAIOps as a continuous workflow and identifies open challenges including trade-offs, immature evaluation, and risks from large pre-trained models.
Key findings
- The paper defines eight trustworthiness aspects—robustness, generalization, explainability, transparency, reproducibility, fairness, privacy preservation, and accountability—and argues that predictive accuracy alone is insufficient; trustworthiness must be assessed systematically across data preparation, algorithm design, development, deployment, and management.
- It proposes TrustAIOps, extending MLOps/DevOps with aligned trustworthiness principles, interdisciplinary collaboration, extensive artifact management, and continuous feedback loops across the five lifecycle stages.
- It documents frictions and trade-offs: transparency can leak private data or intellectual property and enable targeted hacking; adversarial robustness can reduce accuracy; robustness and fairness can negatively affect each other; and explanations can be fragile, inconsistent with human interpretation, or hard to judge as faithful.
- In privacy evaluation, it cites work showing that 15 demographic attributes were sufficient to render 99% of participants unique.
Methods
- Design: narrative review/survey, not an empirical study; it synthesizes technical, engineering, and governance literature on trustworthy AI.
- Data or sample: no dataset or sample is used; the paper draws on cited literature and industrial case studies of face recognition, autonomous driving, and NLP in the appendix.
- Main analysis: conceptual organization of approaches by trustworthiness aspect and by lifecycle stage, culminating in a proposed systematic framework and TrustAIOps workflow.
Limitations
- The paper states that explainability, transparency, and accountability are still seldom evaluated quantitatively; approaches to robustness and explainability remain immature; trade-offs and frictions exist between trustworthiness aspects; large pre-trained models are costly and difficult to reproduce and may carry bias or privacy leakage; and end-user awareness is limited.
- Evident from the text: as a survey, it presents no original experiments or quantitative validation of the proposed TrustAIOps workflow, and its recommendations are conceptual rather than empirically tested in the paper.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
39 Valid · 28 Not valid · 96 Not checked · 119 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“It is widely recognized that AI models might be vulnerable to attacks by adversaries with malicious intentions.”
What the sources say
- Partly supportedcites [68] [68] Andreas Rossler, Davide Cozzolino, Luisa Verdoliva, Christian Riess, Justus Thies, and Matthias Nießner. Faceforensics++: Learning to detect manipulated facial images. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 1-11, 2019. - sourceExperimental — accuracy not yet measured
The Face Manipulation Methods section says forgery detectors have robustness issues and that compression and resizing can erase traces they use to detect manipulation. That supports vulnerability in these particular models, but the paper does not establish the broader claim that AI models are vulnerable to intentional attacks by malicious adversaries. (LLM, conf=0.78)
- Not matchedcites [304]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not checked — no full text of the sourcecites [11] [11] Sarah Bankins and Paul Formosa. Ethical ai at work: The social contract for artificial intelligence and its implications for the workplace psychological contract. Redefining the Psychological Contract in the Digital Era: Issues for Research and Practice , pages 55-72, 2021. - source
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“many current AI systems are found vulnerable to imperceptible attacks, biased against underrepresented groups, lacking in user privacy protection.”
What the sources say
- Could not verifycites [68] [68] Andreas Rossler, Davide Cozzolino, Luisa Verdoliva, Christian Riess, Justus Thies, and Matthias Nießner. Faceforensics++: Learning to detect manipulated facial images. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 1-11, 2019. - sourceExperimental — accuracy not yet measured
The Face Manipulation Methods section discusses robustness of facial-forgery detectors to compression and resizing, and §4.2.2 reports lower detection performance on compressed videos. Searches for imperceptible adversarial attacks and perturbations found no passage saying that many current AI systems are vulnerable to such attacks. (LLM, conf=0.74)
- Not matchedcites [304]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not checked — no full text of the sourcecites [11] [11] Sarah Bankins and Paul Formosa. Ethical ai at work: The social contract for artificial intelligence and its implications for the workplace psychological contract. Redefining the Psychological Contract in the Digital Era: Issues for Research and Practice , pages 55-72, 2021. - source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“However, liveness detection is still vulnerable against new types of attacks by fake videos generated by AI techniques like DeepFake [54].”
What the sources say
- Partly supportedcites [54] [54] Thanh Thi Nguyen, Quoc Viet Hung Nguyen, Cuong M Nguyen, Dung Nguyen, Duc Thanh Nguyen, and Saeid Nahavandi. Deep learning for deepfakes creation and detection: A survey. arXiv preprint arXiv:1909.11573 , 2019. - sourceExperimental — accuracy not yet measured
Section 3, “Deepfake Detection,” reports that tested face-recognition systems failed to detect deepfakes effectively and that other methods had high error rates on a newly produced deepfake-video dataset. This supports vulnerability to AI-generated fake videos, but the passage does not specifically assess liveness detection. (LLM, conf=0.84)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Research has shown that face recognition systems can be easily spoofed by impersonation attackers wearing a small printed adversarial patch [90, 30, 43, 73].”
What the sources say
- Supportedcites [90] [90] Zihao Xiao, Xianfeng Gao, Chilin Fu, Yinpeng Dong, Wei Gao, Xiaolu Zhang, Jun Zhou, and Jun Zhu. Improving transferability of adversarial patches on face recognition with generative models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 11845-11854, 2021. - source
The Introduction describes adversarial eyeglass frames used to impersonate another identity and says physical patches can fool face-recognition models. Section 4.5 reports an attacker wearing printed adversarial eyeglass frames and finding that the patches retained high transferability after printing and photographing. The physical-world experiment involved one volunteer and three target identities. (LLM, conf=0.86)
- Partly supportedcites [30] [30] Ying Guo, Xingxing Wei, Guoqiu Wang, and Bo Zhang. Meaningful adversarial stickers for face recognition in physical world. arXiv:2104.06728 , 2021. - sourceExperimental — accuracy not yet measured
Section 5.3 reports successful impersonation attacks using facial stickers and shows examples in the physical world. But Section 3.5 explicitly says the method does not involve printing: subjects wear existing stickers placed according to calculated attack parameters. Thus the paper supports the impersonation point, but not the description of a printed adversarial patch. (LLM, conf=0.89)
- Partly supportedcites [43] [43] Stepan Komkov and Aleksandr Petiushko. Advhat: Real-world adversarial attack on arcface face id system. In 2020 25th International Conference on Pattern Recognition (ICPR) , pages 819-826. IEEE, 2021. - sourceExperimental — accuracy not yet measured
The Abstract and Introduction (p. 1) describe an easily reproducible attack using a printed sticker on a hat to confuse a Face ID model. But the paper’s results concern making the wearer less recognizable, not impersonating another person: Section 2.1 (p. 2) distinguishes impersonation from dodging, and the Testing Protocol (p. 5) describes its attacks as dodging attacks. (LLM, conf=0.90)
- Partly supportedcites [73] [73] Mahmood Sharif, Sruti Bhagavatula, Lujo Bauer, and Michael K Reiter. A general framework for adversarial examples with objectives. ACM Transactions on Privacy and Security (TOPS) , 22(3):1-30, 2019. - sourceExperimental — accuracy not yet measured
The Introduction and Experiment Setup (pages unavailable in our copy) describe impersonation attacks against face-recognition DNNs using printed adversarial patterns affixed to eyeglasses worn by attackers. The source does not establish that face-recognition systems generally are easily spoofed by a small patch: it tests particular DNNs and eyeglass-based attacks under controlled conditions, and notes that some physical impersonation attempts failed. (LLM, conf=0.84)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“Some pre-trained models are reported to output training text data containing private user information such as addresses [63].”
What the sources say
- Could not verifycites [63] [63] Inioluwa Deborah Raji, Andrew Smart, Rebecca N White, Margaret Mitchell, Timnit Gebru, Ben Hutchinson, Jamila Smith-Loud, Daniel Theron, and Parker Barnes. Closing the ai accountability gap: Defining an end-to-end framework for internal algorithmic auditing. In Proceedings of the 2020 conference on fairness, accountability, and transparency , pages 33-44, 2020. - sourceExperimental — accuracy not yet measured
Section 4.4.2 discusses documentation for trained models, and Section 4.6 discusses privacy practices for face data; neither reports a model outputting private information from its training text. Searches for pre-trained models, training-text leakage, and addresses found no passage addressing the claim. The retrieved passages provide section labels but no page numbers. (LLM, conf=0.76)
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
72 more claims · 5 with their cited sources checked
Create a free account to see all 282 claims and to verify the 38 references still open.
- Probably not a substantial claim. 191
- It may state two separate claims as one. 43
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“An effective automated audit system for authorship and provenance compliance functions as a continuous pipeline rather than a retrospective review tool [28].”
From Designing automated audit mechanisms to evaluate compliance of generative AI platforms with federal authorship and ownership disclosure requirements. (2023) · cites this paper as [28]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Reliable provenance begins at the moment content is produced, not after distribution [28].”
From Designing automated audit mechanisms to evaluate compliance of generative AI platforms with federal authorship and ownership disclosure requirements. (2023) · cites this paper as [28]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“This high-level understanding will help optimize trust, as operators will be aware of the capabilities and limitations of the AI systems they work with and know the appropriate settings for use104.”
From Adopting and expanding ethical principles for generative artificial intelligence from military to healthcare (2023) · cites this paper as [104]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“This may be deliberate, to protect brand perception or avoid licensing obligations, or unintentional, especially in creative interfaces that mask algorithmic influence behind seemingly manual controls [28].”
From Designing automated audit mechanisms to evaluate compliance of generative AI platforms with federal authorship and ownership disclosure requirements. (2023) · cites this paper as [28]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 3 Trustworthy AI: A Systematic Approach · p. 13
“the lifecycle of the development of a typical AI product can be partitioned into data preparation, algorithm design, development-deployment, and management [25].”
§ 2.6 Privacy protection · p. 12
“the five main stages of the lifecycle of an AI system, i.e., data preparation, algorithm design, development, deployment, and management.”
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
The Nature of Statistical Learning Theory
Vapnik, V. N. (1995) · doi:10.1007/978-1-4757-2440-0 · Cited by 39,138
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834 · cited 1×
“For example, in computer vision, an established successful paradigm involves pre-training models on a largescale dataset, such as ImageNet, and then fine-tuning them on target tasks with fewer training data [131, 224, 375].”
MethodRich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684 · cited 1×
“For example, in computer vision, an established successful paradigm involves pre-training models on a largescale dataset, such as ImageNet, and then fine-tuning them on target tasks with fewer training data [131, 224, 375].”
MethodGrad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.74 · Cited by 24,513
The Elements of Statistical Learning
Hastie, T., Friedman, J., & Tibshirani, R. (2001) · Springer Series in Statistics · doi:10.1007/978-0-387-21606-5 · Cited by 24,162
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707
Pan, S. J., & Yang, Q. (2010) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2009.191 · Cited by 23,387 · cited 2×
“This challenge arises from various practical AI tasks [343, 391] in the area of transfer learning [255, 350].”
BackgroundLanguage Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Research works printed but not linked: 60
- Dalal Al-Azizy, David Millard, Iraklis Symeonidis, Kieron O’Hara, and Nigel Shadbolt. 2015. A literature survey and classifications on data deanonymisation. In Proceedings of the International Conference on Risks and Security of Internet and Systems. Springer, 36–51.
- Dzmitry Bahdanau Kyunghyun Cho and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In 3rd International Conference on Learning Representations (ICLR’15) .
- Joy Buolamwini and Timnit Gebru. 2018. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on Fairness, Accountability and Transparency. PMLR, 77–91.
- 10.5555/3294996.3295155 · doi:10.5555/3294996.3295155
- Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, et al. 2021. Extracting training data from large language models. In Proceedings of the 30th USENIX Security Symposium (USENIX Security’21). 2633–2650.
- Yu Cheng Mo Yu Xiaoxiao Guo and Bowen Zhou. 2019. Few-shot learning with meta metric learners. In NIPS 2017 Workshop on Meta-Learning .
- Moustapha Cisse, Piotr Bojanowski, Edouard Grave, Yann Dauphin, and Nicolas Usunier. 2017. Parseval networks: Improving robustness to adversarial examples. In International Conference on Machine Learning. PMLR, 854–863.
- Bo Cowgill and Catherine Tucker. 2017. Algorithmic bias: A counterfactual perspective. NSF Trustworthy Algorithms.
- Francesco Croce, Maksym Andriushchenko, Vikash Sehwag, Edoardo Debenedetti, Nicolas Flammarion, Mung Chiang, Prateek Mittal, and Matthias Hein. 2021. RobustBench: A standardized adversarial robustness benchmark. In Proceedings of the 35th Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2).
- Luiz Marcio Cysneiros and Vera Maria Benjamim Werneck. 2009. An initial analysis on how software transparency and trust influence each other. In Proceedings of the Workshop on Requirements Engineering (WER’09). Citeseer.
- Alexey Dosovitskiy, German Ros, Felipe Codevilla, Antonio Lopez, and Vladlen Koltun. 2017. CARLA: An open urban driving simulator. In Conference on Robot Learning. PMLR, 1–16.
- 10.5555/1791834.1791836 · doi:10.5555/1791834.1791836
- Exforsys. 2011. What Is Monkey Testing. Retrieved July 9, 2021 from http://www.exforsys.com/tutorials/testing-types/ monkey-testing.html.
- Chelsea Finn, Pieter Abbeel, and Sergey Levine. 2017. Model-agnostic meta-learning for fast adaptation of deep networks. In International Conference on Machine Learning. PMLR, 1126–1135.
- Shixiang Gu and Luca Rigazio. 2015. Towards deep neural network architectures robust to adversarial examples. In 3rd International Conference on Learning Representations (ICLR’15) San Diego CA USA May 7-9 2015 Workshop Track Proceedings .
- IBM. Trusting AI. Retrieved February 20, 2021 from https://www.research.ibm.com/artificial-intellig ence/trusted-ai/.
- Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International Conference on Machine Learning. PMLR, 448–456.
- Shouling Ji, Weiqing Li, Prateek Mittal, Xin Hu, and Raheem Beyah. 2015. \(\lbrace\) SecGraph \(\rbrace\) : A uniform and open-source evaluation system for graph data anonymization and de-anonymization. In Proceedings of the 24th USENIX Security Symposium (USENIX Security’15). 303–318.
- Gregory Koch, Richard Zemel, and Ruslan Salakhutdinov. 2015. Siamese neural networks for one-shot image recognition. In ICML Deep Learning Workshop, Vol. 2.
- Pang Wei Koh and Percy Liang. 2017. Understanding black-box predictions via influence functions. In International Conference on Machine Learning. PMLR, 1885–1894.
- Klas Leino, Zifan Wang, and Matt Fredrikson. 2021. Globally-robust neural networks. In International Conference on Machine Learning. PMLR, 6212–6222.
- Ke Li and Jitendra Malik. 2017. Learning to optimize. In International Conference on Learning Representations .
- Yiming Li Yong Jiang Zhifeng Li and Shu-Tao Xia. 2017. Backdoor learning: A survey. IEEE Transactions on Neural Networks and Learning Systems .
- Kang Liu, Brendan Dolan-Gavitt, and Siddharth Garg. 2018. Fine-pruning: Defending against backdooring attacks on deep neural networks. In International Symposium on Research in Attacks, Intrusions, and Defenses. Springer, 273–294.
- Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. 2018. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations.
- Pratyush Maini, Eric Wong, and Zico Kolter. 2020. Adversarial robustness against the union of multiple perturbation models. In International Conference on Machine Learning. PMLR, 6640–6650.
- Andre Martins and Ramon Astudillo. 2016. From softmax to sparsemax: A sparse model of attention and multi-label classification. In International Conference on Machine Learning. PMLR, 1614–1623.
- Jan Hendrik Metzen Tim Genewein Volker Fischer and Bastian Bischoff. 2017. On detecting adversarial perturbations. In 5th International Conference on Learning Representations (ICLR’17) Toulon France April 24-26 2017 Conference Track Proceedings .
- Silvio Micali, Oded Goldreich, and Avi Wigderson. 1987. How to play any mental game. In Proceedings of the 19th ACM Symposium on Theory of Computing (STOC’87). ACM, 218–229.
- Christoph Molnar. 2020. Interpretable Machine Learning. Retrieved from Lulu.com.
- OpenAI. 2018. OpenAI Charter. Retrieved February 20, 2021 from https://openai.com/charter/.
- Andrea Paudice, Luis Muñoz-González, and Emil C. Lupu. 2018. Label sanitization against label flipping poisoning attacks. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer, 5–15.
- Aditi Raghunathan, Jacob Steinhardt, and Percy Liang. 2018. Certified defenses against adversarial examples. In Proceedings of the International Conference on Learning Representations.
- Aditi Raghunathan Sang Michael Xie Fanny Yang John C. Duchi and Percy Liang. 2019. Adversarial training can hurt generalization. In ICML 2019 Workshop Deep Phenomena .
- Yuji Roh, Kangwook Lee, Steven Whang, and Changho Suh. 2020. FR-train: A mutual information-based approach to fair and robust training. In International Conference on Machine Learning. PMLR, 8147–8157.
- Christian Sandvig, Kevin Hamilton, Karrie Karahalios, and Cedric Langbort. 2014. Auditing algorithms: Research methods for detecting discrimination on internet platforms. In Proceedings of the Data and Discrimination: Converting Critical Concerns Into Productive Inquiry 22 (2014), 4349–4357.
- Adam Santoro, Sergey Bartunov, Matthew Botvinick, Daan Wierstra, and Timothy Lillicrap. 2016. Meta-learning with memory-augmented neural networks. In International Conference on Machine Learning. PMLR, 1842–1850.
- Sebastian Schelter, Joos-Hendrik Boese, Johannes Kirschnick, Thoralf Klein, and Stephan Seufert. 2017. Automatically tracking metadata and provenance of machine learning experiments. In Machine Learning Systems Workshop at NIPS. 27–29.
- Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. 2017. Learning important features through propagating activation differences. In International Conference on Machine Learning. PMLR, 3145–3153.
- Simone Stumpf, Adrian Bussone, and Dympna O’sullivan. 2016. Explanations considered harmful? user interactions with machine learning systems. In Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems (CHI).
- Weijie Su, Xizhou Zhu, Yue Cao, Bin Li, Lewei Lu, Furu Wei, and Jifeng Dai. 2019. VL-BERT: Pre-training of generic visual-linguistic representations. In Proceedings of the International Conference on Learning Representations.
- Mukund Sundararajan, Ankur Taly, and Qiqi Yan. 2017. Axiomatic attribution for deep networks. In International Conference on Machine Learning. PMLR, 3319–3328.
- Supreme Audit Institutions of Finland, Germany, the Netherlands, Norway and the UK. 2020. Auditing Machine Learning Algorithms. Retrieved from https://www.auditingalgorithms.net/index.html.
- Liang Tong, Bo Li, Chen Hajaj, Chaowei Xiao, Ning Zhang, and Yevgeniy Vorobeychik. 2019. Improving robustness of \(\lbrace\) ML \(\rbrace\) classifiers against realizable evasion attacks using conserved features. In Proceedings of the 28th USENIX Security Symposium (USENIX Security’19). 285–302.
- Cecília Torres, Walter Franklin, and Laura Martins. 2018. Accessibility in chatbots: The state of the art in favor of users with visual impairment. In International Conference on Applied Human Factors and Ergonomics. Springer, 623–635.
- Florian Tramèr, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Boneh, and Patrick McDaniel. 2018. Ensemble adversarial training: Attacks and defenses. In Proceedings of the International Conference on Learning Representations.
- Florian Tramèr, Fan Zhang, Ari Juels, Michael K. Reiter, and Thomas Ristenpart. 2016. Stealing machine learning models via prediction \(\lbrace\) APIs \(\rbrace\) . In Proceedings of the 25th USENIX Security Symposium (USENIX Security’16). 601–618.
- Dimitris Tsipras Shibani Santurkar Logan Engstrom Alexander Turner and Aleksander Ma̧dry. 2019. Robustness may be at odds with accuracy. In 7th International Conference on Learning Representations (ICLR’19) New Orleans LA USA May 6-9 2019 .
- UNI Global Union. 2017. Top 10 principles for ethical artificial intelligence. In The Future World of Work.
- Yisen Wang, Xingjun Ma, James Bailey, Jinfeng Yi, Bowen Zhou, and Quanquan Gu. 2019. On the convergence and robustness of adversarial training. In Proceedings of the International Conference on Machine Learning (ICML’19), Vol. 1. 2.
- Yingxu Wang and Jingqiu Shao. 2003. Measurement of the cognitive functional complexity of software. In Proceedings of the 2nd IEEE International Conference on Cognitive Informatics. IEEE, 67–74.
- Tsui-Wei Weng, Huan Zhang, Pin-Yu Chen, Jinfeng Yi, Dong Su, Yupeng Gao, Cho-Jui Hsieh, and Luca Daniel. 2018. Evaluating the robustness of neural networks: An extreme value theory approach. In Proceedings of the International Conference on Learning Representations.
- Tong Wu Liang Tong and Yevgeniy Vorobeychik. 2020. Defending against physically realizable attacks on image classification. In 8th International Conference on Learning Representations (ICLR’20) Addis Ababa Ethiopia April 26-30 2020 .
- Weiyi Xia, Yongtai Liu, Zhiyu Wan, Yevgeniy Vorobeychik, Murat Kantacioglu, Steve Nyemba, Ellen Wright Clayton, and Bradley A. Malin. 2021. Enabling realistic health data re-identification risk assessment through adversarial modeling. J. Am. Med. Inf. Assoc. 28, 4 (2021), 744–752.
- Han Xu, Xiaorui Liu, Yaxin Li, Anil Jain, and Jiliang Tang. 2021. To be robust or to be fair: Towards fairness in adversarial training. In International Conference on Machine Learning. PMLR, 11492–11501.
- Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhudinov, Rich Zemel, and Yoshua Bengio. 2015. Show, attend and tell: Neural image caption generation with visual attention. In International Conference on Machine Learning. PMLR, 2048–2057.
- 10.5555/1382436.1382751 · doi:10.5555/1382436.1382751
- Ting Yao, Yiheng Zhang, Zhaofan Qiu, Yingwei Pan, and Tao Mei. 2021. Seco: Exploring sequence supervision for unsupervised representation learning. In AAAI, Vol. 2. 7.
- Matei Zaharia, Andrew Chen, Aaron Davidson, Ali Ghodsi, Sue Ann Hong, Andy Konwinski, Siddharth Murching, Tomas Nykodym, Paul Ogilvie, Mani Parkhe, et al. 2018. Accelerating the machine learning lifecycle with MLflow. IEEE Data Eng. Bull. 41, 4 (2018), 39–45.
- Rich Zemel, Yu Wu, Kevin Swersky, Toni Pitassi, and Cynthia Dwork. 2013. Learning fair representations. In International Conference on Machine Learning. PMLR, 325–333.
Cited by
Augmenting large language models with chemistry tools
M. Bran, A., Cox, S., Schilter, O., Baldassari, C., White, A. D., & Schwaller, P. (2024) · Nature Machine Intelligence · doi:10.1038/s42256-024-00832-8 · Cited by 794
Interpretable deep learning: interpretation, interpretability, trustworthiness, and beyond
Li, X., Xiong, H., Li, X., Wu, X., Zhang, X., Liu, J., Bian, J., & Dou, D. (2022) · Knowledge and Information Systems · doi:10.1007/s10115-022-01756-8 · Cited by 497
“Researchers [29, 47, 81, 101, 107, 183] also proposed many other desiderata for interpretations, interpretation algorithms or interpretability, such as fairness, privacy, reliability, robustness, causality, trust, fidelity, faithfulness, transferability, informativeness, transparency, plausibility, satisfaction, accou…”
BackgroundSelf-citationEsmaeilzadeh, P. (2024) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2024.102861 · Cited by 377
Joyce, D. W., Kormilitzin, A., Smith, K. A., & Cipriani, A. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00751-9 · Cited by 238
Post-hoc vs ante-hoc explanations: xAI design guidelines for data scientists
Retzlaff, C. O., Angerschmid, A., Saranti, A., Schneeberger, D., Röttger, R., Müller, H., & Holzinger, A. (2024) · Cognitive Systems Research · doi:10.1016/j.cogsys.2024.101243 · Cited by 194
“In any case, trustworthy AI encompasses elements of security (Li et al., 2023), safety, privacy, fairness (Angerschmid et al., 2022), accountability, auditability, and robustness (Holzinger, Dehmer et al., 2022).”
Neuro-symbolic artificial intelligence: a survey
Bhuyan, B. P., Ramdane-Cherif, A., Tomar, R., & Singh, T. P. (2024) · Neural Computing and Applications · doi:10.1007/s00521-024-09960-z · Cited by 190
“These AI models may also lack the trans-parency and explainability of other forms of AI due to the complexity involved in understanding how they arrive at their predictions or choices [8].”
The METRIC-framework for assessing data quality for trustworthy AI in medicine: A systematic review
Schwabe, D., Becker, K., Seyferth, M., Klaß, A., & Schaeffter, T. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01196-4 · Cited by 169
Sensing and Machine Learning for Automotive Perception: A Review
Pandharipande, A., Cheng, C.-H., Dauwels, J., Gurbuz, S. Z., Ibanez-Guzman, J., Li, G., Piazzoni, A., Wang, P., & Santra, A. (2023) · IEEE Sensors Journal · doi:10.1109/jsen.2023.3262134 · Cited by 161
“Aspects such as explainability and reproducibility of ML processing become critical in ADAS/ADS and, given their safety-critical nature, remain a challenge [194], [195].”
BackgroundA review of uncertainty estimation and its application in medical imaging
Zou, K., Chen, Z., Yuan, X., Shen, X., Wang, M., & Fu, H. (2023) · Meta-Radiology · doi:10.1016/j.metrad.2023.100003 · Cited by 158
Li, Y., Wu, B., Huang, Y., & Luan, S. (2024) · Frontiers in Psychology · doi:10.3389/fpsyg.2024.1382693 · Cited by 159
Trustworthy Graph Neural Networks: Aspects, Methods, and Trends
Zhang, H., Wu, B., Yuan, X., Pan, S., Tong, H., & Pei, J. (2024) · Proceedings of the IEEE · doi:10.1109/jproc.2024.3369017 · Cited by 144
“For example, trustworthy systems should be able to obtain knowledge from limited training data and achieve convincing competence (i.e., high accuracy) on unseen data [11].”
BackgroundInfluentialSelf-citationOniani, D., Hilsman, J., Peng, Y., Poropatich, R. K., Pamplin, J. C., Legault, G. L., & Wang, Y. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00965-x · Cited by 143
“This high-level understanding will help optimize trust, as operators will be aware of the capabilities and limitations of the AI systems they work with and know the appropriate settings for use 104 .”
BackgroundHow to teach responsible AI in Higher Education: challenges and opportunities
Aler Tubella, A., Mora-Cantallops, M., & Nieves, J. C. (2024) · Ethics and Information Technology · doi:10.1007/s10676-023-09733-7 · Cited by 131
“New recommendations regarding Trustworthy AI are appearing in the literature; nevertheless, most of those newer recommendations identify AI principles that are either a subset or an extension of the AI principles introduced by the EC (Li et al., 2023).”
Laine, J., Minkkinen, M., & Mäntymäki, M. (2024) · Information & Management · doi:10.1016/j.im.2024.103969 · Cited by 222
“Alongside the concepts of trustworthy AI [63,81] and AI governance [13,92,105], the auditing of AI systems has been put forth as one response to these challenges [16,102,128].”
Liesenfeld, A., Lopez, A., & Dingemanse, M. (2023) · Proceedings of the 5th International Conference on Conversational User Interfaces · doi:10.1145/3571884.3604316 · Cited by 111
“Despite strong evidence of the scientific and engineering benefits of open research practices, openness is not a given in machine learning and AI research [18, 20, 30].”
BackgroundFault Detection for Point Machines: A Review, Challenges, and Perspectives
Hu, X., Tang, T., Tan, L., & Zhang, H. (2023) · Actuators · doi:10.3390/act12100391 · Cited by 132
Toward Trustworthy Artificial Intelligence (TAI) in the Context of Explainability and Robustness
chander, B., John, C., Warrier, L., & Gopalakrishnan, K. (2025) · ACM Computing Surveys · doi:10.1145/3675392 · Cited by 108
“Proving malicious data into an AI system during training and upda ting may alter the system‟s behavior.”
Advanced Deep Learning Models for 6G: Overview, Opportunities, and Challenges
Jiao, L., Shao, Y., Sun, L., Liu, F., Yang, S., Ma, W., Li, L., Liu, X., Hou, B., Zhang, X., et al. (2024) · IEEE Access · doi:10.1109/access.2024.3418900 · Cited by 147
Automated detection and forecasting of COVID-19 using deep learning techniques: A review
Shoeibi, A., Khodatars, M., Jafari, M., Ghassemi, N., Sadeghi, D., Moridian, P., Khadem, A. A., Alizadehsani, R., Hussain, S., Zare, A., et al. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.127317 · Cited by 329
Tariq, A., Serhani, M. A., Sallabi, F., Barka, E. S., Qayyum, T., Khater, H. M., & Shuaib, K. (2024) · IEEE Open Journal of the Communications Society · doi:10.1109/ojcoms.2024.3438264 · Cited by 153
Aldoseri, A., Al‐Khalifa, K. N., & Hamouda, A. M. S. (2024) · Applied System Innovation · doi:10.3390/asi7010014 · Cited by 128
“This includes skills in programing languages such as Python or R, understanding machine learning algorithms, statistical analysis, data manipulation, and data visualization [87,88].”
Diffusion model-driven smart design and manufacturing: Prospects and challenges
Leng, J., Su, X., Liu, Z., Zhou, L., Chen, C., Guo, X., Wang, Y., Wang, R., Zhang, C., Liu, Q., et al. (2025) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2025.07.011 · Cited by 101
Alsalem, M. A., Alamoodi, A. H., Albahri, O. S., Albahri, A. S., Martínez, L. M., Yera, R., Duhaim, A. M., & Sharaf, I. M. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.123066 · Cited by 95
Wang, X., Wang, B., Wu, Y., Ning, Z., Guo, S., & Yu, F. R. (2024) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2024.3446585 · Cited by 127
“Authors in [19] provide various requirements for ensuring the trust-worthiness of AI, along with corresponding methods, all from a human-centered perspective.”
Evaluating Trustworthiness in AI: Risks, Metrics, and Applications Across Industries
Nastoska, A., Jancheska, B., Rizinski, M., & Trajanov, D. (2025) · Electronics · doi:10.3390/electronics14132717 · Cited by 167
Mei, S., Lian, J., Wang, X., Su, Y., Ma, M., & Chau, L. (2024) · Journal of Remote Sensing · doi:10.34133/remotesensing.0219 · Cited by 95
Kong, S.-C., Lee, J. C.-K., & Tsang, O. (2024) · Research and Practice in Technology Enhanced Learning · doi:10.58459/rptel.2024.19030 · Cited by 69
Explainable machine learning for public policy: Use cases, gaps, and research directions
Amarasinghe, K., Rodolfa, K. T., Lamba, H., & Ghani, R. (2023) · Data & Policy · doi:10.1017/dap.2023.2 · Cited by 73
“Furthermore, there has been significant emphasis laid on developing “trustworthy ML” systems of which explainable ML is considered to be a key element (Li et al., 2022).”
BackgroundFederatedTrust: A solution for trustworthy federated learning
Sánchez, P. M. S., Celdrán, A. H., Xie, N., Bovet, G., Pérez, G. M., & Stiller, B. (2023) · Future Generation Computer Systems · doi:10.1016/j.future.2023.10.013 · Cited by 103
Benk, M., Kerstan, S., von Wangenheim, F., & Ferrario, A. (2024) · AI & Society · doi:10.1007/s00146-024-02059-y · Cited by 122
“4 In particular, there is a growing interest in defining what makes the trustee in an interaction, that is, the AI system, worthy of trust (Jobin et al. 2019; Li et al. 2023; Petersen et al. 2022; Kaur et al. 2022).”
Establishing and evaluating trustworthy AI: overview and research challenges
Kowald, D., Scher, S., Pammer-Schindler, V., Müllner, P., Waxnegger, K., Demelius, L., Fessl, A., Toller, M., Mendoza Estrada, I. G., Šimić, I., et al. (2024) · Frontiers in Big Data · doi:10.3389/fdata.2024.1467222 · Cited by 59
“…last years, various frameworks, guidelines, survey articles, and collections of requirements of trustworthy AI have been developed and published by researchers, governments, and private organizations (Smuha, 2019; Kaur et al., 2021; Floridi, 2021; Kaur et al., 2022; Li et al., 2023; Yeung, 2020).”
Niri, M. F., Aslansefat, K., Haghi, S., Hashemian, M., Daub, R., & Marco, J. (2023) · Energies · doi:10.3390/en16176360 · Cited by 71
“Addressing the example techniques in Figure 1 is out of the scope of this work and readers are invited to review the relevant resources such as [6,7] for further information.”
Making It Possible for the Auditing of AI: A Systematic Review of AI Audits and AI Auditability
Li, Y., & Goel, S. (2024) · Information Systems Frontiers · doi:10.1007/s10796-024-10508-8 · Cited by 103
“Previous research sees auditability of AI systems as a dimension of accountability (AI HLEG, 2019; Li et al., 2023; Liu et al., 2022; Salvi del Pero et al., 2022; Wier-inga, 2020).”
InfluentialOn TinyML and Cybersecurity: Electric Vehicle Charging Infrastructure Use Case
Dehrouyeh, F., Yang, L., Ajaei, F. B., & Shami, A. A. (2024) · IEEE Access · doi:10.1109/access.2024.3437192 · Cited by 85
“Such an all-encompassing strategy is vital for aligning AI with ethical norms and societal expectations, essential for improving trust among users and stakeholders [122].”
BackgroundPhysics-enhanced deep surrogates for partial differential equations
Pestourie, R., Mroueh, Y., Rackauckas, C., Das, P., & Johnson, S. G. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00761-y · Cited by 95
“We extensively compared PEDS against traditional surrogate models and found that they outperformed all models for more complex problems (Supplementary Information, Comparison to mainstream surrogate models).”
Trustworthiness Assurance Assessment for High-Risk AI-Based Systems
Stettinger, G., Weissensteiner, P., & Khastgir, S. (2024) · IEEE Access · doi:10.1109/access.2024.3364387 · Cited by 90
“The foundations of Trustworthy AI are based on fundamental rights as moral and legal entitlements [24], [25] and are reflected in four ethical principles, namely Authorized licensed use limited to the terms of the applicable license agreement with IEEE.”
BackgroundManaging workplace AI risks and the future of work
Howard, J., & Schulte, P. A. (2024) · American Journal of Industrial Medicine · doi:10.1002/ajim.23653 · Cited by 63
Domain Generalization in Computational Pathology: Survey and Guidelines
Jahanifar, M., Raza, M., Xu, K., Vuong, T. T. L., Jewsbury, R., Shephard, A., Tajeddin, N. Z., Kwak, J. T., Raza, S. E. A., Minhas, F., et al. (2025) · ACM Computing Surveys · doi:10.1145/3724391 · Cited by 105
Schiller, J., Stiller, S., & Ryo, M. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11165-2 · Cited by 78
“Given the high variability in their needs, along with technical and societal changes, the design for AI to be human-centered and trustworthy must be adjusted iteratively and with a participatory approach and extends beyond providing model interpretations (Bostrom et al. 2024; Li et al. 2023a, b).”
InfluentialTowards Trustworthy Machine Learning in Production: An Overview of the Robustness in MLOps Approach
Bayram, F., & Ahmed, B. S. (2025) · ACM Computing Surveys · doi:10.1145/3708497 · Cited by 48
“These AI systems, equipped with various toolkits, offer a wide range of solutions that can be adopted to help decision makers take rapid action in various applications [3].”
Trustworthy Artificial Intelligence in Dentistry: Learnings from the EU AI Act
Ducret, M., Wahal, E., Gruson, D., Amrani, S., Richert, R., Mouncif-Moungache, M., & Schwendicke, F. (2024) · Journal of Dental Research · doi:10.1177/00220345241271160 · Cited by 64
“Further aspects of human-centeredness include diversity and fairness (Li et al. 2023).”
BackgroundAli, W., Zhou, X., & Shao, J. (2024) · ACM Computing Surveys · doi:10.1145/3708982 · Cited by 70
“This could have a number of negative consequences, such as discrimination, targeted advertising, and identity theft [48, 88].”
LEGAL CHALLENGES OF ARTIFICIAL INTELLIGENCE AND ROBOTICS: A COMPREHENSIVE REVIEW
Akpuokwe, C. U., Adeniyi, A. O., Bakare, S. S., & Eneh, N. E. (2024) · Computer Science & IT Research Journal · doi:10.51594/csitrj.v5i3.860 · Cited by 73
Madathil, N. T., Dankar, F. K., Gergely, M., Belkacem, A. N., & Alrabaee, S. (2025) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2025.06.009 · Cited by 76
“Furthermore, configuring DP parameters is often challenging [128,129], but the DP approach is generally favored for its lower computation overhead compared to other privacy mechanisms [130–132].”
Anomaly diagnosis of connected autonomous vehicles: A survey
Fang, Y., Min, H., Wu, X., Wang, W., Zhao, X., Martínez‐Pastor, B., & Teixeira, R. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102223 · Cited by 57
Trust and Trustworthiness in AI
Durán, J. M., & Pozzi, G. (2025) · Philosophy & Technology · doi:10.1007/s13347-025-00843-2 · Cited by 47
“The general principles for trustworthy AI outlined by the EU Commission (2019), echoed throughout the specialized literature (e.g., Li et al., 2023; Kaur et al., 2022), advocate for caution and the pursuit of robust solutions.”
Trustworthy AI for human-centric smart manufacturing: A survey
Li, D., Liu, S., Wang, B., Yu, C., Zheng, P., & Li, W. (2024) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2024.11.020 · Cited by 59
Wang, X., Zhuang, B., Smyl, D., Zhou, H., & Naser, M. Z. (2025) · Engineering Structures · doi:10.1016/j.engstruct.2025.119652 · Cited by 50
Holding AI-Based Systems Accountable in the Public Sector: A Systematic Review
Yuan, Q., & Chen, T. (2025) · Public Performance & Management Review · doi:10.1080/15309576.2025.2469784 · Cited by 70
“Moreover, both literatures adopt a life-cycle perspective on AI-based systems, stressing the necessity of implementing accountability measures throughout the stages of design, development, and deployment (Li et al., 2023; Saeed & Omlin, 2023).”
Assessing trustworthy AI: Technical and legal perspectives of fairness in AI
Kattnig, M., Angerschmid, A., Reichel, T., & Kern, R. (2024) · Computer law & security review · doi:10.1016/j.clsr.2024.106053 · Cited by 77
Security, privacy, and robustness for trustworthy AI systems: A review
Saeed, M. M., & Alsharidah, M. (2024) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2024.109643 · Cited by 54
Editorial: Human-Centered Artificial Intelligence in Industry 5.0
Mentzas, G. N., Hribernik, K., Stahre, J., Romero, D., & Soldatos, J. K. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1429186 · Cited by 68
“As AI systems become increasingly autonomous, it becomes imperative to demystify their inner workings and ensure that they align with high ethical and moral standards within trustworthy systems (Díaz-Rodríguez et al., 2023; Li et al., 2023; Mentzas et al., 2024).”
Re-evaluating GPT-4’s bar exam performance
Martínez, E. (2025) · Artificial Intelligence and Law · doi:10.1007/s10506-024-09396-9 · Cited by 54
“To the extent that transparency is important to ensuring the safe deployment of AI, a lack of transparency could undermine our confidence in the prospect of safe deployment of AI (Brundage et al. 2020; Li et al. 2023).”
Wang, S., Zhang, Y., Xiao, Y., & Liang, Z. (2025) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2025.123971 · Cited by 73
Design and Evaluation of Trustworthy Knowledge Tracing Model for Intelligent Tutoring System
Lu, Y., Wang, D., Chen, P., & Zhang, Z. (2024) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2024.3403135 · Cited by 52
“Trustworthy AI was initially introduced in the general AI domain, with the main focus being on how AI can achieve its maximum potential by establishing trust at every stage of its life cycle, including design, development, deployment, and use [4].”
BackgroundExplainable Machine Learning in Critical Decision Systems: Ensuring Safe Application and Correctness
Wiggerthale, J., & Reich, C. (2024) · AI · doi:10.3390/ai5040138 · Cited by 61
Jiang, J., Zhao, H., Li, S., Jin, B., Wang, R., Wei, Y., Li, H., & Dai, Y. (2025) · ACS Nano · doi:10.1021/acsnano.5c06265 · Cited by 37
Lalor, J. P., Abbasi, A., Oketch, K., Yang, Y., & Forsgren, N. (2024) · ACM Transactions on Information Systems · doi:10.1145/3641276 · Cited by 58
Ouifak, H., & Idri, A. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.130602 · Cited by 48
Machine learning based eddy current testing: A review
Munir, N., Huang, J., Wong, C., & Song, S.-J. (2024) · Results in Engineering · doi:10.1016/j.rineng.2024.103724 · Cited by 51
Enabling AutoML for Zero-Touch Network Security: Use-Case Driven Analysis
Yang, L., Rajab, M. E., Shami, A. A., & Muhaidat, S. (2024) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2024.3376631 · Cited by 55
“Key aspects of AI trustworthiness include robustness, reproducibility, explainability, privacy, accountability, and fairness [103].”
Rehman, H. U., Jamil, M., Wang, N., et al. (2024) · Materials Today Communications · doi:10.1016/j.mtcomm.2024.109806 · Cited by 30
A Systematic Review of Responsible Artificial Intelligence Principles and Practice
Gunasekara, L., El-Haber, N., Nagpal, S., Moraliyage, H., Issadeen, Z., Manic, M., & Silva, D. D. (2025) · Applied System Innovation · doi:10.3390/asi8040097 · Cited by 67
Miedema, E., Waschull, S., & Emmanouilidis, C. (2025) · Computers in Industry · doi:10.1016/j.compind.2025.104409 · Cited by 48
Wang, M., Yang, N., Gunasinghe, D. H., & Weng, N. (2023) · Computers · doi:10.3390/computers12100209 · Cited by 46
Security Risk and Attacks in AI: A Survey of Security and Privacy
Rahman, M. M., Siddika, A., Hasan, M. M., Mishu, S. F., Shahriar, H., & Wu, F. (2023) · IEEE Annual International Computer Software and Applications Conference (COMPSAC) · doi:10.1109/compsac57700.2023.00284 · Cited by 48
“Furthermore, they must ensure that their AI systems are transparent and subject to oversight, conforming to privacy laws and regulations and adopting frameworks like trustworthy AI [3].”
BackgroundMethodTransparency and Accountability in Explainable AI: Best Practices
Akhtar, M. A. K., Kumar, M., & Nayyar, A. (2024) · Studies in systems, decision and control · doi:10.1007/978-3-031-66489-2_5 · Cited by 53
From Efficiency to Innovation: LLM 5.0 and the Future of Industry
Gantayat, P. K., Kaur, T., Majhi, M., Jena, U. K., Das, S., & Soni (2025) · International Conference on Multi-Agent Systems for Collaborative Intelligence (ICMSCI) · doi:10.1109/icmsci62561.2025.10894335 · Cited by 33
“Automobile industry: Cobots are used by automakers such as BMW to assemble cars, freeing up workers to focus on quality assurance and optimization [10].”
Semantic Communications Using Foundation Models: Design Approaches and Open Issues
Jiang, P., Wen, C.-K., Yi, X., Li, X., Jin, S., & Zhang, J. (2024) · IEEE Wireless Communications · doi:10.1109/mwc.002.2300460 · Cited by 39
“Further research should focus on intelligently harnessing the capabilities of big models.”
BackgroundState-of-the-Art in Responsible, Explainable, and Fair AI for Medical Image Analysis
Amirian, S., Gao, F., Littlefield, N. G., Hill, J. H., Yates, A. J., Plate, J. F., Pantanowitz, L., Rashidi, H. H., & Tafti, A. P. (2025) · IEEE Access · doi:10.1109/access.2025.3555543 · Cited by 37
“Reliability in REF-AI systems in healthcare means consistently performing as expected across diverse healthcare settings and scenarios, managing uncertainties and errors, and preventing breakdowns or performance issues to maintain core functionalities [101], [102].”
Optimising the use of electronic medical records for large scale research in psychiatry
Newby, D., Taylor, N., Joyce, D. W., & Winchester, L. M. (2024) · Translational Psychiatry · doi:10.1038/s41398-024-02911-1 · Cited by 39
“If clinicians and patients are to trust an AI/ML model, they will likely favour model transparency and simplicity — often described as intrinsically interpretable models [83] — over the often modest performance gains given by complex DL models [84].”
Backgroundde Campos, L. M., Fernández-Luna, J. M., & Huete, J. F. (2024) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-024-09400-6 · Cited by 41
“Recognizing the significance of explainable AI both in a general context (Minh et al. 2022; Li et al. 2023; Barredo-Arrieta et al. 2020 and specifically within RS (Pavitha et al. 2022; Zhang and Chen 2020), the provision of explanations for recommended content serves multiple purposes.”
Ethics of trustworthy AI in healthcare: Challenges, principles, and practical pathways
Ahadian, P., Xu, W. F., Liu, D., & Guan, Q. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.131942 · Cited by 43
Explainable artificial intelligence in ophthalmology
Tan, T. F., Dai, P., Zhang, X., Jin, L., Poh, S. S. J., Hong, D. C., Lim, J., Lim, G., Teo, Z. L., Liu, N., et al. (2023) · Current Opinion in Ophthalmology · doi:10.1097/icu.0000000000000983 · Cited by 29
Zheng, Y., Chang, C.-H., Huang, S.-H., Chen, P., & Picek, S. (2024) · IEEE Journal on Emerging and Selected Topics in Circuits and Systems · doi:10.1109/jetcas.2024.3477348 · Cited by 47
“Human-annotated datasets are insufficient for comprehensively testing a machine learning system [233].”
A model-agnostic, network theory-based framework for supporting XAI on classifiers
Bonifazi, G., Cauteruccio, F., Corradini, E., Marchetti, M., Terracina, G., Ursino, D., & Virgili, L. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.122588 · Cited by 25
Yang, L., Allen, G., Zhang, Z., & Zhao, Y. (2024) · Buildings · doi:10.3390/buildings15010021 · Cited by 33
Towards a Better Understanding of Evaluating Trustworthiness in AI Systems
Kemmerzell, N., Schreiner, A., Khalid, H., Schalk, M., & Bordoli, L. (2025) · ACM Computing Surveys · doi:10.1145/3721976 · Cited by 45
Akbasli, I. T., Bırbılen, A. Z., & Tekşam, Ö. (2025) · JAMIA Open · doi:10.1093/jamiaopen/ooae138 · Cited by 33
“42,43 By integrating automation, Continuous Integration (CI), and Continuous Delivery (CD), MLOps facilitates efficient deployment and sustained accuracy.”
Re-evaluating GPT-4's bar exam performancePreprint
Martínez, E. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4441311 · Cited by 22
Kang, Y., Gu, H., Tang, X., He, Y., Zhang, Y., He, J., Han, Y., Fan, L., Chen, K., & Yang, Q. (2024) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3701039 · Cited by 49
Trustworthy AI-based Performance Diagnosis Systems for Cloud Applications: A Review
Xin, R., Wang, J., Chen, P., & Zhao, Z. (2025) · ACM Computing Surveys · doi:10.1145/3701740 · Cited by 30
“For data collection and preprocessing, methods must be in place to ensure the privacy and security of sensitive information [104].”
InfluentialA Theoretical Framework for AI Models Explainability with Application in Biomedicine
Rizzo, M., Veneri, A., Albarelli, A., Lucchese, C., Nobile, M. S., & Conati, C. (2023) · IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB) · doi:10.1109/cibcb56990.2023.10264877 · Cited by 30
“A design-oriented approach to AI development should involve taking humans into the loop, thus fostering a human-centered AI which is more intelligible by design and is expected to increase trust in the end-users [24].”
BackgroundThe audit revolution: Integrating artificial intelligence in detecting accounting fraud
Supriadi, I. (2024) · Akuntansi dan Teknologi Informasi · doi:10.24123/jati.v17i1.6279 · Cited by 61
“Although AI has provided some advantages in auditing, the use of AI technology in the audit process also leads to some issues, such as the lack of public trust in the audit results produced by AI (Li et al., 2021).”
ResultMachine learning robustness: a primer
Braiek, H. B., & Khomh, F. (2025) · Elsevier eBooks · doi:10.1016/b978-0-44-323761-4.00012-2 · Cited by 78
Measuring the Interpretability and Explainability of Model Decisions of Five Large Language ModelsPreprint
Fujiwara, K., Sasaki, M., Nakamura, A., & Watanabe, N. (2024) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/d4ntw · Cited by 63
Rule-Based Detection of Anomalous Patterns in Device Behavior for Explainable IoT Security
Costa, G., Forestiero, A., & Ortale, R. (2023) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2023.3327822 · Cited by 24
“Recently, several approaches have been devised in the area of explainable artificial intelligence [12], [13], [14], [15], [16], [17], [18], [19], [20], [21], [22] to approach cybersecurity [23], [24], [25] from various viewpoints across distinct domains.”
BackgroundTrustworthy machine learning in the context of security and privacy
Upreti, R., Lind, P. G., Elmokashfi, A. M., & Yazidi, A. (2024) · International Journal of Information Security · doi:10.1007/s10207-024-00813-3 · Cited by 31
“Most of the trustworthy AI frameworks are in the immature phase [114].”
Chettri, S. K., Deka, R. K., & Saikia, M. J. (2025) · AI · doi:10.3390/ai6010010 · Cited by 35
Fu, K., Ye, C., Wang, Z., Wu, M., Liu, Z., & Yuan, Y. (2025) · Frontiers in Digital Health · doi:10.3389/fdgth.2025.1618169 · Cited by 24
Badawy, W. (2023) · AI and Ethics · doi:10.1007/s43681-023-00376-4 · Cited by 26
FVFL: A Flexible and Verifiable Privacy-Preserving Federated Learning Scheme
Wang, G., Zhou, L., Li, Q., Yan, X., Liu, X., & Wu, Y. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3385479 · Cited by 25
“W ITH the emergence of artificial intelligence technology, the processing and operation of big data has become very efficiency and simple, and has been widely used, such as face recognition, online medical diagnosis, and voice recognition etc. [1].”
MethodInsuasti, J., Roa, F., & Jaramillo, C. M. Z. (2023) · Big Data and Cognitive Computing · doi:10.3390/bdcc7040182 · Cited by 23
“This process is vital to yield AI applications that are ethical and fair, reducing the risk of unintended consequences arising from biased predictions [36].”
BackgroundExploring the landscape of trustworthy artificial intelligence: Status and challenges
Mentzas, G. N., Fikardos, M., Lepenioti, K., & Apostolou, D. (2024) · Intelligent Decision Technologies · doi:10.3233/idt-240366 · Cited by 25
“For example, [105] reviews tools and methods in order to help translate principles into practice, while [86] introduces a framework that consolidates the existing fragmented approaches to trustworthy AI into a unified, systematic approach.”
Naik, D. S. B., & Dondeti, V. (2025) · Peer-to-Peer Networking and Applications · doi:10.1007/s12083-024-01835-3 · Cited by 22
FairDgcl: Fairness-Aware Recommendation With Dynamic Graph Contrastive Learning
Chen, W., Yuan, M., Zhang, Z., Xie, R., Zhuang, F., Wang, D., & Liu, R. (2025) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2025.3580087 · Cited by 44
“As machine learning systems increasingly impact real-world applications, the importance of developing trustworthy AI has become paramount [27], [28], [29].”
Salazar, T., Araújo, H. J., Cano, A., & Abreu, P. H. (2026) · Artificial Intelligence Review · doi:10.1007/s10462-025-11475-5 · Cited by 38
Pierre, K., Gupta, M., Raviprasad, A. G., Razavi, S. M. S., Patel, A. B., Peters, K., Hochhegger, B., Mancuso, A., & Forghani, R. (2023) · Expert Review of Anticancer Therapy · doi:10.1080/14737140.2023.2286001 · Cited by 19
“Instead, it should encompass the entire lifecycle of an AI product that includes data preparation, algorithmic design, development, deployment, operation, monitoring, and governance [116].”
BackgroundMethodEyes in the Sky: Privacy and Ethical Considerations in Drone Cybersecurity
Sindiramutty, S. R., Tan, C. E., & Goh, W. W. (2024) · Advances in information security, privacy, and ethics book series · doi:10.4018/979-8-3693-0774-8.ch017 · Cited by 26
Alelyani, T. (2024) · Frontiers in Digital Health · doi:10.3389/fdgth.2024.1474692 · Cited by 22
Metadata
- Authors
- Li, B., Qi, P., Liu, B., Di, S., Liu, J., Pei, J., Yi, J., & Zhou, B.
- Affiliations
- Tsinghua University (CN)
- Amazon (United States) (US)
- Walmart Inc., Mountain View, CA, USA
- Jingdong (China) (CN)
- JD Technology, Mountain View, CA, USA
- Frontis.AI, Beijing, China
- Venue
- ACM Computing Surveys
- Published
- 2023-01-16
- Volume
- 55
- Issue
- 9
- Pages
- 1-46
- ISSN
- 0360-0300 (ISSN-L)
- 1557-7341
- OpenAlex
- W3202183072
- Cited by
- 612
- References
- 384
- Citations by year
- 2026: 164 · 2025: 207 · 2024: 169 · 2023: 65
- FWCI
- 129.72
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- ACM Computing Surveys (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Safety Systems Engineering in Autonomy
- Keywords
- trustworthy AI, robustness, generalization, explainability, transparency, reproducibility, fairness, privacy preservation, accountability, AI lifecycle, AI governance, continuous monitoring, algorithmic bias, imperceptible attacks
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-29, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-29 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.