Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims
Open accessAbstract
With the recent wave of progress in artificial intelligence (AI) has come a growing awareness of the large-scale impacts of AI systems, and recognition that existing regulations and norms in industry and academia are insufficient to ensure responsible AI development. In order for AI developers to earn trust from system users, customers, civil society, governments, and other stakeholders that they are building AI responsibly, they will need to make verifiable claims to which they can be held accountable. Those outside of a given organization also need effective means of scrutinizing such claims. This report suggests various steps that different stakeholders can take to improve the verifiability of claims made about AI systems and their associated development processes, with a focus on providing evidence about the safety, security, fairness, and privacy protection of AI systems. We analyze ten mechanisms for this purpose--spanning institutions, software, and hardware--and make recommendations aimed at implementing, exploring, or improving those mechanisms.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Provenance records may contain sensitive timestamps or workflow traces.
“Provenance records may contain sensitive timestamps or workflow traces”
From Designing automated audit mechanisms to evaluate compliance of generative AI platforms with federal authorship and ownership disclosure requirements. (2023) · cites this paper as [37]
The “Problem” section describes audit trails as traceable logs of steps in AI design, development, and operation, supporting the workflow-traces part of the claim. The passages found do not say that those records contain timestamps or that their contents are sensitive; no page numbers are available in our copy. (LLM, conf=0.82)
Experimental — accuracy not yet measuredVendor audits ensure provenance capture and persistence obligations are met across integrated systems.
“Where external model providers or plug-ins are used, vendor audits ensure provenance capture and persistence obligations are met across integrated systems [37,34].”
From Designing automated audit mechanisms to evaluate compliance of generative AI platforms with federal authorship and ownership disclosure requirements. (2023) · cites this paper as [37]
The paper’s Problem sections say that third-party auditors can verify claims about AI development and that audit trails provide traceable logs. They do not say that vendor audits ensure provenance capture or persistence obligations across external model providers, plug-ins, or integrated systems; the paper also describes AI audit trails as not yet mature. (LLM, conf=0.81)
Experimental — accuracy not yet measuredThe blockchain has recently been considered a promising tool to certify data provenance in AI.
“To ensure that the provenance is tamper proof, the blockchain has been recently considered as a promising tool to certify data provenance in AI [96, 14].”
From Trustworthy AI: From Principles to Practices (2023) · cites this paper as [14]
The Problem section discusses traceable audit trails and verifiable logs of data use, but does not identify blockchain as a way to certify AI data provenance. Searches for “blockchain” and “provenance” found no passage making that connection; the provenance passage concerns learned models instead. (LLM, conf=0.75)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
SMOTE synthetic minority over-sampling technique
Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002) · Journal of Artificial Intelligence Research · doi:10.1613/jair.953 · Cited by 32,865 · cited 1× · see citing works
“Training data can be rigorously evaluated for representativeness [63] [64]; assumptions can be characterized by evaluating modular components of an AI model and by clearly communicating output uncertainties; and performance can be characterized by measuring generalization, fairness, and performance heterogeneity acros…”
BackgroundVisualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839 · see citing works
An integrative model of organizational trust
Mayer, R. C., Davis, J. H., & Schoorman, F. D. (1995) · The Academy of Management Review · doi:10.2307/258792 · Cited by 10,736 · see citing works
Rudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561 · cited 1× · see citing works
“This lack of interpretability in AI systems has raised concerns about using AI models in high stakes decision-making contexts where human welfare may be compromised [73].”
BackgroundExplaining and Harnessing Adversarial Examples
Goodfellow, I. J., Shlens, J., & Szegedy, C. (2014) · arXiv · doi:10.48550/arxiv.1412.6572 · Cited by 10,381 · cited 1× · see citing works
“• This system is robust even to adversarial examples [57] [58];”
BackgroundThe future of employment: How susceptible are jobs to computerisation?
Frey, C. B., & Osborne, M. A. (2017) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2016.08.019 · Cited by 8,727 · see citing works
Intriguing properties of neural networks
Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., & Fergus, R. (2013) · arXiv · doi:10.48550/arxiv.1312.6199 · Cited by 8,143 · cited 1× · see citing works
“• This system is robust even to adversarial examples [57] [58];”
BackgroundAdvances and Open Problems in Federated Learning
Kairouz, P., & McMahan, H. B. (2021) · Foundations and Trends in Machine Learning · doi:10.1561/2200000083 · Cited by 7,772 · cited 2× · see citing works
“, service provider), while keeping the training data decentralized [94].”
BackgroundSelf-citationIntroduction: epistemic communities and international policy coordination
Haas, P. M. (1992) · International Organization · doi:10.1017/s0020818300001442 · Cited by 7,603 · see citing works
Dissecting racial bias in an algorithm used to manage the health of populations
Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019) · Science · doi:10.1126/science.aax2342 · Cited by 7,137 · cited 1× · see citing works
“uncovered racial bias in a widely used algorithm affecting millions of patients [47].”
BackgroundTowards Deep Learning Models Resistant to Adversarial Attacks
Madry, A., Makelov, A., Schmidt, L., Tsipras, D., & Vladu, A. (2017) · arXiv · doi:10.48550/arxiv.1706.06083 · Cited by 6,602 · cited 1× · see citing works
“45 Currently, the security balance is tilted in favor of attacks rather than defenses, with only adversarial training [65] having stood the test of multiple years of attack research.”
BackgroundThe global landscape of AI ethics guidelines
Jobin, A., Ienca, M., & Vayena, E. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0088-2 · Cited by 6,220 · cited 1× · see citing works
“Further, they are amenable to various interpretations [13][14] and agreement on them can mask deeper disagreements [5].”
BackgroundThe mythos of model interpretability
Lipton, Z. C. (2018) · Communications of the ACM · doi:10.1145/3233231 · Cited by 5,427 · cited 1× · see citing works
“Characterizing desiderata for interpretable models is a helpful way to formalize interpretability [74] [75].”
BackgroundDeep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
Simonyan, K., Vedaldi, A., & Zisserman, A. (2013) · arXiv · doi:10.48550/arxiv.1312.6034 · Cited by 5,370 · see citing works
Axiomatic attribution for deep networks
Sundararajan, M., Taly, A., & Yan, Q. (2017) · arXiv · doi:10.48550/arxiv.1703.01365 · Cited by 4,087 · cited 1× · see citing works
“Characterizing desiderata for interpretable models is a helpful way to formalize interpretability [74] [75].”
BackgroundStriving for Simplicity: The All Convolutional Net
Springenberg, J. T., Dosovitskiy, A., Brox, T., & Riedmiller, M. (2014) · arXiv · doi:10.48550/arxiv.1412.6806 · Cited by 3,447 · see citing works
InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial NetsPreprint
Chen, X., Duan, Y., Houthooft, R., Schulman, J., Sutskever, I., & Abbeel, P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.03657 · Cited by 3,379 · see citing works
Algorithm aversion: People erroneously avoid algorithms after seeing them err.
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015) · Journal of Experimental Psychology: General · doi:10.1037/xge0000033 · Cited by 3,232 · cited 1× · see citing works
“, studying the efficacy of interpretability tools or exploring possible interfaces–would be welcome, as would further exploration of how practitioners currently use such tools [81] [82] [83] [78] [84].”
BackgroundImbalanced-learn: A Python Toolbox to Tackle the Curse of Imbalanced Datasets in Machine LearningPreprint
Lemaître, G., Nogueira, F., & Aridas, C. K. (2016) · Journal of machine learning research · doi:10.48550/arxiv.1609.06570 · Cited by 2,733 · cited 1× · see citing works
“Training data can be rigorously evaluated for representativeness [63] [64]; assumptions can be characterized by evaluating modular components of an AI model and by clearly communicating output uncertainties; and performance can be characterized by measuring generalization, fairness, and performance heterogeneity acros…”
BackgroundModel Cards for Model Reporting
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287596 · Cited by 2,681 · cited 3× · see citing works
“(18)Model cards for model reporting [28] and data sheets for datasets [29] reveal information about AI systems publicly, and future work in third party auditing could build on such tools, as advocated by Raji and Smart et al.”
BackgroundMethodGebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., III, H. D., & Crawford, K. (2021) · Communications of the ACM · doi:10.1145/3458723 · Cited by 2,469 · cited 3× · see citing works
“(18)Model cards for model reporting [28] and data sheets for datasets [29] reveal information about AI systems publicly, and future work in third party auditing could build on such tools, as advocated by Raji and Smart et al.”
BackgroundMethodCooperation under the security dilemma
Jervis, R. (1978) · World Politics · doi:10.2307/2009958 · Cited by 2,305 · see citing works
Ethics Guidelines for Trustworthy AI
Cannarsa, M. (2021) · The Cambridge Handbook of Lawyering in the Digital Age · doi:10.1017/9781108936040.022 · Cited by 2,095 · see citing works
Buchanan, J. (1965) · Economica · doi:10.2307/2552442 · Cited by 1,957 · cited 1× · see citing works
“Voluntary membership organizations with stricter rules and norms have been implemented in other industries and might also be a useful model for AI developers [31].”
BackgroundAnanny, M., & Crawford, K. (2018) · New Media & Society · doi:10.1177/1461444816676645 · Cited by 1,836 · see citing works
Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks
Katz, G., Barrett, C., Dill, D. L., Julian, K., & Kochenderfer, M. J. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-63387-9_5 · Cited by 1,706 · cited 1× · see citing works
“Without approximation, certificates are computationally prohibitive for all but the smallest real world tasks [69].”
BackgroundRISE: Randomized Input Sampling for Explanation of Black-box ModelsPreprint
Petsiuk, V., Das, A., & Saenko, K. (2018) · British Machine Vision Conference · doi:10.48550/arxiv.1806.07421 · Cited by 1,702 · see citing works
Ulyanov, D., Vedaldi, A., & Lempitsky, V. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-020-01303-4 · Cited by 1,661 · see citing works
Understanding Black-box Predictions via Influence Functions
Koh, P. W., & Liang, P. (2017) · arXiv · doi:10.48550/arxiv.1703.04730 · Cited by 1,514 · see citing works
Self-citationNetwork Dissection: Quantifying Interpretability of Deep Visual Representations
Bau, D., Zhou, B., Khosla, A., Oliva, A., & Torralba, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.354 · Cited by 1,389 · see citing works
Raji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372873 · Cited by 1,372 · cited 3× · see citing works
“[36] for a discussion of some lessons for AI from auditing in other industries.”
BackgroundConcrete Problems in AI Safety
Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., & Mané, D. (2016) · arXiv · doi:10.48550/arxiv.1606.06565 · Cited by 1,314 · cited 2× · see citing works
“Algorithms or models that are purported to have favorable safety properties, such as enabling safe exploration or robustness to distributional shifts [49], could be scrutinized via bounty programs.”
BackgroundSanity Checks for Saliency Maps
Adebayo, J., Gilmer, J., Muelly, M., Goodfellow, I., Hardt, M., & Kim, B. (2018) · arXiv · doi:10.48550/arxiv.1810.03292 · Cited by 1,294 · cited 1× · see citing works
“, the reliability of existing methods is an active area of research [85] [86] [87] [88] [89] [90] [91] [92] [93].”
BackgroundCertified Adversarial Robustness via Randomized SmoothingPreprint
Cohen, J. M., Rosenfeld, E., & Kolter, J. Z. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.02918 · Cited by 1,172 · cited 1× · see citing works
“not generalizing well to unseen data [66] [67] [68]).”
BackgroundInnovative instructions and software model for isolated execution
Mckeen, F., Alexandrovich, I., Berenzon, A., Rozas, C. V., Shafi, H., Shanbhogue, V., & Savagaonkar, U. R. (2013) · Hardware and Architectural Support for Security and Privacy · doi:10.1145/2487726.2488368 · Cited by 1,103 · cited 1× · see citing works
“A secure enclave is a set of software and hardware features that together provide an isolated execution environment that enables a set of strong guarantees regarding security for applications running inside the enclave [109].”
BackgroundInterpretable Explanations of Black Boxes by Meaningful Perturbation
Fong, R. C., & Vedaldi, A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.371 · Cited by 1,097 · see citing works
Self-citationDota 2 with Large Scale Deep Reinforcement Learning
OpenAI, :, Berner, C., Brockman, G., Chan, B., Cheung, V., Dębiak, P., Dennison, C., Farhi, D., Fischer, Q., et al. (2019) · arXiv · doi:10.48550/arxiv.1912.06680 · Cited by 1,065 · see citing works
BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain
Gu, T., Dolan-Gavitt, B., & Garg, S. (2017) · arXiv · doi:10.48550/arxiv.1708.06733 · Cited by 1,061 · cited 1× · see citing works
“, the reliability of existing methods is an active area of research [85] [86] [87] [88] [89] [90] [91] [92].”
BackgroundThe Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation
Brundage, M., Avin, S., Clark, J., Toner, H., Eckersley, P., Garfinkel, B., Dafoe, A., Scharre, P., Zeitzoff, T., Filar, B., et al. (2018) · Apollo - University of Cambridge Repository · doi:10.17863/cam.22520 · Cited by 849 · see citing works
Self-citationInterpretable Convolutional Neural Networks
Zhang, Q., Wu, Y. N., & Zhu, S.-C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00920 · Cited by 833 · see citing works
Barth, S., & de Jong, M. D. T. (2017) · Telematics and Informatics · doi:10.1016/j.tele.2017.04.013 · Cited by 798 · cited 1× · see citing works
“, a concern for personal privacy) because other factors such as convenience and brand loyalty also play a role in influencing their behavior [127].”
BackgroundCertified Robustness to Adversarial Examples with Differential Privacy
Lecuyer, M., Atlidakis, V., Geambasu, R., Hsu, D., & Jana, S. (2019) · 2019 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2019.00044 · Cited by 750 · cited 1× · see citing works
“not generalizing well to unseen data [66] [67] [68]).”
BackgroundManipulating and Measuring Model Interpretability
Poursabzi-Sangdeh, F., Goldstein, D. G., Hofman, J. M., Wortman Vaughan, J. W., & Wallach, H. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445315 · Cited by 728 · cited 2× · see citing works
“This is already the case in regulated domains such as finance [78].”
BackgroundWhitman, N. A. (1999) · Academic medicine : journal of the Association of American Medical Colleges · doi:10.1097/00001888-199901000-00001 · Cited by 689
Interpretable Decision Sets: A Joint Framework for Description and Prediction
Lakkaraju, H., Bach, S. H., & Leskovec, J. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939874 · Cited by 687 · see citing works
Fairness in Machine Learning: Lessons from Political Philosophy
Binns, R. (2017) · arXiv · doi:10.48550/arxiv.1712.03586 · Cited by 676 · see citing works
The (Un)reliability of Saliency Methods
Kindermans, P.-J., Hooker, S., Adebayo, J., Alber, M., Schütt, K. T., Dähne, S., Erhan, D., & Kim, B. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-28954-6_14 · Cited by 652 · cited 1× · see citing works
“, the reliability of existing methods is an active area of research [85] [86] [87] [88] [89] [90] [91] [92].”
BackgroundSelf-citationExplainable machine learning in deployment
Bhatt, U., Xiang, A., Sharma, S., Weller, A., Taly, A., Jia, Y., Ghosh, J., Puri, R., Moura, J. M. F., & Eckersley, P. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3375624 · Cited by 633 · cited 1× · see citing works
“, studying the efficacy of interpretability tools or exploring possible interfaces–would be welcome, as would further exploration of how practitioners currently use such tools [81] [82] [83] [78] [84].”
BackgroundSelf-citationThe Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks
Carlini, N., Liu, C., Erlingsson, Ú., Kos, J., & Song, D. (2018) · arXiv · doi:10.48550/arxiv.1802.08232 · Cited by 629 · cited 1× · see citing works
“These measures include quantifying the degree to which models memorize training data [95], and incorporating differential privacy techniques to limit the contribution of individual clients in the federated setting [96].”
BackgroundRoss, A. S., & Doshi‐Velez, F. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11504 · Cited by 621 · see citing works
A Multiscale Visualization of Attention in the Transformer Model
Vig, J. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations · doi:10.18653/v1/p19-3007 · Cited by 600 · see citing works
The Building Blocks of Interpretability
Olah, C., Satyanarayan, A., Johnson, I., Carter, S., Schubert, L., Ye, K., & Mordvintsev, A. (2018) · Distill · doi:10.23915/distill.00010 · Cited by 591 · see citing works
Formal Guarantees on the Robustness of a Classifier against Adversarial ManipulationPreprint
Hein, M., & Andriushchenko, M. (2017) · Neural Information Processing Systems · doi:10.48550/arxiv.1705.08475 · Cited by 566 · cited 1× · see citing works
“not generalizing well to unseen data [66] [67] [68]).”
BackgroundActionable Recourse in Linear Classification
Ustun, B., Spangher, A., & Liu, Y. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287566 · Cited by 501 · cited 1× · see citing works
“, bank loans) [77];”
Artificial Intelligence: American Attitudes and Trends
Zhang, B., & Dafoe, A. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3312874 · Cited by 464 · cited 1× · see citing works
“Indeed, studies [17], surveys [18], and trends in worker and community organizing [2] [3] make clear that large swaths of the public are concerned about the risks of AI development, and do not trust the organizations currently dominating such development to self-regulate effectively.”
BackgroundSelf-citationA Benchmark for Interpretability Methods in Deep Neural NetworksPreprint
Hooker, S., Erhan, D., Kindermans, P.-J., & Kim, B. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.10758 · Cited by 438 · cited 1× · see citing works
“, the reliability of existing methods is an active area of research [85] [86] [87] [88] [89] [90] [91] [92].”
BackgroundSelf-citationSynthesizing the preferred inputs for neurons in neural networks via deep generator networksPreprint
Nguyen, A., Dosovitskiy, A., Yosinski, J., Brox, T., & Clune, J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.09304 · Cited by 435 · see citing works
Adler, E. (1992) · International Organization · doi:10.1017/s0020818300001466 · Cited by 408 · see citing works
Real Time Image Saliency for Black Box ClassifiersPreprint
Dabkowski, P., & Gal, Y. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.07857 · Cited by 404 · see citing works
The Role and Limits of Principles in AI Ethics: Towards a Focus on Tensions
Whittlestone, J., Nyrup, R., Alexandrova, A., & Cave, S. (2019) · Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3306618.3314289 · Cited by 399 · cited 2× · see citing works
“Further, they are amenable to various interpretations [13][14] and agreement on them can mask deeper disagreements [5].”
BackgroundCalibrating Noise to Sensitivity in Private Data Analysis
Dwork, C., McSherry, F., Kobbi Nissim, B., & Smith, A. (2017) · Journal of Privacy and Confidentiality · doi:10.29012/jpc.v7i3.405 · Cited by 390 · cited 1× · see citing works
“Differential privacy [97] is a system for publicly sharing information derived from a dataset by describing the patterns of groups within the dataset, while withholding information about individuals in the dataset; it allows for precise measurements of privacy risks for current and potential data owners, and can addre…”
BackgroundInterpreting CNNs via Decision Trees
Zhang, Q., Yang, Y., Ma, H., & Wu, Y. N. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00642 · Cited by 350 · see citing works
Trust: Making and Breaking Cooperative Relations.
Barber, B., & Gambetta, D. (1992) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2076328 · Cited by 340 · see citing works
Algorithms for Verifying Deep Neural Networks
Liu, C., Arnon, T., Lazarus, C., Strong, C. A., Barrett, C., & Kochenderfer, M. J. (2020) · Foundations and Trends® in Optimization · doi:10.1561/2400000035 · Cited by 336 · see citing works
Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNet
Brendel, W., & Bethge, M. (2019) · arXiv · doi:10.48550/arxiv.1904.00760 · Cited by 323 · see citing works
An Abstraction-Refinement Approach to Verification of Artificial Neural Networks
Pulina, L., & Tacchella, A. (2010) · International Conference on Computer Aided Verification · doi:10.1007/978-3-642-14295-6_24 · Cited by 311 · see citing works
The relationship between trust in AI and trustworthy machine learning technologies
Toreini, E., Aitken, M., Coopamootoo, K., Elliott, K., Zelaya, C. G., & van Moorsel, A. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372834 · Cited by 296 · see citing works
Release Strategies and the Social Impacts of Language Models
Solaiman, I., Brundage, M., Clark, J., Askell, A., Herbert-Voss, A., Wu, J., Radford, A., Krueger, G., Kim, J. W., Kreps, S., et al. (2019) · arXiv · doi:10.48550/arxiv.1908.09203 · Cited by 292 · cited 3× · see citing works
“, "Threat Modeling AI/ML Systems and Dependencies" [43] (25)In the context of language models, for example, 2019 saw a degree of communication and coordination across AI developers to assess the relative risks of different language understanding and generation systems [10].”
BackgroundSelf-citationWho Cares about Auditor Reputation
Barton, J. (2005) · Contemporary Accounting Research · doi:10.1506/c27u-23k8-e1vl-20r0 · Cited by 279 · cited 1× · see citing works
“Reputational considerations help to ensure auditing integrity in the case of financial accounting, where firms prefer to engage with credible auditors [38].”
Artificial Intelligence and National Security
Cimbala, S. (1986) · doi:10.1007/978-3-031-06709-9 · Cited by 273 · see citing works
Irresistible: The Rise of Addictive Technology and the Business of Keeping Us Hooked
Hunter, M. T., & Morganstein, J. (2021) · Psychiatry · doi:10.1080/00332747.2021.1925508 · Cited by 272
On the importance of single directions for generalizationPreprint
Morcos, A. S., Barrett, D. G. T., Rabinowitz, N. C., & Botvinick, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.06959 · Cited by 269 · see citing works
Slalom: Fast, Verifiable and Private Execution of Neural Networks in Trusted HardwarePreprint
Tramèr, F., & Boneh, D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.03287 · Cited by 263 · cited 1× · see citing works
“67 Demonstrations to date have been carried out on commodity hardware (CPUs [112] [113] and GPUs [114]) or have secure and verifiable outsourcing of parts of the computation to less secure hardware [115] [116], rather than on hardware directly optimized for machine learning (such as TPUs).”
BackgroundAmershi, S., Chickering, M., Drucker, S. M., Lee, B., Simard, P., & Suh, J. (2015) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702509 · Cited by 263 · see citing works
The Ethical Algorithm: The Science of Socially Aware Algorithm Design
Kearns, M. J., & Roth, A. (2021) · Perspectives on Science and Christian Faith · doi:10.56315/pscf3-21kearns · Cited by 261 · cited 2× · see citing works
“For an accessible discussion of the ideas behind differential privacy and its applications, from which this short summary was adapted, see [99].”
Privacy-preserving Machine Learning as a Service
Hesamifard, E., Takabi, H., Ghasemi, M., & Wright, R. N. (2018) · Proceedings on Privacy Enhancing Technologies · doi:10.1515/popets-2018-0024 · Cited by 245 · cited 1× · see citing works
“67 Demonstrations to date have been carried out on commodity hardware (CPUs [113] [114] and GPUs [115]) or have secure and verifiable outsourcing of parts of the computation to less secure hardware [116] [117], rather than on hardware directly optimized for machine learning (such as TPUs).”
BackgroundLeike, J., Martic, M., Krakovna, V., Ortega, P. A., Everitt, T., Lefrancq, A., Orseau, L., & Legg, S. (2017) · arXiv · doi:10.48550/arxiv.1711.09883 · Cited by 240 · cited 1× · see citing works
“• This system is robust to ’natural’ distributional shifts [49] [56];”
BackgroundHohman, F., Head, A., Caruana, R., DeLine, R. A., & Drucker, S. M. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300809 · Cited by 225 · cited 1× · see citing works
“, studying the efficacy of interpretability tools or exploring possible interfaces–would be welcome, as would further exploration of how practitioners currently use such tools [81] [82] [83] [78] [84].”
BackgroundInterpreting CNN Knowledge via an Explanatory Graph
Zhang, Q., Cao, R., Shi, F., Wu, Y. N., & Zhu, S. C. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11819 · Cited by 219 · see citing works
Semantic photo manipulation with a generative image prior
Bau, D., Strobelt, H., Peebles, W., Wulff, J., Zhou, B., Zhu, J.-Y., & Torralba, A. (2019) · ACM Transactions on Graphics · doi:10.1145/3306346.3323023 · Cited by 207 · cited 1× · see citing works
“, while limited in treatment, has primarily centered on comparing methods using human surveys [80].”
"The human body is a black box": supporting clinical decision-making with deep learning
Sendak, M., Elish, M. C., Gao, M., Futoma, J., Ratliff, W., Nichols, M., Bedoya, A., Balu, S., & O'Brien, C. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372827 · Cited by 196 · cited 1× · see citing works
“[76] which focuses on building trust in a hospital context, and contextualizes the role of interpretability in this process.”
BackgroundRaji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P. (2020) · arXiv · doi:10.48550/arxiv.2001.00973 · Cited by 179 · see citing works
Programmatically Interpretable Reinforcement LearningPreprint
Verma, A., Murali, V., Singh, R., Kohli, P., & Chaudhuri, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.02477 · Cited by 158 · see citing works
Narayanan, M., Chen, E., He, J., Kim, B., Gershman, S., & Doshi-Velez, F. (2018) · arXiv · doi:10.48550/arxiv.1802.00682 · Cited by 153 · cited 1× · see citing works
“, studying the efficacy of interpretability tools or exploring possible interfaces–would be welcome, as would further exploration of how practitioners currently use such tools [81] [82] [83] [78] [84].”
BackgroundCarter, S., Armstrong, Z., Schubert, L., Johnson, I., & Olah, C. (2019) · Distill · doi:10.23915/distill.00015 · Cited by 147 · see citing works
Increasing Trust in AI Services through Supplier's Declarations of Conformity
Hind, M., Mehta, S., Mojsilovic, A., et al. (2018) · arXiv.org · doi:10.48550/arxiv.1808.07261 · Cited by 134
Errudite: Scalable, Reproducible, and Testable Error Analysis
Wu, T., Ribeiro, M. T., Heer, J., & Weld, D. S. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1073 · Cited by 129 · see citing works
Lost Causes: Agenda Vetting in Global Issue Networks and the Shaping of Human Security
Carpenter, C. (2014) · doi:10.7591/cornell/9780801448850.001.0001 · Cited by 127 · see citing works
Fooling Neural Network Interpretations via Adversarial Model ManipulationPreprint
Heo, J., Joo, S., & Moon, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.02041 · Cited by 126 · cited 1× · see citing works
“, the reliability of existing methods is an active area of research [85] [86] [87] [88] [89] [90] [91] [92].”
BackgroundInterpreting Blackbox Models via Model ExtractionPreprint
Bastani, O., Kim, C., & Bastani, H. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.08504 · Cited by 126 · see citing works
Representer Point Selection for Explaining Deep Neural NetworksPreprint
Yeh, C., Kim, J. S., Yen, I. E.-H., & Ravikumar, P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.09720 · Cited by 125 · see citing works
Salient Deconvolutional Networks
Mahendran, A., & Vedaldi, A. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46466-4_8 · Cited by 124 · cited 1× · see citing works
“, the reliability of existing methods is an active area of research [85] [86] [87] [88] [89] [90] [91] [92] [93].”
BackgroundArtificial Intelligence: A Revolution in Strategic Affairs?
Payne, K. (2018) · Survival · doi:10.1080/00396338.2018.1518374 · Cited by 124 · see citing works
SafetyNets: Verifiable Execution of Deep Neural Networks on an Untrusted CloudPreprint
Ghodsi, Z., Gu, T., & Garg, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.10268 · Cited by 119 · cited 1× · see citing works
“For example, one could have assurances that a cloud-based AI service is not skimping on computations by running a less powerful model than advertised, operating on private data in a disallowed fashion, or compromised by malware [107].”
BackgroundComputational Power and the Social Impact of Artificial IntelligencePreprint
Hwang, T. (2018) · SSRN Electronic Journal · doi:10.2139/ssrn.3147971 · Cited by 107 · cited 1× · see citing works
“Hardware relevant to AI development ranges from sensors, networking, and memory, to, perhaps most crucially, processing power [102].”
BackgroundUnderstanding Image Representations by Measuring Their Equivariance and Equivalence
Lenc, K., & Vedaldi, A. (2019) · International Journal of Computer Vision · doi:10.1007/s11263-018-1098-y · Cited by 98 · see citing works
SGX-LKL: Securing the Host OS Interface for Trusted ExecutionPreprint
Priebe, C., Muthukumaran, D., Lind, J., Zhu, H., Cui, S., Sartakov, V. A., & Pietzuch, P. R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.11143 · Cited by 96 · cited 1× · see citing works
“67 Demonstrations to date have been carried out on commodity hardware (CPUs [112] [113] and GPUs [114]) or have secure and verifiable outsourcing of parts of the computation to less secure hardware [115] [116], rather than on hardware directly optimized for machine learning (such as TPUs).”
BackgroundRutherford, K. (2000) · doi:10.1017/s0043887100009382 · Cited by 95
Linking Artificial Intelligence PrinciplesPreprint
Zeng, Y., Lu, E.-M., & Huangfu, C. (2018) · SafeAI@AAAI · doi:10.48550/arxiv.1812.04814 · Cited by 94 · cited 1× · see citing works
“[16] for examples of non-Western AI principles.”
Political Realism and Political Idealism: a Study in Theories and Realities
GanzH., F. (1952) · doi:10.2307/2604027 · Cited by 94
Efficient Deep Learning on Multi-Source Private DataPreprint
Hynes, N., Cheng, R., & Song, D. X. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.06689 · Cited by 93 · cited 2× · see citing works
“4% [112].”
BackgroundTowards a program logic for JavaScript
Gardner, P., Maffeis, S., & Smith, G. (2012) · ACM-SIGACT Symposium on Principles of Programming Languages · doi:10.1145/2103656.2103663 · Cited by 92
Differential Adaptive Stress Testing of Airborne Collision Avoidance Systems
Lee, R., Mengshoel, O. J., Saksena, A., et al. (2018) · 2018 AIAA Modeling and Simulation Technologies Conference · doi:10.2514/6.2018-1923 · Cited by 87 · cited 1×
“For example, adaptive stress testing (AST) of an AI system allows users to find the most likely failure of a system for a given scenario using reinforcement learning [61], and is being used by to validate the next generation of aircraft collision avoidance software [62].”
MethodAI Ethics – Too Principled to Fail?
Mittelstadt, B. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3391293 · Cited by 82 · cited 3× · see citing works
“Over 80 AI organizations [5], including technology companies such as Google [22], OpenAI [23], and Microsoft [24] have publicly stated the principles they will follow in developing AI.”
DeepAttest: An End-to-End Attestation Framework for Deep Neural Networks
Chen, H., Fu, C., Rouhani, B., et al. (2019) · International Symposium on Computer Architecture · doi:10.1145/3307650.3322251 · Cited by 82 · cited 1×
“Remote attestation is often used to attest that a certain version of software is currently running, or that a computation took a certain amount of time (which can then be compared to a reference by the remote party to detect tampering) [106].”
BackgroundRepeatability, reproducibility, and rigor in systems research
Vítek, J., & Kalibera, T. (2011) · ACM International Conference on Embedded Software (EMSOFT) · doi:10.1145/2038642.2038650 · Cited by 80 · cited 1× · see citing works
“, Vitek & Kalibera [123] and Hoefler & Belli [124].”
BackgroundPolitical Realism and Political Idealism: A Study in Theories and Realities.
Flechtheim, O. K., & Herz, J. H. (1951) · doi:10.2307/2087376 · Cited by 79
The Antitrust Consumer Welfare Paradox
Orbach, B. (2011) · doi:10.1093/joclec/nhq019 · Cited by 78
The Role of Cooperation in Responsible AI DevelopmentPreprint
Askell, A., Brundage, M., & Hadfield, G. K. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.04534 · Cited by 75 · cited 3× · see citing works
“Greater verifiability of claims about AI development would help enable more effective oversight and reduce pressure to cut corners for the sake of gaining a competitive edge [1].”
BackgroundSelf-citationSchwartz, R., Dodge, J., Smith, N. A., & Etzioni, O. (2019) · arXiv · doi:10.48550/arxiv.1907.10597 · Cited by 73 · cited 1× · see citing works
“have recommended that researchers always publish financial and computational costs alongside performance increases [121].”
BackgroundFinding Influential Training Samples for Gradient Boosted Decision Trees
Sharchilev, B., Ustinovsky, Y., Serdyukov, P., et al. (2018) · International Conference on Machine Learning · doi:10.48550/arxiv.1802.06640 · Cited by 73
Robustness Testing of Autonomy Software
Hutchison, C., Zizyte, M., Lanigan, P. E., et al. (2017) · 2018 IEEE/ACM 40th International Conference on Software Engineering: Software Engineering in Practice Track (ICSE-SEIP) · doi:10.1145/3183519.3183534 · Cited by 72 · see citing works
Arms Control, Stability, and Causes of War
Jervis, R. (1993) · doi:10.2307/2152010 · Cited by 67
Developing Bug-Free Machine Learning Systems With Formal MathematicsPreprint
Selsam, D., Liang, P., & Dill, D. (2017) · International Conference on Machine Learning · doi:10.48550/arxiv.1706.08605 · Cited by 66 · see citing works
Understanding Deep Networks via Extremal Perturbations and Smooth Masks
Fong, R., Patrick, M., & Vedaldi, A. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00304 · Cited by 63 · see citing works
Self-citationInterpreting Black Box Predictions using Fisher KernelsPreprint
Khanna, R., Kim, B., Ghosh, J., & Koyejo, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.10118 · Cited by 61 · see citing works
You shouldn’t trust me: Learning models which conceal unfairness from multiple explanation methods.
Dimanov, B., Bhatt, U., Jamnik, M., & Weller, A. (2020) · Apollo (University of Cambridge) · doi:10.17863/cam.48825 · Cited by 58 · cited 1× · see citing works
“, the reliability of existing methods is an active area of research [85] [86] [87] [88] [89] [90] [91] [92] [93].”
BackgroundSelf-citationThe battle for ethical AI at the world’s biggest machine-learning conference
Gibney, E. (2020) · Nature · doi:10.1038/d41586-020-00160-y · Cited by 54 · cited 1× · see citing works
“Nor are they able to hold developers to account when principles and behavior diverge, fueling accusations of "ethics washing" [4].”
BackgroundBelfield, H. (2020) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3375627.3375814 · Cited by 52 · see citing works
Self-citationCoe, A., & Vaynman, J. (2019) · American Political Science Review · doi:10.1017/s000305541900073x · Cited by 51
A Safety Framework for Critical Systems Utilising Deep Neural Networks
Zhao, X., Banks, A., Sharp, J. J., Robu, V., Flynn, D., Fisher, M., & Huang, X. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-54549-9_16 · Cited by 46 · see citing works
Toward Scalable Verification for Safety-Critical Deep NetworksPreprint
Kuper, L., Katz, G., Gottschlich, J. E., Julian, K. D., Barrett, C. W., & Kochenderfer, M. J. (2018) · arXiv.org · doi:10.48550/arxiv.1801.05950 · Cited by 40 · see citing works
Autonomous Weapons Systems: Law, Ethics, Policy
Bhuta, N., Beck, S., Geis, R., Liu, H.-Y., & Kress, C. (2016) · doi:10.1017/cbo9781316597873 · Cited by 40
Survey of Attacks and Defenses on Edge-Deployed Neural Networks
Isakov, M., Gadepally, V. N., Gettings, K., & Kinsy, M. A. (2019) · IEEE High Performance Extreme Computing Conference · doi:10.1109/hpec.2019.8916519 · Cited by 40 · cited 1× · see citing works
“56 These techniques do not, however, protect the model itself from theft [100].”
BackgroundThe Windfall Clause: Distributing the Benefits of AI for the Common Good
O'Keefe, C., Cihon, P., Garfinkel, B., Flynn, C., Leung, J., & Dafoe, A. (2020) · Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3375627.3375842 · Cited by 38 · cited 1× · see citing works
“The Windfall Clause [30] is one proposal along these lines, and involves an ex ante commitment by AI firms to donate a significant amount of any eventual extremely large profits.”
BackgroundSelf-citationRaji, I. D., & Yang, J. (2019) · arXiv · doi:10.48550/arxiv.1912.06166 · Cited by 36 · cited 2× · see citing works
“Inspiration might be taken from recent work on internal algorithmic auditing [36] and ongoing work on the documentation of AI systems more generally, such as the ABOUT ML project [27].”
BackgroundSelf-citationRobust artificial intelligence and robust human organizations
Dietterich, T. G. (2018) · Frontiers of Computer Science · doi:10.1007/s11704-018-8900-4 · Cited by 36 · see citing works
On the Legal Compatibility of Fairness Definitions
Xiang, A., & Raji, I. D. (2019) · arXiv · doi:10.48550/arxiv.1912.00761 · Cited by 34 · see citing works
Regulatory Markets for AI SafetyPreprint
Clark, J., & Hadfield, G. K. (2019) · RePEc: Research Papers in Economics · doi:10.48550/arxiv.2001.00078 · Cited by 33 · cited 1× · see citing works
“One way that third party auditing could connect to government policies, and be funded, is via a "regulatory market" [42].”
BackgroundSelf-citationEncyclopedia of arms control and disarmament
Cohen, E., & Burns, R. (1993) · doi:10.2307/20045731 · Cited by 33
Autonomous Weapons: Regulation Tolerant or Regulation Resistant?
Watts, S. (2015) · doi:10.2139/ssrn.2681283 · Cited by 30
Monitoring Misuse for Accountable 'Artificial Intelligence as a Service'
Javadi, S. A., Cloete, R., Cobbe, J., Lee, M. S. A., & Singh, J. P. (2020) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3375627.3375873 · Cited by 29 · cited 1× · see citing works
“These mechanisms could be extended to cover AI misuse [107].”
BackgroundInterpretable & Explorable Approximations of Black Box Models
Lakkaraju, H., Kamar, E., Caruana, R., & Leskovec, J. (2017) · arXiv · doi:10.48550/arxiv.1707.01154 · Cited by 23 · see citing works
Disruptive Innovations and Disruptive Assurance: Assuring Machine Learning and Autonomy
Bloomfield, R., Khlaaf, H., Ryan Conmy, P., & Fletcher, G. (2019) · Computer · doi:10.1109/mc.2019.2914775 · Cited by 23 · see citing works
Self-citationThe Role of Civilian Innovation in the Development of Lethal Autonomous Weapon Systems
Verbruggen, M. (2019) · Global Policy · doi:10.1111/1758-5899.12663 · Cited by 22
(1974) · International Legal Materials · doi:10.1017/s0020782900039085 · Cited by 18
Ovadya, A., & Whittlestone, J. (2019) · arXiv · doi:10.48550/arxiv.1907.11274 · Cited by 15 · cited 1× · see citing works
“Incident sharing is closely related to but distinct from responsible publication practices in AI and coordinated disclosure of cybersecurity vulnerabilities [55].”
BackgroundImbrie, A., & Kania, E. B. (2019) · doi:10.51593/20190051 · Cited by 15
The Adaptive Stress Testing Formulation
Koren, M., Corso, A., & Kochenderfer, M. J. (2020) · arXiv.org · doi:10.48550/arxiv.2004.04293 · Cited by 12 · cited 1×
“For example, adaptive stress testing (AST) of an AI system allows users to find the most likely failure of a system for a given scenario using reinforcement learning [61], and is being used by to validate the next generation of aircraft collision avoidance software [62].”
MethodSelf-citationBurns, R. (2009) · doi:10.5040/9798400648540 · Cited by 10
Goldblat, J. (2020) · doi:10.4324/9781003053491 · Cited by 8
Civil Society Responds to the AWS : Growing Activist Networks and Shifting Frames
Bahçecik, Ş. O. (2019) · Global Policy · doi:10.1111/1758-5899.12671 · Cited by 7 · see citing works
Barner-Barry, C. (1990) · Politics and the life sciences · doi:10.1017/s0730938400010352 · Cited by 7
Daniloff, N. (2016) · Journal of Cold War Studies · doi:10.1162/jcws_r_00693 · Cited by 1
Flagged first, then by citations
85 of the 230 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
A Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI
Tjoa, E., & Guan, C. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3027314 · Cited by 2,524 · see citing works
Ali, S., Abuhmed, T., El-Sappagh, S., Muhammad, K., Alonso-Moral, J. M., Confalonieri, R., Guidotti, R., Del Ser, J., Díaz-Rodríguez, N., & Herrera, F. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101805 · Cited by 1,762 · see citing works
Secure, privacy-preserving and federated machine learning in medical imaging
Kaissis, G. A., Makowski, M. R., Rückert, D., & Braren, R. F. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-0186-1 · Cited by 1,501 · see citing works
“(8) The development of auditable and objectively trustworthy systems 99 (that is, not relying on subjective assertions—for example, by governments) will promote the universal acceptance of secure and private AI solutions by individuals and policymakers.”
BackgroundGenerative AI and ChatGPT: applications, challenges, and AI-human collaboration
Fui-Hoon Nah, F., Zheng, R., Cai, J., Siau, K., & Chen, L. (2023) · Journal of Information Technology Case and Application Research · doi:10.1080/15228053.2023.2233814 · Cited by 1,363 · see citing works
Interpretable machine learning: Fundamental principles and 10 grand challenges
Rudin, C., Chen, C., Chen, Z., Huang, H., Semenova, L., & Zhong, C. (2022) · Statistics Surveys · doi:10.1214/21-ss133 · Cited by 958 · see citing works
“It is universally agreed that interpretability is a key element of trust for AI models (Wagstaff, 2012; Rudin and Wagstaff, 2014; Lo Piano, 2020; Ashoori and Weisz, 2019; Thiebes et al., 2020; Spiegelhalter, 2020; Brundage et al., 2020).”
Backgroundvon Rueden, L., Mayer, S., Beckh, K., Georgiev, B., Giesselbach, S., Heese, R., Kirsch, B., Walczak, M., Pfrommer, J., Pick, A., et al. (2021) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3079836 · Cited by 878 · see citing works
“Another important aspect is that a purely data-driven model might not meet constraints such as dictated by natural laws, or given through regulatory or security guidelines, which are important for trustworthy AI [8].”
BackgroundDíaz-Rodríguez, N., Del Ser, J., Coeckelbergh, M., López de Prado, M., Herrera-Viedma, E., & Herrera, F. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101896 · Cited by 800 · see citing works
Industrial Artificial Intelligence in Industry 4.0 - Systematic Review, Challenges and Outlook
Peres, R. S., Jia, X., Lee, J., Sun, K., Colombo, A. W., & Barata, J. (2020) · IEEE Access · doi:10.1109/access.2020.3042874 · Cited by 754 · see citing works
Trustworthy AI: From Principles to Practices
Li, B., Qi, P., Liu, B., Di, S., Liu, J., Pei, J., Yi, J., & Zhou, B. (2023) · ACM Computing Surveys · doi:10.1145/3555803 · Cited by 612 · see citing works
“• We consider ethical requirements with broad concerns in recent literature [9, 57, 121, 145, 178, 199, 218, 301, 334], including fairness, privacy, and accountability.”
BackgroundInfluentialThe Ethics of Artificial Intelligence
Floridi, L. (2023) · doi:10.1093/oso/9780198883098.001.0001 · Cited by 605 · see citing works
Artificial Intelligence in Education (AIEd): a high-level academic and industry note 2021
Chaudhry, M. A., & Kazim, E. (2022) · AI and Ethics · doi:10.1007/s43681-021-00074-z · Cited by 425 · see citing works
“1 3 and engineering focused on developing Trustworthy AI [11].”
BackgroundThe ethics of algorithms: key problems and solutions
Tsamados, A., Aggarwal, N., Cowls, J., Morley, J., Roberts, H., Taddeo, M., & Floridi, L. (2022) · AI & SOCIETY · doi:10.1007/s00146-021-01154-8 · Cited by 384 · see citing works
“This includes information ranging from characteristics like geographical location and age to information on specific behaviour and preferences, including what type of content a person is likely to seek the most on a given platform (Chakraborty et al. 2019).”
Perifanis, N.-A., & Kitsios, F. (2023) · Information · doi:10.3390/info14020085 · Cited by 407 · see citing works
Rakova, B., Yang, J., Cramer, H., & Chowdhury, R. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449081 · Cited by 329 · see citing works
“[10]) based originally on discussions within workshops held under Chatham house rules, while others have created new formal organizations, conferences such as AIES or FAccT, or research institutes.”
BackgroundSelf-citationEthics as a Service: A Pragmatic Operationalisation of AI Ethics
Morley, J., Elhalal, A., Garcia, F., Kinsey, L., Mökander, J., & Floridi, L. (2021) · Minds and Machines · doi:10.1007/s11023-021-09563-w · Cited by 257 · see citing works
“However, such audits inevitably impose both financial and administrative costs (Brundage et al., 2020).”
BackgroundEthics-Based Auditing to Develop Trustworthy AI
Mökander, J., & Floridi, L. (2021) · Minds and Machines · doi:10.1007/s11023-021-09557-8 · Cited by 268 · see citing works
Trustworthy ai: A computational perspective
Liu, H., Wang, Y., Fan, W., Liu, X., Li, Y., Jain, S., Liu, Y., Jain, A., & Tang, J. (2023) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3546872 · Cited by 261 · see citing works
“Based on these four principles, AI researchers, practitioners, and governments propose various speciic dimensions for trustworthy AI [55, 325, 337].”
BackgroundPutting AI ethics to work: are the tools fit for purpose?
Ayling, J., & Chapman, A. (2022) · AI and Ethics · doi:10.1007/s43681-021-00084-x · Cited by 253 · see citing works
“Brundage et al. define as this as problem space for current AI production in that they ‘lack traceable logs of steps taken in problem-definition, design, development, and operation, leading to a lack of accountability for subsequent claims about those systems’ properties and impacts’ [55].”
BackgroundGoverning AI safety through independent audits
Falco, G., Shneiderman, B., Badger, J., Carrier, R., Dahbura, A., Danks, D., Eling, M., Goodloe, A., Gupta, J., Hart, C., et al. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00370-7 · Cited by 250 · see citing works
“Various industry, academic, government bodies, and notfor-profit organizations have proposed AI principles in recent years, with many sharing core concepts and mechanisms [1], [11].”
BackgroundIntroduction to artificial intelligence in ultrasound imaging in obstetrics and gynecology
Drukker, L., Noble, J. A., & Papageorghiou, A. T. (2020) · Ultrasound in Obstetrics & Gynecology · doi:10.1002/uog.22122 · Cited by 240 · see citing works
“Towards this goal, a recent multinational academic report addressing both medical and non-medical AI systems, entitled ‘Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims’ 61 , provides a list of measures and mechanisms for AI developers and regulatory bodies to ensure responsible AI develop…”
BackgroundInterpretable Machine Learning
Boehmke, B., & Greenwell, B. (2019) · Chapman & Hall/CRC eBooks · doi:10.1201/9780367816377-16 · Cited by 271 · see citing works
Predictability and Surprise in Large Generative Models
Ganguli, D., Hernandez, D., Lovitt, L., Askell, A., Bai, Y., Chen, A., Conerly, T., Dassarma, N., Drain, D., Elhage, N., et al. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533229 · Cited by 207 · see citing works
Self-citationAkinrinola, O., Okoye, C. C., Ofodile, O. C., & Ugochukwu, C. E. (2024) · GSC Advanced Research and Reviews · doi:10.30574/gscarr.2024.18.3.0088 · Cited by 206 · see citing works
“Open data sharing also allows for external scrutiny, enabling researchers, developers, and the public to assess the quality and fairness of datasets used for training AI models (Brundage et al., 2020).”
BackgroundAuditing large language models: a three-layered approach
Mökander, J., Schuett, J., Kirk, H. R., & Floridi, L. (2024) · AI and Ethics · doi:10.1007/s43681-023-00289-2 · Cited by 289 · see citing works
Who Audits the Auditors? Recommendations from a field scan of the algorithmic auditing ecosystem
Costanza-Chock, S., Raji, I. D., & Buolamwini, J. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533213 · Cited by 191 · see citing works
“For example, some argue that AI impact assessments are a more effective accountability mechanism than audits, or that red teams should be the preferred approach [8, 53].”
BackgroundHelp Me Help the AI": Understanding How Explainability Can Support Human-AI Interaction
Kim, S. S. Y., Watkins, E. A., Russakovsky, O., Fong, R., & Monroy-Hernández, A. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581001 · Cited by 196 · see citing works
“See [7, 19, 41, 46, 49, 107, 109] for more comprehensive overviews.”
Self-citationRapid Trust Calibration through Interpretable and Uncertainty-Aware AI
Tomsett, R., Preece, A., Braines, D., Cerutti, F., Chakraborty, S., Srivastava, M., Pearson, G., & Kaplan, L. (2020) · Patterns · doi:10.1016/j.patter.2020.100049 · Cited by 186 · see citing works
Singhal, A., Neveditsin, N., Tanveer, H., & Mago, V. (2024) · JMIR Medical Informatics · doi:10.2196/50048 · Cited by 172 · see citing works
“Such transparency fosters enhanced scrutiny and accountability of the system by external parties, including regulators and the general public [95].”
BackgroundArtificial Intelligence Governance For Businesses
Schneider, J., Abraham, R., Meske, C., & Brocke, J. V. (2022) · Information Systems Management · doi:10.1080/10580530.2022.2085825 · Cited by 227 · see citing works
“Brundage et al. (2020) presented data governance intending to achieve trust.”
BackgroundInfluentialKourou, K. D., Exarchos, K., Papaloukas, C. L., Sakaloglou, P., Exarchos, T., & Fotiadis, D. I. (2021) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2021.10.006 · Cited by 167 · see citing works
“According to the guidelines for the ethical development of ML-based systems [101], an ethics by design approach has been proposed for trust-worthy AI/ML for GDPR-compliant, ethical and robust systems.”
Operationalising ethics in artificial intelligence for healthcare: a framework for AI developers
Solanki, P., Grundy, J., & Hussain, W. (2023) · AI and Ethics · doi:10.1007/s43681-022-00195-z · Cited by 157 · see citing works
“Perform ethics-based auditing to compare aspired vs practiced ethics and take corrective measures [150, 151]”
Background3DALL-E: Integrating Text-to-Image AI in 3D Design Workflows
Liu, V., Vermeulen, J., Fitzmaurice, G. W., & Matejka, J. (2023) · Designing Interactive Systems Conference · doi:10.1145/3563657.3596098 · Cited by 179 · see citing works
“These models have extraordinary generative capacity, but their ability to be used nefariously has also inspired new approaches to safeguarding AI outputs from redteaming [6] to large scale audits for social and gender biases [12].”
BackgroundMapping global AI governance: a nascent regime in a fragmented landscape
Schmitt, L. (2021) · AI and Ethics · doi:10.1007/s43681-021-00083-y · Cited by 152 · see citing works
Mökander, J., Axente, M., Casolari, F., & Floridi, L. (2022) · Minds and Machines · doi:10.1007/s11023-021-09577-4 · Cited by 153 · see citing works
“Because external audits help address concerns about the incentives for accuracy in self-reporting (Brundage et al., 2020), they are typically required for formal verification and certification procedures (Yanisky-Ravid & Hallisey, 2018).”
BackgroundEthical machines: The human-centric use of artificial intelligence
Lepri, B., Oliver, N., & Pentland, A. (2021) · iScience · doi:10.1016/j.isci.2021.102249 · Cited by 149 · see citing works
Trustworthy Graph Neural Networks: Aspects, Methods, and Trends
Zhang, H., Wu, B. Y., Yuan, X., Pan, S., Tong, H., & Pei, J. (2024) · Proceedings of the IEEE · doi:10.1109/jproc.2024.3369017 · Cited by 173 · see citing works
“There are a number of fields in which “trust” is defined and studied, including philosophy, psychology, sociology, economics, technology, and organizational management [10], [12], [13].”
BackgroundAuditing of AI: Legal, Ethical and Technical Approaches
Mökander, J. (2023) · Digital Society · doi:10.1007/s44206-023-00074-y · Cited by 152 · see citing works
“However, previous research suggests that technology providers too have strong incentives to subject the AI systems they design and deploy to independent audits (Falco et al., 2021; Raji et al., 2020).”
Rewards, risks and responsible deployment of artificial intelligence in water systems
Richards, C. E., Tzachor, A., Avin, S., & Fenner, R. A. (2023) · Nature Water · doi:10.1038/s44221-023-00069-6 · Cited by 141 · see citing works
Self-citationReviewable Automated Decision-Making: A Framework for Accountable Algorithmic Systems
Cobbe, J., Lee, M. S. A., & Singh, J. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445921 · Cited by 150 · see citing works
“Any such testing should be presented in the form of “verifiable claims” with audit trails and explanations of model predictions [9].”
BackgroundHarms from Increasingly Agentic Algorithmic Systems
Chan, A., Salganik, R., Markelius, A., Pang, C., Rajkumar, N., Krasheninnikov, D., Langosco, L., He, Z., Duan, Y., Carroll, M., et al. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594033 · Cited by 149 · see citing works
“The likely failures of a more agentic system (relative to what has been deployed already) are certainly a barrier to deployment – given the disproportionate impact of these failures on already marginalized groups, we would hope and advocate for restrictions on deployment [Brundage et al. 2020].”
BackgroundMethodInfluentialSelf-citationZ-Inspection <sup>®</sup> : A Process to Assess Trustworthy AI
Zicari, R. V., Brodersen, J., Brusseau, J., Dudder, B., Eichhorn, T., Ivanov, T., Kararigas, G., Kringen, P., McCullough, M., Moslein, F., et al. (2021) · IEEE Transactions on Technology and Society · doi:10.1109/tts.2021.3066209 · Cited by 134 · see citing works
“Brundage et al. [13] further stated that “techniques and best practices have not yet been established for auditing AI systems.””
BackgroundResultInfluentialSustainable AI: An integrated model to guide public sector decision-making
Wilson, C., & van der Velden, M. (2022) · Technology in Society · doi:10.1016/j.techsoc.2022.101926 · Cited by 134 · see citing works
Actionable Principles for Artificial Intelligence Policy: Three Pathways
Stix, C. (2021) · Science and Engineering Ethics · doi:10.1007/s11948-020-00277-3 · Cited by 129 · see citing works
“…with their ability to shape the implications and features of their AI development process, and from industry actors who are in a position to create more diverse hiring processes (Crawford 2016 ), to civil society actors with their power to demand and engage in multi-stakeholder dialogues.”
BackgroundSelf-citationMökander, J., Axente, M., Casolari, F., & Floridi, L. (2022) · PhilPapers (PhilPapers Foundation) · Cited by 115 · see citing works
Artificial intelligence explainability: the technical and ethical dimensions
McDermid, J. A., Jia, Y., Porter, Z., & Habli, I. (2021) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2020.0363 · Cited by 136 · see citing works
A Comparison of Natural Language Understanding Platforms for Chatbots in Software Engineering
Abdellatif, A., Badran, K., Costa, D. E., & Shihab, E. (2021) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2021.3078384 · Cited by 106 · see citing works
González-Gonzalo, C., Thee, E. F., Klaver, C. C. W., Lee, A. Y., Schlingemann, R. O., Tufail, A., Verbraak, F., & Sánchez, C. I. (2022) · Progress in Retinal and Eye Research · doi:10.1016/j.preteyeres.2021.101034 · Cited by 103 · see citing works
“A main problem for reproducibility is that AI systems lack traceable logs of steps taken in problem-definition, design, development, and operation, decreasing the ability of outside parties to verify claims made about AI systems and leading to a lack of accountability for subsequent claims about those systems’ propert…”
BackgroundArtificial intelligence in government: Concepts, standards, and a unified framework
Straub, V. J., Morgan, D., Bright, J., & Margetts, H. Z. (2023) · Government Information Quarterly · doi:10.1016/j.giq.2023.101881 · Cited by 120 · see citing works
Koshiyama, A., Kazim, E., Treleaven, P., Rai, P., Szpruch, L., Pavey, G., Ahamat, G., Leutner, F., Goebel, R., Knight, A., et al. (2024) · Royal Society Open Science · doi:10.1098/rsos.230859 · Cited by 110 · see citing works
“Issues of explainability, fairness, privacy, governance and robustness are now popular research themes among AI researchers—an area that falls under the umbrella of ‘trustworthy AI’ [1].”
BackgroundInfluentialSelf-citationExplaining the Principles to Practices Gap in AI
Schiff, D. S., Rakova, B., Ayesh, A., Fanti, A., & Lennon, M. (2021) · IEEE Technology and Society Magazine · doi:10.1109/mts.2021.3056286 · Cited by 93 · see citing works
“Structural elements include software and engineering development processes [37], their integration with larger business development cycles, and organizational incentives and hierarchy.”
BackgroundInfluentialJacobs, A. S., Beltiukov, R., Willinger, W., Ferreira, R. A., Gupta, A. K., & Granville, L. Z. (2022) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3548606.3560609 · Cited by 91 · see citing works
“, łinterpretable MLž [51], łexplainable AI (XAI)ž [59], and łtrustworthy AIž [12]).”
InfluentialTariq, A., Serhani, M. A., Sallabi, F., Barka, E. S., Qayyum, T., Khater, H. M., & Shuaib, K. (2024) · IEEE Open Journal of the Communications Society · doi:10.1109/ojcoms.2024.3438264 · Cited by 110 · see citing works
Trustworthy safety improvement for autonomous driving using reinforcement learning
Cao, Z. L., Xu, S., Jiao, X., et al. (2022) · Transportation Research Part C Emerging Technologies · doi:10.1016/j.trc.2022.103656 · Cited by 85 · see citing works
Edge Intelligence—Research Opportunities for Distributed Computing Continuum Systems
Pujol, V. C., Donta, P. K., Morichetta, A., Murturi, I., & Dustdar, S. (2023) · IEEE Internet Computing · doi:10.1109/mic.2023.3284693 · Cited by 84 · see citing works
“It is essential to build systems and processes where accountability, transparency, and integrity are first-class citizens.(96) In the last years, researchers have been inspecting ways to address this.”
BackgroundIbáñez, J., & Olmeda, M. V. (2021) · Ai & Society · doi:10.1007/s00146-021-01267-0 · Cited by 88 · see citing works
“Several studies have reviewed the various initiatives, guidelines, and principles (Brundage et al. 2020; Fjeld et al. 2020; Greene et al. 2019; Whittlestone et al. 2019).”
BackgroundCiravegna, G., Barbiero, P., Giannini, F., Gori, M., Lió, P., Maggini, M., & Melacci, S. (2022) · Artificial Intelligence · doi:10.1016/j.artint.2022.103822 · Cited by 88 · see citing works
Black-Box Access is Insufficient for Rigorous AI Audits
Casper, S., Ezell, C., Siegmann, C., Kolt, N., Curtis, T. L., Bucknall, B., Haupt, A. A., Wei, K., Scheurer, J., Hobbhahn, M., et al. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3659037 · Cited by 104 · see citing works
“Norms for AI audits are not yet well established, and their effectiveness can vary depending on the degree of system access granted to auditors [11, 33, 252].”
BackgroundSelf-citationThe Gradient of Generative AI Release: Methods and Considerations
Solaiman, I. (2023) · 2023 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3593013.3593981 · Cited by 90 · see citing works
“Bounty programs, from bug bounties to bias bounties [82], can raise unforeseen safety issues and strengthen trust in a system [18].”
BackgroundPetersen, E., Potdevin, Y., Mohammadi, E., Zidowitz, S., Breyer, S., Nowotka, D., Henn, S., Pechmann, L., Leucker, M., Rostalski, P., et al. (2022) · IEEE Access · doi:10.1109/access.2022.3178382 · Cited by 83 · see citing works
“Moreover, parts of this responsible process can be formalized, by, for example, invoking a red team responsible for breaking the MML system [228], stan-dardizing algorithm audits [88], or aiming for an inclusive and transparent development process [7].”
BackgroundMethodHIVE: Evaluating the Human Interpretability of Visual Explanations
Kim, S. S. Y., Meister, N., Ramaswamy, V. V., Fong, R., & Russakovsky, O. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-19775-8_17 · Cited by 72 · see citing works
“Interpretability research can be described along several axes: first, whether a method is post-hoc or interpretable-by-design; second, whether it is global or local; and third, the form of an explanation (see [4,11,16,24,28,30,57,59] for surveys).”
BackgroundSelf-citationMacrae, C. (2022) · Risk Analysis · doi:10.1111/risa.13850 · Cited by 76 · see citing works
Principles to Practices for Responsible AI: Closing the GapPreprint
Schiff, D. S., Rakova, B., Ayesh, A., Fanti, A., & Lennon, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.04707 · Cited by 65 · see citing works
The alignment problem from a deep learning perspective
Ngo, R., Chan, L., & Mindermann, S. (2022) · arXiv · doi:10.48550/arxiv.2209.00626 · Cited by 68 · see citing works
“Such cooperation would become more viable given mechanisms for allowing AI developers to certify properties of training runs without leaking information about the code or data they used [Brundage et al., 2020].”
BackgroundAdversarial interference and its mitigations in privacy-preserving collaborative machine learning
Usynin, D., Ziller, A., Makowski, M. R., Braren, R., Rueckert, D., Glocker, B., Kaissis, G. A., & Passerat‐Palmbach, J. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00390-3 · Cited by 64 · see citing works
Mökander, J., & Axente, M. (2021) · Ai & Society · doi:10.1007/s00146-021-01286-x · Cited by 49 · see citing works
“For example, the sharing of past systems’ failures could help mitigate future harms (Brundage et al., 2020).”
BackgroundInfluentialIt is Not “Accuracy vs. Explainability”—We Need Both for Trustworthy AI Systems
Petkovic, D. (2023) · IEEE Transactions on Technology and Society · doi:10.1109/tts.2023.3239921 · Cited by 63 · see citing works
“the internal development stage, as per guidance in [7], [8], [9], [10], [11], [12], [13], [14], [17], [18] and proportional to anticipated risk and safety issues.”
BackgroundLM vs LM: Detecting Factual Errors via Cross Examination
Cohen, R., Hamri, M., Geva, M., & Globerson, A. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.778 · Cited by 60 · see citing works
Mediating artificial intelligence developments through negative and positive incentives
Han, T. A., Pereira, L. M., Lenaerts, T., & Santos, F. C. (2021) · PLoS ONE · doi:10.1371/journal.pone.0244592 · Cited by 57 · see citing works
“Future studies may look at more complex incentivising mechanisms [47] such as reputation and public image manipulation [51,52], emotional motives of guilt and apology-forgiveness [53,54], institutional and coordinated incentives [28, 36], and the subtle combination of different forms of incentive (e.”
BackgroundFilling gaps in trustworthy development of AI
Avin, S., Belfield, H., Brundage, M., Krueger, G. M., Wang, J., Weller, A., Anderljung, M., Krawczuk, I., Krueger, D., Lebensold, J., et al. (2021) · Science · doi:10.1126/science.abi7176 · Cited by 57 · see citing works
Self-citationZicari, R. V., Brusseau, J., Blomberg, S. N. F., Christensen, H. C., Coffee, M. P., Ganapini, M. B., Gerke, S., Gilbert, T. K., Hickman, E., Hildt, E., et al. (2021) · Frontiers in Human Dynamics · doi:10.3389/fhumd.2021.673104 · Cited by 56 · see citing works
“One way to understand this use case we have been looking at is to see the “ medical discussions ” as possible forms of validation metrics rather than simply mechanisms for “ veri fi able claims, ” as OpenAI has recently argued (Brundage et al., 2020).”
MethodSelf-citationRengasamy, D., Rothwell, B. C., & Figueredo, G. P. (2021) · Applied Sciences · doi:10.3390/app112411854 · Cited by 54 · see citing works
Koshiyama, A., Kazim, E., & Treleaven, P. (2022) · Computer · doi:10.1109/mc.2021.3067225 · Cited by 55 · see citing works
Ethical implications of AI in the Metaverse
Zhuk, A. (2024) · AI and Ethics · doi:10.1007/s43681-024-00450-5 · Cited by 56 · see citing works
“Transparent reporting of AI development and deployment processes can foster trust among users, enabling them to hold AI systems accountable [34],Brundage et al. [11].”
Horneber, D., & Laumer, S. (2023) · Business & Information Systems Engineering · doi:10.1007/s12599-023-00817-8 · Cited by 81 · see citing works
“Another frequently suggested accountability mechanism, that can be implemented via independent external stakeholders, is the collection and reporting of critical incidents (Brundage et al. 2020).”
BackgroundTowards training reproducible deep learning models
Chen, B., Wen, M., Shi, Y., Lin, D., Rajbahadur, G. K., & Jiang, Z. M. (2022) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3510003.3510163 · Cited by 50 · see citing works
“, if/else), the DL models within AI-based systems are constructed in a stochastic way due to the underlying DL algorithms, whose behavior may not be reproducible and trustworthy [26, 54].”
BackgroundCertification Labels for Trustworthy AI: Insights From an Empirical Mixed-Method Study
Scharowski, N., Benk, M., Kühne, S. J., Wettstein, L., & Brühlmann, F. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3593994 · Cited by 53 · see citing works
“Motivated by these challenges, this work builds on research highlighting the pivotal role of auditability as an enabler of trust in AI [7, 65] and its crucial role in creating an "AI trustworthiness ecosystem" [2] by ensuring that the principles of trustworthy AI are met.”
A Framework for Assurance Audits of Algorithmic Systems
Lam, K., Lange, B., Blili-Hamelin, B., Davidovic, J., Brown, S., & Hasan, A. (2024) · The 2024 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3630106.3658957 · Cited by 48 · see citing works
“15 This mirrors the standard procedures for obtaining evidence in finan-cial audits [55] and builds upon proposals calling for adaptation of claim-based assurance frameworks to the AI context [11, 32].”
MethodHagendorff, T. (2021) · Minds and Machines · doi:10.1007/s11023-021-09573-8 · Cited by 47 · see citing works
“By following such an integrated approach, unintended cognitive biases in the selection off intended training data biases can be tracked, but it can also be ensured that the selection of values and traits stands in accordance with democratic consensuses (Floridi et al., 2018; Brundage et al. 2020; Daly et al. 2019b; Da…”
BackgroundProtecting society from AI misuse: when are restrictions on capabilities warranted?
Anderljung, M., Hazell, J., & von Knebel, M. (2024) · AI & Society · doi:10.1007/s00146-024-02130-8 · Cited by 65 · see citing works
“Further, they can invite external red teams to scrutinize their models and provide assessments before model release or deployment [11].”
Self-citationThe Foundation Model Transparency Index
Bommasani, R., Klyman, K., Longpre, S., Kapoor, S., Maslej, N., Xiong, B., Zhang, D., & Liang, P. (2023) · arXiv · doi:10.48550/arxiv.2310.12941 · Cited by 46 · see citing works
Logic-oriented fuzzy neural networks: A survey
Alateeq, M. M., & Pedrycz, W. (2024) · Expert systems with applications · doi:10.1016/j.eswa.2024.125120 · Cited by 41 · see citing works
Ethics as a service: a pragmatic operationalisation of AI EthicsPreprint
Morley, J., Elhalal, A., Garcia, F., Kinsey, L., Mökander, J., & Floridi, L. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3784238 · Cited by 45 · see citing works
Digital ethicswashing: a systematic review and a process-perception-outcome framework
Schultz, M. D., Conti, L. G., & Seele, P. (2024) · AI and Ethics · doi:10.1007/s43681-024-00430-9 · Cited by 46 · see citing works
“Moreover, many scholars are calling for the new regulations to include external oversight by an independent entity with enforcement powers to make the violators accountable [14, 45, 75, 126].”
Isotope identification using deep learning: An explanation
Gomez-Fernandez, M., Wong, W., Tokuhiro, A., Welter, K. B., Alhawsawi, A. M., Yang, H., & Higley, K. A. (2020) · Nuclear Instruments and Methods in Physics Research Section A Accelerators Spectrometers Detectors and Associated Equipment · doi:10.1016/j.nima.2020.164925 · Cited by 41 · see citing works
AI Regulation Is (not) All You Need
Lucaj, L., van der Smagt, P., & Benbouzid, D. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594079 · Cited by 40 · see citing works
“Auditing is not new and has significantly con-tributedtopromotingaccountabilityandconsistency inotherhighly-regulated industries such as finance and air mobility [13, 74].”
InfluentialBeyond Text-to-Image: Multimodal Prompts to Explore Generative AI
Liu, V. (2023) · Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544549.3577043 · Cited by 38 · see citing works
“Auditing work has unearthed social and gender biases [6], and strategies such as redteaming [5] have been implemented to discover and patch loopholes that can produce nefarious outputs.”
BackgroundResponsible AI: A Primer for the Legal Community
Golbin, I., Rao, A. S., Hadjarian, A., & Krittman, D. (2020) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata50022.2020.9377738 · Cited by 38 · see citing works
“These include audit trails for accountability of high-stakes or regulated AI systems, interpretability to foster better understanding and scrutiny of underlying models, and privacy-preserving machine learning to help confirm privacy protection [34].”
BackgroundInfluentialTowards Verifiable Federated Learning
Zhang, Y., & Yu, H. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/792 · Cited by 38 · see citing works
Monitoring and Diagnosability of Perception Systems
Antonante, P., Spivak, D. I., & Carlone, L. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636497 · Cited by 37 · see citing works
“In particular, we address two questions (adapted from [47]): (i) Can I (as a developer) verify that the perception”
MethodComparative Analysis of Generative AI Risks in the Public Sector
Navarro, M. A. B., Mondragón, M. I. R., & Han, S. H. (2024) · Annual International Conference on Digital Government Research · doi:10.1145/3657054.3657125 · Cited by 43 · see citing works
Multilayered review of safety approaches for machine learning-based systems in the days of AI
Dey, S., & Lee, S.-W. (2021) · Journal of Systems and Software · doi:10.1016/j.jss.2021.110941 · Cited by 37 · see citing works
“, not having a formal consensus on metric to measure general property like safety of these systems, there will be always a tension between the verifiability claims and the generality of such claims [160].”
BackgroundLearning to run a power network with trust
Marot, A., Donnot, B., Chaouache, K., Kelly, A., Huang, Q., Hossain, R. R., & Cremer, J. L. (2022) · Electric Power Systems Research · doi:10.1016/j.epsr.2022.108487 · Cited by 34 · see citing works
“amount of evidence it has gathered[28], [7].”
Carugati, F., & Levi, M. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108872942 · Cited by 38 · see citing works
Self-citationThe impact of Artificial Intelligence on productivity, distribution and growth
Filippucci, F., Gál, P., Jona‐Lasinio, C., Leandro, Á., & Nicoletti, G. (2024) · OECD artificial intelligence papers · doi:10.1787/8d900037-en · Cited by 85 · see citing works
Aljanabi, M. (2023) · Babylonian Journal of Internet of Things · doi:10.58496/bjiot/2023/005 · Cited by 34 · see citing works
“This paper specifically focuses on augmenting IoMT anomaly detectors and IDS components with trustworthy AI techniques to strengthen reliability and security[8-11].”
BackgroundExplainable Machine Learning with Prior Knowledge: An Overview
Beckh, K., Müller, S., Jakobs, M., Toborek, V., Tan, H., Fischer, R., Welke, P., Houben, S., & von Rueden, L. (2021) · Lancaster EPrints (Lancaster University) · doi:10.48550/arxiv.2105.10172 · Cited by 40 · see citing works
“With trustworthy artificial intelligence (AI) becoming more important [10], another purpose of IML is to ensure knowledge conformity or to improve the interpretability of a model [78].”
BackgroundSeger, E., Dreksler, N., Moulange, R., Dardaman, E., Schuett, J., Wei, K., Winter, C., Arnold, M., Ó hÉigeartaigh, S., Korinek, A., et al. (2023) · doi:10.2139/ssrn.4596436 · Cited by 31 · see citing works
“Another possibility Brundage et al. [136] suggests, is to extend red-teams to elicit input from a wider community of ‘red-team professionals’ .”
BackgroundSelf-citationLeung, R., Hill, A. J., & Melkumyan, A. (2023) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2023.3328457 · Cited by 30 · see citing works
Discerning Between the “Easy” and “Hard” Problems of AI Governance
Minkkinen, M., & Mäntymäki, M. (2023) · IEEE Transactions on Technology and Society · doi:10.1109/tts.2023.3267382 · Cited by 33 · see citing works
“However, given a particular stage of AI regulation, organizations and AI systems can reach a sufficient level of compliance, which can be verified by auditing and independent oversight, given that the required governance mechanisms are available [34].”
BackgroundLessons Learned from Assessing Trustworthy AI in Practice
Vetter, D., Amann, J., Bruneault, F., Coffee, M. P., Düdder, B., Gallucci, A., Gilbert, T. K., Hagendorff, T., van Halem, I., Hickman, E., et al. (2023) · Digital Society · doi:10.1007/s44206-023-00063-1 · Cited by 31 · see citing works
Self-citation
- Authors
- Brundage, M., Avin, S., Wang, J., Belfield, H., Krueger, G., Hadfield, G., Khlaaf, H., Yang, J., Toner, H., Fong, R., Maharaj, T., Koh, P. W., Hooker, S., Leung, J., Trask, A., Bluemke, E., Lebensold, J., O'Keefe, C., Koren, M., Ryffel, T., Rubinovitz, J., Besiroglu, T., Carugati, F., Clark, J., Eckersley, P., de Haas, S., Johnson, M., Laurie, B., Ingerman, A., Krawczuk, I., Askell, A., Cammarota, R., Lohn, A., Krueger, D., Stix, C., Henderson, P., Graham, L., Prunkl, C., Martin, B., Seger, E., Zilberman, N., hÉigeartaigh, S. Ó., Kroeger, F., Sastry, G., Kagan, R., Weller, A., Tse, B., Barnes, E., Dafoe, A., Scharre, P., Herbert-Voss, A., Rasser, M., Sodhani, S., Flynn, C., Gilbert, T. K., Dyer, L., Khan, S., Bengio, Y., & Anderljung, M.
- Affiliations
- Stanford University (US)
- Centre Universitaire de Mila (DZ)
- University of Toronto (CA)
- Adelard (GB)
- Center for Security and Emerging Technology (US)
- University of Oxford (GB)
- École Normale Supérieure - PSL (FR)
- University of Cambridge (GB)
- Google (United States) (US)
- École Polytechnique Fédérale de Lausanne (CH)
- RAND Corporation (US)
- Eindhoven University of Technology (NL)
- Coventry University (GB)
- Center for a New American Security (US)
- University of California, Berkeley (US)
- ORCID
- Avin, Shahar
- Belfield, Haydn
- Hadfield, Gillian
- Yang, Jingying
- Fong, Ruth
- Maharaj, Tegan
- Koh, Pang Wei
- Hooker, Sara
- Trask, Andrew
- Bluemke, Emma
- Ryffel, Théo
- Besiroglu, Tamay
- Carugati, Federica
- Laurie, Ben
- Ingerman, Alex
- Krawczuk, Igor
- Cammarota, Rosario
- Lohn, Andrew
- Krueger, David
- Stix, Charlotte
- Henderson, Peter
- Prunkl, Carina
- Seger, Elizabeth
- Zilberman, Noa
- hÉigeartaigh, Seán Ó
- Kroeger, Frens
- Weller, Adrian
- Barnes, Elizabeth
- Dafoe, Allan
- Sodhani, Shagun
- Gilbert, Thomas Krendl
- Khan, Saif
- Bengio, Yoshua
- Anderljung, Markus
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2020
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2004.07213
- OpenAlex
- W4287552728
- Cited by
- 306
- References
- 230
- Citations by year
- 2026: 11 · 2025: 39 · 2024: 52 · 2023: 80 · 2022: 42 · 2021: 65 · 2020: 16 · 2019: 1
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- HAL (submittedVersion, other-oa)
- Pure (Coventry University) (submittedVersion)
- TU/e Research Portal (submittedVersion, other-oa)
- arXiv (Cornell University)
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Safety Systems Engineering in Autonomy
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Keywords
- trustworthy AI, responsible AI development, accountability, AI safety, AI security, algorithmic fairness, privacy preservation, stakeholder scrutiny, verification methods, AI governance
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
