AI Regulation Is (not) All You Need
Open accessThe DOI matches its Crossref record; 60 of the 60 references it cites were found in a registry or an index.
Abstract
The development of processes and tools for ethical, trustworthy, and legal AI is only beginning. At the same time, legal requirements are emerging in various jurisdictions, following a deluge of ethical guidelines. It is therefore key to explore the necessary practices that must be adopted to ensure the quality of AI systems, mitigate their potential risks and enable legal compliance. Ensuring that the potential negative impacts of AI on individuals, society, and the environment are mitigated will depend on many factors, including the capacity to properly regulate its deployment and to mandate necessary internal best practices along lifecycles. Regulatory frameworks must evolve from abstract requirements to providing concrete operational mandates that enable better oversight mechanisms in the way AI systems operate, how they are developed, and how they are deployed. In view of the above, this paper explores the necessary practices that can be adopted throughout a comprehensive lifecycle audit as a key practice to ensure the quality of AI systems and enable the development of compliance mechanisms. It also discusses novel governance tools that enable bridging the current operational gaps. Such gaps were identified by interviewing experts, analysing adaptable tools and methodologies from the software engineering domain, and by exploring the state of the art of auditing. The results present recommendations for novel tools and oversight mechanisms for governing AI systems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In addition, in cases where substantive differences of opinion exist between expertssuch as in the discourse on computational definitions of fairness [6,8,37]-designers of audit standards must navigate these nuances while also balancing to achieve a set of criteria that provides compliance and is operationalizable at scale.”
From A Framework for Assurance Audits of Algorithmic Systems (2024) · cites this paper as [37]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The global landscape of AI ethics guidelines
Jobin, A., Ienca, M., & Vayena, E. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0088-2 · Cited by 6,220 · cited 1×
“A significant amount of research has been devoted to crafting ethical guidelines for practitioners, to guide their development of AI systems [27, 35, 38].”
Doyle, T. (2017) · The Information Society · doi:10.1080/01972243.2017.1354593 · Cited by 3,071 · cited 5×
“Certification emerged throughout four interviews as fundamental in improving the ecosystem (Candi-dates 1, 3, 4, 5).”
InfluentialModel Cards for Model Reporting
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287596 · Cited by 2,681 · cited 4×
“Emerging guidelines have addressed documentation practices for certain features and particular steps along the pipeline of an ML model such as model cards and datasheets [30, 51].”
InfluentialHierarchical Text-Conditional Image Generation with CLIP Latents
Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., & Chen, M. (2022) · arXiv · doi:10.48550/arxiv.2204.06125 · Cited by 2,511 · cited 1×
“AI systems are rapidly transforming many areas of life with disruptive applications, such as the impressive advancement of generative AI models like ChatGPT, a text-generating chatbot, or the text-to-image generation provided by cascaded diffusion models such as DALL·E [64].”
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Daumé, H., & Crawford, K. (2018) · arXiv · doi:10.48550/arxiv.1803.09010 · Cited by 2,474 · cited 5×
“Datasheets are inspired by the standardised forms of information sharing and rigorous testing, established in the electronic hardware industry, to enable the overview of the components performances under different test conditions [30].”
InfluentialGebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., III, H. D., & Crawford, K. (2021) · Communications of the ACM · doi:10.1145/3458723 · Cited by 2,469 · cited 5×
“Datasheets are inspired by the standardised forms of information sharing and rigorous testing, established in the electronic hardware industry, to enable the overview of the components performances under different test conditions [30].”
InfluentialRaji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372873 · Cited by 2,245 · cited 9×
“This process allows the developers to detect the potential unintended consequences before the deployment [63].”
InfluentialThe Ethics of AI Ethics: An Evaluation of Guidelines
Hagendorff, T. (2020) · Minds and Machines · doi:10.1007/s11023-020-09517-8 · Cited by 1,911
The Ethics of AI Ethics -- An Evaluation of Guidelines
Hagendorff, T. (2019) · arXiv · doi:10.48550/arxiv.1903.03425 · Cited by 1,907
Bellamy, R. K. E., Dey, K., Hind, M., Hoffman, S. C., Houde, S., Kannan, K., Lohia, P., Martino, J., Mehta, S., Mojsilovic, A., et al. (2018) · arXiv · doi:10.48550/arxiv.1810.01943 · Cited by 1,581
(2002) · Routledge eBooks · doi:10.4324/9780203413081 · Cited by 1,445
Software Engineering for Machine Learning: A Case Study
Amershi, S., Begel, A., Bird, C., DeLine, R., Gall, H., Kamar, E., Nagappan, N., Nushi, B., & Zimmermann, T. (2019) · 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1109/icse-seip.2019.00042 · Cited by 1,135 · cited 2×
“Looking at the software engineering field can provide insights into the essential technical challenges that arise as organisations develop large-scale AI solutions, such as keeping the systems robust and secure, as well as the adoption of quality assurance practices [1, 13].”
Handbook of Research Methods in Health Social Sciences
(2019) · doi:10.1007/978-981-10-5251-4 · Cited by 1,082
Shneiderman, B. (2022) · doi:10.1093/oso/9780192845290.001.0001 · Cited by 955
AI Fairness 360: An extensible toolkit for detecting and mitigating algorithmic bias
Bellamy, R. K. E., Dey, K., Hind, M., Hoffman, S. C., Houde, S., Kannan, K., Lohia, P., Martino, J., Mehta, S., Mojsilović, A., et al. (2019) · IBM Journal of Research and Development · doi:10.1147/jrd.2019.2942287 · Cited by 884 · cited 1×
“diversity, bias in datasets [5, 9, 72] or explainability [4, 5, 53] have been developed.”
BackgroundShneiderman, B. (2020) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3419764 · Cited by 768
Raji, I. D., & Buolamwini, J. (2019) · Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3306618.3314244 · Cited by 651 · cited 1×
“Algorithmic auditing of ML systems has gained considerable recognition as an opportunity to harness the potential of ML models, as well as detect and mitigate the problematic patterns and consequences of their deployment in sensitive decision-making contexts [42, 62, 80].”
Software maintenance and evolution: a roadmap
Bennett, K. H., & Rajlich, V. T. (2000) · Proceedings of the Conference on The Future of Software Engineering · doi:10.1145/336512.336534 · Cited by 606 · cited 1×
“Similarly to the field of AI, software engineering requires necessary maintenance and evolution practices that carry enormous costs and implementation challenges [10, 76].”
Recommender systems and their ethical challenges
Milano, S., Taddeo, M., & Floridi, L. (2020) · AI & SOCIETY · doi:10.1007/s00146-020-00950-y · Cited by 556 · cited 1×
“Ethical challenges with the deployment of AI systems are not a new phenomenon and have been widely recognised in the literature [16, 43, 49].”
BackgroundAequitas: A Bias and Fairness Audit Toolkit
Saleiro, P., Kuester, B., Hinkson, L., London, J., Stevens, A., Anisfeld, A., Rodolfa, K. T., & Ghani, R. (2018) · arXiv · doi:10.48550/arxiv.1811.05577 · Cited by 496 · cited 1×
“Hence, a number of auditing tools and frameworks tackle specific areas, e.g. diversity, bias in datasets [5, 9, 72] or explainability [4, 5, 53] have been developed.”
Ethical and social risks of harm from Language Models
Weidinger, L., Mellor, J., Rauh, M., Griffin, C., Uesato, J., Huang, P.-S., Cheng, M., Glaese, M., Balle, B., Kasirzadeh, A., et al. (2021) · arXiv · doi:10.48550/arxiv.2112.04359 · Cited by 477 · cited 1×
“The risks associated with the deployment of ML models have been widely analysed and range from robustness issues due to private data leaks to providing and spreading misinformation, or the manipulation of users leading them to overestimate its capabilities and use it unsafely [79].”
BackgroundFactSheets: Increasing trust in AI services through supplier's declarations of conformity
Arnold, M., Bellamy, R. K. E., Hind, M., Houde, S., Mehta, S., Mojsilović, A., Nair, R., Ramamurthy, K. N., Olteanu, A., Piorkowski, D., et al. (2019) · IBM Journal of Research and Development · doi:10.1147/jrd.2019.2942288 · Cited by 434 · cited 2×
“Analogous to a supplier’s declaration of conformity (SDoC) which provides information on how a product conforms to the technical standards or regulations enforced in the country it is deployed, Factsheets have been adopted, to present a comprehensive documentation framework [3, 66].”
BackgroundEstablishing the rules for building trustworthy AI
Floridi, L. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0055-y · Cited by 394 · cited 1×
“A significant amount of research has been devoted to crafting ethical guidelines for practitioners, to guide their development of AI systems [27, 35, 38].”
BackgroundInterpretML: A Unified Framework for Machine Learning Interpretability
Nori, H., Jenkins, S., Koch, P., & Caruana, R. (2019) · arXiv · doi:10.48550/arxiv.1909.09223 · Cited by 322 · cited 1×
“diversity, bias in datasets [5, 9, 72] or explainability [4, 5, 53] have been developed.”
BackgroundFairness is not static: deeper understanding of long term fairness via simulation studies
D’Amour, A., Srinivasan, H., Atwood, J. R., Baljekar, P. N., Sculley, D., & Halpern, Y. (2020) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3351095.3372878 · Cited by 314 · cited 1×
“Moreover, open-source software frameworks have been proposed, to enable the analysis of fairness metrics and manage trade-offs between fairness and optimal model performance [12, 22].”
Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims
Brundage, M., Avin, S., Wang, J., Belfield, H., Krueger, G., Hadfield, G., Khlaaf, H., Yang, J., Toner, H., Fong, R., et al. (2020) · arXiv · doi:10.48550/arxiv.2004.07213 · Cited by 306 · cited 10×
“Auditing is not new and has significantly con-tributedtopromotingaccountabilityandconsistency inotherhighly-regulated industries such as finance and air mobility [13, 74].”
InfluentialWho Audits the Auditors? Recommendations from a field scan of the algorithmic auditing ecosystem
Costanza-Chock, S., Raji, I. D., & Buolamwini, J. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533213 · Cited by 277 · cited 3×
“Audit-ing enables the evaluation of a system against a determined set of criteria and standards, that could enable the exchange of relevant information about its performance, and provide recommendations on how to improve it [19].”
InfluentialOne Explanation Does Not Fit All: A Toolkit and Taxonomy of AI Explainability Techniques
Arya, V., Bellamy, R. K. E., Chen, P.-Y., Dhurandhar, A., Hind, M., Hoffman, S. C., Houde, S., Liao, Q. V., Luss, R., Mojsilović, A., et al. (2019) · arXiv · doi:10.48550/arxiv.1909.03012 · Cited by 268 · cited 1×
“Hence, a number of auditing tools and frameworks tackle specific areas, e.g. diversity, bias in datasets [5, 9, 72] or explainability [4, 5, 53] have been developed.”
Governing AI safety through independent audits
Falco, G., Shneiderman, B., Badger, J., Carrier, R., Dahbura, A., Danks, D., Eling, M., Goodloe, A., Gupta, J., Hart, C., et al. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00370-7 · Cited by 250 · cited 2×
“The practices, such as bug bounties, audit trails, incident analysis and red team exercises, conducted in software engineering, can provide concrete methods to enable AI developers to assess the performance of the system as well as enabling to gain trustworthiness [7, 13, 26, 29, 60, 75].”
BackgroundUncovering and Mitigating Algorithmic Bias through Learned Latent Structure
Amini, A., Soleimany, A. P., Schwarting, W., Bhatia, S. N., & Rus, D. (2019) · Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3306618.3314243 · Cited by 206 · cited 1×
“…Machine Learning (ML) systems can produce predictions that disproportionately affect vulnerable minorities, particularly in sensitive contexts such as criminal justice, hiring, and healthcare; and contribute to the re-enforcement of the structural bias already present in society [2, 15, 57, 63].”
BackgroundSecurity and Privacy for Artificial Intelligence: Opportunities and ChallengesPreprint
Oseni, A., Moustafa, N., Janicke, H., Liu, P., Tari, Z., & Vasilakos, A. V. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.04661 · Cited by 199 · cited 1×
“There is a multitude of literature focusing on fairness [39, 69], transparency and explainability [8, 18], accountability [24, 61, 63], privacy and many other principles around the development of ethical AI [48, 58].”
BackgroundArtificial Intelligence: Risks to Privacy and DemocracyPreprint
Manheim, K. M., & Kaplan, L. (2018) · SSRN Electronic Journal · Cited by 180
Raji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P. (2020) · arXiv · doi:10.48550/arxiv.2001.00973 · Cited by 179 · cited 9×
“This process allows the developers to detect the potential unintended consequences before the deployment [63].”
InfluentialFrom Reality to World. A Critical Perspective on AI Fairness
John-Mathews, J.-M., Cardon, D., & Balagué, C. (2022) · Journal of Business Ethics · doi:10.1007/s10551-022-05055-8 · Cited by 174 · cited 1×
“There is a multitude of literature focusing on fairness [39, 69], transparency and explainability [8, 18], accountability [24, 61, 63], privacy and many other principles around the development of ethical AI [48, 58].”
BackgroundMinkkinen, M., Laine, J., & Mäntymäki, M. (2022) · Digital Society · doi:10.1007/s44206-022-00022-2 · Cited by 161 · cited 1×
“Ad-vancing Ad-vancing the research on the application of such methodologies to AI, in parallel to governance tools, can enable to hold the organisations that develop and deploy AI accountable, by addressing risks through the implementation of control and oversight mechanisms [42, 50, 52].”
Toward situated interventions for algorithmic equity: lessons from the field
Katell, M., Young, M., Dailey, D., Herman, B., Guetler, V., Tam, A., Bintz, C., Raz, D., & Krafft, P. M. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372874 · Cited by 153 · cited 1×
“To address the gaps that are caused by a focus on technical solutions to provide algorithmic equity, tools such as the Algorithmic Equity Toolkit can promote awareness to users of the impact that automated decision-making systems can have on their communities [40].”
Mökander, J., Axente, M., Casolari, F., & Floridi, L. (2022) · Minds and Machines · doi:10.1007/s11023-021-09577-4 · Cited by 153
The Role of Cooperation in Responsible AI DevelopmentPreprint
Askell, A., Brundage, M., & Hadfield, G. K. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.04534 · Cited by 133 · cited 1×
“…AI incidents that can expose the potential relation, between the deployment of certain systems and the contexts, nonetheless, reputations concerns and competitive pressures play a significant role in disincentivising such practices and result in a classic “collective action problem scenario” [6].”
Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned
Ganguli, D., Lovitt, L., Kernion, J., Askell, A., Bai, Y., Kadavath, S., Mann, B., Perez, E., Schiefer, N., Ndousse, K., et al. (2022) · arXiv · doi:10.48550/arxiv.2209.07858 · Cited by 130 · cited 1×
“The practices, such as bug bounties, audit trails, incident analysis and red team exercises, conducted in software engineering, can provide concrete methods to enable AI developers to assess the performance of the system as well as enabling to gain trustworthiness [7, 13, 26, 29, 60, 75].”
BackgroundMökander, J., Axente, M., Casolari, F., & Floridi, L. (2022) · PhilPapers (PhilPapers Foundation) · Cited by 115
Transparency and Explainability of AI Systems: Ethical Guidelines in Practice
Balasubramaniam, N., Kauppinen, M., Hiekkanen, K., & Kujala, S. (2022) · Lecture notes in computer science · doi:10.1007/978-3-030-98464-9_1 · Cited by 103 · cited 1×
“There is a multitude of literature focusing on fairness [39, 69], transparency and explainability [8, 18], accountability [24, 61, 63], privacy and many other principles around the development of ethical AI [48, 58].”
BackgroundKoshiyama, A., Kazim, E., Treleaven, P. C., Rai, P., Szpruch, Ł., Pavey, G., Ahamat, G., Leutner, F., Goebel, R. G., Knight, A., et al. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3778998 · Cited by 98
Filling gaps in trustworthy development of AI
Avin, S., Belfield, H., Brundage, M., Krueger, G., Wang, J., Weller, A., Anderljung, M., Krawczuk, I., Krueger, D. M., Lebensold, J., et al. (2021) · Science · doi:10.1126/science.abi7176 · Cited by 67 · cited 5×
“Bias and safety bounties are an established practice in software engineering to analyse potential risks and vulnerabilities that might have been overlooked, in the development phase, but necessarily need to be addressed before deployment [7, 13, 23].”
BackgroundInfluentialOn Predicting Recidivism: Epistemic Risk, Tradeoffs, and Values in Machine Learning
Biddle, J. B. (2022) · Canadian Journal of Philosophy · doi:10.1017/can.2020.27 · Cited by 66 · cited 1×
“…has been shown to disproportionally predict high-risk scores of recidivism for African-American defendants, resulting in high false positive rates, whilst giving white defendants lower scores for similar cases, in turn resulting in high false negatives rates for the white population [11, 46].”
The Model Card Authoring Toolkit: Toward Community-centered, Deliberation-driven AI Design
Shen, H., Wang, L., Deng, W. H., Brusse, C., Velgersdijk, R., & Zhu, H. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533110 · Cited by 64 · cited 1×
“Moreover, the Model Card Authoring Toolkit enables community members to understand, whether ML models operate in alignment with their collective values [73].”
BackgroundTesting and quality validation for AI software–perspectives, issues, and practices
Tao, C., Gao, J., & Wang, T. (2019) · IEEE Access · doi:10.1109/access.2019.2937107 · Cited by 60 · cited 1×
“Similarly to the field of AI, software engineering requires necessary maintenance and evolution practices that carry enormous costs and implementation challenges [10, 76].”
AI Explainability 360: An Extensible Toolkit for Understanding Data and Machine Learning Models
Arya, V., Bellamy, R., Chen, P., Dhurandhar, A., Hind, M., Hoffman, S. C., Houde, S., Liao, Q. V., Luss, R., Mojsilović, A., et al. (2020) · Journal of Machine Learning Research · Cited by 54
Powering Africa׳s sustainable development: The potential role of nuclear energy
Kessides, I. N. (2014) · Energy Policy · doi:10.1016/j.enpol.2014.04.037 · Cited by 48 · cited 1×
“More-over, safety-critical industries rely on robust practices to assess the performance of systems such as the aerospace and aviation industries, nuclear energy, and healthcare, heavily rely on standardised practices and internal audits to maintain a required level of quality [41, 63, 74].”
Red-Teaming the Stable Diffusion Safety FilterPreprint
Rando, J., Paleka, D., Lindner, D., Heim, L., & Tramèr, F. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2210.04610 · Cited by 34 · cited 2×
“These findings display how the lack of documentation has prevented developers to access sufficient information for ex-ante mitigation measures, to properly understand and detect the safety risks [65].”
Standardizing AI - The Case of the European Commission's Proposal for an Artificial Intelligence ActPreprint
Ebers, M. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3900378 · Cited by 29 · cited 1×
“Delegating regulatory powers to private bodies is problematic due to the lack of democratic over-sight mechanisms by European legislative institutions, the scarce involvement of all relevant stakeholders such as consumers, and the current immunity of standards to judicial control [25].”
A taxonomic system for failure cause analysis of open source AI incidentsPreprint
Pittaras, N., & McGregor, S. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2211.07280 · Cited by 28 · cited 1×
“The practices, such as bug bounties, audit trails, incident analysis and red team exercises, conducted in software engineering, can provide concrete methods to enable AI developers to assess the performance of the system as well as enabling to gain trustworthiness [7, 13, 26, 29, 60, 75].”
BackgroundA Methodology for Creating AI FactSheets
Richards, J., Piorkowski, D., Hind, M., Houde, S., & Mojsilović, A. (2020) · arXiv · doi:10.48550/arxiv.2006.13796 · Cited by 28 · cited 2×
“Analogous to a supplier’s declaration of conformity (SDoC) which provides information on how a product conforms to the technical standards or regulations enforced in the country it is deployed, Factsheets have been adopted, to present a comprehensive documentation framework [3, 66].”
BackgroundIntroduction to the Special Issue on AI Fairness, Trust, and Ethics
Robert, L., Bansal, G., Melville, N. P., & Stafford, T. F. (2020) · AIS Transactions on Human-Computer Interaction · doi:10.17705/1thci.00134 · Cited by 24 · cited 1×
“There is a multitude of literature focusing on fairness [39, 69], transparency and explainability [8, 18], accountability [24, 61, 63], privacy and many other principles around the development of ethical AI [48, 58].”
BackgroundTiwari, S. (2023) · Journal of Cyber Policy · doi:10.1080/23738871.2023.2237981 · Cited by 24
Ethical hacking for boosting IoT vulnerability management
Ding, A. Y., Jesus, G. L. D., & Janssen, M. (2019) · International Conference on Telecommunications and Remote Sensing (ICTRS) · doi:10.1145/3357767.3357774 · Cited by 19 · cited 1×
“Bias and safety bounties are an established practice in software engineering to analyse potential risks and vulnerabilities that might have been overlooked, in the development phase, but necessarily need to be addressed before deployment [7, 13, 23].”
BackgroundAccounting software evaluation: hardware, audit trails, backup, error recovery and security
Rushinek, A., & Rushinek, S. F. (1995) · Managerial Auditing Journal · doi:10.1108/02686909510100887 · Cited by 15 · cited 1×
“Audit trails are another fundamental practice that can clarify accountability, by recording who was responsible for what, and at which stage [70, 74].”
BackgroundCombating disinformation with AI: Epistemic and ethical challenges
Lange, B., & Lechterman, T. M. (2021) · IEEE International Symposium on Technology and Society (ISTAS) · doi:10.1109/istas52410.2021.9629122 · Cited by 12 · cited 3×
“Ethical challenges with the deployment of AI systems are not a new phenomenon and have been widely recognised in the literature [16, 43, 49].”
The Ethical Guidelines for Trustworthy AI – A Procrastination of Effective Law Enforcement
Vasse’i, R. M. (2019) · Computer Law Review International · doi:10.9785/cri-2019-200502 · Cited by 7 · cited 1×
“One of the solutions is to invest in significant in-house training of the technical expertise within the governments to enable proper independence instead of heavily relying on private partners [77].”
Cybersecurity Exercises: Wargaming and Red Teaming
Lorenzo, R., Francesco, B., & Alessio, D. A. (2019) · NATO science for peace and security series. D, Information and communication security · doi:10.3233/nicsp190008 · Cited by 6
Research works printed but not linked: 23
- Sarah Bird, Miro Dudík, Richard Edgar, Brandon Horn, Roman Lutz, Vanessa Milan, Mehrnoosh Sameki, Hanna Wallach, and Kathleen Walker. 2020. Fairlearn: A toolkit for assessing and improving fairness in AI. Microsoft, Tech. Rep. MSR-TR-2020-32 (2020).
- Alan Bryman, Robert G Burgess, 1994. Analyzing qualitative data. Vol. 11. Routledge London.
- Joy Buolamwini and Timnit Gebru. 2018. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on fairness, accountability and transparency. PMLR, 77–91.
- Danton S Char, Nigam H Shah, and David Magnus. 2018. Implementing machine learning in health care—addressing ethical challenges. The New England journal of medicine 378, 11 (2018), 981.
- João Figueiredo Nobre Brito Cortese, Fabio Gagliardi Cozman, Marcos Paulo Lucca-Silveira, and Adriano Figueiredo Bechara. 2022. Should explainability be a fifth ethical principle in AI ethics?AI and Ethics (2022), 1–12.
- Financial Reporting Council. 2021. Auditors I Audit and Assurance I Standards and Guidance for Auditors I Financial Reporting Council (2020).
- Kate Crawford. 2021. The atlas of AI: Power, politics, and the planetary costs of artificial intelligence. Yale University Press.
- Finale Doshi-Velez, Mason Kortz, Ryan Budish, Chris Bavitz, Sam Gershman, David O’Brien, Kate Scott, Stuart Schieber, James Waldo, David Weinberger, 2017. Accountability of AI under the law: The role of explanation. arXiv preprint arXiv:1711.01134 (2017). · arXiv
- Nicola K Gale, Gemma Heath, Elaine Cameron, Sabina Rashid, and Sabi Redwood. 2013. Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC medical research methodology 13, 1 (2013), 1–8.
- 10.5555/3297863.3297908 · doi:10.5555/3297863.3297908
- Ellen P Goodman and Julia Trehu. 2022. AI Audit Washing and Accountability. Available at SSRN 4227350 (2022).
- Jaber F Gubrium and James A Holstein. 2002. From the individual interview to the interview society. Handbook of interview research: Context and method (2002), 3–32.
- Shin-Shin Hua and Haydn Belfield. 2020. AI & Antitrust: Reconciling Tensions between Competition Law and Cooperative AI Development. Yale JL & Tech. 23 (2020), 415.
- Johann Laux, Sandra Wachter, and Brent Mittelstadt. 2023. Three Pathways for Standardisation and Ethical Disclosure by Default under the European Union Artificial Intelligence Act. Available at SSRN (2023).
- Heidi Ledford. 2019. Millions of black people affected by racial bias in health-care algorithms. Nature 574, 7780 (2019), 608–610.
- Carina Lewandowski. 2021. Machine (Un) learning: An Investigation of Racial Bias in Predictive Recidivism Algorithms as a Product of Real-World, Structural Discrimination. (2021).
- Ziad Obermeyer, Brian Powers, Christine Vogeli, and Sendhil Mullainathan. 2019. Dissecting racial bias in an algorithm used to manage the health of populations. Science 366, 6464 (2019), 447–453.
- House of Commons of Canada. 2022. Bill C-27: An Act to enact the Consumer Privacy Protection Act, the Personal Information and Data Protection Tribunal Act and the Artificial Intelligence and Data Act and to make consequential and related amendments to other Acts. 1st Session, 44th Parliament, 70-71 Elizabeth II, 2021-2022 (2022).
- Cathy O’neil. 2016. Weapons of math destruction: How big data increases inequality and threatens democracy. Broadway Books.
- Junaid Qadir, Mohammad Qamar Islam, and Ala Al-Fuqaha. 2022. Toward accountable human-centered AI: rationale and promising directions. Journal of Information, Communication and Ethics in Society (2022).
- Jane Ritchie, Jane Lewis, Carol McNaughton Nicholls, Rachel Ormston, 2013. Qualitative research practice: A guide for social science students and researchers. sage.
- Carol AB Warren. 2002. Qualitative interviewing. Handbook of interview research: Context and method 839101 (2002), 103–116.
- Jenna Wiens, W Nicholson Price, and Michael W Sjoding. 2020. Diagnosing bias in data-driven algorithms for healthcare. Nature medicine 26, 1 (2020), 25–26.
Cited by
Black-Box Access is Insufficient for Rigorous AI Audits
Casper, S., Ezell, C., Siegmann, C., Kolt, N., Curtis, T. L., Bucknall, B., Haupt, A., Wei, K., Scheurer, J., Hobbhahn, M., et al. (2024) · The 2024 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3630106.3659037 · Cited by 86
Neuromarketing algorithms’ consumer privacy and ethical considerations: challenges and opportunities
Goncalves, M. V., Hu, Y., Aliagas, I., & Suárez, L. M. C. (2024) · Cogent Business & Management · doi:10.1080/23311975.2024.2333063 · Cited by 103
“One practical approach to promoting ethical and standardized ai/Ml algorithms is through certification, which can help reduce information asymmetries (cihon et al., 2021; lucaj et al., 2023).”
Benk, M., Kerstan, S., von Wangenheim, F., & Ferrario, A. (2024) · AI & Society · doi:10.1007/s00146-024-02059-y · Cited by 122
“…humans trust AI systems is fueled by increasingly capable AI-based technologies that are transforming the way humans work and collaborate in a multitude of society-impacting applications (Jacobs et al. 2021; Chiou and Lee 2021; Lucaj et al. 2023; Shin 2021; Wamba et al. 2021; Feher et al. 2024).”
Rethinking open source generative AI: open washing and the EU AI Act
Liesenfeld, A., & Dingemanse, M. (2024) · The 2024 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3630106.3659005 · Cited by 123
A Framework for Assurance Audits of Algorithmic Systems
Lam, K., Lange, B., Blili-Hamelin, B., Davidovic, J., Brown, S., & Hasan, A. (2024) · The 2024 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3630106.3658957 · Cited by 75
Metadata
- Authors
- Lucaj, L., van der Smagt, P., & Benbouzid, D.
- Affiliations
- Volkswagen Group (Germany) (DE)
- Machine Learning Research Lab, Volkswagen Group, Germany
- Venue
- 2023 ACM Conference on Fairness Accountability and Transparency
- Publisher
- ACM
- Published
- 2023-06-12
- Pages
- 1267-1279
- OpenAlex
- W4380320348
- Cited by
- 40
- References
- 45
- Citations by year
- 2026: 8 · 2025: 14 · 2024: 18
- FWCI
- 10.02
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Law › Digitalization, Law, and Regulation
- Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Safety Systems Engineering in Autonomy
- Keywords
- AI regulation, AI governance, legal compliance, trustworthy AI, ethical AI, AI risk mitigation, oversight mechanisms, regulatory framework
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.