Continuous Auditing of Artificial Intelligence: a Conceptualization and Assessment of Tools and Frameworks
Open accessThe DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index.
Abstract
Artificial intelligence (AI), which refers to both a research field and a set of technologies, is rapidly growing and has already spread to application areas ranging from policing to healthcare and transport. The increasing AI capabilities bring novel risks and potential harms to individuals and societies, which auditing of AI seeks to address. However, traditional periodic or cyclical auditing is challenged by the learning and adaptive nature of AI systems. Meanwhile, continuous auditing (CA) has been discussed since the 1980s but has not been explicitly connected to auditing of AI. In this paper, we connect the research on auditing of AI and CA to introduce CA of AI (CAAI). We define CAAI as a (nearly) real-time electronic support system for auditors that continuously and automatically audits an AI system to assess its consistency with relevant norms and standards. We adopt a bottom-up approach and investigate the CAAI tools and methods found in the academic and grey literature. The suitability of tools and methods for CA is assessed based on criteria derived from CA definitions. Our study findings indicate that few existing frameworks are directly suitable for CAAI and that many have limited scope within a particular sector or problem area. Hence, further work on CAAI frameworks is needed, and researchers can draw lessons from existing CA frameworks; however, this requires consideration of the scope of CAAI, the human–machine division of labour, and the emerging institutional landscape in AI governance. Our work also lays the foundation for continued research and practical applications within the field of CAAI.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Floridi, L., Cowls, J., Beltrametti, M., Chatila, R., Chazerand, P., Dignum, V., Luetge, C., Madelin, R., Pagallo, U., Rossi, F., et al. (2018) · Minds and Machines · doi:10.1007/s11023-018-9482-5 · Cited by 4,774 · cited 3×
“The rationale for CA is linked to the aspired human oversight of AI systems (Floridi et al., 2018; Shneiderman, 2020).”
BackgroundThe Black Box Society: The Secret Algorithms That Control Money and Information
Pasquale, F. (2015) · doi:10.4159/harvard.9780674736061 · Cited by 3,796
Kaplan, A., & Haenlein, M. (2019) · Business Horizons · doi:10.1016/j.bushor.2018.08.004 · Cited by 3,140 · cited 2×
“In general, AI is integrated into information systems and refers to the capabilities of data interpretation, learning, and adaptation that aim to attain human-level capabilities in particular tasks (Kaplan & Haenlein, 2019; Russell & Norvig, 2021).”
BackgroundRaji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372873 · Cited by 2,245 · cited 3×
“Because CAAI requires continuous access to AI systems, it appears most relevant to internal audit functions within organizations (cf. Raji et al., 2020b; Tronto et al., 2021) as opposed to external auditing conducted by independent auditors.”
BackgroundBellamy, R. K. E., Dey, K., Hind, M., Hoffman, S. C., Houde, S., Kannan, K., Lohia, P., Martino, J., Mehta, S., Mojsilovic, A., et al. (2018) · arXiv · doi:10.48550/arxiv.1810.01943 · Cited by 1,581
AI Fairness 360: An extensible toolkit for detecting and mitigating algorithmic bias
Bellamy, R. K. E., Dey, K., Hind, M., Hoffman, S. C., Houde, S., Kannan, K., Lohia, P., Martino, J., Mehta, S., Mojsilović, A., et al. (2019) · IBM Journal of Research and Development · doi:10.1147/jrd.2019.2942287 · Cited by 884 · cited 1×
“AI Fairness 360 by Bellamy et al. (2019) and FlipTest by Black et al. (2020) focused on fairness issues in AI systems.”
BackgroundThe Emergence of Artificial Intelligence: How Automation is Changing Auditing
Kokina, J., & Davenport, T. H. (2017) · Journal of Emerging Technologies in Accounting · doi:10.2308/jeta-51730 · Cited by 790 · cited 1×
“There is a separate and growing stream of literature on the use of AI and other novel technologies to aid auditing (e.g. Kokina & Davenport, 2017).”
BackgroundShneiderman, B. (2020) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3419764 · Cited by 768 · cited 2×
“The rationale for CA is linked to the aspired human oversight of AI systems (Floridi et al., 2018; Shneiderman, 2020).”
BackgroundEthical Implications and Accountability of Algorithms
Martin, K. (2019) · Journal of Business Ethics · doi:10.1007/s10551-018-3921-3 · Cited by 757
Aequitas: A Bias and Fairness Audit Toolkit
Saleiro, P., Kuester, B., Hinkson, L., London, J., Stevens, A., Anisfeld, A., Rodolfa, K. T., & Ghani, R. (2018) · arXiv · doi:10.48550/arxiv.1811.05577 · Cited by 496
Machine Learning, Social Learning and the Governance of Self-Driving Cars
Stilgoe, J. (2017) · doi:10.2139/ssrn.2937316 · Cited by 376
Machine learning, social learning and the governance of self-driving cars
Stilgoe, J. (2018) · Social Studies of Science · doi:10.1177/0306312717741687 · Cited by 375 · cited 1×
“Artificial intelligence (AI), which refers to both a research field and a set of technologies, is rapidly growing and has already spread to application areas ranging from policing to healthcare and transport (e.g. Rezende, 2020; Stilgoe, 2018; Trocin et al., 2021).”
BackgroundSaving Face: Investigating the Ethical Concerns of Facial Recognition Auditing
Raji, I. D., Gebru, T., Mitchell, M., Buolamwini, J., Lee, J., & Denton, R. (2020) · Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3375627.3375820 · Cited by 299 · cited 3×
“Because CAAI requires continuous access to AI systems, it appears most relevant to internal audit functions within organizations (cf. Raji et al., 2020b; Tronto et al., 2021) as opposed to external auditing conducted by independent auditors.”
BackgroundGoverning AI safety through independent audits
Falco, G., Shneiderman, B., Badger, J., Carrier, R., Dahbura, A., Danks, D., Eling, M., Goodloe, A., Gupta, J., Hart, C., et al. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00370-7 · Cited by 250
Responsible AI for Digital Health: a Synthesis and a Research Agenda
Trocin, C., Mikalef, P., Papamitsiou, Z., & Conboy, K. (2023) · Information Systems Frontiers · doi:10.1007/s10796-021-10146-4 · Cited by 205 · cited 1×
“Artificial intelligence (AI), which refers to both a research field and a set of technologies, is rapidly growing and has already spread to application areas ranging from policing to healthcare and transport (e.g. Rezende, 2020; Stilgoe, 2018; Trocin et al., 2021).”
BackgroundThe algorithm audit: Scoring the algorithms that score us
Brown, S., Davidovic, J., & Hasan, A. (2021) · Big Data & Society · doi:10.1177/2053951720983865 · Cited by 204 · cited 5×
“On the ethics-based side, Brown et al. (2021) presented an auditing framework to guide the ethical assessment of an algorithm.”
BackgroundInfluentialAlgorithmic Impact Assessments and Accountability: The Co-construction of Impacts
Metcalf, J., Moss, E., Watkins, E. A., Singh, R., & Elish, M. C. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445935 · Cited by 195
Multiaccuracy: Black-Box Post-Processing for Fairness in Classification
Kim, M. P., Ghorbani, A., & Zou, J. (2019) · Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3306618.3314287 · Cited by 193
Raji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P. (2020) · arXiv · doi:10.48550/arxiv.2001.00973 · Cited by 179
Floridi, L., Holweg, M., Taddeo, M., Amaya, J., Mökander, J., & Wen, Y. (2022) · doi:10.2139/ssrn.4064091 · Cited by 157
Mökander, J., Axente, M., Casolari, F., & Floridi, L. (2022) · Minds and Machines · doi:10.1007/s11023-021-09577-4 · Cited by 153 · cited 12×
“EBA is defined as “a structured process whereby an entity’s present or past behaviour is assessed for consistency with relevant principles or norms” (Mökander et al., 2021, p. 1).”
BackgroundMethodResultInfluentialToward situated interventions for algorithmic equity: lessons from the field
Katell, M., Young, M., Dailey, D., Herman, B., Guetler, V., Tam, A., Bintz, C., Raz, D., & Krafft, P. M. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372874 · Cited by 153 · cited 1×
“Greater fairness, accountability, and transparency in algorithmic systems were also the objectives of the Algorithmic Equity Toolkit by Katell et al. (2020) and the Counterfactual Explanations for Robustness, Transpar-ency, Interpretability, and Fairness of Artificial Intelligence models (CERTIFAI;…”
Reviewable Automated Decision-Making: A Framework for Accountable Algorithmic Systems
Cobbe, J., Lee, M. S. A., & Singh, J. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445921 · Cited by 150
Z-Inspection ® : A Process to Assess Trustworthy AI
Zicari, R. V., Brodersen, J., Brusseau, J., Dudder, B., Eichhorn, T., Ivanov, T., Kararigas, G., Kringen, P., McCullough, M., Moslein, F., et al. (2021) · IEEE Transactions on Technology and Society · doi:10.1109/tts.2021.3066209 · Cited by 134
Koshiyama, A., Kazim, E., Treleaven, P., Rai, P., Szpruch, L., Pavey, G., Ahamat, G., Leutner, F., Goebel, R., Knight, A., et al. (2024) · Royal Society Open Science · doi:10.1098/rsos.230859 · Cited by 110
Shah, H. (2018) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2017.0362 · Cited by 101 · cited 1×
“…2018; Shiue et al., 2021; Vasarhelyi & Halper, 1991), we were unable to find an established literature stream specifically on CA of AI (CAAI) beyond general calls for monitoring the impacts of algorithmic systems (e.g. Doneda & Almeida, 2016; Metcalf et al., 2021; Shah, 2018; Yeung et al., 2020).”
BackgroundQuantifying the Impact of User Attentionon Fair Group Representation in Ranked Lists
Sapiezynski, P., Zeng, W., E Robertson, R., Mislove, A., & Wilson, C. (2019) · Companion Proceedings of The 2019 World Wide Web Conference · doi:10.1145/3308560.3317595 · Cited by 92
FairLens: Auditing black-box clinical decision support systems
Panigutti, C., Perotti, A., Panisson, A., Bajardi, P., & Pedreschi, D. (2021) · Information Processing & Management · doi:10.1016/j.ipm.2021.102657 · Cited by 86
Saving Face: Investigating the Ethical Concerns of Facial Recognition Auditing
Raji, I. D., Gebru, T., Mitchell, M., Buolamwini, J., Lee, J., & Denton, E. (2020) · arXiv · doi:10.48550/arxiv.2001.00964 · Cited by 69
Machine Learning for Health: Algorithm Auditing & Quality Control
Oala, L., Murchison, A. G., Balachandran, P., Choudhary, S., Fehr, J., Leite, A. W., Goldschmidt, P. G., Johner, C., Schörverth, E. D. M., Nakasi, R., et al. (2021) · Journal of Medical Systems · doi:10.1007/s10916-021-01783-y · Cited by 48
Putting AI Ethics into Practice: The Hourglass Model of Organizational AI Governance
Mäntymäki, M., Minkkinen, M., Birkstedt, T., & Viljanen, M. (2022) · arXiv · doi:10.48550/arxiv.2206.00335 · Cited by 38 · cited 1×
“These challenges and risks related to AI systems underscore the importance of AI governance at the organizational, interorganizational, and societal levels (Laato et al., 2022; Mäntymäki et al., 2022a, b; Minkkinen et al., 2022a, b; Schneider et al., 2022; Seppälä et al., 2021).”
BackgroundSelf-citationSharma, S., Henderson, J., & Ghosh, J. (2019) · arXiv · doi:10.48550/arxiv.1905.07857 · Cited by 28
FAIRVIS: Visual Analytics for Discovering Intersectional Bias in Machine Learning
Cabrera, Á. A., Epperson, W., Hohman, F., Kahng, M., Morgenstern, J., & Chau, D. H. (2019) · 2019 IEEE Conference on Visual Analytics Science and Technology (VAST) · doi:10.1109/vast47406.2019.8986948 · Cited by 14
Tamper-Proof Privacy Auditing for Artificial Intelligence Systems
Sutton, A., & Samavi, R. (2018) · Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/756 · Cited by 3
Cited by
AI governance: themes, knowledge gaps and future agendas
Birkstedt, T., Minkkinen, M., Tandon, A., & Mäntymäki, M. (2023) · Internet Research · doi:10.1108/intr-01-2022-0042 · Cited by 309
Self-citationAuditing Large Language Models: A Three-Layered Approach
Mökander, J., Schuett, J., Kirk, H. R., & Floridi, L. (2023) · doi:10.2139/ssrn.4361607 · Cited by 292
Auditing large language models: a three-layered approach
Mökander, J., Schuett, J., Kirk, H. R., & Floridi, L. (2024) · AI and Ethics · doi:10.1007/s43681-023-00289-2 · Cited by 289
Bias and ethics of AI systems applied in auditing - A systematic review
Murikah, W., Nthenge, J. K., & Musyoka, F. M. (2024) · Scientific African · doi:10.1016/j.sciaf.2024.e02281 · Cited by 148
“Regardless of the category, maintaining human discretion and judgment is crucial for prioritizing the public interest [86,97,110,114,120].”
Auditing of AI: Legal, Ethical and Technical Approaches
Mökander, J. (2023) · Digital Society · doi:10.1007/s44206-023-00074-y · Cited by 152
“…challenges associated with the use of AI systems in the railway sector from a legal point of view; Light and Panai (2022) conduct an ethical analysis of the principles against which AI systems are being audited; and Minkkinen et al. (2022) focus on the technical aspects of how audits are conducted.”
Background
Metadata
- Authors
- Minkkinen, M., Laine, J., & Mäntymäki, M.
- Venue
- Digital Society
- Published
- 2022-10-04
- Volume
- 1
- Issue
- 3
- ISSN
- 2731-4650 (ISSN-L)
- 2731-4669
- OpenAlex
- W4301186400
- Cited by
- 88
- References
- 34
- FWCI
- 18.19
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.