CoChat

Auditing of AI: Legal, Ethical and Technical Approaches

Open access
Mökander, Jakob · 2023 · Digital Society

The DOI matches its Crossref record; 176 of the 176 references it cites were found in a registry or an index; 10 of the 102 sources its claims cite were checked for support.

Abstract

AI auditing is a rapidly growing field of research and practice. This review article, which doubles as an editorial to Digital Society’s topical collection on ‘Auditing of AI’, provides an overview of previous work in the field. Three key points emerge from the review. First, contemporary attempts to audit AI systems have much to learn from how audits have historically been structured and conducted in areas like financial accounting, safety engineering and the social sciences. Second, both policymakers and technology providers have an interest in promoting auditing as an AI governance mechanism. Academic researchers can thus fill an important role by studying the feasibility and effectiveness of different AI auditing procedures. Third, AI auditing is an inherently multidisciplinary undertaking, to which substantial contributions have been made by computer scientists and engineers as well as social scientists, philosophers, legal scholars and industry practitioners. Reflecting this diversity of perspectives, different approaches to AI auditing have different affordances and constraints. Specifically, a distinction can be made between technology-oriented audits, which focus on the properties and capabilities of AI systems, and process-oriented audits, which focus on technology providers’ governance structures and quality management systems. The next step in the evolution of auditing as an AI governance mechanism, this article concludes, should be the interlinking of these available—and complementary—approaches into structured and holistic procedures to audit not only how AI systems are designed and used but also how they impact users, societies and the natural environment in applied settings over time.

Overview

CoChat Summary

TL;DR

This review/editorial surveys AI auditing as a multidisciplinary governance mechanism, arguing that it should learn from financial, safety, and social-science audits and combine technology-oriented and process-oriented approaches. It concludes that the next step is to interlink complementary auditing tools into structured, holistic procedures, while remaining realistic about auditing’s conceptual, technical, economic, and institutional limits.

Key findings

  • Contemporary AI auditing can learn from established audit domains: financial accounting, safety engineering, and social-science audit studies. The paper notes financial auditing is a large industry with an estimated market size of over $110 bn.
  • Both policymakers and technology providers have incentives to promote AI auditing: top-down regulatory pressure and bottom-up corporate risk-management pressure. Academic researchers are positioned to study the feasibility and effectiveness of different procedures.
  • AI auditing is multidisciplinary and can be divided into narrow, technology-oriented audits focused on AI system outputs/properties and broad, process-oriented audits focused on providers’ governance structures and quality management systems. These approaches are complementary and should be interlinked.
  • The paper cites examples such as Bertrand and Mullainathan’s audit study, where résumés with white-sounding names were 50% more likely to receive callbacks than those with black-sounding names, and Dawson et al.’s estimate that fixing a bug can cost up to 15 times more depending on when it is found.

Methods

  • Design: a review article that doubles as an editorial for *Digital Society*’s topical collection on “Auditing of AI.” It provides an overview of previous work rather than presenting new empirical research.
  • Data/sample: prior academic literature, policy documents, standards, guidance, and real-world examples from financial auditing, safety auditing, social-science audit studies, and AI auditing.
  • Analysis: conceptual and narrative synthesis. It distinguishes narrow vs. broad conceptions of AI auditing and technical, legal, and ethical approaches, then introduces six articles in the topical collection. The paper states “Materials and Method Not applicable.”

Limitations

  • The paper states that AI auditing, even when following best practices, is subject to conceptual, technical, economic, and institutional limitations; some can be addressed by policy or technology, while others are intrinsic.
  • It states that claims about the limitations and affordances of AI auditing as a governance mechanism have yet to be substantiated by empirical research, and that well-established standards for AI auditing have yet to emerge.
  • Evident from the text: as a review/editorial, it does not use a systematic review protocol, formal search strategy, or primary data collection; its conclusions are synthetic and illustrative rather than empirically tested.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1007/s44206-023-00074-y

Cites

176 of 176 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

8 Valid · 15 Not valid · 57 Not checked · 62 Own results and interpretations

45 more claims · 1 with their cited sources checked

Create a free account to see all 142 claims and to verify the 92 references still open.

See all claims — free account
  • It may state two separate claims as one. 30
  • Probably not a substantial claim. 78

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

3 of 3

Citations

Cites

Show all 176 references

Research works printed but not linked: 7

  1. Allford, L., & Carson, P. (2015). Safety practice safety, health, and environment audits with selected case histories. In Loss Prevention Bulletin (241). Retrieved July 20, 2023, from www.researchgate.net/publication/307978324
  2. Brown, R. G. (1962). Changing audit objectives and techniques. The Accounting Review, 37(4), 696–703. Retrieved July 20, 2023, from https://www.proquest.com/docview/1301318804
  3. Cosserat, G. W. (2004). Modern auditing (2nd ed.). John Wiley & Sons, Ltd.
  4. Dawson, M., Burrell, D. N., Rahim, E., & Brewster, S. (2010). Integrating software assurance into the software development life cycle (SDLC) meeting department of defense (DOD) demands. Journal of Information Systems Technology and Planning, 3(6), 49–53. Retrieved July 20, 2023, from www.academia.edu/22484322
  5. Gupta, K. (2004). Comtemporary auditing. McGraw Hill.
  6. Klinect, J., Murray, P., Merritt, A., & Helmreich, R. (2003). Line operation safety audits: Definition and operating characteristics (pp. 663–668). Proceedings of the 12th International Symposium on Aviation Psychology.
  7. Mittelstadt, B. (2016). Auditing for transparency in content personalization systems. International Journal of Communication, 10, 4991–5002. Retrieved July 20, 2023, from www.researchgate.net/publication/309136069

Cited by

See all 152

Metadata

Authors
Mökander, J.
Affiliations
Princeton University (US)
Corresponding author
Mökander, Jakob
Venue
Digital Society
Published
2023-11-08
Volume
2
Issue
3
ISSN
  • 2731-4650 (ISSN-L)
  • 2731-4669
OpenAlex
W4388499666
Cited by
152
References
167
Citations by year
2026: 69 · 2025: 53 · 2024: 29 · 2023: 1
FWCI
38.06
Type
journal article
Language
en
Open access
Licence
Locations
  • Digital Society (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Funders
Center for Information Technology Policy (CITP), Princeton University
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Social Sciences › Law › Digitalization, Law, and Regulation
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
Keywords
artificial intelligence in auditing, AI governance, quality management systems, history of auditing, societal impact of AI

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-07, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-07 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Auditing of AI: Legal, Ethical and Technical Approaches. https://app.cochat.ai/papers/10.1007/s44206-023-00074-y
The paper
Mökander, J. (2023). Auditing of AI: Legal, Ethical and Technical Approaches. Digital Society. https://doi.org/10.1007/s44206-023-00074-y

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.