Closing the AI accountability gap: defining an end-to-end framework for internal algorithmic auditing
Open accessThe DOI matches its Crossref record; 96 of the 96 references it cites were found in a registry or an index; 11 of the 44 sources its claims cite were checked for support.
Abstract
Rising concern for the societal implications of artificial intelligence systems has inspired a wave of academic and journalistic literature in which deployed systems are audited for harm by investigators from outside the organizations deploying the algorithms. However, it remains challenging for practitioners to identify the harmful repercussions of their own systems prior to deployment, and, once deployed, emergent issues can become difficult or impossible to trace back to their source.
Overview
CoChat Summary
TL;DR
The paper proposes SMACTR, an end-to-end internal algorithmic auditing framework intended to help organizations assess AI systems against declared ethical principles throughout development. It argues that internal audits can complement external accountability by providing pre-deployment access, structured documentation, and risk analysis drawn from aerospace, medical device, and finance practices.
Key findings
- Introduces SMACTR—Scoping, Mapping, Artifact Collection, Testing, and Reflection—as five audit stages, each with documentation requirements that together form an overall audit report and assess decisions against organizational values or AI principles.
- Argues external audits are limited by lack of access to internal processes, intermediate models, and training data, while internal audits can enable proactive pre-deployment intervention and generate artifacts for external review.
- Draws lessons from other industries, citing aerospace’s one commercial airline accident per two million flights, Boeing 787 software estimated at 13 million lines of code, and a 1-in-1,000,000,000 per use hour maximum failure probability for critical aerospace systems; also cites medical-device design controls and finance internal audit infrastructure.
- Uses hypothetical cases—a child abuse screening tool and a smile-detection photo booth—to illustrate the framework; for the smile detector’s CelebA datasheet, it reports 58.1% female and 42% male labels, 77.8% aged 0–45 and 22.1% over 46, and 14.2% lighter-skinned versus 85.8% darker-skinned.
Methods
- Conceptual framework paper: develops SMACTR from auditing practices and artifacts in aerospace, medical devices, and finance, plus responsible-innovation dimensions of anticipation, reflexivity, inclusion, and responsiveness.
- Uses a hypothetical Company X Inc. consultancy with two invented client projects—a child abuse screening tool and Happy-Go-Lucky Inc.’s smile-detection photo booth—to demonstrate audit stages and artifacts; a worked example for the smile case is provided as supplementary material.
- Analysis is narrative and design-oriented, referencing existing standards and tools such as FMEA, model cards, datasheets, design controls, and design history files; no empirical sample or quantitative evaluation is reported.
Limitations
- The paper states that internal auditors necessarily share an organizational interest with the audit target, making an independent and objective viewpoint challenging; audits are not an objective “view from nowhere” and risk becoming reputation management.
- The paper states that internal audits are only one aspect of a broader system of required quality checks and balances, and that determining what systems to audit and why is not covered and is a separate research topic.
- Evident from the text: the framework is not empirically validated, the illustrative cases are hypothetical, and no real deployment outcomes or measured audit effectiveness are reported.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures


Verification
Claims the paper makes
9 Valid · 6 Not valid · 27 Not checked · 30 Own results and interpretations
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“While not without faults, safety-critical and regulated industries such as aerospace and medicine have long traditions of auditable processes and design controls that have dramatically improved safety [77,81].”
§ Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing > 3 LESSONS FROM AUDITING PRACTICES IN OTHER INDUSTRIES
What the sources say
- Not checked — no full text of the sourcecites [77] source
- Could not verifycites [81] sourceExperimental — accuracy not yet measured
The Introduction and §3 discuss biomedical ethics and standards or regulation, but do not say that aerospace or medicine has long used auditable processes and design controls, or that these dramatically improved safety. Searches for aerospace, auditable processes, design controls, and safety-critical industries found no passage addressing that claim. Page numbers are unavailable in our copy. (LLM, conf=0.76)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“This is a repeatedly observed phenomenon in tax compliance auditing, where several international surveys of tax compliance demonstrate that a fixed and vetted tax audit methodology is one of the most effective strategies to convince companies to respect audit results and pay their full taxes [22,53].”
§ Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing > 2.3 Audit Integrity and Procedural Justice
What the sources say
- Partly supportedcites [22] sourceExperimental — accuracy not yet measured
The Regression Test section reports that Malaysian respondents would be more compliant if tax authorities treated them fairly in tax procedures. The Research Methodology section describes a questionnaire of academics, not several international surveys of companies. The paper does not establish that a fixed, vetted audit methodology is especially effective at securing acceptance of audit results or full tax payment. (LLM, conf=0.86)
- Partly supportedcites [53]Experimental — accuracy not yet measured
The “Justice in the taxation context” section reports that taxpayers are generally more compliant when treated fairly and respectfully, and the General discussion links perceived unfair treatment to resistance to paying back tax (page numbers unavailable). It does not establish that a fixed, vetted audit methodology is among the most effective strategies, or report international surveys demonstrating that effect for companies. (LLM, conf=0.83)
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Traditional metrics for artificial intelligence like loss may conceal fairness concerns, social impact risks or abstraction errors [69].”
§ Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing > 4.3.2 Artifact: Ethnographic Field Study.
What the sources say
- Supportedcites [69] source
The Introduction (page not provided) says that narrow assessments of machine-learning performance abstract away social context and can miss information needed for fair outcomes—an abstraction error. The Formalism Trap section adds that mathematical measures cannot capture the full range of fairness concerns, while the Takeaways discuss unintended social consequences. The paper names AUC, precision and recall rather than loss, but the claim uses loss as an illustrative metric. (LLM, conf=0.88)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Internal auditors' direct access to systems can thus help extend traditional external auditing paradigms by incorporating additional information typically unavailable for external evaluations to reveal previously unidentifiable risks.”
§ Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing > 2.4 The Internal Audit
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“The essential inclusion of independent domain experts and marginalized groups in the ethical review process “has the potential to lead to more rigorous critical reflection because their experiences will often be precisely those that are most needed in identifying problematic background assumptions and revealing limitations with research questions, models, or methodologies” [33].”
§ Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing > 4.2.1 Artifact: Ethical Review of System Use Case.
What the sources say
- Not checked — no full text of the sourcecites [33] source
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
19 more claims
Create a free account to see all 72 claims and every source verdict.
- It may state two separate claims as one. 25
- Probably not a substantial claim. 37
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This disparity poses risks, as practitioners may unintentionally embed their own biases into AI system design and implementation (Martin 2019;Raji et al. 2020).”
From Accountability in Government Use of AI : Citizen Concerns and Preferences (2025) · cites this paper as Raji et al. 2020
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Work on AI safety and reproducibility has long pointed out the crucial importance of training data for understanding model performance, ensuring reproducibility, and assessing legal exposure [2, 25, 29, 46, 58].”
From Rethinking open source generative AI: open washing and the EU AI Act (2024) · cites this paper as [46]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“While there is a growing interest in auditing algorithmic systems [125,126,128,138,154], this feld generally lacks defned standards and records could be incomplete or not relating to information needed for accountability purposes [15,127].”
From Out of Context: Investigating the Bias and Fairness Concerns of “Artificial Intelligence as a Service” (2023) · cites this paper as [128]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Raji et al. (2020) proposed an internal algorithmic auditing framework spanning data collection, model development, testing, deployment, and monitoring, with stage-specific artifacts that enable traceability and accountability.”
From The Impact of Structured Validation and Audit Frameworks on the Fairness and Efficiency of AI-Driven Hiring Systems (2025) · cites this paper as Raji et al., 2020
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Practical approaches to implement such ideas typically revolve around the systematic collection and curation of data on various aspects of systems [2, 28, 46].”
From Rethinking open source generative AI: open washing and the EU AI Act (2024) · cites this paper as [46]
Experimental — accuracy not yet measured
13 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Deep Learning Face Attributes in the Wild
Liu, Z., Luo, P., Wang, X., & Tang, X. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.425 · Cited by 7,747 · cited 1×
“A hypothetical datasheet for this system is filled out using studies on the CelebA dataset, with which the smile detector is built [44, 47].”
MethodThe global landscape of AI ethics guidelines
Jobin, A., Ienca, M., & Vayena, E. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0088-2 · Cited by 6,220
Product development: Past research, present findings and future directions
Brown, S. L., & Eisenhardt, K. M. (1995) · The Academy of Management Review · doi:10.2307/258850 · Cited by 3,777 · cited 1×
“There are also intersections between this framework and just efective product development theory [5], as many of the components of audit design refocus the product development process to prioritize the user and their ultimate well-being, resulting in a more efective product performance outcome.”
Background(2018) · Cited by 3,510
Christiane, Druml, Wolzt, & Pleiner (2001) · Acta Scientiarum Naturalium Universitatis Sunyatseni · Cited by 3,419
How the machine “thinks”: Understanding opacity in machine learning algorithms
Burrell, J. (2016) · Big Data & Society · doi:10.1177/2053951715622512 · Cited by 3,196
Developing a framework for responsible innovation
Stilgoe, J., Owen, R., & Macnaghten, P. (2013) · Research Policy · doi:10.1016/j.respol.2013.05.008 · Cited by 3,184 · cited 1×
“To design our audit procedure, we suggest complementing formal risk assessment methodologies with ideas from responsible innovation, which stresses four key dimensions: anticipation, reflexivity, inclusion and responsiveness [73], as well as system-theoretic concepts that help grapple with increasing complexity and co…”
MethodAlgorithms of Oppression: How Search Engines Reinforce Racism
Noble, S. U. (2020) · doi:10.2307/j.ctt1pwt9w5 · Cited by 2,709
Model Cards for Model Reporting
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287596 · Cited by 2,681
Self-citation(2014) · Journal of the Korean Medical Association · doi:10.5124/jkma.2014.57.11.899 · Cited by 2,566
Research works printed but not linked: 7
- Virginia Eubanks . 2018. A child abuse prediction model fails poor families. Wired Magazine ( 2018 ). Virginia Eubanks. 2018. A child abuse prediction model fails poor families. Wired Magazine (2018).
- Jeremy Goldhaber-Fiebert and Lea Prince . 2019. Impact Evaluation of a Predictive Risk Modeling Tool for Allegheny CountyâĂŹs Child Welfare Office . Pittsburgh : Allegheny County.[Google Scholar ] ( 2019 ). Jeremy Goldhaber-Fiebert and Lea Prince. 2019. Impact Evaluation of a Predictive Risk Modeling Tool for Allegheny CountyâĂŹs Child Welfare Office. Pittsburgh: Allegheny County.[Google Scholar] (2019).
- Nitin Kohli , Renata Barreto , and Joshua A Kroll . 2018 . Translation Tutorial: A Shared Lexicon for Research and Practice in Human-Centered Software Systems. In 1st Conference on Fairness, Accountability, and Transparancy . New York, NY, USA. 7. Nitin Kohli, Renata Barreto, and Joshua A Kroll. 2018. Translation Tutorial: A Shared Lexicon for Research and Practice in Human-Centered Software Systems. In 1st Conference on Fairness, Accountability, and Transparancy. New York, NY, USA. 7.
- Joshua A Kroll , Solon Barocas , Edward W Felten , Joel R Reidenberg , David G Robinson , and Harlan Yu . 2016 . Accountable algorithms . U. Pa. L. Rev. 165 (2016), 633 . Joshua A Kroll, Solon Barocas, Edward W Felten, Joel R Reidenberg, David G Robinson, and Harlan Yu. 2016. Accountable algorithms. U. Pa. L. Rev. 165 (2016), 633.
- Institute of Internal Auditors. Research Foundation and Institute of Internal Auditors . 2007 . The Professional Practices Framework. Inst of Internal Auditors . Institute of Internal Auditors. Research Foundation and Institute of Internal Auditors. 2007. The Professional Practices Framework. Inst of Internal Auditors.
- Christian Sandvig , Kevin Hamilton , Karrie Karahalios , and Cedric Langbort . 2014. Auditing algorithms: Research methods for detecting discrimination on internet platforms. Data and discrimination: converting critical concerns into productive inquiry 22 ( 2014 ). Christian Sandvig, Kevin Hamilton, Karrie Karahalios, and Cedric Langbort. 2014. Auditing algorithms: Research methods for detecting discrimination on internet platforms. Data and discrimination: converting critical concerns into productive inquiry 22 (2014).
- Tim Vanderveen . 2005 . Averting highest-risk errors is first priority . Patient Safety and Quality Healthcare 2 (2005), 16 -- 21 . Tim Vanderveen. 2005. Averting highest-risk errors is first priority. Patient Safety and Quality Healthcare 2 (2005), 16--21.
Cited by
Items 1,101–1,200 of 1,410
Lee, S.-L., Hsu, M., Wang, M. Y.-F., & Wang, Y.-F. (2026) · Science Progress · doi:10.1177/00368504261462858
Retracted(2025) · IEEE Standards · doi:10.1109/ieeestd.2026.11570895 · Cited by 1
AI Incident Auditor: A Retrieval-Augmented Framework for Post-Hoc Forensics of LLM Outputs
Chandel, Y. S., Varshney, S., Joshi, V., et al. (2026) · doi:10.1109/icici68867.2026.11565049
“This provides a strong supervised baseline using a lightweight architecture [9].”
InfluentialZeka, B., & Johns, R. (2026) · Financial Planning Review · doi:10.1002/cfp2.70040 · Cited by 2
Ahmetoglu, Y., Constantinides, M., & Cox, A. L. (2026) · Annual Symposium on Human-Computer Interaction for Work (CHIWORK) · doi:10.1145/3808045.3808056 · Cited by 8
Authorised Interruption Capability in AI-intensive organisations
Velotto, G. (2026) · AI and Ethics · doi:10.1007/s43681-026-01207-y
"Are you biased right now, AI?": Investigating Supporting Youths' Systematic Evaluation of GenAI
Solyst, J., Walker, L., Handa, S., Nurdin, F., & Shapiro, R. B. (2026) · ACM International Conference on Interaction Design and Children (IDC) · doi:10.1145/3773077.3806125
“…and harm in ways that are de-fensible and usable for reflection and redesign, including implications for the current use and trajectories for improvement in the future [32, 44] Science education literature provides a complementary theoretical framework for teaching systematic model evaluation.”
Algorithmic Bias in AI Systems: Examining Its Impact on Fairness and Decision-MakingPreprint
Patel, D. (2026) · Research Square · doi:10.21203/rs.3.rs-10083731/v1
Urrutia-Onate, D., Perallos, A., & Onieva, E. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6987345
Responsible Leapfrogging: Countering the Automation of Inequality in Global Mental Health
DeFranza, D., & Bae, S. Y. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3812349
“These bodies must be capable of technically auditing the code and data of foreign AI vendors [55].”
Accountability Challenges in AI-Automated Life Cycle Assessments
Preuss, N., & You, F. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3812254 · Cited by 1
“Like with XAI, there are numerous forms an audit can take, such as internal audits [61], reviewability [17], audits of datasets [38], and traceability or reviewability [43].”
Deva, R., Thapa, A., Mehta, Z., Kapile, S. K., Jalota, S., Karusala, N., & Ismail, A. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3806721 · Cited by 1
“Building on this perspective, we argue that evaluation is not simply a neutral or one-time assessment of model performance, but an ongoing process through which values are negotiated, responsibilities are assigned, and judgments are made about acceptable system behavior [43, 46].”
InfluentialKhairallah, M. (2026) · International Symposium of Systems, Advanced Technologies and Knowledge (ISSATK) · doi:10.1109/issatk69667.2026.11566843
How Formerly Incarcerated People Envision Technologies for Prison Parole
Savage, S., Nava, J., Zhou, W. I., & Lee, H. J. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3812258
The Paradox of ‘Algorithmic’ Accountability and its Implications for Design
Ramesh, D., Parthasarathy, S., & Banović, N. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3812374
Don't Trust the Process: When Verifiability Undermines AI Accountability
Paris, T., Moon, A., & Guo, J. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3812373 · Cited by 1
Voices in the Loop: Mapping Participatory AI
Mushkani, R. A. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3812241
Kálmán, J., & Lapsánszky, A. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6993478
Ethical Frictions in AI-Enhanced Learning Environments: A Comparative European Analysis
Moch, E., Vrba, J., & Oberdieck, T. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-9883-9.ch014
The Moral-Constraint Divide in AI Development: Safety, Legitimacy, and Pluralism
Bailey, P. (2026) · IGI Global eBooks · doi:10.4018/979-8-3373-8192-3.ch010
Chopra, A. (2026) · VINE Journal of Information and Knowledge Management Systems · doi:10.1108/vjikms-07-2025-0288
Hong, P. C., & Park, Y. S. (2026) · IGI Global eBooks · doi:10.4018/979-8-3373-8192-3.ch011
Interdisciplinary AI Competencies as the Foundation of Digital Transformation
Kopyto, A., Wachnik, B., & Kizińska, A. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-29918-5_35
Rodríguez-Espinosa, N. (2026) · AI & Society · doi:10.1007/s00146-026-03196-2
Wang, Y. Y., Irion, K., Groth, P., & Harmouch, H. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3812313
Piridi, S., Asundi, S., Kamineni, S., & Dadi, C. B. (2026) · Human-Centric Intelligent Systems · doi:10.1007/s44230-026-00161-6
An Integrated Oversight Framework for Adaptive Instructional Systems
Estevam, M. C., Adlakha-Hutcheon, G., & Ojo, A. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-30015-7_28
Responsible NLP: Why performance is no longer enough
Lamsiyah, S. (2026) · Natural language processing. · doi:10.1017/nlp.2026.10030 · Cited by 1
“Model cards (Mitchell et al. 2019), audits (Raji et al. 2020), and risk management frameworks (National Institute of Standards and Technology 2023) offer partial tools for this broader view.”
Auditing a Robotic System for the AI Act
Lucaj, L., Bok, F., & van der Smagt, P. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3812223 · Cited by 2
“…Groves et al. 2024; Wilson et al. 2021], ensuring accountability in algorithmic decision-making [Birhane et al. 2024; Brown et al. 2021; Cen and Alur 2024; Raji et al. 2020], and qualitative studies to delineate auditing best practices [Bandy 2021; Costanza-Chock et al. 2022; Li and Goel 2024].”
Bureaucratic Silences: What the Canadian AI Register Reveals, Omits, and Obscures
Das, D., Tessono, C., Ahmed, S. I., & Guha, S. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3812336 · Cited by 6
Qadir, S., Ni, C., Vaidya, M. S., Ryu, H., Mulvaney, S. A., Kantarcıoğlu, M., Novak, L. L., Malin, B., Rose, S. L., & Yin, Z. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3806456 · Cited by 1
Algorithm Auditing Policies Rest on Flawed Assumptions About Public Sector Systems
Escher, N., Banović, N., & Green, B. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3812280 · Cited by 2
“In contrast, internal audits are performed by parties who have a financial relationship with algorithm developers, raising questions about their objectivity and usefulness as mechanisms for accountability [45, 68, 102, 103].”
Haidemariam, T., & Zheng, X. (2026) · Frontiers in Virtual Reality · doi:10.3389/frvir.2026.1860842 · Cited by 1
Bachmann, P. A., Boehmer, N., Klausner, L. D., & Lackner, M. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3806808
“The authors confirm that we did not use any generative AI tools during any stages of this research work.”
Strategic Opacity: The Paradox of Open Source AI Governance in DeepSeek and OpenAI
Dodds, T., & Chengyuan, L. (2026) · The Information Society · doi:10.1080/01972243.2026.2680886
“An effective governance framework is likely to require moving beyond code visibility and directly address protocol-layer control mechanisms (Raji et al. 2020).”
Human-Centered AI vs.Traditional AI
Anwar, N., Nanda, A. R., Kuba, M. S. S., et al. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-8217-3.ch001
HITLAP: Human in the Loop AI-Privacy Architecture for Next Generation Banking Automation
Koganti, H., Refonaa, J., Ignatius, A. G., Rao, A. V. S. S. R., Praba, R. S., & Keerthana, N. V. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-8217-3.ch006
Montshiwa, A. L. (2026) · AI and Ethics · doi:10.1007/s43681-026-01210-3
“Iterative open and axial coding identified recurring themes and implementation gaps across the literature, particularly the persistent divide between high-level ethical principles and operational implementation mechanisms [43, 49, 54].”
InfluentialContext-Aware Auditing of Facial Analysis Pipelines for Autism Support Dashboards
Chen, W., Fu, Z., & Yang, W. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-30833-7_20
Dignity-Aware AI: Positioning Edge and Federated Intelligence for Ethical Elderly Monitoring
Shao, X. (2026) · ACM AI Letters · doi:10.1145/3828755
“Internal auditing, update traceability, and institution-independent verification [9] are essential for sustaining trust and accountability in such systems.”
InfluentialMeenal, & Kumari, S. (2026) · International Conference on Connected Intelligence for Industrial Applications (CI2A) · doi:10.1109/ci2a69097.2026.11576816
Algorithmic Accountability in Greenwashing Detection: Hybrid Governance in ESG Auditing
Harke, G. (2026) · IGI Global eBooks · doi:10.4018/979-8-3373-8392-7.ch008
How continuity distinguishes autonomy from agency in agentic AI
Kahl, P. (2026) · Discover Artificial Intelligence · doi:10.1007/s44163-026-01675-5 · Cited by 3
“Accountable-algorithm and algorithmic-audit literature makes this explicit: the aim is to create records, procedures, and review mechanisms by which computational decisions can be checked against articulated standards [20, 28].”
On the definition and detection of cherry-picking in counterfactual explanations
Hinns, J., Goethals, S., der Veeken, S. V., Evgeniou, T., & Martens, D. (2026) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-026-01234-7 · Cited by 2
Razak, M. R. A., Nasrul, M. A. D., & Abbas, I. (2026) · Millah Journal of Religious Studies · doi:10.20885/millah.vol25.iss1.art2 · Cited by 3
“Hence, technical advances must be paired with content governance (shared schemas, publication SLAs, and release notes) and audit mechanisms (Raji et al., 2020).”
InfluentialRethinking technology adoption: responsible AI use as ethical practice in rural SMEs
Cenizo, C. (2026) · Small Enterprise Research · doi:10.1080/13215906.2026.2685206
Zureigat, Q. M. (2026) · EuroMid Journal of Business and Tech-innovation (EJBTI) · doi:10.51325/d4w0yq25
“The business ethics literature on AI in organisations has made significant advances in theorising the moral challenges of algorithmic decision-making from the perspectives of designers, deployers, and regulators (Mittelstadt et al., 2016; Diakopoulos, 2016; Raji et al., 2020).”
MoghadasNian, S., Basereh, A., & MoghadasNian, S. (2026) · doi:10.24108/preprints-3115752
AbdelAziz, K. (2026) · EuroMid Journal of Business and Tech-innovation (EJBTI) · doi:10.51325/3yx5qe22
“Recent academic work has pointed to the importance of algorithmic accountability as a specific governance challenge that warrants specific institutional responses (Diakopoulos, 2016; Kearns and Roth, 2019; Raji et al., 2020).”
Osman, M. (2026) · AI and Ethics · doi:10.1007/s43681-026-01228-7
“Third, they create information asymmetries between AI system operators and their principals—regula-tors, boards, and customers—that undermine the monitoring mechanisms through which accountability is conventionally enforced [13, 18].”
Artificial human leaders: The Transport–Ethics Model of trust and governance in digital human CEOs
Bonelli, M. I. (2026) · Computers in Human Behavior Artificial Humans · doi:10.1016/j.chbah.2026.100344
Counterfactual Red Teaming and Dialogical Ethics in Distributed Human–Computer Interaction
Martín-Peña, R. E. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-29583-5_31
Fang, S., & Chen, S. (2026) · AI and Ethics · doi:10.1007/s43681-026-01244-7
Cho, B.-C., & Ahn, S. (2026) · Information Systems Frontiers · doi:10.1007/s10796-026-10775-7
When AI Speaks the Curriculum: From Generic Chatbots to Context-Aware Learning Systems
Ahsan, A., McDonald, H., Saha, R., & Davison, C. (2026) · Systems · doi:10.3390/systems14070791 · Cited by 2
Ali, S., Raza, S. M., & Dey, S. (2026) · International Conference on Computing, Mathematics and Engineering Technologies (iCoMET) · doi:10.1109/icomet69771.2026.11591638
“Digital transformation research provides macro-level accounts of change [1]–[3]; capability studies quantify performance impacts [4]–[7]; governance and documentation tools specify controls [8]–[10], [31]–[34].”
InfluentialKumar, S., Kumari, S., Huzaifa, M., & Shamsi, F. (2026) · International Conference on Computing, Mathematics and Engineering Technologies (iCoMET) · doi:10.1109/icomet69771.2026.11592044
“By combining controlled enrollment with demographically representative data collection, the proposed system aims to improve reliability, fairness, and operational feasibility in large scale examination environments [10].”
InfluentialGender, Race, and Class in the AI-Driven Workplace
Subrahmanyam, S., & Azzi, G. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-2600-2900-8.ch003
Banaybanay, R. J., & Ricafort, R. B. (2026) · International Journal of Latest Technology in Engineering Management & Applied Science · doi:10.51583/ijltemas.2026.150600080
Stanizzi, S. (2026) · AI Law Politics · doi:10.5709/alp-02.01.2026-06 · Cited by 1
Boban, M., Gombar, M., & Bachal, M. P. (2026) · International Convention on Information and Communication Technology, Electronics and Microelectronics · doi:10.1109/mipro70003.2026.11591781
A Conceptual Framework for Human-in-the-Loop Multi-attribute Decision Making Under Uncertainty
Çakır, E., Demırcıoğlu, E., & Tolga, A. Ç. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-2497-5_12
Shaping Work and Creativity in the Age of AI: Benefits, Challenges, and Design Guidelines
Banerjee, U., & Saha, K. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-30826-9_1
Huang, Y., Ke, G., Du, Y., Feng, K., & Su, Y. (2026) · Energies · doi:10.3390/en19143230 · Cited by 1
Algorithmic Governance and Transparency in Automated Decision-Making
Sedano, J. J. F., Valadez, M. E., & Castillo, A. I. D. (2026) · IGI Global eBooks · doi:10.4018/979-8-3373-8592-1.ch005
Zhang, Y., Yu, G., & Shang, S. (2026) · International Communication of Chinese Culture · doi:10.1007/s40636-026-00372-3
“Related work on GPT-3 and algorithmic auditing also treats model capability and accountability gaps as institutional rather than merely technical questions (Floridi and Chiriatti 2020; Raji et al. 2020) (Fig.”
WHAT: The Proposition (Optimization)
Walther, C. C. (2026) · Springer eBooks · doi:10.1007/978-3-032-06278-9_2
Human-AI collaboration in internal auditing: the moderating role of financial reporting quality
Jurczuk, A., Alsqour, M., & Zaqeeba, N. (2026) · Engineering Management in Production and Services · doi:10.2478/emj-2026-0008 · Cited by 1
“This enhances the ability to detect unusual transactions, identify control weaknesses, and prioritise audit areas with higher risk exposure (Boskou et al., 2019; Raji et al., 2020).”
Walther, C. C. (2026) · Springer eBooks · doi:10.1007/978-3-032-06278-9_1
From Retrieval to Responsibility: Designing Creative MRAG with Control and Governance
Yu, H., Jiang, Z., Wilson, E., & Yang, Y. (2026) · Creativity and Cognition · doi:10.1145/3803784.3816842
Will AI Replace Humans in Strategizing Communications? Ethical Considerations
Selaković, M., Ljepava, N., Gačić, M., & Filipović, N. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-23890-0_19
MoghadasNian, S., AlizadehMousavi, S., HosseinPoor, S., Farmanian, M., & Bagheri, A. (2026) · doi:10.24108/preprints-3115839
Ersozlu, Z., Ledger, S., Babic, M., & Parkes, R. (2026) · Contemporary Educational Technology · doi:10.30935/cedtech/18970 · Cited by 3
When Human-in-the-Loop Fails: Accountability Across AI Adoption Levels in Human-Machine Systems
Roy, A. (2026) · IEEE International Conference on Human-Machine Systems (ICHMS) · doi:10.1109/ichms69701.2026.11602198
“HITL often functions as accountability theater: procedural indications of oversight that do not meaningfully constrain system behaviour.”
Auditing Agentic AI Systems: Governance, Risk, and Control Frameworks
Kadali, R. S. (2026) · IGI Global eBooks · doi:10.4018/979-8-3373-8277-7.ch008
Managed Agents vs. Workflow Automation
Khare, V. (2026) · IGI Global eBooks · doi:10.4018/979-8-3373-8277-7.ch007
Governing Artificial Womb Technology: AI, Ethics, and Policy at the Boundaries of Human Development
Abadla, R., Alahmed, Y., & Alabadla, S. (2026) · Artificial intelligence · doi:10.5772/intechopen.1015650
A Mandate Beyond Accuracy: Forging the Algorithm’s Social Contract
Singh, N., & Foulds, J. R. (2026) · Artificial intelligence · doi:10.5772/intechopen.1015619
Data Governance for Generative AI: Curation, Documentation, and Lifecycle Management
Armand, K. W. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-2600-0368-8.ch006
From Autonomy to Alienation: Exploring Organisational Dehumanisation Among Ride-Hailing Drivers
Shaiwalini, S. (2026) · Group & Organization Management · doi:10.1177/10596011261469248 · Cited by 1
Guyeux, C. (2026) · IEEE International Conference on Smart Sustainable Systems for Computer and Engineering Applications (3SCEA) · doi:10.1109/3scea68071.2026.11603160 · Cited by 2
“Prior work on algorithmic auditing highlights the need for end-to-end evidence chains that connect outputs to inputs and decision context, enabling external review and legal defensibility [10].”
AI Persona Design as Interaction Specification: Agency and Accountability
Park, G. (2026) · IEEE International Conference on Smart Sustainable Systems for Computer and Engineering Applications (3SCEA) · doi:10.1109/3scea68071.2026.11603096
Badillo, R. (2026) · Qeios · doi:10.32388/28u0sw
Sobuj, M. A. M., Akter, S., Bhowmick, P. K., Sohan, M. F. A. A., & Nahar, A. (2026) · International Conference on Frontiers of Engineering and Emerging Technologies (FET) · doi:10.1109/fet68771.2026.11601710
“Raji et al. [9] proposed an algorithmic auditing framework that integrates documentation, continuous monitoring, and stakeholder oversight throughout the lifecycle of the model.”
The shared blind spot: why diverse AI governance approaches fail for the same reason
Kim, S. H. (2026) · AI & Society · doi:10.1007/s00146-026-03257-6 · Cited by 1
Audit in the Era of Digital Transformation: Challenges in the Evolution from Audit 4.0 to Audit 5.0
GĂBOROI, I. C., Cazoni, C., Pîrcălăboiu, A., & Ciocoiu, C. N. (2026) · Proceedings of the ... International Conference on Business Excellence · doi:10.2478/picbe-2026-0027
“Scholars such as Vinuesa et al. (2020) and Raji et al. (2020) place particular emphasis on the ethical dimension of Audit 5.0, highlighting the importance of addressing risks such as algorithmic bias, model opacity, decision accountability, and the social impact of AI systems.”
InfluentialEmbodied AI Framework for Self-Adaptive Decision Reliability in Autonomous Engineering Environments
Aaron, S. A., Newaz, A., Abir, S. H., Rahman, M. M., Ahosan, N., & Hossain, M. M. (2026) · International Symposium on AI-Driven Engineering Systems (ISADES) · doi:10.1109/isades69945.2026.11608160
“…for safe learning [27], model-based reinforcement learning [28], shielding [29], autonomous vehicle assurance [30], neural network verification [31][32][33][34], interpretability [35][36][37], audits for accountability [38], governance principles [39], and converging ethics guidelines [40].”
Krasteva-Hristova, R., & Georgieva, V. (2026) · Accounting and Auditing · doi:10.3390/accountaudit2030015
Mohanty, P., & Mishra, S. (2026) · Business Strategy and the Environment · doi:10.1002/bse.71333
Dai, Y., Zhou, J., Song, Y., Li, T., & Xue, B. (2026) · Sustainability · doi:10.3390/su18147500 · Cited by 1
“A visibility audit should make its assumptions, data boundaries, and interpretive limits explicit [43,44].”
Autonomous Bias Mitigation in Talent Acquisition: An Intelligent Redaction Framework for Fair Hiring
Lata, S. (2026) · IEEE International Conference on AI and Data Analytics (ICAD) · doi:10.1109/icad69378.2026.11608591
Voinea, D. V. (2026) · Social Sciences · doi:10.3390/socsci15080496 · Cited by 1
“No single measure can establish democratic adequacy, and audits do not create accountability on their own: they must operate within institutions that give independent, technically capable auditors enough authority and access to act (Raji et al. 2020, 2022; Birhane et al. 2024).”
Kalluru, V. (2026) · ACM SIGCAS/SIGCHI Conference on Computing and Sustainable Societies (COMPASS) · doi:10.1145/3811242.3827541
Kun, A. L. (2026) · Cambridge University Press eBooks · doi:10.1017/9781009625180.005 · Cited by 1
Prüfung, Zertifizierung und Assurance
Barenkamp, M. (2026) · Springer eBooks · doi:10.1007/978-3-658-52322-0_15
Barón, M. J. S. (2026) · Springer proceedings in business and economics · doi:10.1007/978-3-032-24600-4_9
Xu, Y., Sampaio, S. C. D. B., & Cormican, K. T. (2026) · Journal of Responsible Innovation · doi:10.1080/23299460.2026.2688598
Epistemic Delegation and Africa’s Future in the Age of AI
Negedu, I. A., Mamah, J. C., & Akpan, T. M. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2026.2706239
Consent collapse in agentic AI: moral authorization beyond the boundaries of task delegation
Brobbey, A. A., & Blu, R. (2026) · AI and Ethics · doi:10.1007/s43681-026-01287-w
Metadata
- Authors
- Raji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P.
- Affiliations
- Partnership on AI
- Google (United States) (US)
- ORCID
- Venue
- Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency
- Publisher
- ACM
- Published
- 2020-01-27
- Pages
- 33-44
- OpenAlex
- W3001807593
- Cited by
- 2,245
- References
- 83
- Citations by year
- 2026: 1097 · 2025: 454 · 2024: 243 · 2023: 177 · 2022: 149 · 2021: 99 · 2020: 24
- FWCI
- 35.91
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Safety Systems Engineering in Autonomy
- Keywords
- AI accountability, algorithmic auditing, harm detection, algorithmic transparency, societal impact of AI, algorithmic governance
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-07, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-07 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.