Reviewable Automated Decision-Making: A Framework for Accountable Algorithmic Systems
Open accessThe DOI matches its Crossref record; 60 of the 60 references it cites were found in a registry or an index; 33 of the 53 sources its claims cite were checked for support.
Abstract
This paper introduces reviewability as a framework for improving the accountability of automated and algorithmic decisionmaking (ADM) involving machine learning. We draw on an understanding of ADM as a socio-technical process involving both human and technical elements, beginning before a decision is made and extending beyond the decision itself. While explanations and other model-centric mechanisms may assist some accountability concerns, they often provide insufficient information of these broader ADM processes for regulatory oversight and assessments of legal compliance. Reviewability involves breaking down the ADM process into technical and organisational elements to provide a systematic framework for determining the contextually appropriate record-keeping mechanisms to facilitate meaningful review - both of individual decisions and of the process as a whole. We argue that a reviewability framework, drawing on administrative law's approach to reviewing human decision-making, offers a practical way forward towards more a more holistic and legally-relevant form of accountability for ADM.
Overview
CoChat Summary
TL;DR
This paper introduces “reviewability” as a framework for improving accountability of automated and algorithmic decision-making (ADM) involving machine learning by breaking ADM into technical and organisational stages and specifying contextually appropriate record-keeping and logging. It argues that model-centric explanations are insufficient for regulatory oversight and legal compliance, and that a holistic, process-based approach drawing on English administrative law can better support meaningful accountability.
Key findings
- Model-focused transparency and explanation mechanisms are insufficient because ADM is a socio-technical process with human and organisational elements; problems often originate outside the model, in purposes, assumptions, design, and use.
- Reviewability requires contextually appropriate information that is (i) relevant to the accountability relationships involved, (ii) accurate, complete, and representative, (iii) proportionate to the required transparency, and (iv) comprehensible by the relevant forums.
- The paper proposes a framework with four stages: Commissioning (procurement, problem definition, impact assessment), Model building (data collection, pre-processing, model training, model testing), Decision-making (deployment, use, consequences), and Investigation (audit, disclosure).
- It draws on accountability as involving an actor, a forum, an account, and consequences, and cites five kinds of accountability relationship: political, legal, administrative, professional, and social. It also identifies three generic actor roles: managers, developers, and users.
Methods
- Conceptual and legal-theoretical framework development, drawing on English administrative law, accountability literature, and prior work applying administrative law to ADM.
- No empirical sample or quantitative analysis is reported; the paper synthesises related research and proposes a systematic framework for reviewable ADM.
- Main analysis breaks the ADM process into stages and steps, then identifies record-keeping and logging mechanisms that could provide contextually appropriate information for review of individual decisions and the process as a whole.
Limitations
- The paper states that reviewability cannot on its own address all potential problems with ADM; wider examination of political economy and socio-economic context, as well as legal protections for individuals and groups, is also needed.
- It states that transparency has limits: reviewable systems may not capture broader business processes, wider context, interactions with other socio-technical processes, business models, or questions of power; transparency also has temporal limitations and is open to subjective interpretation and contestation.
- It states that implementation research is needed on what record-keeping and logging are appropriate at each step, what information to retain, and how best to present it to forums; recording everything can be impractical and can create a “transparency paradox.”
- Evident from the text: the framework is conceptual and normative, with no empirical evaluation or deployment study reported.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
23 Valid · 10 Not valid · 12 Not checked · 78 Own results and interpretations
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Administrative law as a framework and judicial review as an oversight mechanism understand decision-making as a process beginning before decisions are made and extending to their consequences and effects.”
§ Reviewable Automated Decision-Making: A Framework for Accountable Algorithmic Systems > 1 INTRODUCTION
What the sources say
- Supportedcites [13] 13. Jennifer Cobbe. 2019. Administrative Law and the Machines of Government: Judicial Review of Automated Public-Sector Decision-Making. Legal Studies (2019). source
The Review of ADM (p. 9) says judicial review of an individual decision may need to examine earlier choices about training data, system design and testing. The discussion of reasons (p. 22) says administrative-law requirements depend on how serious and consequential a decision is. Together, these support the claim that oversight considers both what precedes a decision and its effects. (LLM, conf=0.86)
- Not checked — no full text of the sourcecites [14] 14. Jennifer Cobbe, Michelle Seng Ah Lee, Heleen Janssen, and Jatinder Singh. 2020. Centring the Rule of Law in the Digital State. IEEE Computer (2020). source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Models are only one aspect of ADM and cannot by themselves provide accounts of the process as a whole [66], whether to technical, legal, or other forums.”
§ Reviewable Automated Decision-Making: A Framework for Accountable Algorithmic Systems > 2.2.1 Meaningful accountability needs more than models.
What the sources say
- Supportedcites [66] 66. Jatinder Singh, Jennifer Cobbe, and Chris Norval. 2020. Decision Provenance: Harnessing Data Flow for Accountable Systems. IEEE Access 7 (2020). source
In “V. Case Study: Machine Learning Pipelines” (page unavailable), the paper describes models as components of interconnected systems and says accountability requires information about data flows, model use, and the downstream effects of decisions. “B. Technical Assistance” (page unavailable) connects system-wide transparency to both technical oversight and legal accountability. This supports the claim that a model alone cannot account for the whole decision-making process. (LLM, conf=0.88)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Rather, people are concerned with broader aspects of ADM: the purposes, roles, and outcomes of decision-making processes as a whole, as it is the whole process that affects them rather than simply certain technical aspects of it [26].”
§ Reviewable Automated Decision-Making: A Framework for Accountable Algorithmic Systems > 2.2.1 Meaningful accountability needs more than models.
What the sources say
- Partly supportedcites [26] 26. Lilian Edwards and Michael Veale. 2017. Slave to the Algorithm? Why a 'Right to an Explanation' is Probably Not the Remedy You Are Looking For. Duke Law & Technology Review 17 (2017). sourceExperimental — accuracy not yet measured
The Introduction (p. 2) says the public is troubled by systems that govern their lives and produce damaging or unfair results. The Conclusion (p. 64) favors broader algorithmic governance over explanations of a black box’s interior. These passages support the contrast with purely technical explanations, but they do not establish that people are specifically concerned with the purposes and roles of decision-making processes as a whole. (LLM, conf=0.78)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: unclear — It is unclear whether it could be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“However, understanding ADM as a sociotechnical process, we argue that there is another form of opacityunwitting opacity, where those responsible for designing, developing, deploying, and using systems simply don't think to record relevant organisational aspects of ADM processes (perhaps unaware of their relevance for meaningful accountability [12]).”
§ Reviewable Automated Decision-Making: A Framework for Accountable Algorithmic Systems > 2.1 Accountability in ADM
What the sources say
- Partly supportedcites [12] 12. Richard Cloete, Chris Norval, and Jatinder Singh. 2020. A Call for Auditable Virtual, Augmented and Mixed Reality. sourceExperimental — accuracy not yet measured
Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Though explanations can assist some specific concerns, like model engineering [7], explanations or other approaches focused on how the model itself works or has reached a particular outcome may miss much of what is important [75].”
§ Reviewable Automated Decision-Making: A Framework for Accountable Algorithmic Systems > 2.2 Limitations of model-focused mechanisms
What the sources say
- Supportedcites [7] 7. Umang Bhatt, Alice Xiang, Shubham Sharma, Adrian Weller, Ankur Taly, Yun- han Jia, Joydeep Ghosh, Ruchir Puri, José M. F. Moura, and Peter Eckersley. 2020. Explainable Machine Learning in Deployment. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (Barcelona, Spain) (FAT* '20). Association for Computing Machinery, New York, NY, USA, 648?657. https://doi.org/10.1145/3351095.3375624 sourceExperimental — accuracy not yet measured
The paper’s §3.4 Key Takeaways says that a description of how a model responds to an input change may not explain anything useful to stakeholders or reveal the underlying causal mechanism. Section 5.1 Establish Clear Desiderata likewise emphasizes identifying stakeholders’ needs and the purpose of an explanation. Together, these support the hedged claim that focusing on model behavior or a particular prediction can miss important concerns. (LLM, conf=0.86)
- Supportedcites [75] 75. Maranke Wieringa. 2020. What to account for when accounting for algorithms: a systematic literature review on algorithmic accountability. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 1-18. sourceExperimental — accuracy not yet measured
Section 3.2 says that understanding a system’s technical workings can be useful but is insufficient for accountability because it does not explain why the system was adopted or who set its specifications. Section 3.4 similarly argues that explanations focused on technical aspects can miss the system’s social context, development, and effects. (LLM, conf=0.94)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: unclear — It is unclear whether it could be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
24 more claims · 8 with their cited sources checked
Create a free account to see all 123 claims and every source verdict.
- Probably not a substantial claim. 87
- It may state two separate claims as one. 24
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Accountability has five types: legal responsibility to judicial bodies; political accountability to elected representatives and similar entities; administrative accountability to auditors and regulatory agencies; professional responsibility to internal and external colleagues; and social responsibility to community members and the public (Cobbe et al., 2021).”
From Reconciling Administrative Efficiency with Procedural Safeguards in the ADM System (2025) · cites this paper as Cobbe et al., 2021
In the citing paper
“Transparency which supports broader accountability processes on training sets, models and internal processes behind these services, as well as in the widersense of acknowledgement of known-limitations, past failures, and lessons learnt [27,131] can help move AIaaS from the opaque, unknown and potentially harmful 'black-boxes' of today, to a more understandable, accepting of their own limitations,”
From Out of Context: Investigating the Bias and Fairness Concerns of “Artificial Intelligence as a Service” (2023) · cites this paper as [27]
Citations
Cites
Arrieta, A. B., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., García, S., Gil-López, S., Molina, D., Benjamins, R., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.10045 · Cited by 11,321
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423 · cited 2×
“…of the research landscape show a vast number of mechanisms for developing ‘interpretable’ or ‘explainable’ ML systems have been proposed, seeking to provide explanations and other interpretable accounts of model behaviour to a range of forums, many of which have come in the last few years [4, 32].”
BackgroundThe Black Box Society: The Secret Algorithms That Control Money and Information
Pasquale, F. (2015) · doi:10.4159/harvard.9780674736061 · Cited by 3,796
A few useful things to know about machine learning
Domingos, P. (2012) · Communications of the ACM · doi:10.1145/2347736.2347755 · Cited by 3,326
How the machine “thinks”: Understanding opacity in machine learning algorithms
Burrell, J. (2016) · Big Data & Society · doi:10.1177/2053951715622512 · Cited by 3,196 · cited 5×
“Burrell identifies three forms of algorithmic opacity [10]; while Burrell refers primarily to model opacity, these also relate to broader ADM processes.”
BackgroundMethodInfluentialModel Cards for Model Reporting
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287596 · Cited by 2,681
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Daumé, H., & Crawford, K. (2018) · arXiv · doi:10.48550/arxiv.1803.09010 · Cited by 2,474
Raji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372873 · Cited by 2,245 · cited 1×
“Even where assessments are not legally required, they are good practice, and robust internal (in-organisation) assessment processes have value [60].”
BackgroundEuropean Union Regulations on Algorithmic Decision Making and a “Right to Explanation”
Goodman, B., & Flaxman, S. (2017) · AI Magazine · doi:10.1609/aimag.v38i3.2741 · Cited by 2,131 · cited 1×
“Some proposals have seemingly been prompted by academic debate about the existence, nature, extent, and utility of a so-called ‘right to an explanation’ [26, 31, 44, 64, 74] in GDPR, which was passed in 2016.”
BackgroundAnanny, M., & Crawford, K. (2018) · New Media & Society · doi:10.1177/1461444816676645 · Cited by 1,836 · cited 8×
“There are limits to transparency itself as a mechanism for supporting accountability [2, 26, 68].”
BackgroundInfluentialThe Black Box Society: The Secret Algorithms That Control Money and Information
(2016) · Contemporary Sociology: A Journal of Reviews · doi:10.1177/0094306116641409c · Cited by 1,334
Wachter, S., Mittelstadt, B., & Floridi, L. (2017) · International Data Privacy Law · doi:10.1093/idpl/ipx005 · Cited by 1,308
Amazon Scraps Secret AI Recruiting Tool that Showed Bias against Women *
Dastin, J. (2022) · Ethics of Data and Analytics · doi:10.1201/9781003278290-44 · Cited by 1,165
Consumer Credit Risk Models Via Machine-Learning AlgorithmsPreprint
Khandani, A., Kim, A., & Lo, A. W. (2010) · SSRN Electronic Journal · doi:10.2139/ssrn.1568864 · Cited by 895
Consumer credit-risk models via machine-learning algorithms
Khandani, A. E., Kim, A. J., & Lo, A. W. (2010) · Journal of Banking & Finance · doi:10.1016/j.jbankfin.2010.06.001 · Cited by 864
Bender, E. M., & Friedman, B. (2018) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00041 · Cited by 854
Accountability in algorithmic decision making
Diakopoulos, N. (2016) · Communications of the ACM · doi:10.1145/2844110 · Cited by 763
Algorithmic Accountability: Journalistic investigation of computational power structures
Diakopoulos, N. (2015) · Digital Journalism · doi:10.1080/21670811.2014.976411 · Cited by 733
Explainable machine learning in deployment
Bhatt, U., Xiang, A., Sharma, S., Weller, A., Taly, A., Jia, Y., Ghosh, J., Puri, R., Moura, J. M. F., & Eckersley, P. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3375624 · Cited by 633
Data Preprocessing For Supervised Leaning
Kotsiantis, S. B., Kanellopoulos, D. N., & Pintelas, P. (2007) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.1082414 · Cited by 631
A Survey of Methods for Explaining Black Box Models
Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., & Pedreschi, D. (2019) · ACM Computing Surveys · doi:10.1145/3236009 · Cited by 629 · cited 1×
“Surveys of the research landscape show a vast number of mechanisms for developing ‘interpretable’ or ‘explainable’ ML systems have been proposed, seeking to provide explanations and other interpretable accounts of model behaviour to a range of forums, many of which have come in the last few years [4, 32].”
BackgroundKuziemski, M., & Misuraca, G. (2020) · Telecommunications Policy · doi:10.1016/j.telpol.2020.101976 · Cited by 610 · cited 1×
“cial services [38], and immigration and employment [41], and is”
BackgroundThe Black Box Society: The Secret Algorithms That Control Money and Information
(2015) · Choice Reviews Online · doi:10.5860/choice.190706 · Cited by 533 · cited 1×
“This includes proposals for transparency of algorithms themselves, making code, model benchmarks, or other aspects of the model lifecycle available for scrutiny [10, 23, 45, 50, 56, 57].”
MethodEdwards, L., & Veale, M. (2017) · doi:10.31228/osf.io/97upg · Cited by 524
Meaningful information and the right to explanation
Selbst, A. D., & Powles, J. (2017) · International Data Privacy Law · doi:10.1093/idpl/ipx022 · Cited by 516
FactSheets: Increasing trust in AI services through supplier's declarations of conformity
Arnold, M., Bellamy, R. K. E., Hind, M., Houde, S., Mehta, S., Mojsilović, A., Nair, R., Ramamurthy, K. N., Olteanu, A., Piorkowski, D., et al. (2019) · IBM Journal of Research and Development · doi:10.1147/jrd.2019.2942288 · Cited by 434 · cited 2×
“approaches such as service ‘factsheets’ [3] that describe aspects of their service beyond technical capabilities.”
BackgroundWieringa, M. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372833 · Cited by 321
Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims
Brundage, M., Avin, S., Wang, J., Belfield, H., Krueger, G., Hadfield, G., Khlaaf, H., Yang, J., Toner, H., Fong, R., et al. (2020) · arXiv · doi:10.48550/arxiv.2004.07213 · Cited by 306 · cited 1×
“Any such testing should be presented in the form of “verifiable claims” with audit trails and explanations of model predictions [9].”
BackgroundMalgieri, G., & Comandé, G. (2017) · International Data Privacy Law · doi:10.1093/idpl/ipx019 · Cited by 271 · cited 1×
“Some proposals have seemingly been prompted by academic debate about the existence, nature, extent, and utility of a so-called ‘right to an explanation’ [26, 31, 44, 64, 74] in GDPR, which was passed in 2016.”
BackgroundTransparent to whom? No algorithmic accountability without a critical audience
Kemper, J., & Kolkman, D. (2019) · Information, Communication & Society · doi:10.1080/1369118x.2018.1477967 · Cited by 260
On the (im)possibility of fairness
Friedler, S. A., Scheidegger, C., & Venkatasubramanian, S. (2016) · arXiv · doi:10.48550/arxiv.1609.07236 · Cited by 247
Algorithmic Accountability Reporting: On the Investigation of Black Boxes
Diakopoulos, N. A. (2014) · Columbia Academic Commons (Columbia University) · doi:10.7916/d8zk5tw2 · Cited by 221 · cited 1×
“Some have argued for equipping the general public with skills and knowledge to understand how models work [10], or that journalists could reverse engineer algorithms to inform the public about their workings [21, 22].”
BackgroundTrustworthy whole-system provenance for the Linux kernel
Bates, A., Tian, D. J., Butler, K. R. B., & Moyer, T. (2015) · USENIX Security Symposium · Cited by 190
Raji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P. (2020) · arXiv · doi:10.48550/arxiv.2001.00973 · Cited by 179
Cobbe, J. (2019) · Legal Studies · doi:10.1017/lst.2019.9 · Cited by 174 · cited 4×
“Our previous work considered how administrative law and judicial review can apply to public sector ADM [13].”
BackgroundInfluentialSelf-citationToward situated interventions for algorithmic equity: lessons from the field
Katell, M., Young, M., Dailey, D., Herman, B., Guetler, V., Tam, A., Bintz, C., Raz, D., & Krafft, P. M. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372874 · Cited by 153 · cited 1×
“Accountability also must consider algorithmic systems as they are situated – not just technical features, but how they affect different people in their contexts of use [36].”
Background(2020) · Iowa law review · Cited by 151
The Rule of Law and Automation of Government Decision-Making
Zalnieriute, M., Bennett Moses, L., & Williams, G. (2019) · doi:10.2139/ssrn.3348831 · Cited by 140
Oswald, M. (2018) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2017.0359 · Cited by 132 · cited 1×
“lish or otherwise) to ADM in various ways [11, 15, 46, 55, 70, 78].”
BackgroundRequirements Engineering for Software and Systems
Laplante, P. A. (2013) · Auerbach Publications eBooks · doi:10.1201/b15939 · Cited by 128 · cited 1×
“Information from business analysis or requirements engineering activities, common for many organisational technology undertakings [42], will often be pertinent, as will any documents such as minutes from board”
BackgroundDecision Provenance: Harnessing Data Flow for Accountable Systems
Singh, J., Cobbe, J., & Norval, C. (2019) · IEEE Access · doi:10.1109/access.2018.2887201 · Cited by 110
Self-citationA governance framework for algorithmic accountability and transparency
Koene, A. R., Clifton, C., Hatada, Y., Webb, H., & Richardson, R. (2019) · Repository@Nottingham (University of Nottingham) · doi:10.2861/59990 · Cited by 104
A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle
Suresh, H., & Guttag, J. V. (2019) · arXiv · doi:10.48550/arxiv.1901.10002 · Cited by 82
A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle
Suresh, H., & Guttag, J. (2021) · Equity and Access in Algorithms, Mechanisms, and Optimization · doi:10.1145/3465416.3483305 · Cited by 82
Automatically Tracking Metadata and Provenance of Machine Learning Experiments
Schelter, S., Böse, J.-H., Kirschnick, J., Klein, T., & Seufert, S. (2017) · Cited by 78
Procurement As Policy: Administrative Process for Machine Learning
Mulligan, D. K., & Bamberger, K. A. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3464203 · Cited by 75
Duboc, L., Betz, S., Penzenstadler, B., Akinli Kocak, S., Chitchyan, R., Leifler, O., Porras, J., Seyff, N., & Venters, C. C. (2019) · 2019 IEEE 27th International Requirements Engineering Conference (RE) · doi:10.1109/re.2019.00013 · Cited by 69
An approach for a fundamental rights impact assessment to automated decision-making
Janssen, H. (2020) · Apollo (University of Cambridge) · doi:10.17863/cam.54393 · Cited by 50
Raji, I. D., & Yang, J. (2019) · arXiv · doi:10.48550/arxiv.1912.06166 · Cited by 36
An approach for a fundamental rights impact assessment to automated decision-making
Janssen, H. L. (2020) · International Data Privacy Law · doi:10.1093/idpl/ipz028 · Cited by 36 · cited 2×
“There are many materials on various topics that can assist assessment undertakings [34, 35, 61].”
BackgroundResponsibility & Machine Learning: Part of a ProcessPreprint
Singh, J., Walden, I., Crowcroft, J., & Bacon, J. (2016) · SSRN Electronic Journal · doi:10.2139/ssrn.2860048 · Cited by 34 · cited 1×
“This might include checking for model skew, or any inappropriate discrimination or other behaviour that may be manifesting [67].”
BackgroundSelf-citationHow Portable is Portable?: Exercising the GDPR's Right to Data Portability
Wong, J., & Henderson, T. (2018) · Proceedings of the 2018 ACM International Joint Conference and 2018 International Symposium on Pervasive and Ubiquitous Computing and Wearable Computers · doi:10.1145/3267305.3274152 · Cited by 27 · cited 1×
“Note that, in practice, many organisations appear ill-prepared for meeting their disclosure obligations [65, 76] Records about disclosures themselves can also be relevant – both of the processes for disclosure as well as what was actually released, how information was compiled, how it was delivered, in what format, to…”
BackgroundThe Security Implications of Data Subject Rights
Singh, J., & Cobbe, J. (2019) · IEEE Security & Privacy · doi:10.1109/msec.2019.2914614 · Cited by 17 · cited 1×
“Note that, in practice, many organisations appear ill-prepared for meeting their disclosure obligations [65, 76] Records about disclosures themselves can also be relevant – both of the processes for disclosure as well as what was actually released, how information was compiled, how it was delivered, in what format, to…”
BackgroundSelf-citationA Call for Auditable Virtual, Augmented and Mixed Reality
Cloete, R., Norval, C., & Singh, J. (2020) · 26th ACM Symposium on Virtual Reality Software and Technology · doi:10.1145/3385956.3418960 · Cited by 8 · cited 3×
“Technical toolkits may also play a role in facilitating the logging and information to be extracted from technical components in a common manner [12].”
BackgroundSelf-citationAuditable Augmented/Mixed/Virtual Reality: The Practicalities of Mobile System Transparency
Cloete, R., Norval, C., & Singh, J. (2021) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3495001 · Cited by 7
Self-citation6. Algorithmic Accountability Reporting
Diakopoulos, N. A. (2019) · Harvard University Press eBooks · doi:10.4159/9780674239302-007 · Cited by 6
Proving Public Law Error in Automated Decision-Making SystemsPreprint
Tomlinson, J., Sheridan, K., & Harkens, A. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3476657 · Cited by 4 · cited 1×
“lish or otherwise) to ADM in various ways [11, 15, 46, 55, 70, 78].”
BackgroundCentering the Rule of Law in the Digital StatePreprint
Cobbe, J., Lee, M. S. A., Janssen, H., & Singh, J. P. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3673843 · Cited by 2
Self-citationRequirements Engineering for Software and Systems, Third Edition
Laplante, P. A. (2017) · Auerbach Publications eBooks · doi:10.1201/9781315303710 · Cited by 2
The Security Implications of Data Subject RightsPreprint
Singh, J. P., & Cobbe, J. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3385779
Self-citation
Research works printed but not linked: 14
- Philip Alston. 2019. Report of the Special Rapporteur on extreme poverty and human rights. United Nations General Assembly A/74/493 (2019). https://undocs.org/A/74/493
- Mark Bovens. 2006. Analysing and Assessing Public Accountability. A Conceptual Framework. EUROGOV, European Governance Papers No. C-06-01 (2006).
- Cary Coglianese and Daniel Lehr. 2017. Regulating by Robot: Administrative Decision Making in the Machine-Learning Era. Georgetown Law Journal 105 (2017).
- European Commission. 2020. White Paper On Artificial Intelligence - A European Approach to Excellence and Trust. COM(2020) 65 (2020).
- 10.1145/2347736 · doi:10.1145/2347736
- Avi Feller, Emma Pierson, Sam Corbett-Davies, and Sharad Goel. 2016. A computer program used for bail and sentencing decisions was labeled biased against blacks. It's actually not that clear. The Washington Post (2016).
- Agata Foryciarz, Daniel Leufer, and Katarzyna Szymielewicz. 2020. Black-Boxed Politics: Opacity is a Choice in AI Systems. Panoptykon Foundation (17 January 2020). https://en.panoptykon.org/articles/black-boxed-politics-opacity-choice-ai-systems
- 10.1093/idpl · doi:10.1093/idpl
- David Lehr and Paul Ohm. 2017. Playing with the Data: What Legal Scholars Should Learn About Machine Learning. UC Davis Law Review 51 (2017).
- House of Common Science and Technology Committee. 2018. Algorithms in Decision-Making. Fourth Report of Session 2017-19 HC 351 (2018).
- Douglas Pyper. 2020. Research Briefing: The Public Sector Equality Duty and Equality Impact Assessments. House of Commons Library (2020). https://commonslibrary.parliament.uk/research-briefings/sn06591/
- Nick Seaver. 2013. Knowing Algorithms. paper presented at Media in Transition 8 (2013).
- Cynthia Stohl, Michael Stohl, and Paul M Leonardi. 2016. Managing Opacity: Information Visibility and the Paradox of Transparency in the Digital Age. International Journal of Communication 10 (2016).
- European Union. 2016. Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation). Official Journal of the European Union L119 (4 May 2016), 1--88.
Cited by
Auditing large language models: a three-layered approach
Mökander, J., Schuett, J., Kirk, H. R., & Floridi, L. (2024) · AI and Ethics · doi:10.1007/s43681-023-00289-2 · Cited by 289
“Three ideas underpin the promise of auditing as an AI governance mechanism: that procedural regularity and transparency contribute to good governance [8], [9]; that proactivity in the design of AI systems helps identify risks and prevent harm before it occurs [10], [11]; and, that the independence between the auditor…”
BackgroundA literature review on applications of explainable artificial intelligence (XAI)
Kalasampath, K., Spoorthi, K. N., Sajeev, S., Kuppa, S. S., Ajay, K., & Maruthamuthu, A. (2025) · IEEE Access · doi:10.1109/access.2025.3546681 · Cited by 180
“By comprehending the capabilities and restrictions of each kind of explanation, developers, and stakeholders can develop AI systems that are more efficient and reliable, thus advancing ethical and responsible AI practices [114].”
Auditing of AI: Legal, Ethical and Technical Approaches
Mökander, J. (2023) · Digital Society · doi:10.1007/s44206-023-00074-y · Cited by 152
“Reflecting that convergence, previous research has developed procedures to audit AI systems for transparency and explainability (Cobbe et al., 2021; Mittelstadt, 2016), bias and fairness (Bartley et al., 2021; Morina et al., 2019) and accountability (Busuioc, 2021; Metcalf et al., 2021).”
MethodAI and Global Governance: Modalities, Rationales, Tensions
Veale, M., Matus, K., & Gorwa, R. (2023) · Annual Review of Law and Social Science · doi:10.1146/annurev-lawsocsci-020223-040749 · Cited by 134
“Issues of accountability can be separated from neither the much longer sociotechnical process (Cobbe et al. 2021) nor the need for an active, critical audience (Kemper & Kolkman 2018).”
BackgroundMethodOperationalising AI governance through ethics-based auditing: an industry case study
Mökander, J., & Floridi, L. (2023) · AI and Ethics · doi:10.1007/s43681-022-00171-7 · Cited by 129
“National regulators like the UK Information Commissioner’s Office have provided guidance on how to audit AI systems [18], and professional services firms like PwC, EY, Deloitte and KPMG have all developed auditing (or ‘assurance’) procedures to help clients ensure that the AI systems they design and deploy are legal,…”
Background
Metadata
- Authors
- Cobbe, J., Lee, M. S. A., & Singh, J.
- Affiliations
- University of Cambridge (GB)
- Venue
- Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency
- Publisher
- ACM
- Published
- 2021-03
- Pages
- 598-609
- OpenAlex
- W3134395196
- Semantic Scholar
- 1b94ddf67a6e0bef90ef83fad8301a3d71c41096
- Cited by
- 150
- References
- 52
- Citations by year
- 2026: 63 · 2025: 29 · 2024: 15 · 2023: 13 · 2022: 18 · 2021: 11
- FWCI
- 12.86
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion, cc-by)
- Funders
- Engineering and Physical Sciences Research Council (EP/P024394/1, EP/R033501/1)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Law › Digitalization, Law, and Regulation
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Keywords
- reviewability, automated decision-making, algorithmic decision-making, accountability, machine learning, administrative law, regulatory oversight, legal compliance, record keeping, socio-technical processes, administrative review
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-09-29, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-09-29 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.