The DOI matches its Crossref record; 60 of the 60 references it cites were found in a registry or an index; 33 of the 53 sources its claims cite were checked for support.
Reviewable Automated Decision-Making: A Framework for Accountable Algorithmic Systems
Open accessAbstract
This paper introduces reviewability as a framework for improving the accountability of automated and algorithmic decisionmaking (ADM) involving machine learning. We draw on an understanding of ADM as a socio-technical process involving both human and technical elements, beginning before a decision is made and extending beyond the decision itself. While explanations and other model-centric mechanisms may assist some accountability concerns, they often provide insufficient information of these broader ADM processes for regulatory oversight and assessments of legal compliance. Reviewability involves breaking down the ADM process into technical and organisational elements to provide a systematic framework for determining the contextually appropriate record-keeping mechanisms to facilitate meaningful review - both of individual decisions and of the process as a whole. We argue that a reviewability framework, drawing on administrative law's approach to reviewing human decision-making, offers a practical way forward towards more a more holistic and legally-relevant form of accountability for ADM.
Overview
CoChat Summary
TL;DR
This paper introduces “reviewability” as a framework for improving accountability of automated and algorithmic decision-making (ADM) involving machine learning by breaking ADM into technical and organisational stages and specifying contextually appropriate record-keeping and logging. It argues that model-centric explanations are insufficient for regulatory oversight and legal compliance, and that a holistic, process-based approach drawing on English administrative law can better support meaningful accountability.
Key findings
- Model-focused transparency and explanation mechanisms are insufficient because ADM is a socio-technical process with human and organisational elements; problems often originate outside the model, in purposes, assumptions, design, and use.
- Reviewability requires contextually appropriate information that is (i) relevant to the accountability relationships involved, (ii) accurate, complete, and representative, (iii) proportionate to the required transparency, and (iv) comprehensible by the relevant forums.
- The paper proposes a framework with four stages: Commissioning (procurement, problem definition, impact assessment), Model building (data collection, pre-processing, model training, model testing), Decision-making (deployment, use, consequences), and Investigation (audit, disclosure).
- It draws on accountability as involving an actor, a forum, an account, and consequences, and cites five kinds of accountability relationship: political, legal, administrative, professional, and social. It also identifies three generic actor roles: managers, developers, and users.
Methods
- Conceptual and legal-theoretical framework development, drawing on English administrative law, accountability literature, and prior work applying administrative law to ADM.
- No empirical sample or quantitative analysis is reported; the paper synthesises related research and proposes a systematic framework for reviewable ADM.
- Main analysis breaks the ADM process into stages and steps, then identifies record-keeping and logging mechanisms that could provide contextually appropriate information for review of individual decisions and the process as a whole.
Limitations
- The paper states that reviewability cannot on its own address all potential problems with ADM; wider examination of political economy and socio-economic context, as well as legal protections for individuals and groups, is also needed.
- It states that transparency has limits: reviewable systems may not capture broader business processes, wider context, interactions with other socio-technical processes, business models, or questions of power; transparency also has temporal limitations and is open to subjective interpretation and contestation.
- It states that implementation research is needed on what record-keeping and logging are appropriate at each step, what information to retain, and how best to present it to forums; recording everything can be impractical and can create a “transparency paradox.”
- Evident from the text: the framework is conceptual and normative, with no empirical evaluation or deployment study reported.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
23 Valid · 10 Not valid · 12 Not checked · 78 Own results and interpretations
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Administrative law as a framework and judicial review as an oversight mechanism understand decision-making as a process beginning before decisions are made and extending to their consequences and effects.”
§ Reviewable Automated Decision-Making: A Framework for Accountable Algorithmic Systems > 1 INTRODUCTION
What the sources say
- Supportedcites [13] 13. Jennifer Cobbe. 2019. Administrative Law and the Machines of Government: Judicial Review of Automated Public-Sector Decision-Making. Legal Studies (2019). source
The Review of ADM (p. 9) says judicial review of an individual decision may need to examine earlier choices about training data, system design and testing. The discussion of reasons (p. 22) says administrative-law requirements depend on how serious and consequential a decision is. Together, these support the claim that oversight considers both what precedes a decision and its effects. (LLM, conf=0.86)
- Not checked — no full text of the sourcecites [14] 14. Jennifer Cobbe, Michelle Seng Ah Lee, Heleen Janssen, and Jatinder Singh. 2020. Centring the Rule of Law in the Digital State. IEEE Computer (2020). source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Models are only one aspect of ADM and cannot by themselves provide accounts of the process as a whole [66], whether to technical, legal, or other forums.”
§ Reviewable Automated Decision-Making: A Framework for Accountable Algorithmic Systems > 2.2.1 Meaningful accountability needs more than models.
What the sources say
- Supportedcites [66] 66. Jatinder Singh, Jennifer Cobbe, and Chris Norval. 2020. Decision Provenance: Harnessing Data Flow for Accountable Systems. IEEE Access 7 (2020). source
In “V. Case Study: Machine Learning Pipelines” (page unavailable), the paper describes models as components of interconnected systems and says accountability requires information about data flows, model use, and the downstream effects of decisions. “B. Technical Assistance” (page unavailable) connects system-wide transparency to both technical oversight and legal accountability. This supports the claim that a model alone cannot account for the whole decision-making process. (LLM, conf=0.88)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Rather, people are concerned with broader aspects of ADM: the purposes, roles, and outcomes of decision-making processes as a whole, as it is the whole process that affects them rather than simply certain technical aspects of it [26].”
§ Reviewable Automated Decision-Making: A Framework for Accountable Algorithmic Systems > 2.2.1 Meaningful accountability needs more than models.
What the sources say
- Partly supportedcites [26] 26. Lilian Edwards and Michael Veale. 2017. Slave to the Algorithm? Why a 'Right to an Explanation' is Probably Not the Remedy You Are Looking For. Duke Law & Technology Review 17 (2017). sourceExperimental — accuracy not yet measured
The Introduction (p. 2) says the public is troubled by systems that govern their lives and produce damaging or unfair results. The Conclusion (p. 64) favors broader algorithmic governance over explanations of a black box’s interior. These passages support the contrast with purely technical explanations, but they do not establish that people are specifically concerned with the purposes and roles of decision-making processes as a whole. (LLM, conf=0.78)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: unclear — It is unclear whether it could be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“However, understanding ADM as a sociotechnical process, we argue that there is another form of opacityunwitting opacity, where those responsible for designing, developing, deploying, and using systems simply don't think to record relevant organisational aspects of ADM processes (perhaps unaware of their relevance for meaningful accountability [12]).”
§ Reviewable Automated Decision-Making: A Framework for Accountable Algorithmic Systems > 2.1 Accountability in ADM
What the sources say
- Partly supportedcites [12] 12. Richard Cloete, Chris Norval, and Jatinder Singh. 2020. A Call for Auditable Virtual, Augmented and Mixed Reality. sourceExperimental — accuracy not yet measured
Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Though explanations can assist some specific concerns, like model engineering [7], explanations or other approaches focused on how the model itself works or has reached a particular outcome may miss much of what is important [75].”
§ Reviewable Automated Decision-Making: A Framework for Accountable Algorithmic Systems > 2.2 Limitations of model-focused mechanisms
What the sources say
- Supportedcites [7] 7. Umang Bhatt, Alice Xiang, Shubham Sharma, Adrian Weller, Ankur Taly, Yun- han Jia, Joydeep Ghosh, Ruchir Puri, José M. F. Moura, and Peter Eckersley. 2020. Explainable Machine Learning in Deployment. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (Barcelona, Spain) (FAT* '20). Association for Computing Machinery, New York, NY, USA, 648?657. https://doi.org/10.1145/3351095.3375624 sourceExperimental — accuracy not yet measured
The paper’s §3.4 Key Takeaways says that a description of how a model responds to an input change may not explain anything useful to stakeholders or reveal the underlying causal mechanism. Section 5.1 Establish Clear Desiderata likewise emphasizes identifying stakeholders’ needs and the purpose of an explanation. Together, these support the hedged claim that focusing on model behavior or a particular prediction can miss important concerns. (LLM, conf=0.86)
- Supportedcites [75] 75. Maranke Wieringa. 2020. What to account for when accounting for algorithms: a systematic literature review on algorithmic accountability. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 1-18. sourceExperimental — accuracy not yet measured
Section 3.2 says that understanding a system’s technical workings can be useful but is insufficient for accountability because it does not explain why the system was adopted or who set its specifications. Section 3.4 similarly argues that explanations focused on technical aspects can miss the system’s social context, development, and effects. (LLM, conf=0.94)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: unclear — It is unclear whether it could be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
24 more claims · 8 with their cited sources checked
Create a free account to see all 123 claims and every source verdict.
- Probably not a substantial claim. 87
- It may state two separate claims as one. 24
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Accountability has five types: legal responsibility to judicial bodies; political accountability to elected representatives and similar entities; administrative accountability to auditors and regulatory agencies; professional responsibility to internal and external colleagues; and social responsibility to community members and the public (Cobbe et al., 2021).”
From Reconciling Administrative Efficiency with Procedural Safeguards in the ADM System (2025) · cites this paper as Cobbe et al., 2021
In the citing paper
“Transparency which supports broader accountability processes on training sets, models and internal processes behind these services, as well as in the widersense of acknowledgement of known-limitations, past failures, and lessons learnt [27,131] can help move AIaaS from the opaque, unknown and potentially harmful 'black-boxes' of today, to a more understandable, accepting of their own limitations,”
From Out of Context: Investigating the Bias and Fairness Concerns of “Artificial Intelligence as a Service” (2023) · cites this paper as [27]
Citations
Cites
Arrieta, A. B., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., García, S., Gil-López, S., Molina, D., Benjamins, R., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.10045 · Cited by 11,321
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423 · cited 2×
“…of the research landscape show a vast number of mechanisms for developing ‘interpretable’ or ‘explainable’ ML systems have been proposed, seeking to provide explanations and other interpretable accounts of model behaviour to a range of forums, many of which have come in the last few years [4, 32].”
BackgroundThe Black Box Society: The Secret Algorithms That Control Money and Information
Pasquale, F. (2015) · doi:10.4159/harvard.9780674736061 · Cited by 3,796
A few useful things to know about machine learning
Domingos, P. (2012) · Communications of the ACM · doi:10.1145/2347736.2347755 · Cited by 3,326
How the machine “thinks”: Understanding opacity in machine learning algorithms
Burrell, J. (2016) · Big Data & Society · doi:10.1177/2053951715622512 · Cited by 3,196 · cited 5×
“Burrell identifies three forms of algorithmic opacity [10]; while Burrell refers primarily to model opacity, these also relate to broader ADM processes.”
BackgroundMethodInfluentialModel Cards for Model Reporting
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287596 · Cited by 2,681
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Daumé, H., & Crawford, K. (2018) · arXiv · doi:10.48550/arxiv.1803.09010 · Cited by 2,474
Raji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372873 · Cited by 2,245 · cited 1×
“Even where assessments are not legally required, they are good practice, and robust internal (in-organisation) assessment processes have value [60].”
BackgroundEuropean Union Regulations on Algorithmic Decision Making and a “Right to Explanation”
Goodman, B., & Flaxman, S. (2017) · AI Magazine · doi:10.1609/aimag.v38i3.2741 · Cited by 2,131 · cited 1×
“Some proposals have seemingly been prompted by academic debate about the existence, nature, extent, and utility of a so-called ‘right to an explanation’ [26, 31, 44, 64, 74] in GDPR, which was passed in 2016.”
BackgroundAnanny, M., & Crawford, K. (2018) · New Media & Society · doi:10.1177/1461444816676645 · Cited by 1,836 · cited 8×
“There are limits to transparency itself as a mechanism for supporting accountability [2, 26, 68].”
BackgroundInfluential
Research works printed but not linked: 14
- Philip Alston. 2019. Report of the Special Rapporteur on extreme poverty and human rights. United Nations General Assembly A/74/493 (2019). https://undocs.org/A/74/493
- Mark Bovens. 2006. Analysing and Assessing Public Accountability. A Conceptual Framework. EUROGOV, European Governance Papers No. C-06-01 (2006).
- Cary Coglianese and Daniel Lehr. 2017. Regulating by Robot: Administrative Decision Making in the Machine-Learning Era. Georgetown Law Journal 105 (2017).
- European Commission. 2020. White Paper On Artificial Intelligence - A European Approach to Excellence and Trust. COM(2020) 65 (2020).
- 10.1145/2347736 · doi:10.1145/2347736
- Avi Feller, Emma Pierson, Sam Corbett-Davies, and Sharad Goel. 2016. A computer program used for bail and sentencing decisions was labeled biased against blacks. It's actually not that clear. The Washington Post (2016).
- Agata Foryciarz, Daniel Leufer, and Katarzyna Szymielewicz. 2020. Black-Boxed Politics: Opacity is a Choice in AI Systems. Panoptykon Foundation (17 January 2020). https://en.panoptykon.org/articles/black-boxed-politics-opacity-choice-ai-systems
- 10.1093/idpl · doi:10.1093/idpl
- David Lehr and Paul Ohm. 2017. Playing with the Data: What Legal Scholars Should Learn About Machine Learning. UC Davis Law Review 51 (2017).
- House of Common Science and Technology Committee. 2018. Algorithms in Decision-Making. Fourth Report of Session 2017-19 HC 351 (2018).
- Douglas Pyper. 2020. Research Briefing: The Public Sector Equality Duty and Equality Impact Assessments. House of Commons Library (2020). https://commonslibrary.parliament.uk/research-briefings/sn06591/
- Nick Seaver. 2013. Knowing Algorithms. paper presented at Media in Transition 8 (2013).
- Cynthia Stohl, Michael Stohl, and Paul M Leonardi. 2016. Managing Opacity: Information Visibility and the Paradox of Transparency in the Digital Age. International Journal of Communication 10 (2016).
- European Union. 2016. Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation). Official Journal of the European Union L119 (4 May 2016), 1--88.
Cited by
Auditing large language models: a three-layered approach
Mökander, J., Schuett, J., Kirk, H. R., & Floridi, L. (2024) · AI and Ethics · doi:10.1007/s43681-023-00289-2 · Cited by 289
“Three ideas underpin the promise of auditing as an AI governance mechanism: that procedural regularity and transparency contribute to good governance [8], [9]; that proactivity in the design of AI systems helps identify risks and prevent harm before it occurs [10], [11]; and, that the independence between the auditor…”
BackgroundA literature review on applications of explainable artificial intelligence (XAI)
Kalasampath, K., Spoorthi, K. N., Sajeev, S., Kuppa, S. S., Ajay, K., & Maruthamuthu, A. (2025) · IEEE Access · doi:10.1109/access.2025.3546681 · Cited by 180
“By comprehending the capabilities and restrictions of each kind of explanation, developers, and stakeholders can develop AI systems that are more efficient and reliable, thus advancing ethical and responsible AI practices [114].”
Auditing of AI: Legal, Ethical and Technical Approaches
Mökander, J. (2023) · Digital Society · doi:10.1007/s44206-023-00074-y · Cited by 152
“Reflecting that convergence, previous research has developed procedures to audit AI systems for transparency and explainability (Cobbe et al., 2021; Mittelstadt, 2016), bias and fairness (Bartley et al., 2021; Morina et al., 2019) and accountability (Busuioc, 2021; Metcalf et al., 2021).”
MethodAI and Global Governance: Modalities, Rationales, Tensions
Veale, M., Matus, K., & Gorwa, R. (2023) · Annual Review of Law and Social Science · doi:10.1146/annurev-lawsocsci-020223-040749 · Cited by 218
“Issues of accountability can be separated from neither the much longer sociotechnical process (Cobbe et al. 2021) nor the need for an active, critical audience (Kemper & Kolkman 2018).”
BackgroundMethodOperationalising AI Governance through Ethics-Based Auditing: An Industry Case Study
Mökander, J., & Floridi, L. (2022) · AI and Ethics · doi:10.1007/s43681-022-00171-7 · Cited by 239
“National regulators like the UK Information Commissioner’s Office have provided guidance on how to audit AI systems [18], and professional services firms like PwC, EY, Deloitte and KPMG have all developed auditing (or ‘assurance’) procedures to help clients ensure that the AI systems they design and deploy are legal,…”
Background
Metadata
- Authors
- Cobbe, J., Lee, M. S. A., & Singh, J.
- Affiliations
- University of Cambridge (GB)
- Venue
- Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency
- Publisher
- ACM
- Published
- 2021-03
- Pages
- 598-609
- OpenAlex
- W3134395196
- Semantic Scholar
- 1b94ddf67a6e0bef90ef83fad8301a3d71c41096
- Cited by
- 150
- References
- 52
- Citations by year
- 2026: 63 · 2025: 29 · 2024: 15 · 2023: 13 · 2022: 18 · 2021: 11
- FWCI
- 12.86
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion, cc-by)
- Funders
- Engineering and Physical Sciences Research Council (EP/P024394/1, EP/R033501/1)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Law › Digitalization, Law, and Regulation
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Keywords
- reviewability, automated decision-making, algorithmic decision-making, accountability, machine learning, administrative law, regulatory oversight, legal compliance, record keeping, socio-technical processes, administrative review
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-09-29, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-09-29 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Reviewable Automated Decision-Making: A Framework for Accountable Algorithmic Systems. https://app.cochat.ai/papers/10.1145/3442188.3445921
- The paper
- Cobbe, J., Lee, M. S. A., & Singh, J. (2021). Reviewable Automated Decision-Making: A Framework for Accountable Algorithmic Systems. Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency. https://doi.org/10.1145/3442188.3445921
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.