The DOI matches its Crossref record; 23 of the 30 references it cites were found in a registry or an index; none of the 51 sources its claims cite is checked for support yet.
Proceduralizing control and discretion: Human oversight in artificial intelligence policy
Open accessAbstract
This article is an examination of human oversight in EU policy for controlling algorithmic systems in automated legal decision making. Despite the shortcomings of human control over complex technical systems, human oversight is advocated as a solution against the risks of increasing reliance on algorithmic tools. For law, human oversight provides an attractive, easily implementable and observable procedural safeguard. However, without awareness of its inherent limitations, human oversight is in danger of becoming a value in itself, an empty procedural shell used as a stand-in justification for algorithmization but failing to provide protection for fundamental rights. By complementing socio-legal analysis with Science and Technology Studies, critical algorithm studies, organization studies and human-computer interaction research, the author explores the importance of keeping the human in the loop and asks what the human element at the core of legal decision making is. Through algorithmization it is made visible how law conceptualises decision making through human actors, personalises legal decision making through the decision-maker’s discretionary power that provides proportionality and common sense, prevents gross miscarriages of justice and establishes the human encounter deemed essential for the feeling of being heard. The analysis demonstrates the necessary human element embedded in legal decision making, against which the meaningfulness of human oversight needs to be examined.
Overview
CoChat Summary
TL;DR
The article argues that human oversight is promoted in EU AI policy as an easily implementable procedural safeguard for algorithmic legal decision making, but it risks becoming an empty procedural shell or rubberstamping unless tied to an understanding of legal decision making. It shows that algorithmisation makes visible law’s reliance on human discretion and encounter, while also diminishing or displacing discretion upstream.
Key findings
- EU policy frames human oversight as a key requirement for trustworthy AI: the 2019 Commission Communication listed human agency and oversight as the first of seven key requirements, and the February 2020 digital package included four documents echoing “trustworthy” and “human-centric” AI. The White Paper on AI proposes four possible forms of human oversight: review/validate before the decision becomes effective, review after it becomes effective, monitoring and intervention during operation, and design-based operational constraints.
- GDPR Article 22 and WP29 guidelines set the stage for human oversight, but WP29 emphasises that fabricated human involvement is insufficient: the overseer must have the authority and competence to change the decision.
- Despite its appeal, human oversight has practical limits: humans generally perform badly as supervisors of automated systems due to boredom, automation bias and alert fatigue; Elish’s “moral crumple zone” describes misattributed responsibility to a human with limited control. Human oversight may become a value in itself, turning decision-makers into rubberstamps.
- Algorithmisation reveals the human element in legal decision making: discretion as both freedom and control, and the human encounter behind the feeling of being heard. The paper makes three observations about context-dependency, including that decision making is moved upstream to system architects and away from street-level bureaucrats, contributing to loss of human discretion.
Methods
- Conceptual socio-legal analysis, complemented with Science and Technology Studies, critical algorithm studies, organisation studies and human-computer interaction research.
- No empirical sample or primary data collection is reported; the analysis draws on EU regulation and policy documents (GDPR, WP29 guidelines, 2019 Communication, 2020 digital package, White Paper on AI, CEPEJ ethical guidelines) and secondary literature.
- The argument proceeds in four steps: contextualising algorithmisation of legal decision making; describing EU regulation and emerging AI policy; contesting the feasibility of human oversight; and locating the human element in the decision-maker’s discretionary power.
Limitations
- The paper states that more research, both theoretical and empirical, is needed to understand how the tension between the personal dimension and organisational context of legal decision making plays out with algorithmisation.
- It states that the EU’s emerging AI policy provides little guidance on what constitutes “meaningful” human oversight, and that its ex ante emphasis marginalises ex post mechanisms.
- Evident from the text: the analysis is conceptual and relies on secondary literature and policy documents rather than empirical data; the paper also notes that context-dependency and complexity make a comprehensive overview of long-term implications difficult.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 Valid · 2 Not valid · 52 Not checked · 73 Own results and interpretations
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“I argue that algorithmisation challenges existing conceptualisations of legal decision making by making visible the inherently human face of procedural justice, articulated both through the procedural norms regulating decision-makers' use of discretion and the procedural experiences of those seeking justice.”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: unclear — It is unclear whether it could be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Ultimately, law still perceives the human decision-maker as constitutive for fair decision making.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
None of the sources cited for this claim could be matched to a paper in Crossref, so there is nothing to check it against.
In the paper
“there is a widespread consensus regarding the need to reform dispute resolution mechanisms, shared both by access to justice scholars and socio-legal algorithm studies.”
What the sources say
- Not matchedcites J. Resnik
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites J. Cohen, Between Truth and Power
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This approach might clash with the ex post mechanisms, based on which legal protection typically is produced, although the topic deserves more thorough analysis.”
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The EU's emerging AI policy seems to focus on ex ante assessment, at the expense of marginalising the elaboration of ex post mechanisms.”
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
22 more claims
Create a free account to see all 127 claims and every source verdict.
- Probably not a substantial claim. 86
- It may state two separate claims as one. 46
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Koulu warns that human oversight may become a procedural shell where law preserves the appearance of control without ensuring its practical conditions (Koulu, 2020).”
From Human-centric AI governance in the European Union. Accountability, fundamental rights, and institutional resilience in public administration (2026) · cites this paper as Koulu, 2020
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Human oversight in EU regulation and policy
“'Guidelines on Automated individual decision making and Profiling for the purposes of Regulation 2106/679, wp251rev.01'”
§ Human oversight in EU regulation and policy
“Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation)”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Human oversight in EU regulation and policy
“the EU's General Data Protection Regulation (GDPR, 679/2016)”
§ Human oversight in EU regulation and policy
“Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation)”
Citations
Cites
The ethics of algorithms: Mapping the debate
Mittelstadt, B. D., Allo, P., Taddeo, M., Wachter, S., & Floridi, L. (2016) · Big Data & Society · doi:10.1177/2053951716679679 · Cited by 4,567
Sociomaterial practices: exploring technology at work
Orlikowski, W. J. (2007) · Organization Studies · doi:10.1177/0170840607081138 · Cited by 3,248
Normal Accidents: Living with High-Risk Technologies
Cummings, L. L., & Perrow, C. (1984) · Administrative Science Quarterly · doi:10.2307/2392945 · Cited by 2,861
The Ethics of AI Ethics: An Evaluation of Guidelines
Hagendorff, T. (2020) · Minds and Machines · doi:10.1007/s11023-020-09517-8 · Cited by 1,911
The Ethics of AI Ethics -- An Evaluation of Guidelines
Hagendorff, T. (2019) · arXiv · doi:10.48550/arxiv.1903.03425 · Cited by 1,907
Gillespie, T. (2014) · Media Technologies · doi:10.7551/mitpress/9042.003.0013 · Cited by 1,016
Technologies of Humility: Citizen Participation in Governing Science
Jasanoff, S. (2003) · Minerva · doi:10.1023/a:1025557512320 · Cited by 1,004
Greene, D., Hoffmann, A. L., & Stark, L. (2019) · Proceedings of the Annual Hawaii International Conference on System Sciences · doi:10.24251/hicss.2019.258 · Cited by 386
Moral crumple zones: cautionary tales in human–robot interaction
Elish, M. C. (2019) · Engaging Science, Technology, and Society · doi:10.17351/ests2019.260 · Cited by 321 · cited 1×
“41 For various reasons, from boredom at routine monitoring to automation bias and alert fatigue, humans generally perform badly as supervisors of automated technical systems.”
BackgroundGlobal Labor: Algocratic Modes of Organization
Aneesh, A. (2009) · Sociological Theory · doi:10.1111/j.1467-9558.2009.01352.x · Cited by 258
Cited by
Lekadir, K., Frangi, A. F., Porras, A. R., Glocker, B., Cintas, C., Langlotz, C. P., Weicken, E., Asselbergs, F. W., Prior, F., Collins, G. S., et al. (2025) · BMJ · doi:10.1136/bmj-2024-081554 · Cited by 957
On the Quest for Effectiveness in Human Oversight: Interdisciplinary Perspectives
Sterz, S., Baum, K., Biewer, S., Hermanns, H., Lauber-Rönsberg, A., Meinel, P., & Langer, M. (2024) · The 2024 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3630106.3659051 · Cited by 91
Towards Risk‐Free Trustworthy Artificial Intelligence: Significance and Requirements
Alzubaidi, L., Al-Sabaawi, A., Bai, J., Dukhan, A., Alkenani, A. H., Al‐Asadi, A., Alwzwazy, H. A., Manoufali, M., Fadhel, M. A., Albahri, A. S., et al. (2023) · International Journal of Intelligent Systems · doi:10.1155/2023/4459198 · Cited by 145
‘Human oversight’ in the EU artificial intelligence act: what, when and by whom?
Enqvist, L. (2023) · Law, Innovation and Technology · doi:10.1080/17579961.2023.2245683 · Cited by 142
Automation bias in public administration – an interdisciplinary perspective from law and psychology
Ruschemeier, H., & Hondrich, L. J. (2024) · Government Information Quarterly · doi:10.1016/j.giq.2024.101953 · Cited by 59
Metadata
- Authors
- Koulu, R.
- Affiliations
- University of Helsinki (FI)
- Corresponding author
- Koulu, Riikka
- ORCID
- Koulu, Riikka
- Venue
- Maastricht Journal of European and Comparative Law
- Publisher
- SAGE Publications
- Published
- 2020-12
- Volume
- 27
- Issue
- 6
- Pages
- 720-735
- ISSN
- 1023-263X (ISSN-L)
- 2399-5548
- OpenAlex
- W3109629996
- Cited by
- 72
- References
- 30
- Citations by year
- 2026: 22 · 2025: 19 · 2024: 17 · 2023: 7 · 2022: 4 · 2021: 3
- FWCI
- 12.08
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Maastricht Journal of European and Comparative Law (publishedVersion, cc-by)
- Työväentutkimus Vuosikirja (submittedVersion)
- Topics
- Social Sciences › Law › Artificial Intelligence in Law
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Law › Digitalization, Law, and Regulation
- Keywords
- human oversight, algorithmic decision-making, algorithmic accountability, discretionary power, proportionality, fundamental rights, algorithmization, human-computer interaction, legal decision-making, science and technology studies, critical algorithm studies, miscarriage of justice
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Proceduralizing control and discretion: Human oversight in artificial intelligence policy. https://app.cochat.ai/papers/10.1177/1023263x20978649
- The paper
- Koulu, R. (2020). Proceduralizing control and discretion: Human oversight in artificial intelligence policy. Maastricht Journal of European and Comparative Law. https://doi.org/10.1177/1023263x20978649
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.