Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 23 of the 30 references it cites were found in a registry or an index; none of the 51 sources its claims cite is checked for support yet.

Proceduralizing control and discretion: Human oversight in artificial intelligence policy

Open access
Koulu, Riikka · 2020 · Maastricht Journal of European and Comparative Law

Abstract

This article is an examination of human oversight in EU policy for controlling algorithmic systems in automated legal decision making. Despite the shortcomings of human control over complex technical systems, human oversight is advocated as a solution against the risks of increasing reliance on algorithmic tools. For law, human oversight provides an attractive, easily implementable and observable procedural safeguard. However, without awareness of its inherent limitations, human oversight is in danger of becoming a value in itself, an empty procedural shell used as a stand-in justification for algorithmization but failing to provide protection for fundamental rights. By complementing socio-legal analysis with Science and Technology Studies, critical algorithm studies, organization studies and human-computer interaction research, the author explores the importance of keeping the human in the loop and asks what the human element at the core of legal decision making is. Through algorithmization it is made visible how law conceptualises decision making through human actors, personalises legal decision making through the decision-maker’s discretionary power that provides proportionality and common sense, prevents gross miscarriages of justice and establishes the human encounter deemed essential for the feeling of being heard. The analysis demonstrates the necessary human element embedded in legal decision making, against which the meaningfulness of human oversight needs to be examined.

Overview

CoChat Summary

TL;DR

The article argues that human oversight is promoted in EU AI policy as an easily implementable procedural safeguard for algorithmic legal decision making, but it risks becoming an empty procedural shell or rubberstamping unless tied to an understanding of legal decision making. It shows that algorithmisation makes visible law’s reliance on human discretion and encounter, while also diminishing or displacing discretion upstream.

Key findings

  • EU policy frames human oversight as a key requirement for trustworthy AI: the 2019 Commission Communication listed human agency and oversight as the first of seven key requirements, and the February 2020 digital package included four documents echoing “trustworthy” and “human-centric” AI. The White Paper on AI proposes four possible forms of human oversight: review/validate before the decision becomes effective, review after it becomes effective, monitoring and intervention during operation, and design-based operational constraints.
  • GDPR Article 22 and WP29 guidelines set the stage for human oversight, but WP29 emphasises that fabricated human involvement is insufficient: the overseer must have the authority and competence to change the decision.
  • Despite its appeal, human oversight has practical limits: humans generally perform badly as supervisors of automated systems due to boredom, automation bias and alert fatigue; Elish’s “moral crumple zone” describes misattributed responsibility to a human with limited control. Human oversight may become a value in itself, turning decision-makers into rubberstamps.
  • Algorithmisation reveals the human element in legal decision making: discretion as both freedom and control, and the human encounter behind the feeling of being heard. The paper makes three observations about context-dependency, including that decision making is moved upstream to system architects and away from street-level bureaucrats, contributing to loss of human discretion.

Methods

  • Conceptual socio-legal analysis, complemented with Science and Technology Studies, critical algorithm studies, organisation studies and human-computer interaction research.
  • No empirical sample or primary data collection is reported; the analysis draws on EU regulation and policy documents (GDPR, WP29 guidelines, 2019 Communication, 2020 digital package, White Paper on AI, CEPEJ ethical guidelines) and secondary literature.
  • The argument proceeds in four steps: contextualising algorithmisation of legal decision making; describing EU regulation and emerging AI policy; contesting the feasibility of human oversight; and locating the human element in the decision-maker’s discretionary power.

Limitations

  • The paper states that more research, both theoretical and empirical, is needed to understand how the tension between the personal dimension and organisational context of legal decision making plays out with algorithmisation.
  • It states that the EU’s emerging AI policy provides little guidance on what constitutes “meaningful” human oversight, and that its ex ante emphasis marginalises ex post mechanisms.
  • Evident from the text: the analysis is conceptual and relies on secondary literature and policy documents rather than empirical data; the paper also notes that context-dependency and complexity make a comprehensive overview of long-term implications difficult.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1177/1023263x20978649

Cites

23 of 23 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

0 Valid · 2 Not valid · 52 Not checked · 73 Own results and interpretations

22 more claims

Create a free account to see all 127 claims and every source verdict.

See all claims — free account
  • Probably not a substantial claim. 86
  • It may state two separate claims as one. 46

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

2 of 2

Citations

Cites

Show all 30 references

Cited by

See all 72

Metadata

Authors
Koulu, R.
Affiliations
University of Helsinki (FI)
Corresponding author
Koulu, Riikka
Venue
Maastricht Journal of European and Comparative Law
Published
2020-12
Volume
27
Issue
6
Pages
720-735
ISSN
  • 1023-263X (ISSN-L)
  • 2399-5548
OpenAlex
W3109629996
Cited by
72
References
30
Citations by year
2026: 22 · 2025: 19 · 2024: 17 · 2023: 7 · 2022: 4 · 2021: 3
FWCI
12.08
Type
journal article
Language
en
Open access
Locations
  • Maastricht Journal of European and Comparative Law (publishedVersion, cc-by)
  • Työväentutkimus Vuosikirja (submittedVersion)
Topics
  • Social Sciences › Law › Artificial Intelligence in Law
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Social Sciences › Law › Digitalization, Law, and Regulation
Keywords
human oversight, algorithmic decision-making, algorithmic accountability, discretionary power, proportionality, fundamental rights, algorithmization, human-computer interaction, legal decision-making, science and technology studies, critical algorithm studies, miscarriage of justice

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Proceduralizing control and discretion: Human oversight in artificial intelligence policy. https://app.cochat.ai/papers/10.1177/1023263x20978649
The paper
Koulu, R. (2020). Proceduralizing control and discretion: Human oversight in artificial intelligence policy. Maastricht Journal of European and Comparative Law. https://doi.org/10.1177/1023263x20978649

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.