From Human Intervention to Human Involvement: A Critical Examination of the Role of Humans in (Semi-)Automated Administrative Decision-Making
Open accessThe DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index.
Abstract
In view of recent scandals, evolving regulatory demands, and changing perceptions of human involvement in automated systems, this study examines how the role of humans in semi-automated administrative decision-making (AADM) is conceptualized in academic literature. We conducted a qualitative systematic literature review from the public administration, law and socio-technological fields, to critically examine what is understood as human involvement in AADM. Our findings suggest that the literature largely focuses on human intervention in AADM, where individual decision-makers either review automated outputs before deciding (human-in-the-loop) or review automated decisions (human-out-of-the-loop). However, a broader scope on human involvement in AADM—which also includes system designers (human-in-control) and monitoring and supervisory roles responsible for ongoing oversight (human-on-the-loop)—is crucial to fully capture the human-machine dynamic in AADM. These insights can be used to advance the understanding of the human-automation interaction in AADM through a more comprehensive conceptualization of human involvement. This broader, governance-oriented, view has the potential to optimize the interrelationship between human involvement and automation, ultimately improving accountability, effectiveness, and adaptability in AADM. New legislation underscores the importance of meaningful human involvement beyond isolated review points, encompassing design and ongoing oversight.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
The Coding Manual for Qualitative Researchers
Lungu, M. (2022) · American Journal of Qualitative Research · doi:10.29333/ajqr/12085 · Cited by 10,826 · cited 1×
“To be able to develop a greater understanding of human involvement, in Section 3.1 , we first explore where the author’s position the human in the automated decision-making process (RQ1).”
InfluentialCampbell, F., Tricco, A. C., Munn, Z., Pollock, D., Saran, A., Sutton, A., White, H., & Khalil, H. (2023) · Systematic Reviews · doi:10.1186/s13643-023-02178-5 · Cited by 345
Disparate Interactions: An Algorithm-in-the-Loop Analysis of Fairness in Risk Assessments
Green, B., & Chen, Y. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287563 · Cited by 275 · cited 1×
“Concerns for arbitrary use of discretion can be tackled by developing internal accountability structures, which ensures freedom to decision-making that is “not random or arbitrary but instead creates space for contextual consideration, understanding for the exceptional circumstances unique to that…”
The flaws of policies requiring human oversight of government algorithms
Green, B. (2022) · Computer Law & Security Review · doi:10.1016/j.clsr.2022.105681 · Cited by 263 · cited 62×
“This is problematic, considering human decision-makers can be a source of inconsistency, bias, and discrimination, potentially influenced consciously and unconsciously by individual experiences [Laux 2023 ; Green 2022 ].”
InfluentialThe Flaws of Policies Requiring Human Oversight of Government Algorithms
Green, B. (2021) · doi:10.2139/ssrn.3921216 · Cited by 259
(2024) · doi:10.1093/oso · Cited by 138
Liable, but not in control? Ensuring meaningful human agency in automated decision-making systems
Wagner, B. (2019) · Policy & Internet · doi:10.1002/poi3.198 · Cited by 136 · cited 22×
“Partly this is found in better equipping individual decision-makers, by training human operators, providing them with better information, guidance, and more time [Wagner 2019 ; Green 2022 ; Lazcos & de Hert 2023 ; Laux 2023 ].”
InfluentialHuman Judgment in algorithmic loops: Individual justice and automated decision‐making
Binns, R. (2022) · Regulation & Governance · doi:10.1111/rego.12358 · Cited by 134 · cited 9×
“To come to a tailored judgment under complex circumstances, human discretion is needed.”
InfluentialLazcoz, G., & de Hert, P. (2023) · Computer Law & Security Review · doi:10.1016/j.clsr.2023.105833 · Cited by 93 · cited 4×
“One can ask for an ex-post assessment and for additional information to be taken into account, which bolsters the protection of material and procedural rights and the control over automated systems [Koulu 2020b ; also Lazcoz & de Hert 2023 ].”
InfluentialProceduralizing control and discretion: Human oversight in artificial intelligence policy
Koulu, R. (2020) · Maastricht Journal of European and Comparative Law · doi:10.1177/1023263x20978649 · Cited by 72 · cited 7×
“To come to a tailored judgment under complex circumstances, human discretion is needed.”
Influential
Research works printed but not linked: 4
- 10.1145/10.1145/3287560.3287563 · doi:10.1145/10.1145/3287560.3287563
- 10.1007/s00146-023-01777 · doi:10.1007/s00146-023-01777
- D. Stokmans and S. Moeliker. 2024. Het UWV wil “menselijker” worden, maar bij de afdeling Handhaving tellen vooral de targets. NRC. Retrieved from https://www.nrc.nl/nieuws/2024/04/19/bij-handhaving-blijft-de-menselijke-maat-zoek-a4196601?t=1717060846.
- 10.17176/20230512-181843-0 · doi:10.17176/20230512-181843-0
Cited by
AI regulatory strategies: a global public health perspective
Léger-Riopel, N. (2025) · Journal of Public Health · doi:10.1093/pubmed/fdaf116 · Cited by 3
From cause to consequence: ethical considerations and the causal governance maturity model
Edokobi, O. (2026) · AI and Ethics · doi:10.1007/s43681-026-00992-w
“…decision‑making and the logic and signifi ‑ cance of processing, offered accessible channels to contest outcomes and request meaningful human review, and, for high‑impact decisions, the default is a human‑in‑the‑loop workflow whose validation cadence and depth are propor ‑ tionate to risk [94, 95].”
Integrated Decision Support Systems (DSS) for Responsible AI Governance in Higher Education
Chua, K. P., & Vrba, J. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-2600-0467-8.ch005
A Human–AI Agency Interaction Framework for Education
Nurcahyo, E., & Sekti, B. A. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-7467-3.ch001
Right-to-Override (R2O) Interfaces: Safe Circuit Breakers for Frontline Automation
GUDIPATI, S. K., & PONUGOTI, S. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-30044-7_3
Alves, J. N., & Tseng, H. (2026) · AI and Ethics · doi:10.1007/s43681-026-01343-5
“Robodebt [66] and the Dutch childcare benefits scandal [ 4, 32, 33] represent accountability breakdowns in algorithmically mediated welfare admin - istration.”
InfluentialWang, J., Wang, Q., Yang, P., Li, G., Wang, X., Sun, Y., Feng, Z., Gao, Y., Li, Y., Guo, W., et al. (2026) · npj Digital Public Health · doi:10.1038/s44482-026-00045-3
“Forecast performance remains important, but it is insuf fi cient on its own to assess whether ACCESS improves public health decision-making 66,67 .”
Metadata
- Authors
- Haitsma, L., & Brink, B.
- Affiliations
- University of Groningen (NL)
- Venue
- Digital Government: Research and Practice
- Published
- 2025-08-27
- Volume
- 6
- Issue
- 3
- Pages
- 1-17
- ISSN
- 2639-0175 (ISSN-L)
- 2691-199X
- OpenAlex
- W4407131097
- Semantic Scholar
- 4739e08d6eb2814b36b8aa2ec375e2950740daee
- Cited by
- 7
- References
- 14
- Citations by year
- 2026: 6 · 2025: 1
- FWCI
- 26.13
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Digital Government Research and Practice (publishedVersion, cc-by)
- University of Groningen research database (University of Groningen / Centre for Information Technology) (submittedVersion, cc-by)
- Topics
- Social Sciences › Political Science and International Relations › E-Government and Public Services
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Law › Digitalization, Law, and Regulation
- Keywords
- human-in-the-loop, human-on-the-loop, human-machine interaction, system design, accountability, meaningful human control
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.