Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 47 of the 47 references it cites were found in a registry or an index; 4 of the 13 sources its claims cite were checked for support.

Will Algorithms Blind People? The Effect of Explainable AI and Decision-Makers’ Experience on AI-supported Decision-Making in Government

Open access
Janssen, Marijn, Hartog, Martijn, Matheus, Ricardo, Yi Ding, Aaron, Kuk, George · 2022 · Social Science Computer Review

Abstract

Computational artificial intelligence (AI) algorithms are increasingly used to support decision making by governments. Yet algorithms often remain opaque to the decision makers and devoid of clear explanations for the decisions made. In this study, we used an experimental approach to compare decision making in three situations: humans making decisions (1) without any support of algorithms, (2) supported by business rules (BR), and (3) supported by machine learning (ML). Participants were asked to make the correct decisions given various scenarios, while BR and ML algorithms could provide correct or incorrect suggestions to the decision maker. This enabled us to evaluate whether the participants were able to understand the limitations of BR and ML. The experiment shows that algorithms help decision makers to make more correct decisions. The findings suggest that explainable AI combined with experience helps them detect incorrect suggestions made by algorithms. However, even experienced persons were not able to identify all mistakes. Ensuring the ability to understand and traceback decisions are not sufficient for avoiding making incorrect decisions. The findings imply that algorithms should be adopted with care and that selecting the appropriate algorithms for supporting decisions and training of decision makers are key factors in increasing accountability and transparency.

Verification finished with gaps

Some checks didn’t finish. The results below cover the steps that did.

  1. Registry recordVerified
  2. Full textRead the full text
  3. Reference list42 references found
  4. Reference check47 of 47 references checked
  5. StatisticsRecomputed
  6. Claims58 claims anchored to a quote
  7. Cited sourcesChecked
  8. Self-checkDidn’t finish — the rest of the report still applies.
  9. VerificationFindings verified

Overview

CoChat Summary

TL;DR

In an online quasi-experiment using Dutch immigration/asylum cases, algorithmic support helped participants make more correct decisions than human-only decision making, and transparent business rules (BR) outperformed opaque machine learning (ML). Explainability plus experience helped detect incorrect algorithmic suggestions, but even experienced participants missed mistakes, suggesting algorithms can “blind” decision makers.

Key findings

  • Algorithms improved correctness: mean correct decisions out of 30 were 23.20 for human-only, 24.65 for ML-supported, and 25.80 for BR-supported decision making.
  • Transparent BR outperformed opaque ML: BR had the highest mean correct decisions (25.80 vs. 24.65) and the lowest variance (8.30 vs. 9.04 for ML and 15.69 for human-only), suggesting more consistent decisions.
  • Experience mattered: experienced participants averaged 26.21 correct decisions versus 22.89 for nonexperienced participants; within conditions, experienced BR reached 27.27, experienced ML 26.17, and experienced human-only 25.20.
  • ANOVA found significant differences between decision modes (columns, p = 0.011) and between experience groups (rows, p = 0.002), but no significant interaction (p = 0.868); even experienced participants did not identify all algorithmic mistakes.

Methods

  • Quasi-experiment comparing three conditions—human-only, BR-supported, and ML-supported decision making—using asylum request and residence permit cases derived from the Dutch IND; algorithms sometimes gave incorrect suggestions, with half of suggestions wrong and equal wrong counts for ML and BR.
  • Participants were given 30 different cases in randomized order with no feedback; they had to grant or reject each application, and decisions were marked correct if they matched legislation. Table 2 reports 180 decisions across conditions and experience groups.
  • Participants were master’s students oriented toward government work, plus a few civil servants; most were 22–30 years old, about one quarter claimed prior public administration experience, and international students were categorized as experienced. Analysis used ANOVA.

Limitations

  • Stated: the experiment was too simplistic and designed as a setup for making wrong decisions; only two algorithm types were used; participants were forced to choose and could not investigate further; some quit; a larger sample was needed; input data were predefined and structured, and contested cases were excluded.
  • Stated: the study did not examine trust, fairness, transparency, accountability, user-interface effects, time pressure, or learning over time; it also used students rather than public decision makers, with only a few civil servants, and could not statistically prove that civil servant results differed.
  • Evident: the paper reports ANOVA F and p values but no effect sizes or confidence intervals, and the simplified cases/interfaces limit ecological validity for real government decision-making.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1177/0894439320980118

Cites

47 of 47 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

4 Valid · 0 Not valid · 8 Not checked · 46 Own results and interpretations

26 more claims

Create a free account to see all 58 claims and every source verdict.

See all claims — free account
  • Probably not a substantial claim. 20
  • It may state two separate claims as one. 9

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

This check didn’t finish.

Citations

Cites

Show all 42 references

Research works printed but not linked: 4

  1. Barocas S., 2016, California Law Review, 104, 671
  2. Brauneis R., 2018, Yale Journal of Law & Technology, 20, 103
  3. Kroll J. A., 2015, Accountable algorithms
  4. Simon H. A., 1972, Decision and Organization, 1, 161

Cited by

See all 192

Metadata

Authors
Janssen, M., Hartog, M., Matheus, R., Yi Ding, A., & Kuk, G.
Affiliations
  • Delft University of Technology (NL)
  • Delft University of Technology, the Netherlands
  • Nottingham Trent University (GB)
Corresponding author
Janssen, Marijn
Venue
Social Science Computer Review
Published
2020-12-28
Volume
40
Issue
2
Pages
478-493
ISSN
  • 0894-4393 (ISSN-L)
  • 1552-8286
OpenAlex
W3114194962
Cited by
192
References
42
Citations by year
2026: 46 · 2025: 44 · 2024: 45 · 2023: 28 · 2022: 19 · 2021: 10
FWCI
5.59
Type
journal article
Language
en
Open access
Locations
  • Social Science Computer Review (publishedVersion, cc-by)
  • Nottingham Trent University's Institutional Repository (Nottingham Trent Repository) (acceptedVersion)
  • Lirias (submittedVersion, cc-by)
  • Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
  • Research Repository (Delft University of Technology) (submittedVersion)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Social Sciences › Political Science and International Relations › E-Government and Public Services
Keywords
explainable AI, AI-supported decision-making, government decision-making, algorithmic opacity, decision maker experience, business rules, machine learning, algorithmic bias, accountability, transparency, algorithmic accountability

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-15, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-15 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Will Algorithms Blind People? The Effect of Explainable AI and Decision-Makers’ Experience on AI-supported Decision-Making in Government. https://app.cochat.ai/papers/10.1177/0894439320980118
The paper
Janssen, M., Hartog, M., Matheus, R., Yi Ding, A., & Kuk, G. (2022). Will Algorithms Blind People? The Effect of Explainable AI and Decision-Makers’ Experience on AI-supported Decision-Making in Government. Social Science Computer Review. https://doi.org/10.1177/0894439320980118

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.