The DOI matches its Crossref record; 47 of the 47 references it cites were found in a registry or an index; 4 of the 13 sources its claims cite were checked for support.
Will Algorithms Blind People? The Effect of Explainable AI and Decision-Makers’ Experience on AI-supported Decision-Making in Government
Open accessAbstract
Computational artificial intelligence (AI) algorithms are increasingly used to support decision making by governments. Yet algorithms often remain opaque to the decision makers and devoid of clear explanations for the decisions made. In this study, we used an experimental approach to compare decision making in three situations: humans making decisions (1) without any support of algorithms, (2) supported by business rules (BR), and (3) supported by machine learning (ML). Participants were asked to make the correct decisions given various scenarios, while BR and ML algorithms could provide correct or incorrect suggestions to the decision maker. This enabled us to evaluate whether the participants were able to understand the limitations of BR and ML. The experiment shows that algorithms help decision makers to make more correct decisions. The findings suggest that explainable AI combined with experience helps them detect incorrect suggestions made by algorithms. However, even experienced persons were not able to identify all mistakes. Ensuring the ability to understand and traceback decisions are not sufficient for avoiding making incorrect decisions. The findings imply that algorithms should be adopted with care and that selecting the appropriate algorithms for supporting decisions and training of decision makers are key factors in increasing accountability and transparency.
Verification finished with gaps
Some checks didn’t finish. The results below cover the steps that did.
- Registry recordVerified
- Full textRead the full text
- Reference list42 references found
- Reference check47 of 47 references checked
- StatisticsRecomputed
- Claims58 claims anchored to a quote
- Cited sourcesChecked
- Self-checkDidn’t finish — the rest of the report still applies.
- VerificationFindings verified
Overview
CoChat Summary
TL;DR
In an online quasi-experiment using Dutch immigration/asylum cases, algorithmic support helped participants make more correct decisions than human-only decision making, and transparent business rules (BR) outperformed opaque machine learning (ML). Explainability plus experience helped detect incorrect algorithmic suggestions, but even experienced participants missed mistakes, suggesting algorithms can “blind” decision makers.
Key findings
- Algorithms improved correctness: mean correct decisions out of 30 were 23.20 for human-only, 24.65 for ML-supported, and 25.80 for BR-supported decision making.
- Transparent BR outperformed opaque ML: BR had the highest mean correct decisions (25.80 vs. 24.65) and the lowest variance (8.30 vs. 9.04 for ML and 15.69 for human-only), suggesting more consistent decisions.
- Experience mattered: experienced participants averaged 26.21 correct decisions versus 22.89 for nonexperienced participants; within conditions, experienced BR reached 27.27, experienced ML 26.17, and experienced human-only 25.20.
- ANOVA found significant differences between decision modes (columns, p = 0.011) and between experience groups (rows, p = 0.002), but no significant interaction (p = 0.868); even experienced participants did not identify all algorithmic mistakes.
Methods
- Quasi-experiment comparing three conditions—human-only, BR-supported, and ML-supported decision making—using asylum request and residence permit cases derived from the Dutch IND; algorithms sometimes gave incorrect suggestions, with half of suggestions wrong and equal wrong counts for ML and BR.
- Participants were given 30 different cases in randomized order with no feedback; they had to grant or reject each application, and decisions were marked correct if they matched legislation. Table 2 reports 180 decisions across conditions and experience groups.
- Participants were master’s students oriented toward government work, plus a few civil servants; most were 22–30 years old, about one quarter claimed prior public administration experience, and international students were categorized as experienced. Analysis used ANOVA.
Limitations
- Stated: the experiment was too simplistic and designed as a setup for making wrong decisions; only two algorithm types were used; participants were forced to choose and could not investigate further; some quit; a larger sample was needed; input data were predefined and structured, and contested cases were excluded.
- Stated: the study did not examine trust, fairness, transparency, accountability, user-interface effects, time pressure, or learning over time; it also used students rather than public decision makers, with only a few civil servants, and could not statistically prove that civil servant results differed.
- Evident: the paper reports ANOVA F and p values but no effect sizes or confidence intervals, and the simplified cases/interfaces limit ecological validity for real government decision-making.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
4 Valid · 0 Not valid · 8 Not checked · 46 Own results and interpretations
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The p values show that the differences between the human, ML, and BR-based decision making are significant.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, even experienced persons were not able to identify all mistakes.”
- Also stated in: § Conclusions · p. 15 · plain
“In our experiment, no single decision maker was able to detect all mistakes made by algorithms.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Conclusions · p. 15 · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Despite the experiment's relatively simple nature, only a small portion of the participants made all decisions correctly in the experiment.”
- Also stated in: § Discussion · p. 12 · plain
“There were only a few participants who made no or hardly any mistakes.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Discussion · p. 12 · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Also, the difference between experienced and nonexperienced persons is found to be significant.”
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Consequently, BRs were not able to arrive at a correct decision for all situations, as not all exceptions were codified in BRs and the BRs were not able to interpret the ambiguity in some of the formulations.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
26 more claims
Create a free account to see all 58 claims and every source verdict.
- Probably not a substantial claim. 20
- It may state two separate claims as one. 9
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Janssen et al., 2022's experiment shows transparency leads to more correct decisions when algorithmic options are used to support human decisions.”
From AI adoption and diffusion in public administration: A systematic literature review and future research agenda (2023) · cites this paper as Janssen et al., 2022
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
This check didn’t finish.
Citations
Cites
Artificial intelligence: a modern approach
(1995) · Choice Reviews Online · doi:10.5860/choice.33-1577 · Cited by 21,974
Dilemmas in a general theory of planning
Rittel, H. W. J., & Webber, M. M. (1973) · Policy Sciences · doi:10.1007/bf01405730 · Cited by 16,338 · cited 1×
“The wicked nature of the problems faced by public administrators or governments is often emphasized: the fact that they often need to be solved based on incomplete, contradictory, and changing information (Churchman, 1967; Rittel & Webber, 1973).”
BackgroundMachine learning: Trends, perspectives, and prospects
Jordan, M. I., & Mitchell, T. M. (2015) · Science · doi:10.1126/science.aaa8415 · Cited by 9,955 · cited 3×
“Jordan and Mitchell (2015) found that for many applications, it is easier to train a system by showing it examples of desired input–output behavior than to program it manually.”
BackgroundInfluentialThe Science of “Muddling Through”
Lindblom, C. E. (1959) · Public Administration Review · doi:10.2307/973677 · Cited by 8,513
The Quarterly Journal of Economics
Kuznets, S. (2024) · Business Cycle Theory, Part I Volume 4 · doi:10.4324/9781003549741-8 · Cited by 7,176
Organizing and the process of sensemaking
Weick, K. E., Sutcliffe, K. M., & Obstfeld, D. (2005) · Organization Science · doi:10.1287/orsc.1050.0133 · Cited by 6,943 · cited 1×
“Sense-making is the process of giving meaning to an ambiguous situation (Weick et al., 2005).”
BackgroundPeeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294 · cited 1×
“Algorithms differ in their level of explainability (Adadi & Berrada, 2018).”
BackgroundExplanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302 · cited 5×
“XAI refers to “an explanatory agent revealing underlying causes to its or another agent’s decision making” (Miller, 2019, p. 4).”
BackgroundInfluentialHow the machine “thinks”: Understanding opacity in machine learning algorithms
Burrell, J. (2016) · Big Data & Society · doi:10.1177/2053951715622512 · Cited by 3,196 · cited 3×
“However, many of these AI applications have been criticized for their opaqueness and the lack of explanation of the causal link between input and output (Burrell, 2016).”
BackgroundInfluentialBarocas, S., & Selbst, A. D. (2014) · doi:10.2139/ssrn.2477899 · Cited by 3,048 · cited 1×
“The use of algorithms might not result in correct and accurate decisions (Barocas & Selbst, 2016).”
Method
Research works printed but not linked: 4
- Barocas S., 2016, California Law Review, 104, 671
- Brauneis R., 2018, Yale Journal of Law & Technology, 20, 103
- Kroll J. A., 2015, Accountable algorithms
- Simon H. A., 1972, Decision and Organization, 1, 161
Cited by
Madan, R., & Ashok, M. (2023) · Government Information Quarterly · doi:10.1016/j.giq.2022.101774 · Cited by 468
“…• Ability to game the system with higher transparency (Chen et al., 2019; Harrison & Luna-Reyes, 2022; Henman, 2019; Janssen, Brous, et al., 2020; Janssen et al., 2022; Makasi et al., 2021; Mulligan & Bamberger, 2019; Ojo et al., 2019; Sousa et al., 2019; Veale et al., 2018; Veale & Brass, 2019;…”
Darwin, Rusdin, D., Mukminatien, N., Suryati, N., Laksmi, E. D., & Marzuki (2024) · Cogent Education · doi:10.1080/2331186x.2023.2290342 · Cited by 290
“Keyes et al. (2021) and Janssen et al. (2020) have warned that these algorithms often lack alignment with individual criteria for scholarly paper selection, and as such, could limit opportunities for in-depth critical analysis.”
Haque, A. B., Islam, A. K. M. N., & Mikalef, P. (2023) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2022.122120 · Cited by 240
“To promote transparency, different attributes, such as age, gender, income, profession, and other related specifics, can be included in the explanation (Janssen et al., 2020).”
BackgroundMethodInfluentialUnveiling the shadows: beyond the hype of AI in education
Al-Zahrani, A. M. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e30696 · Cited by 225
“Transparency and explainability in AI systems are further marred by technical complexities [34 – 40].”
BackgroundArtificial intelligence in public services: When and why citizens accept its usage
Gesk, T. S., & Leyer, M. (2022) · Government Information Quarterly · doi:10.1016/j.giq.2022.101704 · Cited by 195
Metadata
- Authors
- Janssen, M., Hartog, M., Matheus, R., Yi Ding, A., & Kuk, G.
- Affiliations
- Delft University of Technology (NL)
- Delft University of Technology, the Netherlands
- Nottingham Trent University (GB)
- Corresponding author
- Janssen, Marijn
- Venue
- Social Science Computer Review
- Publisher
- SAGE Publications
- Published
- 2020-12-28
- Volume
- 40
- Issue
- 2
- Pages
- 478-493
- ISSN
- 0894-4393 (ISSN-L)
- 1552-8286
- OpenAlex
- W3114194962
- Cited by
- 192
- References
- 42
- Citations by year
- 2026: 46 · 2025: 44 · 2024: 45 · 2023: 28 · 2022: 19 · 2021: 10
- FWCI
- 5.59
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Social Science Computer Review (publishedVersion, cc-by)
- Nottingham Trent University's Institutional Repository (Nottingham Trent Repository) (acceptedVersion)
- Lirias (submittedVersion, cc-by)
- Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
- Research Repository (Delft University of Technology) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Political Science and International Relations › E-Government and Public Services
- Keywords
- explainable AI, AI-supported decision-making, government decision-making, algorithmic opacity, decision maker experience, business rules, machine learning, algorithmic bias, accountability, transparency, algorithmic accountability
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-15, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-15 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Will Algorithms Blind People? The Effect of Explainable AI and Decision-Makers’ Experience on AI-supported Decision-Making in Government. https://app.cochat.ai/papers/10.1177/0894439320980118
- The paper
- Janssen, M., Hartog, M., Matheus, R., Yi Ding, A., & Kuk, G. (2022). Will Algorithms Blind People? The Effect of Explainable AI and Decision-Makers’ Experience on AI-supported Decision-Making in Government. Social Science Computer Review. https://doi.org/10.1177/0894439320980118
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.