Cited by 264Explainable Artificial Intelligence (XAI)Ethics and Social Impacts of AIE-Government and Public Services

The perils and pitfalls of explainable AI: Strategies for explaining algorithmic decision-making

Open access
de Bruijn, Hans, Warnier, Martijn, Janssen, Marijn·2022·Government Information Quarterly

Abstract

Governments look at explainable artificial intelligence's (XAI) potential to tackle the criticisms of the opaqueness of algorithmic decision-making with AI. Although XAI is appealing as a solution for automated decisions, the wicked nature of the challenges governments face complicates the use of XAI. Wickedness means that the facts that define a problem are ambiguous and that there is no consensus on the normative criteria for solving this problem. In such a situation, the use of algorithms can result in distrust. Whereas there is much research advancing XAI technology, the focus of this paper is on strategies for explainability. Three illustrative cases are used to show that explainable, data-driven decisions are often not perceived as objective by the public. The context might raise strong incentives to contest and distrust the explanation of AI, and as a consequence, fierce resistance from society is encountered. To overcome the inherent problems of XAI, decisions-specific strategies are proposed to lead to societal acceptance of AI-based decisions. We suggest strategies to embrace explainable decisions and processes, co-create decisions with societal actors, move away from an instrumental to an institutional approach, use competing and value-sensitive algorithms, and mobilize the tacit knowledge of professionals

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.giq.2021.101666

Cites

39 of 39 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • In complex public-law settings, explanations are embedded in contested legal standards, institutional purposes and discretionary judgments.

    “In complex public-law settings, explanations are embedded in contested legal standards, institutional purposes and discretionary judgments (de Bruijn et al., 2022;Oswald, 2018).”

    From From Human Oversight to Cognitive Sovereignty: A Process-Based Governance Standard for AI-Assisted Legal Reasoning (2026) · cites this paper as de Bruijn et al., 2022

    Section 7 (page unavailable) says government AI explanations can be contested and that the problems they address are ambiguous. Section 6.4 (page unavailable) says transparency also requires institutional rules and oversight. These passages support the claim’s broader point, but they do not specifically describe explanations as embedded in contested legal standards, institutional purposes, and discretionary judgments. (LLM, conf=0.82)

    Experimental — accuracy not yet measured
  • De Bruijn, Warnier and Janssen (2022) show that explainable AI is not a simple cure for public-sector algorithmic decisionmaking because explanations operate within contested legal and administrative contexts.

    “De Bruijn, Warnier and Janssen (2022) show that explainable AI is not a simple cure for public-sector algorithmic decisionmaking because explanations operate within contested legal and administrative contexts.”
  • Explainable AI, at least in its current form, may not address the informational needs of laypeople, politicians, professionals, or scientists because the information is too technical.

    “Explainable AI, at least in its current form, may not address the informational needs of laypeople, politicians, professionals, or scientists because the information is too technical [58].”
  • There is very little agreement about what explainability means in practice, because what makes AI explainable depends on the context of its use.

    “there is very little agreement about what explainability means in practice, because what makes AI explainable depends on the context of its use [58].”

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 3

  1. Biran, O., & Cotton, C. (2017). Explanation and justification in machine learning: A survey. In Paper presented at the In IJCAI-17 workshop on explainable AI (XAI).
  2. LeCun, Y., Bengio, Y., & Hinton, G. J.n. (2015). Deep learning., 521(7553), 436-444.
  3. McDonald, H. (2019). AI system for granting UK visas is biased, rights groups claim. The Gaurdian, 29(10), 29 Oct 2019. Retrieved from https://amp.theguardian.com/uk- news/2019/oct/29/ai-system-for-granting-uk-visas-is-biased-rights-groups-claim.

Cited by

Sorted by citations

Authors
de Bruijn, H., Warnier, M., & Janssen, M.
Affiliations
Delft University of Technology (NL)
Corresponding author
Janssen, Marijn
Venue
Government Information Quarterly
Publisher
Elsevier BV
Published
2022-04
Volume
39
Issue
2
Pages
101666
ISSN
0740-624X (ISSN-L)
OpenAlex
W4205177168
Cited by
264
References
37
Citations by year
2026: 66 · 2025: 74 · 2024: 63 · 2023: 44 · 2022: 15 · 2021: 1 · 2017: 1
FWCI
10.87
Type
journal article
Language
en
Open access
Locations
  • Government Information Quarterly (publishedVersion, cc-by)
  • Lirias (KU Leuven) (submittedVersion, cc-by)
  • Research Repository (Delft University of Technology) (submittedVersion, cc-by)
  • Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Social Sciences › Political Science and International Relations › E-Government and Public Services
Keywords
explainable AI, algorithmic decision-making, algorithmic transparency, public trust in AI, wicked problems, automated decision-making, government AI policy

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review