CoChat
Sign Up
Cited by 334Ethics and Social Impacts of AIExplainable Artificial Intelligence (XAI)E-Government and Public Services

Artificial intelligence, transparency, and public decision-making: Why explanations are key when trying to produce perceived legitimacy

Open access
de Fine Licht, Karl, de Fine Licht, Jenny·2020·AI & SOCIETY
DOI 10.1007/s00146-020-00960-w
Open this paper in CoChat — free accountOpen PDF

Abstract

The increasing use of Artificial Intelligence (AI) for making decisions in public affairs has sparked a lively debate on the benefits and potential harms of self-learning technologies, ranging from the hopes of fully informed and objectively taken decisions to fear for the destruction of mankind. To prevent the negative outcomes and to achieve accountable systems, many have argued that we need to open up the “black box” of AI decision-making and make it more transparent. Whereas this debate has primarily focused on how transparency can secure high-quality, fair, and reliable decisions, far less attention has been devoted to the role of transparency when it comes to how the general public come to perceive AI decision-making as legitimate and worthy of acceptance. Since relying on coercion is not only normatively problematic but also costly and highly inefficient, perceived legitimacy is fundamental to the democratic system. This paper discusses how transparency in and about AI decision-making can affect the public’s perception of the legitimacy of decisions and decision-makers and produce a framework for analyzing these questions. We argue that a limited form of transparency that focuses on providing justifications for decisions has the potential to provide sufficient ground for perceived legitimacy without producing the harms full transparency would bring.

AI snapshot

Written from the paper’s full text.

TL;DR

The paper argues that for AI in public decision-making, full transparency about decision-making processes is not necessarily legitimacy-enhancing and may even backfire. It proposes that a limited transparency focused on justifications—transparency in rationale—can better support public perceptions of legitimacy.

Key findings

  • The paper distinguishes transparency about final decisions, transparency in process, and transparency in rationale; it argues decision-makers should generally opt for transparency in rationale, meaning justifications for decisions plus information on who is accountable.
  • It identifies four arguments for full transparency—public scrutiny, public understanding, feelings of control, and fairness perceptions—and argues each can fail or harm perceived legitimacy due to problems such as information overload, the “transparency paradox,” the “myth of hidden politics,” and disappointment with real-world “muddling through.”
  • A justifications approach is argued to support accountability, higher-quality decisions through the giving-reasons requirement, public understanding, and perceptions of fair treatment; AI assistants should explain decisions in accessible language and make clear who is accountable or where to appeal.
  • The paper offers a framework dividing AI public decision-making into three phases: goal-setting, coding, and implementation; transparency in rationale can be applied in each phase.

Methods

  • Conceptual/theoretical paper; no empirical data, sample, or statistical analysis are reported.
  • It synthesizes social science literature on transparency, legitimacy, principal-agent theory, deliberative democracy, and explainable AI to build an argument and framework.
  • The main analysis is normative and argumentative, proposing a framework for transparency and perceived legitimacy in AI decision-making in the socio-technical system.

Limitations

  • The authors state that their analysis “builds on a considerable number of assumptions and guesses” and call for theoretical and empirical work on justifications, normatively acceptable reasons, how justifications should be designed and presented, empirically grounded policy design, and how decisions are perceived when made by humans versus AIs.
  • The text makes evident that the argument is not empirically tested and relies on literature-based reasoning and examples rather than new data.
  • The paper’s main focus is public policymaking and authoritative public decisions, though the authors say the general argument should also apply to AI use in more private operations.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1007/s00146-020-00960-w

Cites

61 of 61 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 39 cited sources checked · 0 supported · 0 flagged

  • Interpretationreasoning

    Quick and efficient decision-making may be vital when implementing AI assistants in Phases 1 and 2 and adjusting them in Phase 3.

    “Furthermore, since AI assistants work quickly, thereby producing potentially positive and negative outcomes in a short timeframe, quick and efficient decision-making may be vital when implementing AI assistants (Phases 1 and 2) and adjusting them (Phase 3).”
    Quote in the paperNot a substantial claim
  • Interpretationreasoning

    Making Phases 1 and 2 available is not necessarily beneficial.

    “This suggests that making Phases 1 and 2 available is not necessarily beneficial.”
    Quote in the paperSubstantial claim
  • Interpretationreasoning

    If the public needs a chance to understand the code, the code would likely have to be much simpler than it is in today's systems.

    “assuming that the public needs a chance to actually understand the code, the code would likely have to be much simpler than it is in today's systems.”
    Quote in the paperNot a substantial claim
    • The authors state this as an assumption: The public needs a chance to actually understand the code for transparency to have positive effects.
  • Interpretationreasoning

    A limited form of transparency that focuses on providing justifications for decisions has the potential to provide sufficient grounds for perceived legitimacy in AI decision-making without producing the harms full transparency would bring.

    “We argue that a limited form of transparency that focuses on providing justifications for decisions has the potential to provide sufficient ground for perceived legitimacy without producing the harms full transparency would bring.”
    • Also stated in: § 1 Introduction · hedged
      “Based on our reading of the literature, we argue that a limited form of transparency that focuses on providing justifications for decisions has the potential to provide sufficient grounds for perceived legitimacy in AI decision-making.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Relying on coercion is normatively problematic, costly, and highly inefficient, and perceived legitimacy is fundamental to the democratic system.
  • Interpretationreasoning

    Not all forms of transparency lead to greater perceived legitimacy.

    “Intuitively, however, not all forms of transparency lead to greater perceived legitimacy.”
    Quote in the paperSubstantial claim
  • Classified as probably not a substantial claim. 29
  • May combine two claims. 32

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationreasoning

    If explicability means making decision-making processes fully transparent, it is not suitable for producing perceived legitimacy; if it means providing a narrative explanation of how the decision was made, it might be applicable for perceived legitimacy.

    “However, if explicability means that we actually make the decisionmaking processes fully transparent, then we do not believe it suitable in relation to the production of perceived legitimacy; but, if decision-makers should provide an explanation in the form of a narrative where it is explained how the decision has been made, then this might be applicable for perceived legitimacy.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    Demands for transparency would probably produce a lower degree of quality in decisions and thereby less perceived legitimacy.

    “Thus, in this sense, though, demands on transparency would probably produce a lower degree of quality in the decisions and thereby less perceived legitimacy.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationcitation

    Simpler code would arguably make it easier to game the system and would provide industry with fewer incentives to innovate.

    “This would arguably make it easier to game the system, and it would provide the industry with fewer incentives to innovate (e.g., Zarsky 2016; Lepri et al. 2017;De Laat 2018; Carabantes 2019).”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites Zarsky 2016 source

      Not checked in the free run.

    • cites Lepri et al. 2017

      Not checked in the free run.

    • cites De Laat 2018 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S1 (asserted), S2 (checkable), S3 (load-bearing), S4 (based).

    Premisecitation

    The capacity to act is even more important in AI decision-making than in regular policymaking.

    “Thus, the”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Warren and Mansbridge 2013

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Decision-makers and professionals might become more interested in avoiding blame than in finding optimal solutions, meaning open discussion may become shallow and fail to produce the best possible outcomes.

    “Decision-makers and professionals might also become more interested in avoiding blame than in finding optimal solutions (Hood 2007), meaning that the open discussion may become shallow (Chambers 2004(Chambers , 2005) ) and fail to produce the best possible outcomes.”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites Hood 2007

      Not checked in the free run.

    • cites Chambers 2004

      Not checked in the free run.

    • cites Chambers 2005

      Not checked in the free run.

  • May combine two claims.

    Interpretationcitation

    Transparency about a process imposes incentives on decision-makers to look credible in the eyes of observers, but this is not necessarily positive for reaching a good decision.

    “One reason for this is that transparency about a process imposes incentives on the decision-makers to look credible in the eyes of the observers during their deliberations, but this is not necessarily positive in relation to the prospect of reaching a good decision.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    All organizations need some shelter in order to arrive at choices and carry them out; fully exposing reasoning, planning, accommodation, and choice from the outset hampers these processes.

    “Bok (1989, p. 175) argues that all organizations need ⋯some shelter in order to be able to arrive at choices and to carry them out.The processes of reasoning, planning, accommodation, and choice are hampered if fully exposed from the outset, no matter how great the corresponding dangers of secrecy.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Bok (1989, p. 175)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationreasoning

    Asking stupid questions is especially important in relation to AI assistants, since these matters are so complex that there will often be many seemingly stupid questions to ask.

    “Asking”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    In front of an audience, it may be difficult to change an opinion, present dissenting opinions, or ask necessary but seemingly stupid questions.

    “In front of an audience, it may be difficult to change an opinion (even with a good argument), present dissenting opinions, or ask necessary but seemingly”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites Elster 1998

      Not checked in the free run.

    • cites Meade and Stasavage 2006

      Not checked in the free run.

    • cites Mansbridge 2009 source

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    Finding the relevant information would be as difficult as finding a needle in a haystack, so we will not witness the positive outcomes of making the information available.

    “Finding the relevant information would be as difficult as finding the proverbial needle in a haystack-something that decision-makers and the public will understand and adapt to.Hence, we will not witness the positive outcomes of making the information available.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    That decision-makers are given some shelter for discussion is especially important in Phases 1 and 2, since these phases are likely to contain both technically complex and morally controversial tradeoffs.

    “That decision-makers are given some shelter for discussion is especially important in Phases 1 and 2, since these phases are likely to contain both technically complex and morally controversial tradeoffs.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Transparency is generally associated with the 'myth of hidden politics'.

    “Transparency is generally associated with the”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Fenster 2006, p. 931

      Not checked in the free run.

    • cites West and Sanders 2003

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given

    In Phases 1 and 2, the public eye monitors the reasons on which the decisions are based.

    “in Phases 1 and 2, the public eye monitors the reasons on which the decisions are based.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

    Interpretationreasoning

    A cooperative working climate that can spark efficiency and problem-solving when it comes to AI in decision-making is crucial, as algorithms can potentially cause considerable damage if they are not properly adjusted when necessary.

    “A cooperative working climate that can spark efficiency and problem-solving when it comes to AI in decision-making is crucial, as algorithms can potentially cause considerable damage if they are not properly adjusted when necessary.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationcitation

    Decision-makers are likely to properly weigh the pros and cons when making decisions, thus leading to higher-quality decisions.

    “Thus, decision-makers are likely to properly weigh the pros and cons when making decisions (Shapiro 1992), thus leading to higher-quality decisions.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Shapiro 1992

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    No general framework exists to analyze the perceived legitimacy of AI in the broader context of the socio-technological system to elucidate issues of transparency and perceived legitimacy.

    “As well as this, no general framework exists to analyze the perceived legitimacy of AI in the broader context of the socio-technological system to elucidate the issues of today and tomorrow regarding transparency and perceived legitimacy.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    Far less attention has been devoted to the role of transparency in how the general public come to perceive AI decision-making as legitimate and worthy of acceptance.

    “However, though a lot of interesting work has been done in the area of transparency, far less attention has been devoted to the role of transparency in terms of how those who are ultimately affected (i.e., the general public) come to perceive AI decision-making as being legitimate and worthy of acceptance.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationnone given

    It may be difficult to establish goals and guarantee that their respective importance is sufficiently precise for programmers' needs.

    “Furthermore, it may be difficult to establish goals and guarantee that their respective importance is sufficiently precise for programmers' needs.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Interpretationcitation

    Even though this is persuasive, it may be just a story.

    “Even though this is persuasive, it may be just a story.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationcitation

    These qualities can increase both quality and efficiency in decision-making.

    “These qualities can increase both quality and efficiency in decision-making (Mansbridge 2009).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Mansbridge 2009 source

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    A policy of justifications of decisions will inform the public of what the decision is, on which grounds it has been made, and identify who the responsible actor is.

    “a policy of justifications of decisions will inform the public of what the decision is, on which grounds it has been made, and in doing so, identify who the responsible actor is.”
    • Also stated in: § 4 The benefits of a justifications approach to AI transparency · plain
      “A policy of justification will tell the public what the decision is, on which grounds it has been made, and who the responsible actor is.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationcitation

    The widespread myth of secret power is sparked by the fact that modern popular culture generally tells us that we should be suspicious of anyone in a powerful position.

    “The widespread myth of secret power is sparked by the fact that modern popular culture generally tells us that we should be suspicious of anyone in a powerful position (Brin 1998).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Brin 1998

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    Due to the staggering amount of information available, it may be difficult to understand who to hold accountable for what.

    “As such, due to the staggering amount of information available, it may be difficult to understand who to hold accountable for what.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    Decision-makers will have incentives to behave better when it comes to producing such reasons, thus producing higher-quality decisions.

    “Hence, the decision-makers will have incentives to behave better when it comes to producing such reasons, thus producing higher-quality decisions.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The notion that AI assistants should be able to provide justifications or explanations for their decisions fits with the core components of explainable AI (XAI).

    “Of course, the notion that AI assistants should be able to provide justifications or explanations for their decisions is not novel.In fact, it fits nicely with the core components of the rapidly evolving research field of explainable AI (XAI) (e.g., Gunning 2017Gunning , 2019;;Thelisson et al. 2017).”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites Gunning 2017

      Not checked in the free run.

    • cites Gunning 2019

      Not checked in the free run.

    • cites Thelisson et al. 2017

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    Source code and data used to train and test AI are often too complex for most individuals to fully understand, so full transparency might make the public less prone to believing they have more control over the AI than before.

    “Similarly, the source code and data that the AI has been trained and tested on are often too complex for most individuals to fully understand, meaning that full transparency might actually make the public less prone to believing that they have more control over the AI than they did before they had access to the data.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The general public will not understand the source code.

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationreasoning

    For a general public that is not particularly interested in political matters, justifications may actually be a more attractive policy than full transparency.

    “For a general public that is not particularly interested in political matters, this means that justifications may actually be a more attractive policy than full transparency.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The general public is not particularly interested in political matters.

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationreasoning

    The public is likely to get sufficient information to understand the different considerations and perspectives underlying the decision, thus making them feel included.

    “Similarly, they are likely to get sufficient information to understand the different considerations and perspectives underlying the decision, thus making them feel included.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationcitation

    When the actual decision-making process remains comparatively secret, decision-makers can ask important but seemingly 'stupid' questions, change their minds in light of new arguments instead of negotiating on fixed preferences, and search for allies.

    “At the same time, when the actual decision-making process remains comparatively secret, decision-makers can ask important but seemingly”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    When code and data are made transparent, the public might think the release is hiding the real code or that there is a back door in the code.

    “Thus, when code and data are made transparent, the public might think this release is hiding the”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The general public will not understand the source code.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning

    There is a constant intermingling between Phases 1-3, with all phases deeply connected to each other.

    “Thus, there is a constant intermingling between Phases 1-3, with all phases deeply connected to each other.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationreasoning

    According to principal-agent terminology, the agent (the people) will know who to hold accountable and for what, and this appears true for all phases.

    “according to principal-agent terminology, the agent (the people) will know who to hold accountable and for what, and this appears true for all phases.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    Decision-makers might use this information to make more well-informed decisions, leading to higher-quality decisions that result in higher perceived legitimacy among the public.

    “This is because they might use this information to make more well-informed decisions, thereby leading to higher-quality decisions that result in higher perceived legitimacy among the public.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Binns et al. (2018) have examined how different kinds of explanations affect the fairness judgments of the general public.

    “In particular, Binns et al. (2018) have examined how different kinds of explanations affect the fairness judgments of the general public.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Binns et al. (2018) source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

    Interpretationreasoning

    The public will receive enough information to evaluate the desirability of the decision and, if they so like, demand accountability.

    “Furthermore, the public will receive enough information to evaluate the desirability of the decision and, if they so like, demand accountability.”
    Quote in the paperNot a substantial claim
    • The authors state this as an assumption: Sufficient mechanisms for accountability are in place.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    European Union lobbyists can be expected to have incentives to behave in a civilized manner when it comes to political matters.

    “He considers the example of European Union lobbyists, who can be expected to have incentives to behave in a civilized manner when it comes to political matters.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Naurin (2007)

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    If we received all information from Phases 1 and 2, we would have a lot of information to assimilate, and this problem becomes even worse in Phase 3.

    “This holds that if we received all of the information from Phases 1 and 2, we would have a lot of information to assimilate, and this problem becomes even worse in Phase 3.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites Eppler and Mengis 2004

      Not checked in the free run.

    • cites Richards and King 2013

      Not checked in the free run.

  • May combine two claims.

    Interpretationnone given

    Giving access to auditors, researchers, or other actors who can decipher algorithms may yield higher-quality decisions and a higher degree of perceived legitimacy.

    “Giving these groups access may yield higherquality decisions and a higher degree of perceived legitimacy.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Full transparency reveals the actual reality of decision-making, and real-world decision-making rarely or never lives up to the democratic or professional ideal.

    “The problem is that full transparency reveals the actual reality of decision-making and that real-world decision-making rarely or never lives up to the democratic or professional ideal (e.g., Tsoukas 1997).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Tsoukas 1997

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Recent empirical research has shown that it is far from evident that increased transparency generates trust or acceptance of public policies; in some cases, the effect can even be negative.

    “However, even though it is intuitive that transparent institutions are preferred over non-transparent ones and that they yield higher perceived legitimacy, recent empirical research (e.g., Grimmelikhuijsen 2012; de Fine Licht 2014) has shown that it is far from evident that increased transparency generates trust or acceptance of public policies.In some cases, the effect can even be negative.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites Grimmelikhuijsen 2012

      Not checked in the free run.

    • cites de Fine Licht 2014

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    In most cases, this information is sufficient for members of the public to form an opinion about the desirability of the decision and, if they so like, demand that the responsible actor be accountable.

    “In most cases, this information is sufficient for members of the public to form an opinion about the desirability of the decision and, if they so like, demand that the responsible actor be accountable”
    • Also stated in: § 4 The benefits of a justifications approach to AI transparency · plain
      “In most cases, this information is enough for members of the public to form an opinion about the desirability of the decision and, if they so like, demand accountability of the responsible actor”
    Quote in the paperNot a substantial claim
    • The authors state this as an assumption: Sufficient mechanisms for accountability are in place.

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationreasoning

    This is again because the public will get the information they need in a manageable way, meaning that they will not suffer from the information overload discussed in Sect.3.

    “This is again because the public will get the information they need in a manageable way, meaning that they will not suffer from the information overload discussed in Sect.3.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Interpretationnone given

    The general argument should also apply to cases of AI use in more private operations.

    “Although our main focus is the use of AI technology in public policymaking, authoritative public decision-making in relation to individuals as citizens, and the intermingling of these actors (i.e., the sociotechnical system), the general argument should also apply to cases of AI use in more private operations.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    In Phases 1 and 2, when decision-makers are transparent about the decision and the reasons for taking it, the public should have greater understanding of why these decisions were made, since the information about the decision will be more condensed, thus making it more accessible.

    “In Phases 1 and 2, when decision-makers are transparent about the decision and the reasons for taking it, the public should have greater understanding of why these decisions were made, since the information about the decision will be more condensed, thus making it more accessible.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    Several conditions must be met for an explanation/justification to have the intended effect: It must be exposed to the intended audience, the audience must pay attention to it, the audience must comprehend it, and the audience must accept the explanation/justification as legitimate.

    “As argued by McGraw et al. (1995), several conditions must be met for an explanation/justification to have the intended effect: It must be exposed to the intended audience, the audience must pay attention to it, the audience must comprehend it, and the audience must accept the explanation/justification as legitimate.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites McGraw et al. (1995) source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    These appear to be necessary conditions that make the receiver feel included and respected.

    “These appear to be necessary conditions that make the receiver feel included and respected.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationcitation

    To provide reasons for decisions signals that the decision-makers respect and care about the affected and may lead to more favorable interpretations regarding the motives and intentions of the decision-makers.

    “To provide reasons for decisions signals that the decision-makers respect and care about the affected (e.g., Tyler and Lind 1992) and may lead to more favorable interpretations regarding the motives and intentions of the decision-makers (e.g., Bies and Shapiro 1988;Shapiro et al. 1994;Colquitt 2001;Shaw et al. 2003).”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites Tyler and Lind 1992 source

      Not checked in the free run.

    • cites Bies and Shapiro 1988

      Not checked in the free run.

    • cites Shapiro et al. 1994

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationnone given

    There may be a reasonable level of public transparency in relation to the design of AI and the decisions made by AI.

    “In addition, we argue that there may be a reasonable level of public transparency in relation to the design of AI and the decisions made by AI.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    They must 'get dressed for politics' (i.e., present serious and well-reasoned arguments for their positions).

    “They must”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Naurin (2007)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    When keeping the process more secret, we can avoid many of the potentially negative effects of full transparency previously discussed such as disappointment regarding the decision-makers' behavior in decision-making situations.

    “At the same time, when keeping the process more secret, we can avoid many of the potentially negative effects of full transparency previously discussed such as disappointment regarding the decision-makers' behavior in decisionmaking situations.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationreasoning

    This will then lead to fairer decisions of higher quality, eventually leading to a higher degree of perceived legitimacy.

    “This will then lead to fairer decisions of higher quality, eventually leading to a higher degree of perceived legitimacy.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    Justifications need to be understandable and relevant to the decision at hand.

    “However, justifications need to be understandable and relevant to the decision at hand.”
    Quote in the paperNot a substantial claim
52 of 52

  • The paper relies on this without support.

    Relying on coercion is normatively problematic, costly, and highly inefficient, and perceived legitimacy is fundamental to the democratic system.

    “Since relying on coercion is not only normatively problematic but also costly and highly inefficient, perceived legitimacy is fundamental to the democratic system.”

  • The authors state this as an assumption.

    The public needs a chance to actually understand the code for transparency to have positive effects.

    “assuming that the public needs a chance to actually understand the code”

  • The paper relies on this without support.

    The general public will not understand the source code.

    “even though the general public will not understand it”

  • The paper relies on this without support.

    Perceived legitimacy is the goal of transparency in AI decision-making.

    “if perceived legitimacy is the goal”
    “When realizing perceived legitimacy, we should, as a default, opt for having our AI assistants explain themselves rather than open up their code, etc. for public scrutiny.”

  • The authors state this as an assumption.

    Sufficient mechanisms for accountability are in place.

    “given, of course, that sufficient mechanisms for accountability are in place”
    “given of course that sufficient mechanisms for accountability are in place”

  • The paper relies on this without support.

    The general public is not particularly interested in political matters.

    “For a general public that is not particularly interested in political matters”

  • The paper relies on this without support.

    Decision-makers are servants of the public.

    “their position as servants of the public”
7 of 7

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Providing a decision justification is needed to sustain government legitimacy.

    “In a conceptual piece on the legitimacy of algorithmic public decisions, they argue that providing a decision justification is needed to sustain government legitimacy.”

    From Explaining Why the Computer Says No: Algorithmic Transparency Affects the Perceived Trustworthiness of Automated Decision‐Making (2023) · cites this paper as De Fine Licht and De Fine Licht (2020)

    Section 4 argues that public decision justifications can enhance perceived legitimacy by enabling accountability and helping people feel included. But the paper presents this as a qualified proposal about perceived legitimacy in AI-assisted public decision-making, not as a demonstrated necessity for sustaining government legitimacy generally. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • Citizens want to receive clear reasons for a (negative) decision to assess the fairness of a decision or to be able to contest an algorithmic decision.

    “Citizens want to receive clear reasons for a (negative) decision to assess the fairness of a decision (De Fine Licht and De Fine Licht 2020; Tyler 2006) or to be able to contest an algorithmic decision (Mittelstadt et al. 2019).”

    From Explaining Why the Computer Says No: Algorithmic Transparency Affects the Perceived Trustworthiness of Automated Decision‐Making (2023) · cites this paper as De Fine Licht and De Fine Licht 2020

  • Reasons-focused transparency can be more intelligible and legitimacy-enhancing than indiscriminate disclosure of code or process detail.

    “This distinction accords with research showing that reasons-focused transparency can be more intelligible and legitimacy-enhancing than indiscriminate disclosure of code or process detail (de Fine Licht & de Fine Licht, 2020).”

    From Responsible AI in Polish Higher-Education Administration under the EU AI Act: High-Risk Use Cases, Fundamental-Rights Impact Assessment, and Institutional Accountability (2026) · cites this paper as de Fine Licht & de Fine Licht, 2020

  • Public dissatisfaction with responses to the black box problem may undermine the adoption of AI/ML technologies, especially when these technologies cause harm, invade privacy, or produce biased claims and results.

    “Public dissatisfaction with responses to the black box problem may undermine the adoption of AI/ML technologies, especially when these technologies cause harm, invade privacy, or produce biased claims and results [60, 85, 134, 175].”

    From The ethics of using artificial intelligence in scientific research: new guidance needed for a new tool (2025) · cites this paper as [134]

  • A limited form of transparency offering justification of decisions is more likely to increase perceived legitimacy of a system, rather than the more extensive transparency of the automated decision-making process itself attempted in this project.

    “It may also be the case that a limited form of transparency offering justification of decisions is more likely to increase perceived legitimacy of a system, rather than the more extensive transparency of the automated decision-making process itself attempted in this project (Fine Licht and Fine Licht 2020).”

    From Tensions in transparent urban AI: designing a smart electric vehicle charge point (2023) · cites this paper as Fine Licht and Fine Licht 2020

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • The public is assigned opposite principal-agent roles

    Minor

    The paper first identifies the public as the principal and the representative as the agent, but later calls the people the agent in the same principal-agent framework.

1 of 1

Cites

Flagged first, then by citations

4 of the 65 references couldn’t be matched to a known paper.

Research works printed but not linked: 5

  1. Fearon J (1998) Deliberation and discussion. In: Elster J (ed) Deliberative democracy. Cambridge University Press, Cambridge
  2. Fenster M (2006) The opacity of transparency. Iowa Law Rev 91(3):885–949
  3. Lepri B, Oliver N, Letouzé E, Pentland A, Vinck P (2017) Fair, transparent, and accountable algorithmic decision-making processes. Philos Technol 2017:1–17
  4. Thelisson E, Padh K, Celis LE (2017) Regulatory mechanisms and algorithms towards trust in AI/ML. In: Proceedings of the IJCAI 2017 workshop on explainable artificial intelligence (XAI), Melbourne, Australia
  5. Warren ME, Mansbridge J (2013) Deliberative Negotiation. In: Mansbridge J, Martin CJ (eds) Negotiating agreement in politics: report of the task force on negotiating in politics. American Political Science Association, Washington

Cited by

Sorted by citations

Related

Authors
de Fine Licht, K., & de Fine Licht, J.
Venue
AI & SOCIETY
Publisher
Springer Science and Business Media LLC
Published
2020-03-19
Volume
35
Issue
4
Pages
917-926
ISSN
  • 0951-5666 (ISSN-L)
  • 1435-5655
DOI
10.1007/s00146-020-00960-w
OpenAlex
W3011363941
Semantic Scholar
02471925d8cd9cc14bfb575bd8ef2ded74d2e86f
Cited by
334
References
65
Citations by year
2026: 83 · 2025: 78 · 2024: 56 · 2023: 49 · 2022: 41 · 2021: 21 · 2020: 6
FWCI
39.87
Type
journal article
Language
en
Open access
Licence
Locations
  • AI & Society (publishedVersion, cc-by)
  • Chalmers Research (Chalmers University of Technology) (submittedVersion)
Funders
Chalmers Tekniska Högskola
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Social Sciences › Political Science and International Relations › E-Government and Public Services
Keywords
transparency, perceived legitimacy, black box problem, accountability, justification of decisions, democratic legitimacy, public acceptance of AI

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-08.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-08.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review