'It's Reducing a Human Being to a Percentage': Perceptions of Justice in Algorithmic Decisions
Open accessPublished version of a preprint
Abstract
Data-driven decision-making consequential to individuals raises important questions of accountability and justice. Indeed, European law provides individuals limited rights to 'meaningful information about the logic' behind significant, autonomous decisions such as loan approvals, insurance quotes, and CV filtering. We undertake three experimental studies examining people's perceptions of justice in algorithmic decision-making under different scenarios and explanation styles. Dimensions of justice previously observed in response to human decision-making appear similarly engaged in response to algorithmic decisions. Qualitative analysis identified several concerns and heuristics involved in justice perceptions including arbitrariness, generalisation, and (in)dignity. Quantitative analysis indicates that explanation styles primarily matter to justice perceptions only when subjects are exposed to multiple different styles---under repeated exposure of one style, scenario effects obscure any explanation effects. Our results suggests there may be no 'best' approach to explaining algorithmic decisions, and that reflection on their automated nature both implicates and mitigates justice dimensions.
Verified 2026-10-06 · 5 references not checked yet
AI snapshot
Written from the paper’s full text.
TL;DR
Three experimental studies found that people apply justice dimensions to algorithmic decisions much as they do to human decisions, but explanation-style effects were mostly visible only when participants compared multiple styles for the same case. Case-based explanations were consistently associated with lower justice perceptions than sensitivity-based explanations, while repeated exposure to a single explanation style largely obscured explanation effects.
Key findings
- Justice constructs correlated similarly to the human decision-making literature: Spearman’s rho was 0.69 for fair process–deserved outcome, 0.60 for fair process–appropriate factors, 0.51 for appropriate factors–deserved outcome, 0.40 for fair process–understanding, 0.40 for appropriate factors–understanding, and 0.36 for understanding–deserved outcome.
- In the between-subjects study (325 participants, 5 conditions of n = 65), explanation styles generally did not significantly affect justice perceptions, except for fair process in loans, F(3, 258) = 2.71, p = 0.046, and appropriate factors, F(3, 258) = 5.35, p = 0.001.
- In the within-subjects study (65 participants), significant explanation effects appeared across all questions and contexts: fair process in loans, F(3, 260) = 7.52, p < .001, and insurance, F(3, 260) = 4.5, p = .004; appropriate factors in loans, F(3, 260) = 3.312, p = .02, and insurance, F(3, 260) = 6.44, p = .0003; deserved outcome in loans, F(3, 260) = 7.31, p = .0001. Case-based explanations produced lower appropriateness, fair process, and loan deservedness than sensitivity-based styles.
- Qualitative themes included lack of human touch/indignity, interpreting the system’s reasoning, acceptability of statistical inference and generalisation, actionability, unaccounted aspects, and the meaning/relevance of moral concepts; some participants saw the algorithmic nature as mitigating justice judgments.
Methods
- Three studies: Phase 1 was an in-person lab study with 19 UK participants, using semi-structured think-aloud interviews over 15 fictional negative-outcome cases across 5 contexts (loan, promotion, car insurance, airline re-routing, bank account freeze), each with one of 4 explanation styles (input influence, sensitivity, case-based, demographic) or a no-explanation control; data were thematically analysed.
- Phase 2 comprised two online Prolific Academic studies with UK participants over 18: a between-subjects study with 325 participants randomly assigned to 5 conditions (n = 65 each) and 12 cases with one explanation style or control, and a within-subjects study with 65 participants using only loan and insurance cases, presenting four different explanation styles for a single case in random order.
- Measures were agreement with five 5-point Likert justice statements (agreement, understanding, appropriateness of factors, fair process, deserved outcome); analysis used Spearman’s rho correlations and ANOVA with Tukey’s post-hoc paired tests.
Limitations
- Stated: samples were not representative—the lab study drew from an affluent, academically dominated UK city, and the online studies were not gender balanced, with almost double the number of females to males; scenarios were hypothetical and did not directly affect participants, lacking first-person consequences and options such as human review or appeal; explanations were not drawn from real machine learning model outputs; differences in explanation text length and format may have confounded results; further confirmatory work is needed on why explanation styles differ.
- Stated/evident: the airline context was excluded from the online studies because lab participants almost universally judged it very unfair and undeserved, preventing useful conclusions; the lab sample was small (19 participants), and online participants were limited to the UK.
- Evident: the between-subjects design repeated a single explanation style, which may have led to habituation, while the within-subjects design made style differences more salient through direct comparison; neither design may fully capture real-world exposure patterns.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
0 of 27 cited sources checked · 0 supported · 0 flagged
- Findingown data§ ABSTRACT · p. 1
Qualitative analysis identified several concerns and heuristics involved in justice perceptions, including arbitrariness, generalisation, and (in)dignity.
“Qualitative analysis identified several concerns and heuristics involved in justice perceptions including arbitrariness, generalisation, and (in)dignity.”
Quote in the paperSubstantial claim - Findingown data§ ABSTRACT · p. 1
Dimensions of justice previously observed in response to human decision-making appear similarly engaged in response to algorithmic decisions.
“Dimensions of justice previously observed in response to human decision-making appear similarly engaged in response to algorithmic decisions.”
Quote in the paperSubstantial claim - Findingown data§ Explanation styles · p. 4
The use of dialog boxes focused attention on explanations.
“Through iterative testing we found that the use of dialog boxes focused attention on explanations (see Figure 2).”
Quote in the paperSubstantial claim - Findingcitation§ The (lack of) human touch · p. 6
Some participants attempted to humanise or anthropomorphise the system.
“In contrast to this apparent lack of humanity (or perhaps to counteract it), some participants attempted to humanise or anthropomorphise the system [48], one participant for instance describing it as 'rude' ( OR ).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [48] 48. Clifford Nass, Jonathan Steuer, Ellen Tauber, and Heidi Reeder. 1993. Anthropomorphism, agency, and ethopoeia: computers as social actors. In INTERACT'93 and CHI'93 Conference Companion on Human Factors in Computing Systems . ACM, 111-112. source
Not checked in the free run.
- Rests on an assumption: The fictionalised scenarios used in the lab and online studies are realistic and representative enough to elicit valid responses about algorithmic decision-making.
- Rests on an assumption: The thematic analysis themes identified by the coders accurately represent participants' interpretations and reasoning.
- Findingown data§ Actionability · p. 7
In judging appropriateness of features or fairness of a decision, issues of personal responsibility and possibility of acting differently were frequently raised.
“In judging the appropriateness of certain features, or the fairness of a decision, issues of personal responsibility and the possibility of acting differently were frequently raised.”
Quote in the paperSubstantial claim- Rests on an assumption: The fictionalised scenarios used in the lab and online studies are realistic and representative enough to elicit valid responses about algorithmic decision-making.
- Rests on an assumption: The thematic analysis themes identified by the coders accurately represent participants' interpretations and reasoning.
- Findingown data§ Actionability · p. 7
Some participants were happy to endorse negative decisions if they could identify a reasonable alternative course of action available to the participant which would result in a different outcome, as in sensitivity-based explanations.
“Some participants were happy to endorse negative decisions if they were able to identify a reasonable alternative course of action available to the participant which would resulted in a different outcome, as in sensitivity -based explanations:”
Quote in the paperNot a substantial claim- Rests on an assumption: The fictionalised scenarios used in the lab and online studies are realistic and representative enough to elicit valid responses about algorithmic decision-making.
- Rests on an assumption: The thematic analysis themes identified by the coders accurately represent participants' interpretations and reasoning.
- Findingown data§ Explanation styles · p. 4
People generally did not pay attention to explanations independent of scenario descriptions.
“We found that people generally did not pay attention to explanations independent of scenario descriptions.”
Quote in the paperSubstantial claim - Findingown data§ The (lack of) human touch · p. 6
Many participants reacted to algorithmic decision-making as indignified or 'weird' compared to human decision-making.
“An initial reaction from many participants concerned the indignity or 'weirdness' of algorithmic decision-making compared to human decision-making.”
Quote in the paperSubstantial claim- Rests on an assumption: The fictionalised scenarios used in the lab and online studies are realistic and representative enough to elicit valid responses about algorithmic decision-making.
- Rests on an assumption: The thematic analysis themes identified by the coders accurately represent participants' interpretations and reasoning.
- Findingown data§ Justice constructs · p. 4
People tended to interpret the justice construct statements as a proxy for the simpler question of whether or not they agreed with the decision.
“We also found that people tended to interpret these justice construct statements as a proxy for the simpler question of whether or not they agreed with the decision;”
Quote in the paperSubstantial claim - Findingown data§ Acceptability of statistical inference · p. 7
Many participants expressed strong objections to the use of statistical inferences to make judgements about people in itself.
“However, many other participants expressed strong objections to the use of statistical inferences to make judgements about people in itself:”
Quote in the paperNot a substantial claim- Rests on an assumption: The fictionalised scenarios used in the lab and online studies are realistic and representative enough to elicit valid responses about algorithmic decision-making.
- Rests on an assumption: The thematic analysis themes identified by the coders accurately represent participants' interpretations and reasoning.
- Findingown data§ Quantitative results · p. 8
We also found significant positive correlations between the two construct connections we were not expecting.
“Surprisingly, we also found significant positive correlations between the two construct connections we were not expecting.”
Quote in the paperNot a substantial claim- Rests on an assumption: The justice constructs developed for human decision-making (informational, procedural, distributive) are applicable to and valid for evaluating algorithmic decision-making contexts.
- Rests on an assumption: Participants' Likert-scale responses to the justice statements accurately measure their perceived justice.
- Findingown data§ Interpreting the system's 'reasoning' · p. 7
Some subjects made their assessments based on whether the system approximated their own reasoning or knowledge.
“Some subjects made their assessments on the basis of whether the system approximated their own reasoning or knowledge:”
Quote in the paperNot a substantial claim- Rests on an assumption: The fictionalised scenarios used in the lab and online studies are realistic and representative enough to elicit valid responses about algorithmic decision-making.
- Rests on an assumption: The thematic analysis themes identified by the coders accurately represent participants' interpretations and reasoning.
- Findingown data§ Justice constructs · p. 4
Preliminary testing found that certain questions from Colquitt's scale, particularly those relating to interactional justice such as 'Have you been able to express your views and feelings during those procedures?' were difficult to interpret.
“Preliminary testing found that certain questions from Colquitt's scale, particularly those relating to interactional justice such as 'Have you been able to express your views and feelings during those procedures?' were difficult to interpret.”
Quote in the paperSubstantial claim - Findingown data§ Do different explanation styles result in differences in perceived levels of justice? · p. 9
In the within-subjects study, significant effects were observed across all questions and decision-making contexts.
“In the within-subjects study-where multiple explanation styles were presented for the same decision-significant effects were observed across all questions and decision-making contexts.”
Quote in the paperSubstantial claim- Rests on an assumption: The justice constructs developed for human decision-making (informational, procedural, distributive) are applicable to and valid for evaluating algorithmic decision-making contexts.
- Rests on an assumption: Participants' Likert-scale responses to the justice statements accurately measure their perceived justice.
- Rests on an assumption: The four explanation styles are sufficiently distinct and were correctly perceived by participants as intended.
- Rests on an assumption: The sample of Prolific Academic participants is similar enough to the lab study participants to allow comparison.
- Findingown data§ Do different explanation styles result in differences in perceived levels of justice? · p. 9
Case-based explanations seem to have the most consistent negative impact on justice perceptions when presented alongside alternative explanation styles.
“Case-based explanations seem to have the most consistent negative impact on justice perceptions when presented alongside alternative explanation styles.”
Quote in the paperSubstantial claim- Rests on an assumption: The justice constructs developed for human decision-making (informational, procedural, distributive) are applicable to and valid for evaluating algorithmic decision-making contexts.
- Rests on an assumption: Participants' Likert-scale responses to the justice statements accurately measure their perceived justice.
- Rests on an assumption: The four explanation styles are sufficiently distinct and were correctly perceived by participants as intended.
- Rests on an assumption: The sample of Prolific Academic participants is similar enough to the lab study participants to allow comparison.
- Findingown data§ Acceptability of statistical inference · p. 7
Participants commonly reflected on the (un)acceptability of generalisation and statistical inference as a basis for prediction and decision-making.
“In addition to concerns about indignity and opaque reasoning, participants commonly reflected on the (un)acceptability of generalisation and statistical inference as a basis for prediction and decision-making in general.”
Quote in the paperSubstantial claim- Rests on an assumption: The fictionalised scenarios used in the lab and online studies are realistic and representative enough to elicit valid responses about algorithmic decision-making.
- Rests on an assumption: The thematic analysis themes identified by the coders accurately represent participants' interpretations and reasoning.
- Findingown data§ DISCUSSION · p. 9
In the study discussions, people strongly engaged with the details of each explanation when discussing each case.
“people strongly engaged with the details of each explanation when discussing each case”
Quote in the paperSubstantial claim - Findingown data§ DISCUSSION · p. 9
In the within-subjects experiment, significant differences were observed in justice perceptions between different explanation styles.
“This was borne out in the within-subjects experiment, where significant differences were observed in justice perceptions between different explanation styles”
Quote in the paperSubstantial claim - Findingown data§ DISCUSSION · p. 10
In this study, the procedural and distributive justice dimensions had a correlation coefficient of 0.69.
“Procedural and distributive dimensions had a relatively high correlation coefficient of 0.69”
Quote in the paperNot a substantial claim - Findingown data§ DISCUSSION · p. 9
Case-based explanation styles impacted negatively on justice-related judgements, especially compared to sensitivity-based explanation styles.
“in particular that case-based explanation styles impacted negatively on justice-related judgements especially compared to sensitivity-based explanation styles”
Quote in the paperSubstantial claim - Findingown data§ DISCUSSION · p. 9
In the between-subjects design where individuals were exposed to only one explanation style repeated across multiple cases, explanation effects on justice judgements largely disappeared.
“in the between-subjects design where individuals were exposed to only one explanation style repeated across multiple cases, these explanation effects largely disappeared”
Quote in the paperSubstantial claim - Interpretationown data§ ABSTRACT · p. 1
Reflection on the automated nature of algorithmic decisions both implicates and mitigates justice dimensions.
“reflection on their automated nature both implicates and mitigates justice dimensions”
Quote in the paperSubstantial claim - Interpretationown data§ ABSTRACT · p. 1
There may be no 'best' approach to explaining algorithmic decisions.
“Our results suggests there may be no 'best' approach to explaining algorithmic decisions”
Quote in the paperSubstantial claim - Interpretationown data§ DISCUSSION · p. 9
Faced with a range of algorithmic decision-making scenarios, people appear conflicted about how to respond to questions of justice.
“Faced with a range of algorithmic decision-making scenarios, people appear conflicted about how to respond to questions of justice.”
Quote in the paperSubstantial claim- Rests on an assumption: The fictionalised scenarios used in the lab and online studies are realistic and representative enough to elicit valid responses about algorithmic decision-making.
- Rests on an assumption: The thematic analysis themes identified by the coders accurately represent participants' interpretations and reasoning.
- Premisecitation§ INTRODUCTION · p. 1
A significant body of research in legal and organisational psychology suggests that people do not only care about whether the outcome of a decision benefits them, but also whether it meets standards of justice.
“A significant body of research in legal and organisational psychology suggests that people do not only care about whether the outcome of a decision benefits them, but also whether it meets standards of justice [71, 40, 11].”
Quote in the paperSubstantial claim0 of 3 sources checked- cites [71] 71. John Thibaut, Laurens Walker, Stephen LaTour, and Pauline Houlden. 1973. Procedural justice as fairness. Stan. L. Rev. 26 (1973), 1271. source
Not checked in the free run.
- cites [40]
Not checked in the free run.
- cites [11] 11. Jason A Colquitt and Jessica B Rodell. 2015. Measuring justice and fairness. In Oxford Handbook of Justice in the Workplace , Russell Cropanzano and Maureen L Ambrose (Eds.). Oxford University Press, Oxford, 187-202.
Not checked in the free run.
- Premisecitation§ Perceptions of justice regarding decision-making · p. 3
Receiving a thorough explanation (informational justice) is important in helping people assess whether the decision-making procedure is just (procedural justice).
“Receiving a thorough explanation (informational justice) is important in helping people to assess whether the decision-making procedure is just (procedural justice) [10].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [10] 10. Jason A Colquitt, Donald E Conlon, Michael J Wesson, Christopher OLH Porter, and K Yee Ng. 2001. Justice at the Millennium: A Meta-Analytic Review of 25 Years of Organizational Justice Research. Journal of Applied Psychology 86, 3 (2001). DOI: http://dx.doi.org/10.1037//0021-9010.86.3.425 source
Not checked in the free run.
- Premisecitation§ INTRODUCTION · p. 1
In their basic form, algorithmic decision-making systems do not naturally provide information relevant to justice judgements.
“In their basic form, algorithmic decision-making systems do not naturally provide information relevant to justice judgements”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [6] 6. Jenna Burrell. 2016. How the machine 'thinks': Understanding opacity in machine learning algorithms. Big Data & Society 3, 1 (2016), 2053951715622512. source
Not checked in the free run.
- Premisecitation§ STUDY DESIGN AND METHODOLOGY · p. 3
Those machine learning model explanation systems which have been subjected to user evaluations often only test people's ability to predict a model's outputs, or answer questions about its logic.
“those which have often only test people's ability to predict a model's outputs, or answer questions about its logic (e.g. [39])”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [39] 39. Brian Y Lim, Anind K Dey, and Daniel Avrahami. 2009. Why and why not explanations improve the intelligibility of context-aware intelligent systems. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems . ACM, 2119-2128. source
Not checked in the free run.
- Premisecitation§ INTRODUCTION · p. 1
In their basic form, algorithmic decision-making systems threaten the capacity for accountability inherent in human decision-making.
“threaten the capacity for accountability inherent in human decision-making”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [6] 6. Jenna Burrell. 2016. How the machine 'thinks': Understanding opacity in machine learning algorithms. Big Data & Society 3, 1 (2016), 2053951715622512. source
Not checked in the free run.
- Premisecitation§ Perceptions of justice regarding decision-making · p. 3
Decisions perceived to be procedurally just are more likely to be perceived as distributively just.
“In turn, decisions perceived to be procedurally just are more likely to be perceived as distributively just [79].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [79] 79. Kees Van den Bos, Lynn Van der Velden, and Allan Lind. 2014. On the role of perceived procedural justice in citizens' reactions to government decisions and the handling of conflicts. Utrecht Law Review 10, 4 (2014). DOI: http://dx.doi.org/10.18352/ulr.287
Not checked in the free run.
- Premisecitation§ Scenarios and application contexts · p. 4
Considerations of procedural justice have been found to be more strongly felt in cases where a decision has a negative outcome for the individual.
“considerations of procedural justice have been found to be more strongly felt in such cases [4]”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [4] 4. Joel Brockner and Batia M Wiesenfeld. 1996. An integrative framework for explaining reactions to decisions: interactive effects of outcomes and procedures. Psychological bulletin 120, 2 (1996), 189.
Not checked in the free run.
- Premisecitation§ INTRODUCTION · p. 1
The European Union's General Data Protection Regulation requires organisations deploying certain systems to provide affected individuals with 'meaningful information about the logic' behind their outputs.
“The European Union's General Data Protection Regulation (GDPR) (and similarly, the preceding 1995 Directive) requires organisations deploying certain systems to provide affected individuals with 'meaningful information about the logic' behind their outputs [81].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [81] 81. Sandra Wachter, Brent Mittelstadt, and Luciano Floridi. 2017. Why a right to explanation of automated decision-making does not exist in the General Data Protection Regulation. International Data Privacy Law 7, 2 (2017), 76-99. DOI: http://dx.doi.org/10.1093/idpl/ipx005 source
Not checked in the free run.
- Interpretationown data§ CONCLUSION · p. 10
People do consider justice-related aspects of algorithmic decision-making systems, much as they do for manual decision-making processes.
“This paper suggests that people do consider justice-related aspects of algorithmic decision-making systems, much as they do for manual decision-making processes.”
Quote in the paperSubstantial claim - Interpretationown data§ DISCUSSION · p. 10
The observed correlations between different justice constructs fall largely in line with those observed in the psychology of justice literature amassed since the 1970s.
“the observed correlations between different justice constructs fall largely in line with those observed within the psychology of justice literature amassed since the 1970s”
Quote in the paperNot a substantial claim - Interpretationcitation§ DISCUSSION · p. 10
Strongly-held intuitions about the fairness of using particular features in a model, regardless of their weights, may account for the consistency in justice-related judgements across explanation styles.
“If people have strong intuitions about the fairness of using particular features in a model (regardless of their weights), as has been observed in a previous study by Grgic et al [28], this may account for such consistency.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Grgic et al [28]
Not checked in the free run.
- Study design and conductreasoning§ Threats to validity · p. 10
The explanations used in the studies were not drawn from real machine learning model outputs.
“The explanations were also not drawn from real machine learning model outputs”
Quote in the paperSubstantial claim - Premisecitation§ DISCUSSION · p. 9
People frequently interact with computers as if they were social actors.
“On the other hand, people frequently interact with computers as if they were social actors [49]; why stop when justice is involved?”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [49] 49. Clifford Nass, Jonathan Steuer, and Ellen R Tauber. 1994. Computers are social actors. In Proceedings of the SIGCHI conference on Human factors in computing systems . ACM, 72-78. source
Not checked in the free run.
- Study design and conductreasoning§ Threats to validity · p. 10
The samples in these studies were not representative of the general population.
“First, the samples were not representative of the general population.”
Quote in the paperNot a substantial claim - Interpretationcitation§ DISCUSSION · p. 9
Such attempts to delineate moral culpability between systems and people, or between different standards of fairness or evidence, may reflect a more general psychological need to see the world as morally coherent.
“Such attempts to delineate moral culpability between systems and people , or between different standards of fairness or evidence, may reflect a more general psychological need to see the world as morally coherent [9].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [9] 9. Cory J Clark, Eric Evan Chen, and Peter H Ditto. 2015. Moral coherence processes: Constructing culpability and consequences. Current Opinion in Psychology 6 (2015), 123-128. source
Not checked in the free run.
- Rests on an assumption: The fictionalised scenarios used in the lab and online studies are realistic and representative enough to elicit valid responses about algorithmic decision-making.
- Rests on an assumption: The thematic analysis themes identified by the coders accurately represent participants' interpretations and reasoning.
- Study design and conductnone given§ Threats to validity · p. 10
Participants in the online studies were not gender balanced, with almost double the number of females to males.
“Participants in the online studies were not gender balanced, with almost double the number of females to males.”
Quote in the paperNot a substantial claim - Interpretationown data§ CONCLUSION · p. 10
Depending on how and when they are deployed, explanations may or may not help individuals to evaluate the fairness of algorithmic decisions.
“depending on how and when they are deployed, explanations may or may not help individuals to evaluate the fairness of such decisions”
Quote in the paperSubstantial claim - Premisecitation§ DISCUSSION · p. 10
A 2001 meta-analysis of 45 studies in a variety of organisational and institutional settings reported a correlation coefficient of 0.48 for the procedural and distributive justice constructs.
“a 2001 meta-analysis of 45 studies in a variety of organisational and institutional settings reported a correlation coefficient of 0.48 for the same constructs [10]”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [10] 10. Jason A Colquitt, Donald E Conlon, Michael J Wesson, Christopher OLH Porter, and K Yee Ng. 2001. Justice at the Millennium: A Meta-Analytic Review of 25 Years of Organizational Justice Research. Journal of Applied Psychology 86, 3 (2001). DOI: http://dx.doi.org/10.1037//0021-9010.86.3.425 source
Not checked in the free run.
- Premisecitation§ DISCUSSION · p. 10
Research on justice judgements in non-automated contexts has found that individuals pay more attention to other people's outcomes than to information about the procedure behind the decision that affects them.
“Research on justice judgements in non-automated contexts has found that individuals pay more attention to other people's outcomes than they do to information about the procedure behind the decision that affects them [78, 84].”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [78] 78. Kees Van den Bos, E Allan Lind, Riël V ermunt, and Henk AM Wilke. 1997. How do I judge my outcome when I do not know the outcome of others? The psychology of the fair process effect. Journal of Personality and Social Psychology 72, 5 (1997), 1034. source
Not checked in the free run.
- cites [84] 84. Lan Xia, Kent B Monroe, and Jennifer L Cox. 2004. The price is unfair! A conceptual framework of price fairness perceptions. Journal of Marketing 68, 4 (2004), 1-15. source
Not checked in the free run.
- May combine two claims. 25
- Classified as probably not a substantial claim. 47
May combine two claims.
Findingown data§ ABSTRACT · p. 1Quantitative analysis indicates that explanation styles primarily matter to justice perceptions only when subjects are exposed to multiple different styles; under repeated exposure of one style, scenario effects obscure any explanation effects.
“Quantitative analysis indicates that explanation styles primarily matter to justice perceptions only when subjects are exposed to multiple different styles-under repeated exposure of one style, scenario effects obscure any explanation effects.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown data§ Acceptability of statistical inference · p. 7Some participants were more comfortable with inductive inferences; AS agreed with the computer's decision to deny a loan applicant with existing debts because they had already 'proven not to be able to pay it back'.
“However in this case, some participants were more comfortable with inductive inferences; AS agreed with the computer's decision to deny a loan applicant with existing debts because they had already 'proven not to be able to pay it back'.”
Quote in the paperNot a substantial claim- Rests on an assumption: The fictionalised scenarios used in the lab and online studies are realistic and representative enough to elicit valid responses about algorithmic decision-making.
- Rests on an assumption: The thematic analysis themes identified by the coders accurately represent participants' interpretations and reasoning.
May combine two claims.
Findingown data§ Within-subjects study · p. 5The airline context was excluded at this stage, because experiment 1 had revealed that participants almost universally judged the cases to be very unfair and undeserved, thus preventing any useful conclusions to be drawn from this context.
“The airline context was excluded at this stage, because experiment 1 had revealed that participants almost universally judged the cases to be very unfair and undeserved, thus preventing any useful conclusions to be drawn from this context.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown data§ Actionability · p. 7Where an individual was deemed at fault for risky or otherwise undesirable past behaviour, participant AS believed the decision was particularly justified.
“Where an individual was deemed at fault for risky or otherwise undesirable past behaviour, participant AS believed the decision was particularly justified:”
Quote in the paperNot a substantial claim- Rests on an assumption: The fictionalised scenarios used in the lab and online studies are realistic and representative enough to elicit valid responses about algorithmic decision-making.
- Rests on an assumption: The thematic analysis themes identified by the coders accurately represent participants' interpretations and reasoning.
Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing).
Findingown data§ Meaning and relevance of moral concepts · p. 8One participant argued that a truly random system could also be seen as more 'fair', at least for certain definitions of fairness.
“One participant argued that a truly random system could also be seen as more 'fair', at least for certain definitions of fairness:”
Quote in the paperNot a substantial claim- Rests on an assumption: The fictionalised scenarios used in the lab and online studies are realistic and representative enough to elicit valid responses about algorithmic decision-making.
- Rests on an assumption: The thematic analysis themes identified by the coders accurately represent participants' interpretations and reasoning.
May combine two claims.
Findingown data§ Do different explanation styles result in differences in perceived levels of justice? · p. 9In the between-subjects study, explanation styles generally did not significantly affect justice perceptions, with the exception of fair process in loans only and appropriate factors showing significant between group differences.
“In the between-subjects study, explanation styles generally did not significantly affect justice perceptions, with the exception of fair process in loans only (loan F(3, 258) = 2.71, p = 0.046) and appropriate factors (F(3, 258) = 5.35, p = 0.001) showing significant between group differences.”
Quote in the paperSubstantial claim- Rests on an assumption: The justice constructs developed for human decision-making (informational, procedural, distributive) are applicable to and valid for evaluating algorithmic decision-making contexts.
- Rests on an assumption: Participants' Likert-scale responses to the justice statements accurately measure their perceived justice.
- Rests on an assumption: The four explanation styles are sufficiently distinct and were correctly perceived by participants as intended.
- Rests on an assumption: The sample of Prolific Academic participants is similar enough to the lab study participants to allow comparison.
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingnone given§ Unaccounted aspects · p. 8In some cases, what was missing was not just information, but also complexity.
“In some cases, what was missing was not just information, but also complexity:”
Quote in the paperNot a substantial claim- Rests on an assumption: The fictionalised scenarios used in the lab and online studies are realistic and representative enough to elicit valid responses about algorithmic decision-making.
- Rests on an assumption: The thematic analysis themes identified by the coders accurately represent participants' interpretations and reasoning.
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingown data§ DISCUSSION · p. 9For some participants, the very idea of an algorithmic system making an important decision on the basis of past data seemed unfair.
“For some participants, the very idea of an algorithmic system making an important decision on the basis of past data seemed unfair;”
Quote in the paperNot a substantial claim- Rests on an assumption: The fictionalised scenarios used in the lab and online studies are realistic and representative enough to elicit valid responses about algorithmic decision-making.
- Rests on an assumption: The thematic analysis themes identified by the coders accurately represent participants' interpretations and reasoning.
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingown data§ Unaccounted aspects · p. 8In others, the missing element was a sense of proportionality.
“In others, the missing element was a sense of proportionality.”
Quote in the paperNot a substantial claim- Rests on an assumption: The fictionalised scenarios used in the lab and online studies are realistic and representative enough to elicit valid responses about algorithmic decision-making.
- Rests on an assumption: The thematic analysis themes identified by the coders accurately represent participants' interpretations and reasoning.
Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing). May combine two claims.
Findingown data§ DISCUSSION · p. 9For others, delegation of such responsibility to a computer, and the authority of statistical models, actually removed the relevance of questions of fairness altogether; the system 'just does what it is supposed to', and as long as its inferences are accurate, it could be seen as 'statistically fair'.
“Whereas for others, delegation of such responsibility to a computer, and the authority of statistical models, actually removed the relevance of questions of fairness altogether; the system 'just does what it is supposed to', and as long as its inferences are accurate, it could be seen as 'statistically fair' ( PC ).”
Quote in the paperNot a substantial claim- Rests on an assumption: The fictionalised scenarios used in the lab and online studies are realistic and representative enough to elicit valid responses about algorithmic decision-making.
- Rests on an assumption: The thematic analysis themes identified by the coders accurately represent participants' interpretations and reasoning.
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingown data§ Meaning and relevance of moral concepts · p. 8The meaning and relevance of moral concepts like 'fairness' and 'desert' in the context of a computer system was occasionally questioned by certain participants.
“The meaning and relevance of moral concepts like 'fairness' and 'desert' in the context of a computer system was occasionally questioned by certain participants.”
Quote in the paperNot a substantial claim- Rests on an assumption: The fictionalised scenarios used in the lab and online studies are realistic and representative enough to elicit valid responses about algorithmic decision-making.
- Rests on an assumption: The thematic analysis themes identified by the coders accurately represent participants' interpretations and reasoning.
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingown data§ Interpreting the system's 'reasoning' · p. 7In some cases, while individual premises seemed understandable, their relevance to the overall decision was not.
“In some cases, while the individual premises seemed understandable, their relevance to the reasoning behind the overall decision was not:”
Quote in the paperNot a substantial claim- Rests on an assumption: The fictionalised scenarios used in the lab and online studies are realistic and representative enough to elicit valid responses about algorithmic decision-making.
- Rests on an assumption: The thematic analysis themes identified by the coders accurately represent participants' interpretations and reasoning.
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingown data§ Acceptability of statistical inference · p. 7Issues raised included the importance of scientific rigour, including the use of sufficient sample sizes.
“Issues raised in this regard were the importance of scientific rigour, including the use of sufficient sample sizes:”
Quote in the paperNot a substantial claim- Rests on an assumption: The fictionalised scenarios used in the lab and online studies are realistic and representative enough to elicit valid responses about algorithmic decision-making.
- Rests on an assumption: The thematic analysis themes identified by the coders accurately represent participants' interpretations and reasoning.
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingown data§ Unaccounted aspects · p. 8Many criticisms of the decisions came down to things that were missing or unaccounted for.
“Many criticisms of the decisions came down to things that were missing or unaccounted for.”
Quote in the paperNot a substantial claim- Rests on an assumption: The fictionalised scenarios used in the lab and online studies are realistic and representative enough to elicit valid responses about algorithmic decision-making.
- Rests on an assumption: The thematic analysis themes identified by the coders accurately represent participants' interpretations and reasoning.
May combine two claims.
Findingown data§ Quantitative results · p. 8The results partially confirmed our hypotheses, as the stronger correlations were connected to the construct connections we believed would exist.
“The results (see Table 1) partially confirmed our hypotheses, as the stronger correlations were connected to the construct connections we believed would exist.”
Quote in the paperSubstantial claim- Rests on an assumption: The justice constructs developed for human decision-making (informational, procedural, distributive) are applicable to and valid for evaluating algorithmic decision-making contexts.
- Rests on an assumption: Participants' Likert-scale responses to the justice statements accurately measure their perceived justice.
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingown data§ Acceptability of statistical inference · p. 7Participants also commented on the acceptability of judging a single individual's future behaviour on the basis of their past behaviour.
“In addition to comparisons between different individuals, participants also commented on the acceptability of judging a single individual's future behaviour on the basis of their past behaviour.”
Quote in the paperNot a substantial claim- Rests on an assumption: The fictionalised scenarios used in the lab and online studies are realistic and representative enough to elicit valid responses about algorithmic decision-making.
- Rests on an assumption: The thematic analysis themes identified by the coders accurately represent participants' interpretations and reasoning.
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingown data§ Actionability · p. 8Participant CS was concerned that behaviour change in response to an algorithm could prove dangerous.
“Although participant CS was concerned that behaviour change in response to an algorithm could prove dangerous”
Quote in the paperNot a substantial claim- Rests on an assumption: The fictionalised scenarios used in the lab and online studies are realistic and representative enough to elicit valid responses about algorithmic decision-making.
- Rests on an assumption: The thematic analysis themes identified by the coders accurately represent participants' interpretations and reasoning.
May combine two claims.
Findingown data§ Do different explanation styles result in differences in perceived levels of justice? · p. 9Tukey's post-hoc paired tests showed that case-based explanations result in lower perceptions of appropriateness, fair process perception, and (in the loans case) deservedness, consistently compared to sensitivity based styles and occasionally compared to other styles.
“Tukey's post-hoc paired tests (see Figure 3) showed that casebased explanations result in lower perceptions of appropriateness, fair process perception, and (in the loans case) deservedness, consistently compared to sensitivity based styles and occasionally compared to other styles.”
Quote in the paperSubstantial claim- Rests on an assumption: The justice constructs developed for human decision-making (informational, procedural, distributive) are applicable to and valid for evaluating algorithmic decision-making contexts.
- Rests on an assumption: Participants' Likert-scale responses to the justice statements accurately measure their perceived justice.
- Rests on an assumption: The four explanation styles are sufficiently distinct and were correctly perceived by participants as intended.
- Rests on an assumption: The sample of Prolific Academic participants is similar enough to the lab study participants to allow comparison.
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingown data§ Actionability · p. 7Several responses suggested that explanations ought to help guide future action, something particularly lacking in demographic explanations.
“Several responses suggested that explanations ought to help guide future action, something which was particularly lacking in the demographic explanations:”
Quote in the paperNot a substantial claim- Rests on an assumption: The fictionalised scenarios used in the lab and online studies are realistic and representative enough to elicit valid responses about algorithmic decision-making.
- Rests on an assumption: The thematic analysis themes identified by the coders accurately represent participants' interpretations and reasoning.
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingown data§ Interpreting the system's 'reasoning' · p. 7Others tried to articulate the system's reasoning as a series of rules and consequences.
“Others tried to articulate the system's reasoning as a series of rules and consequences:”
Quote in the paperNot a substantial claim- Rests on an assumption: The fictionalised scenarios used in the lab and online studies are realistic and representative enough to elicit valid responses about algorithmic decision-making.
- Rests on an assumption: The thematic analysis themes identified by the coders accurately represent participants' interpretations and reasoning.
May combine two claims.
Findingown data§ Do different explanation styles result in differences in perceived levels of justice? · p. 9There was a significant effect of explanation styles on perceptions of fair process, on the perception of appropriate use of factors, and on the perception of deserved outcome (only in the loans case).
“There was a significant effect of explanation styles on perceptions of fair process (loan scenario F(3, 260) = 7.52, p < .001; insurance scenario F(3, 260) = 4.5, p = .004), on the perception of appropriate use of factors (loan F(3, 260) = 3.312, p = .02; insurance F(3, 260) = 6.44, p = .0003), and on the perception of deserved outcome (only in the loans case, F(3, 260) = 7.31, p = .0001).”
Quote in the paperSubstantial claim- Rests on an assumption: The justice constructs developed for human decision-making (informational, procedural, distributive) are applicable to and valid for evaluating algorithmic decision-making contexts.
- Rests on an assumption: Participants' Likert-scale responses to the justice statements accurately measure their perceived justice.
- Rests on an assumption: The four explanation styles are sufficiently distinct and were correctly perceived by participants as intended.
- Rests on an assumption: The sample of Prolific Academic participants is similar enough to the lab study participants to allow comparison.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown data§ The (lack of) human touch · p. 6For one participant, the specificity of figures in input influence and sensitivity explanations led her to conclude the computer must be using arbitrary thresholds, which seemed 'mean'.
“For the same participant, the specificity of the figures used in some explanations (particularly, input influence and sensitivity ) lead her to conclude that the computer must be using arbitrary thresholds, which in itself seemed 'mean':”
Quote in the paperNot a substantial claim- Rests on an assumption: The fictionalised scenarios used in the lab and online studies are realistic and representative enough to elicit valid responses about algorithmic decision-making.
- Rests on an assumption: The thematic analysis themes identified by the coders accurately represent participants' interpretations and reasoning.
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingown data§ Interpreting the system's 'reasoning' · p. 7Many reactions concerned attempts to decipher the logic of the algorithm.
“Many of the reactions concerned attempts to decipher the logic of the algorithm.”
Quote in the paperNot a substantial claim- Rests on an assumption: The fictionalised scenarios used in the lab and online studies are realistic and representative enough to elicit valid responses about algorithmic decision-making.
- Rests on an assumption: The thematic analysis themes identified by the coders accurately represent participants' interpretations and reasoning.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown data§ Interpreting the system's 'reasoning' · p. 7Faced with seemingly inexplicable reasoning processes, some participants concluded the system must be simply 'making it up' and therefore on the face of it unfair.
“Faced with seemingly inexplicable reasoning processes, some participants concluded that the system must be simply 'making it up' ( OR ) and therefore on the face of it unfair.”
Quote in the paperNot a substantial claim- Rests on an assumption: The fictionalised scenarios used in the lab and online studies are realistic and representative enough to elicit valid responses about algorithmic decision-making.
- Rests on an assumption: The thematic analysis themes identified by the coders accurately represent participants' interpretations and reasoning.
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingown data§ The (lack of) human touch · p. 6Some participants considered it problematic that with an automated system there is no sense of negotiation and no opportunity for human interaction.
“Others considered it problematic that with an automated system, 'there's no sense of negotiation' ( DR ), and no opportunity for 'human interaction' ( CS ).”
Quote in the paperNot a substantial claim- Rests on an assumption: The fictionalised scenarios used in the lab and online studies are realistic and representative enough to elicit valid responses about algorithmic decision-making.
- Rests on an assumption: The thematic analysis themes identified by the coders accurately represent participants' interpretations and reasoning.
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingown data§ Unaccounted aspects · p. 8For some, it was the absence of important information.
“For some, it was the absence of important information:”
Quote in the paperNot a substantial claim- Rests on an assumption: The fictionalised scenarios used in the lab and online studies are realistic and representative enough to elicit valid responses about algorithmic decision-making.
- Rests on an assumption: The thematic analysis themes identified by the coders accurately represent participants' interpretations and reasoning.
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingown data§ Meaning and relevance of moral concepts · p. 8Participant PC struggled to evaluate the moral dimensions of computer-driven decisions.
“Participant PC struggled to evaluate the moral dimensions of computer-driven decisions:”
Quote in the paperNot a substantial claim- Rests on an assumption: The fictionalised scenarios used in the lab and online studies are realistic and representative enough to elicit valid responses about algorithmic decision-making.
- Rests on an assumption: The thematic analysis themes identified by the coders accurately represent participants' interpretations and reasoning.
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingown data§ The (lack of) human touch · p. 6Some participants commented that the use of an algorithmic decision-making system could be impersonal and dehumanising for the subject of the decision.
“A series of comments reflected upon the ways in which the very use of an algorithmic decision-making system could be impersonal and dehumanising for the subject of the decision.”
Quote in the paperNot a substantial claim- Rests on an assumption: The fictionalised scenarios used in the lab and online studies are realistic and representative enough to elicit valid responses about algorithmic decision-making.
- Rests on an assumption: The thematic analysis themes identified by the coders accurately represent participants' interpretations and reasoning.
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingown data§ Acceptability of statistical inference · p. 7Some emphasised that an individual who seems similar to past individuals may in fact have extenuating circumstances, particularly in response to case-based explanations.
“Some emphasised that an individual who seems similar to past individuals may in fact have extenuating circumstances, particularly in response to case-based explanations:”
Quote in the paperNot a substantial claim- Rests on an assumption: The fictionalised scenarios used in the lab and online studies are realistic and representative enough to elicit valid responses about algorithmic decision-making.
- Rests on an assumption: The thematic analysis themes identified by the coders accurately represent participants' interpretations and reasoning.
May combine two claims.
Findingown data§ Qualitative Results: Themes and reflections · p. 6The think-aloud responses from the lab study resulted in five major themes: the lack of human touch; interpretations of the system's reasoning; the use of statistical inference; the degree of actionability in an explanation; important aspects which were seen to be unaccounted for by the system; and the meaning and relevance of moral concepts.
“The think-aloud responses from the lab study resulted in five major themes: the lack of human touch; interpretations of the system's reasoning; the use of statistical inference; the degree of actionability in an explanation; important aspects which were seen to be unaccounted for by the system; and the meaning and relevance of moral concepts.”
Quote in the paperSubstantial claim- Rests on an assumption: The fictionalised scenarios used in the lab and online studies are realistic and representative enough to elicit valid responses about algorithmic decision-making.
- Rests on an assumption: The thematic analysis themes identified by the coders accurately represent participants' interpretations and reasoning.
Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing).
Findingown data§ Meaning and relevance of moral concepts · p. 8For DR, there was a sense that the very notion of 'fairness' may not be attributable to a system primarily designed to serve a different organisational goal like efficiency.
“For DR , here was a sense that the very notion of 'fairness' may not be attributable to a system that is primarily designed to serve a different organisational goal like efficiency:”
Quote in the paperNot a substantial claim- Rests on an assumption: The fictionalised scenarios used in the lab and online studies are realistic and representative enough to elicit valid responses about algorithmic decision-making.
- Rests on an assumption: The thematic analysis themes identified by the coders accurately represent participants' interpretations and reasoning.
Classified as probably not a substantial claim. May combine two claims.
Findingnone given§ Actionability · p. 7Conversely, where a judgement was made on the basis of circumstances outside the individual's control, or the course of action suggested was deemed unrealistic, the outcome was seen as undeserved.
“Conversely, where a judgement was made on the basis of circumstances which were outside the individual's control, or the course of action suggested was deemed unrealistic, the outcome was seen as undeserved:”
Quote in the paperSubstantial claim- Rests on an assumption: The fictionalised scenarios used in the lab and online studies are realistic and representative enough to elicit valid responses about algorithmic decision-making.
- Rests on an assumption: The thematic analysis themes identified by the coders accurately represent participants' interpretations and reasoning.
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingown data§ DISCUSSION · p. 9One participant described how she might have it both ways, simultaneously entertaining dual standards of justice, one juridical, the other computational.
“One participant described how she might have it both ways, simultaneously entertaining dual standards of justice, one juridical, the other computational;”
Quote in the paperNot a substantial claim- Rests on an assumption: The fictionalised scenarios used in the lab and online studies are realistic and representative enough to elicit valid responses about algorithmic decision-making.
- Rests on an assumption: The thematic analysis themes identified by the coders accurately represent participants' interpretations and reasoning.
Classified as probably not a substantial claim. May combine two claims.
Interpretationcitation§ Perceptions of justice regarding decision-making · p. 3Requiring decision-makers to explain their decisions should promote informational justice, which should help people assess procedural justice and distributive justice in turn.
“Such findings provide a rationale for giving individuals rights to information about significant decisions; requiring decision-makers to explain their decisions should promote informational justice, which should help people assess procedural justice and distributive justice in turn.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S4 (based). May combine two claims.
Premisenone given§ Key questions and contributions · p. 3Work on perceptions of justice reveals much about the role of explanations in ethical assessments of human decisions, but the applicability of these findings to algorithmic decisions and the range of proposed explanation systems proposed for them remains largely unexplored.
“Work on perceptions of justice reveals much about the role of explanations in ethical assessments of human decisions, but the applicability of these findings to algorithmic decisions and the range of proposed explanation systems proposed for them remains largely unexplored.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable). May combine two claims.
Interpretationown data§ The (lack of) human touch · p. 7Such remarks suggest that if algorithmic decisions are held to norms of social behaviour, interactional justice may be a relevant dimension for evaluation.
“Such remarks suggest that if algorithmic decisions are held to norms of social behaviour, interactional justice may be a relevant dimension for evaluation.”
Quote in the paperNot a substantial claim- Rests on an assumption: The fictionalised scenarios used in the lab and online studies are realistic and representative enough to elicit valid responses about algorithmic decision-making.
- Rests on an assumption: The thematic analysis themes identified by the coders accurately represent participants' interpretations and reasoning.
Classified as probably not a substantial claim.
Premisecitation§ Perceptions of justice regarding decision-making · p. 3Justice perceptions can be broken down into procedural, distributive, interactional, and informational aspects.
“According to Colquitt and others, justice perceptions can be broken down into several aspects or antecedents [10].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [10] 10. Jason A Colquitt, Donald E Conlon, Michael J Wesson, Christopher OLH Porter, and K Yee Ng. 2001. Justice at the Millennium: A Meta-Analytic Review of 25 Years of Organizational Justice Research. Journal of Applied Psychology 86, 3 (2001). DOI: http://dx.doi.org/10.1037//0021-9010.86.3.425 source
Not checked in the free run.
Classified as probably not a substantial claim. May combine two claims.
Premisecitation§ Quantitative results · p. 8According to the literature on justice perceptions, we should expect to find correlations between fair process and deserved outcome, understanding and fair process, appropriateness of factors and deserved outcome, and appropriateness of factors and fair process.
“According to the literature on justice perceptions above, we should expect to find correlations between fair process and deserved outcome , understanding and fair process , appropriateness of factors and deserved outcome , and appropriateness of factors and fair process [10].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [10] 10. Jason A Colquitt, Donald E Conlon, Michael J Wesson, Christopher OLH Porter, and K Yee Ng. 2001. Justice at the Millennium: A Meta-Analytic Review of 25 Years of Organizational Justice Research. Journal of Applied Psychology 86, 3 (2001). DOI: http://dx.doi.org/10.1037//0021-9010.86.3.425 source
Not checked in the free run.
- Rests on an assumption: The justice constructs developed for human decision-making (informational, procedural, distributive) are applicable to and valid for evaluating algorithmic decision-making contexts.
May combine two claims.
Interpretationown data§ Do different explanation styles result in differences in perceived levels of justice? · p. 9This is an effect primarily observed in the within subject study design, indicating that the act of comparison in a particular scenario is important for these differences to become apparent.
“This is an effect primarily observed, like most effects in the quantitative part of our study, in the within subject study design, indicating that the act of comparison in a particular scenario is important for these differences to become apparent.”
Quote in the paperSubstantial claim- Rests on an assumption: The justice constructs developed for human decision-making (informational, procedural, distributive) are applicable to and valid for evaluating algorithmic decision-making contexts.
- Rests on an assumption: Participants' Likert-scale responses to the justice statements accurately measure their perceived justice.
- Rests on an assumption: The four explanation styles are sufficiently distinct and were correctly perceived by participants as intended.
- Rests on an assumption: The sample of Prolific Academic participants is similar enough to the lab study participants to allow comparison.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductnone given§ Phase 1: Lab study · p. 5Thematic codes were independently developed by three coders, after which a single set was jointly agreed upon, and the data were re-coded using the new set of codes.
“Thematic codes were independently developed by three coders, after which a single set was jointly agreed upon. The data were then re-coded using the new set of codes.”
Quote in the paperNot a substantial claim- Rests on an assumption: The thematic analysis themes identified by the coders accurately represent participants' interpretations and reasoning.
May combine two claims.
Interpretationcitation§ DISCUSSION · p. 9While algorithmic decision-making implicates dimensions of justice, it may also mitigate them, in so far as people resist imputing morality to a computer system.
“In this sense, while algorithmic decision-making implicates dimensions of justice, it may also mitigate them, in so far as people resist imputing morality to a computer system [25, 5].”
Quote in the paperSubstantial claim0 of 2 sources checked- cites [25] 25. Batya Friedman and Peter H Kahn. 1992. Human agency and responsible computing: Implications for computer system design. Journal of Systems and Software 17, 1 (1992), 7-14. source
Not checked in the free run.
- cites [5] 5. Joanna J Bryson. 2010. Robots should be slaves. Close Engagements with Artificial Companions: Key social, psychological, ethical and design issues (2010), 63-74.
Not checked in the free run.
- Rests on an assumption: The fictionalised scenarios used in the lab and online studies are realistic and representative enough to elicit valid responses about algorithmic decision-making.
- Rests on an assumption: The thematic analysis themes identified by the coders accurately represent participants' interpretations and reasoning.
Classified as probably not a substantial claim.
Study design and conductreasoning§ Justice constructs · p. 4An additional statement about agreement was included prior to the others to avoid the proxy interpretation.
“to avoid this, an additional statement about agreement (1, agreement ) was also included prior to the others.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Premisecitation§ STUDY DESIGN AND METHODOLOGY · p. 3Much of this research is based on field surveys of individuals with personal experience of being affected by decisions, although controlled experiments involving simulated decision-making and scenario-based methods have also been used.
“Much of this research is based on field surveys of individuals with personal experience of being affected by decisions (e.g. [79]), although controlled experiments involving simulated decisionmaking and scenario-based methods have also been used [78].”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [79] 79. Kees Van den Bos, Lynn Van der Velden, and Allan Lind. 2014. On the role of perceived procedural justice in citizens' reactions to government decisions and the handling of conflicts. Utrecht Law Review 10, 4 (2014). DOI: http://dx.doi.org/10.18352/ulr.287
Not checked in the free run.
- cites [78] 78. Kees Van den Bos, E Allan Lind, Riël V ermunt, and Henk AM Wilke. 1997. How do I judge my outcome when I do not know the outcome of others? The psychology of the fair process effect. Journal of Personality and Social Psychology 72, 5 (1997), 1034. source
Not checked in the free run.
Probably not a substantial claim: may not pass S4 (based). May combine two claims.
Interpretationreasoning§ Perceptions of justice regarding decision-making · p. 3Requiring organisations to explain the logic behind their algorithmic decision-making systems (informational justice) enables affected individuals to assess whether the logic of the system is just (procedural justice), which in turn might moderate their assessments of fairness of the decision outcomes (distributive justice).
“Requiring organisations to explain the logic behind their algorithmic decision-making systems (informational justice) enables affected individuals to assess whether the logic of the system is just (procedural justice), which in turn might moderate their assessments of fairness of the decision outcomes (distributive justice).”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationown data§ Justice constructs · p. 4The difficulty in interpreting interactional justice questions was partly due to the hypothetical nature of the scenarios and partly due to being too human-specific to apply to a computer-based interaction.
“This was partly due to the hypothetical nature of the scenarios, and partly due to being too human-specific to apply to a computer-based interaction.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Premisecitation§ INTRODUCTION · p. 1The United States' Equal Credit Opportunity Act requires lenders to provide 'statements of reasons' when applications are denied loans.
“Similarly, the United States' Equal Credit Opportunity Act (ECOA) requires lenders to provide 'statements of reasons' when applications are denied loans [32].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [32] 32. David C Hsia. 1978. Credit scoring and the equal credit opportunity act. Hastings LJ 30 (1978), 371.
Not checked in the free run.
Probably not a substantial claim: may not pass S4 (based).
Premisenone given§ STUDY DESIGN AND METHODOLOGY · p. 3Many of the previously mentioned machine learning model explanation systems have not been subjected to user evaluations.
“Many of the previously mentioned machine learning model explanation systems have not been subjected to user evaluations”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based).
Premisenone given§ Key questions and contributions · p. 3The extent to which findings on correlations between perceptions of justice from human decision-making contexts hold in the context of algorithmic decisions is not yet well known.
“We also do not yet know much the extent to which findings on correlations between perceptions of justice from human decision-making contexts hold in the context of algorithmic decisions.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based).
Premisenone given§ Key questions and contributions · p. 3The nuances of the different explanation approaches recently developed in interpretable ML have not yet been explored in relation to perceptions of justice.
“However the nuances of the different explanation approaches recently developed in interpretable ML have not yet been explored in relation to perceptions of justice.”
Quote in the paperNot a substantial claimMay combine two claims.
Premisecitation§ Perceptions of justice regarding decision-making · p. 3These aspects of justice are distinct but correlated.
“These aspects of justice are distinct but correlated [10].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [10] 10. Jason A Colquitt, Donald E Conlon, Michael J Wesson, Christopher OLH Porter, and K Yee Ng. 2001. Justice at the Millennium: A Meta-Analytic Review of 25 Years of Organizational Justice Research. Journal of Applied Psychology 86, 3 (2001). DOI: http://dx.doi.org/10.1037//0021-9010.86.3.425 source
Not checked in the free run.
Classified as probably not a substantial claim. May combine two claims.
Premisecitation§ Justice constructs · p. 4Statement 2 (understanding) aimed to capture informational justice; statements 3 and 4 (appropriateness of factors, fair process) pertained to procedural justice; and statement 5 (deserved outcome) addressed distributive dimensions.
“Statement 2 ( understanding ) aimed to capture informational justice [27]; statements 3 and 4 ( appropriateness of factors, fair process ) pertained to procedural justice; and statement 5 ( deserved outcome ) addressed distributive dimensions [11].”
Quote in the paperSubstantial claim0 of 2 sources checked- cites [27] 27. Jerald Greenberg and R Cropanzano. 1993. The social side of fairness: Interpersonal and informational classes of organizational justice. Justice in the workplace: Approaching fairness in human resource management, Lawrence Erlbaum Associates, Hillsdale, NJ (1993).
Not checked in the free run.
- cites [11] 11. Jason A Colquitt and Jessica B Rodell. 2015. Measuring justice and fairness. In Oxford Handbook of Justice in the Workplace , Russell Cropanzano and Maureen L Ambrose (Eds.). Oxford University Press, Oxford, 187-202.
Not checked in the free run.
- Rests on an assumption: The justice constructs developed for human decision-making (informational, procedural, distributive) are applicable to and valid for evaluating algorithmic decision-making contexts.
Probably not a substantial claim: may not pass S3 (load-bearing).
Interpretationreasoning§ DISCUSSION · p. 9Habituation to a repeatedly seen explanation style, leading people to attend more to case features than to explanation features, is a possible explanation for the disappearance of explanation effects in the between-subjects design.
“One possible explanation is that when people repeatedly see the same explanation style, they become habituated and pay more attention to the features of the cases rather than any specific features of the explanation provided.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductreasoning§ Threats to validity · p. 10The scenarios considered were hypothetical, did not affect the participants directly, and therefore lacked the first-person consequences and significance of a real-world decision.
“the scenarios considered were hypothetical, not affecting the participants directly, and therefore lacked the first-person consequences and significance of a real world decision”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).
Interpretationreasoning§ DISCUSSION · p. 9The introduction of algorithmic systems seems likely to shake up previously stable categories and concepts, requiring adjustment and incorporation into one's moral world-view.
“The introduction of algorithmic systems seems likely to shake up previously stable categories and concepts, requiring adjustment and incorporation into one's moral world-view.”
Quote in the paperNot a substantial claim- Rests on an assumption: The fictionalised scenarios used in the lab and online studies are realistic and representative enough to elicit valid responses about algorithmic decision-making.
- Rests on an assumption: The thematic analysis themes identified by the coders accurately represent participants' interpretations and reasoning.
Probably not a substantial claim: may not pass S2 (checkable), S4 (based).
Interpretationnone given§ CONCLUSION · p. 10The algorithmic nature of these systems results in an array of novel considerations which are not captured by traditional research on perceptions of justice.
“The algorithmic nature of these systems results in an array of novel considerations which are not captured by traditional research on perceptions of justice.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Interpretationreasoning§ DISCUSSION · p. 9People may realistically face multiple explanations for similar algorithmic decisions when applying for a loan, mortgage or credit.
“people may realistically face multiple explanations for similar algorithmic decisions when applying for a loan, mortgage or credit”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based).
Interpretationnone given§ CONCLUSION · p. 10Creators of machine learning explanation systems have not typically designed them with the information needs of individuals facing significant personal consequences of model outputs in mind.
“creators of ML explanation systems have not typically designed them with the information needs of those individuals facing significant personal consequences of model outputs in mind”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Interpretationreasoning§ DISCUSSION · p. 9When people are exposed to multiple explanation styles regarding the same case, the differences between styles may become more salient.
“When exposed to multiple explanation styles regarding the same case, the differences may become more salient.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Study design and conductnone given§ Threats to validity · p. 10Differences in the length and format of the text required to impart an explanation between explanation conditions may have been a confounding factor.
“differences in the length and format of the text required to impart an explanation between explanation conditions may have been a confounding factor”
Quote in the paperNot a substantial claim
The authors state this as an assumption.
The GDPR automated-decision hurdle ('based solely on automated processing') can be read less restrictively than legal analysis has indicated, so the selected scenarios may fall under GDPR.
“The hurdle that a decision must be 'based solely on automated processing' is here read less restrictively than legal analysis has indicated [81, 20].”
The paper relies on this without support.
Explanations are most likely to be requested in response to a decision with a negative outcome for the individual.
“Because explanations are most likely to be requested in response to a decision which has a negative outcome for the individual, and considerations of procedural justice have been found to be more strongly felt in such cases [4], each scenario ended in a negative outcome for the individual.”
The paper relies on this without support.
The justice constructs developed for human decision-making (informational, procedural, distributive) are applicable to and valid for evaluating algorithmic decision-making contexts.
“The justice constructs used in all three studies were based on those developed in the psychology of justice research, summarised by Colquitt et al [11].”
“The remaining questions pertained to informational, procedural and distributive justice.”
The paper relies on this without support.
Participants' Likert-scale responses to the justice statements accurately measure their perceived justice.
“Participants were asked to rate their agreement with five statements on a 5-point Likert scale”
“Participants were asked to indicate their agreement with the 5 measures described in phase 1 using the same Likert scale.”
The paper relies on this without support.
The fictionalised scenarios used in the lab and online studies are realistic and representative enough to elicit valid responses about algorithmic decision-making.
“The first study consisted of in-person semi-structured interviews focused around a series of fictionalised cases as described above.”
“Cases were presented with a brief description of the individual, the decision outcome, the explanation (except in the control condition) and an illustrative image depicting the context.”
The paper relies on this without support.
The four explanation styles are sufficiently distinct and were correctly perceived by participants as intended.
“Each case was presented with one of the 4 explanation styles described above, or a control condition in which no explanation was provided.”
“Each decision was explained using a different style (in a random order), allowing participants to directly compare multiple styles applied to an individual subject.”
The paper relies on this without support.
The thematic analysis themes identified by the coders accurately represent participants' interpretations and reasoning.
“Thematic analysis was undertaken to identify common themes in participants' interpretation, evaluation and reasoning in response to cases. Thematic codes were independently developed by three coders, after which a single set was jointly agreed upon. The data were then re-coded using the new set of codes.”
The paper relies on this without support.
The sample of Prolific Academic participants is similar enough to the lab study participants to allow comparison.
“Participants were recruited via the study platform Prolific Academic, 4 and filtered to include only individuals over 18 based in the UK, in order to maintain similarity with the participants recruited in phase 1.”
The paper relies on this without support.
There is still much to learn about what people want and need to know about algorithms in order to hold them accountable to justice.
“there is still much to learn about what people want and need to know about algorithms in order to hold them accountable to justice”
The paper relies on this without support.
The explanation styles proposed in the interpretable machine learning literature vary in the information they contain.
“the explanation styles proposed in the interpretable machine learning literature vary in the information they contain”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Participants considered that humans would treat decision subjects with dignity and compassion.
“and that humans would treat decision subjects with dignity and compassion (e.g., Q.31; also in line with previous research [17,68,94]).”
From Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability (2023) · cites this paper as [17]
In “The (lack of) human touch” (p. 6), participants described algorithmic decisions as undignified, impersonal and dehumanising compared with human decision-making. That supports the dignity comparison, but the passage does not establish that participants expected humans to treat decision subjects with compassion. (LLM, conf=0.83)
Experimental — accuracy not yet measuredHumans in the loop could explain the system's outputs, rendering algorithmic decisions more accountable and transparent.
“Furthermore, humans”
From Artificial Intelligence and Management: The Automation–Augmentation Paradox (2021) · cites this paper as Binns, Van Kleek, Veale, Lyngs, Zhao, & Shadbolt, 2018
The Introduction (p. 1) says human decision-makers can be asked to rationalise their decisions and held accountable. The paper also discusses explanations of algorithmic outputs as a route to transparency and accountability (Interpreting intelligent systems, p. 2), but it does not say that humans in the loop would explain those outputs. (LLM, conf=0.83)
Experimental — accuracy not yet measuredPeople perceive decisions made by algorithms as less trustworthy and fair than those made by humans, and perceive being evaluated by algorithms as dehumanizing.
“This philosophically grounded notion is matched by human perceptions that decisions made by algorithms are less trustworthy and fair than those made by humans and that being evaluated by algorithms is dehumanizing ( Binns et al., 2018 ;Lee, 2018 ).”
From The flaws of policies requiring human oversight of government algorithms (2022) · cites this paper as Binns et al., 2018
“The (lack of) human touch” (p. 6) reports that participants found algorithmic decisions impersonal and dehumanising compared with human decisions. The Discussion (p. 9) describes mixed views of fairness: some participants considered algorithmic decisions unfair, while others considered them fair. These passages do not establish that people generally perceive algorithmic decisions as less trustworthy or fair than human decisions. (LLM, conf=0.87)
Experimental — accuracy not yet measuredThere is evidence that decision subjects care about justice-related aspects in algorithmic decision-making, as they care in human decision-making.
“There is evidence that suggests that decision subjects care about justice-related aspects in algorithmic decisionmaking, as they care in human decision-making [17].”
From Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability (2023) · cites this paper as [17]
Statistics recomputed
Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.
- ConsistentF(3, 258) = 2.71reported p = .046 · recomputed p = .046
“F(3, 258) = 2.71, p = .046”
- ConsistentF(3, 258) = 5.35reported p = .001 · recomputed p = .001
“F(3, 258) = 5.35, p = .001”
- ConsistentF(3, 260) = 4.5reported p = .004 · recomputed p = .004
“F(3, 260) = 4.5, p = .004”
- ConsistentF(3, 260) = 3.312reported p = .02 · recomputed p = .021
“F(3, 260) = 3.312, p = .02”
- ConsistentF(3, 260) = 6.44reported p = .0003 · recomputed p < .001
“F(3, 260) = 6.44, p = .0003”
- ConsistentF(3, 260) = 7.31reported p = .0001 · recomputed p < .001
“F(3, 260) = 7.31, p = .0001”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Within-subject effects are claimed across every context, but one is loan-only
MinorThe claim that significant effects occurred across all questions and decision-making contexts conflicts with the stated finding that the deserved-outcome effect occurred only for loans.
“In the within-subjects study-where multiple explanation styles were presented for the same decision-significant effects were observed across all questions and decision-making contexts.”
“on the perception of deserved outcome (only in the loans case, F(3, 260) = 7.31, p = .0001).”
Conflicting confidence levels for the correlation intervals
MinorThe text specifies a 0.05 level for the bootstrapped intervals, while the caption for the resulting table specifies 0.02.
“we calculated Spearman's rho correlation coefficients (with 0.05 confidence level bootstrapped intervals).”
“Table 1. Spearman's rho correlations for justice constructs with bootstrapped (1000 rep) confidence interval at 0.02 level.”
Five themes are claimed, but six are listed
MinorThe results say the lab responses produced five major themes, then name six.
“The think-aloud responses from the lab study resulted in five major themes”
“the lack of human touch; interpretations of the system's reasoning; the use of statistical inference; the degree of actionability in an explanation; important aspects which were seen to be unaccounted for by the system; and the meaning and relevance of moral concepts.”
Cites
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555 · cited 4× · see citing works
“are applicable to any kind of learning algorithm, such as LIME [62], rather than methods which only work for particular ML models (e.”
BackgroundMethodInfluential“Why Should I Trust You?”: Explaining the Predictions of Any Classifier
Ribeiro, M., Singh, S., & Guestrin, C. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations · doi:10.18653/v1/n16-3020 · Cited by 6,682 · see citing works
The Social Psychology of Procedural Justice
Lind, E. A., & Tyler, T. R. (1988) · Critical Issues in Social Justice · doi:10.1007/978-1-4899-2115-4 · Cited by 6,585 · cited 2× · see citing works
“A significant body of research in legal and organisational psychology suggests that people do not only care about whether the outcome of a decision benefits them, but also whether it meets standards of justice [71, 40, 11].”
BackgroundJustice at the millennium: A meta-analytic review of 25 years of organizational justice research.
Colquitt, J. A., Conlon, D. E., Wesson, M. J., Porter, C. O. L. H., & Ng, K. Y. (2001) · Journal of Applied Psychology · doi:10.1037/0021-9010.86.3.425 · Cited by 5,942 · cited 5× · see citing works
“According to the literature on justice perceptions above, we should expect to find correlations between fair process and deserved outcome , understanding and fair process , appropriateness of factors and deserved outcome , and appropriateness of factors and fair process [10].”
BackgroundResultInfluentialThe mythos of model interpretability
Lipton, Z. C. (2018) · Communications of the ACM · doi:10.1145/3233231 · Cited by 5,427 · cited 3× · see citing works
“2Although we acknowledge this is an important challenge if explanations are to be more than just comforting stories, as cautioned by Lipton [41].”
BackgroundLikert scales, levels of measurement, and the “laws” of statistics
Norman, G. (2010) · Advances in Health Sciences Education · doi:10.1007/s10459-010-9222-y · Cited by 4,850 · cited 1× · see citing works
“6Some readers might have reservations over parametric tests being applied to ordinal data: we point to recent literature that has emphasised [53] that Likert data are robust to parametric assumptions, and that ‘parametric methods can be utilized without concern for “getting the wrong answer”’.”
BackgroundComparing Individual Means in the Analysis of Variance
Tukey, J. W. (1949) · Biometrics · doi:10.2307/3001913 · Cited by 4,302 · cited 1× · see citing works
“We used ANOVA tests, followed by Tukey’s post-hoc paired tests [75], to analyse the relative effects of different explanation styles within different study designs.”
MethodBias in random forest variable importance measures: Illustrations, sources and a solution
Strobl, C., Boulesteix, A., Zeileis, A., & Hothorn, T. (2007) · BMC Bioinformatics · doi:10.1186/1471-2105-8-25 · Cited by 3,755 · see citing works
How the machine “thinks”: Understanding opacity in machine learning algorithms
Burrell, J. (2016) · Big Data & Society · doi:10.1177/2053951715622512 · Cited by 3,196 · see citing works
Towards A Rigorous Science of Interpretable Machine Learning
Doshi-Velez, F., & Kim, B. (2017) · arXiv · doi:10.48550/arxiv.1702.08608 · Cited by 3,121 · cited 2× · see citing works
“Furthermore, as research into practically achievable explanations for machine learning systems accelerates, researchers are turning to consider which explanation styles are more desirable, rather than simply possible to implement [41, 15].”
BackgroundWeapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy
Doyle, T. (2017) · The Information Society · doi:10.1080/01972243.2017.1354593 · Cited by 3,110 · cited 1× · see citing works
“3 After consulting relevant literature on how ML systems are being applied in each context (e.g. [43, 38, 43, 55, 72]), we settled on the following contexts: 1.”
BackgroundNass, C., Steuer, J., & Tauber, E. R. (1994) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/191666.191703 · Cited by 2,148 · cited 1× · see citing works
“On the other hand, people frequently interact with computers as if they were social actors [49]; why stop when justice is involved? One participant described how she might have it both ways, simultaneously entertaining dual standards of justice, one juridical, the other computational;”
Certifying and Removing Disparate Impact
Feldman, M., Friedler, S. A., Moeller, J., Scheidegger, C., & Venkatasubramanian, S. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2783311 · Cited by 1,827 · cited 1× · see citing works
“Pr epr in t correct discrimination in data mining exist, these are aimed at data scientists implementing systems rather providing information to the individual decision-subjects affected by them [3, 59, 29, 23, 28].”
BackgroundExplaining collaborative filtering recommendations
Herlocker, J. L., Konstan, J. A., & Riedl, J. (2000) · Proceedings of the 2000 ACM conference on Computer supported cooperative work · doi:10.1145/358916.358995 · Cited by 1,731 · cited 2× · see citing works
“In some cases, explanation increases trust and reliance [33, 85, 1, 30, 60], but in others an explanation may have the opposite effect if the level of detail it contains is deemed insufficient [7].”
BackgroundBig data: a revolution that will transform how we live, work, and think
Armstrong, K. (2014) · Information Communication & Society · doi:10.1080/1369118x.2014.923482 · Cited by 1,643 · see citing works
Some observations on mental models
Norman, D. A. (1983) · Medical Entomology and Zoology · Cited by 1,549 · see citing works
The price is unfair! A conceptual framework of price fairness perceptions
Xia, L., Monroe, K. B., & Cox, J. L. (2004) · Journal of Marketing · doi:10.1509/jmkg.68.4.1.42733 · Cited by 1,474 · cited 1× · see citing works
“Research on justice judgements in non-automated contexts has found that individuals pay more attention to other people’s outcomes than they do to information about the procedure behind the decision that affects them [78, 84].”
BackgroundBrockner, J., & Wiesenfeld, B. M. (1996) · Psychological Bulletin · doi:10.1037/0033-2909.120.2.189 · Cited by 1,368 · cited 1× · see citing works
“Because explanations are most likely to be requested in response to a decision which has a negative outcome for the individual, and considerations of procedural justice have been found to be more strongly felt in such cases [4], each scenario ended in a negative outcome for the individual.”
BackgroundExplaining nonlinear classification decisions with deep taylor decomposition
Montavon, G., Lapuschkin, S., Binder, A., Samek, W., & Müller, K.-R. (2017) · Pattern Recognition · doi:10.1016/j.patcog.2016.11.008 · Cited by 1,357 · cited 2× · see citing works
“1Layer-wise Relevance Propagation/Deep Taylor Expansion [45], decomposes the innards of deep learning systems to provide similar results to LIME (without directionality), and is also relevant here.”
BackgroundResultWachter, S., Mittelstadt, B., & Floridi, L. (2017) · International Data Privacy Law · doi:10.1093/idpl/ipx005 · Cited by 1,308 · see citing works
Benchmarking state-of-the-art classification algorithms for credit scoring: an update of research
Lessmann, S., Baesens, B., Seow, H.-V., & Thomas, L. C. (2015) · European Journal of Operational Research · doi:10.1016/j.ejor.2015.05.030 · Cited by 1,284 · cited 1× · see citing works
“[43, 38, 43, 55, 72]), we settled on the following contexts:”
The role of trust in automation reliance
Dzindolet, M. T., Peterson, S. A., Pomranky, R. A., Pierce, L. G., & Beck, H. P. (2003) · International Journal of Human-Computer Studies · doi:10.1016/s1071-5819(03)00038-7 · Cited by 1,276 · cited 1× · see citing works
“Various research suggests that users—again, primarily experts— may in certain cases systematically over-rely on the outputs of intelligent systems [26, 19], while in others they may systematically distrust systems, despite reassurances as to their accuracy [57, 24].”
BackgroundEvaluating the Visualization of What a Deep Neural Network Has Learned
Samek, W., Binder, A., Montavon, G., Lapuschkin, S., & Muller, K.-R. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2599820 · Cited by 1,164 · see citing works
Automation bias: a systematic review of frequency, effect mediators, and mitigators
Goddard, K., Roudsari, A., & Wyatt, J. C. (2012) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000089 · Cited by 1,162 · cited 1× · see citing works
“Various research suggests that users—again, primarily experts— may in certain cases systematically over-rely on the outputs of intelligent systems [26, 19], while in others they may systematically distrust systems, despite reassurances as to their accuracy [57, 24].”
BackgroundThe social side of fairness: Interpersonal and informational classes of organizational justice.
Greenberg, J. (1993) · Cited by 963 · see citing works
MUIR, B. M. (1994) · Ergonomics · doi:10.1080/00140139408964957 · Cited by 873 · cited 1× · see citing works
“As users form mental models [52] of machine learning systems over time, such long-term exposure may lead to greater understanding [76] and greater trust and reliance [47].”
BackgroundDatta, A., Sen, S., & Zick, Y. (2016) · 2016 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2016.42 · Cited by 728 · cited 2× · see citing works
“Input Influence : Presents a list of input variables alongside a quantitative measure of their ‘influence’, positive or negative, on a decision (e.g. [13, 86]) 2.”
BackgroundMethodThe epistemology of a rule-based expert system —a framework for explanation
Clancey, W. J. (1983) · Artificial Intelligence · doi:10.1016/0004-3702(83)90008-5 · Cited by 712 · cited 2× · see citing works
“Early knowledge representation and reasoning systems typically aimed to meet this requirement by producing explanations for their outputs, for instance by presenting derivations from rules in rule-based architecture [69, 8], or graphical representations of nodes and edges in the case of Bayesian networks [37].”
BackgroundDiscrimination-aware data mining
Pedreshi, D., Ruggieri, S., & Turini, F. (2008) · Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1401890.1401959 · Cited by 676 · cited 1× · see citing works
“Pr epr in t correct discrimination in data mining exist, these are aimed at data scientists implementing systems rather providing information to the individual decision-subjects affected by them [3, 59, 29, 23, 28].”
BackgroundHow Much Information?: Effects of Transparency on Trust in an Algorithmic Interface
Kizilcec, R. F. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858402 · Cited by 615 · cited 1× · see citing works
“A notable exception is Kizilcec, who studied the effect of levels of transparency on perceptions of procedural justice [35].”
BackgroundNass, C., Steuer, J., & Tauber, E. R. (1994) · Conference companion on Human factors in computing systems - CHI '94 · doi:10.1145/259963.260288 · Cited by 579 · see citing works
<i>Why and why not</i> explanations improve the intelligibility of context-aware intelligent systems
Lim, B. Y., Dey, A. K., & Avrahami, D. (2009) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1518701.1519023 · Cited by 552 · cited 2× · see citing works
“Inspired by different theoretical models of argumentation, various explanation types have been explored and tested, with the style of explanation provided affecting the extent to which end-users understand the decisions of expert systems [46, 39].”
BackgroundEdwards, L., & Veale, M. (2017) · doi:10.31228/osf.io/97upg · Cited by 524 · see citing works
Self-citationDirect manipulation vs. interface agents
Shneiderman, B., & Maes, P. (1997) · Interactions · doi:10.1145/267505.267514 · Cited by 518 · cited 1× · see citing works
“As a general principle, systems should be able account for their own operation in ways that help users understand how their tasks are being accomplished [16, 66].”
BackgroundFildes, R. A., Goodwin, P., Lawrence, M. J., & Nikolopoulos, K. I. (2009) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2008.11.010 · Cited by 447 · cited 1× · see citing works
“Various research suggests that users—again, primarily experts— may in certain cases systematically over-rely on the outputs of intelligent systems [26, 19], while in others they may systematically distrust systems, despite reassurances as to their accuracy [57, 24].”
BackgroundThe Role of Explanations on Trust and Reliance in Clinical Decision Support Systems
Bussone, A., Stumpf, S., & O'Sullivan, D. (2015) · 2015 International Conference on Healthcare Informatics · doi:10.1109/ichi.2015.26 · Cited by 441 · cited 1× · see citing works
“In some cases, explanation increases trust and reliance [33, 85, 1, 30, 60], but in others an explanation may have the opposite effect if the level of detail it contains is deemed insufficient [7].”
BackgroundTickle, A. B., Andrews, R., Golea, M., & Diederich, J. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/72.728352 · Cited by 437 · cited 2× · see citing works
“The increasing popularity of machine learning models as decision making tools presents novel challenges for explanation, as it may not be as straightforward to extract the embedded ‘knowledge’ behind their outputs [73].”
BackgroundThe Oxford Handbook of Justice in the Workplace
(2015) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780199981410.001.0001 · Cited by 419 · see citing works
van den Bos, K., Lind, E. A., Vermunt, R., & Wilke, H. A. M. (1997) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.72.5.1034 · Cited by 392 · see citing works
UX Design Innovation: Challenges for Working with Machine Learning as a Design Material
Dove, G., Halskov, K., Forlizzi, J., & Zimmerman, J. (2017) · Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025739 · Cited by 391 · cited 1× · see citing works
“This is part of a wider need to consider how to re-engineer better user experiences for multiple stakeholders interacting with machine learning systems in general [17].”
BackgroundIntrigue: Personalized recommendation of tourist attractions for desktop and hand held devices
Ardissono, L., Goy, A., Petrone, G., Segnan, M., & Torasso, P. (2003) · Applied Artificial Intelligence · doi:10.1080/713827254 · Cited by 384 · cited 2× · see citing works
“rom the model’s training data which most closely resemble the output in question [12,54], while ‘demographic’ explanations provide statistics on the outcome classes for relevant demographic variables [2]. While these different explanation stylessuggest promising ways to explain the outputs of any classifier, explanati…”
BackgroundFirst I "like" it, then I hide it: Folk Theories of Social Feeds
Eslami, M., Karahalios, K., Sandvig, C., Vaccaro, K., Rickman, A., Hamilton, K., & Kirlik, A. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858494 · Cited by 375 · cited 1× · see citing works
“But in the absence of transparency from platform providers, end-users may develop folk theories of algorithms, leading to a diverse range of strategies and countermeasures with varying degrees of effectiveness [22, 14].”
BackgroundExplaining Recommendations: Design and Evaluation
Tintarev, N., & Masthoff, J. (2015) · Springer eBooks · doi:10.1007/978-1-4899-7637-6_10 · Cited by 344 · cited 1× · see citing works
“Tintarev and Mastho identify seven purposes for recommender system explanations, namely: transparency, scrutability, trust, effectiveness, persuasiveness, efficiency and satisfaction [74].”
BackgroundThe relative influence of advice from human experts and statistical methods on forecast adjustments
Önkal, D., Goodwin, P., Thomson, M. E., Gonul, M. S., & Pollock, A. C. (2009) · Journal of Behavioral Decision Making · doi:10.1002/bdm.637 · Cited by 331 · cited 1× · see citing works
“Various research suggests that users—again, primarily experts— may in certain cases systematically over-rely on the outputs of intelligent systems [26, 19], while in others they may systematically distrust systems, despite reassurances as to their accuracy [57, 24].”
BackgroundAn analysis of physician attitudes regarding computer-based clinical consultation systems
Teach, R. L., & Shortliffe, E. H. (1981) · Computers and Biomedical Research · doi:10.1016/0010-4809(81)90012-4 · Cited by 330 · cited 1× · see citing works
“Early knowledge representation and reasoning systems typically aimed to meet this requirement by producing explanations for their outputs, for instance by presenting derivations from rules in rule-based architecture [69, 8], or graphical representations of nodes and edges in the case of Bayesian networks [37].”
BackgroundGetting access to what goes on in people's heads?
Nielsen, J., Clemmensen, T., & Yssing, C. (2002) · Nordic Conference on Human-Computer Interaction · doi:10.1145/572020.572033 · Cited by 318 · cited 1× · see citing works
“The cases were used to prompt reflection on various aspects of the decision, and any explanatory information provided, using a concurrent think-aloud process [51].”
MethodDeVito, M. A., Gergle, D., & Birnholtz, J. (2017) · Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025659 · Cited by 311 · cited 1× · see citing works
“But in the absence of transparency from platform providers, end-users may develop folk theories of algorithms, leading to a diverse range of strategies and countermeasures with varying degrees of effectiveness [22, 14].”
BackgroundCredit Scoring and Its Applications
Thomas, L. C., Crook, J. N., & Edelman, D. B. (2017) · Cited by 308 · see citing works
Vig, J., Sen, S., & Riedl, J. (2009) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/1502650.1502661 · Cited by 304 · see citing works
Trust building with explanation interfaces
Pu, P., & Chen, L. (2006) · Proceedings of the 11th international conference on Intelligent user interfaces · doi:10.1145/1111449.1111475 · Cited by 300 · cited 1× · see citing works
“In some cases, explanation increases trust and reliance [33, 85, 1, 30, 60], but in others an explanation may have the opposite effect if the level of detail it contains is deemed insufficient [7].”
BackgroundThe Impact of Explanation Facilities on User Acceptance of Expert Systems Advice
Ye, L. R., & Johnson, P. E. (1995) · MIS Quarterly · doi:10.2307/249686 · Cited by 290 · cited 1× · see citing works
“In some cases, explanation increases trust and reliance [33, 85, 1, 30, 60], but in others an explanation may have the opposite effect if the level of detail it contains is deemed insufficient [7].”
BackgroundSlave to the Algorithm? Why a Right to Explanationn is Probably Not the Remedy You are Looking for
Edwards, L., & Veale, M. (2017) · SSRN Electronic Journal · doi:10.2139/ssrn.2972855 · Cited by 244 · cited 5× · see citing works
“ncluding both technical work on interpretable machine learning models, and legal discussions on the requirements of existing and forthcoming regulations on transparency of algorithmic decisions (e.g. [81,65,20]. Our aim was to find explanation styles which could plausibly meet or exceed the regulatory requirements reg…”
BackgroundMethodInfluentialSelf-citationA review of explanation methods for Bayesian networks
Lacave, C., & Díez, F. J. (2002) · The Knowledge Engineering Review · doi:10.1017/s026988890200019x · Cited by 240 · cited 1× · see citing works
“Early knowledge representation and reasoning systems typically aimed to meet this requirement by producing explanations for their outputs, for instance by presenting derivations from rules in rule-based architecture [69, 8], or graphical representations of nodes and edges in the case of Bayesian networks [37].”
BackgroundInterpretable classification models for recidivism prediction
Zeng, J., Ustun, B., & Rudin, C. (2017) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/rssa.12227 · Cited by 220 · see citing works
Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy
Verma, S. (2019) · Vikalpa: The Journal for Decision Makers · doi:10.1177/0256090919853933 · Cited by 219 · see citing works
Enhanced Maintenance and Explanation of Expert Systems Through Explicit Models of Their Development
Neches, R. T., Swartout, W., & Moore, J. D. (1985) · IEEE Transactions on Software Engineering · doi:10.1109/tse.1985.231882 · Cited by 165 · cited 2× · see citing works
“Explanation facilities could be further augmented by explicitly modeling the knowledge engineering and design process and revealing information about them to end-users [50].”
BackgroundProcedural Justice as Fairness
Thibaut, J., Walker, L., LaTour, S., & Houlden, P. (1974) · Stanford Law Review · doi:10.2307/1227990 · Cited by 164 · see citing works
Credit Scoring and Its Applications, Second Edition
Thomas, L. C., Crook, J. N., & Edelman, D. (2017) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9781611974560 · Cited by 164 · see citing works
Anthropomorphism, agency, and ethopoeia: computers as social actors
Nass, C., Steuer, J., Tauber, E., & Reeder, H. (1993) · INTERACT '93 and CHI '93 conference companion on Human factors in computing systems - CHI '93 · doi:10.1145/259964.260137 · Cited by 162 · cited 1× · see citing works
“In contrast to this apparent lack of humanity (or perhaps to counteract it), some participants attempted to humanise or anthropomorphise the system [48], one participant for instance describing it as ‘rude’ (OR).”
ResultMeasuring Justice and Fairness
Cropanzano, R., Ambrose, M. L., Colquitt, J. A., & Rodell, J. B. (2015) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780199981410.013.8 · Cited by 149 · see citing works
Williams, J. J., Kim, J., Rafferty, A. N., Maldonado, S., Gajos, K. Z., Lasecki, W. S., & Heffernan, N. T. (2016) · ACM Conference on Learning @ Scale (L@S) · doi:10.1145/2876034.2876042 · Cited by 147 · cited 1× · see citing works
“Contexts studied include aiding medical experts in diagnosis, increasing test performance of users of personalised learning environments, or enabling network security analysts to correctly identify incidents (e.g. [34, 83, 21]).”
BackgroundProfiling the European Citizen: Cross-Disciplinary Perspectives
Hildebrandt, M., & Gutwirth, S. (2008) · Springer eBooks · doi:10.1007/978-1-4020-6914-7 · Cited by 144 · see citing works
Tullio, J., Dey, A. K., Chalecki, J., & Fogarty, J. A. (2007) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1240624.1240630 · Cited by 137 · cited 1× · see citing works
“As users form mental models [52] of machine learning systems over time, such long-term exposure may lead to greater understanding [76] and greater trust and reliance [47].”
BackgroundThe Case for Process Fairness in Learning: Feature Selection for Fair Decision Making
Grgić-Hlača, N., Zafar, M. B., Gummadi, K. P., & Weller, A. (2016) · MPG.PuRe (Max Planck Society) · Cited by 135 · see citing works
Some Observations on Mental Models
Norman, D. A. (2014) · Psychology Press eBooks · doi:10.4324/9781315802725-5 · Cited by 128 · cited 1× · see citing works
“As users form mental models [52] of machine learning systems over time, such long-term exposure may lead to greater understanding [76] and greater trust and reliance [47].”
BackgroundHuman agency and responsible computing: Implications for computer system design
Friedman, B., & Kahn, P. H. (1992) · Journal of Systems and Software · doi:10.1016/0164-1212(92)90075-u · Cited by 123 · cited 1× · see citing works
“In this sense, while algorithmic decision-making implicates dimensions of justice, it may also mitigate them, in so far as people resist imputing morality to a computer system [25, 5].”
BackgroundAn Evaluation of the Usefulness of Case-Based Explanation
Cunningham, P., Doyle, D., & Loughrey, J. (2007) · Springer eBooks · doi:10.1007/3-540-45006-8_12 · Cited by 118 · see citing works
Towards improving trust in context-aware systems by displaying system confidence
Antifakos, S., Kern, N., Schiele, B., & Schwaninger, A. (2005) · International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI) · doi:10.1145/1085777.1085780 · Cited by 98 · see citing works
(2017) · arXiv (Cornell University) · Cited by 86 · see citing works
A Case-Based Explanation System for Black-Box Systems
Nugent, C., & Cunningham, P. (2005) · Artificial Intelligence Review · doi:10.1007/s10462-005-4609-5 · Cited by 84 · cited 1× · see citing works
“1 Case-based explanations provide examples from the model’s training data which most closely resemble the output in question [12, 54], while ‘demographic’ explanations provide statistics on the outcome classes for relevant demographic variables [2].”
BackgroundBureaucratic Justice: Managing Social Security Disability Claims
Hahn, H., & Mashaw, J. L. (1984) · Public Administration Review · doi:10.2307/975875 · Cited by 81 · see citing works
User Preferences for Hybrid Explanations
Kouki, P., Schaffer, J., Pujara, J., O’Donovan, J., & Getoor, L. C. (2017) · ACM Conference on Recommender Systems (RecSys) · doi:10.1145/3109859.3109915 · Cited by 73 · cited 1× · see citing works
“While early recommender system explanation approaches provided a uniform explanation style for single-source collaborative filtering [30], more recent work explores how to derive explanations for recommender systems based on ‘hybrid’ multiple sources [36] and matrix factorisation [61].”
BackgroundExplanation facilities and interactive systems
Johnson, H., & Johnson, P. (1993) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/169891.169951 · Cited by 71 · cited 1× · see citing works
“In some cases, explanation increases trust and reliance [33, 85, 1, 30, 60], but in others an explanation may have the opposite effect if the level of detail it contains is deemed insufficient [7].”
BackgroundA Reactive Approach to Explanation: Taking the User’s Feedback into Account
Moore, J. D., & Swartout, W. (1991) · Kluwer international series in engineering and computer science · doi:10.1007/978-1-4757-5945-7_1 · Cited by 67 · cited 1× · see citing works
“Inspired by different theoretical models of argumentation, various explanation types have been explored and tested, with the style of explanation provided affecting the extent to which end-users understand the decisions of expert systems [46, 39].”
BackgroundA Review of Explanation and Explanation in Case-Based Reasoning
Doyle, D., Tsymbal, A., & Cunningham, P. (2003) · Trinity's Access to Research Output (TARA) (Trinity College Dublin) · Cited by 61 · see citing works
van den Bos, K., van der Velden, L., & Lind, E. A. (2014) · Utrecht Law Review · doi:10.18352/ulr.287 · Cited by 59 · cited 2× · see citing works
“[79]), although controlled experiments involving simulated decisionmaking and scenario-based methods have also been used [78].”
BackgroundCredit Scoring and the Equal Credit Opportunity Act
Hsia, D. C. (1978) · bepress Legal Repository · Cited by 56 · see citing works
Moral coherence processes: constructing culpability and consequences
Clark, C. J., Chen, E. E., & Ditto, P. H. (2015) · Current Opinion in Psychology · doi:10.1016/j.copsyc.2015.07.016 · Cited by 50 · cited 1× · see citing works
“Such attempts to delineate moral culpability between systems and people, or between different standards of fairness or evidence, may reflect a more general psychological need to see the world as morally coherent [9].”
BackgroundHuman Agency and Responsible Computing: Implications for Computer System Design
Friedman, B., & Kahn, P. H. (2017) · Routledge eBooks · doi:10.4324/9781315259697-26 · Cited by 46 · see citing works
Berendt, B., & Preibusch, S. (2014) · Artificial Intelligence and Law · doi:10.1007/s10506-013-9152-0 · Cited by 44 · cited 2× · see citing works
“Pr epr in t correct discrimination in data mining exist, these are aimed at data scientists implementing systems rather providing information to the individual decision-subjects affected by them [3, 59, 29, 23, 28].”
BackgroundTaking advice from intelligent systems: the double-edged sword of explanations
Ehrlich, K., Kirk, S. E., Patterson, J., Rasmussen, J. C., Ross, S. I., & Gruen, D. (2011) · International Conference on Intelligent User Interfaces · doi:10.1145/1943403.1943424 · Cited by 42 · cited 1× · see citing works
“Contexts studied include aiding medical experts in diagnosis, increasing test performance of users of personalised learning environments, or enabling network security analysts to correctly identify incidents (e.g. [34, 83, 21]).”
BackgroundEvaluating the visualization of what a Deep Neural Network has learnedPreprint
Samek, W., Binder, A., Montavon, G., Bach, S., & Müller, K. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1509.06321 · Cited by 35 · cited 1× · see citing works
“Sensitivity : For each input variable used in a decision, sensitivity analysis shows how much the value of that variable would have to differ in order to change the output class (not to be confused with the notion of sensitivity used in ML evaluation) (e.g. [63, 65]).”
MethodTintarev, N. (2007) · Lecture notes in computer science · doi:10.1007/978-3-540-73078-1_67 · Cited by 34 · see citing works
Vehicle Telematics at an Italian Insurer: New Auto Insurance Products and a New Industry Ecosystem
Vaia, G., Carmel, E., DeLone, W. H., Trautsch, H., & Menichetti, F. (2012) · Journal of the Association for Information Systems · Cited by 32 · see citing works
Direct and Indirect Discrimination Prevention Methods
Hajian, S., & Domingo-Ferrer, J. (2013) · Studies in Applied Philosophy, Epistemology and Rational Ethics · doi:10.1007/978-3-642-30487-3_13 · Cited by 27 · cited 1× · see citing works
“Pr epr in t correct discrimination in data mining exist, these are aimed at data scientists implementing systems rather providing information to the individual decision-subjects affected by them [3, 59, 29, 23, 28].”
BackgroundDo News Consumers Want Explanations for Personalized News Rankings?
ter Hoeve, M., Heruer, M., Odijk, D., Schuth, A., Spitters, M., & de Rijke, M. (2018) · doi:10.18122/b24d7n · Cited by 20 · cited 1× · see citing works
“However, a reported need for explanation may not always correspond to differences in behaviour or performance; in a study of news recommender systems, end-users expressed a desire for explanations, but the number of news items they opened did not change when provided with reasons for their recommendations [70].”
BackgroundSchwartz, P. M. (1991) · bepress Legal Repository · Cited by 16 · see citing works
FATREC Workshop on Responsible Recommendation Proceedings
Ekstrand, M. D., & Sharma, A. K. (2017) · Scholar Works (Boise State University) · Cited by 11 · see citing works
Exploring Explanations for Matrix Factorization Recommender Systems
Rastegarpanah, B., Crovella, M. E., & Gummadi, K. P. (2018) · doi:10.18122/b2r717 · Cited by 11 · cited 1× · see citing works
“While early recommender system explanation approaches provided a uniform explanation style for single-source collaborative filtering [30], more recent work explores how to derive explanations for recommender systems based on ‘hybrid’ multiple sources [36] and matrix factorisation [61].”
BackgroundKim, B., Chacha, C. M., & Shah, J. (2013) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v27i1.8548 · Cited by 8 · cited 1× · see citing works
“Contexts studied include aiding medical experts in diagnosis, increasing test performance of users of personalised learning environments, or enabling network security analysts to correctly identify incidents (e.g. [34, 83, 21]).”
BackgroundInferring team task plans from human meetings
KimBeen, ChachaCaleb, M., & ShahJulie, A. (2015) · doi:10.5555/2831407.2831415 · Cited by 3 · see citing works
Hendrickx, F. (2000) · Cited by 2 · see citing works
Flagged first, then by citations
Research works printed but not linked: 7
- Joanna J Bryson. 2010. Robots should be slaves. Close Engagements with Artificial Companions: Key social, psychological, ethical and design issues (2010), 63--74.
- 10.5555/270318.270324 · DOI
- 10.5555/1386322 · DOI
- Gregory Mitchell. 2004. Libertarian Paternalism is an Cxymoron. Nw. UL Rev. 99 (2004), 1245.
- 10.5555/3175762 · DOI
- Cathy O'Neill. 2016. How algorithms rule our working lives. The Guardian (2016).
- Frank Pasquale. 2015. Digital Star Chamber. Aeon (2015). https://aeon.co/essays/ judge-jury-and-executioner-the-unaccountable-algorithm
Cited by
Sorted by citations
Artificial Intelligence and Management: The Automation–Augmentation Paradox
Raisch, S., & Krakowski, S. (2021) · Academy of Management Review · doi:10.5465/amr.2018.0072 · Cited by 2,755 · see citing works
Ali, S., Abuhmed, T., El-Sappagh, S., Muhammad, K., Alonso-Moral, J. M., Confalonieri, R., Guidotti, R., Del Ser, J., Díaz-Rodríguez, N., & Herrera, F. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101805 · Cited by 1,762 · see citing works
Rahwan, I., Cebrian, M., Obradovich, N., Bongard, J., Bonnefon, J.-F., Breazeal, C., Crandall, J. W., Christakis, N. A., Couzin, I. D., Jackson, M. O., et al. (2019) · Nature · doi:10.1038/s41586-019-1138-y · Cited by 1,364 · see citing works
Morley, J., Floridi, L., Kinsey, L., & Elhalal, A. (2020) · Science and Engineering Ethics · doi:10.1007/s11948-019-00165-5 · Cited by 882 · see citing works
“…a system is explicable (explainable and interpretable) it is inherently more transparent and therefore more accountable in terms of its decision-making properties and the extent to which they include human oversight and are fair, robust and justifiable (Binns et al. 2018; Cath 2018; Lipton 2016).”
BackgroundImproving Fairness in Machine Learning Systems: What Do Industry Practitioners Need?
Holstein, K., Wortman Vaughan, J., Daumé, H., Dudik, M., & Wallach, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300830 · Cited by 777 · see citing works
“For example, HCI researchers have empirically studied users’ expectations and perceptions related to fairness in algorithmic systems, finding that these do not always align with existing statistical definitions [16, 71, 72, 108].”
BackgroundQuestioning the AI: Informing Design Practices for Explainable AI User Experiences
Liao, Q. V., Gruen, D., & Miller, S. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376590 · Cited by 925 · see citing works
Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., & Weld, D. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445717 · Cited by 757 · see citing works
“n natural language (Section3.5). Admittedly, explanations can also be presented in other ways. For structured data, most studies present the salient features, sometimes paired with importance weights [8, 21, 47, 52, 63]. On vision tasks, some work takes the demonstrate-by-example approach, i.e., by explaining an examp…”
MethodTrustworthy Artificial Intelligence: A Review
Kaur, D., Uslu, S., Rittichier, K. J., & Durresi, A. (2023) · ACM Computing Surveys · doi:10.1145/3491209 · Cited by 709 · see citing works
“Accountability of algorithms is an incremental process [18], requiring proper governance of the AI lifecycle and discussion between different stakeholders [144].”
BackgroundManipulating and Measuring Model Interpretability
Poursabzi-Sangdeh, F., Goldstein, D. G., Hofman, J. M., Wortman Vaughan, J. W., & Wallach, H. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445315 · Cited by 728 · see citing works
Langer, M., Oster, D., Speith, T., Hermanns, H., Kästner, L., Schmidt, E., Sesing, A., & Baum, K. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2021.103473 · Cited by 652 · see citing works
“Regulators call, for instance, for trustworthy systems [5, 6, 10, 11, 29, 96, 115, 131].”
A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems
Mohseni, S., Zarei, N., & Ragan, E. D. (2021) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3387166 · Cited by 785 · see citing works
“User Understanding of Model Interview ([40]) and Self-explanation ([20, 47, 164]) Likert-scale Questionnaire ([99, 111, 113, 124, 133, 171]) Model Output Prediction User Prediction of Model Output ([96, 172, 173]) Model Failure Prediction User Prediction of Model Failure ([15, 161])”
BackgroundMethodInfluentialTowards a standard for identifying and managing bias in artificial intelligence
Schwartz, R., Vassilev, A., Greene, K., Perine, L., Burt, A., & Hall, P. (2022) · doi:10.6028/nist.sp.1270 · Cited by 548 · see citing works
Actionable Recourse in Linear Classification
Ustun, B., Spangher, A., & Liu, Y. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287566 · Cited by 501 · see citing works
“Seeing how the lack of autonomy is perceived as a source of injustice in algorithmic decision-making [6, 14, 33], recourse should be considered whenever humans are subject to the predictions of a machine learning model.”
BackgroundTransparency in Algorithmic and Human Decision-Making: Is There a Double Standard?
Zerilli, J., Knott, A., Maclaurin, J., & Gavaghan, C. (2019) · Philosophy & Technology · doi:10.1007/s13347-018-0330-6 · Cited by 420 · see citing works
“Often knowing the reasons why a particular decision has been taken, even if only in rough outline, can engender trust in the process that led to it and confidence that the people in charge of the process acted fairly and reasonably (Binns et al. 2018).”
BackgroundInfluentialThe Principles and Limits of Algorithm-in-the-Loop Decision Making
Green, B., & Chen, Y. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359152 · Cited by 392 · see citing works
“…making, prior research has explored topics such as what interactions can facilitate the development of machine learning models [9, 29, 47], how to improve human performance with an algorithm’s assistance [12, 37, 48], and the ways in which laypeople perceive algorithmic decisions [4, 28, 52].”
BackgroundLanger, M., & Landers, R. N. (2021) · Computers in Human Behavior · doi:10.1016/j.chb.2021.106878 · Cited by 338 · see citing works
Yam, K. C., Tang, P. M., Jackson, J. C., Su, R., & Gray, K. J. (2022) · Journal of Applied Psychology · doi:10.1037/apl0001045 · Cited by 352 · see citing works
de Fine Licht, K., & de Fine Licht, J. (2020) · AI & SOCIETY · doi:10.1007/s00146-020-00960-w · Cited by 334 · see citing works
Explanations in Autonomous Driving: A Survey
Omeiza, D., Webb, H., Jirotka, M., & Kunze, L. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3122865 · Cited by 355 · see citing works
“b) Explanation Style: explanations are categorised based on the type of information or elements referenced in the explanation and the forms they are presented in [72].”
BackgroundMethodFairness perceptions of algorithmic decision-making: A systematic review of the empirical literature
Starke, C., Baleis, J., Keller, B., & Marcinkowski, F. (2022) · Big Data & Society · doi:10.1177/20539517221115189 · Cited by 316 · see citing works
“Explanations for an algorithmic decision significantly increased respondents’ perceptions of fairness in several studies (Binns et al., 2018; Dodge et al., 2019; Shin, 2021; Shulner-Tal et al., 2022).”
BackgroundMethodResultInfluentialBias and Discrimination in AI: A Cross-Disciplinary Perspective
Ferrer, X., Nuenen, T. V., Such, J. M., Cote, M., & Criado, N. (2021) · IEEE Technology and Society Magazine · doi:10.1109/mts.2021.3056293 · Cited by 308 · see citing works
“[27] find a strong split in their respondents: some find the general idea of algorithmic discrimination immoral, others resist imputing morality to a computer system altogether “the computer is just doing its job” [27].”
BackgroundWieringa, M. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372833 · Cited by 321 · see citing works
Wang, X., & Yin, M. (2021) · 26th International Conference on Intelligent User Interfaces · doi:10.1145/3397481.3450650 · Cited by 320 · see citing works
“…the direction of change in the model’s prediction when a feature’s value is altered [16, 18, 29], to simulate the model’s predictions [16, 24, 40, 47, 59], to answer “what-if” questions about the model behavior [7, 16, 24, 33, 54], and to detect mistakes of the model and debug the model [59, 60].”
BackgroundArtificial Intelligence and Patient-Centered Decision-Making
Bjerring, J. C., & Busch, J. (2021) · Philosophy & Technology · doi:10.1007/s13347-019-00391-6 · Cited by 274 · see citing works
“It is well-known that “ the provision of explanations can affect levels of trust and acceptance of algorithmic decisions ” (Binns et al. 2018, p. 377).”
BackgroundExplaining models: an empirical study of how explanations impact fairness judgment
Dodge, J., Liao, Q. V., Zhang, Y., Bellamy, R. K. E., & Dugan, C. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302310 · Cited by 286 · see citing works
Counterfactual Explanations and Algorithmic Recourses for Machine Learning: A Review
Verma, S., Boonsanong, V., Hoang, M., Hines, K., Dickerson, J. P., & Shah, C. S. (2024) · ACM Computing Surveys · doi:10.1145/3677119 · Cited by 456 · see citing works
Counterfactual Explanations and Algorithmic Recourses for Machine Learning: A Review
Verma, S., Boonsanong, V., Hoang, M., Hines, K. E., Dickerson, J. P., & Shah, C. (2020) · arXiv · doi:10.48550/arxiv.2010.10596 · Cited by 267 · see citing works
Towards Human-Centered Explainable AI: A Survey of User Studies for Model Explanations
Rong, Y., Leemann, T., Nguyen, T.-T., Fiedler, L., Qian, P., Unhelkar, V., Seidel, T., Kasneci, G., & Kasneci, E. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3331846 · Cited by 278 · see citing works
“One of the most critical design choices when assessing objective understanding is the selection of a suitable proxy task.”
BackgroundHaque, A. B., Islam, A. K. M. N., & Mikalef, P. (2022) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2022.122120 · Cited by 280 · see citing works
“Another use case related to the flight re-routing system offers the latest flight data to users so that travel is flexible and comfortable (Binns et al., 2018).”
BackgroundMethodInfluentialProcedural Justice in Algorithmic Fairness
Lee, M. K., Jain, A., Jin, H., Ojha, S., & Kusbit, D. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359284 · Cited by 236 · see citing works
Bankins, S., Formosa, P., Griep, Y., & Richards, D. (2022) · Information Systems Frontiers · doi:10.1007/s10796-021-10223-8 · Cited by 249 · see citing works
“…with the lack of human interaction often attendant in AI decision making, can lead workers to think that they are “being reduced to a percentage” (Binns et al., 2018, p. 1; Lee et al., 2019, p. 16; Lee, 2018); which constitutes a set of feelings often captured under the umbrella term of…”
BackgroundInfluentialPassi, S., & Jackson, S. J. (2018) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3274405 · Cited by 225 · see citing works
“This is particularly challenging because the systems are not usually designed “with the information needs of those individuals facing significant personal consequences of model outputs” [7:10].”
BackgroundA Case for Humans-in-the-Loop: Decisions in the Presence of Erroneous Algorithmic Scores
De-Arteaga, M., Fogliato, R., & Chouldechova, A. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376638 · Cited by 221 · see citing works
Efficient Search for Diverse Coherent Explanations
Russell, C. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287569 · Cited by 218 · see citing works
“Binns et al. (2018) found evidence that users prefer counterfactual explanations over case-based reasoning.”
BackgroundInfluentialThe flaws of policies requiring human oversight of government algorithms
Green, B. (2022) · Computer Law & Security Review · doi:10.1016/j.clsr.2022.105681 · Cited by 263 · see citing works
“This philosophically grounded notion is matched by human perceptions that decisions made by algorithms are less trustworthy and fair than those made by humans and that being evaluated by algorithms is dehumanizing (Binns et al., 2018; Lee, 2018).”
BackgroundAutomated rationale generation: a technique for explainable AI and its effects on human perceptions
Ehsan, U., Tambwekar, P., Chan, L., Harrison, B., & Riedl, M. O. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302316 · Cited by 210 · see citing works
Lee, M. K., Kusbit, D., Kahng, A., Kim, J. T., Yuan, X., Chan, A., See, D., Noothigattu, R., Lee, S., Psomas, A., et al. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359283 · Cited by 217 · see citing works
Brown, A., Chouldechova, A., Putnam-Hornstein, E., Tobin, A., & Vaithianathan, R. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300271 · Cited by 207 · see citing works
Highly automated job interviews: Acceptance under the influence of stakes
Langer, M., König, C. J., & Papathanasiou, M. (2019) · International Journal of Selection and Assessment · doi:10.1111/ijsa.12246 · Cited by 203 · see citing works
“It seems possible that participants might have interpreted that a computer gains decision power over a human being, which may be seen as less fair than a person being in charge of conducting high-stake interviews and making high-stake selection decisions (Binns et al., 2018; M. K. Lee, 2018; Ötting & Maier, 2018).”
InfluentialAI-generated vs. Human Artworks. A Perception Bias Towards Artificial Intelligence?
Ragot, M., Martin, N., & Cojean, S. (2020) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3334480.3382892 · Cited by 221 · see citing works
Lai, V., Chen, C., Smith-Renner, A., Liao, Q. V., & Tan, C. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594087 · Cited by 204 · see citing works
“Subjective metrics of understanding typically ask participants to directly rate their understanding of the AI [5, 12, 15, 18, 23, 75, 91, 105, 108], or some variations of it such as confidence in understanding [57], ease of understanding [46, 83], or confidence in simulation [2, 80].”
BackgroundInfluentialThe ethical use of artificial intelligence in human resource management: a decision-making framework
Bankins, S. (2021) · Ethics and Information Technology · doi:10.1007/s10676-021-09619-6 · Cited by 181 · see citing works
“…denial of human interaction can degrade individuals’ justice perceptions, or the perceived fairness of a decision, generating feelings of impersonal treatment, devaluation, and dehumanization (such as “being reduced to a percentage”) (Binns et al., 2018, p. 1; Lee et al., 2019, p. 16; Lee, 2018).”
BackgroundSrivastava, M., Heidari, H., & Krause, A. (2019) · Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3292500.3330664 · Cited by 179 · see citing works
Administration by Algorithm? Public Management meets Public Sector Machine Learning
Veale, M., & Brass, I. (2019) · doi:10.31235/osf.io/mwhnb · Cited by 316 · see citing works
“At this level—the level where individual decisions are made concerning citizens and entities—legal regimes and scholars have been concerned about the threats of such automation to procedural justice in both its substance (Crawford & Schultz 2014; Keats Citron & Pasquale 2014) and the perception of it by citizens (Binn…”
InfluentialSelf-citationHow Explainability Contributes to Trust in AI
Ferrario, A., & Loi, M. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533202 · Cited by 163 · see citing works
Fairness and Explanation in AI-Informed Decision Making
Angerschmid, A., Zhou, J., Theuermann, K., Chen, F., & Holzinger, A. (2022) · Machine Learning and Knowledge Extraction · doi:10.3390/make4020026 · Cited by 161 · see citing works
Kabir, S., Udo-Imeh, D. N., Kou, B., & Zhang, T. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642596 · Cited by 160 · see citing works
“Therefore, it gives an extremely verbose response that contains the definitions of encapsulation, inheritance, etc., which is not useful in terms of identifying the differences originally asked for.”
BackgroundHow Do Fairness Definitions Fare?
Saxena, N. A., Huang, K., DeFilippis, E., Radanović, G., Parkes, D. C., & Liu, Y. (2019) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3306618.3314248 · Cited by 202 · see citing works
“…in targeted online advertising, Grgić-Hlača et al. (2018a) and Grgić-Hlača et al. (2018b) studied human perceptions of features used in algorithmic decision making, and Binns et al. (2018) examined people’s perception of justice in algorithmic decision making under different explanation styles.”
BackgroundMarcinkowski, F., Kieslich, K., Starke, C., & Lünich, M. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372867 · Cited by 159 · see citing works
“Some studies have looked into the determinants of distributive fairness perceptions and have found that process-related variables impact how people evaluate the outcome of ADM [8,16].”
BackgroundMethodInfluentialHow to explain AI systems to end users: a systematic literature review and research agenda
Laato, S., Tiainen, M., Najmul Islam, A. K. M., & Mäntymäki, M. (2022) · Internet Research · doi:10.1108/intr-08-2021-0600 · Cited by 155 · see citing works
Discrimination, artificial intelligence, and algorithmic decision-making
Borgesius, F. Z. (2025) · arXiv · doi:10.48550/arxiv.2510.13465 · Cited by 152 · see citing works
AI Systems and Respect for Human Autonomy
Laitinen, A., & Sahlgren, O. (2021) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2021.705164 · Cited by 154 · see citing works
“Empirical research also supports the notion that algorithmic decisions involve risks of objecti fi cation, of individuals being reduced to mere numbers or percentages, as it were (Binns et al., 2018).”
BackgroundUnderstanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations
Chen, V., Liao, Q. V., Wortman Vaughan, J., & Bansal, G. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610219 · Cited by 145 · see citing works
“This approach helps people understand not only how the model makes decisions but also how it might make mistakes [8, 25, 49].”
BackgroundThe impact of using algorithms for managerial decisions on public employees' procedural justice
Nagtegaal, R. (2021) · Government Information Quarterly · doi:10.1016/j.giq.2020.101536 · Cited by 166 · see citing works
Adversarial XAI Methods in Cybersecurity
Kuppa, A., & Le‐Khac, N. (2021) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2021.3117075 · Cited by 146 · see citing works
Ma, S., Lei, Y., Wang, X., Zheng, C., Shi, C., Yin, M., & Ma, X. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581058 · Cited by 143 · see citing works
Artificial intelligence explainability: the technical and ethical dimensions
McDermid, J. A., Jia, Y., Porter, Z., & Habli, I. (2021) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2020.0363 · Cited by 136 · see citing works
“Research suggests that the provision of sufficiently detailed explanations can affect the acceptance of algorithmic decisions by users—primarily expert-users [25].”
BackgroundHuman-LLM Collaborative Annotation Through Effective Verification of LLM Labels
Wang, X., Kim, H., Rahman, S., Mitra, K., & Miao, Z. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3641960 · Cited by 139 · see citing works
Artificial intelligence and the value of transparency
Walmsley, J. (2021) · AI & SOCIETY · doi:10.1007/s00146-020-01066-z · Cited by 131 · see citing works
Formosa, P., Rogers, W., Griep, Y., Bankins, S., & Richards, D. (2022) · Computers in Human Behavior · doi:10.1016/j.chb.2022.107296 · Cited by 129 · see citing works
“This finding fits with existing research about the dehumanizing impacts of AI decision making (Binns et al., 2018) and perceptions about the inap-propriateness of AI making some morally significant decisions (Formosa & Ryan, 2021).”
BackgroundResultInfluentialHuman Judgment in algorithmic loops: Individual justice and automated <scp>decision‐making</scp>
Binns, R. (2022) · Regulation & Governance · doi:10.1111/rego.12358 · Cited by 134 · see citing works
Self-citationMorley, J., Floridi, L., Kinsey, L., & Elhalal, A. (2021) · Philosophical studies series · doi:10.1007/978-3-030-81907-1_10 · Cited by 178 · see citing works
Molina, M. D., & Sundar, S. S. (2024) · New Media & Society · doi:10.1177/14614448221103534 · Cited by 124 · see citing works
“…set of studies, participants expressed more acceptance of AI compared with humans for content moderation because decisions made by AI are at least based on the same constant rules being applied (Binns et al., 2018; Jhaver et al., 2018), and, thus, are “statisti-cally fair” (Binns et al., 2018: 9).”
Background"Why is 'Chicago' deceptive?" Towards Building Model-Driven Tutorials for Humans
Lai, V., Liu, H., & Tan, C. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376873 · Cited by 123 · see citing works
“In addition to studies mentioned in the introduction, researchers have investigated other desiderata of explanations [5, 8, 17, 18, 32, 49, 65].”
BackgroundHuman-XAI Interaction: A Review and Design Principles for Explanation User Interfaces
Chromik, M., & Butz, A. M. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-85616-8_36 · Cited by 128 · see citing works
“They emphasize either (i) the descriptive accuracy of an explanation to describe the underlying AI behavior [26, 30, 48, 56, 68] or (ii) the capacity of a single explanation style [4] or differences between explanation styles [9, 14, 23, 25, 106, 83] to convey information about the behavior to the human.”
BackgroundThe Who in XAI: How AI Background Shapes Perceptions of AI Explanations
Ehsan, U., Passi, S., Liao, Q. V., Chan, L., Lee, I.-H., Muller, M., & Riedl, M. O. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642474 · Cited by 123 · see citing works
Helberger, N., Araujo, T., & de Vreese, C. H. (2020) · Computer law & security review · doi:10.1016/j.clsr.2020.105456 · Cited by 121 · see citing works
Springer, A., & Whittaker, S. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302322 · Cited by 115 · see citing works
“These systems provide information ranging from routes to work to recommendations about criminal parole [3,8].”
BackgroundExploring Explainability: A Definition, a Model, and a Knowledge Catalogue
Chazette, L., Brunotte, W., & Speith, T. (2021) · IEEE International Requirements Engineering Conference (RE) · doi:10.1109/re51729.2021.00025 · Cited by 120 · see citing works
XAI Systems Evaluation: A Review of Human and Computer-Centred Methods
Lopes, P., Silva, E., Braga, C., Oliveira, T., & Rosado, L. (2022) · Applied Sciences · doi:10.3390/app12199423 · Cited by 112 · see citing works
This Thing Called Fairness: Disciplinary Confusion Realizing a Value in Technology
Mulligan, D. K., Kroll, J. A., Kohli, N., & Wong, R. Y. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359221 · Cited by 111 · see citing works
“Research by social scientists and humanities scholars has similarly advanced the state of theory and application [5, 11, 80, 136, 149].”
BackgroundResponsible AI Systems: Who are the Stakeholders?
Deshpande, A., & Sharp, H. (2022) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3514094.3534187 · Cited by 110 · see citing works
“The need for the HCI community to draw on organisational behavioural psychology literature to inform the design of AI systems and their decision-making is highlighted by [45] [11] [31].”
Investigating How Practitioners Use Human-AI Guidelines: A Case Study on the People + AI Guidebook
Yildirim, N., Pushkarna, M., Goyal, N., WATTENBERG, M. P., & Viégas, F. B. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580900 · Cited by 107 · see citing works
“AI products1 canmake inference errors that are difficult to anticipate, leading to unpredictable behaviors, breakdowns in user experience (UX) and harm [10].”
BackgroundApplicants’ Fairness Perceptions of Algorithm-Driven Hiring Procedures
Lavanchy, M., Reichert, P., Narayanan, J., & Savani, K. (2023) · Journal of Business Ethics · doi:10.1007/s10551-022-05320-w · Cited by 120 · see citing works
“Recent work studying justice perceptions of algorithms has found that there is ‘no best approach’ in explaining algorithmic decisions (Binns et al., 2018).”
BackgroundResultFrom Responsibility to Reason-Giving Explainable Artificial Intelligence
Baum, K., Mantel, S., Schmidt, E., & Speith, T. (2022) · Philosophy & Technology · doi:10.1007/s13347-022-00510-w · Cited by 103 · see citing works
“…problems like those mentioned in the previous paragraphs; they provide arguments for XAI from the broader context of morality or society in general (e.g., Asaro, 2015; Binns et al., 2018; Cave et al., 2018; Floridi et al., 2018; Langer, Oster, et al., 2021; Lipton, 2018; Wachter et al., 2017).”
BackgroundAnik, A. I., & Bunt, A. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445736 · Cited by 103 · see citing works
“Evaluating the Impact of Explanations In parallel to developing different explanation approaches, numerous studies have investigated the impact of explanations on user perceptions of and interactions with the systems [5,14,20,33,59,60,86,102,105].”
BackgroundMethodInfluentialAlgorithmic discrimination in the credit domain: what do we know about it?
García, A. C. B., García, M., & C, R. R. B. (2023) · AI & Society · doi:10.1007/s00146-023-01676-3 · Cited by 127 · see citing works
“Binns et al. (2018) developed an empirical lab study asking 19 UK participants for their perception of fairness upon decisions proposed by an ML algorithm in three different scenarios, including credit loan application.”
DeVos, A., Dhabalia, A., Shen, H., Holstein, K., & Eslami, M. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517441 · Cited by 98 · see citing works
“behaviors will be shaped, in part, by folk theories they hold about the workings of a given algorithmic system [26, 28] and by their existing perceptions and beliefs about algorithmic bias and unfairness in a given context [12, 15, 24, 38, 75, 80] — both of which are”
BackgroundShifting Concepts of Value: Designing Algorithmic Decision-Support Systems for Public Services
Holten Møller, N., Shklovski, I., & Hildebrandt, T. T. (2020) · Proceedings of the 11th Nordic Conference on Human-Computer Interaction: Shaping Experiences, Shaping Society · doi:10.1145/3419249.3420149 · Cited by 97 · see citing works
“Metrics can show an improvement in the public registered status of a person, the number of complaints over a process, or if criteria were met for due process [3, 34, 36].”
BackgroundExplainable AI for Industry 5.0: Vision, Architecture, and Potential Directions
Trivedi, C., Bhattacharya, P., Prasad, V. K., Patel, V., Singh, A., Tanwar, S., Sharma, R., Aluvala, S., Pau, G., & Sharma, G. (2024) · IEEE Open Journal of Industry Applications · doi:10.1109/ojia.2024.3399057 · Cited by 106 · see citing works
“To determine how users perceive fairness in rational judgments, Binns et al. [42] studied explanations in systems for everyday tasks such as calculating premium rates and loan application clearances.”
Background”Because AI is 100% right and safe”: User Attitudes and Sources of AI Authority in India
Kapania, S., Siy, O., Clapper, G., SP, A. M., & Sambasivan, N. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517533 · Cited by 90 · see citing works
An Overview of Fairness in Clustering
Chhabra, A., Masalkovaitė, K., & Mohapatra, P. (2021) · IEEE Access · doi:10.1109/access.2021.3114099 · Cited by 92 · see citing works
“There is also a lot of scope in borrowing from similar efforts that assess perceived fairness in algorithmic decision-making systems such as [186].”
BackgroundHuman intervention in automated decision-making: Toward the construction of contestable systems
Almada, M. (2019) · Proceedings of the Seventeenth International Conference on Artificial Intelligence and Law · doi:10.1145/3322640.3326699 · Cited by 95 · see citing works
Progressive Disclosure: When, Why, and How Do Users Want Algorithmic Transparency Information?
Springer, A., & Whittaker, S. (2020) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3374218 · Cited by 84 · see citing works
“These systems provide information ranging from route planning to recommendations about criminal parole [3, 8, 10, 64].”
BackgroundEffect of Information Presentation on Fairness Perceptions of Machine Learning Predictors
van Berkel, N., Goncalves, J., Russo, D., Hosio, S., & Skov, M. B. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445365 · Cited by 84 · see citing works
“A case-based explanation, in which a representative example from the training data is presented, resulted in the most consistent negative impact on participants’ justice perceptions [7].”
BackgroundFairness Perceptions of Artificial Intelligence: A Review and Path Forward
Narayanan, D., Nagpal, M., McGuire, J., Schweitzer, S., & Cremer, D. D. (2023) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2023.2210890 · Cited by 87 · see citing works
“:3) ca et al. (2018b) and Binns et al. (2018), it was found that when fairness perceptions of the decision-making process are high (procedural fairness), this in turn also generally promoted fairness perceptions of decision outcome (distributive fairness).”
BackgroundMaking data science systems work
Passi, S., & Sengers, P. (2020) · Big Data & Society · doi:10.1177/2053951720939605 · Cited by 81 · see citing works
“One reason for this is that the perspective of people facing the most significant effects from a system’s output is often not included in the system’s design and development process (Binns et al., 2018).”
BackgroundHuman-AI Collaboration for UX Evaluation: Effects of Explanation and Synchronization
Fan, M., Yang, X., Yu, T., Liao, Q. V., & Zhao, J. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3512943 · Cited by 81 · see citing works
“Following this collaborative perspective, researchers studied how people perceive algorithmic decision assistance [10, 52], how they use AI systems for decision-making [13, 14], often in unexpected or suboptimal ways [18, 93], and how to improve the human-AI joint decision-making outcomes [8, 89, 95].”
BackgroundSelf-citationXAIR: A Framework of Explainable AI in Augmented Reality
Xu, X., Yu, A., Jonker, T. R., Todi, K., Lu, F., Qian, X., Belo, J. M. E., Wang, T., Li, M., Mun, A., et al. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581500 · Cited by 84 · see citing works
“Explanations can fulfill users’ curiosity by providing more information [33, 115, 172].”
BackgroundVaccaro, K., Sandvig, C., & Karahalios, K. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3415238 · Cited by 102 · see citing works
Bridging Machine Learning and Mechanism Design towards Algorithmic Fairness
Finocchiaro, J., Maio, R., Monachou, F., Patro, G. K., Raghavan, M., Stoica, A.-A., & Tsirtsis, S. (2021) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3442188.3445912 · Cited by 83 · see citing works
Choi, D., Lee, S., Kim, S.-I., Lee, K., Yoo, H. J., Lee, S., & Hong, H. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3641989 · Cited by 79 · see citing works
“Propelled by research examining AI bias related to gender and race [9, 34, 42, 63, 72], recent years have seen growing attention to AI’s inclusivity toward identity-based minority groups, such as people with disabilities [38–40, 51, 96, 99, 101, 109] and LGBTQ communities [27, 82, 103].”
BackgroundEffects of Explanations in AI-Assisted Decision Making: Principles and Comparisons
Wang, X., & Yin, M. (2022) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3519266 · Cited by 82 · see citing works
“Typical methods include asking people to rank the input data features based on their influence to overall predictions [21, 37], to indicate the direction of change in the model’s prediction when a feature’s value is altered [18, 21, 33], to simulate the model’s predictions [18, 27, 47, 54, 68], to answer “what-if” que…”
BackgroundMethodCrowdsourcing Perceptions of Fair Predictors for Machine Learning
van Berkel, N., Gonçalves, J., Hettiachchi, D., Wijenayake, S., Kelly, R., & Kostakos, V. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359130 · Cited by 73 · see citing works
Yurrita, M., Draws, T., Balayn, A., Murray-Rust, D., Tintarev, N., & Bozzon, A. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581161 · Cited by 73 · see citing works
Algorithms in the public sector. Why context matters
Wenzelburger, G., König, P. D., Felfeli, J., & Achtziger, A. (2022) · Public Administration · doi:10.1111/padm.12901 · Cited by 77 · see citing works
Explanations, Fairness, and Appropriate Reliance in Human-AI Decision-Making
Schoeffer, J., De‐Arteaga, M., & Kühl, N. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642621 · Cited by 87 · see citing works
“At the time of taking the survey, 13.5% of participants were 18–24 years old, 32.6% were 25–34 years old, 21.3% between 35–44, 13.8% between 45– 54,”
BackgroundMethodInfluentialHow Experienced Designers of Enterprise Applications Engage AI as a Design Material
Yildirim, N., Kass, A., Tung, T., Upton, C., Costello, D., Giusti, R., Lacin, S., Lovic, S., O'Neill, J. M., Meehan, R. O., et al. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517491 · Cited by 71 · see citing works
“For example, researchers studied users’ perceptions related to fairness in AI systems [5, 48, 89], and practitioners’ challenges and needs around designing responsible AI [37, 84].”
BackgroundToward involving end-users in interactive human-in-the-loop AI fairness
Nakao, Y., Stumpf, S., Ahmed, S., Naseer, A., & Strappelli, L. (2022) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3514258 · Cited by 69 · see citing works
“The need to directly involve end-users in assessing the fairness of AI systems and making ML models fairer has been emphasized [10, 21, 40, 69, 78, 80], and we believe our tool is a step in that direction.”
Leading with AI in critical care nursing: challenges, opportunities, and the human factor
Hassan, E. A., & El-Ashry, A. M. (2024) · BMC Nursing · doi:10.1186/s12912-024-02363-4 · Cited by 72 · see citing works
“The need for transparency is further supported by a study conducted by Binns et al. (2018) [33], which found that healthcare professionals are likelier to trust and utilize AI systems that provide understandable and transparent decision-making processes.”
Background
- Authors
- Binns, R., Van Kleek, M., Veale, M., Lyngs, U., Zhao, J., & Shadbolt, N.
- Affiliations
- University of Oxford (GB)
- University of Oxford, Oxford, Oxfordshire , United Kingdom
- University College London (GB)
- Venue
- Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2018-04-21
- Pages
- 1-14
- OpenAlex
- W4296978576
- Semantic Scholar
- aba609e6c5c9b58f9dac272b75983f75a7f190e5
- Cited by
- 691
- References
- 92
- Citations by year
- 2026: 139 · 2025: 204 · 2024: 87 · 2023: 60 · 2022: 76 · 2021: 53 · 2020: 35 · 2019: 31 · 2018: 6
- FWCI
- 56.17
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Funders
- UK Engineering and Physical Sciences Research Council (EP/J017728/2)
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Political Science and International Relations › Artificial Intelligence in Law
- Keywords
- algorithmic decision-making, perceived justice, algorithmic transparency, explanation styles, accountability, arbitrariness, indignity, generalization, automated decision-making, EU data protection law, lending, insurance quotes
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-06.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-06.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
