CoChat

Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability

Open access
Yurrita, Mireia, Draws, Tim, Balayn, Agathe, Murray-Rust, Dave, Tintarev, Nava, Bozzon, Alessandro · 2023 · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

The DOI matches its Crossref record; 112 of the 114 references it cites were found in a registry or an index; 22 of the 48 sources its claims cite were checked for support.

Abstract

Recent research claims that information cues and system attributes of algorithmic decision-making processes affect decision subjects’ fairness perceptions. However, little is still known about how these factors interact. This paper presents a user study (N = 267) investigating the individual and combined effects of explanations, human oversight, and contestability on informational and procedural fairness perceptions for high- and low-stakes decisions in a loan approval scenario. We find that explanations and contestability contribute to informational and procedural fairness perceptions, respectively, but we find no evidence for an effect of human oversight. Our results further show that both informational and procedural fairness perceptions contribute positively to overall fairness perceptions but we do not find an interaction effect between them. A qualitative analysis exposes tensions between information overload and understanding, human involvement and timely decision-making, and accounting for personal circumstances while maintaining procedural consistency. Our results have important design implications for algorithmic decision-making processes that meet decision subjects’ standards of justice.

Overview

CoChat Summary

TL;DR

This preregistered user study (N=267) tests how explanations, human oversight, contestability, and task stakes affect informational, procedural, and overall fairness perceptions in a fictional loan approval scenario. Explanations increased informational fairness and contestability increased procedural fairness, while human oversight and task stakes showed no evidence of effects; informational and procedural fairness independently predicted overall fairness.

Key findings

  • Explanations had a main effect on informational fairness perceptions: F(1, 260)=74.21, p<0.001, partial η²=0.22, BF₁₀>1000. Contestability had a main effect on procedural fairness perceptions: F(2, 254)=20.60, p<0.001, partial η²=0.14, BF₁₀>1000.
  • No evidence supported effects of human oversight on procedural fairness: F(1, 254)=0.004, p=0.95, BF₀₁=7.43; nor task stakes moderating explanations on informational fairness: F(1, 260)=0.01, p=0.92, BF₀₁=7.44.
  • Informational fairness (β=0.27, p<0.001) and procedural fairness (β=0.87, p<0.001) both predicted overall fairness in a regression model (R²=0.46, F(3, 263)=76.02, p<0.001), but their interaction was not significant (β=-0.09, p=0.07).
  • Exploratory analyses found contestability mainly improved correctability (F(2, 254)=108.29, p<0.001, partial η²=0.46), yet procedural voice and outcome influence means remained below zero across all contestability conditions; human oversight slightly negatively affected process consistency and lack of bias.

Methods

  • Preliminary study (N=58) selected explanation styles and contestation targets; main preregistered online study recruited 279 Prolific participants, with 267 retained after attention checks.
  • Main study used a 2×2×3×2 between-subjects design in a fictional loan approval scenario, manipulating explanations (no/with), human oversight (no/with), contestability (no contest / contest initial decision with additional information / contest decision-maker), and task stakes (holiday loan vs. home loan).
  • Dependent variables were informational, procedural, and overall fairness perceptions; analyses used multi-way ANOVAs, multiple linear regression, Bonferroni-corrected threshold p<0.0045, Bayesian tests, exploratory sub-element analyses, and reflexive thematic analysis of open-ended responses.

Limitations

  • The paper states the between-subjects design may have prevented participants from comparing configurations, potentially diluting task-stakes and human-oversight effects; the scenario was close to the decision boundary; the third-person, one-shot, no-demographics design was chosen to reduce outcome favorability bias but may limit realism; findings are limited to a loan scenario; terminology may affect perceptions; participants were from the Global North with English proficiency, limiting representativeness.
  • The paper also notes that using a fairness measurement tool designed for human decision-making may not capture algorithmic-specific concerns, such as the temporal dimension of decision-making, which participants valued in qualitative responses.
  • Evident from the text: the study relies on a fictional loan scenario and Prolific sample, so results may not generalize to other decision contexts or populations.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1145/3544548.3581161

Cites

112 of 114 references verified1 citation errors · 1 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

18 Valid · 1 Not valid · 19 Not checked · 101 Own results and interpretations

68 more claims · 2 with their cited sources checked

Create a free account to see all 139 claims and every source verdict.

See all claims — free account
  • Probably not a substantial claim. 56
  • It may state two separate claims as one. 19
  • The quote only partly states this claim. 2

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 114 references

Research works printed but not linked: 9

  1. Kars Alfrink, T. Turel, A. I. Keller, N. Doorn, and G. W. Kortuem. 2020. Contestable City Algorithms. International Conference on Machine Learning Workshop.
  2. Simone Bae Reeva Lederman and Tingru Cui. 2022. Understanding User Perception of Explainable Algorithmic Decision-Making Systems: A Systematic Literature Review. (2022).
  3. David Chan. 2011. Perceptions of fairness. Research Collection School of Social Sciences (2011).
  4. Nina Grgic-Hlaca Muhammad Bilal Zafar Krishna P. Gummadi and Adrian Weller. 2016. The Case for Process Fairness in Learning: Feature Selection for Fair Decision Making. In NIPS SYMPOSIUM ON MACHINE LEARNING AND THE LAW 8.
  5. 10.5555/3157382.3157469 · doi:10.5555/3157382.3157469
  6. 10.1093/jcr · doi:10.1093/jcr
  7. Cathy O’neil. 2016. Weapons of math destruction: How big data increases inequality and threatens democracy.Broadway Books.
  8. 10.1093/idpl · doi:10.1093/idpl
  9. Elizabeth Anne Watkins. 2021. The tension between information justice and security: Perceptions of facial recognition targeting.. In Joint Proceedings of the ACM IUI 2021 Workshops.

Cited by

See all 108

Metadata

Authors
Yurrita, M., Draws, T., Balayn, A., Murray-Rust, D., Tintarev, N., & Bozzon, A.
Affiliations
  • Delft University of Technology (NL)
  • Maastricht University (NL)
Venue
Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems
Publisher
ACM
Published
2023-04-19
Pages
1-21
OpenAlex
W4366549927
Cited by
108
References
106
Citations by year
2026: 39 · 2025: 35 · 2024: 28 · 2023: 6
FWCI
12.13
Type
conference paper
Language
en
Open access
Locations
  • CHI Conference on Human Factors in Computing Systems (publishedVersion, cc-by)
  • Research Publications (Maastricht University) (publishedVersion, cc-by)
  • Research Repository (Delft University of Technology) (submittedVersion)
  • Data Archiving and Networked Services (DANS) (submittedVersion, cc-by)
Funders
European Commission (HORIZON2020, COVID-19, 101016233, 955990)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
Keywords
algorithmic decision-making, perceived fairness, algorithmic explanations, human oversight, contestability, informational fairness, procedural justice, lending, high-stakes decisions, information overload

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. https://app.cochat.ai/papers/10.1145/3544548.3581161
The paper
Yurrita, M., Draws, T., Balayn, A., Murray-Rust, D., Tintarev, N., & Bozzon, A. (2023). Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3544548.3581161

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.