Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability
Open accessThe DOI matches its Crossref record; 112 of the 114 references it cites were found in a registry or an index; 22 of the 48 sources its claims cite were checked for support.
Abstract
Recent research claims that information cues and system attributes of algorithmic decision-making processes affect decision subjects’ fairness perceptions. However, little is still known about how these factors interact. This paper presents a user study (N = 267) investigating the individual and combined effects of explanations, human oversight, and contestability on informational and procedural fairness perceptions for high- and low-stakes decisions in a loan approval scenario. We find that explanations and contestability contribute to informational and procedural fairness perceptions, respectively, but we find no evidence for an effect of human oversight. Our results further show that both informational and procedural fairness perceptions contribute positively to overall fairness perceptions but we do not find an interaction effect between them. A qualitative analysis exposes tensions between information overload and understanding, human involvement and timely decision-making, and accounting for personal circumstances while maintaining procedural consistency. Our results have important design implications for algorithmic decision-making processes that meet decision subjects’ standards of justice.
Overview
CoChat Summary
TL;DR
This preregistered user study (N=267) tests how explanations, human oversight, contestability, and task stakes affect informational, procedural, and overall fairness perceptions in a fictional loan approval scenario. Explanations increased informational fairness and contestability increased procedural fairness, while human oversight and task stakes showed no evidence of effects; informational and procedural fairness independently predicted overall fairness.
Key findings
- Explanations had a main effect on informational fairness perceptions: F(1, 260)=74.21, p<0.001, partial η²=0.22, BF₁₀>1000. Contestability had a main effect on procedural fairness perceptions: F(2, 254)=20.60, p<0.001, partial η²=0.14, BF₁₀>1000.
- No evidence supported effects of human oversight on procedural fairness: F(1, 254)=0.004, p=0.95, BF₀₁=7.43; nor task stakes moderating explanations on informational fairness: F(1, 260)=0.01, p=0.92, BF₀₁=7.44.
- Informational fairness (β=0.27, p<0.001) and procedural fairness (β=0.87, p<0.001) both predicted overall fairness in a regression model (R²=0.46, F(3, 263)=76.02, p<0.001), but their interaction was not significant (β=-0.09, p=0.07).
- Exploratory analyses found contestability mainly improved correctability (F(2, 254)=108.29, p<0.001, partial η²=0.46), yet procedural voice and outcome influence means remained below zero across all contestability conditions; human oversight slightly negatively affected process consistency and lack of bias.
Methods
- Preliminary study (N=58) selected explanation styles and contestation targets; main preregistered online study recruited 279 Prolific participants, with 267 retained after attention checks.
- Main study used a 2×2×3×2 between-subjects design in a fictional loan approval scenario, manipulating explanations (no/with), human oversight (no/with), contestability (no contest / contest initial decision with additional information / contest decision-maker), and task stakes (holiday loan vs. home loan).
- Dependent variables were informational, procedural, and overall fairness perceptions; analyses used multi-way ANOVAs, multiple linear regression, Bonferroni-corrected threshold p<0.0045, Bayesian tests, exploratory sub-element analyses, and reflexive thematic analysis of open-ended responses.
Limitations
- The paper states the between-subjects design may have prevented participants from comparing configurations, potentially diluting task-stakes and human-oversight effects; the scenario was close to the decision boundary; the third-person, one-shot, no-demographics design was chosen to reduce outcome favorability bias but may limit realism; findings are limited to a loan scenario; terminology may affect perceptions; participants were from the Global North with English proficiency, limiting representativeness.
- The paper also notes that using a fairness measurement tool designed for human decision-making may not capture algorithmic-specific concerns, such as the temporal dimension of decision-making, which participants valued in qualitative responses.
- Evident from the text: the study relies on a fictional loan scenario and Prolific sample, so results may not generalize to other decision contexts or populations.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
18 Valid · 1 Not valid · 19 Not checked · 101 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Our results (e.g., Q.13) suggest that providing decision subjects with information that goes beyond outcome explanations could support contestations that are not only limited to post-decision mechanisms but that apply to the system lifecycle as a whole [2].”
§ Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Efects of Explanations, Human Oversight, and Contestability > Implications.
What the sources say
- Partly supportedcites [2] sourceExperimental — accuracy not yet measured
In “Experts believe explanations are actionable by citizens” (page unavailable), the paper describes information about inputs, processes, outcomes, design, and ownership—not just decision outcomes. In the discussion of citizens’ responses (page unavailable), it proposes dialogue with designers, developers, and operators, but says information alone is insufficient without channels for citizens to voice disagreements. It does not establish that providing broader information would support contestation across the system lifecycle as a whole. (LLM, conf=0.82)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“However, contrary to our expectations, and to suggestions from earlier work [83], we did not fnd evidence that explanations moderate the efect of contestability on procedural fairness, i.e., help participants question structural aspects of the decision-making process such as the factors requested by the bank and how these are used.”
§ Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Efects of Explanations, Human Oversight, and Contestability > 6.1 Leveraging Transparency Beyond Outcome Explanations
What the sources say
- Supportedcites [83] source
The citing sentence presents the absence of a moderation effect as its authors’ own finding, contrary to suggestions from earlier work [83]; it does not attribute that null finding to this source. In §6.1 (p. 7), the source reports that learning which factors the system used raised participants’ questions about why those factors were considered and how they affected decisions, supporting the earlier-work suggestion. (LLM, conf=0.88)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“Including humans in algorithmic decision-making processes costs time [20,29,68]”
§ Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Efects of Explanations, Human Oversight, and Contestability > 6.2 Designing Appropriate Human-AI Confgurations
What the sources say
- Not checked — no full text of the sourcecites [20] source
- Could not verifycites [29] sourceExperimental — accuracy not yet measured
The introduction (p. 1) contrasts human review with algorithmic forecasting, and the General Discussion (pp. 10–11) addresses algorithm aversion and forecasting accuracy. Neither addresses whether including humans in algorithmic decision-making costs time; searches for time and human involvement found no passage establishing that point. (LLM, conf=0.76)
- Not checked — no full text of the sourcecites [68] source
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Every source the paper cites for this claim was checked and supports it.
In the paper
“There is evidence that suggests that decision subjects care about justice-related aspects in algorithmic decisionmaking, as they care in human decision-making [17].”
§ Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Efects of Explanations, Human Oversight, and Contestability > 6.4 Multi-dimensional Measurement of Fairness Perceptions
What the sources say
- Supportedcites [17] source
The Conclusion (p. 10) directly states that people consider justice-related aspects of algorithmic decision-making much as they do in manual decision-making. The claim preserves the paper’s tentative framing by saying the evidence “suggests” this. (LLM, conf=0.94)
hedgedstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Despite our promising fndings, using a tool that was designed for human decision-making to evaluate algorithmic decision-making may not encompass the unique challenges that the inclusion of algorithmic systems bring to existing processes (as it is the case for other felds such as human-agent collaboration [23]).”
§ Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Efects of Explanations, Human Oversight, and Contestability > 6.4 Multi-dimensional Measurement of Fairness Perceptions
What the sources say
- Supportedcites [23] source
The citation supports the parenthetical analogy to human-agent collaboration, not a claim that Cila evaluated a decision-making tool. In the Discussion, Cila questions how far a framework for human-human collaboration transfers to human-agent collaboration and identifies distinct practical and ethical challenges. (LLM, conf=0.90)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
68 more claims · 2 with their cited sources checked
Create a free account to see all 139 claims and every source verdict.
- Probably not a substantial claim. 56
- It may state two separate claims as one. 19
- The quote only partly states this claim. 2
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Other explanation methods, such as LIME or Shapley values, are also commonly discussed as potentially fostering contestability (Mulligan, Kluttz, and Kohli 2019;Yurrita et al. 2023).”
From Explainable AI Isn't Enough! Rethinking Algorithmic Contestability (2026) · cites this paper as Yurrita et al. 2023
Experimental — accuracy not yet measured
Citations
Cites
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414
Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007) · Behavior Research Methods · doi:10.3758/bf03193146 · Cited by 65,971 · cited 2×
“We computed the required sample size using the software G*Power [34] for an ANOVA with main efects and interactions; specifying the default efect size of 0.”
MethodInfluentialAdams, J. S. (1965) · Advances in Experimental Social Psychology · doi:10.1016/s0065-2601(08)60108-2 · Cited by 10,818 · cited 2×
“, distributive fairness perceptions) [1, 28], the nature of the process that leads to those decisions (i.”
BackgroundThe Social Psychology of Procedural Justice
Lind, E. A., & Tyler, T. R. (1988) · Critical Issues in Social Justice · doi:10.1007/978-1-4899-2115-4 · Cited by 6,585 · cited 2×
“, procedural fairness perceptions) [62, 64, 89], the treatment received by decision subjects (i.”
BackgroundOn the dimensionality of organizational justice: A construct validation of a measure.
Colquitt, J. A. (2001) · Journal of Applied Psychology · doi:10.1037/0021-9010.86.3.386 · Cited by 4,114 · cited 6×
“For instance, the approach suggested by Colquitt [24] does not explicitly include the temporal dimension of the decision-making process as an attribute that contributes to perceptions of procedural fairness.”
BackgroundMethodInfluentialOrganizational Justice: Yesterday, Today, and Tomorrow
Greenberg, J. (1990) · Journal of Management · doi:10.1177/014920639001600208 · Cited by 3,651 · cited 2×
“Studies dealing with procedural fairness in human decision-making processes [39, 89] demonstrated that participants with a strong infuence over the decision-making process were more likely to perceive a negative outcome as fair [47].”
BackgroundDwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012) · Proceedings of the 3rd Innovations in Theoretical Computer Science Conference · doi:10.1145/2090236.2090255 · Cited by 3,501 · cited 1×
“Motivated by concerns about bias and discrimination in algorithmic decision-making [73], recent work has developed fairness-aware algorithmic systems [6, 33, 108] that ensure outcome distribution equity [32, 42].”
BackgroundAlgorithm aversion: People erroneously avoid algorithms after seeing them err.
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015) · Journal of Experimental Psychology: General · doi:10.1037/xge0000033 · Cited by 3,232
Doyle, T. (2017) · The Information Society · doi:10.1080/01972243.2017.1354593 · Cited by 3,071
What Should Be Done with Equity Theory?
Leventhal, G. S. (1980) · Social Exchange · doi:10.1007/978-1-4613-3087-5_2 · Cited by 3,010 · cited 7×
“, explanations, human oversight, contestability) that we manipulate in our experimental setting have been related to perceptions of informational and procedural fairness in human decision-making [62, 85], we capture perceptions of fairness across those two dimensions.”
BackgroundMethodInfluential
Research works printed but not linked: 9
- Kars Alfrink, T. Turel, A. I. Keller, N. Doorn, and G. W. Kortuem. 2020. Contestable City Algorithms. International Conference on Machine Learning Workshop.
- Simone Bae Reeva Lederman and Tingru Cui. 2022. Understanding User Perception of Explainable Algorithmic Decision-Making Systems: A Systematic Literature Review. (2022).
- David Chan. 2011. Perceptions of fairness. Research Collection School of Social Sciences (2011).
- Nina Grgic-Hlaca Muhammad Bilal Zafar Krishna P. Gummadi and Adrian Weller. 2016. The Case for Process Fairness in Learning: Feature Selection for Fair Decision Making. In NIPS SYMPOSIUM ON MACHINE LEARNING AND THE LAW 8.
- 10.5555/3157382.3157469 · doi:10.5555/3157382.3157469
- 10.1093/jcr · doi:10.1093/jcr
- Cathy O’neil. 2016. Weapons of math destruction: How big data increases inequality and threatens democracy.Broadway Books.
- 10.1093/idpl · doi:10.1093/idpl
- Elizabeth Anne Watkins. 2021. The tension between information justice and security: Perceptions of facial recognition targeting.. In Joint Proceedings of the ACM IUI 2021 Workshops.
Cited by
Deng, W. H., Yildirim, N., Chang, M., Eslami, M., Holstein, K., & Madaio, M. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594037 · Cited by 57
“tools and processes supporting mixed-method evaluation and development for AI fairness could potentially re-shape the quantification culture around AI development that runs the risk of compromising the socio-technical, contested nature of AI fairness [32, 34, 77, 98].”
BackgroundTransparency in AI for emergency management: building trust and accountability
Visave, J. (2025) · AI and Ethics · doi:10.1007/s43681-025-00692-x · Cited by 77
Karusala, N., Upadhyay, S., Veeraraghavan, R., & Gajos, K. Z. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3641898 · Cited by 58
“They and other researchers suggest that reflections on how algorithms were designed and why, decision-making flows, and known limitations could support scrutiny and protective action [17, 39, 109, 110].”
BackgroundInfluentialVereschak, O., Alizadeh, F., Bailly, G., & Caramiaux, B. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642018 · Cited by 72
Ghasemaghaei, M., & Kordzadeh, N. (2024) · Information Systems Journal · doi:10.1111/isj.12572 · Cited by 46
“Research across these fields has extensively examined the root causes and potential remedies for unfairness in algorithms, as well as their impacts on fairness perceptions and decision-making, often emphasising individual-level effects of algorithmic outputs (Yurrita et al. 2023).”
Metadata
- Authors
- Yurrita, M., Draws, T., Balayn, A., Murray-Rust, D., Tintarev, N., & Bozzon, A.
- Affiliations
- Delft University of Technology (NL)
- Maastricht University (NL)
- Venue
- Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2023-04-19
- Pages
- 1-21
- OpenAlex
- W4366549927
- Cited by
- 108
- References
- 106
- Citations by year
- 2026: 39 · 2025: 35 · 2024: 28 · 2023: 6
- FWCI
- 12.13
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion, cc-by)
- Research Publications (Maastricht University) (publishedVersion, cc-by)
- Research Repository (Delft University of Technology) (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion, cc-by)
- Funders
- European Commission (HORIZON2020, COVID-19, 101016233, 955990)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Keywords
- algorithmic decision-making, perceived fairness, algorithmic explanations, human oversight, contestability, informational fairness, procedural justice, lending, high-stakes decisions, information overload
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. https://app.cochat.ai/papers/10.1145/3544548.3581161
- The paper
- Yurrita, M., Draws, T., Balayn, A., Murray-Rust, D., Tintarev, N., & Bozzon, A. (2023). Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3544548.3581161
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.