The DOI matches its Crossref record; 90 of the 91 references it cites were found in a registry or an index; 14 of the 25 sources its claims cite were checked for support.
Contestable AI by Design: Towards a Framework
Open accessAbstract
As the use of AI systems continues to increase, so do concerns over their lack of fairness, legitimacy and accountability. Such harmful automated decision-making can be guarded against by ensuring AI systems are contestable by design: responsive to human intervention throughout the system lifecycle. Contestable AI by design is a small but growing field of research. However, most available knowledge requires a significant amount of translation to be applicable in practice. A proven way of conveying intermediate-level, generative design knowledge is in the form of frameworks. In this article we use qualitative-interpretative methods and visual mapping techniques to extract from the literature sociotechnical features and practices that contribute to contestable AI, and synthesize these into a design framework.
Overview
CoChat Summary
TL;DR
The paper synthesizes literature on contestable AI into a preliminary design framework of five system features and six development practices, using qualitative-interpretative methods and visual mapping. It aims to provide intermediate-level generative design knowledge for making AI systems open and responsive to human intervention throughout their lifecycle.
Key findings
- The framework identifies five features contributing to contestable AI: built-in safeguards against harmful behavior; interactive control over automated decisions; explanations of system behavior; human review and intervention requests; and tools for scrutiny by subjects or third parties.
- It identifies six practices: ex-ante safeguards; agonistic approaches to ML development; quality assurance during development; quality assurance after deployment; risk mitigation strategies; and third-party oversight.
- Features are mapped to actors in typical AI systems (system developers, human controllers, decision subjects, third parties), while practices are mapped to AI lifecycle phases from the ICO auditing framework.
- Contestability is framed as a deep system property and procedural justice requiring dialectical exchange, not merely a “human in the loop”; explanations should include both the “how” and the “why”/justification of decisions.
Methods
- Systematic review broadly following Moher et al. (2009): Scopus search for journal articles and conference papers published between 2016 and 2021 mentioning “AI,” “contestability,” and “design” plus synonyms; duplicates removed, titles/abstracts screened in Rayyan, full-text eligibility assessed, then backward and forward snowballing.
- Analysis adapted from reflexive thematic analysis (Braun & Clarke, 2006) in Atlas.ti version 22 on MacOS: passages on actionable sociotechnical properties were selected, grouped, labeled, and examined for hierarchical and lateral relationships.
- Synthesis used visual mapping techniques and existing diagrams; the preliminary framework was checked against selected passages and an end-to-end read-through of the source literature.
Limitations
- Stated in the paper: the framework is based on a small sample of academic papers, may have gaps from lack of coverage, draws on sources from specific fields (e.g., ethics of technology, computer science, law), and many source papers are not empirically validated; most content remains “context-free,” validation was not part of the paper, and usability is limited because no directions for using the framework to design contestable AI are offered.
- Stated in the paper: the framework assumes a generic automated decision-making setting with significant impact and relatively low time-sensitivity; it likely does not cover extreme high-stakes contexts such as lethal autonomous weapons or shared control of autonomous vehicles, and the authors’ empirical work is situated in local government public services in OECD countries.
- Evident from the text: the framework is explicitly preliminary and provisional, intended as a starting point for subsequent testing and validation in specific application contexts; no empirical evaluation of its effectiveness is reported.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
13 Valid · 4 Not valid · 6 Not checked · 37 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“As a result, AI systems contain a particularly high degree of uncertainty and unpredictability.”
§ Contestable AI by Design: Towards a Framework > 3 Responsible Design for AI
What the sources say
- Partly supportedcites van de Poel, 2020 sourceExperimental — accuracy not yet measured
The passage labeled “1 3” (page unavailable) says unintended consequences are “probably more endemic” in AI systems because of their learning capabilities. The Conclusion likewise discusses unintended and unforeseen consequences. These passages support heightened unpredictability, but they hedge the comparison and do not establish the claim’s unqualified “particularly high degree” of uncertainty in general. (LLM, conf=0.82)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Corrections, appeals, and additional contextual information from human controllers and decision subjects can be used as feedback signals for the decision-making process as a whole (Hirsch et al., 2017;Vaccaro et al., 2020).”
§ Contestable AI by Design: Towards a Framework > 6.2.4 Quality Assurance After Deployment
What the sources say
- Supportedcites Hirsch et al., 2017 source
In “Facilitating Contestation” (page unavailable), the paper calls for users to challenge predictions and provide arguments and contextual details for decisions informed by the system. “Incentivizing Participation” says user corrections revise session reports immediately and user edits can improve models and feedback over time. Together, these passages support the claim that such input can feed back into the decision-making process. (LLM, conf=0.86)
- Not checked — no full text of the sourcecites Vaccaro et al., 2020 source
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The practices are: 1. ex-ante safeguards; 2. agonistic approaches to machine learning (ML) development; 3. quality assurance during development; 4. quality assurance after deployment; 5. risk mitigation strategies; and 6. third-party oversight.”
§ Contestable AI by Design: Towards a Framework > 1 Introduction
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The features are: 1. built-in safeguards against harmful behavior; 2. interactive control over automated decisions; 3. explanations of system behavior; 4. human review and intervention requests; and 5. tools for scrutiny by subjects or third parties.”
§ Contestable AI by Design: Towards a Framework > 1 Introduction
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The practices are: 1. ex-ante safeguards; 2. agonistic approaches to machine learning (ML) development; 3. quality assurance during development; 4. quality assurance after deployment; 5. risk mitigation strategies; and 6. third-party oversight.”
§ Contestable AI by Design: Towards a Framework > 1 Introduction
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
15 more claims
Create a free account to see all 60 claims and every source verdict.
- It may state two separate claims as one. 11
- Probably not a substantial claim. 36
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“The feld of contestable AI is still growing [3] and many of the guidelines on how to design for contestability are conceptual in nature [3,44,67].”
From Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability (2023) · cites this paper as [3]
Citations
Cites
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & The PRISMA Group (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000097 · Cited by 64,042 · cited 1×
“Our data-collection procedure broadly follows Moher et al. (2009).”
Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & the PRISMA Group* (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-4-200908180-00135 · Cited by 37,554
Rayyan-a web and mobile app for systematic reviews.
Ouzzani, M., Hammady, H., Fedorowicz, Z., & Elmagarmid, A. (2016) · Systematic Reviews · doi:10.1186/s13643-016-0384-4 · Cited by 26,092 · cited 1×
“Then, using Rayyan (Ouzzani et al., 2016), we manually screened records’ titles and abstracts for actually referring to contestability (rather than e.g. “contest” in the sense of a competition).”
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555 · cited 1×
“However, once again we emphasize not to fixate on the technical components of AI systems in isolation (Hirsch et al., 2017; Leahu, 2016; Mahendran & Vedaldi, 2015; Ribeiro et al., 2016; Tickle et al., 1998 in Edwards & Veale, 2018).”
“Why Should I Trust You?”: Explaining the Predictions of Any Classifier
Ribeiro, M., Singh, S., & Guestrin, C. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations · doi:10.18653/v1/n16-3020 · Cited by 6,682
The global landscape of AI ethics guidelines
Jobin, A., Ienca, M., & Vayena, E. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0088-2 · Cited by 6,220 · cited 2×
“To illustrate this point we briefly summarize a number of prominent systematic reviews and meta-analyses drawn from across disciplines (Jobin et al., 2019; Morley et al., 2019; Shneiderman, 2020).”
Model Cards for Model Reporting
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287596 · Cited by 2,681 · cited 1×
“Mitchell et al. (2019) and Gebru et al. (2020) offer examples of possible documentation approaches.”
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Daumé, H., & Crawford, K. (2018) · arXiv · doi:10.48550/arxiv.1803.09010 · Cited by 2,474
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., III, H. D., & Crawford, K. (2021) · Communications of the ACM · doi:10.1145/3458723 · Cited by 2,469 · cited 1×
“Mitchell et al. (2019) and Gebru et al. (2020) offer examples of possible documentation approaches.”
Research works printed but not linked: 3
- Goodman, B. (2016). Economic models of (algorithmic) discrimination. In 29th conference on neural information processing systems
- Kluttz, D. N., Mulligan, D. K., Mulligan, D. K., et al. (2019). Shaping our tools: Contestability as a means to promote responsible algorithmic decision making in the professions. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.3311894. · DOI · Paper page
- Winner, L. (1980). Do artifacts have politics? Daedalus, 109(1), 121–136.
Cited by
Mehrotra, S., Degachi, C., Vereschak, O., Jonker, C. M., & Tielman, M. L. (2024) · ACM Journal on Responsible Computing · doi:10.1145/3696449 · Cited by 254
Yurrita, M., Draws, T., Balayn, A., Murray-Rust, D., Tintarev, N., & Bozzon, A. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581161 · Cited by 108
“Contestability is said to “surface values” [92] and to be a “form of procedural justice, a way of giving voice to decision subjects, which increases perceptions of fairness” [3].”
BackgroundTrust, trustworthiness and AI governance
Lahusen, C., Maggetti, M., & Slavkovik, M. (2024) · Scientific Reports · doi:10.1038/s41598-024-71761-0 · Cited by 68
Verma, H., Mlynář, J., Schaer, R., Reichenbach, J., Jreige, M., Prior, J. O., Évéquoz, F., & Depeursinge, A. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581506 · Cited by 78
“To overcome the challenges posed by the functional realization of explainability, recommendations have been made to calibrate (clinical) practices surrounding the use of AI, so that their decisions and impact can be contested [2, 40, 62, 74, 103].”
MethodAlfrink, K., Keller, I., Doorn, N., & Kortuem, G. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580984 · Cited by 69
“Together they show how, as Alfrink et al. [2] propose, “contestability leverages conflict for continuous system improvement.”
BackgroundMethodInfluentialSelf-citation
Metadata
- Authors
- Alfrink, K., Keller, I., Kortuem, G., & Doorn, N.
- Affiliations
- Delft University of Technology (NL)
- Corresponding author
- Alfrink, Kars
- Venue
- Minds and Machines
- Published
- 2022-08-13
- Volume
- 33
- Issue
- 4
- Pages
- 613-639
- ISSN
- 0924-6495 (ISSN-L)
- 1572-8641
- OpenAlex
- W4291197616
- Cited by
- 169
- References
- 79
- Citations by year
- 2026: 97 · 2025: 38 · 2024: 21 · 2023: 13
- FWCI
- 4.82
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Minds and Machines (publishedVersion, cc-by)
- Research Repository (Delft University of Technology) (submittedVersion, cc-by)
- Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
- Funders
- Nederlandse Organisatie voor Wetenschappelijk Onderzoek (CISC.CC.018)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Safety Systems Engineering in Autonomy
- Keywords
- design framework, human intervention, automated decision-making, fairness, accountability, legitimacy, visual mapping
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Contestable AI by Design: Towards a Framework. https://app.cochat.ai/papers/10.1007/s11023-022-09611-z
- The paper
- Alfrink, K., Keller, I., Kortuem, G., & Doorn, N. (2022). Contestable AI by Design: Towards a Framework. Minds and Machines. https://doi.org/10.1007/s11023-022-09611-z
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.