Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 101 of the 109 references it cites were found in a registry or an index; 24 of the 96 sources its claims cite were checked for support.

Explainable recommendation: when design meets trust calibration

Open access
Naiseh, Mohammad, Al-Thani, Dena, Jiang, Nan, Ali, Raian · 2021 · World Wide Web

Abstract

Human-AI collaborative decision-making tools are being increasingly applied in critical domains such as healthcare. However, these tools are often seen as closed and intransparent for human decision-makers. An essential requirement for their success is the ability to provide explanations about themselves that are understandable and meaningful to the users. While explanations generally have positive connotations, studies showed that the assumption behind users interacting and engaging with these explanations could introduce trust calibration errors such as facilitating irrational or less thoughtful agreement or disagreement with the AI recommendation. In this paper, we explore how to help trust calibration through explanation interaction design. Our research method included two main phases. We first conducted a think-aloud study with 16 participants aiming to reveal main trust calibration errors concerning explainability in AI-Human collaborative decision-making tools. Then, we conducted two co-design sessions with eight participants to identify design principles and techniques for explanations that help trust calibration. As a conclusion of our research, we provide five design principles: Design for engagement, challenging habitual actions, attention guidance, friction and support training and learning. Our findings are meant to pave the way towards a more integrated framework for designing explanations with trust calibration as a primary goal.

Overview

CoChat Summary

## TL;DR

This qualitative study examines how explanation interface design can support trust calibration in Human-AI collaborative decision-making, using a screening-prescription mock-up with medical experts. From a think-aloud phase and two co-design sessions, the authors derive five design principles for XAI: engagement, challenging habitual actions, attention guidance, friction, and training/learning.

## Key findings

  • In the exploration phase, many participants made trust calibration errors by skipping or misapplying explanations; skipping was grouped into five categories: lack of curiosity, perceived goal impediment, redundant information, perceived complexity, and lack of context.
  • The study recruited 24 participants total: 16 in the exploration phase and 8 in the co-design phase.
  • In the design phase, participants proposed four main Human-Explanation interaction themes: abstraction, cues, control, and adaptation.
  • The final contribution is five design principles for calibrated-trust XAI: design for engagement, challenging habitual actions, attention guidance, friction, and support training and learning.

## Methods

  • Two-phase qualitative design: (1) a think-aloud exploration phase and (2) two co-design sessions to derive interface principles and techniques.
  • Sample and setting: medical experts from oncology departments in three UK organisations; the use case was an AI-supported screening prescription task in a web-based mock-up.
  • Analysis: audio transcriptions, observations, notes, and sketches were analyzed using content analysis with iterative coding and verification by the coauthors.

## Limitations

  • The paper states that the results are qualitative and that future controlled experiments are needed to map trust calibration errors to explanation types and to test whether the proposed design techniques have the intended effects.
  • The paper notes it was unable to pair configurations of the XAI model with the type of bias and error; this limits how specific the design guidance can be.
  • Evident from the text: the study used a mock-up and a single domain-specific scenario (screening prescriptions), so generalization beyond this context is not established in the paper.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1007/s11280-021-00916-0

Cites

101 of 101 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Figure 1 Screening prescription classification AI-based system classification
Figure 1 Screening prescription classification AI-based system classificationCC BY · PubMed Central
Figure 2 A sample of prescribing system interface supported with AI recommendations
Figure 2 A sample of prescribing system interface supported with AI recommendationsCC BY · PubMed Central
Figure 3 Exploration phase workflow
Figure 3 Exploration phase workflowCC BY · PubMed Central
Figure 4 Co-design session workflow
Figure 4 Co-design session workflowCC BY · PubMed Central
Figure 5 An example of two levels of abstraction design presented in our co-design study
Figure 5 An example of two levels of abstraction design presented in our co-design studyCC BY · PubMed Central
Fig. 6 An example of suggested control techniques designs for Local explanations in the co-design sessions
Fig. 6 An example of suggested control techniques designs for Local explanations in the co-design sessionsCC BY · PubMed Central
Figure 7 Friction design example for calibrated trust goal
Figure 7 Friction design example for calibrated trust goalCC BY · PubMed Central

Verification

Claims the paper makes

19 Valid · 0 Not valid · 55 Not checked · 129 Own results and interpretations

89 more claims · 4 with their cited sources checked

Create a free account to see all 203 claims and every source verdict.

See all claims — free account
  • Probably not a substantial claim. 82
  • It may state two separate claims as one. 49
  • The quote only partly states this claim. 1

Create a free account to see these claims.

Citations

Cites

Show all 109 references

Research works printed but not linked: 2

  1. Biran, O., Cotton, C: Explanation and justification in machine learning: a survey, pp. 8–13 (2017)
  2. Bradley, S.: Design Fundamentals: Elements, Attributes, & Principles. Vanseo design, Colorado (2013)

Cited by

See all 68

Metadata

Authors
Naiseh, M., Al-Thani, D., Jiang, N., & Ali, R.
Affiliations
  • Bournemouth University (GB)
  • Hamad bin Khalifa University (QA)
Corresponding author
  • Naiseh, Mohammad
  • Jiang, Nan
  • Ali, Raian
Venue
World Wide Web
Published
2021-08-02
Volume
24
Issue
5
Pages
1857-1884
ISSN
  • 1386-145X (ISSN-L)
  • 1573-1413
OpenAlex
W3191257699
Cited by
68
References
109
Citations by year
2026: 20 · 2025: 20 · 2024: 17 · 2023: 3 · 2022: 7 · 2021: 1
FWCI
2.46
Type
journal article
Language
en
Open access
Licence
Locations
  • World Wide Web (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Bournemouth University Research Online (Bournemouth University) (acceptedVersion)
  • ePrints Soton (University of Southampton) (submittedVersion, cc-by)
  • Aberdeen University Research Archive (Aberdeen University) (submittedVersion)
  • Figshare (submittedVersion, cc-by)
Funders
iQ HealthTech (01101)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
  • Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
Keywords
explainable recommendation, trust calibration, AI transparency, attention guidance, clinical decision support, think-aloud study, co-design

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-09-30, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-09-30 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Explainable recommendation: when design meets trust calibration. https://app.cochat.ai/papers/10.1007/s11280-021-00916-0
The paper
Naiseh, M., Al-Thani, D., Jiang, N., & Ali, R. (2021). Explainable recommendation: when design meets trust calibration. World Wide Web. https://doi.org/10.1007/s11280-021-00916-0

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.