Explainable recommendation: when design meets trust calibration
Open accessThe DOI matches its Crossref record; 101 of the 109 references it cites were found in a registry or an index; 24 of the 96 sources its claims cite were checked for support.
Abstract
Human-AI collaborative decision-making tools are being increasingly applied in critical domains such as healthcare. However, these tools are often seen as closed and intransparent for human decision-makers. An essential requirement for their success is the ability to provide explanations about themselves that are understandable and meaningful to the users. While explanations generally have positive connotations, studies showed that the assumption behind users interacting and engaging with these explanations could introduce trust calibration errors such as facilitating irrational or less thoughtful agreement or disagreement with the AI recommendation. In this paper, we explore how to help trust calibration through explanation interaction design. Our research method included two main phases. We first conducted a think-aloud study with 16 participants aiming to reveal main trust calibration errors concerning explainability in AI-Human collaborative decision-making tools. Then, we conducted two co-design sessions with eight participants to identify design principles and techniques for explanations that help trust calibration. As a conclusion of our research, we provide five design principles: Design for engagement, challenging habitual actions, attention guidance, friction and support training and learning. Our findings are meant to pave the way towards a more integrated framework for designing explanations with trust calibration as a primary goal.
Overview
CoChat Summary
## TL;DR
This qualitative study examines how explanation interface design can support trust calibration in Human-AI collaborative decision-making, using a screening-prescription mock-up with medical experts. From a think-aloud phase and two co-design sessions, the authors derive five design principles for XAI: engagement, challenging habitual actions, attention guidance, friction, and training/learning.
## Key findings
- In the exploration phase, many participants made trust calibration errors by skipping or misapplying explanations; skipping was grouped into five categories: lack of curiosity, perceived goal impediment, redundant information, perceived complexity, and lack of context.
- The study recruited 24 participants total: 16 in the exploration phase and 8 in the co-design phase.
- In the design phase, participants proposed four main Human-Explanation interaction themes: abstraction, cues, control, and adaptation.
- The final contribution is five design principles for calibrated-trust XAI: design for engagement, challenging habitual actions, attention guidance, friction, and support training and learning.
## Methods
- Two-phase qualitative design: (1) a think-aloud exploration phase and (2) two co-design sessions to derive interface principles and techniques.
- Sample and setting: medical experts from oncology departments in three UK organisations; the use case was an AI-supported screening prescription task in a web-based mock-up.
- Analysis: audio transcriptions, observations, notes, and sketches were analyzed using content analysis with iterative coding and verification by the coauthors.
## Limitations
- The paper states that the results are qualitative and that future controlled experiments are needed to map trust calibration errors to explanation types and to test whether the proposed design techniques have the intended effects.
- The paper notes it was unable to pair configurations of the XAI model with the type of bias and error; this limits how specific the design guidance can be.
- Evident from the text: the study used a mock-up and a single domain-specific scenario (screening prescriptions), so generalization beyond this context is not established in the paper.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures







Verification
Claims the paper makes
19 Valid · 0 Not valid · 55 Not checked · 129 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“They found that communicating explanations, on average, is not improving trust calibration, i.e., users still end-up in situations where they over-trust or under-trust AI-based recommendation.”
- Also stated in: § Theoretical background and related work · plain
“Recent studies showed that even though explanations were communicated to people, trust calibration is not improved [16, 96].”
- Also stated in: § Theoretical background and related work · plain
“Overall, users of XAI systems fail, on average, to calibrate their trust, i.e., human decision-maker working collaboratively with an AI can still be notably following incorrect recommendations or rejecting correct ones.”
§ Explainable recommendation: when design meets trust calibration > Introduction
What the sources say
- Supportedcites [7] Bansal, G., Wu, T., Zhu, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M.T., Weld, D.S.: Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance. arXiv preprint arXiv:2006.14779 (2020) source
The Introduction (p. 2) reports no significant overall improvement from explanations over showing AI confidence alone, despite gains when the AI was correct and losses when it erred. Section 6.2 (p. 12) says explanations increased blind trust rather than appropriate reliance, supporting the claim that explanations did not reliably calibrate users’ trust. (LLM, conf=0.87)
- Partly supportedcites [16] Bussone, A., Stumpf, S. & O'sullivan, D.: The role of explanations on trust and reliance in clinical decision support systems, pp. 160–169. IEEE (2015) sourceExperimental — accuracy not yet measured
The Conclusion (p. 11) reports that comprehensive explanations promoted over-reliance, while selective explanations promoted self-reliance, supporting the claim that users can still over- or under-rely on recommendations. It does not report an average effect showing that communicating explanations fails to improve trust calibration; the Discussion (p. 11) instead says explanations can help users assess suggestions and that their effects require further study. (LLM, conf=0.86)
- Partly supportedcites [96] Zhang, Y., Liao, Q.V., Bellamy, R.K.E.: Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. Proceedings of the 2020 conference on fairness, accountability, and transparency. Association for Computing Machinery, Barcelona (2020) sourceExperimental — accuracy not yet measured
The Results (§4.2) report that local explanations did not improve trust calibration, and §2 describes calibration as distinguishing when to trust or distrust AI. The claim broadens this finding from the particular local explanations tested to communicating explanations generally; the paper (§5.3) allows that other explanation methods may be more effective. (LLM, conf=0.85)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Theoretical background and related work · plain
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Another research [49] showed that current explainable models are designed to be used by ML engineers for debugging purposes where end-users are not the main target of these explanations.”
§ Explainable recommendation: when design meets trust calibration > Introduction
What the sources say
- Partly supportedcites [49] Miller, T., Howe, P., Sonenberg, L.: Explainable AI: beware of inmates running the asylum or: how I learnt to stop worrying and love the social and behavioural sciences. arXiv preprint arXiv:1712.00547 (2017) sourceExperimental — accuracy not yet measured
The Abstract (p. 1) argues that most AI researchers are building explanatory agents for themselves rather than their intended users, supporting the point that end-users are often not the main target. The paper does not say that current explainable models are designed for ML engineers to use for debugging; 'explanatory debugging' appears only in the title of a paper listed in the appendix (p. 6). (LLM, conf=0.88)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“However, this introduces a lack of transparency which can weaken the users’ ability to calibrate trust in the Human-AI collaborative decision-making environments.”
- Also stated in: § Introduction · plain
“This means the human decision-maker is less able to judge when to follow a system output or reject it [33].”
§ Explainable recommendation: when design meets trust calibration > Introduction
What the sources say
- Partly supportedcites [33] Gui, C., Chan, V.: Machine learning in medicine. Unive. West. Ont. Med. J. 86, 76–78 (2017) sourceExperimental — accuracy not yet measured
The 'challenges to translation' section says that black-box ML models can make the medical community uncomfortable because their mechanisms are unknown. This supports a concern about transparency and trust, but it does not address users’ ability to calibrate trust in human–AI collaborative decision-making environments. Page information is unavailable in our copy. (LLM, conf=0.82)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Introduction · plain
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Although, many studies showed that explanation could indeed improve trust calibration [80].”
§ Explainable recommendation: when design meets trust calibration > Theoretical background and related work
What the sources say
- Partly supportedcites [80] Tomsett, R., Preece, A., Braines, D., Cerutti, F., Chakraborty, S., Srivastava, M., Pearson, G., Kaplan, L.: Rapid trust calibration through interpretable and uncertainty-aware AI. Patterns 1, 100049 (2020) sourceExperimental — accuracy not yet measured
The Results section proposes that explanations, alongside uncertainty estimates, could help people calibrate trust in AI. But the Discussion section says research on explanations’ effects remains to be done and notes that explanations can sometimes make users overly trusting. The paper does not report that many studies have shown explanations improve trust calibration. (LLM, conf=0.87)
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Recently, trust calibration became an essential metric for evaluating collaborative Human-AI decision-making tools [7, 16, 96].”
§ Explainable recommendation: when design meets trust calibration > Introduction
What the sources say
- Partly supportedcites [7] Bansal, G., Wu, T., Zhu, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M.T., Weld, D.S.: Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance. arXiv preprint arXiv:2006.14779 (2020) sourceExperimental — accuracy not yet measured
Section 6.2 (p. 12) treats appropriate reliance rather than blind trust as important to human–AI collaboration. But Section 2 (p. 3) says the paper evaluates these tools using end-to-end team accuracy; it does not establish trust calibration as an essential evaluation metric. (LLM, conf=0.82)
- Partly supportedcites [16] Bussone, A., Stumpf, S. & O'sullivan, D.: The role of explanations on trust and reliance in clinical decision support systems, pp. 160–169. IEEE (2015) sourceExperimental — accuracy not yet measured
The paper treats appropriately balanced trust and reliance as important in clinical decision support (Assessing System Reliability and Trust, p. 4; Discussion, p. 11), and measures participants’ trust and reliance in its study (Data Collection and Analysis, p. 6). It does not establish trust calibration as an essential evaluation metric for collaborative human–AI tools generally. (LLM, conf=0.86)
- Supportedcites [96] Zhang, Y., Liao, Q.V., Bellamy, R.K.E.: Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. Proceedings of the 2020 conference on fairness, accountability, and transparency. Association for Computing Machinery, Barcelona (2020) source
The Introduction (page unavailable) calls prediction-specific trust calibration especially important to the success of AI-assisted decision-making. Section 5.3 (page unavailable) says its effect should be evaluated with users in the loop and presents the study as an example of that evaluation, supporting its role as an important evaluation measure. (LLM, conf=0.88)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
89 more claims · 4 with their cited sources checked
Create a free account to see all 203 claims and every source verdict.
- Probably not a substantial claim. 82
- It may state two separate claims as one. 49
- The quote only partly states this claim. 1
Citations
Cites
(2011) · Cited by 14,317
(1999) · Cited by 7,995
An introduction to cybernetics
Ashby, W. R. (1956) · doi:10.5962/bhl.title.5851 · Cited by 7,578
Affect and Cognition Based Trust as Foundations for Interpersonal Cooperation in Organizations
McAllister, D. J. (1995) · Academy of Management Journal · doi:10.2307/256727 · Cited by 7,346 · cited 1×
“1 3 scenarios, it is highly likely that cognition-based trust components are more significant for trust calibration [41, 44].”
Background“Why Should I Trust You?”: Explaining the Predictions of Any Classifier
Ribeiro, M., Singh, S., & Guestrin, C. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations · doi:10.18653/v1/n16-3020 · Cited by 6,682
Trust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605 · cited 4×
“Previous research [40, 88] identified five primary contexts where trust calibration errors in automation occur, their reasons of occurrences and potential design solutions.”
BackgroundResultInfluentialA Simple Model of Herd Behavior
Banerjee, A. V. (1992) · The Quarterly Journal of Economics · doi:10.2307/2118364 · Cited by 6,556
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294
Designing the user interface: Strategies for effective human-computer interaction
(1993) · Applied Ergonomics · doi:10.1016/0003-6870(93)90471-k · Cited by 5,587
Explanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302 · cited 2×
“Some users prefer a detailed explanation, while others need a brief explanation based on their curiosity level [47].”
BackgroundPetty, R. E., & Cacioppo, J. T. (1986) · doi:10.1007/978-1-4612-4964-1 · Cited by 5,219
Central and Peripheral Routes to Advertising Effectiveness: The Moderating Role of Involvement
Petty, R. E., Cacioppo, J. T., & Schumann, D. (1983) · Journal of Consumer Research · doi:10.1086/208954 · Cited by 4,610
Lewis, J. D., & Weigert, A. (1985) · Social Forces · doi:10.2307/2578601 · Cited by 4,163 · cited 1×
“1 3 scenarios, it is highly likely that cognition-based trust components are more significant for trust calibration [41, 44].”
BackgroundOuellette, J. A., & Wood, W. (1998) · Psychological Bulletin · doi:10.1037/0033-2909.124.1.54 · Cited by 3,480
(1986) · Research in Organizational Behavior · Cited by 3,434
The Elaboration Likelihood Model of Persuasion
Petty, R. E., & Cacioppo, J. T. (1986) · Communication and Persuasion · doi:10.1007/978-1-4612-4964-1_1 · Cited by 3,202
(1981) · Cited by 3,038
Interventions to Break and Create Consumer Habits
Verplanken, B., & Wood, W. (2006) · Journal of Public Policy & Marketing · doi:10.1509/jppm.25.1.90 · Cited by 1,221
Counterfactual explanations without opening the black box: automated decisions and the GDPR
Wachter, S., Mittelstadt, B., & Russell, C. (2017) · SSRN Electronic Journal · doi:10.2139/ssrn.3063289 · Cited by 1,061
Zhang, Y., Liao, Q. V., & Bellamy, R. K. E. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372852 · Cited by 895 · cited 5×
“These findings provide a possible interpretation for why exposure to explanations did not improve trust calibration [16, 96].”
BackgroundInfluential(2017) · Research Methods in Human Computer Interaction · doi:10.1016/b978-0-12-805390-4.09991-x · Cited by 889
MUIR, B. M. (1994) · Ergonomics · doi:10.1080/00140139408964957 · Cited by 873
Deep Learning: A Critical Appraisal
Marcus, G. (2018) · arXiv · doi:10.48550/arxiv.1801.00631 · Cited by 853
Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., & Weld, D. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445717 · Cited by 757 · cited 2×
“Several studies have investigated the relationship between trust calibration and explanations [7, 16, 96].”
BackgroundDoes the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., & Weld, D. S. (2020) · arXiv · doi:10.48550/arxiv.2006.14779 · Cited by 754
Abdul, A., Vermeulen, J., Wang, D., Lim, B. Y., & Kankanhalli, M. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174156 · Cited by 743
Changing circumstances, disrupting habits.
Wood, W., Tam, L., & Witt, M. G. (2005) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.88.6.918 · Cited by 678
Cai, C. J., Winter, S., Steiner, D., Wilcox, L., & Terry, M. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359206 · Cited by 594 · cited 2×
“This finding relates to earlier work in Human-AI trust calibration [18], which showed that training clinicians to use the AI-based decision-making tool reduced users’ errors and increased the Human-AI collaborative decision-making performance.”
ResultAn Introduction to Cybernetics
Clark, J. H., & Ashby, W. R. (1965) · OR · doi:10.2307/3006723 · Cited by 583
23 Ways to Nudge: A Review of Technology-Mediated Nudging in Human-Computer Interaction
Caraban, A., Karapanos, E., Gonçalves, D., & Campos, P. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300733 · Cited by 475
Overtrust of robots in emergency evacuation scenarios
Robinette, P., Li, W., Allen, R., Howard, A. M., & Wagner, A. R. (2016) · 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1109/hri.2016.7451740 · Cited by 468 · cited 1×
“Such faulty design has shown critical safety issues [68].”
BackgroundThe Role of Explanations on Trust and Reliance in Clinical Decision Support Systems
Bussone, A., Stumpf, S., & O'Sullivan, D. (2015) · 2015 International Conference on Healthcare Informatics · doi:10.1109/ichi.2015.26 · Cited by 441 · cited 4×
“Several studies have investigated the relationship between trust calibration and explanations [7, 16, 96].”
BackgroundInfluentialMiller, C. A., & Parasuraman, R. (2007) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872007779598037 · Cited by 434
de Koning, B. B., Tabbers, H. K., Rikers, R. M. J. P., & Paas, F. (2009) · Educational Psychology Review · doi:10.1007/s10648-009-9098-7 · Cited by 428
Interacting with Predictions: Visual Inspection of Black-box Machine Learning Models
Krause, J., Perer, A., & Ng, K. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858529 · Cited by 390
Research Methods for Human-Computer Interaction
(2008) · Cambridge University Press eBooks · doi:10.1017/cbo9780511814570 · Cited by 379 · cited 1×
“We chose to recruit different participants to avoid the learning effect [39] and increase the credibility of our findings as existing users already learned the objective of the study and were part of the underpinnings for this next study.”
MethodA Longitudinal Study of Herd Behavior in the Adoption and Continued Use of Technology1
Sun, H. (2013) · MIS Quarterly · doi:10.25300/misq/2013/37.4.02 · Cited by 375
How Does Popularity Information Affect Choices? A Field Experiment
Tucker, C., & Zhang, J. (2011) · Management Science · doi:10.1287/mnsc.1110.1312 · Cited by 322
Tiering Drug-Drug Interaction Alerts by Severity Increases Compliance Rates
Paterno, M. D., Maviglia, S. M., Gorman, P. N., Seger, D. L., Yoshida, E., Seger, A. C., Bates, D. W., & Gandhi, T. K. (2009) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2808 · Cited by 263
Persuasive system design: state of the art and future directions
Torning, K., & Oinas-Kukkonen, H. (2009) · Proceedings of the 4th International Conference on Persuasive Technology · doi:10.1145/1541948.1541989 · Cited by 230 · cited 1×
“They were more motivated to engage with an explanation structure representing a reduction principle of the Persuasive Systems Design model [81], i.e. structures that present information in pieces and stages.”
BackgroundDriver Behavior and Situation Aware Brake Assistance for Intelligent Vehicles
McCall, J. C., & Trivedi, M. M. (2007) · Proceedings of the IEEE · doi:10.1109/jproc.2006.888388 · Cited by 229
Parayitam, S., & Dooley, R. S. (2009) · Journal of Business Research · doi:10.1016/j.jbusres.2008.02.006 · Cited by 224
The effects of example-based explanations in a machine learning interface
Cai, C. J., Jongejan, J., & Holbrook, J. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302289 · Cited by 220
Miller, T., Howe, P., & Sonenberg, L. (2017) · arXiv · doi:10.48550/arxiv.1712.00547 · Cited by 218 · cited 1×
“Another research [49] showed that current explainable models are designed to be used by ML engineers for debugging purposes where end-users are not the main target of these explanations.”
BackgroundEvaluating Effects of User Experience and System Transparency on Trust in Automation
Yang, X. J., Unhelkar, V. V., Li, K., & Shah, J. A. (2017) · Proceedings of the 2017 ACM/IEEE International Conference on Human-Robot Interaction · doi:10.1145/2909824.3020230 · Cited by 208 · cited 1×
“One goal of explainable artificial intelligence is to mitigate trust calibration errors [92].”
BackgroundRapid Trust Calibration through Interpretable and Uncertainty-Aware AI
Tomsett, R., Preece, A., Braines, D., Cerutti, F., Chakraborty, S., Srivastava, M., Pearson, G., & Kaplan, L. (2020) · Patterns · doi:10.1016/j.patter.2020.100049 · Cited by 186
Sokol, K., & Flach, P. (2020) · KI - Künstliche Intelligenz · doi:10.1007/s13218-020-00637-y · Cited by 186 · cited 2×
“Also, recent research showed that static explanations often fail to provide a satisfying explanation among all users [74].”
BackgroundResult“Order Effects” in Survey Research: Activation and Information Functions of Preceding Questions
Strack, F. (1992) · Context Effects in Social and Psychological Research · doi:10.1007/978-1-4612-2848-6_3 · Cited by 171 · cited 1×
“It studies the influence of the information order on decision-making outcome in a more or less systematic fashion [77].”
BackgroundSoll, J. B., Milkman, K. L., & Payne, J. W. (2015) · The Wiley Blackwell Handbook of Judgment and Decision Making · doi:10.1002/9781118468333.ch33 · Cited by 162 · cited 1×
“Our deriving framework to discover protentional design solutions was digital nudging [20] and also the principles of de-biasing [75].”
MethodNarayanan, M., Chen, E., He, J., Kim, B., Gershman, S., & Doshi-Velez, F. (2018) · arXiv · doi:10.48550/arxiv.1802.00682 · Cited by 153
CoNoteS2: A Software Tool for Promoting Self-Regulation
Hadwin, A. F., & Winne, P. H. (2001) · Educational Research and Evaluation · doi:10.1076/edre.7.2.313.3868 · Cited by 152
The attention-guiding effect and cognitive load in the comprehension of animations
Amadieu, F., Mariné, C., & Laimay, C. (2011) · Computers in Human Behavior · doi:10.1016/j.chb.2010.05.009 · Cited by 137
Duff, B. R. L., & Faber, R. J. (2011) · Journal of Advertising · doi:10.2753/joa0091-3367400204 · Cited by 137
Understanding Why Consumers Don't Skip Pre-Roll Video Ads
Campbell, C., Mattison Thompson, F., Grimm, P. E., & Robson, K. (2017) · Journal of Advertising · doi:10.1080/00913367.2017.1334249 · Cited by 112 · cited 1×
“These techniques have shown an effective way to increase users’ level of understanding and enable consciousness when interacting with the presented information [19, 46].”
Background(2002) · Singapore Management University Institutional Knowledge (InK) (Singapore Management University) · Cited by 108
Abstract data views: an interface specification concept to enhance design for reuse
Cowan, D. D., & Lucena, C. J. P. (1995) · IEEE Transactions on Software Engineering · doi:10.1109/32.372150 · Cited by 103
Establishing Appropriate Trust via Critical States
Huang, S. H., Bhatia, K., Abbeel, P., & Dragan, A. D. (2018) · 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8593649 · Cited by 94 · cited 1×
“Human-AI collaborative decision-making tools are usually built to support trust calibration through the recommendations interface design [10, 35, 70].”
BackgroundEffects of adaptive robot dialogue on information exchange and social relations
Torrey, C., Powers, A., Marge, M., Fussell, S. R., & Kiesler, S. (2006) · Proceedings of the 1st ACM SIGCHI/SIGART conference on Human-robot interaction · doi:10.1145/1121241.1121264 · Cited by 93
Differentiation through customer involvement in production or delivery
Song, J. H., & Adams, C. R. (1993) · Journal of Consumer Marketing · doi:10.1108/07363769310039085 · Cited by 84
(1995) · PubMed · Cited by 79
On the costs of accessible attitudes: Detecting that the attitude object has changed.
Fazio, R. H., Ledbetter, J. E., & Towles-Schwen, T. (2000) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.78.2.197 · Cited by 69
Designing for Situation Awareness and Trust in Automation
Wickens, C. D. (1995) · IFAC Proceedings Volumes · doi:10.1016/s1474-6670(17)46646-8 · Cited by 68 · cited 2×
“Previous research [40, 88] identified five primary contexts where trust calibration errors in automation occur, their reasons of occurrences and potential design solutions.”
BackgroundBuskermolen, D. O., & Terken, J. (2012) · Proceedings of the 12th Participatory Design Conference: Exploratory Papers, Workshop Descriptions, Industry Cases - Volume 2 · doi:10.1145/2348144.2348156 · Cited by 65
Modeling dual role preferences for trust-aware recommendation
Yao, W., He, J., Huang, G., & Zhang, Y. (2014) · Proceedings of the 37th international ACM SIGIR conference on Research & development in information retrieval · doi:10.1145/2600428.2609488 · Cited by 62 · cited 1×
“Machine learning (ML) models can be designed to appear as a black box where end-users cannot or are not required to understand “Why” the system makes a particular recommendation [38, 43, 93].”
BackgroundWhen people and algorithms meet: user-reported problems in intelligent everyday applications
Eiband, M., Völkel, S. T., Buschek, D., Cook, S., & Hussmann, H. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302262 · Cited by 62 · cited 1×
“Our observations support previous work which found that people working collaboratively with an AI agent are not willing to engage in what they perceived effortful behaviour with the AI explanations [27, 28].”
Result(2004) · Cited by 61
Using human knowledge awareness to adapt collaborative plan generation, explanation and monitoring
Milliez, G., Lallement, R., Fiore, M., & Alami, R. (2016) · 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1109/hri.2016.7451732 · Cited by 60 · cited 1×
“Previous research explained this result when they showed that the explanation interface’s complexity varies between humans and affects willingness to interact with the explanation [12, 50, 59].”
BackgroundA Graph-based model for context-aware recommendation using implicit feedback data
Yao, W., He, J., Huang, G., Cao, J., & Zhang, Y. (2015) · World Wide Web · doi:10.1007/s11280-014-0307-z · Cited by 54
Scheepers-Hoeks, A.-M. J., Grouls, R. J., Neef, C., Ackerman, E. W., & Korsten, E. H. (2013) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2013.05.002 · Cited by 53
Mejtoft, T., Hale, S., & Söderström, U. (2019) · Proceedings of the 31st European Conference on Cognitive Ergonomics · doi:10.1145/3335082.3335106 · Cited by 51
An experimental evaluation of the impact of data display format on recall performance
Schmell, R. W., & Umanath, N. S. (1988) · Communications of the ACM · doi:10.1145/42411.42419 · Cited by 50
Chaffee, B. W., & Zimmerman, C. R. (2010) · American Journal of Health-System Pharmacy · doi:10.2146/ajhp090153 · Cited by 44
Gui, C., & Chan, V. (2017) · University of Western Ontario Medical Journal · doi:10.5206/uwomj.v86i2.2060 · Cited by 42
Personalising Explainable Recommendations: Literature and Conceptualisation
Naiseh, M., Jiang, N., Ma, J., & Ali, R. (2020) · Advances in Intelligent Systems and Computing · doi:10.1007/978-3-030-45691-7_49 · Cited by 42 · cited 1×
“Also, our previous literature review revealed that explanation shall be adapted to users’ level of knowledge and expertise in a given Human-AI collaborative decision-making task [56].”
BackgroundSelf-citationOrder and Disorder in Everyday Action: the Roles of Contention Scheduling and Supervisory Attention
Cooper, R. (2002) · Neurocase · doi:10.1093/neucas/8.1.61 · Cited by 41 · cited 1×
“Habits reduce sensitivity to minor changes in the explanation interface, curtail explanation utilisation, and reduce assessment and reflection about the decision [23].”
BackgroundTowards Global Explanations of Convolutional Neural Networks With Concept Attribution
Wu, W., Su, Y., Chen, X., Zhao, S., King, I., Lyu, M. R., & Tai, Y.-W. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00868 · Cited by 40 · cited 1×
“Global explanation: explanations of this type provide the general logic and rules of AI reasoning [90].”
BackgroundOrder effects in belief updating with consistent and inconsistent evidence
Tubbs, R. M., Gaeth, G. J., Levin, I. P., & Van Osdol, L. A. (1993) · Journal of Behavioral Decision Making · doi:10.1002/bdm.3960060404 · Cited by 38
Probing, mocking and prototyping: participatory approaches to identity infrastructuring
Clement, A., McPhail, B., Smith, K. L., & Ferenbok, J. (2012) · Proceedings of the 12th Participatory Design Conference: Research Papers - Volume 1 · doi:10.1145/2347635.2347639 · Cited by 37
Reflecting on the invisible: understanding end-user perceptions of ubiquitous computing
Poole, E. S., Le Dantec, C. A., Eagan, J. R., & Edwards, W. K. (2008) · Proceedings of the 10th international conference on Ubiquitous computing · doi:10.1145/1409635.1409662 · Cited by 35
Causality Learning: A New Perspective for Interpretable Machine Learning
Xu, G., Duong, T. D., Li, Q., Liu, S., & Wang, X. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.16789 · Cited by 35 · cited 1×
“A significant work in XAI has focused on introducing novel explanation techniques with a focus on two primary methods [2, 91].”
BackgroundSokol, K., & Flach, P. A. (2020) · Explore Bristol Research · doi:10.48550/arxiv.2001.09734 · Cited by 33 · cited 2×
“Also, recent research showed that static explanations often fail to provide a satisfying explanation among all users [74].”
BackgroundResultA Model of the Relationship among Consumer Trust, Web Design and User Attributes
Zhang, X., Prybutok, V. R., Ryan, S., & Pavur, R. (2009) · Journal of Organizational and End User Computing · doi:10.4018/joeuc.2009040103 · Cited by 32
Primacy Effects in Clinical Judgments of Contingency
Curley, S. P., Young, M. J., Kingry, M. J., & Yates, J. F. (1988) · Medical Decision Making · doi:10.1177/0272989x8800800310 · Cited by 32
(2008) · Cited by 31
Explainable Recommendations and Calibrated Trust: Two Systematic User Errors
Naiseh, M., Cemiloglu, D., Al Thani, D., Jiang, N., & Ali, R. (2021) · Computer · doi:10.1109/mc.2021.3076131 · Cited by 30 · cited 1×
“Such failure of XAI systems in enhancing trust calibration has been linked to factors such as humans’ cognitive biases, e.g., people are selective of what they read and rely on [57].”
BackgroundSelf-citationExplainable Recommendations in Intelligent Systems: Delivery Methods, Modalities and Risks
Naiseh, M., Jiang, N., Ma, J., & Ali, R. (2020) · Lecture Notes in Business Information Processing · doi:10.1007/978-3-030-50316-1_13 · Cited by 27
Self-citationAn explanation is not an excuse: Trust calibration in an age of transparent robots
Wagner, A. R., & Robinette, P. (2021) · Trust in Human-Robot Interaction · doi:10.1016/b978-0-12-819472-0.00009-5 · Cited by 20 · cited 1×
“Also, others showed that XAI failed to improve calibrated trust because of undesired human behaviour with AI-based explanations, e.g., human laziness to engage in what they perceived as effortful behaviour [87].”
BackgroundSupporting protocol-based care in medicine via multiple coordinated views
Aigner, W., & Miksch, S. (2004) · doi:10.1109/cmv.2004.19 · Cited by 18 · cited 1×
“Cues refer to additional elements in the interface that can draw users’ attention and guide them in understanding and reading the explanation [3].”
BackgroundColor for Characters - Effects of Visual Explanations of AI on Trust and Observability
Schrills, T., & Franke, T. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-50334-5_8 · Cited by 17
A context-based approach to training situation awareness
Bass, E. J., Zenyuh, J. P., Small, R. L., & Fortin, S. T. (2002) · Proceedings Third Annual Symposium on Human Interaction with Complex Systems. HICS'96 · doi:10.1109/huics.1996.549497 · Cited by 14 · cited 1×
“Informational cues have used in the literature as a signal of information quality [32] and increase users’ ability to detect errors [8, 88].”
BackgroundLevels of abstraction in designs of human–computer interaction: The case of e-mail
Te'eni, D., & Sani-Kuperberg, Z. (2005) · Computers in Human Behavior · doi:10.1016/j.chb.2004.02.028 · Cited by 14
Garcia-Lopez, E., de-Marcos, L., Garcia-Cabot, A., & Martinez-Herraiz, J.-J. (2015) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2015.1064640 · Cited by 10
A study of dynamic information display and decision-making in abstract trust games
Schaffer, J., O’Donovan, J., Marusich, L., Yu, M., Gonzalez, C., & Höllerer, T. (2018) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2018.01.002 · Cited by 10
Customized explanation in expert system for earthquake prediction
Bofeng, Z., & Yue, L. (2005) · 17th IEEE International Conference on Tools with Artificial Intelligence (ICTAI'05) · doi:10.1109/ictai.2005.54 · Cited by 8 · cited 1×
“Previous research explained this result when they showed that the explanation interface’s complexity varies between humans and affects willingness to interact with the explanation [12, 50, 59].”
BackgroundSupporting protocol-based care in medicine via multiple coordinated views
Aigner, W., & Miksch, S. (2004) · Proceedings. Second International Conference on Coordinated and Multiple Views in Exploratory Visualization, 2004. · doi:10.1109/cmv.2004.1319532 · Cited by 6
Lazar, J. K., Feng, J. H., & Hochheiser, H. (2017) · Elsevier eBooks · doi:10.1016/b978-0-12-805390-4.00009-1 · Cited by 6
The importance of task analysis in Usability Context Analysis - designing for fitness for purpose
Mills, S. (2000) · Behaviour and Information Technology · doi:10.1080/014492900118795 · Cited by 5 · cited 1×
“Overall, the developers of explanation interfaces might require applying usability techniques like task analysis [51] to determine different contexts and scenarios where adapting explanation is required.”
Method(1995) · ACR European Advances · Cited by 3
Madsen, W. (2000) · Computer Fraud & Security · doi:10.1016/s1361-3723(00)05024-7 · Cited by 1
Order and disorder in everyday action: the roles of contention scheduling and supervisory attention
Cooper, R. (2002) · Neurocase · doi:10.1093/neucas/8.1.79 · Cited by 1
(1973) · Medical Entomology and Zoology
Research works printed but not linked: 2
- Biran, O., Cotton, C: Explanation and justification in machine learning: a survey, pp. 8–13 (2017)
- Bradley, S.: Design Fundamentals: Elements, Attributes, & Principles. Vanseo design, Colorado (2013)
Cited by
Rosenbacke, R., Melhus, Å., McKee, M., & Stuckler, D. (2024) · JMIR AI · doi:10.2196/53207 · Cited by 305
“Naiseh et al [28,29] noted that explanations affected both cognitive and affect-based trust and could result in either overtrust or undertrust.”
BackgroundMethodInfluentialNaiseh, M., Al-Thani, D., Jiang, N., & Ali, R. (2023) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2022.102941 · Cited by 170
Self-citationMehrotra, S., Degachi, C., Vereschak, O., Jonker, C. M., & Tielman, M. L. (2024) · ACM Journal on Responsible Computing · doi:10.1145/3696449 · Cited by 144
“The most common is agreement percentage , that is the percentage of trials in Another measure of appropriate trust is related to switch percentage [178], A final method is to measure ideal trusting behavior during the task beforehand and compare to which extent the”
InfluentialBeyond AI-powered context-aware services: the role of human–AI collaboration
Jiang, N., Liu, X., Liu, H., Lim, E. T. K., Tan, C., & Gu, J. (2022) · Industrial Management & Data Systems · doi:10.1108/imds-03-2022-0152 · Cited by 157
Self-citationRezaeian, O., Bayrak, A. E., & Asan, O. (2026) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2025.2539458 · Cited by 39
“Explainable Artificial Intelligence (XAI) has become essential in fields like medicine[42, 31], finance, and autonomous systems, where understanding AI decisions is critical.”
Metadata
- Authors
- Naiseh, M., Al-Thani, D., Jiang, N., & Ali, R.
- Affiliations
- Bournemouth University (GB)
- Hamad bin Khalifa University (QA)
- Corresponding author
- Naiseh, Mohammad
- Jiang, Nan
- Ali, Raian
- Venue
- World Wide Web
- Published
- 2021-08-02
- Volume
- 24
- Issue
- 5
- Pages
- 1857-1884
- ISSN
- 1386-145X (ISSN-L)
- 1573-1413
- PMC
- PMC8327305
- OpenAlex
- W3191257699
- Semantic Scholar
- e632f1191b3c261cdae48698c97eb2e9a5a45c78
- Cited by
- 68
- References
- 109
- Citations by year
- 2026: 20 · 2025: 20 · 2024: 17 · 2023: 3 · 2022: 7 · 2021: 1
- FWCI
- 2.46
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- World Wide Web (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Bournemouth University Research Online (Bournemouth University) (acceptedVersion)
- ePrints Soton (University of Southampton) (submittedVersion, cc-by)
- Aberdeen University Research Archive (Aberdeen University) (submittedVersion)
- Figshare (submittedVersion, cc-by)
- Funders
- iQ HealthTech (01101)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Keywords
- explainable recommendation, trust calibration, AI transparency, attention guidance, clinical decision support, think-aloud study, co-design
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-09-30, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-09-30 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.