Trust, distrust, and appropriate reliance in (X)AI: A conceptual clarification of user trust and survey of its empirical evaluation

Probably not a substantial claim. The paper’s conclusions may not depend on it.

The lack of a multidisciplinary approach is a potential pitfall for both the development and evaluation of XAI methods.

“They note that the lack of a multidisciplinary approach is a potential pitfall for both the development and evaluation of XAI methods.”

• cites Lopes et al., 2022 sourceSupported

Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

Most XAI methods are developed in a more technically, computationally focused environment, and as a result potentially useful contributions/insights from other fields, such as HCI, are often ignored.

“(Lopes et al., 2022) point out that most XAI methods are developed in a more technically, computationally, focused environment and, as a result, potentially useful contributions/insights from other fields, such as HCI, are often ignored.”

• cites Lopes et al., 2022 sourceSupported

Probably not a substantial claim. The paper’s conclusions may not depend on it.

Distrust or suspicion led to an increase of creativity.

“Distrust or suspicion led, for example, to an increase of creativity (J.Mayer & Mussweiler, 2011) or a reduction of the correspondence bias (Fein, 1996).”

• cites J.Mayer & Mussweiler, 2011 sourcePartly supported

  The Discussion reports that distrust enhanced creativity in Study 2 under conditions of privacy. The claim omits that qualification: the Social vs. Private Creativity section explains that distrust can instead hinder creativity when ideas must be shared publicly. (LLM, conf=0.88)

  Experimental — accuracy not yet measured

It may state two separate claims as one.

Explanations can also indicate false predictions and enable distrust.

“Yet, explanations can also indicate false predictions and, thus, sometimes another utility of explanations is also identified, e.g., not trusting predictions (Ribeiro, Singh, & Guestrin, 2016), critical reflection (Ehsan & Riedl, 2020), or enabling distrust (Jacovi et al., 2021).”

• cites Ribeiro, Singh, & Guestrin, 2016 sourceNot checked — no full text of the source
• cites Ehsan & Riedl, 2020 sourceSupported
• cites Jacovi et al., 2021 sourceNot checked yet

  Checked before — its earlier verdict is held back until a new check runs.

Probably not a substantial claim. The paper’s conclusions may not depend on it.

Vilone and Longo distinguish two types of human-centric evaluation studies: qualitative and quantitative.

“(Vilone & Longo, 2020) distinguish two types of human-centric evaluation studies: qualitative and quantitative.”

• cites Vilone & Longo, 2020 sourceSupported

Probably not a substantial claim. The paper’s conclusions may not depend on it.

Research into XAI systems covers a wide variety of application domains, which is supported by the paper's survey.

“Research into XAI systems cover a wide variety of application domains (Adadi & Berrada, 2018;Ferreira & Monteiro, 2020;Lai et al., 2021), which is supported by our survey.”

• cites Adadi & Berrada, 2018 sourceNot checked — no full text of the source
• cites Ferreira & Monteiro, 2020 sourceNot checked — no full text of the source
• cites Lai et al., 2021 sourceSupported

Probably not a substantial claim. The paper’s conclusions may not depend on it.

Common terms that relate to users and are frequently employed to analyze and evaluate trust include user knowledge, technical proficiency, familiarity, confidence, beliefs, faith, emotions, and personal attachments.

“Common terms that relate to users that are frequently employed to analyze and evaluate trust, include user knowledge, technical proficiency, familiarity, confidence, beliefs, faith, emotions, and personal attachments (Mohseni et al., 2021).”

• cites Mohseni et al., 2021 sourceSupported

Probably not a substantial claim. The paper’s conclusions may not depend on it.

Kohn et al. (2021) describe 16 different types of self-report methods used in trust in automation.

“Kohn et al. (2021) describe 16 different types of self-report methods used in trust in automation.”

• cites Kohn et al. (2021) sourceSupported

Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

Kohn et al. (2021) review different measurements of trust in automation and describe three categories: self-report, physiological, and behavioral.

“In Kohn et al. (2021) the authors give a review of different measurements of trust in automation.They describe three categories in which these methods can be placed: self-report, physiological, and behavioral.”

• cites Kohn et al. (2021) sourceSupported

It may state two separate claims as one.

Questionnaires measure participants' perception of their trust, which does not always correspond to their actions.

“Such questionnaires measure the participants' perception of their trust (T.Miller, 2022) that do not always correspond to their actions.”

• cites T.Miller, 2022 sourceSupported

It may state two separate claims as one.

Rusk (2018) introduces a scale that measures trust and distrust separately and provides a first validation.

“In his dissertation Rusk (2018) sets out to close a research gap and introduces a scale that measures trust and distrust separately and provides a first validation.”

• cites Rusk (2018)Supported

Probably not a substantial claim. The paper’s conclusions may not depend on it.

A lack of realism can hinder the validity and generalizability of study results.

“While a lack of realism can hinder the validity and generalizability of study results (Lai et al., 2021).”

• cites Lai et al., 2021 sourceSupported

It may state two separate claims as one.

The Wizard of Oz setup allows researchers full control over the outputs that the user is seeing and allows them to specifically tune the actual trustworthiness of the system to determine to what extent this affects the users.

“The benefit of this is that it allows the researchers full control over the outputs that the user is seeing and allows them to specifically tune the (actual) trustworthiness of the system to determine to what extend this affects the users (Lai et al., 2021).”

• cites Lai et al., 2021 sourceSupported

Probably not a substantial claim. It may state two separate claims as one.

Scharowski and Perrig (2023) clarify that trust as an attitude should be measured via questionnaires whereas behavioural measures are suitable to assess reliance.

“With respect to the use of self-report and behavioral measures, Scharowski and Perrig (2023) clarify that trust as an attitude should be measured via questionnaires whereas behavioural measures are suitable to assess reliance.”

• cites Scharowski and Perrig (2023) sourceSupported

Probably not a substantial claim. The paper’s conclusions may not depend on it.

Custom self-report scales are also observed in the evaluation of XAI methods and in the broad context of trust in information systems.

“A procedure that is also observed in the evaluation of XAI methods (Lopes et al., 2022) or in the broad context of trust in information systems (Rusk, 2018).”

• cites Lopes et al., 2022 sourcePartly supported

  Section 4.1 (Trust) says Likert-scale questionnaires are widely used to assess trust in XAI systems, and Table 3 lists a five-point questionnaire for users’ perceptions of trust. The copy does not establish that these scales were custom-made. The separate claim about trust in information systems is attached to Rusk (2018), not Lopes et al. (LLM, conf=0.78)

  Experimental — accuracy not yet measured
• cites Rusk, 2018Supported

Probably not a substantial claim. The paper’s conclusions may not depend on it.

In the paper's survey, the user studies focus primarily on direct users of AI systems.

“While in our survey the user studies focus primarily on direct users of AI system”

Probably not a substantial claim. The paper’s conclusions may not depend on it.

The surveyed papers are concerned with various domains, ranging from higher risk ones (e.g. medical diagnosis, self-driving cars, finance, or law) to low-medium risk (e.g. entertainment or social media) applications.

“The surveyed papers are concerned with various domains, ranging from higher risk ones (e.g.medical diagnosis, self-driving cars, finance, or law, e.g.recidivism prediction) to low-medium risk (e.g.entertainment or social media related such as movie recommendation) applications.”

Probably not a substantial claim. The paper’s conclusions may not depend on it.

In the paper's survey, generic, mixed participants are often recruited via crowdsource platforms such as Amazon Mechanical Turk or Prolific.

“Often generic, mixed, participants are recruited via crowdsource platforms, such as Amazon Mechanical Turk or Prolific.”

Probably not a substantial claim. The paper’s conclusions may not depend on it. The quote only partly states this claim.

Different types of applications are used in the user studies, such as decision support systems, recommender systems, image classification, text annotation, or speech recognition.

“Different types of applications are used in the user studies, such as decision support systems”

Probably not a substantial claim. The paper’s conclusions may not depend on it.

In the paper's survey, some authors select specifically one user type such as non-experts or domain experts.

“Other authors select specifically one user type such as non-experts”

Probably not a substantial claim. The paper’s conclusions may not depend on it.

In most studies, a variety of demographic information about the user is collected, even when the authors do not target any group specifically.

“In most studies a variety of demographic information about the user is collected (e.g.education level (Omeiza et al., 2021)), even when the authors do not target any group specifically.”

Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.

Most researchers study the effects of different models whether they are proxy or mock models or actual underlying models.

“Most researchers study the effects of different models whether they are proxy or mock models (i.e.Wizard of Oz setups) or actual underlying models.”

It may state two separate claims as one.

In the paper's survey, textual interfaces, such as showing a worded explanation of the output, are the most frequently used interface modality (45 papers), followed by visual or graphical descriptions (28 papers).

“In our survey we find different interface modalities being used, where textual interfaces, such as showing a worded explanation of the output, are most frequently used (45 papers), followed by visual or graphical descriptions (28 papers), such as image based methods”

Probably not a substantial claim. The paper’s conclusions may not depend on it.

Some papers perform visual highlighting of different elements of the interface, such as highlighting the words with the highest feature importances.

“Some papers perform visual highlighting of different elements of the interface (Papenmeier et al., 2022(Papenmeier et al., , 2021;;Bansal et al., 2021;Linder et al., 2021;Schmidt & Biessmann, 2019;Wang & Yin, 2023;Chen et al., 2023;Guesmi et al., 2023), such as highlighting the words with the highest feature importances, to make them stand out more compared to the other parts of the interface and”

It may state two separate claims as one.

Many types of local explanations are used, namely example based, contrastive, (pixel) attribution, counterfactual or 'What if', nearest neighbor explanations, or describing the model's reasoning or logical rationale for the output.

“Many types of local explanations are used, namely example based (F.Yang et al., 2020;Chen et al., 2023;Alam & Mueller, 2021;Leichtmann et al., 2023;Kulesza et al., 2013), contrastive (Omeiza et al., 2021), (pixel) attribution (Nourani et al., 2019;Mahsan Nourani, 2020;Leichtmann et al., 2023;Bussone et al., 2015), counterfactual or 'What if' (Guesmi et al., 2023;Wang & Yin, 2022), nearest neighbor”

Probably not a substantial claim. The paper’s conclusions may not depend on it.

Users are often provided with some form of model performance metrics, such as accuracy, confusion matrices, or class-based error rates.

“User are also often provided with some form of model performance metrics, such as the accuracy (Papenmeier et al., 2022(Papenmeier et al., , 2021;;Ming Yin, 2019;Lim et al., 2009;Rechkemmer & Yin, 2022;Lu & Yin, 2021;Thaler & Schmid, 2021;Suresh et al., 2020;Drozdal et al., 2020), confusion matrices (Drozdal et al., 2020), or class-based error rates (Thaler & Schmid, 2021).”

Probably not a substantial claim. The paper’s conclusions may not depend on it.

The papers that state a specific ML method make use of a range of different models of varying complexity and interpretability, from decision trees, SVMs, logistic regression, and Naive Bayes to Random Forest, CNN, language, time series, and Reinforcement Learning models.

“The papers that do state a specific ML method make use of a range of different models of varying complexity and interpretability from decision trees”

It may state two separate claims as one.

Mostly there is a (moderately) positive effect on user trust from the application of explanations, while a clear negative effect is not as common especially for behavioral trust measures.

“From this we can see that mostly there is a (moderately) positive effect on user trust from the application of explanations, while a clear negative effect is not as common especially for behavioral trust measures.”

It may state two separate claims as one.

The most common approach is to use models of varying accuracy and to observe whether this has an effect on the user or whether users can accurately detect the change and adjust their perceived trustworthiness of the model and reliance accordingly.

“The most common approach is to use models of varying accuracy (Yu et al., 2018(Yu et al., , 2017(Yu et al., , 2019;;Bansal et al., 2021;Donald R. Honeycutt, 2020;Ming Yin, 2019;Zhou, Yu, & Chen, 2018;Zhou et al., 2019;Nourani et al., 2019) and to observe whether this has an effect on the user or whether or not users can accurately detect the change and adjust their perceived trustworthiness of the”

Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

It is common that a mock model is used and not an actual underlying ML model, so the specific local explanation method is not always described in detail but rather users receive some sort of local explanation of which a high-level description is given.

“As we detail later in this section, it is common that a mock model is used and not an actual underlying ML model, therefore, in these cases, the specific local explanation method is not always described in detail but rather that the users receive some sort of local explanation of which a high-level description is given.”

Probably not a substantial claim. The paper’s conclusions may not depend on it.

Some miscellaneous evaluation criteria are classified in the 'Other' category.

“Some miscellaneous criteria we classify in the 'Other' category.”

Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.

In most of the papers, the actual underlying model that is used is not clearly specified.

“In most of the papers the actual underlying model that is used is not clearly specified.”

Probably not a substantial claim. The paper’s conclusions may not depend on it.

Many studies make use of feature based display methods, such as input features and output explanation, or global feature importance to describe the model.

“Many studies make use of feature based display methods (Kulesza et al., 2013;Bussone et al., 2015;Zhang et al., 2020;Guesmi et al., 2021;Jiang et al., 2022;Cheng et al., 2019;Wang & Yin, 2022;Ming Yin, 2019;Linder et al., 2021;Lim et al., 2009;Wanner et al., 2020;Buçinca et al., 2021;Zhou et al., 2019;Drozdal et al., 2020;Lu & Yin, 2021), such as input features (e.g.showing the values of the input”

Probably not a substantial claim. The paper’s conclusions may not depend on it.

Some less frequently used descriptors are providing users with introductory information about the system and underlying model, such as model architecture, which algorithm was used, system limitations, what the explanatory process is and how it works, or information about training data, data distributions, and process of feature engineering.

“Some less frequently used descriptors are providing the users with (introductory) information about the system and underlying model (Körber et al., 2018;Chen et al., 2023), such as the model architecture (Suresh et al., 2020), which algorithm was used, system limitations (Körber et al., 2018), what the explanatory process is and how it works (Zhao et al., 2019), or information about the training”

Probably not a substantial claim. The paper’s conclusions may not depend on it.

In the paper's survey, the most common strategy to test the impact of different descriptors is to either show or not show a specific descriptor, usually applied for various local explanations and uncertainty estimates.

“The most common strategy is to either show or not show a specific descriptor, which is usually applied for various local explanations and uncertainty estimates.”

Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.

Another strategy is to supply the user with explanations of various levels of detail or various combinations of different explanations.

“Another strategy is to supply the user with explanations of various levels of detail (e.g.only showing an uncertainty estimate, combining it with a pixel attribution) or various combinations of different explanations (e.g.combining uncertainty estimates with an example based explanations).”

Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.

The modality of the explanation and the interface can also vary or be combined.

“The modality of the explanation and the interface can also vary or be combined (e.g.showing a text based explanation along with an input image).”

Probably not a substantial claim. It may be too vague to check against evidence.

When comparing descriptors, modulation strategies, and (XAI) methods used with user questions and needs, types of explanations, descriptors, and XAI methods described in overviews, there is still a number of areas and methods that have been either not investigated yet or for which results are currently inconclusive.

“When comparing the descriptors, modulation strategies, and (XAI) methods used with those user questions and needs, types of explanations, descriptors, and XAI methods that are described in the various overviews we find that there is still a number of areas and methods that have been either not investigated yet or those for which results are currently inconclusive.”

Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.

Many different setups for various application domains, types, and users remain un(der)investigated for many of the potential antecedent and confounding factors of trust.

“Many different setups for various application domains, types, and users remain un(der)investigated for many of the potential antecedent and confounding factors of trust.”

Probably not a substantial claim. It may state two separate claims as one.

There is growing interest in user trust in (X)AI and the development of methods that can foster appropriate reliance in trustworthy AI methods.

“We have shown that there is growing interest in user trust in (X)AI and the development of methods that can foster appropriate reliance in trustworthy AI methods.”

Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.

Beneath the desideratum of increased appropriate trust lies the desideratum of increased appropriate reliance.

“So, beneath the desideratum of increased appropriate trust lies the desideratum of increased appropriate reliance.”

Probably not a substantial claim. It may state two separate claims as one.

In the two-dimensional approach, low trust is not the same as high distrust, and low distrust is not equal to high trust.

“In the two-dimensional approach, by definition, low trust is not the same as high distrust, and low distrust is not equal to high trust (Lewicki et al., 1998).”

• cites Lewicki et al., 1998 sourceNot checked — no full text of the source

It may state two separate claims as one.

Different users have different objectives and levels of expertise and might be impacted differently by different types of applications.

“Different users have different objectives and levels of expertise and might, therefore, be impacted differently by different types of applications (Liao & Varshney, 2021).”

• cites Liao & Varshney, 2021Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it.

Trust triggers a perception focus on similarities that makes it harder to remember single entities.

“According to Posten and Gino (2021), trust triggers a perception focus on similarities that makes it harder to remember single entities.”

• cites Posten and Gino (2021) sourceNot checked — no full text of the source

Probably not a substantial claim. The paper’s conclusions may not depend on it.

The distrust mindset leads to an activation of incongruent and alternative associations.

“The distrust mindset leads to an activation of incongruent and alternative associations, which aligns well with the increase of creativity, reflection, and innovation.”

• cites Mayo, 2015 sourceNot checked — no full text of the source

It may state two separate claims as one.

Users may perceive a system's trustworthiness incorrectly and may, therefore, undertrust or disuse trustworthy systems and overtrust untrustworthy ones.

“To further complicate the matter of trustworthiness, users may perceive a system's trustworthiness incorrectly and may, therefore, undertrust, or disuse, trustworthy systems and overtrust untrustworthy ones (Schlicker & Langer, 2021).”

• cites Schlicker & Langer, 2021 sourceNot checked — no full text of the source

Probably not a substantial claim. The paper’s conclusions may not depend on it.

Zhou et al. distinguish between subjective and objective metrics for human-centric evaluation of XAI.

“Similarly, (Zhou et al., 2021) distinguish between subjective and objective metrics for human-centric evaluation of XAI.”

• cites Zhou et al., 2021Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

It may state two separate claims as one.

Trustworthiness influences trust but does not fully determine it.

“Overall, trustworthiness influences trust but does not fully determine it.”

• Also stated in: § 2 Trust in AI · plain

  “A high level of these three factors of trustworthiness does not necessarily lead to trust, and trust can also go along with little trustworthiness (R. C. Mayer et al., 1995a).”

• cites R. C. Mayer et al., 1995a sourceNot checked — no full text of the source

Probably not a substantial claim. The paper’s conclusions may not depend on it.

Distrust or suspicion led to a reduction of the correspondence bias.

“Distrust or suspicion led, for example, to an increase of creativity (J.Mayer & Mussweiler, 2011) or a reduction of the correspondence bias (Fein, 1996).”

• cites Fein, 1996 sourceNot checked — no full text of the source

Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

XAI systems allow users to assess how reliable the system is and, therefore, to correctly calibrate their perception of the system's accuracy.

“XAI systems allow users to assess how reliable the system is and, therefore, to correctly calibrate their perception of the system's accuracy (Mohseni et al., 2021) or, in other words, to calibrate their perceived trustworthiness of the system.”

• cites Mohseni et al., 2021 sourceNot checked yet

It may state two separate claims as one.

Trust does not lead to appropriate reliance because trust is not only influenced by the correctness, i.e. the performance, of an AI.

“Trust does not lead to this because trust is not only influenced by the correctness, i.e. the performance of an AI.”

Probably not a substantial claim.

In order to properly compare results of different studies and experiments, a user evaluation must be both reliable and valid.

“In order to be able to properly compare the results of different studies and experiments it is necessary that a user evaluation is both reliable and valid (Joppe, 2000).”

• cites Joppe, 2000Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

Distrust shifts the perception focus towards differences and increases memory performance.

“Distrust shifts the perception focus towards differences and, therefore, increases memory performance.”

• cites Posten and Gino (2021) sourceNot checked — no full text of the source

Probably not a substantial claim. It may be too vague to check against evidence.

Explainability is an inherently human-centric property.

“As explainability is an inherently human-centric property (Lopes, Silva, Braga, Oliveira, & Rosado, 2022), (Mohseni et al., 2021) and (Liao & Varshney, 2021) believe that HCI and human-centric evaluation can contribute to solving XAI algorithms and applications' limitations.”

• cites Lopes, Silva, Braga, Oliveira, & Rosado, 2022 sourceNot checked yet

Probably not a substantial claim. The paper’s conclusions may not depend on it.

Many ways exist to group and identify different user types and stakeholders in the context of XAI.

“Already many ways exist to group and identify different user types and stakeholder in the context of XAI (Mohseni et al., 2021;Liao & Varshney, 2021;Meske et al., 2020).”

• cites Mohseni et al., 2021 sourceNot checked yet
• cites Liao & Varshney, 2021Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Meske et al., 2020 sourceNot checked yet

  Checked before — its earlier verdict is held back until a new check runs.

Probably not a substantial claim. The paper’s conclusions may not depend on it.

Many different models to classify and categorize the factors that influence trust exist.

“Many different models to classify and categorize the factors that influence trust (i.e. its antecedents) exist (Toreini et al., 2020;Siau & Wang, 2018;Hoff & Bashir, 2015;Schaefer, Chen, Szalma, & Hancock, 2016;Riegelsberger, Sasse, & McCarthy, 2005;Kee & Knox, 1970;Kaplan et al., 2023;R. Yang & Wibowo, 2022;R. C. Mayer, Davis, & Schoorman, 1995b;Dietz & Hartog, 2006).”

• cites Toreini et al., 2020Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Siau & Wang, 2018Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Hoff & Bashir, 2015 sourceNot checked — no full text of the source

It may state two separate claims as one.

The dynamics between explainability and trust are not well established, and empirical investigations of their relation remain mixed or inconclusive.

“However, the dynamics between explainability and trust are not well established and empirical investigations of their relation remain mixed or inconclusive.”

• Also stated in: § 1 Introduction · plain

  “Problematically, the dynamics between explainability and trust are far from being clarified, with both terms lacking precise definitions (Ferrario & Loi, 2022) and results from empirical investigations of their relation remaining mixed and inconclusive (Kastner et al., 2021).”

• cites Kastner et al., 2021 sourceNot checked yet

  Checked before — its earlier verdict is held back until a new check runs.

• cites Ferrario & Loi, 2022 sourceNot checked — no full text of the source

It may state two separate claims as one.

A lower willingness to rely, a lower trust, would decrease the likelihood of overtrust but would also lead to less reliance overall.

“While it may be true that a lower willingness to rely, a lower trust, would decrease the likelihood of overtrust, it would also lead to less reliance overall.”

It may state two separate claims as one.

An exclusive focus on trust, rather than healthy scepticism and distrust, is potentially dangerous, as a mismatch between actual and perceived trustworthiness can result in overtrust on the system.

“This makes an exclusive focus on trust, rather than healthy scepticism and distrust, potentially dangerous, as a mismatch between actual and perceived trustworthiness can result in overtrust on the system.”

It may state two separate claims as one.

Distrust also has positive consequences and merits separate from those of trust.

“In sum, although often connotated negatively, distrust also has positive consequences and merits separate from those of trust.”

Probably not a substantial claim. It may be too vague to check against evidence. It may state two separate claims as one.

Trust is characterized by hope, faith, or assurance, and distrust by scepticism, fear, suspicion or vigilance.

“Among others, trust is characterized by hope, faith, or assurance, and distrust by scepticism, fear, suspicion or vigilance (Benamati et al., 2006;Cho, 2006;Lewicki et al., 1998).”

• cites Benamati et al., 2006 sourceNot checked — no full text of the source
• cites Cho, 2006 sourceNot checked — no full text of the source
• cites Lewicki et al., 1998 sourceNot checked — no full text of the source

It may state two separate claims as one.

Trust is dynamic and can increase or decrease during interactions.

“Trust is dynamic and can increase or decrease during interactions (Y.Guo & Yang, 2021;Glikson & Woolley, 2020;R. R. Hoffman et al., 2009).”

• cites Y.Guo & Yang, 2021 sourceNot checked yet

  Checked before — its earlier verdict is held back until a new check runs.

• cites Glikson & Woolley, 2020 sourceNot checked — no full text of the source
• cites R. R. Hoffman et al., 2009 sourceNot checked — no full text of the source

Probably not a substantial claim. The paper’s conclusions may not depend on it.

HCI and human-centric evaluation can contribute to solving XAI algorithms and applications' limitations.

“As explainability is an inherently human-centric property (Lopes, Silva, Braga, Oliveira, & Rosado, 2022), (Mohseni et al., 2021) and (Liao & Varshney, 2021) believe that HCI and human-centric evaluation can contribute to solving XAI algorithms and applications' limitations.”

• cites Mohseni et al., 2021 sourceNot checked yet
• cites Liao & Varshney, 2021Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

It may state two separate claims as one.

Of the factors that have been studied, there is a relatively robust picture of which individual antecedents are effective at predicting trust and which are not, but there is a lack of empirical research for many other factors.

“Of the ones that have been studied there is a relatively robust picture of which individual antecedents are effective at predicting trust and which are not, however there is a lack of empirical research for many of the other factors (Kaplan et al., 2023).”

• cites Kaplan et al., 2023 sourceNot checked — no full text of the source

It may state two separate claims as one.

Reverse scoring distrust items and summing them with trust items entails a problematic entanglement of the two factors identified by Spain et al. (2008) and disregards the incremental insight by measuring trust and distrust individually.

“Reverse scoring distrust items to then sum with the trust items entails a problematic entanglement of the two factors identified by Spain et al. (2008) and disregards the incremental insight by measuring trust and distrust individually.”

• cites Spain et al. (2008) sourceNot checked — no full text of the source

Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

The Rusk scale is developed for the context of information systems and might be applicable to the (X)AI context as well.

“The scale is developed for the context of information systems and might be applicable to the (X)AI context as well.”

Probably not a substantial claim. The paper’s conclusions may not depend on it.

In this paper, the factors influencing trust are categorized as related to the human, the AI, and the context.

“In this paper we use a categorization of the factors influencing trust similar to the one used by Kaplan et al. (2023).These factors are related to the human, the AI, and the context (Kaplan et al., 2023).”

• cites Kaplan et al., 2023 sourceNot checked — no full text of the source

Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

Kohn et al. (2021) highlight that researchers either state that they developed their own way of measuring trust, or they do not cite any source for their method.

“Kohn et al. (2021) highlight that researchers either state that they developed their own way of measuring trust, or they do not cite any source for their method.”

• cites Kohn et al. (2021) sourceNot checked yet

  Checked before — its earlier verdict is held back until a new check runs.

Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.

Single item measures are usually less reliable than multiple items and narrow down the complex concept of trust to a single (dis)agreement.

“Despite the advantage in efficiency of single item measures, they are usually less reliable than multiple items and narrow down the complex concept of trust to a single (dis)agreement.”

Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

In one study, the approach of distinguishing appropriate trust, undertrust and overtrust allowed observation that feature importance and feature contribution slightly increased appropriate trust, decreased undertrust, and that for feature importance this came at the cost of a slight increase in overtrust.

“For instance in one study, this approach allowed to observe that feature importance and feature contribution slightly increased appropriate trust, decreased their undertrust, and that, for feature importance, this came at the cost of a slight increase in overtrust (Wang & Yin, 2022).”

• cites Wang & Yin, 2022 sourceNot checked yet

  Checked before — its earlier verdict is held back until a new check runs.

Probably not a substantial claim.

The Checklist for Trust between People and Automation measures trust and distrust as polar opposites along a single dimension.

“This checklist measures trust and distrust as polar opposites along a single dimension.”

It may state two separate claims as one.

Demographic factors such as gender, age, or culture and cultural aspects such as individualism and power relations within a culture can impact a person's propensity to trust an AI system.

“Additionally, demographics factors such as gender, age, or culture (Hoff & Bashir, 2015) and various cultural aspects such as individualism and power relations within a culture can impact a person's propensity to trust an AI system (Chien, Lewis, Hergeth, Semnani-Azad, & Sycara, 2015;Chien, Lewis, & Sycara, 2016).”

• cites Hoff & Bashir, 2015 sourceNot checked — no full text of the source
• cites Chien, Lewis, Hergeth, Semnani-Azad, & Sycara, 2015 sourceNot checked — no full text of the source
• cites Chien, Lewis, & Sycara, 2016 sourceNot checked — no full text of the source

Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

Novice (non-expert) users suffer from over-reliance due to their lack of knowledge, which results in their inability to properly detect errors.

“Furthermore, (Mahsan Nourani, 2020) observe that novice (non-expert) users suffer from over-reliance due to their lack of knowledge which results in their inability to properly detect errors.”

• cites Mahsan Nourani, 2020Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. It may be too vague to check against evidence.

Consistent and standardized items would allow researchers to develop a better understanding of trust and distrust.

“Consistent and standardized items would allow researchers to develop a better understanding of trust and distrust (Rusk, 2018).”

• cites Rusk, 2018Not checked yet

  Checked before — its earlier verdict is held back until a new check runs.

Probably not a substantial claim. The paper’s conclusions may not depend on it.

Along with the explanations' content, the presentation of this information also has an impact on the user.

“Along with the explanations' content (i.e.used descriptors) the presentation of this information also has an impact on the user (Hoff & Bashir, 2015).”

• cites Hoff & Bashir, 2015 sourceNot checked — no full text of the source

Probably not a substantial claim. The paper’s conclusions may not depend on it.

Kohn et al. (2021) note that notwithstanding evidence for the two-dimensional conceptualisation, uni-dimensional scales are the common form for assessing trust in automation.

“Kohn et al. (2021) note, however, that notwithstanding the evidence for the two-dimensional conceptualisation, uni-dimensional scales are the common form for assessing trust in automation (Kohn et al., 2021).”

• cites Kohn et al. (2021) sourceNot checked yet

  Checked before — its earlier verdict is held back until a new check runs.

Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

The first models on trust stem from social sciences, including organisational science and psychology, and are concerned with interpersonal trust.

“The first of these models on trust (Kee & Knox, 1970;R. C. Mayer et al., 1995b;Dietz & Hartog, 2006) stem from social sciences, including organisational science and psychology, and are concerned with interpersonal trust.”

• cites Kee & Knox, 1970 sourceNot checked — no full text of the source
• cites R. C. Mayer et al., 1995b sourceNot checked — no full text of the source
• cites Dietz & Hartog, 2006 sourceNot checked — no full text of the source

It may state two separate claims as one.

A person's personality (disposition to trust) and ability are of concern for trust.

“A person's personality (disposition to trust) (Siau & Wang, 2018;Hoff & Bashir, 2015) and ability are of concern (Siau & Wang, 2018).”

• cites Siau & Wang, 2018Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Hoff & Bashir, 2015 sourceNot checked — no full text of the source

Probably not a substantial claim. The paper’s conclusions may not depend on it.

Given the complexity of model behavior, it can be challenging to design realistic studies using the Wizard of Oz method.

“Additionally, given the complexity of model behavior, it can be challenging to design realistic studies using this method.”

Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

A study on website design showed that trust and distrust are affected by different antecedents, and the performance of a trust-aware recommender system was improved by predicting not only trust but also distrust.

“A study on website design showed that trust and distrust are affected by different antecedents, and the performance of a trust-aware recommender system was improved by predicting not only trust but also distrust (Fang et al., 2015).”

• cites Fang et al., 2015 sourceNot checked — no full text of the source

Probably not a substantial claim. The paper’s conclusions may not depend on it.

A difference between dispositional trust and dispositional distrust was observed in the context of online expert advice.

“A difference between dispositional trust and dispositional distrust was observed in the context of online expert advice (McKnight et al., 2004)”

• cites McKnight et al., 2004 sourceNot checked — no full text of the source

Probably not a substantial claim. The paper’s conclusions may not depend on it.

The interactive effects between the antecedents of trust have been largely unexplored.

“Additionally, the interactive effects between the antecedents of trust have been largely unexplored (Kaplan et al., 2023).”

• cites Kaplan et al., 2023 sourceNot checked — no full text of the source

Probably not a substantial claim.

In a critical validation attempt by Spain et al. (2008), a one-factor model and a two-factor model were compared.

“In a critical validation attempt of this scale by Spain et al. (2008), a one-factor model (indicating the polar opposites along a single dimension) and a two-factor model were compared.”

• cites Spain et al. (2008) sourceNot checked — no full text of the source

It may state two separate claims as one.

While domain expertise does impact user trust, domain expertise alone cannot fully predict people's trust in a model because expectations and personal experiences play a significant role.

“However, while domain expertise does impact user trust (Ooge & Verbert, 2021) observe that as expectations and personal experiences play a significant role, domain expertise alone cannot fully predict people's trust in a model.”

• cites Ooge & Verbert, 2021 sourceNot checked yet

  Checked before — its earlier verdict is held back until a new check runs.

Probably not a substantial claim. The paper’s conclusions may not depend on it.

Domain experts are capable of dynamically adjusting the perceived trustworthiness of an AI model by using its explanations.

“Domain experts are capable of dynamically adjusting the perceived trustworthiness of an AI model by using its explanations (Mahsan Nourani, 2020).”

• cites Mahsan Nourani, 2020Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.

There is an important difference between studying whether a XAI method affects the user's trust in a single interaction and whether it affects the general attitude in AI.

“There is an important difference between studying whether a XAI method affects the user's trust in a single interaction and whether it affects the general attitude in AI, which should not be mixed.”

Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.

Depending on the risk, unwarranted trust or unwarranted distrust can be problematic in different ways.

“Depending on the risk, unwarranted trust or unwarranted distrust can be problematic in different ways.”

Probably not a substantial claim.

Because explanations help identify both correct and wrong outputs, XAI methods may affect both the user's trust and distrust.

“Because explanations help identify both correct and wrong outputs, XAI methods may affect both the user's trust and distrust.”

• AssumptionRests on an assumption: Explanations help identify both correct and wrong outputs.

Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.

The application domain and the entailed risk may inform which reliance problem (disuse or overtrust) is more important.

“The application domain and the entailed risk may inform which reliance problem (disuse or overtrust) is more important.”