CoChat
Sign Up
PreprintCited by 1

The DOI matches its DataCite record; 1 of the 1 references it cites were found in a registry or an index.

Distrust in (X)AI -- Measurement Artifact or Distinct Construct?

Scharowski, Nicolas, Perrig, Sebastian A. C.·2023·arXiv
DOI 10.48550/arxiv.2303.16495arXiv 2303.16495
Open this paper in CoChat — free account

Abstract

Trust is a key motivation in developing explainable artificial intelligence (XAI). However, researchers attempting to measure trust in AI face numerous challenges, such as different trust conceptualizations, simplified experimental tasks that may not induce uncertainty as a prerequisite for trust, and the lack of validated trust questionnaires in the context of AI. While acknowledging these issues, we have identified a further challenge that currently seems underappreciated - the potential distinction between trust as one construct and \emph{distrust} as a second construct independent of trust. While there has been long-standing academic discourse for this distinction and arguments for both the one-dimensional and two-dimensional conceptualization of trust, distrust seems relatively understudied in XAI. In this position paper, we not only highlight the theoretical arguments for distrust as a distinct construct from trust but also contextualize psychometric evidence that likewise favors a distinction between trust and distrust. It remains to be investigated whether the available psychometric evidence is sufficient for the existence of distrust or whether distrust is merely a measurement artifact. Nevertheless, the XAI community should remain receptive to considering trust and distrust for a more comprehensive understanding of these two relevant constructs in XAI.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2303.16495

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Flagged first, then by citations

Research works printed but not linked: 46

  1. Peter Andras, Lukas Esterle, Michael Guckert, The Anh Han, Peter R. Lewis, Kristina Milanovic, Terry Payne, Cedric Perret, Jeremy Pitt, Simon T. Powers, Neil Urquhart, and Simon Wells. 2018. Trusting intelligent machines: Deepening trust within socio-technical systems. IEEE Technology and Society Magazine 37, 4 (dec 2018), 76-83. https://doi.org/10.1109/MTS.2018.2876107 · DOI · Paper page
  2. John Benamati, Mark A Serva, and Mark A Fuller. 2006. Are trust and distrust distinct constructs? An empirical study of the effects of trust and distrust among online banking users. In Proceedings of the 39th Annual Hawaii International Conference on System Sciences (HICSS'06), Vol. 6. IEEE, 121b-121b. https://doi.org/10.1109/HICSS.2006.63 · DOI · Paper page
  3. Joyce Berg, John Dickhaut, and Kevin McCabe. 1995. Trust, reciprocity, and social history. Games and economic behavior 10, 1 (1995), 122-142. https://doi.org/10.1006/game.1995.1027 · DOI · Paper page
  4. Timothy A Brown. 2015. Confirmatory factor analysis for applied research (2nd ed.). The Guilford Press, New York, NY, USA. 462 pages.
  5. Zana Buçinca, Phoebe Lin, Krzysztof Z. Gajos, and Elena L. Glassman. 2020. Proxy Tasks and Subjective Measures Can Be Misleading in Evaluating Explainable AI Systems. In Proceedings of the 25th International Conference on Intelligent User Interfaces (Cagliari, Italy) (IUI '20). ACM, New York, NY, USA, 454-464. https://doi.org/10.1145/3377325.3377498 · DOI · Paper page
  6. John T Cacioppo and Gary G Berntson. 1994. Relationship Between Attitudes and Evaluative Space: A Critical Review, with Emphasis on the Separability of Positive and Negative Substrates. Psychological Bulletin 115, 3 (1994), 401-423. https://doi.org/10.1037/0033-2909.115.3.401 · DOI
  7. Yong-Sheng Chang and Shyh-Rong Fang. 2013. Antecedents and distinctions between online trust and distrust: Predicting high-and low-risk internet behaviors. Journal of Electronic Commerce Research 14, 2 (2013), 149.
  8. Robert F. DeVellis. 2017. Scale development: Theory and applications (4 ed.). SAGE publications, Inc., Thousand Oaks, CA, USA.
  9. Angelika Dimoka. 2010. What does the brain tell us about trust and distrust? Evidence from a functional neuroimaging study. MIS Quarterly 34, 2 (2010), 373-396. https://doi.org/10.2307/20721433 · DOI
  10. Mike Furr. 2011. Scale construction and psychometrics for social and personality psychology. SAGE publications, Ltd., 1 Oliver's Yard, 55 City Road, London EC1Y 1SP. https://dx.doi.org/10.4135/9781446287866 · DOI
  11. Diego Gambetta et al. 2000. Can we trust trust. Trust: Making and breaking cooperative relations 13 (2000), 213-237. http://www.sociology.ox.ac.uk/papers/gambetta213-237.pdf
  12. D Harrison McKnight and Norman Chervany. 2001. While trust is cool and collected, distrust is fiery and frenzied: A model of distrust concepts. In AMCIS 2001 Proceedings. 883-888.
  13. D Harrison McKnight and Norman L Chervany. 2001. Trust and distrust definitions: One bite at a time. In Trust in cyber-societies: Integrating the hu- man and artificial perspectives, Rino Falcone, Munindar Singh, and Yao-Hua Tang (Eds.). Springer, 27-54. https://doi.org/10.1007/3-540-45547-7_3 · DOI · Paper page
  14. Robert R Hoffman, Gary Klein, and Shane T Mueller. 2018. Explaining explanation for "explainable AI". In Proceedings of the human factors and ergonomics society annual meeting, Vol. 62. SAGE Publications Sage CA, Los Angeles, CA, USA, 197-201. https://doi.org/10.1177/1541931218621047 · DOI
  15. Alon Jacovi, Ana Marasović, Tim Miller, and Yoav Goldberg. 2021. Formalizing Trust in Artificial Intelligence: Prerequisites, Causes and Goals of Human Trust in AI. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (Virtual Event, Canada) (FAccT '21). ACM, New York, NY, USA, 624-635. https://doi.org/10.1145/3442188.3445923 · DOI · Paper page
  16. Jiun-Yin Jian, Ann M. Bisantz, and Colin G. Drury. 2000. Foundations for an Empirically Determined Scale of Trust in Automated Systems. Inter- national Journal of Cognitive Ergonomics 4, 1 (March 2000), 53-71. https://doi.org/10.1207/S15327566IJCE0401_04 · DOI · Paper page
  17. E. F. Juniper. 2009. Validated questionnaires should not be modified. European Respiratory Journal 34, 5 (2009), 1015-1017. https://doi.org/10.1183/09031936.00110209 arXiv:https://erj.ersjournals.com/content/34/5/1015.full.pdf · DOI
  18. Chester Chun Seng Kam, John P Meyer, and Shaojing Sun. 2021. Why Do People Agree With Both Regular and Reversed Items? A Logical Response Perspective. Assessment 28, 4 (2021), 1110-1124. https://doi.org/10.1177/10731911211001931 · DOI
  19. Frens Kroeger. 2019. Unlocking the treasure trove: How can Luhmann's theory of trust enrich trust research? Journal of Trust Research 9, 1 (2019), 110-124. https://doi.org/10.1080/21515581.2018.1552592 · DOI
  20. Markus Langer, Tim Hunsicker, Tina Feldkamp, Cornelius J. König, and Nina Grgić-Hlača. 2022. "Look! It's a Computer Program! It's an Algorithm! It's AI!": Does Terminology Affect Human Perceptions and Evaluations of Algorithmic Decision-Making Systems?. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI '22). Association for Computing Machinery, New York, NY, USA, Article 581, 28 pages. https://doi.org/10.1145/3491102.3517527 · DOI · Paper page
  21. Markus Langer, Daniel Oster, Timo Speith, Lena Kästner, Holger Hermanns, Eva Schmidt, Andreas Sesing, and Kevin Baum. 2021. What Do We Want From Explainable Artificial Intelligence (XAI)? A Stakeholder Perspective on XAI and a Conceptual Model Guiding Interdisciplinary XAI Research. Artificial Intelligence 296 (Feb. 2021), 103473. https://doi.org/10.1016/j.artint.2021.103473 · DOI · Paper page
  22. John D. Lee and Katrina A. See. 2004. Trust in automation: Designing for appropriate reliance. Human factors 46, 1 (March 2004), 50-80. https://doi.org/10.1518/hfes.46.1.50_30392 · DOI · Paper page
  23. Roy J Lewicki, Daniel J McAllister, and Robert J Bies. 1998. Trust and distrust: New relationships and realities. Academy of management Review 23, 3 (1998), 438-458. https://doi.org/10.2307/259288 · DOI · Paper page
  24. Roy J Lewicki, Edward C Tomlinson, and Nicole Gillespie. 2006. Models of interpersonal trust development: Theoretical approaches, empirical evidence, and future directions. Journal of management 32, 6 (2006), 991-1022. https://doi.org/10.1177/0149206306294405 · DOI
  25. J. R. Lewis and J. Sauro. 2017. Revisiting the Factor Structure of the System Usability Scale. Journal of Usability Studies 12, 4 (2017), 183-192.
  26. Zachary C. Lipton. 2018. The Mythos of Model Interpretability: In Machine Learning, the Concept of Interpretability is Both Important and Slippery. Queue 16, 3 (June 2018), 31-57. https://doi.org/10.1145/3236386.3241340 · DOI · Paper page
  27. Niklas Luhmann. 1979. Trust and Power. Wiley.
  28. Carol Xiaojuan Ou and Choon Ling Sia. 2009. To Trust or to Distrust, That is the Question: Investigating the Trust-Distrust Paradox. Commun. ACM 52, 5 (may 2009), 135-139. https://doi.org/10.1145/1506409.1506442 · DOI
  29. Sebastian A. C. Perrig, Nicolas Scharowski, and Florian Brühlmann. 2023. Trust Issues with Trust Scales: Examining the Psychometric Quality of Trust Measures in the Context of AI. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI EA '23). Association for Computing Machinery, New York, NY, USA, 7 pages. https://doi.org/10.1145/3544549.3585808 · DOI
  30. William J. Pilotte and Robert K. Gable. 1990. The Impact of Positive and Negative Item Stems on the Validity of a Computer Anxiety Scale. Educational and Psychological Measurement 50, 3 (1990), 603-610. https://doi.org/10.1177/0013164490503016 · DOI
  31. Forough Poursabzi-Sangdeh, Daniel G. Goldstein, Jake M. Hofman, Jennifer Wortman Wortman Vaughan, and Hanna Wallach. 2021. Manipulating and Measuring Model Interpretability. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI '21). ACM, New York, NY, USA, Article 237, 52 pages. https://doi.org/10.1145/3411764.3445315 · DOI · Paper page
  32. Joseph R Priester and Richard E Petty. 1996. The Gradual Threshold Model of Ambivalence: Relating the Positive and Negative Bases of Attitudes to Subjective Ambivalence. Journal of personality and social psychology 71, 3 (1996), 431-449. https://doi.org/10.1037/0022-3514.71.3.431 · DOI · Paper page
  33. Julian B Rotter. 1980. Interpersonal trust, trustworthiness, and gullibility. American psychologist 35, 1 (1980), 1-7. https://doi.org/10.1037/0003-066X.35.1.1 · DOI · Paper page
  34. Mark NK Saunders, Graham Dietz, and Adrian Thornhill. 2014. Trust and distrust: Polar opposites, or independent but co-existing? Human Relations 67, 6 (2014), 639-665. https://doi.org/10.1177/0018726713500831 · DOI
  35. Jeff Sauro and James R. Lewis. 2011. When Designing Usability Questionnaires, Does It Hurt to Be Positive?. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Vancouver, BC, Canada) (CHI '11). Association for Computing Machinery, New York, NY, USA, 2215-2224. https://doi.org/10.1145/1978942.1979266 · DOI · Paper page
  36. Nicolas Scharowski, Sebastian AC Perrig, Nick von Felten, and Florian Brühlmann. 2022. Trust and Reliance in XAI-Distinguishing Between Attitudinal and Behavioral Measures. CHI TRAIT Workshop (2022), 6 pages. https://doi.org/10.48550/arXiv.2203.12318 · DOI · Paper page
  37. Beau G Schelble, Christopher Flathmann, Matthew Scalia, Shiwen Zhou, Christopher Myers, Nathan J Mcneese, Jamie Gorman, and Guo Freeman. 2022. Addressing the Spread of Trust and Distrust in Distributed Human-AI Teaming Constellations. CHI TRAIT Workshop (2022), 11 pages.
  38. Neal Schmitt and Daniel M. Stuits. 1985. Factors Defined by Negatively Keyed Items: The Result of Careless Respondents? Applied Psychological Measurement 9, 4 (1985), 367-373. https://doi.org/10.1177/014662168500900405 · DOI
  39. F David Schoorman, Roger C Mayer, and James H Davis. 2007. An integrative model of organizational trust: Past, present, and future. Academy of Management review 32, 2 (2007), 344-354. https://doi.org/10.5465/amr.2007.24348410 · DOI · Paper page
  40. Chester A. Schriesheim and Kenneth D. Hill. 1981. Controlling Acquiescence Response Bias by Item Reversals: The Effect on Questionnaire Validity. Educational and Psychological Measurement 41, 4 (1981), 1101-1114. https://doi.org/10.1177/001316448104100420 · DOI
  41. Sim B Sitkin and Nancy L Roth. 1993. Explaining the limited effectiveness of legalistic "remedies" for trust/distrust. Organization science 4, 3 (1993), 367-392. https://doi.org/10.1287/orsc.4.3.367 · DOI · Paper page
  42. Randall D. Spain, Ernesto A. Bustamante, and James P. Bliss. 2008. Towards an empirically developed scale for system trust: Take two. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 52, 19 (Sept. 2008), 1335-1339. https://doi.org/10.1177/154193120805201907 · DOI · Paper page
  43. Javier Suárez Álvarez, Ignacio Pedrosa, Luis M Lozano, Eduardo García-Cueto, Marcelino Cuesta, and José Muñiz. 2018. Using reversed items in Likert scales: A questionable practice. Psicothema 30, 2 (2018), 149-158. https://doi.org/10.7334/psicothema2018.33 · DOI
  44. Bert Weijters and Hans Baumgartner. 2012. Misresponse to Reversed and Negated Items in Surveys: A Review. Journal of Marketing Research 49, 5 (2012), 737-747. https://doi.org/10.1509/jmr.11.0368 · DOI · Paper page
  45. Nancy Wong, Aric Rindfleisch, and James E. Burroughs. 2003. Do Reverse-Worded Items Confound Measures in Cross-Cultural Consumer Research? The Case of the Material Values Scale. Journal of Consumer Research 30, 1 (06 2003), 72-91. https://doi.org/10.1086/374697 · DOI
  46. Xijuan Zhang, Ramsha Noor, and Victoria Savalei. 2016. Examining the effect of reverse worded items on the factor structure of the need for cognition scale. PloS one 11, 6 (2016), e0157795. https://doi.org/10.1371/journal.pone.0157795 · DOI

Cited by

Sorted by citations

Metadata

Authors
Scharowski, N., & Perrig, S. A. C.
Venue
arXiv
Publisher
arXiv
Published
2023
DOI
10.48550/arxiv.2303.16495
arXiv
2303.16495
References
1
Type
Preprint
Licence
creativecommons.org/licenses/by/4.0/legalcode

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review