CoChat
Sign Up
PreprintCited by 4

The DOI matches its DataCite record.

Are we measuring trust correctly in explainability, interpretability, and transparency research?

Miller, Tim·2022·arXiv
DOI 10.48550/arxiv.2209.00651arXiv 2209.00651
Open this paper in CoChat — free account

Abstract

This paper presents an argument for why we are not measuring trust sufficiently in explainability, interpretability, and transparency research. Most studies ask participants to complete a trust scale to rate their trust of a model that has been explained/interpreted. If the trust is increased, we consider this a positive. However, there are two issues with this. First, we usually have no way of knowing whether participants should trust the model. Trust should surely decrease if a model is of poor quality. Second, these scales measure perceived trust rather than demonstrated trust. This paper showcases three methods that do a good job at measuring perceived and demonstrated trust. It is intended to be starting point for discussion on this topic, rather than to be the final say. The author invites critique and discussion.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2209.00651

Cites

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

No references could be resolved yet.

Research works printed but not linked: 26

  1. Ofra Amir, David G Rand, and Ya'akov Kobi Gal. 2012. Economic games on the internet: the effect of $1 stakes. PLoS One 7, 2 (Feb. 2012), e31461. https://doi.org/10.1371/journal.pone.0031461 · DOI · Paper page
  2. Joyce Berg, John Dickhaut, and Kevin McCabe. 1995. Trust, reciprocity, and social history. Games and economic behavior 10, 1 (1995), 122-142.
  3. Christiane Bradler, Susanne Neckermann, and Arne Jonas Warnke. 2019. Incentivizing Creativity: A Large-Scale Experiment with Performance Bonuses and Gifts. J. Labor Econ. 37, 3 (July 2019), 793-851. https://doi.org/10.1086/702649 · DOI
  4. Béatrice Cahour and Jean-François Forzy. 2009. Does projection into use improve trust and exploration? An example with a cruise control system. Safety science 47, 9 (2009), 1260-1270.
  5. Colin Camerer and Keith Weigelt. 1988. Experimental tests of a sequential equilibrium reputation model. Econometrica: Journal of the Econometric Society (1988), 1-36.
  6. Lida Ghaemi Dizaji and Yaoping Hu. 2021. Building And Measuring Trust In Human-Machine Systems. In 2021 IEEE International Conference on Autonomous Systems (ICAS). 1-5. https://doi.org/10.1109/ICAS49788.2021.9551131 · DOI
  7. Katherine M Engelke, Valerie Hase, and Florian Wintterlin. 2019. On measuring trust and distrust in journalism: Reflection of the status quo and suggestions for the road ahead. Journal of Trust Research 9, 1 (Jan. 2019), 66-86. https://doi.org/10.1080/21515581.2019.1588741 · DOI
  8. Rotem D Guttman, Jessica Hammer, Erik Harpstead, and Carol J Smith. 2021. Play for Real(ism) -Using Games to Predict Human-AI interactions in the Real World. Proc. ACM Hum.-Comput. Interact. 5, CHI PLAY (Oct. 2021), 1-17. https://doi.org/10.1145/3474655 · DOI
  9. Robert R Hoffman, Shane T Mueller, Gary Klein, and Jordan Litman. 2018. Metrics for explainable AI: Challenges and prospects. arXiv preprint arXiv:1812.04608 (2018). · arXiv
  10. Ming Hou, Geoffrey Ho, and David Dunwoody. 2021. IMPACTS: a trust model for human-autonomy teaming. Hum.-Intell. Syst. Integr. 3, 2 (June 2021), 79-97. https://doi.org/10.1007/s42454-020-00023-x · DOI
  11. Tobias Huber, Katharina Weitz, Elisabeth André, and Ofra Amir. 2021. Local and global explanations of agent behavior: Integrating strategy summaries with saliency maps. Artificial Intelligence 301 (2021), 103571.
  12. Aya Hussein, Sondoss Elsawah, and Hussein A Abbass. 2020. Trust mediating reliability-reliance relationship in supervisory control of human-swarm interactions. Human Factors 62, 8 (2020), 1237-1248.
  13. Alon Jacovi, Ana Marasovic, Tim Miller, and Yoav Goldberg. 2021. Formalizing trust in artificial intelligence: Prerequisites, causes and goals of human trust in AI. In Proceedings of ACM Conference on Fairness, Accountability, and Transparency (ACM FAccT 2021). https://arxiv.org/abs/2010.07487 · arXiv
  14. Jiun-Yin Jian, Ann M Bisantz, and Colin G Drury. 2000. Foundations for an Empirically Determined Scale of Trust in Automated Systems. Int. J. Cogn. Ergon. 4, 1 (March 2000), 53-71. https://doi.org/10.1207/S15327566IJCE0401_04 · DOI · Paper page
  15. Herbert W. Kee and Robert E. Knox. 1970. Conceptual and methodological considerations in the study of trust and suspicion1: Conceptual Considerations Methodological Considerations Summary and Conclusions REFERENCES. The Journal of Conflict Resolution (pre-1986) 14, 3 (09 1970), 357.
  16. Mohammed Laeequddin, B S Sahay, Vinita Sahay, and Waheed K Abdul. 2010. Measuring trust in supply chain partners' relationships. Measuring Business Excellence 14, 3 (Jan. 2010), 53-69. https://doi.org/10.1108/13683041011074218 · DOI
  17. John D Lee and Katrina A See. 2004. Trust in automation: designing for appropriate reliance. Hum. Factors 46, 1 (2004), 50-80. https://doi.org/10. 1518/hfes.46.1.50_30392 · DOI · Paper page
  18. Roger C Mayer, James H Davis, and F David Schoorman. 1995. An Integrative Model Of Organizational Trust. AMRO 20, 3 (July 1995), 709-734. https://doi.org/10.5465/amr.1995.9508080335 · DOI · Paper page
  19. Bill McEvily and Marco Tortoriello. 2011. Measuring trust in organisational research: Review and recommendations. Journal of Trust Research 1, 1 (April 2011), 23-63. https://doi.org/10.1080/21515581.2011.552424 · DOI · Paper page
  20. David Miller, Mishel Johns, Brian Mok, Nikhil Gowda, David Sirkin, Key Lee, and Wendy Ju. 2016. Behavioral Measurement of Trust in Automation: The Trust Fall. Proc. Hum. Fact. Ergon. Soc. Annu. Meet. 60, 1 (Sept. 2016), 1849-1853. https://doi.org/10.1177/1541931213601422 · DOI · Paper page
  21. Jason R C Nurse, Ioannis Agrafiotis, Michael Goldsmith, Sadie Creese, and Koen Lamberts. 2014. Two sides of the coin: measuring and communicating the trustworthiness of online information. Journal of Trust Management 1, 1 (May 2014), 1-20. https://doi.org/10.1186/2196-064X-1-5 · DOI
  22. Raja Parasuraman and Victor Riley. 1997. Humans and Automation: Use, Misuse, Disuse, Abuse. Hum. Factors 39, 2 (June 1997), 230-253. https://doi.org/10.1518/001872097778543886 · DOI · Paper page
  23. Philipp Schmidt and Felix Biessmann. 2019. Quantifying Interpretability and Trust in Machine Learning Systems. (Jan. 2019). arXiv:1901.08558 [cs.LG] http://arxiv.org/abs/1901.08558 · arXiv
  24. Gaurav Tejpal, R K Garg, and Anish Sachdeva. 2013. Trust among supply chain partners: a review. Meas. Bus. Excel. 17, 1 (March 2013), 51-71. https://doi.org/10.1108/13683041311311365 · DOI
  25. Jennifer Wang and Angela Moulden. 2021. AI Trust Score: A User-Centered Approach to Building, Designing, and Measuring the Success of Intelligent Workplace Features. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI EA '21, Article 54). Association for Computing Machinery, New York, NY, USA, 1-7. https://doi.org/10.1145/3411763.3443452 · DOI · Paper page
  26. Lu Wang, Greg A Jamieson, and Justin G Hollands. 2009. Trust and reliance on an automated combat identification system. Human factors 51, 3 (2009), 281-291.

Cited by

Sorted by citations

Metadata

Authors
Miller, T.
Venue
arXiv
Publisher
arXiv
Published
2022
DOI
10.48550/arxiv.2209.00651
arXiv
2209.00651
References
0
Type
Preprint
Licence
creativecommons.org/licenses/by/4.0/legalcode

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review