CoChat

How to Evaluate Trust in AI-Assisted Decision Making? A Survey of Empirical Methodologies

Open access
Vereschak, Oleksandra, Bailly, Gilles, Caramiaux, Baptiste · 2021 · Proceedings of the ACM on Human-Computer Interaction

The DOI matches its Crossref record; 212 of the 215 references it cites were found in a registry or an index.

Abstract

The spread of AI-embedded systems involved in human decision making makes studying human trust in these systems critical. However, empirically investigating trust is challenging. One reason is the lack of standard protocols to design trust experiments. In this paper, we present a survey of existing methods to empirically investigate trust in AI-assisted decision making and analyse the corpus along the constitutive elements of an experimental protocol. We find that the definition of trust is not commonly integrated in experimental protocols, which can lead to findings that are overclaimed or are hard to interpret and compare across studies. Drawing from empirical practices in social and cognitive studies on human-human trust, we provide practical guidelines to improve the methodology of studying Human-AI trust in decision-making contexts. In addition, we bring forward research opportunities of two types: one focusing on further investigation regarding trust methodologies and the other on factors that impact Human-AI trust.

Overview

Integrity signals

Registry

Crossref · 10.1145/3476068

Cites

212 of 212 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

The text we found for this paper looks like another document, so its claims and self-check weren’t run. The registry and citation checks still apply.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The text we found for this paper looks like another document, so the self-check wasn’t run.

Citations

Cites

Research works printed but not linked: 48

  1. 10.1037/0022--3514.46.3.610 · doi:10.1037/0022--3514.46.3.610
  2. S. Boon and J. Holmes. 1991. The dynamics of interpersonal trust: resolving uncertainty in the ace of risk. In Cooperation and Prosocial Behaviour R. Hinde and J. Gorebel (Eds.). Cambridge University Press Cambridge 190--211.
  3. Raja Chatila Virginia Dignum Michael Fisher Fosca Giannotti Katharina Morik Stuart Russell and Karen Yeung. 2021. Trustworthy AI. Springer International Publishing Cham 13--39. https://doi.org/10.1007/978--3-030--69128--8_2 · doi:10.1007/978--3-030--69128--8_2
  4. 10.1037//0021--9010.87.4.611 · doi:10.1037//0021--9010.87.4.611
  5. 10.1111/j.1751--9004.2009.00232.x · doi:10.1111/j.1751--9004.2009.00232.x
  6. 10.1162/JEEA.2009.7.2--3.235 · doi:10.1162/jeea.2009.7.2--3.235
  7. AXA Research Fund. 2019. Artificial Intelligence: Fostering Trust. Technical Report. AXA. 45 pages. https://www.axa-research.org/en/news/AI-research-guide
  8. Diego Gambetta. [n.d.]. Can We Trust Trust? Department of Sociology,University of Oxford, Oxford, United Kingdom.
  9. Nicole Gillespie. 2003. Measuring trust in working relationships: The behavioral trust inventory. Melbourne Business School, Melbourne, Australia.
  10. 10.1108/00483480610682299 · DOI · Paper page
  11. 10.1007/978-0--387--73655--6_19 · doi:10.1007/978-0--387--73655--6_19
  12. 10.1037//0022--3514.49.1.95 · doi:10.1037//0022--3514.49.1.95
  13. 10.1038/s42256-019-0088--2 · doi:10.1038/s42256-019-0088--2
  14. 10.1016/j.joep.2011.05.00 · doi:10.1016/j.joep.2011.05.00
  15. 10.1002/j.2333--8504.1967.tb00991.x · doi:10.1002/j.2333--8504.1967.tb00991.x
  16. 10.1016/j.obhdp.2012.08.0 · doi:10.1016/j.obhdp.2012.08.0
  17. KPMG. 2019. Controlling AI: The imperative for transparency and explainability. Technical Report. KPMG. 28 pages. https://advisory.kpmg.us/articles/2019/controlling-ai.html
  18. 10.1518/hfes.46.1.50.30392 · doi:10.1518/hfes.46.1.50.30392 · Paper page
  19. 10.2307/259288 · DOI · Paper page
  20. 10.1007/978--94-010--1276--8_19 · doi:10.1007/978--94-010--1276--8_19
  21. Niklas Luhmann. 1979. Trust and Power (1 ed.). Wiley, Chichester, Toronto.
  22. Nikolas Luhmann. 2000. Familiarity, Confidence, Trust: Problems and Alternatives. In Trust: Making and Breaking Cooperative Relations, Diego Gambetta (Ed.). Basil Blackwell, Oxford, United Kingdom, 94--107.
  23. Maria A. Madsen and Shirley Gregor. 2000. Measuring Human-Computer Trust. In Proceedings of the 11 th Australasian Conference on Information Systems. Australasian Conference on Information Systems (ACIS), Brisbane, Australia, 6--8.
  24. C. Mantzavinos. 2020. Hermeneutics. In The Stanford Encyclopedia of Philosophy (spring 2020 ed.), Edward N. Zalta (Ed.). Metaphysics Research Lab, Stanford University, Stanford, CA, USA.
  25. 10.1016/S0969--5931(03)00037--4 · doi:10.1016/s0969--5931(03)00037--4
  26. 10.1037/0021--9010.84.1.123 · doi:10.1037/0021--9010.84.1.123
  27. 10.1007/3--540--45547--7_3 · doi:10.1007/3--540--45547--7_3
  28. 10.5465/amr.1998.926622 · DOI · Paper page
  29. Joachim Meyer and John D. Lee. 2013. Trust, Reliance, and Compliance. In The Oxford Handbook of Cognitive Engineering, John D. Lee and Alex Kirlik (Eds.). Oxford University Press, Oxford, UK, 1--29. https://www.oxfordhandbooks.com/view/10.1093/oxfordhb/9780199757183.001.0001/oxfordhb-9780199757183-e-6 · doi:10.1093/oxfordhb/9780199757183.001.0001/oxfordhb-9780199757183-e-6
  30. B.M. Muir. 1989. Operators' Trust in and Use of Automatic Controllers in a Supervisory Process Control Task. University of Toronto, Toronto, Canada. https://books.google.fr/books?id=T94NSwAACAAJ
  31. Domen Novak. 2014. Engineering Issues in Physiological Computing. 17--38. https://doi.org/10.1007/978--1--4471--6392--3_2 · doi:10.1007/978--1--4471--6392--3_2
  32. Claus Offe. [n.d.]. How can we trust our fellow citizens? Cambridge UP, Cambridge, United Kingdom.
  33. 10.1111/2041--210X.12860 · doi:10.1111/2041--210x.12860
  34. 10.5214/ans.0972--7531.1017309 · doi:10.5214/ans.0972--7531.1017309
  35. 10.1111/j.1744--6198.2011.00230.x · doi:10.1111/j.1744--6198.2011.00230.x
  36. Gjalt-Jorn Peters. 2014. The alpha and the omega of scale reliability and validity: Why and how to abandon Conbach's alpha and the route towards more comprehensive assessment od scale quality. Euro Health Psychologist 16 (01 2014), 56--69.
  37. J. Potter and D. Edwards. 1996. Discourse Analysis. Macmillan Education UK London 419--425. https://doi.org/10.1007/978--1--349--24483--6_63 · doi:10.1007/978--1--349--24483--6_63
  38. 10.1016/B978-0--12--800985--7.00007--7 · doi:10.1016/b978-0--12--800985--7.00007--7
  39. Accenture Federal Services. 2019. Responsible AI: A Framework for Building Trust in your AI Solutions. Technical Report. Accenture. 13 pages. https://www.accenture.com/us-en/insights/us-federal-government/ai-is-ready-are-we
  40. 10.1007/s11336-008--9101-0 · doi:10.1007/s11336-008--9101-0
  41. Richard S. Sutton and Andrew G. Barto. 1998. Introduction to Reinforcement Learning (1st ed.). MIT Press, Cambridge, MA, USA.
  42. 10.1111/j.1758--6623.2005.tb00554.x · doi:10.1111/j.1758--6623.2005.tb00554.x
  43. M. Wang, A. Hussein, R. F. Rojas, K. Shafi, and H. A. Abbass. 2018. EEG-Based Neural Correlates of Trust in Human-Autonomy Interaction. In 2018 IEEE Symposium Series on Computational Intelligence (SSCI). IEEE, Bangalore, India, 350--357.
  44. Ning Wang, David V. Pynadath, and Susan G. Hill. 2016. Trust Calibration within a Human-Robot Team: Comparing Automatically Generated Explanations. In The Eleventh ACM/IEEE International Conference on Human Robot Interaction (HRI '16). IEEE Press, New York, NY, USA, 109--116.
  45. 10.1111/j.1468--2958.1977.tb00523.x · doi:10.1111/j.1468--2958.1977.tb00523.x
  46. T. Whelan. 2008. Social Presence in Multi-User Virtual Environments : A Review and Measurement Framework for Organizational Research.
  47. 10.5555/3378680.3378689 · doi:10.5555/3378680.3378689
  48. Louise C. Young and Gerald S. Albaum. 2002. Developing a measure of trust in retail relationships : a direct selling application. School of Marketing, University of Technology of Sydney, Sydney Broadway, N.S.W, Australia.

Cited by

See all 235

Metadata

Authors
Vereschak, O., Bailly, G., & Caramiaux, B.
Affiliations
  • Sorbonne Université (FR)
  • Centre National de la Recherche Scientifique (FR)
Venue
Proceedings of the ACM on Human-Computer Interaction
Published
2021-10-13
Volume
5
Issue
CSCW2
Pages
1-39
ISSN
2573-0142 (ISSN-L)
OpenAlex
W3207598588
Cited by
229
References
215
Citations by year
2026: 58 · 2025: 61 · 2024: 64 · 2023: 31 · 2022: 15
FWCI
26.95
Type
journal article
Language
en
Locations
  • Proceedings of the ACM on Human-Computer Interaction (publishedVersion)
  • HAL (submittedVersion, other-oa)
Funders
ANR-11-LABX-65 (ANR-11-IDEX-0004-02)
Topics
  • Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
  • Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
Keywords
trust in AI, AI-assisted decision-making, experimental protocol, trust measurement, trust definition, research methodology guidelines, trust factors

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.