How to Evaluate Trust in AI-Assisted Decision Making? A Survey of Empirical Methodologies
Open accessThe DOI matches its Crossref record; 212 of the 215 references it cites were found in a registry or an index.
Abstract
The spread of AI-embedded systems involved in human decision making makes studying human trust in these systems critical. However, empirically investigating trust is challenging. One reason is the lack of standard protocols to design trust experiments. In this paper, we present a survey of existing methods to empirically investigate trust in AI-assisted decision making and analyse the corpus along the constitutive elements of an experimental protocol. We find that the definition of trust is not commonly integrated in experimental protocols, which can lead to findings that are overclaimed or are hard to interpret and compare across studies. Drawing from empirical practices in social and cognitive studies on human-human trust, we provide practical guidelines to improve the methodology of studying Human-AI trust in decision-making contexts. In addition, we bring forward research opportunities of two types: one focusing on further investigation regarding trust methodologies and the other on factors that impact Human-AI trust.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The text we found for this paper looks like another document, so the self-check wasn’t run.
Citations
Cites
An integrative model of organizational trust
Mayer, R. C., Davis, J. H., & Schoorman, F. D. (1995) · The Academy of Management Review · doi:10.2307/258792 · Cited by 10,736
Not So Different After All: A Cross-Discipline View Of Trust
Rousseau, D. M., Sitkin, S. B., Burt, R. S., & Camerer, C. (1998) · Academy of Management Review · doi:10.5465/amr.1998.926617 · Cited by 10,274
The critical incident technique.
Flanagan, J. C. (1954) · Psychological Bulletin · doi:10.1037/h0061470 · Cited by 8,191 · cited 1×
“Critical incident technique [57] is a set of procedures used to collect data from narrated past experiences (or observations) to identify and brainstorm about important events related to a pre-defined problem [7].”
Affect and Cognition Based Trust as Foundations for Interpersonal Cooperation in Organizations
McAllister, D. J. (1995) · Academy of Management Journal · doi:10.2307/256727 · Cited by 7,346 · cited 2×
“Madsen [130] (adapted from McAllister [139]) HCI, adapted from Human-Human Trust ✗ ✓ ✓ [48, 109, 237]”
An overview of heart rate variability metrics and norms
Shaffer, F., & Ginsberg, J. P. (2017) · Frontiers in Public Health · doi:10.3389/fpubh.2017.00258 · Cited by 7,185
Trust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605 · cited 11×
“In the rest of our paper, we rely on the Lee and See [116] definition when referring to trust: “An”
InfluentialDeveloping and validating trust measures for e-commerce: an integrative typology
McKnight, D. H., Choudhury, V., & Kacmar, C. (2002) · Information Systems Research · doi:10.1287/isre.13.3.334.81 · Cited by 5,550 · cited 4×
“The examples of such questionnaires are [11, 39, 68, 137, 142].”
Influential(2007) · Encyclopedia of Industrial and Organizational Psychology · doi:10.4135/9781412952651.n55 · Cited by 5,287
Boys will be Boys: Gender, Overconfidence, and Common Stock Investment
Barber, B. M., & Odean, T. (2001) · The Quarterly Journal of Economics · doi:10.1162/003355301556400 · Cited by 5,276 · cited 1×
“It is believed that its magnitude depends on gender [13], age or culture [93].”
Lewis, J. D., & Weigert, A. (1985) · Social Forces · doi:10.2307/2578601 · Cited by 4,163 · cited 1×
“Trust has grounds to form only when one thinks that negative outcomes associated with trusting do not exist or are very unlikely [121].”
Ohanian, R. (1990) · Journal of Advertising · doi:10.1080/00913367.1990.10673191 · Cited by 4,043 · cited 1×
“2 questionnaires were taken fromHuman-Human trust literature [137, 161, 230] (n = 5, 15.”
Dunn, T. J., Baguley, T., & Brunsden, V. (2014) · British Journal of Psychology · doi:10.1111/bjop.12046 · Cited by 3,596
Trust in leadership: Meta-analytic findings and implications for research and practice.
Dirks, K. T., & Ferrin, D. L. (2002) · Journal of Applied Psychology · doi:10.1037/0021-9010.87.4.611 · Cited by 3,550
Rempel, J. K., Holmes, J. G., & Zanna, M. P. (1985) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.49.1.95 · Cited by 3,146 · cited 1×
“As we have discussed before, trust does not always translate to a behavior, but there is definitely a relation between them [43, 44, 86, 88, 116, 145].”
Trust And Distrust: New Relationships and Realities
Lewicki, R. J., McAllister, D. J., & Bies, R. J. (1998) · The Academy of Management Review · doi:10.5465/amr.1998.926620 · Cited by 3,091
The gender similarities hypothesis.
Hyde, J. S. (2005) · American Psychologist · doi:10.1037/0003-066x.60.6.581 · Cited by 3,084
Human trust in artificial intelligence: Review of empirical research
Glikson, E., & Woolley, A. W. (2020) · Academy of Management Annals · doi:10.5465/annals.2018.0057 · Cited by 2,921 · cited 3×
“To address it, we followed the methodology presented in [72] and included the following systems if a paper did not explicitly mention the system is AI-embedded: robots, virtual agents, and automated vehicles.”
Trust in Automation: Integrating Empirical Evidence on Factors That Influence Trust
Hoff, K. A., & Bashir, M. (2015) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720814547570 · Cited by 2,817
On the use, the misuse, and the very limited usefulness of Cronbach’s alpha
Sijtsma, K. (2009) · Psychometrika · doi:10.1007/s11336-008-9101-0 · Cited by 2,814
O.Nyumba, T., Wilson, K., Derrick, C. J., & Mukherjee, N. (2018) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.12860 · Cited by 2,619 · cited 1×
“Moreover, some participants might get too shy to express their real opinions, especially for such personal topics as trust [163].”
Explaining Explanations: An Overview of Interpretability of Machine Learning
Gilpin, L. H., Bau, D., Yuan, B. Z., Bajwa, A., Specter, M., & Kagal, L. (2018) · 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA) · doi:10.1109/dsaa.2018.00018 · Cited by 2,507 · cited 2×
“[70] discuss the theoretical differences between interpretability and explanability when interacting with an AI.”
A General Approach to Confirmatory Maximum Likelihood Factor Analysis
Jöreskog, K. G. (1969) · Psychometrika · doi:10.1007/bf02289343 · Cited by 2,489
Guidelines for Human-AI Interaction
Amershi, S., Weld, D., Vorvoreanu, M., Fourney, A., Nushi, B., Collisson, P., Suh, J., Iqbal, S., Bennett, P. N., Inkpen, K., et al. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300233 · Cited by 2,439 · cited 1×
“[6] present guidelines to help in designing and evaluating AI-embedded systems that users can trust and work with efficiently.”
(2016) · Cited by 2,334
TRUST: THE CONNECTING LINK BETWEEN ORGANIZATIONAL THEORY AND PHILOSOPHICAL ETHICS
Hosmer, L. T. (1995) · The Academy of Management Review · doi:10.5465/amr.1995.9507312923 · Cited by 2,323
The effect of the performance appraisal system on trust for management: A field quasi-experiment.
Mayer, R. C., & Davis, J. H. (1999) · Journal of Applied Psychology · doi:10.1037//0021-9010.84.1.123 · Cited by 2,247
Pay Enough or Don't Pay at All*
Gneezy, U., & Rustichini, A. (2000) · Quarterly Journal of Economics · doi:10.1162/003355300554917 · Cited by 2,175 · cited 1×
“If the bonuses are too small, participants might disregard them and feel unmotivated to perform well [73].”
Deutsch, M. (1958) · Journal of Conflict Resolution · doi:10.1177/002200275800200401 · Cited by 2,077 · cited 1×
“As we have discussed before, trust does not always translate to a behavior, but there is definitely a relation between them [43, 44, 86, 88, 116, 145].”
A Meta-Analysis of Factors Affecting Trust in Human-Robot Interaction
Hancock, P. A., Billings, D. R., Schaefer, K. E., Chen, J. Y. C., de Visser, E. J., & Parasuraman, R. (2011) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720811417254 · Cited by 2,051 · cited 1×
“, [10, 80, 86, 116, 241]) investigate the impact of factors related to user, system and task on trust while interacting with an AI.”
Foundations for an Empirically Determined Scale of Trust in Automated Systems
Jian, J.-Y., Bisantz, A. M., & Drury, C. G. (2000) · International Journal of Cognitive Ergonomics · doi:10.1207/s15327566ijce0401_04 · Cited by 1,799
Interpersonal trust, trustworthiness, and gullibility.
Rotter, J. B. (1980) · American Psychologist · doi:10.1037/0003-066x.35.1.1 · Cited by 1,694 · cited 1×
“While there are some researchers who deem trust and distrust as the opposite ends of one construct [191], recently the community views them as two separate ones [120, 203].”
Baumeister, R. F. (1984) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.46.3.610 · Cited by 1,665
Fundamental Research Statistics for the Behavioral Sciences.
Mandeville, G. K., & Roscoe, J. T. (1971) · Journal of the American Statistical Association · doi:10.2307/2284880 · Cited by 1,618
Reliability: A Review of Psychometric Basics and Recent Marketing Practices
Peter, J. P. (1979) · Journal of Marketing Research · doi:10.1177/002224377901600102 · Cited by 1,592
Explaining the Limited Effectiveness of Legalistic ‘Remedies’ for Trust/Distrust
Sitkin, S. B., & Roth, N. L. (1993) · Organization Science · doi:10.1287/orsc.4.3.367 · Cited by 1,545
The Effects of Financial Incentives in Experiments: A Review and Capital-Labor-Production Framework
Camerer, C. F., Hogarth, R. M., Budescu, D. V., & Eckel, C. (1999) · Elicitation of Preferences · doi:10.1007/978-94-017-1406-8_2 · Cited by 1,545
Trust, control strategies and allocation of function in human-machine systems
LEE, J., & MORAY, N. (1992) · Ergonomics · doi:10.1080/00140139208967392 · Cited by 1,459
The Effects of Financial Incentives in Experiments: A Review and Capital-Labor-Production Framework
Camerer, C. F., & Hogarth, R. M. (1999) · Journal of Risk and Uncertainty · doi:10.1023/a:1007850605129 · Cited by 1,428
Initial trust formation in new organizational relationships
McKnight, D. H., Cummings, L. L., & Chervany, N. L. (1998) · The Academy of Management Review · doi:10.2307/259290 · Cited by 1,296
Luger, E., & Sellen, A. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858288 · Cited by 1,176
How to identify trust and reciprocity
Cox, J. C. (2004) · Games and Economic Behavior · doi:10.1016/s0899-8256(03)00119-2 · Cited by 1,037
Brysbaert, M. (2019) · Journal of Cognition · doi:10.5334/joc.72 · Cited by 996
Johnson, N. D., & Mislin, A. A. (2011) · Journal of Economic Psychology · doi:10.1016/j.joep.2011.05.007 · Cited by 955
Bohner, G., & Dickel, N. (2011) · Annual Review of Psychology · doi:10.1146/annurev.psych.121208.131609 · Cited by 951
Trizano-Hermosilla, I., & Alvarado, J. M. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2016.00769 · Cited by 942
Trust, self-confidence, and operators' adaptation to automation
Lee, J. D., & Moray, N. (1994) · International Journal of Human-Computer Studies · doi:10.1006/ijhc.1994.1007 · Cited by 932
Questioning the AI: Informing Design Practices for Explainable AI User Experiences
Liao, Q. V., Gruen, D., & Miller, S. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376590 · Cited by 925
Designing Theory-Driven User-Centric Explainable AI
Wang, D., Yang, Q., Abdul, A., & Lim, B. Y. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300831 · Cited by 906
Zhang, Y., Liao, Q. V., & Bellamy, R. K. E. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372852 · Cited by 895
At What Level (and in Whom) We Trust: Trust Across Multiple Organizational Levels
Fulmer, C. A., & Gelfand, M. J. (2012) · Journal of Management · doi:10.1177/0149206312439327 · Cited by 886
The Effect of Motivational Orientation upon Trust and Suspicion
Deutsch, M. (1960) · Human Relations · doi:10.1177/001872676001300202 · Cited by 808
Levels of Organizational Trust in Individualist Versus Collectivist Societies: A Seven-Nation Study
Huff, L., & Kelley, L. (2003) · Organization Science · doi:10.1287/orsc.14.1.81.12807 · Cited by 686
On The Economics and Biology of Trust
Fehr, E. (2009) · Journal of the European Economic Association · doi:10.1162/jeea.2009.7.2-3.235 · Cited by 680
Local Standards for Sample Size at CHI
Caine, K. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858498 · Cited by 657
Measuring trust inside organisations
Dietz, G., & Den Hartog, D. N. (2006) · Personnel Review · doi:10.1108/00483480610682299 · Cited by 596
Cai, C. J., Winter, S., Steiner, D., Wilcox, L., & Terry, M. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359206 · Cited by 594
Measuring Trust between Organizational Boundary Role Persons
Currall, S. C., & Judge, T. A. (1995) · Organizational Behavior and Human Decision Processes · doi:10.1006/obhd.1995.1097 · Cited by 592
Bohner, G., & Wanke, M. (2014) · Psychology Press eBooks · doi:10.4324/9781315784786 · Cited by 584
Understanding the Effect of Accuracy on Trust in Machine Learning Models
Yin, M., Wortman Vaughan, J., & Wallach, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300509 · Cited by 527 · cited 6×
“For instance, we mentioned several studies that indicated that 60% - 70% accuracy could be considered to be a threshold for investigating users’ trust in AI-embedded decision-support systems [241, 244, 248].”
InfluentialVeale, M., Van Kleek, M., & Binns, R. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174014 · Cited by 490 · cited 2×
“…81, 84, 87, 89, 91, 97, 107, 112, 118, 122, 123, 126, 136, 150, 154, 156, 165, 166, 170, 176, 177, 180, 185, 196–198, 201, 207, 210, 214–216, 219–221, 225, 227, 228, 233, 234, 238, 241, 243, 248] To better illustrate these elements, let’s imagine a situation where a patient has a serious…”
UX Curve: A method for evaluating long-term user experience
Kujala, S., Roto, V., Väänänen-Vainio-Mattila, K., Karapanos, E., & Sinnelä, A. (2011) · Interacting with Computers · doi:10.1016/j.intcom.2011.06.005 · Cited by 485
Calibration of Probabilities: The State of the Art
Lichtenstein, S., Fischhoff, B., & Phillips, L. D. (1977) · Decision Making and Change in Human Affairs · doi:10.1007/978-94-010-1276-8_19 · Cited by 483
Betrayal Aversion: Evidence from Brazil, China, Oman, Switzerland, Turkey, and the United States
Bohnet, I., Greig, F., Herrmann, B., & Zeckhauser, R. (2008) · American Economic Review · doi:10.1257/aer.98.1.294 · Cited by 481 · cited 1×
“However, such games are criticized for confounding trust with altruism [37] and betrayal aversion [17, 55], and for the lack of stability [25, 99, 208], and hence, should not be preferred for measuring trust-related behaviors.”
Measuring trust in organisational research: Review and recommendations
McEvily, B., & Tortoriello, M. (2011) · Journal of Trust Research · doi:10.1080/21515581.2011.552424 · Cited by 477
THE MEASUREMENT OF TRUST AND ITS RELATIONSHIP TO SELF-DISCLOSURE
WHEELESS, L. R., & GROTZ, J. (1977) · Human Communication Research · doi:10.1111/j.1468-2958.1977.tb00523.x · Cited by 474
Overtrust of robots in emergency evacuation scenarios
Robinette, P., Li, W., Allen, R., Howard, A. M., & Wagner, A. R. (2016) · 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1109/hri.2016.7451740 · Cited by 468
Magaldi, D., & Berler, M. (2020) · Encyclopedia of Personality and Individual Differences · doi:10.1007/978-3-319-24612-3_857 · Cited by 433
Human-Centered Tools for Coping with Imperfect Algorithms During Medical Decision-Making
Cai, C. J., Reif, E., Hegde, N., Hipp, J., Kim, B., Smilkov, D., Wattenberg, M., Viegas, F., Corrado, G. S., Stumpe, M. C., et al. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300234 · Cited by 405
Testing the Difference Between Reliability Coefficients Alpha and Omega
Deng, L., & Chan, W. (2017) · Educational and Psychological Measurement · doi:10.1177/0013164416658325 · Cited by 395
Lai, V., & Tan, C. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287590 · Cited by 377
Evaluation by Moments: Past and Future
Kahneman, D. (2000) · Choices, Values, and Frames · doi:10.1017/cbo9780511803475.039 · Cited by 368
Using multi‐item psychometric scales for research and practice in human resource management
Robinson, M. A. (2018) · Human Resource Management · doi:10.1002/hrm.21852 · Cited by 353
A caveat on using single‐item versus multiple‐item scales
Loo, R. (2002) · Journal of Managerial Psychology · doi:10.1108/02683940210415933 · Cited by 347
Studies in the reliability and validity of the critical incident technique.
Andersson, B.-E., & Nilsson, S.-G. (1964) · Journal of Applied Psychology · doi:10.1037/h0042025 · Cited by 327
Proxy tasks and subjective measures can be misleading in evaluating explainable AI systems
Buçinca, Z., Lin, P., Gajos, K. Z., & Glassman, E. L. (2020) · Proceedings of the 25th International Conference on Intelligent User Interfaces · doi:10.1145/3377325.3377498 · Cited by 307
Trust building with explanation interfaces
Pu, P., & Chen, L. (2006) · Proceedings of the 11th international conference on Intelligent user interfaces · doi:10.1145/1111449.1111475 · Cited by 300 · cited 1×
“…8, 22, 24, 26, 27, 33, 34, 38, 42, 50, 56, 60, 67, 71, 75, 81, 84, 87, 89, 91, 97, 107, 112, 118, 122, 123, 126, 136, 150, 154, 156, 165, 166, 170, 176, 177, 180, 185, 196–198, 201, 207, 210, 214–216, 219–221, 225, 227, 228, 233, 234, 238, 241, 243, 248] To better illustrate these elements, let’s…”
Further Understanding of Trust and Performance in Virtual Teams
Aubert, B. A., & Kelsey, B. L. (2003) · Small Group Research · doi:10.1177/1046496403256011 · Cited by 286
Helldin, T., Falkman, G., Riveiro, M., & Davidsson, S. (2013) · Proceedings of the 5th International Conference on Automotive User Interfaces and Interactive Vehicular Applications · doi:10.1145/2516540.2516554 · Cited by 286 · cited 1×
“No source stated - ✗ ✓ ✓ [193] No source stated - ✓ ✗ ✓ [235] No definition: [1, 4, 8, 22, 24, 26, 27, 33, 34, 38, 42, 50, 56, 60, 67, 71, 75, 81, 84, 87, 89, 91, 97, 107, 112, 118, 122, 123, 126, 136, 150, 154, 156, 165, 166, 170, 176, 177, 180, 185, 196–198, 201, 207, 210, 214–216, 219–221, 225,…”
Friend-or-foe intentionality priming in an extensive form trust game
Burnham, T., McCabe, K., & Smith, V. L. (2000) · Journal of Economic Behavior & Organization · doi:10.1016/s0167-2681(00)00108-6 · Cited by 282
Trust, trustworthiness, and the F scale.
Deutsch, M. (1960) · The Journal of Abnormal and Social Psychology · doi:10.1037/h0046501 · Cited by 266
Affective Processes in Human–Automation Interactions
Merritt, S. M. (2011) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720811411912 · Cited by 260 · cited 1×
“Four of the questionnaires originate from Human-Automation literature [33, 96, 144, 149] and are cited by almost half (n = 15) of the papers with questionnaires.”
Deb, S., Strawderman, L., Carruth, D. W., DuBien, J., Smith, B., & Garrison, T. M. (2017) · Transportation Research Part C: Emerging Technologies · doi:10.1016/j.trc.2017.08.029 · Cited by 259
A Qualitative Exploration of Perceptions of Algorithmic Fairness
Woodruff, A., Fox, S. E., Rousso-Schindler, S., & Warshaw, J. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174230 · Cited by 256
Communication, the Development of Trust, and Cooperative Behavior
Loomis, J. L. (1959) · Human Relations · doi:10.1177/001872675901200402 · Cited by 249
Effects of Anthropomorphism and Accountability on Trust in Human Robot Interaction
Natarajan, M., & Gombolay, M. (2020) · Proceedings of the 2020 ACM/IEEE International Conference on Human-Robot Interaction · doi:10.1145/3319502.3374839 · Cited by 248 · cited 2×
“9% e.g. game points [154, 244–246]), negative virtual consequences for participants ( 𝑛 = 29, 44 .”
Floersch, J., Longhofer, J. L., Kranke, D., & Townsend, L. (2010) · Qualitative Social Work · doi:10.1177/1473325010362330 · Cited by 247
Passi, S., & Jackson, S. J. (2018) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3274405 · Cited by 225
Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy
Verma, S. (2019) · Vikalpa: The Journal for Decision Makers · doi:10.1177/0256090919853933 · Cited by 219
Human factors in model interpretability: Industry practices, challenges, and needs
Hong, S. R., Hullman, J., & Bertini, E. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3392878 · Cited by 219 · cited 1×
“…stated - ✗ ✓ ✓ [193] No source stated - ✓ ✗ ✓ [235] No definition: [1, 4, 8, 22, 24, 26, 27, 33, 34, 38, 42, 50, 56, 60, 67, 71, 75, 81, 84, 87, 89, 91, 97, 107, 112, 118, 122, 123, 126, 136, 150, 154, 156, 165, 166, 170, 176, 177, 180, 185, 196–198, 201, 207, 210, 214–216, 219–221, 225, 227,…”
Toward establishing trust in adaptive agents
Glass, A., McGuinness, D. L., & Wolverton, M. (2008) · Proceedings of the 13th international conference on Intelligent user interfaces · doi:10.1145/1378773.1378804 · Cited by 211
Evaluating Effects of User Experience and System Transparency on Trust in Automation
Yang, X. J., Unhelkar, V. V., Li, K., & Shah, J. A. (2017) · Proceedings of the 2017 ACM/IEEE International Conference on Human-Robot Interaction · doi:10.1145/2909824.3020230 · Cited by 208 · cited 3×
“Lee and See [116] and Lee and Moray [115] Automation, adapted from Human-Human Trust ✓ ✓ ✓ [53, 61, 111, 181, 209, 239, 244, 245]; [246]”
Brown, A., Chouldechova, A., Putnam-Hornstein, E., Tobin, A., & Vaithianathan, R. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300271 · Cited by 207
How do visual explanations foster end users' appropriate trust in machine learning?
Yang, F., Huang, Z., Scholtz, J., & Arendt, D. L. (2020) · Proceedings of the 25th International Conference on Intelligent User Interfaces · doi:10.1145/3377325.3377480 · Cited by 200 · cited 5×
“2-5 days apart) to reduce learning effects [237].”
InfluentialThe Perception And Measurement Of Human-robot Trust
Schaefer, K. E. (2013) · STARS (University of Central Florida) · Cited by 192
Characterizing Children’s Expectations About Expertise and Incompetence: Halo or Pitchfork Effects?
Koenig, M. A., & Jaswal, V. K. (2011) · Child Development · doi:10.1111/j.1467-8624.2011.01618.x · Cited by 187
Landrum, A. R., Mills, C. M., & Johnston, A. M. (2013) · Developmental Science · doi:10.1111/desc.12059 · Cited by 187
The State of the Art in Enhancing Trust in Machine Learning Models with the Use of Visualizations
Chatzimparmpas, A., Martins, R. M., Jusufi, I., Kucher, K., Rossi, F., & Kerren, A. (2020) · Computer Graphics Forum · doi:10.1111/cgf.14034 · Cited by 182
Concept Analysis: Confidence/Self-confidence: Concept Analysis: Self-confidence
Perry, P. (2011) · Nursing Forum · doi:10.1111/j.1744-6198.2011.00230.x · Cited by 177
The Psychology (and Economics) of Trust
Evans, A. M., & Krueger, J. I. (2009) · Social and Personality Psychology Compass · doi:10.1111/j.1751-9004.2009.00232.x · Cited by 171
Getting Off on the Wrong Foot: The Timing of a Breach and the Restoration of Trust
Lount, R. B., Zhong, C.-B., Sivanathan, N., & Murnighan, J. K. (2008) · Personality and Social Psychology Bulletin · doi:10.1177/0146167208324512 · Cited by 171
Handbook of Research Methods on Trust: Second Edition
(2015) · Edward Elgar Publishing eBooks · doi:10.4337/9781782547419 · Cited by 169
Informed Consent and Medical Artificial Intelligence: What to Tell the Patient?
Cohen, I. G. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3529576 · Cited by 165
Face Value? Exploring the Effects of Embodiment for a Group Facilitation Agent
Shamekhi, A., Liao, Q., Wang, D., Bellamy, R., & Erickson, T. D. (2018) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173965 · Cited by 161
Doing Psychological Science by Hand
Freeman, J. B. (2018) · Current Directions in Psychological Science · doi:10.1177/0963721417746793 · Cited by 159
Measuring Trust: Experiments and Surveys in Contrast and CombinationPreprint
Naef, M., & Schupp, J. (2009) · SSRN Electronic Journal · doi:10.2139/ssrn.1369833 · Cited by 159
(2013) · Psychology Press eBooks · doi:10.4324/9780203726389-31 · Cited by 158
Repairing trust with individuals vs. groups
Kim, P. H., Cooper, C. D., Dirks, K. T., & Ferrin, D. L. (2012) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2012.08.004 · Cited by 152
Effects of Direct Social Experience on Trust Decisions and Neural Reward Circuitry
Fareri, D. S., Chang, L. J., & Delgado, M. R. (2012) · Frontiers in Neuroscience · doi:10.3389/fnins.2012.00148 · Cited by 147 · cited 1×
“For example, if the past experience with a system was negative, a participant is more likely to overreact to an error during an experiment, reconfirming their initial expectations [54].”
Impact of robot failures and feedback on real-time trust
Desai, M., Kaniarasu, P., Medvedev, M., Steinfeld, A. M., & Yanco, H. A. (2013) · doi:10.5555/2447556.2447663 · Cited by 139
In AI We Trust? Factors That Influence Trustworthiness of AI-infused Decision-Making ProcessesPreprint
Ashoori, M., & Weisz, J. D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.02675 · Cited by 122
I can do better than your AI: expertise and explanations
Schaffer, J., O'Donovan, J., Michaelis, J., Raglin, A., & Höllerer, T. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302308 · Cited by 119
Drozdal, J., Weisz, J., Wang, D., Dass, G., Yao, B., Zhao, C., Muller, M., Ju, L., & Su, H. (2020) · Proceedings of the 25th International Conference on Intelligent User Interfaces · doi:10.1145/3377325.3377501 · Cited by 119
The Impact of POMDP-Generated Explanations on Trust and Performance in Human-Robot Teams
Wang, N., Pynadath, D. V., & Hill, S. G. (2016) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/mjoe4434 · Cited by 119
Alós-Ferrer, C., & Farolfi, F. (2019) · Frontiers in Neuroscience · doi:10.3389/fnins.2019.00887 · Cited by 116
This Thing Called Fairness: Disciplinary Confusion Realizing a Value in Technology
Mulligan, D. K., Kroll, J. A., Kohli, N., & Wong, R. Y. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359221 · Cited by 111
Israelsen, B., & Ahmed, N. R. (2019) · ACM Computing Surveys · doi:10.1145/3267338 · Cited by 109
Overtrust in External Cues of Automated Vehicles: An Experimental Investigation
Holländer, K., Wintersberger, P., & Butz, A. (2019) · Proceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications · doi:10.1145/3342197.3344528 · Cited by 109 · cited 2×
“…source stated - ✗ ✓ ✓ [193] No source stated - ✓ ✗ ✓ [235] No definition: [1, 4, 8, 22, 24, 26, 27, 33, 34, 38, 42, 50, 56, 60, 67, 71, 75, 81, 84, 87, 89, 91, 97, 107, 112, 118, 122, 123, 126, 136, 150, 154, 156, 165, 166, 170, 176, 177, 180, 185, 196–198, 201, 207, 210, 214–216, 219–221, 225,…”
AI and HCI: Two Fields Divided by a Common Focus
Grudin, J. (2009) · AI Magazine · doi:10.1609/aimag.v30i4.2271 · Cited by 107
Explainable Active Learning (XAL): Toward AI Explanations as Interfaces for Machine Teachers
Ghai, B., Liao, Q. V., Zhang, Y., Bellamy, R., & Mueller, K. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3432934 · Cited by 106
Yang, Q., Zimmerman, J., Steinfeld, A., Carey, L., & Antaki, J. F. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858373 · Cited by 105 · cited 2×
“Observations in the field (n = 1; [238]) is used to understand doctors’ daily routine and decision-making process.”
A Field Study of Pedestrians and Autonomous Vehicles
Reig, S., Norman, S., Morales, C. G., Das, S., Steinfeld, A. M., & Forlizzi, J. (2018) · International Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI) · doi:10.1145/3239060.3239064 · Cited by 103
Frison, A.-K., Wintersberger, P., Riener, A., Schartmüller, C., Boyle, L. N., Miller, E. E., & Weigl, K. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300374 · Cited by 102
Ackermans, S., Dey, D., Ruijten, P. A. M., Cuijpers, R. H., & Pfleging, B. (2020) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376197 · Cited by 98
What can AI do for me?: evaluating machine learning interpretations in cooperative play
Feng, S., & Boyd-Graber, J. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302265 · Cited by 92
When in Rome: the role of culture & context in adherence to robot recommendations
Wang, L., Rau, P. P., Evers, V., Robinson, B., & HINDS, P. J. (2010) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.5555/1734454.1734578 · Cited by 91
User Trust Dynamics: An Investigation Driven by Differences in System Performance
Yu, K., Berkovsky, S., Taib, R., Conway, D., Zhou, J., & Chen, F. (2017) · Proceedings of the 22nd International Conference on Intelligent User Interfaces · doi:10.1145/3025171.3025219 · Cited by 90
How do children weigh competence and benevolence when deciding whom to trust?
Johnston, A. M., Mills, C. M., & Landrum, A. R. (2015) · Cognition · doi:10.1016/j.cognition.2015.07.015 · Cited by 90
Humans Conform to Robots: Disambiguating Trust, Truth, and Conformity
Salomons, N., van der Linden, M., Strohkorb Sebo, S., & Scassellati, B. (2018) · Proceedings of the 2018 ACM/IEEE International Conference on Human-Robot Interaction · doi:10.1145/3171221.3171282 · Cited by 90
Traffic Augmentation as a Means to Increase Trust in Automated Driving Systems
Wintersberger, P., von Sawitzky, T., Frison, A.-K., & Riener, A. (2017) · Proceedings of the 12th Biannual Conference on Italian SIGCHI Chapter · doi:10.1145/3125571.3125600 · Cited by 89
(2011) · SAGE Publications Ltd eBooks · doi:10.4135/9781446287866.n8 · Cited by 89
Rhetorical robots: Making robots more effective speakers using linguistic cues of expertise
Andrist, S., Spannan, E., & Mutlu, B. (2013) · 2013 8th ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1109/hri.2013.6483608 · Cited by 81
Electrodermal Response and Automation Trust during Simulated Self-Driving Car Use
Morris, D. M., Erno, J. M., & Pilcher, J. J. (2017) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1541931213601921 · Cited by 78
Trust in artificial voices: A "congruency effect" of first impressions and behavioural experience
Torre, I., Goslin, J., White, L., & Zanatto, D. (2018) · Proceedings of the Technology, Mind, and Society · doi:10.1145/3183654.3183691 · Cited by 77
Mouse tracking as a window into decision making
Maldonado, M., Dunbar, E., & Chemla, E. (2019) · Behavior Research Methods · doi:10.3758/s13428-018-01194-x · Cited by 75
Do I trust my machine teammate?: an investigation from perception to decision
Yu, K., Berkovsky, S., Taib, R., Zhou, J., & Chen, F. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302277 · Cited by 71 · cited 2×
“9% e.g. game points [154, 244–246]), negative virtual consequences for participants ( 𝑛 = 29, 44 .”
CarePre: An Intelligent Clinical Decision Assistance System
Jin, Z., Cui, S., Guo, S., Gotz, D., Sun, J., & Cao, N. (2020) · ACM Transactions on Computing for Healthcare · doi:10.1145/3344258 · Cited by 70
Kulms, P., & Kopp, S. (2019) · Proceedings of Mensch und Computer 2019 · doi:10.1145/3340764.3340793 · Cited by 68
Effects of Proactive Dialogue Strategies on Human-Computer Trust
Kraus, M., Wagner, N., & Minker, W. (2020) · Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization · doi:10.1145/3340631.3394840 · Cited by 67
Globally distributed system developers: their trust expectations and processes
Al-Ani, B., Bietz, M. J., Wang, Y., Trainer, E. H., Koehne, B., Marczak, S. D. S., Redmiles, D., & Prikladnicki, R. (2013) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2441776.2441840 · Cited by 66
A Slow Algorithm Improves Users' Assessments of the Algorithm's Accuracy
Park, J. S., Barber, R., Kirlik, A., & Karahalios, K. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359204 · Cited by 65
Yu, B., Yuan, Y., Terveen, L., Wu, Z. S., Forlizzi, J., & Zhu, H. (2020) · Designing Interactive Systems Conference · doi:10.1145/3357236.3395528 · Cited by 65
Brains or Beauty: How to Engender Trust in User-Agent Interactions
Yuksel, B. F., Collisson, P., & Czerwinski, M. (2017) · ACM Transactions on Internet Technology · doi:10.1145/2998572 · Cited by 63
Trust It or Not: Effects of Machine-Learning Warnings in Helping Individuals Mitigate Misinformation
Seo, H., Xiong, A., & Lee, D. (2019) · Proceedings of the 10th ACM Conference on Web Science · doi:10.1145/3292522.3326012 · Cited by 63
When people and algorithms meet: user-reported problems in intelligent everyday applications
Eiband, M., Völkel, S. T., Buschek, D., Cook, S., & Hussmann, H. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302262 · Cited by 62
Veale, M., Van Kleek, M., & Binns, R. (2018) · doi:10.31235/osf.io/8kvf4 · Cited by 60
The influence of agent reliability on trust in human-agent collaboration
Fan, X., Oh, S., McNeese, M., Yen, J., Cuevas, H., Strater, L., & Endsley, M. R. (2008) · Proceedings of the 15th European conference on Cognitive ergonomics: the ergonomics of cool interaction · doi:10.1145/1473018.1473028 · Cited by 59
In AI We Trust: Investigating the Relationship between Biosignals, Trust and Cognitive Load in VR
Gupta, K., Hajika, R., Pai, Y. S., Duenser, A., Lochner, M., & Billinghurst, M. (2019) · 25th ACM Symposium on Virtual Reality Software and Technology · doi:10.1145/3359996.3364276 · Cited by 59
The Effect of Culture on Trust in Automation: Reliability and Workload
Chien, S.-Y., Lewis, M., Sycara, K., Liu, J.-S., & Kumru, A. (2018) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3230736 · Cited by 58
Robot Capability and Intention in Trust-Based Decisions Across Tasks
Xie, Y., Bodala, I. P., Ong, D. C., Hsu, D., & Soh, H. (2019) · 2019 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1109/hri.2019.8673084 · Cited by 57
Lee, M. H., Siewiorek, D. P., Smailagic, A., Bernardino, A., & Bermúdez i Badia, S. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3415227 · Cited by 57
Measuring Trust in Organizational Contexts: An Overview of Survey-based Measures
Gillespie, N. A. (2011) · Edward Elgar Publishing eBooks · doi:10.4337/9780857932013.00027 · Cited by 57
More Text Please! Understanding and Supporting the Use of Visualization for Clinical Text Overview
Sultanum, N., Brudno, M., Wigdor, D., & Chevalier, F. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173996 · Cited by 56
Predictive model to assess user trust: a psycho-physiological approach
Ajenaghughrure, I. B., Sousa, S., Kosunen, I., & Lamas, D. (2019) · Indian Conference on Human-Computer Interaction (IndiaHCI) · doi:10.1145/3364183.3364195 · Cited by 56
Monitoring and Trust in Virtual Teams
Robert, L. (2016) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2818048.2820076 · Cited by 52
Torre, I., Carrigan, E., McDonnell, R., Domijan, K., McCabe, K., & Harte, N. (2019) · Motion in Games · doi:10.1145/3359566.3360065 · Cited by 50
Misplaced Trust: Measuring the Interference of Machine Learning in Human Decision-Making
Suresh, H., Lao, N., & Liccardi, I. (2020) · 12th ACM Conference on Web Science · doi:10.1145/3394231.3397922 · Cited by 49 · cited 2×
“Lee and See [116] and Lee and Moray [115] Automation, adapted from Human-Human Trust ✓ ✓ ✓ [53, 61, 111, 181, 209, 239, 244, 245]; [246]”
Hertwig, R., & Ortmann, A. (2003) · Oxford University Press eBooks · doi:10.1093/oso/9780199251063.003.0013 · Cited by 47
Designing a Guardian Angel: Giving an Automated Vehicle the Possibility to Override its Driver
Maurer, S., Erbach, R., Kraiem, I., Kuhnert, S., Grimm, P., & Rukzio, E. (2018) · International Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI) · doi:10.1145/3239060.3239078 · Cited by 46
A Trust Model for an Open, Decentralized Reputation System
Gutscher, A. (2007) · Springer eBooks · doi:10.1007/978-0-387-73655-6_19 · Cited by 44
Dynamics of human trust in recommender systems
Harman, J. L., O’Donovan, J., Abdelzaher, T., & González, C. (2014) · ACM Conference on Recommender Systems (RecSys) · doi:10.1145/2645710.2645761 · Cited by 42
Rhetorical robots: making robots more effective speakers using linguistic cues of expertise
Andrist, S., Spannan, E., & Mutlu, B. (2013) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.5555/2447556.2447678 · Cited by 41
Knowles, B., Rouncefield, M., Harding, M., Davies, N., Blair, L., Hannon, J. T., Walden, J., & Wang, D. (2015) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2675133.2675154 · Cited by 37
Exploratory and Confirmatory Factor Analysis in Gifted Education: Examples with Self-Concept Data
Plucker, J. A. (2003) · journal for the education of the gifted · doi:10.1177/016235320302700103 · Cited by 37
Xiao, J., Stasko, J., & Catrambone, R. (2007) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1240624.1240820 · Cited by 37
ML for UX? - An Inventory and Predictions on the Use of Machine Learning Techniques for UX Research
Chromik, M., Lachner, F., & Butz, A. M. (2020) · Nordic Conference on Human-Computer Interaction · doi:10.1145/3419249.3420163 · Cited by 37
Communicating Uncertainty in Fertility Prognosis
Schneider, H., Wayrauther, J., Hassib, M., & Butz, A. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300391 · Cited by 36
Moderators Of Trust And Reliance Across Multiple Decision Aids
Ross, J. M. (2008) · Journal of International Crisis and Risk Communication Research · Cited by 33
Trust and Reliance Based on System Accuracy
Yu, K., Berkovsky, S., Conway, D., Taib, R., Zhou, J., & Chen, F. (2016) · Proceedings of the 2016 Conference on User Modeling Adaptation and Personalization · doi:10.1145/2930238.2930290 · Cited by 32
Marshall, R. S. (2003) · International Business Review · doi:10.1016/s0969-5931(03)00037-4 · Cited by 30
Who is Generation A? Investigating the Experience of Automated Driving for Different Age Groups
Frison, A.-K., Aigner, L. I., Wintersberger, P., & Riener, A. (2018) · International Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI) · doi:10.1145/3239060.3239087 · Cited by 30
Wendt, E. K., Fridlund, B., & Lidell, E. A.-S. (2004) · Acta Obstetricia Et Gynecologica Scandinavica · doi:10.1111/j.0001-6349.2004.00597.x · Cited by 30
Sun, H., Verbeke, W. J. M. I., Pozharliev, R., Bagozzi, R. P., Babiloni, F., & Wang, L. (2019) · Social Neuroscience · doi:10.1080/17470919.2019.1566171 · Cited by 30
Creating Appropriate Trust for Autonomous Vehicle Systems: A Framework for HMI Design
Ekman, F., Johansson, M., & Sochor, J. (2016) · Chalmers Publication Library (Chalmers University of Technology) · Cited by 29
A field study of human-agent interaction for electricity tariff switching
Alan, A., Costanza, E., Fischer, J. E., Ramchurn, S. D., Rodden, T. A., & Jennings, N. R. (2014) · ePrints Soton (University of Southampton) · doi:10.5555/2615731.2617400 · Cited by 28
van Huysduynen, H. H., Terken, J. M. B., & Eggen, B. H. (2018) · International Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI) · doi:10.1145/3239060.3239063 · Cited by 28
Misplaced Trust: Measuring the Interference of Machine Learning in Human Decision-Making
Suresh, H., Lao, N., & Liccardi, I. (2020) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 28
Cramer, H. S. M., Evers, V., Kemper, N., & Wielinga, B. J. (2008) · IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT) · doi:10.1109/wiiat.2008.326 · Cited by 26
Study of driver trust during cooperation with adaptive cruise control
Rajaonah, B., Anceaux, F., & Vienne, F. (2006) · Le travail humain · doi:10.3917/th.692.0099 · Cited by 25
Personal Influences on Dynamic Trust Formation in Human-Agent Interaction
Huang, H., & Bashir, M. N. (2017) · International Conference on Human-Agent Interaction (HAI) · doi:10.1145/3125739.3125749 · Cited by 25
Fischhoff, B. (2013) · Judgment and Decision Making · doi:10.4324/9780203141939-11 · Cited by 23
Drozdal, J., Weisz, J. D., Wang, D., Dass, G., Yao, B., Zhao, C., Muller, M. J., Lin, J., & Su, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.06509 · Cited by 21
Differences in trust between human and automated decision aids
Pearson, C. J., Welk, A. K., Boettcher, W. A., Mayer, R. C., Streck, S., Simons-Rudolph, J. M., & Mayhorn, C. B. (2016) · Symposium and Bootcamp on Science of Security · doi:10.1145/2898375.2898385 · Cited by 19
Trust Lengthens Decision Time on Unexpected Recommendations in Human-agent Interaction
Tokushige, H., Narumi, T., Ono, S., Fuwamoto, Y., Tanikawa, T., & Hirose, M. (2017) · International Conference on Human-Agent Interaction (HAI) · doi:10.1145/3125739.3125751 · Cited by 17
Fu, E., Johns, M., Hyde, D., Sibi, S., Fischer, M., & Sirkin, D. M. (2020) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376300 · Cited by 17
Where Do You Think You're Going? Characterizing Spatial Mental Models from Planned Routes
Perelman, B. S., Evans, A. W., & Schaefer, K. E. (2020) · ACM Transactions on Human-Robot Interaction · doi:10.1145/3385008 · Cited by 16
Using critical incident technique in trust research
Münscher, R., & Kühlmann, T. M. (2015) · Edward Elgar Publishing eBooks · doi:10.4337/9781782547419.00029 · Cited by 15
Socio‐Cognitive Model of Trust: Basic Ingredients
Castelfranchi, C., & Falcone, R. (2010) · Wiley eBooks · doi:10.1002/9780470519851.ch2 · Cited by 15
Cultivating trust : how do public policies become trusted
Breeman, G. E. (2006) · Leiden Repository (Leiden University) · Cited by 15
Examining Profiles for Robotic Risk Assessment: Does a Robot's Approach to Risk Affect User Trust?
Bridgwater, T., Giuliani, M., van Maris, A., Baker, G., Winfield, A., & Pipe, T. (2020) · Proceedings of the 2020 ACM/IEEE International Conference on Human-Robot Interaction · doi:10.1145/3319502.3374804 · Cited by 14 · cited 5×
““Rank the agents in order of trust” [21].”
InfluentialClustering Behavior to Recognize Subjective Beliefs in Human-Agent Teams
Pynadath, D. V., Wang, N., Rovira, E. M., & Barnes, M. (2018) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/xwaa3999 · Cited by 13
Trust and uncertainty: a critical re-assessment
Lascaux, A. (2008) · International Review of Sociology · doi:10.1080/03906700701823613 · Cited by 13
Li, I., Forlizzi, J., Dey, A. K., & Kiesler, S. B. (2007) · Conference on Designing Pleasurable Products and Interfaces (DPPI) · doi:10.1145/1314161.1314179 · Cited by 13
The Role of Environmental Predictability and Costs in Relying on Automation
Sutherland, S. C., Harteveld, C., & Young, M. E. (2015) · Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702609 · Cited by 12
Hermeneutic Methods in Trust Research
Breeman, G. E. (2011) · Edward Elgar Publishing eBooks · doi:10.4337/9780857932013.00024 · Cited by 11
Deciding when to trust automation in a policy-based city management game
Oduor, K. F., & Campbell, C. S. (2007) · Computer Human Interaction for Management of Information Technology · doi:10.1145/1234772.1234775 · Cited by 10
Effects of Reliance Support on Team Performance by Advising and Adaptive Autonomy
van Maanen, P.-P., Wisse, F., van Diggelen, J., & Beun, R. J. (2011) · IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT) · doi:10.1109/wi-iat.2011.117 · Cited by 10
Do not disturb! trust in decision support systems improves work outcomes under certain conditions
Müller, L. S., Meeßen, S. M., Thielsch, M. T., Nohe, C., Riehle, D. M., & Hertel, G. (2020) · Mensch und Computer (MuC) · doi:10.1145/3404983.3405515 · Cited by 10
Do people believe in Artificial Intelligence?: A cross-topic multicultural study
Kolasinska, A., Lauriola, I., & Quadrio, G. (2019) · Proceedings of the 5th EAI International Conference on Smart Objects and Technologies for Social Good · doi:10.1145/3342428.3342667 · Cited by 9
Can Semi-Anthropomorphism Influence Trust and Compliance?: Exploring Image Use in App Interfaces
Gruber, D., Aune, A., & Koutstaal, W. (2018) · Proceedings of the Technology, Mind, and Society · doi:10.1145/3183654.3183700 · Cited by 8
Ashleigh, M., & Meyer, E. (2015) · Edward Elgar Publishing eBooks · doi:10.4337/9781782547419.00026 · Cited by 8
Research works printed but not linked: 48
- 10.1037/0022--3514.46.3.610 · doi:10.1037/0022--3514.46.3.610
- S. Boon and J. Holmes. 1991. The dynamics of interpersonal trust: resolving uncertainty in the ace of risk. In Cooperation and Prosocial Behaviour R. Hinde and J. Gorebel (Eds.). Cambridge University Press Cambridge 190--211.
- Raja Chatila Virginia Dignum Michael Fisher Fosca Giannotti Katharina Morik Stuart Russell and Karen Yeung. 2021. Trustworthy AI. Springer International Publishing Cham 13--39. https://doi.org/10.1007/978--3-030--69128--8_2 · doi:10.1007/978--3-030--69128--8_2
- 10.1037//0021--9010.87.4.611 · doi:10.1037//0021--9010.87.4.611
- 10.1111/j.1751--9004.2009.00232.x · doi:10.1111/j.1751--9004.2009.00232.x
- 10.1162/JEEA.2009.7.2--3.235 · doi:10.1162/jeea.2009.7.2--3.235
- AXA Research Fund. 2019. Artificial Intelligence: Fostering Trust. Technical Report. AXA. 45 pages. https://www.axa-research.org/en/news/AI-research-guide
- Diego Gambetta. [n.d.]. Can We Trust Trust? Department of Sociology,University of Oxford, Oxford, United Kingdom.
- Nicole Gillespie. 2003. Measuring trust in working relationships: The behavioral trust inventory. Melbourne Business School, Melbourne, Australia.
- 10.1108/00483480610682299 · DOI · Paper page
- 10.1007/978-0--387--73655--6_19 · doi:10.1007/978-0--387--73655--6_19
- 10.1037//0022--3514.49.1.95 · doi:10.1037//0022--3514.49.1.95
- 10.1038/s42256-019-0088--2 · doi:10.1038/s42256-019-0088--2
- 10.1016/j.joep.2011.05.00 · doi:10.1016/j.joep.2011.05.00
- 10.1002/j.2333--8504.1967.tb00991.x · doi:10.1002/j.2333--8504.1967.tb00991.x
- 10.1016/j.obhdp.2012.08.0 · doi:10.1016/j.obhdp.2012.08.0
- KPMG. 2019. Controlling AI: The imperative for transparency and explainability. Technical Report. KPMG. 28 pages. https://advisory.kpmg.us/articles/2019/controlling-ai.html
- 10.1518/hfes.46.1.50.30392 · doi:10.1518/hfes.46.1.50.30392 · Paper page
- 10.2307/259288 · DOI · Paper page
- 10.1007/978--94-010--1276--8_19 · doi:10.1007/978--94-010--1276--8_19
- Niklas Luhmann. 1979. Trust and Power (1 ed.). Wiley, Chichester, Toronto.
- Nikolas Luhmann. 2000. Familiarity, Confidence, Trust: Problems and Alternatives. In Trust: Making and Breaking Cooperative Relations, Diego Gambetta (Ed.). Basil Blackwell, Oxford, United Kingdom, 94--107.
- Maria A. Madsen and Shirley Gregor. 2000. Measuring Human-Computer Trust. In Proceedings of the 11 th Australasian Conference on Information Systems. Australasian Conference on Information Systems (ACIS), Brisbane, Australia, 6--8.
- C. Mantzavinos. 2020. Hermeneutics. In The Stanford Encyclopedia of Philosophy (spring 2020 ed.), Edward N. Zalta (Ed.). Metaphysics Research Lab, Stanford University, Stanford, CA, USA.
- 10.1016/S0969--5931(03)00037--4 · doi:10.1016/s0969--5931(03)00037--4
- 10.1037/0021--9010.84.1.123 · doi:10.1037/0021--9010.84.1.123
- 10.1007/3--540--45547--7_3 · doi:10.1007/3--540--45547--7_3
- 10.5465/amr.1998.926622 · DOI · Paper page
- Joachim Meyer and John D. Lee. 2013. Trust, Reliance, and Compliance. In The Oxford Handbook of Cognitive Engineering, John D. Lee and Alex Kirlik (Eds.). Oxford University Press, Oxford, UK, 1--29. https://www.oxfordhandbooks.com/view/10.1093/oxfordhb/9780199757183.001.0001/oxfordhb-9780199757183-e-6 · doi:10.1093/oxfordhb/9780199757183.001.0001/oxfordhb-9780199757183-e-6
- B.M. Muir. 1989. Operators' Trust in and Use of Automatic Controllers in a Supervisory Process Control Task. University of Toronto, Toronto, Canada. https://books.google.fr/books?id=T94NSwAACAAJ
- Domen Novak. 2014. Engineering Issues in Physiological Computing. 17--38. https://doi.org/10.1007/978--1--4471--6392--3_2 · doi:10.1007/978--1--4471--6392--3_2
- Claus Offe. [n.d.]. How can we trust our fellow citizens? Cambridge UP, Cambridge, United Kingdom.
- 10.1111/2041--210X.12860 · doi:10.1111/2041--210x.12860
- 10.5214/ans.0972--7531.1017309 · doi:10.5214/ans.0972--7531.1017309
- 10.1111/j.1744--6198.2011.00230.x · doi:10.1111/j.1744--6198.2011.00230.x
- Gjalt-Jorn Peters. 2014. The alpha and the omega of scale reliability and validity: Why and how to abandon Conbach's alpha and the route towards more comprehensive assessment od scale quality. Euro Health Psychologist 16 (01 2014), 56--69.
- J. Potter and D. Edwards. 1996. Discourse Analysis. Macmillan Education UK London 419--425. https://doi.org/10.1007/978--1--349--24483--6_63 · doi:10.1007/978--1--349--24483--6_63
- 10.1016/B978-0--12--800985--7.00007--7 · doi:10.1016/b978-0--12--800985--7.00007--7
- Accenture Federal Services. 2019. Responsible AI: A Framework for Building Trust in your AI Solutions. Technical Report. Accenture. 13 pages. https://www.accenture.com/us-en/insights/us-federal-government/ai-is-ready-are-we
- 10.1007/s11336-008--9101-0 · doi:10.1007/s11336-008--9101-0
- Richard S. Sutton and Andrew G. Barto. 1998. Introduction to Reinforcement Learning (1st ed.). MIT Press, Cambridge, MA, USA.
- 10.1111/j.1758--6623.2005.tb00554.x · doi:10.1111/j.1758--6623.2005.tb00554.x
- M. Wang, A. Hussein, R. F. Rojas, K. Shafi, and H. A. Abbass. 2018. EEG-Based Neural Correlates of Trust in Human-Autonomy Interaction. In 2018 IEEE Symposium Series on Computational Intelligence (SSCI). IEEE, Bangalore, India, 350--357.
- Ning Wang, David V. Pynadath, and Susan G. Hill. 2016. Trust Calibration within a Human-Robot Team: Comparing Automatically Generated Explanations. In The Eleventh ACM/IEEE International Conference on Human Robot Interaction (HRI '16). IEEE Press, New York, NY, USA, 109--116.
- 10.1111/j.1468--2958.1977.tb00523.x · doi:10.1111/j.1468--2958.1977.tb00523.x
- T. Whelan. 2008. Social Presence in Multi-User Virtual Environments : A Review and Measurement Framework for Organizational Research.
- 10.5555/3378680.3378689 · doi:10.5555/3378680.3378689
- Louise C. Young and Gerald S. Albaum. 2002. Developing a measure of trust in retail relationships : a direct selling application. School of Marketing, University of Technology of Sydney, Sydney Broadway, N.S.W, Australia.
Cited by
Nazaretsky, T., Ariely, M., Cukurova, M., & Alexandron, G. (2022) · British Journal of Educational Technology · doi:10.1111/bjet.13232 · Cited by 589
Design Principles for Generative AI Applications
Weisz, J. D., He, J., Muller, M., Hoefer, G., Miles, R., & Geyer, W. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642466 · Cited by 232
Tun, H. M., Rahman, H. A., Naing, L., & Malik, O. A. (2025) · Journal of Medical Internet Research · doi:10.2196/69678 · Cited by 210
“Vereschak et al [10] highlight the importance of integrating theoretical elements of trust, such as vulnerability, positive expectations, and attitude, into the understanding of human-AI trust.”
Gomez, C., Cho, S. M., Ke, S., Huang, C.-M., & Unberath, M. (2025) · Frontiers in Computer Science · doi:10.3389/fcomp.2024.1521066 · Cited by 209
Lai, V., Chen, C., Smith-Renner, A., Liao, Q. V., & Tan, C. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594087 · Cited by 204
Metadata
- Authors
- Vereschak, O., Bailly, G., & Caramiaux, B.
- Affiliations
- Sorbonne Université (FR)
- Centre National de la Recherche Scientifique (FR)
- Venue
- Proceedings of the ACM on Human-Computer Interaction
- Published
- 2021-10-13
- Volume
- 5
- Issue
- CSCW2
- Pages
- 1-39
- ISSN
- 2573-0142 (ISSN-L)
- OpenAlex
- W3207598588
- Cited by
- 229
- References
- 215
- Citations by year
- 2026: 58 · 2025: 61 · 2024: 64 · 2023: 31 · 2022: 15
- FWCI
- 26.95
- Type
- journal article
- Language
- en
- Open access
- Locations
- Proceedings of the ACM on Human-Computer Interaction (publishedVersion)
- HAL (submittedVersion, other-oa)
- Funders
- ANR-11-LABX-65 (ANR-11-IDEX-0004-02)
- Topics
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Keywords
- trust in AI, AI-assisted decision-making, experimental protocol, trust measurement, trust definition, research methodology guidelines, trust factors
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.