A Review of Trust in Artificial Intelligence: Challenges, Vulnerabilities and Future Directions
Open accessThe DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
Abstract
Artificial Intelligence (AI) can benefit society, but it is also fraught with risks. Societal adoption of AI is recognized to depend on stakeholder trust in AI, yet the literature on trust in AI is fragmented, and little is known about the vulnerabilities faced by different stakeholders, making it is difficult to draw on this evidence-base to inform practice and policy. We undertake a literature review to take stock of what is known about the antecedents of trust in AI, and organize our findings around five trust challenges unique to or exacerbated by AI. Further, we develop a concept matrix identifying the key vulnerabilities to stakeholders raised by each of the challenges, and propose a multi-stakeholder approach to future research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
An integrative model of organizational trust
Mayer, R. C., Davis, J. H., & Schoorman, F. D. (1995) · The Academy of Management Review · doi:10.2307/258792 · Cited by 10,736
Not So Different After All: A Cross-Discipline View Of Trust
Rousseau, D. M., Sitkin, S. B., Burt, R. S., & Camerer, C. (1998) · Academy of Management Review · doi:10.5465/amr.1998.926617 · Cited by 10,274
Machine learning: Trends, perspectives, and prospects
Jordan, M. I., & Mitchell, T. M. (2015) · Science · doi:10.1126/science.aaa8415 · Cited by 9,955
Trust and TAM in Online Shopping: An Integrated Model1
Gefen, D., Karahanna, E., & Straub, D. W. (2003) · MIS Quarterly · doi:10.2307/30036519 · Cited by 8,635
“Why Should I Trust You?”: Explaining the Predictions of Any Classifier
Ribeiro, M., Singh, S., & Guestrin, C. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations · doi:10.18653/v1/n16-3020 · Cited by 6,682
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294
Dwivedi, Y. K., Hughes, L., Ismagilova, E., Aarts, G., Coombs, C., Crick, T., Duan, Y., Dwivedi, R., Edwards, J., Eirug, A., et al. (2021) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2019.08.002 · Cited by 4,611 · cited 1×
“However, they also change the nature of how IT artifacts are designed and work [26], their capacity for autonomous functioning, creating risks, challenges and uncertainties [27] not inherent in traditional technologies.”
BackgroundLewis, J. D., & Weigert, A. (1985) · Social Forces · doi:10.2307/2578601 · Cited by 4,163 · cited 2×
“Furthermore empirical studies have tended to examine whether a concept (such as accuracy or anthropomorphism) enhances trust, yet high trust is not always appropriate, and encouraging people to trust without ‘good reasons’ [33] can be manipulative.”
BackgroundBig other: Surveillance Capitalism and the Prospects of an Information Civilization
Zuboff, S. (2015) · Journal of Information Technology · doi:10.1057/jit.2015.5 · Cited by 3,205 · cited 1×
“Data extraction is fundamentally different from previous forms of market exchange, as it connotes a one-way taking of data rather than a consensual or reciprocal process [82].”
BackgroundHuman trust in artificial intelligence: Review of empirical research
Glikson, E., & Woolley, A. W. (2020) · Academy of Management Annals · doi:10.5465/annals.2018.0057 · Cited by 2,921 · cited 1×
“A recent review of the empirical literature suggests that AI representation plays an important role in the development of trust [15] and differentially impacts trust over time; for robotic AI, trust tends to start low and increase over time, but for virtual and embedded AI the opposite commonly occurs.”
BackgroundArtificial Intelligence and Management: The Automation–Augmentation Paradox
Raisch, S., & Krakowski, S. (2021) · Academy of Management Review · doi:10.5465/amr.2018.0072 · Cited by 2,755
Algorithm appreciation: People prefer algorithmic to human judgment
Logg, J. M., Minson, J. A., & Moore, D. A. (2019) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2018.12.005 · Cited by 2,061
A Meta-Analysis of Factors Affecting Trust in Human-Robot Interaction
Hancock, P. A., Billings, D. R., Schaefer, K. E., Chen, J. Y. C., de Visser, E. J., & Parasuraman, R. (2011) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720811417254 · Cited by 2,051 · cited 1×
“antecedents to trust in specific applications of AI, such as human-robot interaction [19] and automation [20], but have not taken into account human trust in AI more broadly.”
BackgroundResistance to Medical Artificial Intelligence
Longoni, C., Bonezzi, A., & Morewedge, C. K. (2019) · Journal of Consumer Research · doi:10.1093/jcr/ucz013 · Cited by 2,015 · cited 1×
“human provider’s decision, rather than replacing the human [64].”
BackgroundThe forthcoming artificial intelligence (AI) revolution: Its impact on society and firms
Makridakis, S. (2017) · Futures · doi:10.1016/j.futures.2017.03.006 · Cited by 1,866 · cited 1×
“While trust has been shown to be important for the adoption of a range of technologies [3], AI creates an array of qualitatively different trust challenges compared to more traditional information technologies [4].”
BackgroundThe Effects of Personalization and Familiarity on Trust and Adoption of Recommendation Agents1
Komiak, S. Y. X., & Benbasat, I. (2006) · MIS Quarterly · doi:10.2307/25148760 · Cited by 1,374
Rahwan, I., Cebrian, M., Obradovich, N., Bongard, J., Bonnefon, J.-F., Breazeal, C., Crandall, J. W., Christakis, N. A., Couzin, I. D., Jackson, M. O., et al. (2019) · Nature · doi:10.1038/s41586-019-1138-y · Cited by 1,364
Kokolakis, S. (2017) · Computers & Security · doi:10.1016/j.cose.2015.07.002 · Cited by 1,229
At What Level (and in Whom) We Trust: Trust Across Multiple Organizational Levels
Fulmer, C. A., & Gelfand, M. J. (2012) · Journal of Management · doi:10.1177/0149206312439327 · Cited by 886
Qiu, L., & Benbasat, I. (2009) · Journal of Management Information Systems · doi:10.2753/mis0742-1222250405 · Cited by 737
Langer, E. J., Blank, A., & Chanowitz, B. (1978) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.36.6.635 · Cited by 694
What do we need to build explainable AI systems for the medical domain?
Holzinger, A., Biemann, C., Pattichis, C. S., & Kell, D. B. (2017) · arXiv · doi:10.48550/arxiv.1712.09923 · Cited by 636 · cited 1×
“explainable, while the ones most able to provide the clearest explanations are the least accurate [39].”
BackgroundViewpoint: when will AI exceed human performance? evidence from AI experts
Grace, K., Salvatier, J., Dafoe, A., Zhang, B., & Evans, O. (2018) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.11222 · Cited by 579
Understanding the Effect of Accuracy on Trust in Machine Learning Models
Yin, M., Wortman Vaughan, J., & Wallach, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300509 · Cited by 527
Ethical governance is essential to building trust in robotics and artificial intelligence systems
Winfield, A. F. T., & Jirotka, M. (2018) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2018.0085 · Cited by 493
Artificial Intelligence: American Attitudes and Trends
Zhang, B., & Dafoe, A. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3312874 · Cited by 464
Salles, A., Evers, K., & Farisco, M. (2020) · AJOB Neuroscience · doi:10.1080/21507740.2020.1740350 · Cited by 335 · cited 1×
“anthropomorphism may lead to overestimation of the AI’s capabilities, potentially putting the stakeholder at risk [73], damaging trust [74], and leading to a host of ethical and psychological concerns, including manipulation [75].”
BackgroundThe effect of presence on human-robot interaction
Bainbridge, W. A., Hart, J., Kim, E. S., & Scassellati, B. (2008) · RO-MAN 2008 - The 17th IEEE International Symposium on Robot and Human Interactive Communication · doi:10.1109/roman.2008.4600749 · Cited by 300 · cited 1×
“having a physical form) also increases user trust in the agent, primarily through perceptions of its social presence [9, 77, 78].”
BackgroundWhen Will AI Exceed Human Performance? Evidence from AI Experts
Grace, K., Salvatier, J., Dafoe, A., Zhang, B., & Evans, O. (2017) · arXiv · doi:10.48550/arxiv.1705.08807 · Cited by 285 · cited 1×
“Artificial Intelligence (AI) is an increasingly ubiquitous aspect of modern life that has had a transformative impact on how we live and work [1].”
BackgroundBig data with cognitive computing: A review for the future
Gupta, S., Kar, A. K., Baabdullah, A., & Al-Khowaiter, W. A. A. (2018) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2018.06.005 · Cited by 285
Kim, K., Boelling, L., Haesler, S., Bailenson, J., Bruder, G., & Welch, G. F. (2018) · 2018 IEEE International Symposium on Mixed and Augmented Reality (ISMAR) · doi:10.1109/ismar.2018.00039 · Cited by 206
Trusting Intelligent Machines: Deepening Trust Within Socio-Technical Systems
Andras, P., Esterle, L., Guckert, M., Han, T. A., Lewis, P. R., Milanovic, K., Payne, T., Perret, C., Pitt, J., Powers, S. T., et al. (2018) · IEEE Technology and Society Magazine · doi:10.1109/mts.2018.2876107 · Cited by 153
Kroll, J. A. (2018) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2018.0084 · Cited by 132 · cited 3×
“However, explanations may actually cause overtrust [45] and can be manipulative [46].”
BackgroundDefining AI in Policy versus Practice
Krafft, P. M., Young, M., Katell, M., Huang, K., & Bugingo, G. (2020) · Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3375627.3375835 · Cited by 129 · cited 1×
“We adopt the OECD's [21] definition of AI, as recently recommended by AI experts [22]: "a machine-based system that can, for a given set of human-defined objectives, make predictions, recommendations, or decisions influencing real or virtual environments.”
MethodReview and Theory Symbiosis: An Introspective Retrospective
Baylor University and Lund University, & Leidner, D. (2018) · Journal of the Association for Information Systems · doi:10.17705/1jais.00501 · Cited by 102
Trusting a Virtual Driver That Looks, Acts, and Thinks Like You
Verberne, F. M. F., Ham, J., & Midden, C. J. H. (2015) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720815580749 · Cited by 101
Trust and Transparency in Machine Learning-Based Clinical Decision Support
Gretton, C. (2018) · Human–Computer Interaction Series · doi:10.1007/978-3-319-90403-0_14 · Cited by 27
Assessing Demand for Transparency in Intelligent Systems Using Machine Learning
Vorm, E. S. (2018) · 2018 Innovations in Intelligent Systems and Applications (INISTA) · doi:10.1109/inista.2018.8466328 · Cited by 25
Cited by
Trust in AI and Its Role in the Acceptance of AI Technologies
Choung, H., David, P., & Ross, A. (2023) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2022.2050543 · Cited by 963
“AI is ubiquitous in modern life, enabling smart technologies and applications, such as smart home devices, fitness trackers, autonomous driving systems, and social media platforms (Gorwa et al., 2020; Lockey et al., 2021).”
BackgroundInfluentialTrust in AI: progress, challenges, and future directions
Afroogh, S., Akbari, A., Malone, E., Kargar, M., & Alambeigi, H. (2024) · Humanities and Social Sciences Communications · doi:10.1057/s41599-024-04044-8 · Cited by 507
To trust or not to trust? An assessment of trust in AI-based systems: Concerns, ethics and contexts
Omrani, N., Rivieccio, G., Fiore, U., Schiavone, F., & Agreda, S. G. (2022) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2022.121763 · Cited by 269
How can we manage biases in artificial intelligence systems – A systematic literature review
P.S., D. V. (2023) · International Journal of Information Management Data Insights · doi:10.1016/j.jjimei.2023.100165 · Cited by 239
Students' perceptions of using ChatGPT in a physics class as a virtual tutor
Ding, L., Li, T., Jiang, S., & Gapud, A. (2023) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-023-00434-1 · Cited by 208
“Given the language power of GenAI (particularly ChatGPT) and the lack of transparency regarding how GenAI works, people without AI literacy training are more likely to blindly trust incorrect information provided by AI systems (Lockey et al., 2021).”
Background
Metadata
- Authors
- Lockey, S., Gillespie, N., Holm, D., & Someh, I. A.
- Venue
- Proceedings of the Annual Hawaii International Conference on System Sciences
- Published
- 2021
- ISSN
- 1530-1605 (ISSN-L)
- 2572-6862
- OpenAlex
- W3127461705
- Cited by
- 205
- References
- 38
- FWCI
- 25.54
- Type
- conference paper
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.