Formalizing Trust in Artificial Intelligence: Prerequisites, Causes and Goals of Human Trust in AI
The DOI matches its Crossref record; 74 of the 76 references it cites were found in a registry or an index, and 2 are not checked yet.
Abstract
Trust is a central component of the interaction between people and AI, in that 'incorrect' levels of trust may cause misuse, abuse or disuse of the technology. But what, precisely, is the nature of trust in AI? What are the prerequisites and goals of the cognitive mechanism of trust, and how can we promote them, or assess whether they are being satisfied in a given interaction? This work aims to answer these questions. We discuss a model of trust inspired by, but not identical to, interpersonal trust (i.e., trust between people) as defined by sociologists. This model rests on two key properties: the vulnerability of the user; and the ability to anticipate the impact of the AI model's decisions. We incorporate a formalization of 'contractual trust', such that trust between a user and an AI model is trust that some implicit or explicit contract will hold, and a formalization of 'trustworthiness' (that detaches from the notion of trustworthiness in sociology), and with it concepts of 'warranted' and 'unwarranted' trust. We present the possible causes of warranted trust as intrinsic reasoning and extrinsic behavior, and discuss how to design trustworthy AI, how to evaluate whether trust has manifested, and whether it is warranted. Finally, we elucidate the connection between trust and XAI using our formalization.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Uncertainty is a defining characteristic of trust (Mayer et al., 1995) and has been referred to as a necessary prerequisite of human trust in AI that has been lacking in current XAI studies (Jacovi et al., 2021).”
From Exploring the effects of human-centered AI explanations on trust and reliance (2023) · cites this paper as Jacovi et al., 2021
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“research found that people often ignore recommendations because they do not trust them”
From Explainable AI is Dead, Long Live Explainable AI!: Hypothesis-driven Decision Support using Evaluative AI (2023) · cites this paper as [13]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The primary motivation for clinical XAI is fostering appropriate trust: enabling clinicians to calibrate their reliance on AI recommendations based on the quality of supporting reasoning [156,157].”
From Explainability of decoder-only clinical large language models: A scoping review (2026) · cites this paper as [157]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“However, people find it difficult to correctly calibrate their trust in a decision aid [10,13,21,37].”
From Explainable AI is Dead, Long Live Explainable AI!: Hypothesis-driven Decision Support using Evaluative AI (2023) · cites this paper as [13]
The cited source says what the claim attributes to it.
In the citing paper
“For example, [38] formalizes human-AI trust by considering and extending the concepts of interpersonal trust.”
From Trustworthy AI: From Principles to Practices (2023) · cites this paper as [38]
9 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
An integrative model of organizational trust
Mayer, R. C., Davis, J. H., & Schoorman, F. D. (1995) · The Academy of Management Review · doi:10.2307/258792 · Cited by 10,736 · cited 1×
“on basic definition of trust regards it as a directional transaction between two parties: if A believes that B will act in A’s best interest, and accepts vulnerability to B’s actions, then A trusts B [49]. The goal of trust is to “make social life predictable [by anticipating the impact of behavior], and make it easie…”
BackgroundTrust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605 · cited 2×
“ay that the trust is warranted if it is the result of trustworthiness, and otherwise it is unwarranted [50]. Warranted trust is sometimes referred to as trust which is calibrated with trustworthiness [43]. In other words, trust is the cognitive mechanism to give the ‘illusion’ of anticipating intended behavior, which…”
BackgroundLipton, Z. C. (2018) · Queue · doi:10.1145/3236386.3241340 · Cited by 5,427 · cited 2×
“s can reveal strong correlations between attributes for one of those classes, giving leads to research on causation between them, even if the model was not useful for the original classification task [42, 46]. 2Assume that the performance evaluation is representative of real usage for now, although this is an importan…”
BackgroundInfluentialDeep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
Simonyan, K., Vedaldi, A., & Zisserman, A. (2013) · arXiv · doi:10.48550/arxiv.1312.6034 · Cited by 5,370 · cited 1×
“Transparency High-standard documentation All Technical explainability Factsheets (explainability) Saliency maps [61], self-attention patterns [40], influence functions [38], probing [16] Adaptable user-centered explainability Factsheets (explainability) Counterfactual [21], contrastive [51], natural language [27], by-…”
BackgroundExplanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302
Self-citationHumans and automation: Use, misuse, disuse, abuse
Parasuraman, R., & Riley, V. (1997) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872097778543886 · Cited by 4,554 · cited 2×
“8We consider AI as an automation attributed with intent, and thus, Human-AI trust is a sub-case of human-machine trust [28, 55].”
BackgroundLewis, J. D., & Weigert, A. (1985) · Social Forces · doi:10.2307/2578601 · Cited by 4,163 · cited 1×
“the influence of the trustee’s incentives on Human-AI trust, should build on the existing research in sociology that investigates the influence of relationships and communities on interpersonal trust [44]. Consequently, the question of whether this trust is warranted or unwarranted is ill-defined.Our definition of war…”
BackgroundA large annotated corpus for learning natural language inference
Bowman, S. R., Angeli, G., Potts, C., & Manning, C. D. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1075 · Cited by 3,377 · cited 1×
“We set the “effort boundary” around 774 tokens—a few of minutes of reading—and the “vulnerability boundary” right from natural language inference [7], commonsense QA [70], and multi-sentence”
BackgroundModel Cards for Model Reporting
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287596 · Cited by 2,681
Baier, A. (1986) · Ethics · doi:10.1086/292745 · Cited by 2,567 · cited 1×
“present two directions for extension of our formalization inspired by other factors of interpersonal trust. 9.1 Other Elements of Interpersonal Trust and Trust in Automation Trust vs Reliance. Baier [4] introduces the term reliance for trust in inanimate objects, such as trusting a stool to hold your weight. We may fe…”
BackgroundInfluential
Research works printed but not linked: 27
- [1] David Alvarez Melis and Tommi Jaakkola. 2018. Towards Robust Interpretability with Self-Explaining Neural Networks. In Advances in Neural Information Processing Systems 31 , S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. CesaBianchi, and R. Garnett (Eds.). 7775-7784. http://papers.nips.cc/paper/8003towards-robust-interpretability-with-self-explaining-neural-networks.pdf -
- 10.18653/v1 · doi:10.18653/v1
- 10.18653/v1 · doi:10.18653/v1
- 10.18653/v1 · doi:10.18653/v1
- 10.18653/v1 · doi:10.18653/v1
- 10.18653/v1 · doi:10.18653/v1
- [15] Finale Doshi-Velez and Been Kim. 2017. Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608 (2017). - · arXiv
- 10.18653/v1 · doi:10.18653/v1
- [21] Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. 2015. Explaining and Harnessing Adversarial Examples. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings , Yoshua Bengio and Yann LeCun (Eds.). http://arxiv.org/abs/1412.6572 - · arXiv
- [22] Yash Goyal, Ziyan Wu, Jan Ernst, Dhruv Batra, Devi Parikh, and Stefan Lee. 2019. Counterfactual Visual Explanations (Proceedings of Machine Learning Research, Vol. 97) , Kamalika Chaudhuri and Ruslan Salakhutdinov (Eds.). PMLR, Long Beach, California, USA, 2376-2384. http://proceedings.mlr.press/v97/goyal19a.html -
- 10.18653/v1 · doi:10.18653/v1
- [25] Sven Ove Hansson. 2018. Risk. In The Stanford Encyclopedia of Philosophy (fall 2018 ed.), Edward N. Zalta (Ed.). Metaphysics Research Lab, Stanford University. -
- [29] Robert R Hoffman. 2017. A taxonomy of emergent trusting in the human-machine relationship. Cognitive systems engineering: The future for a changing world (2017), 137-163. -
- 10.18653/v1 · doi:10.18653/v1
- 10.18653/v1 · doi:10.18653/v1
- 10.18653/v1 · doi:10.18653/v1
- [40] Hana Kopecka and Jose M Such. 2020. Explainable AI for Cultural Minds. https://sites.google.com/view/dexahai-at-ecai2020/home Workshop on Dialogue, Explanation and Argumentation for Human-Agent Interaction , DEXAHAI ; Conference date: 07-09-2020. -
- 10.18653/v1 · doi:10.18653/v1
- 10.18653/v1 · doi:10.18653/v1
- 10.18653/v1 · doi:10.18653/v1
- [53] Carolyn McLeod. 2015. Trust. In The Stanford Encyclopedia of Philosophy (fall 2015 ed.), Edward N. Zalta (Ed.). Metaphysics Research Lab, Stanford University. -
- 10.18653/v1 · doi:10.18653/v1
- 10.18653/v1 · doi:10.18653/v1
- [64] Philipp Schmidt and Felix Bießmann. 2019. Quantifying Interpretability and Trust in Machine Learning Systems. CoRR abs/1901.08558 (2019). arXiv:1901.08558 http://arxiv.org/abs/1901.08558 - · arXiv
- [67] Anders Søgaard, Sebastian Ebert, Jasmijn Bastings, and Katja Filippova. 2020. We Need to Talk About Random Splits. CoRR abs/2005.00636 (2020). arXiv:2005.00636 https://arxiv.org/abs/2005.00636 - · arXiv
- 10.18653/v1 · doi:10.18653/v1
- 10.18653/v1 · doi:10.18653/v1
Cited by
Items 201–300 of 747
Trust and Reliance in XAI -- Distinguishing Between Attitudinal and Behavioral Measures
Scharowski, N., Perrig, S. A. C., von Felten, N., & Brühlmann, F. (2022) · arXiv · doi:10.48550/arxiv.2203.12318 · Cited by 18
“[19] argued that using the term trust is appropriate when attributing AI with intent and that in the case of non-anthropomorphized AI, the term reliance should be considered.”
BackgroundInfluentialAnthropomimetic Uncertainty: What Verbalized Uncertainty in Language Models is Missing
Ulmer, D., Lorson, A., Titov, I., & Hardmeier, C. (2026) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl.a.739 · Cited by 22
Cai, J., Fu, X., Gu, Z., & Wu, R. (2023) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2023.2212224 · Cited by 20
“Existing evidence showed that interpersonal trust and trust in AI contained the same ingredients, which were anticipation and vulnerability (Jacovi et al., 2021).”
InfluentialCausalABSC: Causal Inference for Aspect Debiasing in Aspect-Based Sentiment Classification
Zhou, J., Lin, Y., Chen, Q., Zhang, Q., Huang, X., & He, L. (2023) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2023.3340606 · Cited by 24
“Despite the existing neural-based models having obtained state-of-the-art results, these models may be untrustworthy [32].”
BackgroundAliferis, C., & Simon, G. (2024) · Health informatics · doi:10.1007/978-3-031-39355-6_1 · Cited by 22
How Being Outvoted by AI Teammates Impacts Human-AI Collaboration
Hu, M., Zhang, G., Chong, L., Cagan, J., & Goucher-Lambert, K. (2024) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2024.2345980 · Cited by 27
“For example, Jacovi et al. (2021) explored the mental mechanism of trust in AI and the features of trust-worthy AI.”
Xuan, Y., Small, E. A., Sokol, K., Hettiachchi, D., & Sanderson, M. (2024) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2024.103376 · Cited by 20
Development and validation of the trust in AI scale (TAIS)
Wischnewski, M., Doebler, P., & Krämer, N. C. (2026) · Journal of Psychology and AI · doi:10.1080/29974100.2026.2715475 · Cited by 19
What Guides Our Choices? Modeling Developers' Trust and Behavioral Intentions Towards Genai
Choudhuri, R., Trinkenreich, B., Pandita, R., Kalliamvakou, E., Steinmacher, I., Gerosa, M., Sanchez, C., & Sarma, A. (2025) · 2025 IEEE/ACM 47th International Conference on Software Engineering (ICSE) · doi:10.1109/icse55347.2025.00087 · Cited by 17
“This clarity allows individuals to form accurate expectations about system quality (H1) within their task contexts and assess its functional value (H2), thereby fostering appropriate trust [43].”
InfluentialThe AI transparency dilemma: when more is less for trust and adoption
Ngo, V. M. (2025) · Information Discovery and Delivery · doi:10.1108/idd-03-2025-0056 · Cited by 23
Buhr, E., Onder, O., Rudra, P., & Ursin, F. (2025) · Ethics and Information Technology · doi:10.1007/s10676-025-09867-w · Cited by 18
“There is a danger that trust will not only erode, but turn into distrust, which describes not merely the absence of trust, but the expectation of harm, failure, or outcomes contrary to one’s interests (Jacovi et al., 2021; Tallant, 2017).”
Justifying Our Credences in the Trustworthiness of AI Systems: A Reliabilistic Approach
Ferrario, A. (2024) · Science and Engineering Ethics · doi:10.1007/s11948-024-00522-z · Cited by 24
“The necessary contextualization of these principle-based properties is provided by the now influential ‘contractual trust’ approach in human-AI interactions (Jacovi et al., 2021).”
InfluentialZhang, Y., Guo, J., Wang, Y., Li, S., Yang, Q., Zhang, J., & Lu, Z. (2025) · Systems · doi:10.3390/systems13100855 · Cited by 16
Towards Aligning Personalized AI Agents with Users' Privacy Preference
Zhang, S., Ma, Y., Chen, J., Li, S., Yi, X., & Li, H. (2025) · doi:10.1145/3733816.3760752 · Cited by 20
Ulmer, D., Frellsen, J., & Hardmeier, C. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.198 · Cited by 46
Ala-Luopa, S., Olsson, T., Väänänen, K., Hartikainen, M., & Makkonen, J. (2024) · Computer Supported Cooperative Work (CSCW) · doi:10.1007/s10606-024-09499-6 · Cited by 18
“This has raised discussion on trustworthy AI, referring to responsible, reliable, and ethical AI advancement, which is expected to necessitate the involvement of people using AI or affected by it (Jacovi et al. 2021; Zicari et al. 2021; Ashoori and Weisz 2019).”
Bakonyi, Z. (2024) · Journal of Organizational Change Management · doi:10.1108/jocm-01-2023-0026 · Cited by 18
Zhai, X., Jiang, J., Dejl, A., Rago, A., Guo, F., Toni, F., & Sivakumar, A. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103057 · Cited by 21
“However, explainability often represents a trade-off with accuracy [28] and trust-worthiness has been shown to be a complex phenomenon which is affected by explainability in many ways [29, 30, 31, 32, 33].”
Wang, H., Zhang, Y., & Lu, T. (2025) · Management Science · doi:10.1287/mnsc.2022.03844 · Cited by 27
It’s the Meaning That Counts: The State of the Art in NLP and Semantics
Hershcovich, D., & Donatelli, L. E. (2021) · Künstliche Intell. · doi:10.1007/s13218-021-00726-6 · Cited by 17
“Explainable Artificial Intelligence (XAI), an emerging field in the broader AI community, is important for facilitating human trust in AI systems such as deep learning models [86, 192], which are increasingly widespread but tend to be opaque with regard to their decision making.”
BackgroundDDoD: Dual Denial of Decision Attacks on Human-AI Teams
Tag, B., van Berkel, N., Verma, S., Zhao, B. Z. H., Berkovsky, S., Kaafar, D., Kostakos, V., & Ohrimenko, O. (2023) · IEEE Pervasive Computing · doi:10.1109/mprv.2022.3218773 · Cited by 15
“A core component in building trust is the anticipation of intentional behavior in an entity, and the ability to predict (parts of) the decisions made by this entity and their impact [20].”
BackgroundThe ethical issues in generative artificial intelligence: A systematic review
TIRPAN, E. C. (2024) · Business And Management Studies An International Journal · doi:10.15295/bmij.v12i4.2431 · Cited by 20
The Amplifying Effect of Explainability in AI-assisted Decision-making in Groups
Duarte, R. D. B., Abreu, M. C., Campos, J., & Paiva, A. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713534 · Cited by 19
“Without such an element, evaluating AI trust and reliance could be less effective [23, 38].”
The Alchemy of Trust: The Creative Act of Designing Trustworthy Socio-Technical Systems
Thornton, L., Knowles, B., & Blair, G. S. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533196 · Cited by 20
Cherry on Top or Real Need? A Review of Explainable Machine Learning in Kidney Transplantation
de Souza, A. A., Stubbs, A., Hesselink, D. A., Baan, C. C., & Boer, K. (2024) · Transplantation · doi:10.1097/tp.0000000000005063 · Cited by 15
Salimzadeh, S., & Gadiraju, U. (2024) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3627043.3659567 · Cited by 19
Ahn, Y. H., Kim, H. B., & Kim, S. T. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01043 · Cited by 25
“This lack of clarity raises concerns about the stability and reliability of these models, leading to a growing consensus that artificial intelligence should be reliable, robust, and safe [4, 13].”
den Bosch, K. V., van Diggelen, J., Verdult, S., Haije, T., & van der Waa, J. (2025) · AI and Ethics · doi:10.1007/s43681-024-00647-8 · Cited by 16
“This notion considers trust as a ’con - tract’ between humans and AI whereby the human requires the AI to behave in a specific way or to perform a specific action in specific circumstances [ 48].”
AI‐Based Crypto Tokens: The Illusion of Decentralized AI?
Mafrur, R. (2025) · IET Blockchain · doi:10.1049/blc2.70015 · Cited by 18
Artificial Intelligence Project Success Factors: Moral Decision-Making with Algorithms
Miller, G. J. (2021) · Annals of Computer Science and Information Systems · doi:10.15439/2021f26 · Cited by 24
Ünver, M. B. (2023) · International Review of Law Computers & Technology · doi:10.1080/13600869.2023.2192569 · Cited by 20
The Dynamics of Trust: Path Dependence in Interpersonal Trust
Caddell, J., & Nilchiani, R. (2023) · IEEE Engineering Management Review · doi:10.1109/emr.2023.3285098 · Cited by 18
“This initial trustworthiness assessment (often referred to as trust propensity) has garnered much attention [14], [22], [36] though notably a more recent focus in the adoption of products [37], [38], [39], [40] and AI [10], [41], [42].”
BackgroundPerspectives on incorporating expert feedback into model updates
Chen, V., Bhatt, U., Heidari, H., Weller, A., & Talwalkar, A. (2023) · Patterns · doi:10.1016/j.patter.2023.100780 · Cited by 20
“, measures of confidence or trust [71, 122]) may be difficult to translate”
BackgroundMeasuring an artificial intelligence language model’s trust in humans using machine incentives
Johnson, T., & Obradovich, N. (2024) · Journal of Physics Complexity · doi:10.1088/2632-072x/ad1c69 · Cited by 17
“Researchers have maintained this focus because they hypothesize that trust affects AI usage [1] and because they harbor concerns about the alignment problem—that is, the challenge of ensuring that AI models accord with human values and respect humans’ interests [3, 4].”
BackgroundInfluentialPredicting Adoption Intention of ChatGPT- A Study on Business Professionals of BangladeshPreprint
Emon, M. M. H. (2023) · Research Square · doi:10.21203/rs.3.rs-3749611/v1 · Cited by 24
Investigating How Clinicians Form Trust in an AI-Based Mental Health Model: Qualitative Case Study
Kelly, A., Bhardwaj, N., Holmberg, T. T., van de Ven, P. W. J., Melia, R., Williams, J. E., Mathiasen, K., & Nielsen, A. S. (2025) · JMIR Human Factors · doi:10.2196/79658 · Cited by 17
AI in project teams: how trust calibration reconfigures team's collaboration and performance
Dawarka, V., Doargajudhur, M. S., & Dutot, V. (2026) · International Journal of Managing Projects in Business · doi:10.1108/ijmpb-07-2025-0285 · Cited by 18
Moriuchi, E. (2025) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2025.124004 · Cited by 16
Divisional perspectives on trust in generative AI: an empirical study of senior decision-makers
Khac, L. T. D., Leyer, M., Wichmann, J., & Richter, A. (2025) · Enterprise Information Systems · doi:10.1080/17517575.2025.2481557 · Cited by 15
“PII data in Europe is subject to data privacy laws such as the General Data Protection Regulation (GDPR), and therefore rights such as the ‘right to know’ and ‘right to erasure’ along with appropriate data management and data governance measurements have to be in place (Jacovi et al. 2021).”
What Needs Attention? Prioritizing Drivers of Developers' Trust and Adoption of Generative AI
Choudhuri, R., Trinkenreich, B., Pandita, R., Kalliamvakou, E., Steinmacher, I., Gerosa, M., Sanchez, C., & Sarma, A. (2025) · arXiv · doi:10.48550/arxiv.2505.17418 · Cited by 14
“Trust involves users attributing intent and anthropomorphism to the AI [60], leading to feelings of betrayal when trust is violated.”
InfluentialReadying Medical Students for Medical AI: The Need to Embed AI Ethics Education
Quinn, T. P., & Coghlan, S. (2021) · arXiv · doi:10.48550/arxiv.2109.02866 · Cited by 13
A comprehensive introspection on AI risks: taxonomy, challenges, and future directions
Ray, P. P. (2026) · Iran Journal of Computer Science · doi:10.1007/s42044-025-00370-5 · Cited by 17
“When adversaries exploit weaknesses in these boundary conditions—through carefully crafted injection payloads or by manipulating inter-agent communication—they create covert channels that subvert the intended security posture [76].”
Wilkowska, W., Otten, S., Maidhof, C., & Ziefle, M. (2023) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2023.2272075 · Cited by 13
“This can be an e-vendor, applications like Facebook and Twitter, but also intelligent systems, including ambient technologies (Jacovi et al., 2021; Ryan, 2020).”
Exploring NWS Forecasters’ Assessment of AI Guidance Trustworthiness
Cains, M. G., Wirz, C. D., Demuth, J. L., Bostrom, A., Gagne, D. J., McGovern, A., Sobash, R. A., & Madlambayan, D. (2024) · Weather and Forecasting · doi:10.1175/waf-d-23-0180.1 · Cited by 13
“Within the literature are many efforts to de fi ne and measure trustworthy AI and end-user trust in AI (e.g., Hoffman et al. 2018b; Jacovi et al. 2020; Varshney 2021; Stanton and Jensen 2021; Bostrom et al. 2024).”
MethodiSee: A case-based reasoning platform for the design of explanation experiences
Caro-Martínez, M., Recio-García, J. A., Díaz-Agudo, B., Darias, J. M., Wiratunga, N., Martin, K., Wijekoon, A., Nkisi-Orji, I., Corsar, D., Pradeep, P., et al. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112305 · Cited by 16
“For example, explanations may aim to enhance task performance when using the AI [9], build user trust in the system [10], or uncover potential ethical and bias issues [11].”
BackgroundThe Relationship Between AI Self-Efficacy and AI Trust of College Students
Aliño, J. P., Rebato, A. J., Abrenica, A. N., Baguio, J., Villanueva, M., Obenza-Tanudtanud, D. M. N., & Sumatra, K. P. (2024) · International Journal of Multidisciplinary Studies in Higher Education · doi:10.70847/587961 · Cited by 13
“This trust is built on users' expectations regarding the AI's functionality and its ability to act in their best interests, which parallels interpersonal trust concepts found in sociology (Jacovi et al., 2021).”
Swinger, N., Baseman, C. M., Ryu, M., Abdullah, S., Wiese, C. W., Sherrill, A. M., & Arriaga, R. I. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3710917 · Cited by 15
Dahlin, E. (2025) · AI & Society · doi:10.1007/s00146-025-02429-0 · Cited by 13
Adjust for Trust: Mitigating Trust-Induced Inappropriate Reliance on AI Assistance
Srinivasan, T., & Thomason, J. (2026) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3742413.3789136 · Cited by 18
Ferrario, A. (2021) · Journal of Medical Ethics · doi:10.1136/medethics-2021-107482 · Cited by 14
“I refer to Hatherley and Ferrario et al.’s contributions for a recent debate on this point 8 9 and Jacovi et al 10 for a discussion on trust and trustworthiness from the human-machine interaction perspective. reliabilism offers reasons to justify the beliefs in the results generated by the AI system.”
ResultWischnewski, M., & Krämer, N. (2022) · Frontiers in artificial intelligence and applications · doi:10.3233/faia220198 · Cited by 14
Kaleidoscope: Semantically-grounded, context-specific ML model evaluation
Suresh, H., Shanmugam, D., Chen, T., Bryan, A. G., D’Amour, A., Guttag, J. V., & Satyanarayan, A. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581482 · Cited by 19
“The resulting evaluations are There is increasing recognition that evaluations of machine learning valuable, but they require signifcant time, efort, and customization (ML) systems should be grounded in context [25, 41].”
BackgroundSoftware doping analysis for human oversight
Biewer, S., Baum, K. L., Sterz, S., Hermanns, H., Hetmank, S., Langer, M., Lauber-Rönsberg, A., & Lehr, F. (2024) · Formal Methods in System Design · doi:10.1007/s10703-024-00445-2 · Cited by 13
“Among them are enabling warranted trust in systems [61, 64, 100, 109], increasing humansystem decision-making performance [67] for instance through increasing human situation awareness when operating systems [107], enabling responsible decisionmaking and effective human oversight [13, 78, 112], as well as identifying…”
Wang, Z., Wang, J., Tian, C., Ali, A., & Yin, X. (2024) · Kybernetes · doi:10.1108/k-02-2024-0478 · Cited by 15
The Influence of Interdependence on Trust Calibration in Human-Machine Teams
Verhagen, R., Marcu, A., Neerincx, M. A., & Tielman, M. L. (2024) · Frontiers in artificial intelligence and applications · doi:10.3233/faia240203 · Cited by 14
“When appropriate trust is directly caused by information about the actual trustworthiness of the machine, this is called warranted appropriate trust [16,17].”
BackgroundThe Role of Transparency in AI-Driven Technologies: Targeting Healthcare
Subías-Beltrán, P., Pitarch, C., Migliorelli, C., Marte, L., Galofré, M., & Orte, S. (2024) · Artificial intelligence · doi:10.5772/intechopen.1007444 · Cited by 13
Phillips, V. J. (2025) · AI & Society · doi:10.1007/s00146-025-02337-3 · Cited by 16
“Recent work by Jacovi et al. (2021) establishes a formal framework for understanding trust formation in AI systems, while Asan et al. (2020) specifically examines how clinicians develop trust in healthcare AI applications.”
Plume: Scaffolding Text Composition in Dashboards
Lisnic, M., Setlur, V., & Sultanum, N. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713580 · Cited by 14
“While our user study provided valuable insights into dashboard authors’ initial impressions of Plume, as well as their attitudes and trust toward generative AI in dashboard text, it is crucial to explore its impact on longer-term workflows [24].”
xEEGNet: towards explainable AI in EEG dementia classification
Zanola, A., Tshimanga, L. F., Pup, F. D., Baiesi, M., & Atzori, M. (2025) · Journal of Neural Engineering · doi:10.1088/1741-2552/adf6e6 · Cited by 13
“Citing Jacovi et al [9], by designing an AI that users can and will trust to interact with, AI can be safely deployed in society.”
Fraidan, A. A. A. (2025) · SAGE Open · doi:10.1177/21582440251396113 · Cited by 12
“Transparency was measured via a 10-item Likert-style scale adapted from validated instruments in user trust and algorithmic fairness research (Jacovi et al., 2021; Schnackenberg et al., 2021).”
Dynamic calibration of trust and trustworthiness in AI-enabled systems
Liebherr, M., Enkel, E., Law, E. L., Mousavi, M. R., Sammartino, M., & Sieberg, P. M. (2026) · International Journal on Software Tools for Technology Transfer · doi:10.1007/s10009-026-00840-6 · Cited by 17
Starke, G., & Poppe, C. J. (2022) · Ethics and Information Technology · doi:10.1007/s10676-022-09650-1 · Cited by 12
“While the internal trustworthiness of a model depends on the question whether the “rea-soning process aligns with human reasoning” (Jacovi et al., 2021, p. 629) and may be promoted by a Jasperian understanding, the external path to trustworthiness relies on the observation from without and would…”
BackgroundConstantin, A., Atkinson, M., Bernabéu, M. O., Buckmaster, F., Dhillon, B., McTrusty, A., Strang, N. C., & Williams, R. (2023) · JMIR Human Factors · doi:10.2196/40887 · Cited by 15
Towards Responsible AI: Developing Explanations to Increase Human-AI Collaboration
Duarte, R. D. B. (2023) · Frontiers in artificial intelligence and applications · doi:10.3233/faia230126 · Cited by 12
Four facets of AI transparency
Larsson, S., Haresamudram, K., Högberg, C., Lao, Y., Nyström, A., & Söderlund, K. (2023) · Edward Elgar Publishing eBooks · doi:10.4337/9781803928562.00047 · Cited by 18
What Else Do I Need to Know? The Effect of Background Information on Users’ Reliance on QA Systems
Goyal, N., Briakou, E., Liu, A., Baumler, C., Bonial, C., Micher, J. C., Voss, C. R., Carpuat, M. J., & Daumé, H. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.201 · Cited by 19
“This indicates un-warranted trust (Jacovi et al., 2021) in the model to some extent (RQ1).”
BackgroundSIDEs: Separating Idealization from Deceptive 'Explanations' in xAI
Sullivan, E. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3658999 · Cited by 17
Workshop on Trust and Reliance in AI-Human Teams (TRAIT)
Bansal, G., Smith, A., Buçinca, Z., Wu, T., Holstein, K., Hullman, J. R., & Stumpf, S. (2022) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3491101.3503704 · Cited by 17
Robustness and Reliability Testing in Healthcare Using Artificial Intelligence
Khinvasara, T., Shankar, A., & Wong, C. (2024) · Asian Journal of Research in Computer Science · doi:10.9734/ajrcos/2024/v17i7482 · Cited by 12
Zhang, Y., Tian, J., & Deng, F. (2025) · Interacting with Computers · doi:10.1093/iwc/iwaf029 · Cited by 13
Jo, J., Zhang, H., Cai, J., & Goyal, N. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732077 · Cited by 17
Cases Are King: A User Study of Case Presentation to Explain CBR Decisions
Gates, L., Leake, D. B., & Wilkerson, K. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-40177-0_10 · Cited by 11
“Trust can be conceptualized in terms of user vulnerability and the extent to which the user accepts the risk present in an interaction [12, 23, 28].”
BackgroundPredictability and Comprehensibility in Post-Hoc XAI Methods: A User-Centered Analysis
Jalali, A., Haslhofer, B., Kriglstein, S., & Rauber, A. (2023) · Lecture notes in networks and systems · doi:10.1007/978-3-031-37717-4_46 · Cited by 14
Towards Characterizing Trust in Generative Artificial Intelligence among Students
Amoozadeh, M., Daniels, D., Chen, S., Nam, D., Kumar, A., Hilton, M. L., Alipour, M. A., & Ragavan, S. S. (2023) · ACM Conference on International Computing Education Research (ICER) · doi:10.1145/3568812.3603469 · Cited by 13
Decomposing and Measuring Trust in Open-Source Software Supply Chains
Boughton, L., Miller, C., Acar, Y., Wermke, D., & Kästner, C. (2024) · IEEE/ACM International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER) · doi:10.1145/3639476.3639775 · Cited by 21
Why Real Citizens Would Turn to Artificial Leaders
Spatola, N., & MacDorman, K. F. (2021) · Digital Government Research and Practice · doi:10.1145/3447954 · Cited by 12
Accountable AI for Healthcare IoT Systems
Bagave, P., Westberg, M., Dobbe, R., Janssen, M., & Ding, A. Y. (2022) · IEEE International Conference on Trust, Privacy and Security in Intelligent Systems and Applications (TPS-ISA) · doi:10.1109/tps-isa56441.2022.00013 · Cited by 16
“The most common definition of trust from interpersonal studies is: the anticipation of someone’s behaviour in vulnerable situations [24], [25].”
BackgroundBridging Expertise Gaps: The Role of LLMs in Human-AI Collaboration for CybersecurityPreprint
Tariq, S., Singh, R., Chhetri, M. B., Nepal, ., & Paris, C. L. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5462737 · Cited by 14
Perceptions of AI Across Sectors: A Comparative Review of Public AttitudesPreprint
Biały, F., Elliot, M. J., & Meckin, R. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.18233 · Cited by 11
Guglielmucci, F., & Di Basilio, D. (2025) · JMIR Human Factors · doi:10.2196/70698 · Cited by 10
“…in the propensity to place trust in and engage with AI are underpinned by various subjective factors, including, for example, personality traits and characteristics [29,30], which could help predict the likelihood of individuals opting for AI-based therapy over human-based clinical interventions.”
Fiduciary Responsibility: Facilitating Public Trust in Automated Decision Making
Harper, S. B., & Weber, E. (2022) · Journal of Social Computing · doi:10.23919/jsc.2022.0017 · Cited by 11
Trust Development and Explainability: A Longitudinal Study with a Personalized Assistive System
Zafari, S., de Pagter, J., Papagni, G., Rosenstein, A., Filzmoser, M., & Koeszegi, S. T. (2024) · Multimodal Technologies and Interaction · doi:10.3390/mti8030020 · Cited by 14
Explainable Time Series Prediction of Tyre Energy in Formula One Race Strategy
Todd, J., Jiang, J., Russo, A., Winkler, S., Sale, S. E., McMillan, J., & Rago, A. (2025) · ACM Symposium on Applied Computing · doi:10.1145/3672608.3707765 · Cited by 14
An LLM-guided platform for multi-granular collection and management of data provenance
Gregori, L., Lazzaro, P. L., Lazzaro, M. D., Missier, P., & Torlone, R. (2025) · Journal Of Big Data · doi:10.1186/s40537-025-01209-3 · Cited by 10
“Having the ability to explain how data is manipulated before model training and operations is increasingly recognized as the missing component in achieving satisfactory explainability, ultimately leading to trust in the resulting models [3, 4].”
Measuring Social Trust in AI: How Institutions Shape the Usage Intention of AI‐Based Technologies
Amir, S., Luk, S. C. Y., Saha, S., Tsyrulneva, I., & Teo, M. T. L. (2025) · Human Behavior and Emerging Technologies · doi:10.1155/hbe2/4084384 · Cited by 11
Social Misattributions in Conversations with Large Language Models
Ferrario, A., Termine, A., & Facchini, A. (2025) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1609/aies.v8i1.36600 · Cited by 14
Jin, X., & Li, J. (2025) · International Journal of Consumer Studies · doi:10.1111/ijcs.70133 · Cited by 12
Rohlfing, K. J., Främling, K., Lim, B., Alpsancar, S., & Thommes, K. (2026) · Springer eBooks · doi:10.1007/978-981-96-5290-7 · Cited by 10
Societal Frameworks Can Improve LLM Alignment
Stańczak, K., Meade, N., Bhatia, M., Zhou, H., Böttinger, K., Barnes, J., Stanley, J., Papernot, N., Chapados, N., Thérien, D., et al. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3806486 · Cited by 10
Self-citationBasic Issues and Challenges on Explainable Artificial Intelligence (XAI) in Healthcare Systems
Oladipo, I. D., Awotunde, J. B., AbdulRaheem, M., & Salihu, S. A. (2022) · Advances in medical technologies and clinical practice book series · doi:10.4018/978-1-6684-3791-9.ch011 · Cited by 10
Fine-Tuning Languages: Epistemological Foundations for Ethical AI in Journalism
Dierickx, L., & Lindén, C. (2023) · IEEE Swiss Conference on Data Science (SDS) · doi:10.1109/sds57534.2023.00013 · Cited by 12
“Favoring a mutual understanding is also about building trust, which is considered essential for human-AI interactions [138] [139] [140].”
BackgroundCrooks, C. L., Talwalkar, S., Sharma, T., Arora, K., & Venkatesh, K. (2024) · IEEE SoutheastCon · doi:10.1109/southeastcon52093.2024.10500080 · Cited by 10
Practising Appropriate Trust in Human-Centred AI Design
Degachi, C., Mehrotra, S., Yurrita, M., Niforatos, E., & Tielman, M. L. (2024) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3613905.3650825 · Cited by 10
“The current landscape of AI research predominantly emphasises building trust, without delving into the subtleties of appropriateness [18, 27].”
Aspects and Views on Responsible Artificial Intelligence
Brumen, B., Göllner, S., & Tropmann-Frick, M. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-25599-1_29 · Cited by 10
Steinmetz, S., Naugle, A., Schutte, P., Sweitzer, M., Washburne, A., Linville, L. M., Krofcheck, D., Kucer, M., & Myren, S. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.15511 · Cited by 10
“Proper calibration of trust is crucial; a user should trust a model as much as it SAND2025-00999O deserves, but not more [2].”
Zahedi, Z., Sengupta, S., & Kambhampati, S. (2024) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2024.3488559 · Cited by 10
RelAI: an automated approach to judge pointwise ML prediction reliability
Peracchio, L., Nicora, G., Parimbelli, E., Buonocore, T. M., Tavazzi, E., Bergamaschi, R., Dagliati, A., & Bellazzi, R. (2025) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2025.105857 · Cited by 11
Exploring the Impact of Explainability in Large Language Model (LLM) Applications on User Experience
Wang, Y., Fang, X., Xu, Z., Li, J., & Wang, L. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3719941 · Cited by 12
Zhu, Y., Hua, G., Liu, X., Wang, C., & Tang, M. (2025) · Frontiers in Psychology · doi:10.3389/fpsyg.2025.1539054 · Cited by 14
Metadata
- Authors
- Jacovi, A., Marasović, A., Miller, T., & Goldberg, Y.
- Affiliations
- Bar-Ilan University (IL)
- University of Washington (US)
- The University of Melbourne (AU)
- Venue
- Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency
- Publisher
- ACM
- Published
- 2021-03
- Pages
- 624-635
- OpenAlex
- W3134111219
- Cited by
- 545
- References
- 74
- Citations by year
- 2026: 131 · 2025: 137 · 2024: 102 · 2023: 90 · 2022: 60 · 2021: 24
- FWCI
- 31.88
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion)
- Funders
- European Research Council (802774)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Keywords
- trust in AI, interpersonal trust, contractual trust, trustworthiness, user vulnerability, explainable AI, human-AI interaction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.