Let Me Explain: Impact of Personal and Impersonal Explanations on Trust in Recommender Systems
The DOI matches its Crossref record; 53 of the 54 references it cites were found in a registry or an index.
Abstract
Trust in a Recommender System (RS) is crucial for its overall success. However, it remains underexplored whether users trust personal recommendation sources (i.e. other humans) more than impersonal sources (i.e. conventional RS), and, if they do, whether the perceived quality of explanation provided account for the difference. We conducted an empirical study in which we compared these two sources of recommendations and explanations. Human advisors were asked to explain movies they recommended in short texts while the RS created explanations based on item similarity. Our experiment comprised two rounds of recommending. Over both rounds the quality of explanations provided by users was assessed higher than the quality of the system's explanations. Moreover, explanation quality significantly influenced perceived recommendation quality as well as trust in the recommendation source. Consequently, we suggest that RS should provide richer explanations in order to increase their perceived recommendation quality and trustworthiness.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Prior work suggests that subjective self-reported trust may not be a reliable indicator for trusting behaviors [16, 29], which are what ultimately matter in AI-assisted decision tasks.”
From Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making (2020) · cites this paper as [16]
In the citing paper
“several recent empirical studies found little evidence that the level of transparency has significant impact on people's willingness to trust a ML system, whether by using a directly interpretable model, allowing user to inspect the model behavior, showing explanation or reducing the number of features presented [5, 16, 27, 29].”
From Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making (2020) · cites this paper as [16]
In the citing paper
“Some studies use reliance as a proxy measure of trust [4,15,55,97], while others subjectively measure an individual's trust in the system and innate disposition to trust [8-10, 42, 51, 77].”
From Explanations Can Reduce Overreliance on AI Systems During Decision-Making (2023) · cites this paper as [51]
In the citing paper
“Some studies found that explanations increase trust [4,8], some found no significant effect [15,42,97], while others found varying effects under different conditions [49,51].”
From Explanations Can Reduce Overreliance on AI Systems During Decision-Making (2023) · cites this paper as [51]
In the citing paper
“First, the mapping between cues and trustworthiness attributes is not always one-to-one, meaning that a system feature can cue multiple bases of trust [31].”
From Designing for Responsible Trust in AI Systems: A Communication Perspective (2022) · cites this paper as [31]
Citations
Cites
Statistical Power Analysis for the Behavioral Sciences (2nd ed.).
Lachenbruch, P. A., & Cohen, J. (1989) · Journal of the American Statistical Association · doi:10.2307/2290095 · Cited by 37,395
lavaan : An R Package for Structural Equation Modeling
Rosseel, Y. (2012) · Journal of Statistical Software · doi:10.18637/jss.v048.i02 · Cited by 27,955 · cited 1×
“For this, we utilized the R package lavaan , version 0.6-2 [39].”
MethodInfluentialStatistical Power Analysis for the Behavioral Sciences
Cohen, J. (2013) · doi:10.4324/9780203771587 · Cited by 25,373
Matrix Factorization Techniques for Recommender Systems
Koren, Y., Bell, R., & Volinsky, C. (2009) · Computer · doi:10.1109/mc.2009.263 · Cited by 11,673 · cited 1×
“Recommendations were generated using the well-established technique of Matrix Factorization [24].”
MethodTrust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605 · cited 1×
“Trust is an important factor in human-machine interaction [28] and arguably of special interest for RS, since taking an advice is a highly trust-dependent behavior [30, 31].”
BackgroundDeveloping and validating trust measures for e-commerce: an integrative typology
McKnight, D. H., Choudhury, V., & Kacmar, C. (2002) · Information Systems Research · doi:10.1287/isre.13.3.334.81 · Cited by 5,550 · cited 5×
“Interestingly, such trust formation processes also seem to apply to computer systems in general [30] and to RS in particular [22].”
BackgroundMethodInfluentialLower bounds on sample size in structural equation modeling
Christopher Westland, J. (2010) · Electronic Commerce Research and Applications · doi:10.1016/j.elerap.2010.07.003 · Cited by 1,846 · cited 1×
“3) [7, 48].”
Explaining collaborative filtering recommendations
Herlocker, J. L., Konstan, J. A., & Riedl, J. (2000) · Proceedings of the 2000 ACM conference on Computer supported cooperative work · doi:10.1145/358916.358995 · Cited by 1,731 · cited 3×
“Various methods for explaining recommendations based on the computed similarity between items or users have been proposed [e.g. 2, 17, 46].”
BackgroundMethodThe influence of online product recommendations on consumers' online choices
Senecal, S., & Nantel, J. (2004) · Journal of Retailing · doi:10.1016/j.jretai.2004.04.001 · Cited by 1,585 · cited 1×
“The resulting two kinds of recommendation sources (i.e. human and non-human) are often termed as personal and impersonal [41, 44].”
Background(2015) · Springer eBooks · doi:10.1007/978-1-4899-7637-6 · Cited by 1,550
Muthén, B. (1984) · Psychometrika · doi:10.1007/bf02294210 · Cited by 1,544 · cited 1×
“By utilizing tools of causal statistical inference, i.e. structural equation modeling [33] and the counterfactual framework [18, 35, 40], we were able to reveal that the richness of explanations plays a pivotal role in trust-building processes.”
MethodThe Netflix Recommender System: Algorithms, Business Value, and Innovation
Gomez-Uribe, C. A., & Hunt, N. (2016) · ACM Transactions on Management Information Systems · doi:10.1145/2843948 · Cited by 1,418 · cited 1×
“Nearly all contemporary online platforms, such as Amazon, Netflix, and Facebook, make use of RS [14, 16, 43].”
BackgroundInitial trust formation in new organizational relationships
McKnight, D. H., Cummings, L. L., & Chervany, N. L. (1998) · The Academy of Management Review · doi:10.2307/259290 · Cited by 1,296 · cited 5×
“We used items from McKnight et al. [30, 31] to measure trusting beliefs and trusting intentions .”
BackgroundMethodInfluentialBennett, J. R., & Lanning, S. (2007) · Cited by 1,216
Being accurate is not enough: how accuracy metrics have hurt recommender systems
McNee, S. M., Riedl, J., & Konstan, J. A. (2006) · CHI '06 Extended Abstracts on Human Factors in Computing Systems · doi:10.1145/1125451.1125659 · Cited by 1,066 · cited 1×
“While for a long time research in RS focused primarily on algorithmic accuracy, it recently began to shift onto more user-centered qualities [4, 23, 27, 32] such as the degree of control [20], the transparency [46] and the trustworthiness [47] of a RS.”
BackgroundRecommender Systems: Introduction and Challenges
Ricci, F., Rokach, L., & Shapira, B. (2015) · Recommender Systems Handbook · doi:10.1007/978-1-4899-7637-6_1 · Cited by 846 · cited 2×
“RS have become ubiquitous means that proactively filter information in order to help users find interesting items [38].”
BackgroundExplaining the user experience of recommender systems
Knijnenburg, B. P., Willemsen, M. C., Gantner, Z., Soncu, H., & Newell, C. (2012) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-011-9118-4 · Cited by 799 · cited 1×
“Ad-ditionally, prior domain knowledge, the frequency of using online streaming providers and general trust in technology Knijnenburg et al. [21] were measured.”
BackgroundRecommender systems: from algorithms to user experience
Konstan, J. A., & Riedl, J. (2012) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-011-9112-x · Cited by 762 · cited 1×
“While for a long time research in RS focused primarily on algorithmic accuracy, it recently began to shift onto more user-centered qualities [4, 23, 27, 32] such as the degree of control [20], the transparency [46] and the trustworthiness [47] of a RS.”
BackgroundOnline peer and editorial recommendations, trust and choice in virtual markets
Smith, D., Menon, S., & Sivakumar, K. (2005) · Journal of Interactive Marketing · doi:10.1002/dir.20041 · Cited by 683 · cited 2×
“The resulting two kinds of recommendation sources (i.e. human and non-human) are often termed as personal and impersonal [41, 44].”
BackgroundTwo Decades of Recommender Systems at Amazon.com
Smith, B., & Linden, G. (2017) · IEEE Internet Computing · doi:10.1109/mic.2017.72 · Cited by 653 · cited 1×
“Nearly all contemporary online platforms, such as Amazon, Netflix, and Facebook, make use of RS [14, 16, 43].”
BackgroundOn The So-Called “Huber Sandwich Estimator” and “Robust Standard Errors”
Freedman, D. A. (2006) · The American Statistician · doi:10.1198/000313006x152207 · Cited by 498
Formal mode of statistical inference for causal effects
Rubin, D. B. (1990) · Journal of Statistical Planning and Inference · doi:10.1016/0378-3758(90)90077-8 · Cited by 483 · cited 1×
“By utilizing tools of causal statistical inference, i.e. structural equation modeling [33] and the counterfactual framework [18, 35, 40], we were able to reveal that the richness of explanations plays a pivotal role in trust-building processes.”
MethodThe role of transparency in recommender systems
Sinha, R., & Swearingen, K. (2002) · CHI '02 Extended Abstracts on Human Factors in Computing Systems · doi:10.1145/506443.506619 · Cited by 477 · cited 1×
“While explanations have been shown to have potential for increasing transparency [42, 46], this does not necessarily improve trust in RS [8].”
BackgroundA definition of causal effect for epidemiological research
Hernán, M. A. (2004) · Journal of Epidemiology and Community Health · doi:10.1136/jech.2002.006361 · Cited by 462 · cited 1×
“By utilizing tools of causal statistical inference, i.e. structural equation modeling [33] and the counterfactual framework [18, 35, 40], we were able to reveal that the richness of explanations plays a pivotal role in trust-building processes.”
MethodThe effects of transparency on trust in and acceptance of a content-based art recommender
Cramer, H., Evers, V., Ramlal, S., van Someren, M., Rutledge, L., Stash, N., Aroyo, L., & Wielinga, B. (2008) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-008-9051-3 · Cited by 448 · cited 1×
“While explanations have been shown to have potential for increasing transparency [42, 46], this does not necessarily improve trust in RS [8].”
BackgroundEvaluating recommender systems from the user’s perspective: survey of the state of the art
Pu, P., Chen, L., & Hu, R. (2012) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-011-9115-7 · Cited by 377
How should I explain? A comparison of different explanation types for recommender systems
Gedikli, F., Jannach, D., & Ge, M. (2014) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2013.12.007 · Cited by 337 · cited 2×
“Yet, transparency can help users in their decision making [45] and increase user satisfaction [12].”
BackgroundTintarev, N., & Masthoff, J. (2012) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-011-9117-5 · Cited by 334 · cited 1×
“Yet, transparency can help users in their decision making [45] and increase user satisfaction [12].”
BackgroundExplaining Recommendations: Satisfaction vs. Promotion
Bilgic, M., & Mooney, R. J. (2005) · Intelligent User Interfaces · Cited by 334
Explaining Recommendations: Design and Evaluation
Tintarev, N., & Masthoff, J. (2015) · Recommender Systems Handbook · doi:10.1007/978-1-4899-7637-6_10 · Cited by 329 · cited 7×
“In the same line, designers of RS often strive to increase transparency and trustworthiness by providing textual explanatory components for recommendations [5, 46, 50].”
BackgroundMethodInfluentialEvaluating Recommender Systems
Gunawardana, A., & Shani, G. (2015) · Recommender Systems Handbook · doi:10.1007/978-1-4899-7637-6_8 · Cited by 283 · cited 1×
“Even though recommendation algorithms have become highly accurate in terms of estimating a user’s preferences [1, 15], they oftentimes appear as “black boxes” by concealing important details from their users.”
BackgroundTrust-inspiring explanation interfaces for recommender systems
Pu, P., & Chen, L. (2007) · Knowledge-Based Systems · doi:10.1016/j.knosys.2007.04.004 · Cited by 248 · cited 1×
“Hence, several researchers argue that especially the trust-worthiness of a RS should be considered when assessing its quality [2, 22, 36].”
BackgroundInteracting with Recommenders—Overview and Research Directions
Jugovac, M., & Jannach, D. (2017) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3001837 · Cited by 172 · cited 1×
“While for a long time research in RS focused primarily on algorithmic accuracy, it recently began to shift onto more user-centered qualities [4, 23, 27, 32] such as the degree of control [20], the transparency [46] and the trustworthiness [47] of a RS.”
BackgroundHuman Decision Making and Recommender Systems
Jameson, A., Willemsen, M. C., Felfernig, A., de Gemmis, M., Lops, P., Semeraro, G. M., & Chen, L. (2015) · Springer eBooks · doi:10.1007/978-1-4899-7637-6_18 · Cited by 139
Victor, P., Cock, M. D., & Cornelis, C. (2010) · Springer eBooks · doi:10.1007/978-0-387-85820-3_20 · Cited by 128 · cited 1×
“While for a long time research in RS focused primarily on algorithmic accuracy, it recently began to shift onto more user-centered qualities [4, 23, 27, 32] such as the degree of control [20], the transparency [46] and the trustworthiness [47] of a RS.”
BackgroundHuman Decision Making and Recommender Systems
Chen, L., de Gemmis, M., Felfernig, A., Lops, P., Ricci, F., & Semeraro, G. (2013) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/2533670.2533675 · Cited by 120 · cited 1×
“Providing recommendations not only helps users make decisions, thus reducing their cognitive load [19, 37], but also increase purchases and general user satisfaction [38].”
BackgroundThe Mediation Formula: A Guide to the Assessment of Causal Pathways in Nonlinear Models
Pearl, J. (2012) · Wiley Series in Probability and Statistics · doi:10.1002/9781119945710.ch12 · Cited by 114 · cited 2×
“Generalizing to population level is done by taking the expected value which yields the mediation formula [35]:”
BackgroundCrowd-Based Personalized Natural Language Explanations for Recommendations
Chang, S., Harper, F. M., & Terveen, L. G. (2016) · Proceedings of the 10th ACM Conference on Recommender Systems · doi:10.1145/2959100.2959153 · Cited by 114 · cited 4×
“We thus believe that combinations of different explanation styles lead to explanations with a higher perceived value, which is backed up by prior findings [5].”
BackgroundResultInfluentialPanniello, U., Gorgoglione, M., & Tuzhilin, A. (2016) · Information Systems Research · doi:10.1287/isre.2015.0610 · Cited by 80
How to Recommend?: User Trust Factors in Movie Recommender Systems
Berkovsky, S., Taib, R., & Conway, D. (2017) · Proceedings of the 22nd International Conference on Intelligent User Interfaces · doi:10.1145/3025171.3025209 · Cited by 77 · cited 3×
“For instance, explanations indicating a high average rating of a recommendation resulted in a high perceived benevolence [2].”
BackgroundMethodGefen, D., & Straub, D. W. (1997) · Cited by 75
HCI for Recommender Systems: the Past, the Present and the Future
Valdez, A. S. C., Ziefle, M., & Verbert, K. (2016) · ACM Conference on Recommender Systems (RecSys) · doi:10.1145/2959100.2959158 · Cited by 75 · cited 1×
“While for a long time research in RS focused primarily on algorithmic accuracy, it recently began to shift onto more user-centered qualities [4, 23, 27, 32] such as the degree of control [20], the transparency [46] and the trustworthiness [47] of a RS.”
BackgroundAlhajj, R., & Rokne, J. G. (2018) · Springer eBooks · doi:10.1007/978-1-4939-7131-2_101201 · Cited by 72
Exploring Explanation Effects on Consumers’ Trust in Online Recommender Agents
Zhang, J., & Curley, S. P. (2018) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2017.1357904 · Cited by 71 · cited 1×
“In the same line, designers of RS often strive to increase transparency and trustworthiness by providing textual explanatory components for recommendations [5, 46, 50].”
BackgroundFelfernig, A., & Gula, B. (2006) · IEEE Conference on Commerce and Enterprise Computing (CEC) · doi:10.1109/cec-eee.2006.14 · Cited by 70 · cited 1×
“Another approach to enhance trust in RS is to provide the rationale behind a recommendation in the form of textual explanations [10, 17, 46].”
BackgroundKunkel, J., Loepp, B., & Ziegler, J. (2017) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3025171.3025189 · Cited by 56 · cited 1×
“While for a long time research in RS focused primarily on algorithmic accuracy, it recently began to shift onto more user-centered qualities [4, 23, 27, 32] such as the degree of control [20], the transparency [46] and the trustworthiness [47] of a RS.”
BackgroundSelf-citationWho predicts better? results from an online study comparing humans and an online recommender system
Krishnan, V., Narayanashetty, P. K., Nathan, M., Davies, R. T., & Konstan, J. A. (2008) · ACM Conference on Recommender Systems (RecSys) · doi:10.1145/1454008.1454042 · Cited by 50 · cited 2×
“Although humans have been observed to be less accurate when predicting another user’s interests [25], the social cues transmitted by a personal recommendation source create social presence and can foster users’ trust in a system [6, 26].”
BackgroundResultThe formation of trust and distrust in recommendation agents in repeated interactions
Xiao, S., & Benbasat, I. (2003) · International Conference on Electronic Commerce · doi:10.1145/948005.948043 · Cited by 41
Impact of item consumption on assessment of recommendations in user studies
Loepp, B., Donkers, T., Kleemann, T., & Ziegler, J. (2018) · ACM Conference on Recommender Systems (RecSys) · doi:10.1145/3240323.3240375 · Cited by 30 · cited 2×
“Wewere interested in identifying whether the interaction led to differences in the assessment of trust subject to our 4 We, nonetheless, tested for possible differences between pre-and post-consumption and did not find any significant differences which is in line with [29] for the movie domain.”
MethodResultSelf-citationExplaining Recommendations by Means of User Reviews.
Donkers, T., Loepp, B., & Ziegler, J. (2018) · IUI Workshops · Cited by 13
Self-citationGuy, I. (2012) · Springer eBooks · doi:10.1007/978-1-0716-2197-4_22 · Cited by 12
The Influence of Social Presence on Evaluating Personalized Recommender Systems
Choi, J. R., Lee, H. J., & Kim, Y. C. (2008) · AIS Electronic Library (AISeL) · Cited by 10
Trust-related Effects of Expertise and Similarity Cues in Human-Generated Recommendations.
Kunkel, J., Donkers, T., Barbu, C.-M., & Ziegler, J. (2018) · IUI Workshops · Cited by 9
Self-citation
Research works printed but not linked: 3
- 10.5555/2017317.2017325 · doi:10.5555/2017317.2017325
- 10.1518/hfes.46.1.50.30392 · doi:10.1518/hfes.46.1.50.30392 · Paper page
- 10.5465/amr.1998.926622 · DOI · Paper page
Cited by
A Survey of Graph Neural Networks for Recommender Systems: Challenges, Methods, and Directions
Gao, C., Zheng, Y., Li, N., Li, Y., Qin, Y., Piao, J., Quan, Y., Chang, J., Jin, D., He, X., et al. (2023) · ACM Transactions on Recommender Systems · doi:10.1145/3568022 · Cited by 1,082
“Increasing the explainability of recommender systems can enhance users’ perceived transparency [154], persuasiveness [166], and trustworthiness [86], and facilitate practitioners to debug and refine the system [236].”
BackgroundZhang, Y., Liao, Q. V., & Bellamy, R. K. E. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372852 · Cited by 895
“4.16 compensation for completing the half-hour long experiment. 3.2 Results 3.2.1 Trust. Prior work suggests that subjective self-reported trust may not be a reliable indicator for trusting behaviors [16, 29], which are what ultimately matter in AI-assisted decision tasks. Therefore, following recent studies [17, 27,…”
BackgroundResultInfluentialDoes the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., & Weld, D. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445717 · Cited by 757
“ely to benefit the overall process. As a result, various studies have evaluated the effect of explanations from different dimensions, including whether the explanation improves users’ trust on the AI [34, 68] or enables users to simulate the model predictions [13, 55], or assisting developers to debug and train a bett…”
BackgroundDoes the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., & Weld, D. S. (2020) · arXiv · doi:10.48550/arxiv.2006.14779 · Cited by 754
“ely to benefit the overall process. As a result, various studies have evaluated the effect of explanations from different dimensions, including whether the explanation improves users’ trust on the AI [34, 68] or enables users to simulate the model predictions [13, 55], or assisting developers to debug and train a bett…”
BackgroundFormalizing Trust in Artificial Intelligence: Prerequisites, Causes and Goals of Human Trust in AI
Jacovi, A., Marasović, A., Miller, T., & Goldberg, Y. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445923 · Cited by 545
“oversight Foster fundamental human rights Fairness checklists See “Diversity, non-discrimination, fairness” Support users’ agency All User-centered explanations [59] Enable human oversight Explanations in recommender systems [41]”
Background
Metadata
- Authors
- Kunkel, J., Donkers, T., Michael, L., Barbu, C.-M., & Ziegler, J.
- Affiliations
- University of Duisburg-Essen (DE)
- Venue
- Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2019-05-02
- Pages
- 1-12
- OpenAlex
- W2941411956
- Cited by
- 156
- References
- 54
- Citations by year
- 2026: 16 · 2025: 20 · 2024: 26 · 2023: 22 · 2022: 24 · 2021: 29 · 2020: 18 · 2019: 1
- FWCI
- 14.34
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- Universitätsbibliographie, Universität Duisburg-Essen (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Information Systems › Recommender Systems and Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Keywords
- recommender systems, recommendation trust, explanation quality, personal recommendations, item similarity, movie recommendation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.