To explain or not to explain: the effects of personal characteristics when explaining music recommendations
Open accessAbstract
Recommender systems have been increasingly used in online services that we consume daily, such as Facebook, Netflix, YouTube, and Spotify. However, these systems are often presented to users as a "black box", i.e. the rationale for providing individual recommendations remains unexplained to users. In recent years, various attempts have been made to address this black box issue by providing textual explanations or interactive visualisations that enable users to explore the provenance of recommendations. Among other things, results demonstrated benefits in terms of precision and user satisfaction. Previous research had also indicated that personal characteristics such as domain knowledge, trust propensity and persistence may also play an important role on such perceived benefits. Yet, to date, little is known about the effects of personal characteristics on explaining recommendations. To address this gap, we developed a music recommender system with explanations and conducted an online study using a within-subject design. We captured various personal characteristics of participants and administered both qualitative and quantitative evaluation methods. Results indicate that personal characteristics have significant influence on the interaction and perception of recommender systems, and that this influence changes by adding explanations. For people with a low need for cognition are the explained recommendations the most beneficial. For people with a high need for cognition, we observed that explanations could create a lack of confidence. Based on these results, we present some design implications for explaining recommendations.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
R: A Language and Environment for Statistical Computing
R Core Team (2000) · doi:10.32614/r.manuals · Cited by 355,706 · see citing works
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414 · see citing works
Fitting Linear Mixed-Effects Models Using <b>lme4</b>
Bates, D., Mächler, M., Bolker, B., & Walker, S. (2015) · Journal of Statistical Software · doi:10.18637/jss.v067.i01 · Cited by 86,835 · see citing works
<b>lmerTest</b> Package: Tests in Linear Mixed Effects Models
Kuznetsova, A., Brockhoff, P. B., & Christensen, R. H. B. (2017) · Journal of Statistical Software · doi:10.18637/jss.v082.i13 · Cited by 24,346 · see citing works
Generalized expectancies for internal versus external control of reinforcement.
Rotter, J. B. (1966) · Psychological Monographs: General and Applied · doi:10.1037/h0092976 · Cited by 21,937 · see citing works
What's wrong with Bonferroni adjustments
Perneger, T. V. (1998) · BMJ · doi:10.1136/bmj.316.7139.1236 · Cited by 5,900 · see citing works
The Efficient Assessment of Need for Cognition
Cacioppo, J. T., Petty, R. E., & Feng Kao, C. (1984) · Journal of Personality Assessment · doi:10.1207/s15327752jpa4803_13 · Cited by 2,913 · see citing works
More powerful procedures for multiple significance testing
Hochberg, Y., & Benjamini, Y. (1990) · Statistics in Medicine · doi:10.1002/sim.4780090710 · Cited by 2,715 · see citing works
Models of Working Memory: Mechanisms of Active Maintenance and Executive Control
Miyake, A., & Shah, P. (1999) · Cited by 2,404 · see citing works
Doing a thematic analysis: A practical, step-by-step guide for learning and teaching scholars.
Maguire, M., & Delahunt, B. (2017) · All Ireland Journal of Higher Education · doi:10.62707/aishej.v9i3.335 · Cited by 2,099 · see citing works
Explaining collaborative filtering recommendations
Herlocker, J. L., Konstan, J. A., & Riedl, J. (2000) · Proceedings of the 2000 ACM conference on Computer supported cooperative work · doi:10.1145/358916.358995 · Cited by 1,731 · see citing works
He, S., & Lau, R. W. H. (2021) · Computer Graphics Forum · doi:10.1111/(issn)1467-8659 · Cited by 1,236 · see citing works
Müllensiefen, D., Gingras, B., Musil, J., & Stewart, L. (2014) · PLoS ONE · doi:10.1371/journal.pone.0089642 · Cited by 1,155 · see citing works
The Corsi Block-Tapping Task: Standardization and Normative Data
Kessels, R. P. C., van Zandvoort, M. J. E., Postma, A., Kappelle, L. J., & de Haan, E. H. F. (2000) · Applied Neuropsychology · doi:10.1207/s15324826an0704_8 · Cited by 1,027 · see citing works
A user-centric evaluation framework for recommender systems
Pu, P., Chen, L., & Hu, R. J. (2011) · ACM Conference on Recommender Systems (RecSys) · doi:10.1145/2043932.2043962 · Cited by 867 · see citing works
Explaining the user experience of recommender systems
Knijnenburg, B. P., Willemsen, M. C., Gantner, Z., Soncu, H., & Newell, C. (2012) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-011-9118-4 · Cited by 799 · see citing works
A Survey of Explanations in Recommender Systems
Tintarev, N., & Masthoff, J. (2007) · IEEE International Conference on Data Engineering Workshops (ICDEW) · doi:10.1109/icdew.2007.4401070 · Cited by 584 · see citing works
The role of transparency in recommender systems
Sinha, R. R., & Swearingen, K. (2002) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/506443.506619 · Cited by 480 · see citing works
A Principled Way of Assessing Visualization Literacy
Boy, J., Rensink, R. A., Bertini, E., & Fekete, J. (2014) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2014.2346984 · Cited by 280 · see citing works
Bostandjiev, S., O’Donovan, J., & Höllerer, T. (2012) · ACM Conference on Recommender Systems (RecSys) · doi:10.1145/2365952.2365964 · Cited by 274 · see citing works
Visualizing recommendations to support exploration, transparency and controllability
Verbert, K., Parra, D., Brusilovsky, P. L., et al. (2013) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/2449396.2449442 · Cited by 163 · see citing works
Self-citationO’Donovan, J., Smyth, B., Gretarsson, B., et al. (2008) · ACM SIGCHI Conference on Human Factors in Computing Systems (CHI) · doi:10.1145/1357054.1357222 · Cited by 148 · see citing works
Knijnenburg, B. P., Reijmer, N. J. M., & Willemsen, M. C. (2011) · ACM Conference on Recommender Systems (RecSys) · doi:10.1145/2043932.2043960 · Cited by 132 · see citing works
SmallWorlds: Visualizing Social Recommendations
Gretarsson, B., O’Donovan, J., Bostandjiev, S., et al. (2010) · Computer Graphics Forum · doi:10.1111/j.1467-8659.2009.01679.x · Cited by 112 · see citing works
Towards Adaptive Information Visualization: On the Influence of User Characteristics
Toker, D., Conati, C., Carenini, G., et al. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-31454-4_23 · Cited by 111 · see citing works
Self-citationHighlighting interventions and user differences
Carenini, G., Conati, C., Hoque, E., Steichen, B., Toker, D., & Enns, J. T. (2014) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2556288.2557141 · Cited by 79 · see citing works
Self-citationPersonality predicts musical sophistication
Greenberg, D. M., Müllensiefen, D., Lamb, M. E., & Rentfrow, P. J. (2015) · Journal of Research in Personality · doi:10.1016/j.jrp.2015.06.002 · Cited by 78 · see citing works
Dynamic aggregation with circular visual designs
Chuah, M. C. (2002) · IEEE Symposium on Information Visualization · doi:10.1109/infvis.1998.729557 · Cited by 77 · see citing works
Conati, C., Carenini, G., Hoque, E., Steichen, B., & Toker, D. (2014) · Computer Graphics Forum · doi:10.1111/cgf.12393 · Cited by 72 · see citing works
Self-citationAndjelkovic, I., Parra, D., & O’Donovan, J. (2016) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/2930238.2930280 · Cited by 72 · see citing works
Gou, L., You, F., Guo, J. Q., et al. (2011) · doi:10.1145/2016656.2016671 · Cited by 70 · see citing works
The effect of user characteristics on search effectiveness in information retrieval
Al‐Maskari, A., & Sanderson, M. (2011) · Information Processing & Management · doi:10.1016/j.ipm.2011.03.002 · Cited by 68 · see citing works
Trust Building in Recommender Agents
Chen, L., & Pu, P. (2005) · doi:10.5220/0001422901350145 · Cited by 66 · see citing works
Controlling Spotify Recommendations
Millecamp, M., Htun, N. N., Jin, Y., & Verbert, K. (2018) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3209219.3209223 · Cited by 64 · see citing works
Self-citationJin, Y., Tintarev, N., & Verbert, K. (2018) · ACM Conference on Recommender Systems (RecSys) · doi:10.1145/3240323.3240358 · Cited by 53 · see citing works
Self-citationModels of working memory:Mechanisms of active maintenance and executive control (1999)
Buchweitz, A. (2002) · Ilha do Desterro A Journal of English Language Literatures in English and Cultural Studies · Cited by 49 · see citing works
Tsai, C.-H., & Brusilovsky, P. L. (2018) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3172944.3172959 · Cited by 44 · see citing works
Towards User-Adaptive Information Visualization
Conati, C., Carenini, G., Toker, D., et al. (2015) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v29i1.9775 · Cited by 41 · see citing works
Self-citationIndividual differences in human-computer interaction
Aykin, N. M., & Aykin, T. (1991) · Computers & Industrial Engineering · doi:10.1016/0360-8352(91)90009-u · Cited by 40 · see citing works
Jin, Y., Seipp, K., Duval, E., & Verbert, K. (2016) · International Conference on Advanced Visual Interfaces (AVI) · doi:10.1145/2909132.2909269 · Cited by 35 · see citing works
Self-citationEffects of Individual Traits on Diversity-Aware Music Recommender User Interfaces
Jin, Y., Tintarev, N., & Verbert, K. (2018) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3209219.3209225 · Cited by 33 · see citing works
Self-citationLallé, S., Conati, C., & Carenini, G. (2017) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3079628.3079634 · Cited by 30 · see citing works
Self-citationIndividual differences in human-computer interaction
Aykin, N. M. (1989) · Computers & Industrial Engineering · doi:10.1016/0360-8352(89)90135-6 · Cited by 27 · see citing works
Understanding the Utility of Rationale in a Mixed-Initiative System for GUI Customization
Bunt, A., McGrenere, J., & Conati, C. (2007) · Lecture notes in computer science · doi:10.1007/978-3-540-73078-1_18 · Cited by 21 · see citing works
Self-citationLallé, S., Conati, C., & Carenini, G. (2017) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3099023.3099055 · Cited by 19 · see citing works
Self-citationEffects of Individual Differences in Working Memory on Plan Presentational Choices
Tintarev, N., & Masthoff, J. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2016.01793 · Cited by 14 · see citing works
Presenting Diversity Aware Recommendations: Making Challenging News Acceptable
Tintarev, N. (2018) · doi:10.18122/b2hq41 · Cited by 14 · see citing works
Geyer, R. W. (2009) · RMLE Online · doi:10.1080/19404476.2009.11462058 · Cited by 12 · see citing works
Enhancing recommendation diversity through a dual recommendation interface
Tsai, C. H., & Brusilovsky, P. L. (2017) · D-Scholarship@Pitt (University of Pittsburgh) · Cited by 10 · see citing works
Online Daters' Willingness to Use Recommender Technology for Mate Selection Decisions.
Tong, S. T., Corriero, E. F., Matheny, R. G., & Hancock, J. T. (2018) · Conference on Recommender Systems · Cited by 9 · see citing works
Dewan, I., & Benckendorff, P. J. (2013) · Cited by 6 · see citing works
Flagged first, then by citations
Research works printed but not linked: 6
- Ivana Andjelkovic, Denis Parra, and John O'Donovan. 2016. Moodplay: Interactive Mood-based Music Discovery and Recommendation. In Proc. of UMAP '16. ACM, 275-279.
- Peter Brusilovsky and Eva Millán. 2007. User models for adaptive hypermedia and adaptive educational systems. In The adaptive web. Springer, 3-53.
- Gitta O Domik and Bernd Gutkauf. 1994. User modeling for adaptive visualization systems. In Visualization, 1994., Visualization'94, Proceedings., IEEE Conference on. IEEE, 217-223.
- Liang Gou, Fang You, Jun Guo, Luqi Wu, and Xiaolong Luke Zhang. 2011. Sfviz: interest-based friends exploration and recommendation in social networks. In Proc. VINCI '11. ACM, 15.
- John O'Donovan, Barry Smyth, Brynjar Gretarsson, Svetlin Bostandjiev, and Tobias Höllerer. 2008. PeerChooser: visual interactive recommendation. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1085-1088.
- Bodo Winter. 2013. A very basic tutorial for performing linear mixed effects analyses. arXiv preprint arXiv:1308.5499 (2013). · arXiv
Cited by
Sorted by citations
Towards Human-Centered Explainable AI: A Survey of User Studies for Model Explanations
Rong, Y., Leemann, T., Nguyen, T.-T., Fiedler, L., Qian, P., Unhelkar, V., Seidel, T., Kasneci, G., & Kasneci, E. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3331846 · Cited by 278 · see citing works
Buçinca, Z., Malaya, M. B., & Gajos, K. Z. (2021) · arXiv · doi:10.48550/arxiv.2102.09692 · Cited by 1,124 · see citing works
“However, there is also some evidence that people with high NFC benefit less from explanations (in terms of confidence in their decisions) when using recommender systems than people with low NFC [47] although there appear to be limits to the generalizability of this result [48].”
BackgroundAnik, A. I., & Bunt, A. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445736 · Cited by 103 · see citing works
Exploring and Promoting Diagnostic Transparency and Explainability in Online Symptom Checkers
Tsai, C.-H., You, Y., Gui, X., Kou, Y., & Carroll, J. M. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445101 · Cited by 86 · see citing works
Beyond self-diagnosis: how a chatbot-based symptom checker should respond
You, Y., Tsai, C.-H., Li, Y., Ma, F., Heron, C., & Gui, X. (2023) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3589959 · Cited by 60 · see citing works
What's in a User? Towards Personalising Transparency for Music Recommender Interfaces
Millecamp, M., Htun, N. N., Conati, C., & Verbert, K. (2020) · Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization · doi:10.1145/3340631.3394844 · Cited by 50 · see citing works
Self-citationPersonalising Explainable Recommendations: Literature and Conceptualisation
Naiseh, M., Jiang, N., Ma, J., & Ali, R. (2020) · Advances in Intelligent Systems and Computing · doi:10.1007/978-3-030-45691-7_49 · Cited by 42 · see citing works
Explainable Recommendations in Intelligent Systems: Delivery Methods, Modalities and Risks
Naiseh, M., Jiang, N., Ma, J., & Ali, R. (2020) · Lecture Notes in Business Information Processing · doi:10.1007/978-3-030-50316-1_13 · Cited by 27 · see citing works
- Authors
- Millecamp, M., Htun, N. N., Conati, C., & Verbert, K.
- Affiliations
- KU Leuven (BE)
- University of British Columbia (CA)
- Venue
- Proceedings of the 24th International Conference on Intelligent User Interfaces
- Publisher
- ACM
- Published
- 2019-03-17
- Pages
- 397-407
- OpenAlex
- W2915495256
- Cited by
- 148
- References
- 52
- Citations by year
- 2026: 19 · 2025: 17 · 2024: 27 · 2023: 18 · 2022: 30 · 2021: 22 · 2020: 10 · 2019: 5
- FWCI
- 13.24
- Type
- conference paper
- Language
- en
- Open access
- green
- lirias.kuleuven.be
- Locations
- ACM International Conference on Intelligent User Interfaces (IUI) (publishedVersion)
- Lirias (KU Leuven) (submittedVersion, other-oa)
- Funders
- KU Leuven (C24/16/017)
- Topics
- Physical Sciences › Computer Science › Information Systems › Recommender Systems and Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Keywords
- recommender systems, explainable recommendation, black box, personal characteristics, need for cognition, trust propensity, user confidence, music recommendation, user satisfaction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
