The DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index.
Explaining the user experience of recommender systems
Open accessAbstract
Research on recommender systems typically focuses on the accuracy of prediction algorithms. Because accuracy only partially constitutes the user experience of a recommender system, this paper proposes a framework that takes a user-centric approach to recommender system evaluation. The framework links objective system aspects to objective user behavior through a series of perceptual and evaluative constructs (called subjective system aspects and experience, respectively). Furthermore, it incorporates the influence of personal and situational characteristics on the user experience. This paper reviews how current literature maps to the framework and identifies several gaps in existing work. Consequently, the framework is validated with four field trials and two controlled experiments and analyzed using Structural Equation Modeling. The results of these studies show that subjective system aspects and experience variables are invaluable in explaining why and how the user experience of recommender systems comes about . In all studies we observe that perceptions of recommendation quality and/or variety are important mediators in predicting the effects of objective system aspects on the three components of user experience: process (e.g. perceived effort, difficulty), system (e.g. perceived system effectiveness) and outcome (e.g. choice satisfaction). Furthermore, we find that these subjective aspects have strong and sometimes interesting behavioral correlates (e.g. reduced browsing indicates higher system effectiveness). They also show several tradeoffs between system aspects and personal and situational characteristics (e.g. the amount of preference feedback users provide is a tradeoff between perceived system usefulness and privacy concerns). These results, as well as the validated framework itself, provide a platform for future research on the user-centric evaluation of recommender systems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hu, L., & Bentler, P. M. (1999) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705519909540118 · Cited by 108,788
Perceived usefulness, perceived ease of use, and user acceptance of information technology
Davis, F. D. (1989) · MIS Quarterly · doi:10.2307/249008 · Cited by 68,483
User Acceptance of Information Technology: Toward A Unified View1
Venkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. D. (2003) · MIS Quarterly · doi:10.2307/30036540 · Cited by 45,035
Structural equation modeling in practice: A review and recommended two-step approach
Anderson, J. C., & Gerbing, D. W. (1988) · Psychological Bulletin · doi:10.1037/0033-2909.103.3.411 · Cited by 41,941
On the Evaluation of Structural Equation Models
Bagozzi, R. P., & Yi, Y. (1988) · Journal of the Academy of Marketing Science · doi:10.1007/bf02723327 · Cited by 26,163
Intention—Behavior Relations: A Conceptual and Empirical Review
Sheeran, P. (2002) · European Review of Social Psychology · doi:10.1080/14792772143000003 · Cited by 3,688
When choice is demotivating: can one desire too much of a good thing?
Iyengar, S. S., & Lepper, M. R. (2000) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.79.6.995 · Cited by 2,878
Explaining collaborative filtering recommendations
Herlocker, J. L., Konstan, J. A., & Riedl, J. (2000) · Proceedings of the 2000 ACM conference on Computer supported cooperative work · doi:10.1145/358916.358995 · Cited by 1,731
The Effects of Personalization and Familiarity on Trust and Adoption of Recommendation Agents1
Komiak, S. Y. X., & Benbasat, I. (2006) · MIS Quarterly · doi:10.2307/25148760 · Cited by 1,374
The Thing and I: Understanding the Relationship Between User and Product
Hassenzahl, M. (2003) · Human-Computer Interaction Series · doi:10.1007/1-4020-2967-5_4 · Cited by 1,097
Cited by
A Survey on Conversational Recommender Systems
Jannach, D., Manzoor, A., Cai, W., & Chen, L. (2022) · ACM Computing Surveys · doi:10.1145/3453154 · Cited by 521
Kaminskas, M., & Bridge, D. (2017) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/2926720 · Cited by 465
Möller, J., Trilling, D., Helberger, N., & van Es, B. (2018) · Information, Communication & Society · doi:10.1080/1369118x.2018.1444076 · Cited by 387
Oh, C., Song, J., Choi, J., Kim, S., Lee, S., & Suh, B. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174223 · Cited by 301
Towards Human-Centered Explainable AI: A Survey of User Studies for Model Explanations
Rong, Y., Leemann, T., Nguyen, T.-T., Fiedler, L., Qian, P., Unhelkar, V., Seidel, T., Kasneci, G., & Kasneci, E. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3331846 · Cited by 278
Personalization and privacy: a survey of privacy risks and remedies in personalization-based systems
Toch, E., Wang, Y., & Cranor, L. F. (2012) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-011-9110-z · Cited by 226
Beyond user experience: What constitutes algorithmic experiences?
Shin, D., Zhong, B., & Biocca, F. A. (2020) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2019.102061 · Cited by 174
Let Me Explain: Impact of Personal and Impersonal Explanations on Trust in Recommender Systems
Kunkel, J., Donkers, T., Michael, L., Barbu, C.-M., & Ziegler, J. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300717 · Cited by 156
Progress in context-aware recommender systems — An overview
Raza, S., & Ding, C. (2019) · Computer Science Review · doi:10.1016/j.cosrev.2019.01.001 · Cited by 151
Millecamp, M., Htun, N. N., Conati, C., & Verbert, K. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302313 · Cited by 148
Human-centered evaluation of explainable AI applications: a systematic review
Kim, J., Maathuis, H., & Sent, D. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1456486 · Cited by 130
Proactive vs. reactive personalization: Can customization of privacy enhance user experience?
Zhang, B., & Sundar, S. S. (2019) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2019.03.002 · Cited by 109
Behaviorism is Not Enough: Better Recommendations through Listening to Users
Ekstrand, M. D., & Willemsen, M. C. (2016) · Proceedings of the 10th ACM Conference on Recommender Systems · doi:10.1145/2959100.2959179 · Cited by 101
Self-citationExploring and Promoting Diagnostic Transparency and Explainability in Online Symptom Checkers
Tsai, C.-H., You, Y., Gui, X., Kou, Y., & Carroll, J. M. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445101 · Cited by 86
Factors influencing willingness to provide personal information for personalized recommendations
Kim, M. S., & Kim, S. (2018) · Computers in Human Behavior · doi:10.1016/j.chb.2018.06.031 · Cited by 78
Bertrand, A., Viard, T., Belloum, R., Eagan, J. R., & Maxwell, W. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581314 · Cited by 71
What's in a User? Towards Personalising Transparency for Music Recommender Interfaces
Millecamp, M., Htun, N. N., Conati, C., & Verbert, K. (2020) · Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization · doi:10.1145/3340631.3394844 · Cited by 50
Metadata
- Authors
- Knijnenburg, B. P., Willemsen, M. C., Gantner, Z., Soncu, H., & Newell, C.
- Venue
- User Modeling and User-Adapted Interaction
- Published
- 2012-03-10
- Volume
- 22
- Issue
- 4-5
- Pages
- 441-504
- ISSN
- 0924-1868 (ISSN-L)
- 1573-1391
- OpenAlex
- W2125330369
- Cited by
- 799
- References
- 17
- FWCI
- 121.44
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
