The DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index.
Explaining the user experience of recommender systems
Open accessAbstract
Research on recommender systems typically focuses on the accuracy of prediction algorithms. Because accuracy only partially constitutes the user experience of a recommender system, this paper proposes a framework that takes a user-centric approach to recommender system evaluation. The framework links objective system aspects to objective user behavior through a series of perceptual and evaluative constructs (called subjective system aspects and experience, respectively). Furthermore, it incorporates the influence of personal and situational characteristics on the user experience. This paper reviews how current literature maps to the framework and identifies several gaps in existing work. Consequently, the framework is validated with four field trials and two controlled experiments and analyzed using Structural Equation Modeling. The results of these studies show that subjective system aspects and experience variables are invaluable in explaining why and how the user experience of recommender systems comes about . In all studies we observe that perceptions of recommendation quality and/or variety are important mediators in predicting the effects of objective system aspects on the three components of user experience: process (e.g. perceived effort, difficulty), system (e.g. perceived system effectiveness) and outcome (e.g. choice satisfaction). Furthermore, we find that these subjective aspects have strong and sometimes interesting behavioral correlates (e.g. reduced browsing indicates higher system effectiveness). They also show several tradeoffs between system aspects and personal and situational characteristics (e.g. the amount of preference feedback users provide is a tradeoff between perceived system usefulness and privacy concerns). These results, as well as the validated framework itself, provide a platform for future research on the user-centric evaluation of recommender systems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hu, L., & Bentler, P. M. (1999) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705519909540118 · Cited by 108,788
Perceived usefulness, perceived ease of use, and user acceptance of information technology
Davis, F. D. (1989) · MIS Quarterly · doi:10.2307/249008 · Cited by 68,483
User Acceptance of Information Technology: Toward A Unified View1
Venkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. D. (2003) · MIS Quarterly · doi:10.2307/30036540 · Cited by 45,035
Structural equation modeling in practice: A review and recommended two-step approach
Anderson, J. C., & Gerbing, D. W. (1988) · Psychological Bulletin · doi:10.1037/0033-2909.103.3.411 · Cited by 41,941
On the Evaluation of Structural Equation Models
Bagozzi, R. P., & Yi, Y. (1988) · Journal of the Academy of Marketing Science · doi:10.1007/bf02723327 · Cited by 26,163
Intention—Behavior Relations: A Conceptual and Empirical Review
Sheeran, P. (2002) · European Review of Social Psychology · doi:10.1080/14792772143000003 · Cited by 3,688
When choice is demotivating: can one desire too much of a good thing?
Iyengar, S. S., & Lepper, M. R. (2000) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.79.6.995 · Cited by 2,878
Explaining collaborative filtering recommendations
Herlocker, J. L., Konstan, J. A., & Riedl, J. (2000) · Proceedings of the 2000 ACM conference on Computer supported cooperative work · doi:10.1145/358916.358995 · Cited by 1,731
The Effects of Personalization and Familiarity on Trust and Adoption of Recommendation Agents1
Komiak, S. Y. X., & Benbasat, I. (2006) · MIS Quarterly · doi:10.2307/25148760 · Cited by 1,374
The Thing and I: Understanding the Relationship Between User and Product
Hassenzahl, M. (2003) · Human-Computer Interaction Series · doi:10.1007/1-4020-2967-5_4 · Cited by 1,097
Can There Ever Be Too Many Options? A Meta-Analytic Review of Choice Overload
Scheibehenne, B., Greifeneder, R., & Todd, P. M. (2010) · Journal of Consumer Research · doi:10.1086/651235 · Cited by 1,070 · cited 1×
“In conventional choice situations, too few items may restrict the users’ freedom of choice, whereas too many items may lead to choice overload (a process-EXP variable, Schwartz 2004; Iyengar and Lepper 2000; Scheibehenne et al. 2010).”
BackgroundUser experience (UX): towards an experiential perspective on product quality
Hassenzahl, M. (2008) · Proceedings of the 20th Conference on l'Interaction Homme-Machine · doi:10.1145/1512714.1512717 · Cited by 791
The effects of transparency on trust in and acceptance of a content-based art recommender
Cramer, H., Evers, V., Ramlal, S., van Someren, M., Rutledge, L., Stash, N., Aroyo, L., & Wielinga, B. (2008) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-008-9051-3 · Cited by 448
Trust building with explanation interfaces
Pu, P., & Chen, L. (2006) · Proceedings of the 11th international conference on Intelligent user interfaces · doi:10.1145/1111449.1111475 · Cited by 300
Understanding choice overload in recommender systems
Bollen, D., Knijnenburg, B. P., Willemsen, M. C., & Graus, M. (2010) · Proceedings of the fourth ACM conference on Recommender systems · doi:10.1145/1864708.1864724 · Cited by 232
Self-citationLayered evaluation of interactive adaptive systems: framework and formative methods
Paramythis, A., Weibelzahl, S., & Masthoff, J. (2010) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-010-9082-4 · Cited by 145
Ozok, A. A., Fan, Q., & Norcio, A. F. (2010) · Behaviour & Information Technology · doi:10.1080/01449290903004012 · Cited by 60 · cited 5×
“…that the interface of a recommender system may have far larger effects on users’ experience with the recommender than the recommender’s algorithmic performance (McNee et al. 2006a; Baudisch and Terveen 1999; Murray and Häubl 2008; Xiao and Benbasat 2007; Ziegler et al. 2005; Ozok et al. 2010).”
BackgroundInfluential
Cited by
A Survey on Conversational Recommender Systems
Jannach, D., Manzoor, A., Cai, W., & Chen, L. (2022) · ACM Computing Surveys · doi:10.1145/3453154 · Cited by 521
Kaminskas, M., & Bridge, D. (2017) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/2926720 · Cited by 465
Möller, J., Trilling, D., Helberger, N., & van Es, B. (2018) · Information, Communication & Society · doi:10.1080/1369118x.2018.1444076 · Cited by 387
Oh, C., Song, J., Choi, J., Kim, S., Lee, S., & Suh, B. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174223 · Cited by 301
Towards Human-Centered Explainable AI: A Survey of User Studies for Model Explanations
Rong, Y., Leemann, T., Nguyen, T.-T., Fiedler, L., Qian, P., Unhelkar, V., Seidel, T., Kasneci, G., & Kasneci, E. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3331846 · Cited by 278
Metadata
- Authors
- Knijnenburg, B. P., Willemsen, M. C., Gantner, Z., Soncu, H., & Newell, C.
- Venue
- User Modeling and User-Adapted Interaction
- Published
- 2012-03-10
- Volume
- 22
- Issue
- 4-5
- Pages
- 441-504
- ISSN
- 0924-1868 (ISSN-L)
- 1573-1391
- OpenAlex
- W2125330369
- Cited by
- 799
- References
- 17
- FWCI
- 121.44
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
