How Serendipity Improves User Satisfaction with Recommendations? A Large-Scale User Evaluation
Open accessThe DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.
Abstract
Recommendation serendipity is being increasingly recognized as being equally important as the other beyond-accuracy objectives (such as novelty and diversity), in eliminating the “filter bubble” phenomenon of the traditional recommender systems. However, little work has empirically verified the effects of serendipity on increasing user satisfaction and behavioral intention. In this paper, we report the results of a large-scale user survey (involving over 3,000 users) conducted in an industrial mobile e-commerce setting. The study has identified the significant causal relationships from novelty, unexpectedness, relevance, and timeliness to serendipity, and from serendipity to user satisfaction and purchase intention. Moreover, our findings reveal that user curiosity plays a moderating role in strengthening the relationships from novelty to serendipity and from serendipity to satisfaction. Our third contribution lies in the comparison of several recommender algorithms, which demonstrates the significant improvements of the serendipity-oriented algorithm over the relevance- and novelty-oriented approaches in terms of user perceptions. We finally discuss the implications of this experiment, which include the feasibility of developing a more precise metric for measuring recommendation serendipity, and the potential benefit of a curiosity-based personalized serendipity strategy for recommender systems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Caution Regarding Rules of Thumb for Variance Inflation Factors
O’brien, R. M. (2007) · Quality & Quantity · doi:10.1007/s11135-006-9018-6 · Cited by 11,217
Structural Equation Modelling: Guidelines for Determining Model Fit
Hooper, D., Coughlan, J., & Mullen, M. R. (2008) · Dublin Institute of Technology · doi:10.21427/d7cf7r · Cited by 8,746
The Kolmogorov-Smirnov Test for Goodness of Fit
Massey, F. J. (1951) · Journal of the American Statistical Association · doi:10.1080/01621459.1951.10500769 · Cited by 7,210
A Beginner's Guide to Structural Equation Modeling: Fourth Edition
Schumacker, R. E., & Lomax, R. G. (2015) · doi:10.4324/9781315749105 · Cited by 6,604
Evaluating collaborative filtering recommender systems
Herlocker, J. L., Konstan, J. A., Terveen, L. G., & Riedl, J. T. (2004) · ACM Transactions on Information Systems · doi:10.1145/963770.963772 · Cited by 5,798 · cited 7×
“Although it has been recognized that accuracy alone does not always result in user satisfaction [12], few studies have verified whether and how the other beyond-accuracy factors, especially serendipity , have a positive effect in this regard.”
BackgroundInfluentialByrne, B. M. (2013) · doi:10.4324/9780203805534 · Cited by 4,016
A Meta-Analysis of Cronbach's Coefficient Alpha
Peterson, R. A. (1994) · Journal of Consumer Research · doi:10.1086/209405 · Cited by 3,028
The psychology of curiosity: A review and reinterpretation.
Loewenstein, G. (1994) · Psychological Bulletin · doi:10.1037/0033-2909.116.1.75 · Cited by 2,738 · cited 2×
“According to the psychological theory [2, 23, 39], curiosity is an intrinsic human trait, triggered when there is a gap between the person’s current knowledge level and the desired level.”
BackgroundWu, J.-H., & Wang, S.-C. (2005) · Information & Management · doi:10.1016/j.im.2004.07.001 · Cited by 2,284 · cited 1×
“First, the users who took part in our experiment mainly represent the population that has used mobile apps to buy products [43].”
BackgroundFILTER BUBBLE: Wie wir im Internet entmündigt werden
Pariser, E. (2012) · doi:10.3139/9783446431164 · Cited by 1,849
Cited by
Recommender systems in the healthcare domain: state-of-the-art and research issues
Tran, T. N. T., Felfernig, A., Trattner, C., & Holzinger, A. (2021) · Journal of Intelligent Information Systems · doi:10.1007/s10844-020-00633-6 · Cited by 303
Serendipity in Recommender Systems: A Systematic Literature Review
Ziarani, R. J., & Ravanmehr, R. (2021) · Journal of Computer Science and Technology · doi:10.1007/s11390-020-0135-9 · Cited by 123
Deep Learning Models for Serendipity Recommendations: A Survey and New Perspectives
Fu, Z., Niu, X., & Maher, M. L. (2024) · ACM Computing Surveys · doi:10.1145/3605145 · Cited by 30
User Bias in Beyond-Accuracy Measurement of Recommendation Algorithms
Wang, N., & Chen, L. (2021) · Fifteenth ACM Conference on Recommender Systems · doi:10.1145/3460231.3474244 · Cited by 22
“In this work, we used Taobao Serendipity Dataset 1 , which was collected previously [10, 44] Concretely, four algorithms were employed in their survey [10], respectively called HOT , Rel-CF , Nov-CF , and Ser-CF , which aim at recommending popular, relevant, novel, and serendipitous products…”
BackgroundMethodInfluentialSelf-citation
Metadata
- Authors
- Chen, L., Yang, Y., Wang, N., Yang, K., & Yuan, Q.
- Venue
- The World Wide Web Conference
- Publisher
- ACM
- Published
- 2019-05-13
- Pages
- 240-250
- OpenAlex
- W2913853152
- Cited by
- 139
- References
- 14
- FWCI
- 11.60
- Type
- conference paper
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.