Detecting anomalies in dynamic rating data: a robust probabilistic model for rating evolution
Abstract
Rating data is ubiquitous on websites such as Amazon, TripAdvisor, or Yelp. Since ratings are not static but given at various points in time, a temporal analysis of rating data provides deeper insights into the evolution of a product's quality. In this work, we tackle the following question: Given the time stamped rating data for a product or service, how can we detect the general rating behavior of users as well as time intervals where the ratings behave anomalous? We propose a Bayesian model that represents the rating data as sequence of categorical mixture models. In contrast to existing methods, our method does not require any aggregation of the input but it operates on the original time stamped data. To capture the dynamic effects of the ratings, the categorical mixtures are temporally constrained: Anomalies can occur in specific time intervals only and the general rating behavior should evolve smoothly over time. Our method automatically determines the intervals where anomalies occur, and it captures the temporal effects of the general behavior by using a state space model on the natural parameters of the categorical distributions. For learning our model, we propose an efficient algorithm combining principles from variational inference and dynamic programming. In our experimental study we show the effectiveness of our method and we present interesting discoveries on multiple real world datasets.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The study confirms that fraud is perpetrated by experts who control many sockpuppet user accounts.
“Our study thus confirms observations made by existing work, that fraud is perpetrated by experts who control either (1) many sockpuppet user accounts, e.g., [28, 37, recruited online [13,17].”
From The Art and Craft of Fraudulent App Promotion in Google Play (2019) · cites this paper as [17]
Cites
New Introduction to Multiple Time Series Analysis
Lütkepohl, H. (2005) · Springer eBooks · doi:10.1007/978-3-540-27752-1 · Cited by 5,972 · see citing works
Pattern Recognition and Machine Learning
Neal, R. M. (2007) · Technometrics · doi:10.1198/tech.2007.s518 · Cited by 4,629 · see citing works
Blei, D. M., & Lafferty, J. (2006) · International Conference on Machine Learning · doi:10.1145/1143844.1143859 · Cited by 2,425 · see citing works
Jindal, N., & Liu, B. (2008) · Proceedings of the international conference on Web search and web data mining - WSDM '08 · doi:10.1145/1341531.1341560 · Cited by 1,542 · see citing works
Collaborative filtering with temporal dynamics
Koren, Y. (2009) · ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/1557019.1557072 · Cited by 1,217 · see citing works
Forecasting With Bayesian Vector Autoregressions—Five Years of Experience
Litterman, R. B. (1986) · Journal of Business and Economic Statistics · doi:10.1080/07350015.1986.10509491 · Cited by 1,153 · see citing works
Aggarwal, C. C. (2013) · Springer eBooks · doi:10.1007/978-1-4614-6396-2 · Cited by 1,110 · see citing works
Aggarwal, C. C. (2013) · Cited by 832 · see citing works
Latent aspect rating analysis on review text data
Wang, H., Lu, Y., & Zhai, C. (2010) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/1835804.1835903 · Cited by 651 · see citing works
Book Review: Pattern Recognition and Machine Learning
Nasrabadi, N. M. (2007) · Journal of Electronic Imaging · doi:10.1117/1.2819119 · Cited by 636 · see citing works
Continuous Time Dynamic Topic ModelsPreprint
Wang, C., Blei, D. M., & Heckerman, D. E. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1206.3298 · Cited by 317 · see citing works
Koenigstein, N., Dror, G., & Koren, Y. (2011) · ACM Conference on Recommender Systems (RecSys) · doi:10.1145/2043932.2043964 · Cited by 281 · see citing works
Bentley, J. L. (1984) · Communications of the ACM · doi:10.1145/358234.381162 · Cited by 244 · see citing works
Forecasting with Bayesian Vector Autoregression
Karlsson, S. (2013) · Handbook of economic forecasting · doi:10.1016/b978-0-444-62731-5.00015-4 · Cited by 209 · see citing works
Introduction to Stochastic Processes
Lawler, G. F. (2018) · Chapman & Hall/CRC eBooks · doi:10.1201/9781315273600 · Cited by 168 · see citing works
Statistical change detection for multi-dimensional data
Song, X., Wu, M., Jermaine, C., & Ranka, S. (2007) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/1281192.1281264 · Cited by 150 · see citing works
Ahmed, A. A., & Xing, E. P. (2012) · arXiv (Cornell University) · doi:10.1184/r1/6476402 · Cited by 146 · see citing works
An Introduction to Stochastic Processes
Dufour, J. (2006) · CRC Press eBooks · doi:10.1201/b15871-3 · Cited by 96 · see citing works
Robust multivariate autoregression for anomaly detection in dynamic product ratings
Günnemann, N., Günnemann, S., & Faloutsos, C. (2014) · International World Wide Web Conference (WWW) · doi:10.1145/2566486.2568008 · Cited by 61 · see citing works
Self-citationLearning an Outlier-Robust Kalman Filter
Ting, J.-A., Theodorou, E. A., & Schaal, S. (2007) · Lecture notes in computer science · doi:10.1007/978-3-540-74958-5_76 · Cited by 58 · see citing works
Computing Maximum-Scoring Segments in Almost Linear Time
Bengtsson, F., & Chen, J. (2006) · Lecture notes in computer science · doi:10.1007/11809678_28 · Cited by 10 · see citing works
Dynamics of a Collaborative Rating System
Lerman, K. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-00528-2_5 · Cited by 9 · see citing works
Zhang, H., Spiliopoulou, M., Mobasher, B., Giles, C. L., McCallum, A., Nasraoui, O., Srivastava, J., & Yen, J. (2009) · Springer eBooks · Cited by 8 · see citing works
Flagged first, then by citations
3 of the 26 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Bimodal Distribution and Co-Bursting in Review Spam Detection
Li, H., Fei, G., Wang, S., Liu, B., Shao, W., Mukherjee, A., & Shao, J. (2017) · Proceedings of the 26th International Conference on World Wide Web · doi:10.1145/3038912.3052582 · Cited by 141 · see citing works
Analyzing and Detecting Opinion Spam on a Large-scale Dataset via Temporal and Spatial Patterns
Li, H., Chen, Z., Mukherjee, A., Liu, B., & Shao, J. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v9i1.14652 · Cited by 116 · see citing works
BIRDNEST: Bayesian Inference for Ratings-Fraud Detection
Hooi, B., Shah, N., Beutel, A., Günnemann, S., Akoglu, L., Kumar, M., Makhija, D., & Faloutsos, C. (2016) · SIAM International Conference on Data Mining · doi:10.1137/1.9781611974348.56 · Cited by 118 · see citing works
Self-citationLeveraging Joint Interactions for Credibility Analysis in News Communities
Mukherjee, S., & Weikum, G. (2015) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/2806416.2806537 · Cited by 100 · see citing works
Re-scale AdaBoost for attack detection in collaborative filtering recommender systems
Yang, Z., Xu, L., Cai, Z., & Xu, Z. (2016) · Knowledge-Based Systems · doi:10.1016/j.knosys.2016.02.008 · Cited by 92 · see citing works
Deceptive review detection using labeled and unlabeled data
Rout, J. K., Singh, S., Jena, S. K., & Bakshi, S. (2016) · Multimedia Tools and Applications · doi:10.1007/s11042-016-3819-y · Cited by 91 · see citing works
Costa, A. F., Yamaguchi, Y., Traina, A. J. M., Traina, C., & Faloutsos, C. (2015) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2783258.2783294 · Cited by 84 · see citing works
Self-citationBayesian Robust Attributed Graph Clustering: Joint Learning of Partial Anomalies and Group Structure
Bojchevski, A., & Günnemann, S. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11642 · Cited by 77 · see citing works
Self-citationLiu, S., Hooi, B., & Faloutsos, C. (2017) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3132847.3133018 · Cited by 74 · see citing works
Self-citationEstimating user behavior toward detecting anomalous ratings in rating systems
Yang, Z., Cai, Z., & Guan, X. (2016) · Knowledge-Based Systems · doi:10.1016/j.knosys.2016.08.011 · Cited by 71 · see citing works
Robust Spectral Clustering for Noisy Data
Bojchevski, A., Matkovic, Y., & Günnemann, S. (2017) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3097983.3098156 · Cited by 66 · see citing works
Self-citationCombating Crowdsourced Review Manipulators: A Neighborhood-Based Approach
Kaghazgaran, P., Caverlee, J., & Squicciarini, A. (2018) · Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining · doi:10.1145/3159652.3159726 · Cited by 59 · see citing works
A Contrast Metric for Fraud Detection in Rich Graphs
Liu, S., Hooi, B., & Faloutsos, C. (2018) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2018.2876531 · Cited by 36 · see citing works
Self-citationReliable fake review detection via modeling temporal and behavioral patterns
Wu, X., Dong, Y., Tao, J., Huang, C., & Chawla, N. V. (2017) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata.2017.8257963 · Cited by 34 · see citing works
Mukherjee, S., & Weikum, G. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.02667 · Cited by 29 · see citing works
The Art and Craft of Fraudulent App Promotion in Google Play
Rahman, M., Hernandez, N., Recabarren, R., Ahmed, S. I., & Carbunar, B. (2019) · Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3319535.3345658 · Cited by 24 · see citing works
Spotting anomalous ratings for rating systems by analyzing target users and items
Yang, Z., Cai, Z., & Yang, Y. (2017) · Neurocomputing · doi:10.1016/j.neucom.2017.02.052 · Cited by 23 · see citing works
Modeling Temporal Activity to Detect Anomalous Behavior in Social Media
Costa, A. F., Yamaguchi, Y., Traina, A. J. M., Traina, C., & Faloutsos, C. (2017) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3064884 · Cited by 21 · see citing works
Self-citationUncovering anomalous rating behaviors for rating systems
Yang, Z., Sun, Q., Zhang, Y., & Zhang, B. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.05.001 · Cited by 21 · see citing works
Experience-Aware Item Recommendation in Evolving Review Communities
Mukherjee, S., Lamba, H., & Weikum, G. (2015) · Proceedings · doi:10.1109/icdm.2015.111 · Cited by 21 · see citing works
Identifying ground truth in opinion spam: an empirical survey based on review psychology
Li, J., Wang, X., Yang, L., Zhang, P., & Yang, D. (2020) · Applied Intelligence · doi:10.1007/s10489-020-01764-7 · Cited by 17 · see citing works
Identification of Malicious Injection Attacks in Dense Rating and Co-Visitation Behaviors
Yang, Z., Sun, Q., Zhang, Y., & Wang, W. (2020) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2020.3016827 · Cited by 17 · see citing works
HoloScope: Topology-and-Spike Aware Fraud DetectionPreprint
Liu, S., Hooi, B., & Faloutsos, C. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.02505 · Cited by 15 · see citing works
Self-citationYang, Z., Sun, Q., & Zhang, B. (2018) · IEEE Access · doi:10.1109/access.2018.2869271 · Cited by 14 · see citing works
Zhang, L., Wang, S., Lin, Z., & Wu, Y. (2019) · Telematics and Informatics · doi:10.1016/j.tele.2019.01.009 · Cited by 14 · see citing works
Modeling Review Spam Using Temporal Patterns and Co-bursting BehaviorsPreprint
Li, H., Fei, G., Wang, S., Liu, B., Shao, W., Mukherjee, A., & Shao, J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.06625 · Cited by 13 · see citing works
The Power of Certainty: A Dirichlet-Multinomial Model for Belief Propagation
Eswaran, D., Günnemann, S., & Faloutsos, C. (2017) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9781611974973.17 · Cited by 12 · see citing works
Self-citationV ol T ime : Unsupervised Anomaly Detection on Users' Online Activity Volume
Chino, D. Y. T., Costa, A. F., Traina, A. J. M., & Faloutsos, C. (2017) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9781611974973.13 · Cited by 11 · see citing works
Self-citationYang, Z., Sun, Q., & Zhang, Y. (2020) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2020.3023114 · Cited by 10 · see citing works
Yang, Z., Sun, Q., Zhang, Y., Zhu, L., & Ji, W. (2020) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2020.2977023 · Cited by 9 · see citing works
Temporal Anomaly Detection: Calibrating the Surprise
Gutflaish, E., Kontorovich, A., Sivan Sabato, A., Biller, O., & Sofer, O. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33013755 · Cited by 8 · see citing works
Continuous Experience-aware Language Model
Mukherjee, S., Günnemann, S., & Weikum, G. (2016) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2939672.2939780 · Cited by 7 · see citing works
Self-citationModel-based Cluster Analysis for Identifying Suspicious Activity Sequences in Software
Lamba, H., Glazier, T. J., Cámara, J., Schmerl, B. R., Garlan, D., & Pfeffer, J. (2017) · ACM International Workshop on Security and Privacy Analytics (IWSPA) · doi:10.1145/3041008.3041014 · Cited by 7 · see citing works
Detecting Anomalous Ratings in Collaborative Filtering Recommender Systems
Yang, Z., & Cai, Z. (2016) · International Journal of Digital Crime and Forensics · doi:10.4018/ijdcf.2016040102 · Cited by 5 · see citing works
GhostLink: Latent Network Inference for Influence-aware Recommendation
Mukherjee, S., & Guennemann, S. (2019) · International World Wide Web Conference (WWW) · doi:10.1145/3308558.3313449 · Cited by 6 · see citing works
Hernandez, N., Recabarren, R., Carbunar, B., & Ahmed, S. I. (2021) · Proceedings of the 21st ACM Internet Measurement Conference · doi:10.1145/3487552.3487837 · Cited by 3 · see citing works
Venil, P., Vinodhini, G. A. F., & Joseph, K. S. (2021) · Journal of Business Analytics · doi:10.1080/2573234x.2021.1947752 · Cited by 3 · see citing works
Discovering Opinion Changes in Online Reviews via Learning Fine-Grained Sentiments
Liu, Z., Lin, Y., Wang, M., & Lu, Z. (2016) · IEEE International Conference on Collaboration and Internet Computing (CIC) · doi:10.1109/cic.2016.015 · Cited by 3 · see citing works
Detecting Anomalous Ratings Using Matrix Factorization for Recommender Systems
Yang, Z., Cai, Z., & Chen, X. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-39958-4_1
Defending Suspected Ratings in Collaborative Filtering Recommender Systems: A Fast Detection Method
Yang, Z. (2016) · International Conference on Information Science and Cloud Computing Companion (ISCC-C) · doi:10.22323/1.264.0015
Queue Classification for Fraud Types: Banking Domain
Trikha, A., & Khant, S. M. (2018) · Advances in intelligent systems and computing · doi:10.1007/978-981-13-1274-8_20
GhostLink: Latent Network Inference for Influence-aware RecommendationPreprint
Mukherjee, S., & Guennemann, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.05955
User-based Network Embedding for Collective Opinion Spammer DetectionPreprint
Wang, Z., Wei, W., Mao, X.-L., Guo, G., Zhou, P., & Feng, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.07783
Application of Business Big Data Management and Decision Making
Cao, J. (2023) · Springer eBooks · doi:10.1007/978-981-99-3588-8_9
Detection of Fake Reviews in Yelp Dataset Using Machine Learning and Chain Classifier Approach
Abdulzahra, L. S., & Obaid, A. J. (2024) · Lecture notes in networks and systems · doi:10.1007/978-981-99-9562-2_27
Polarity-Driven Ensemble Framework for Online Fake Reviews Detection
Abdullshaheed, Z. Z., & Obaid, A. J. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-23314-1_11
- Authors
- Günnemann, S., Günnemann, N., & Faloutsos, C.
- Affiliations
- Carnegie Mellon University (US)
- Venue
- Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining
- Publisher
- ACM
- Published
- 2014-08-24
- Pages
- 841-850
- OpenAlex
- W2020214545
- Cited by
- 47
- References
- 26
- Citations by year
- 2026: 1 · 2024: 1 · 2023: 1 · 2021: 3 · 2020: 5 · 2019: 5 · 2018: 7 · 2017: 11 · 2016: 10 · 2015: 3
- FWCI
- 4.71
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) (publishedVersion)
- Funders
- German Academic Exchange Service
- U.S. Army Research Laboratory (W911NF-09-2-0053)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Anomaly Detection Techniques and Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Bayesian Methods and Mixture Models
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- anomaly detection, online ratings, Bayesian model, state-space model, variational inference, dynamic programming, time-stamped data
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
