MRR: an unsupervised algorithm to rank reviews by relevance
The DOI matches its Crossref record; 24 of the 24 references it cites were found in a registry or an index.
Abstract
The automatic detection of relevant reviews plays a major role in tasks such as opinion summarization, opinion-based recommendation, and opinion retrieval. Supervised approaches for ranking reviews by relevance rely on the existence of a significant, domain-dependent training data set. In this work, we propose MRR (Most Relevant Reviews), a new unsupervised algorithm that identifies relevant revisions based on the concept of graph centrality. The intuition behind MRR is that central reviews highlight aspects of a product that many other reviews frequently mention, with similar opinions, as expressed in terms of ratings. MRR constructs a graph where nodes represent reviews, which are connected by edges when a minimum similarity between a pair of reviews is observed, and then employs PageRank to compute the centrality. The minimum similarity is graph-specific, and takes into account how reviews are written in specific domains. The similarity function does not require extensive pre-processing, thus reducing the computational cost. Using reviews from books and electronics products, our approach has outperformed the two unsupervised baselines and shown a comparable performance with two supervised regression models in a specific setting. MRR has also achieved a significantly superior run-time performance in a comparison with the unsupervised baselines.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
(1999) · Cited by 12,854
Authoritative sources in a hyperlinked environment
Kleinberg, J. M. (1999) · Journal of the ACM · doi:10.1145/324133.324140 · Cited by 8,959 · cited 1×
“Popular algorithms to calculate node centrality are PageRank [12], and HITS [6].”
Background(2010) · Introduction to Combinatorics · doi:10.1201/b12336-12 · Cited by 5,632
Cumulated gain-based evaluation of IR techniques
Järvelin, K., & Kekäläinen, J. (2002) · ACM Transactions on Information Systems · doi:10.1145/582415.582418 · Cited by 4,841 · cited 1×
“Since each product that composes the gold standard has a distinct helpfulness score, we used the Normalized Discounted Cumulative Gain (NDCG) [3] measure as the evaluation metric of our ranking task.”
MethodWhat Makes a Helpful Online Review? A Study of Customer Reviews on Amazon.Com1
Mudambi, S. M., & Schuff, D. (2010) · MIS Quarterly · doi:10.2307/20721420 · Cited by 2,490
What makes a helpful online review? a study of customer reviews on amazon.com
Mudambi, S. M., & Schuff, D. (2010) · MIS Quarterly · doi:10.5555/2017447.2017457 · Cited by 1,281
Inferring Networks of Substitutable and Complementary Products
McAuley, J., Pandey, R., & Leskovec, J. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2783381 · Cited by 645 · cited 1×
“The dataset comprises reviews of electronic products and books extracted from the Amazon website [9].”
BackgroundAutomatically assessing review helpfulness
Kim, S.-M., Pantel, P., Chklovski, T., & Pennacchiotti, M. (2006) · Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing - EMNLP '06 · doi:10.3115/1610075.1610135 · Cited by 518 · cited 4×
“Previous work on automatic review helpfulness assessment [5, 10] model the relevance as the proportion of users who have given a positive vote about the total of given votes (positive and negative votes).”
BackgroundMethodInfluentialSurvey on mining subjective data on the web
Tsytsarau, M., & Palpanas, T. (2012) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-011-0238-6 · Cited by 443 · cited 1×
“The automatic detection of relevant reviews is an important problem, which plays a major role in tasks such as opinion summarization [7], opinion-based recommendation [1], and opinion-mining [15].”
Background
Research works printed but not linked: 3
- 10.5555/1610075.1610135 · doi:10.5555/1610075.1610135
- 10.5555/2390524.2390570 · doi:10.5555/2390524.2390570
- Frank Wilcoxon , SK Katti , and Roberta A Wilcox . 1970. Critical values and probability levels for the Wilcoxon rank sum test and the Wilcoxon signed rank test. Selected tables in mathematical statistics 1 ( 1970 ), 171--259. Frank Wilcoxon, SK Katti, and Roberta A Wilcox. 1970. Critical values and probability levels for the Wilcoxon rank sum test and the Wilcoxon signed rank test. Selected tables in mathematical statistics 1 (1970), 171--259.
Cited by
DDC-Outlier: Preventing Medication Errors Using Unsupervised Learning
dos Santos, H. D. P., Ulbrich, A. H. D. P. S., Woloszyn, V., & Vieira, R. L. R. (2018) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2018.2828028 · Cited by 79
“The same idea was successfully employed, for instance, to rank textual information of documents and automatically create textual summaries that contain the most common words based on their centrality on a graph [20], [21].”
MethodSelf-citationWang, R., Chen, Z., Zhang, M. R., Li, Z., Liu, Z., Dang, Z., Yu, C., & Chen, X. '. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445547 · Cited by 42
“The next level lower are reviews that contain the query keywords but do not meet any rules, within which they are ranked by relevance based on the concept of graph centrality following prior work [37].”
BackgroundDistrustRank: Spotting False News Domains
Woloszyn, V., & Nejdl, W. (2018) · ACM Web Science Conference (WebSci) · doi:10.1145/3201064.3201083 · Cited by 20
“However, setting an appropriate threshold is a tricky problem [19].”
BackgroundSelf-citationIs there Gender bias and stereotype in Portuguese Word Embeddings?Preprint
Santana, B. S., Woloszyn, V., & Wives, L. K. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.04528 · Cited by 16
“On the other side of the spectrum, there are Unsupervised and semi-supervised learning techniques, that are attractive because they do not imply the cost of corpus annotation [16,14,17,15].”
BackgroundSelf-citationBlogSet-BR: A Brazilian Portuguese Blog Corpus
dos Santos, H. D. P., Woloszyn, V., & Vieira, R. (2018) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/34xgyquk7mxj · Cited by 14
“Such data sets allow the accomplishment of many different tasks, such as relevance ranking for online documents and several other machine learning tasks(Woloszyn et al., 2016; Woloszyn et al., 2017).”
BackgroundSelf-citation
Metadata
- Authors
- Woloszyn, V., dos Santos, H. D. P., Wives, L. K., & Becker, K.
- Affiliations
- Universidade Federal do Rio Grande do Sul (BR)
- Pontificia Universidade, Catolica do Rio Grande do Sul, Porto Alegre, Rio Grande do Sul, Brazil
- Venue
- Proceedings of the International Conference on Web Intelligence
- Publisher
- ACM
- Published
- 2017-08-23
- Pages
- 877-883
- OpenAlex
- W2744303556
- Cited by
- 9
- References
- 23
- Citations by year
- 2026: 1 · 2021: 3 · 2018: 4
- FWCI
- 0.50
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE/WIC/ACM International Conference on Web Intelligence (publishedVersion)
- Funders
- CAPES and CNPq, Brazil (459322/2014-1)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Physical Sciences › Computer Science › Information Systems › Recommender Systems and Techniques
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Keywords
- graph centrality, PageRank, opinion retrieval, opinion summarization, product reviews
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: MRR: an unsupervised algorithm to rank reviews by relevance. https://app.cochat.ai/papers/10.1145/3106426.3106444
- The paper
- Woloszyn, V., dos Santos, H. D. P., Wives, L. K., & Becker, K. (2017). MRR: an unsupervised algorithm to rank reviews by relevance. Proceedings of the International Conference on Web Intelligence. https://doi.org/10.1145/3106426.3106444
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.