Extracting product features and opinions from reviews
Open accessThe DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Abstract
Consumers are often forced to wade through many on-line reviews in order to make an informed product choice. This paper introduces Opine, an unsupervised information-extraction system which mines reviews in order to build a model of important product features, their evaluation by reviewers, and their relative quality across products.Compared to previous work, Opine achieves 22% higher precision (with only 3% lower recall) on the feature extraction task. Opine's novel use of relaxation labeling for finding the semantic orientation of words in context leads to strong performance on the tasks of finding opinion phrases and their polarity.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Mining and summarizing customer reviews
Hu, M., & Liu, B. (2004) · Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1014052.1014073 · Cited by 7,793
Thumbs up?: sentiment classification using machine learning techniques
Pang, B., Lee, L., & Vaithyanathan, S. (2002) · Proceedings of the ACL-02 conference on Empirical methods in natural language processing - EMNLP '02 · doi:10.3115/1118693.1118704 · Cited by 7,024
Thumbs Up or Thumbs Down? Semantic Orientation Applied to Unsupervised Classification of Reviews
Peter, P., & Turney (2002) · Cited by 3,650
Recognizing contextual polarity in phrase-level sentiment analysis
Wilson, T., Wiebe, J., & Hoffmann, P. (2005) · Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing - HLT '05 · doi:10.3115/1220575.1220619 · Cited by 3,345
Thumbs up or thumbs down?: semantic orientation applied to unsupervised classification of reviews
Turney, P. D. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073153 · Cited by 2,352
Thumbs up? Sentiment Classification using Machine Learning Techniques
Pang, B., Lee, L., & Vaithyanathan, S. (2002) · arXiv (Cornell University) · doi:10.48550/arxiv.cs/0205070 · Cited by 2,191
Mining the peanut gallery: opinion extraction and semantic classification of product reviews
Dave, K., Lawrence, S., & Pennock, D. M. (2003) · Proceedings of the twelfth international conference on World Wide Web - WWW '03 · doi:10.1145/775152.775226 · Cited by 1,916
Thumbs Up or Thumbs Down? Semantic Orientation Applied to Unsupervised Classification of Reviews
Turney, P. D. (2002) · arXiv (Cornell University) · doi:10.48550/arxiv.cs/0212032 · Cited by 1,575
Measuring praise and criticism: Inference of semantic orientation from association
Turney, P. D., & Littman, M. L. (2003) · ACM Transactions on Information Systems · doi:10.1145/944012.944013 · Cited by 1,502
Determining the sentiment of opinions
Kim, S.-M., & Hovy, E. (2004) · Proceedings of the 20th international conference on Computational Linguistics - COLING '04 · doi:10.3115/1220355.1220555 · Cited by 1,482
Cited by
Opinion Mining and Sentiment Analysis
Pang, B., & Lee, L. (2008) · now publishers, Inc. eBooks · doi:10.1561/9781601981516 · Cited by 6,720
Sentiment Analysis and Opinion Mining
Liu, B. (2012) · Synthesis Lectures on Human Language Technologies · doi:10.1007/978-3-031-02145-9 · Cited by 3,281
Sentiment Analysis and Opinion Mining
Zhang, L., & Liu, B. (2017) · Encyclopedia of Machine Learning and Data Mining · doi:10.1007/978-1-4899-7687-1_907 · Cited by 3,260
SemEval-2014 Task 4: Aspect Based Sentiment Analysis
Pontiki, M., Galanis, D., Pavlopoulos, J., Papageorgiou, H., Androutsopoulos, I., & Manandhar, S. (2014) · Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014) · doi:10.3115/v1/s14-2004 · Cited by 1,862
Hidden factors and hidden topics: understanding rating dimensions with review text
McAuley, J., & Leskovec, J. (2013) · Proceedings of the 7th ACM conference on Recommender systems · doi:10.1145/2507157.2507163 · Cited by 1,564
Jindal, N., & Liu, B. (2008) · Proceedings of the international conference on Web search and web data mining - WSDM '08 · doi:10.1145/1341531.1341560 · Cited by 1,542
Techniques and applications for sentiment analysis
Feldman, R. (2013) · Communications of the ACM · doi:10.1145/2436256.2436274 · Cited by 1,489
A holistic lexicon-based approach to opinion mining
Ding, X., Liu, B., & Yu, P. S. (2008) · Proceedings of the international conference on Web search and web data mining - WSDM '08 · doi:10.1145/1341531.1341561 · Cited by 1,386
A Survey of Opinion Mining and Sentiment Analysis
Liu, B., & Zhang, L. (2012) · Mining Text Data · doi:10.1007/978-1-4614-3223-4_13 · Cited by 1,323
A survey on opinion mining and sentiment analysis: Tasks, approaches and applications
Ravi, K., & Ravi, V. (2015) · Knowledge-Based Systems · doi:10.1016/j.knosys.2015.06.015 · Cited by 1,306
New Avenues in Opinion Mining and Sentiment Analysis
Cambria, E., Schuller, B., Xia, Y., & Havasi, C. (2013) · IEEE Intelligent Systems · doi:10.1109/mis.2013.30 · Cited by 1,285
Opinion Word Expansion and Target Extraction through Double Propagation
Qiu, G., Liu, B., Bu, J., & Chen, C. (2011) · Computational Linguistics · doi:10.1162/coli_a_00034 · Cited by 1,074
Using linguistic cues for the automatic recognition of personality in conversation and text
Mairesse, F., Walker, M. A., Mehl, M. R., & Moore, R. K. (2007) · Journal of Artificial Intelligence Research · doi:10.1613/jair.2349 · Cited by 1,046
“... Girand, Graciarena, Kathol, Michaelis, Pellom, Shriberg, & Stolcke, 2005), speaker charisma (Rosenberg & Hirschberg, 2005), mood (Mishne, 2005), dominance in meetings (Rienks & Heylen, 2006), point of view or subjectivity (Wilson, Wiebe, & Hwa, 2004; Wiebe, Wilson, Bruce, Bell, & Martin, 2004; Wiebe & Rilo, 2005;…”
Open information extraction from the web
Etzioni, O., Banko, M., Soderland, S., & Weld, D. S. (2008) · Communications of the ACM · doi:10.1145/1409360.1409378 · Cited by 1,012
Self-citationDeriving the Pricing Power of Product Features by Mining Consumer Reviews
Archak, N., Ghose, A., & Ipeirotis, P. G. (2011) · Management Science · doi:10.1287/mnsc.1110.1370 · Cited by 1,007
Movie review mining and summarization
Zhuang, L., Jing, F., & Zhu, X.-Y. (2006) · Proceedings of the 15th ACM international conference on Information and knowledge management - CIKM '06 · doi:10.1145/1183614.1183625 · Cited by 894
Aspect extraction for opinion mining with a deep convolutional neural network
Poria, S., Cambria, E., & Gelbukh, A. (2016) · Knowledge-Based Systems · doi:10.1016/j.knosys.2016.06.009 · Cited by 861
Survey on Aspect-Level Sentiment Analysis
Schouten, K., & Frasincar, F. (2016) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2015.2485209 · Cited by 832
Modeling online reviews with multi-grain topic models
Titov, I., & McDonald, R. (2008) · Proceedings of the 17th international conference on World Wide Web · doi:10.1145/1367497.1367513 · Cited by 778
Aspect and sentiment unification model for online review analysis
Jo, Y., & Oh, A. H. (2011) · Proceedings of the fourth ACM international conference on Web search and data mining · doi:10.1145/1935826.1935932 · Cited by 767
Schuller, B., Batliner, A., Steidl, S., & Seppi, D. (2011) · Speech Communication · doi:10.1016/j.specom.2011.01.011 · Cited by 758
Improving sentiment analysis via sentence type classification using BiLSTM-CRF and CNN
Chen, T., Xu, R., He, Y., & Wang, X. (2017) · Expert Systems with Applications · doi:10.1016/j.eswa.2016.10.065 · Cited by 730
Recognizing Contextual Polarity: An Exploration of Features for Phrase-Level Sentiment Analysis
Wilson, T., Wiebe, J., & Hoffmann, P. (2009) · Computational Linguistics · doi:10.1162/coli.08-012-r1-06-90 · Cited by 722
Detecting product review spammers using rating behaviors
Lim, E.-P., Nguyen, V.-A., Jindal, N., Liu, B., & Lauw, H. W. (2010) · Proceedings of the 19th ACM international conference on Information and knowledge management · doi:10.1145/1871437.1871557 · Cited by 715
Sarawagi, S. (2008) · Foundations and Trends in Databases · doi:10.1561/1900000003 · Cited by 707
Latent aspect rating analysis on review text data: a rating regression approach
Wang, H., Lu, Y., & Zhai, C. (2010) · Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1835804.1835903 · Cited by 650
Ghose, A., Ipeirotis, P. G., & Li, B. (2012) · Marketing Science · doi:10.1287/mksc.1110.0700 · Cited by 649
Inferring Networks of Substitutable and Complementary Products
McAuley, J., Pandey, R., & Leskovec, J. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2783381 · Cited by 645
“A variety of works have used topic models within recommender systems ( e.g. [6, 10, 11, 22, 23, 28, 31, 32]), though generally with the goal of predicting user ratings (or opinions) rather than learning relationships between products as we do here.”
BackgroundHow Do Users Like This Feature? A Fine Grained Sentiment Analysis of App Reviews
Guzman, E., & Maalej, W. (2014) · 2014 IEEE 22nd International Requirements Engineering Conference (RE) · doi:10.1109/re.2014.6912257 · Cited by 590
A review of natural language processing techniques for opinion mining systems
Sun, S., Luo, C., & Chen, J. (2017) · Information Fusion · doi:10.1016/j.inffus.2016.10.004 · Cited by 542
Opinion Mining and Sentiment Analysis
Liu, B. (2011) · Web Data Mining · doi:10.1007/978-3-642-19460-3_11 · Cited by 482
Extracting opinions, opinion holders, and topics expressed in online news media text
Kim, S.-M., & Hovy, E. (2006) · Proceedings of the Workshop on Sentiment and Subjectivity in Text - SST '06 · doi:10.3115/1654641.1654642 · Cited by 460
Bug report, feature request, or simply praise? On automatically classifying app reviews
Maalej, W., & Nabil, H. (2015) · 2015 IEEE 23rd International Requirements Engineering Conference (RE) · doi:10.1109/re.2015.7320414 · Cited by 453
Jindal, N., & Liu, B. (2007) · Proceedings of the 16th international conference on World Wide Web · doi:10.1145/1242572.1242759 · Cited by 449
Identifying comparative sentences in text documents
Jindal, N., & Liu, B. (2006) · Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/1148170.1148215 · Cited by 446
Recursive Neural Conditional Random Fields for Aspect-based Sentiment Analysis
Wang, W., Pan, S. J., Dahlmeier, D., & Xiao, X. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1059 · Cited by 435
Aspect-Level Sentiment Analysis Via Convolution over Dependency Tree
Sun, K., Zhang, R., Mensah, S., Mao, Y., & Liu, X. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1569 · Cited by 430
Coupled Multi-Layer Attentions for Co-Extraction of Aspect and Opinion Terms
Wang, W., Pan, S. J., Dahlmeier, D., & Xiao, X. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10974 · Cited by 425
Double Embeddings and CNN-based Sequence Labeling for Aspect Extraction
Xu, H., Liu, B., Shu, L., & Yu, P. S. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/p18-2094 · Cited by 414
Fully automatic lexicon expansion for domain-oriented sentiment analysis
Kanayama, H., & Nasukawa, T. (2006) · Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing - EMNLP '06 · doi:10.3115/1610075.1610125 · Cited by 408
Recommender systems based on user reviews: the state of the art
Chen, L., Chen, G., & Wang, F. (2015) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-015-9155-5 · Cited by 403
Mohammad, S. M. (2016) · Emotion Measurement · doi:10.1016/b978-0-08-100508-8.00009-6 · Cited by 391
Sentiment Analysis: Mining Opinions, Sentiments, and Emotions
Liu, B. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108639286 · Cited by 362
Phrase dependency parsing for opinion mining
Wu, Y., Zhang, Q., Huang, X., & Wu, L. (2009) · Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing Volume 3 - EMNLP '09 · doi:10.3115/1699648.1699700 · Cited by 361
Tsur, O., Davidov, D., & Rappoport, A. (2010) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v4i1.14018 · Cited by 347
Estimating aggregate consumer preferences from online product reviews
Decker, R., & Trusov, M. (2010) · International Journal of Research in Marketing · doi:10.1016/j.ijresmar.2010.09.001 · Cited by 341
Al-Smadi, M., Qawasmeh, O., Al-Ayyoub, M., Jararweh, Y., & Gupta, B. (2018) · Journal of Computational Science · doi:10.1016/j.jocs.2017.11.006 · Cited by 339
Senti-lexicon and improved Naïve Bayes algorithms for sentiment analysis of restaurant reviews
Kang, H., Yoo, S. J., & Han, D. (2012) · Expert Systems with Applications · doi:10.1016/j.eswa.2011.11.107 · Cited by 336
“Popescu and Etzioni (2005) found a feature by using the same method as Hu and Liu (2004) and, using it as the center, they found a sentiment word with a rule that could express the sentiment word.”
MethodLatent aspect rating analysis without aspect keyword supervision
Wang, H., Lu, Y., & Zhai, C. (2011) · Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/2020408.2020505 · Cited by 327
Recent Named Entity Recognition and Classification techniques: A systematic review
Goyal, A., Gupta, V., & Kumar, M. (2018) · Computer Science Review · doi:10.1016/j.cosrev.2018.06.001 · Cited by 309
Gated Neural Networks for Targeted Sentiment Analysis
Zhang, M., Zhang, Y., & Vo, D.-T. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10380 · Cited by 304
Recognizing stances in online debates
Somasundaran, S., & Wiebe, J. (2009) · Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 1 - ACL-IJCNLP '09 · doi:10.3115/1687878.1687912 · Cited by 303
Wiebe, J., & Mihalcea, R. (2006) · Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the ACL - ACL '06 · doi:10.3115/1220175.1220309 · Cited by 296
Mining opinions in comparative sentences
Ganapathibhotla, M., & Liu, B. (2008) · Proceedings of the 22nd International Conference on Computational Linguistics - COLING '08 · doi:10.3115/1599081.1599112 · Cited by 295
Detection of review spam: A survey
Heydari, A., Tavakoli, M. A., Salim, N., & Heydari, Z. (2015) · Expert Systems with Applications · doi:10.1016/j.eswa.2014.12.029 · Cited by 290
Social media data analytics to improve supply chain management in food industries
Singh, A., Shukla, N., & Mishra, N. (2018) · Transportation Research Part E: Logistics and Transportation Review · doi:10.1016/j.tre.2017.05.008 · Cited by 286
Aspect Term Extraction with History Attention and Selective Transformation
Li, X., Bing, L., Li, P., Lam, W., & Yang, Z. (2018) · Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/583 · Cited by 284
On the automatic classification of app reviews
Maalej, W., Kurtanović, Z., Nabil, H., & Stanik, C. (2016) · Requirements Engineering · doi:10.1007/s00766-016-0251-9 · Cited by 280
Sentiment, emotion, purpose, and style in electoral tweets
Mohammad, S. M., Zhu, X., Kiritchenko, S., & Martin, J. (2015) · Information Processing & Management · doi:10.1016/j.ipm.2014.09.003 · Cited by 274
Which side are you on?: identifying perspectives at the document and sentence levels
Lin, W.-H., Wilson, T., Wiebe, J., & Hauptmann, A. (2006) · Proceedings of the Tenth Conference on Computational Natural Language Learning - CoNLL-X '06 · doi:10.3115/1596276.1596297 · Cited by 269
Kang, D., & Park, Y. (2014) · Expert Systems with Applications · doi:10.1016/j.eswa.2013.07.101 · Cited by 268
OpinionMiner: a novel machine learning system for web opinion mining and extraction
Jin, W., Ho, H. H., & Srihari, R. K. (2009) · Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1557019.1557148 · Cited by 267
A survey of the applications of text mining in financial domain
Kumar, B. S., & Ravi, V. (2016) · Knowledge-Based Systems · doi:10.1016/j.knosys.2016.10.003 · Cited by 265
Feature subsumption for opinion analysis
Riloff, E., Patwardhan, S., & Wiebe, J. (2006) · Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing - EMNLP '06 · doi:10.3115/1610075.1610137 · Cited by 261
Clustering product features for opinion mining
Zhai, Z., Liu, B., Xu, H., & Jia, P. (2011) · Proceedings of the fourth ACM international conference on Web search and data mining · doi:10.1145/1935826.1935884 · Cited by 258
Opinion Mining and Information Fusion: A survey
Balazs, J. A., & Velásquez, J. D. (2016) · Information Fusion · doi:10.1016/j.inffus.2015.06.002 · Cited by 253
Sentiment analysis on tweets for social events
Zhou, X., Tao, X., Yong, J., & Yang, Z. (2013) · Proceedings of the 2013 IEEE 17th International Conference on Computer Supported Cooperative Work in Design (CSCWD) · doi:10.1109/cscwd.2013.6581022 · Cited by 124
Identifying helpful reviews based on customer’s mentions about experiences
Min, H.-J., & Park, J. C. (2012) · Expert Systems with Applications · doi:10.1016/j.eswa.2012.01.116 · Cited by 66
AL-Smadi, M., Hammad, M. M., Al-Zboon, S. A., AL-Tawalbeh, S., & Cambria, E. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.107540 · Cited by 62
“It started with earlier stages of traditional methods such as association mining as in [21,22], and Likelihood ratio as in [23] as unsupervised machine learning techniques.”
A random walk algorithm for automatic construction of domain-oriented sentiment lexicon
Tan, S., & Wu, Q. (2011) · Expert Systems with Applications · doi:10.1016/j.eswa.2011.02.105 · Cited by 55
Feature-Based Visual Sentiment Analysis of Text Document Streams
Rohrdantz, C., Hao, M. C., Dayal, U., Haug, L.-E., & Keim, D. A. (2012) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/2089094.2089102 · Cited by 43
Generating Product Descriptions from User Reviews
Novgorodov, S., Guy, I., Elad, G., & Radinsky, K. (2019) · The World Wide Web Conference · doi:10.1145/3308558.3313532 · Cited by 41
“Most studies focus on identifying the key attributes of an entity, such as a product, a movie, or a hotel, and then extracting key phrases that describe these attributes or the sentiment towards them (e.g., [26, 37, 56, 73]).”
BackgroundRevMiner: an extractive interface for navigating reviews on a smartphone
Huang, J., Etzioni, O., Zettlemoyer, L., Clark, K., & Lee, C. (2012) · Proceedings of the 25th annual ACM symposium on User interface software and technology · doi:10.1145/2380116.2380120 · Cited by 37
“For instance, the OPINE system introduced an unsupervised method for identifying product features and the associated opinions (including their strength and polarity) [13].”
BackgroundMethodSelf-citation
Metadata
- Authors
- Popescu, A.-M., & Etzioni, O.
- Venue
- Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing - HLT '05
- Published
- 2005
- Pages
- 339-346
- OpenAlex
- W2081375810
- Cited by
- 1,698
- References
- 28
- FWCI
- 33.39
- Type
- conference paper
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.