NRC-Canada-2014: Detecting Aspects and Sentiment in Customer Reviews
Open accessThe DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Abstract
Reviews depict sentiments of customers towards various aspects of a product or service. Some of these aspects can be grouped into coarser aspect categories. SemEval-2014 had a shared task (Task 4) on aspect-level sentiment analysis, with over 30 teams participated. In this pa-per, we describe our submissions, which stood first in detecting aspect categories, first in detecting sentiment towards aspect categories, third in detecting aspect terms, and first and second in detecting senti-ment towards aspect terms in the laptop and restaurant domains, respectively. 1
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LIBSVM: A library for support vector machines
Chang, C.-C., & Lin, C.-J. (2011) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/1961189.1961199 · Cited by 40,918 · cited 1×
“To address this multi-class classification problem, we trained a linear SVM classifier using the LibSVM software (Chang and Lin, 2011).”
MethodAn algorithm for suffix stripping
Porter, M. F. (1980) · Program · doi:10.1108/eb046814 · Cited by 8,096 · cited 1×
“Sentences were tokenized and stemmed with Porter stemmer (Porter, 1980).”
MethodMining and summarizing customer reviews
Hu, M., & Liu, B. (2004) · Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1014052.1014073 · Cited by 7,793 · cited 2×
“In addition, we employed existing out-ofdomain sentiment lexicons: (1) large-coverage automatic tweet sentiment lexicons, Hashtag Sentiment lexicons and Sentiment140 lexicons (Kiritchenko et al., 2014), and (2) three manually created sentiment lexicons, NRC Emotion Lexicon (Mohammad and Turney, 2010), Bing Liu’s Lexic…”
BackgroundRecognizing contextual polarity in phrase-level sentiment analysis
Wilson, T., Wiebe, J., & Hoffmann, P. (2005) · Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing - HLT '05 · doi:10.3115/1220575.1220619 · Cited by 3,345 · cited 2×
“In addition, we employed existing out-ofdomain sentiment lexicons: (1) large-coverage automatic tweet sentiment lexicons, Hashtag Sentiment lexicons and Sentiment140 lexicons (Kiritchenko et al., 2014), and (2) three manually created sentiment lexicons, NRC Emotion Lexicon (Mohammad and Turney, 2010), Bing Liu’s Lexic…”
Background(1992) · doi:10.5555/176313.176316 · Cited by 2,892
On the algorithmic implementation of multiclass kernel-based vector machines
Crammer, K., & Singer, Y. (2002) · Journal of Machine Learning Research · Cited by 2,311
Generating Typed Dependency Parses from Phrase Structure Parses
de Marneffe, M.-C., MacCartney, B., & Manning, C. D. (2006) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/3wtkaidk6ea8 · Cited by 2,258 · cited 1×
“Sentences were first tokenized and parsed with the Stanford CoreNLP toolkits2 to obtain part-ofspeech (POS) tags and (collapsed) typed dependency parse trees (de Marneffe et al., 2006).”
MethodCollins, M. (2002) · Proceedings of the ACL-02 conference on Empirical methods in natural language processing - EMNLP '02 · doi:10.3115/1118693.1118694 · Cited by 1,862
SemEval-2014 Task 4: Aspect Based Sentiment Analysis
Pontiki, M., Galanis, D., Pavlopoulos, J., Papageorgiou, H., Androutsopoulos, I., & Manandhar, S. (2014) · Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014) · doi:10.3115/v1/s14-2004 · Cited by 1,862 · cited 1×
“Further details of the task and data can be found in the task description paper (Pontiki et al., 2014).”
MethodOnline Passive-Aggressive Algorithms
Crammer, K., Dekel, O., Keshet, J., Shalev‐Shwartz, S., & Singer, Y. (2003) · Cited by 1,665
Hidden factors and hidden topics: understanding rating dimensions with review text
McAuley, J., & Leskovec, J. (2013) · Proceedings of the 7th ACM conference on Recommender systems · doi:10.1145/2507157.2507163 · Cited by 1,564 · cited 1×
“Amazon laptop reviews corpus: McAuley and Leskovec (2013) collected reviews posted on Amazon.”
BackgroundMeasuring praise and criticism: Inference of semantic orientation from association
Turney, P. D., & Littman, M. L. (2003) · ACM Transactions on Information Systems · doi:10.1145/944012.944013 · Cited by 1,502
Frustratingly Easy Domain Adaptation
Daumé, H. (2009) · arXiv (Cornell University) · doi:10.48550/arxiv.0907.1815 · Cited by 1,383
Emotions Evoked by Common Words and Phrases: Using Mechanical Turk to Create an Emotion Lexicon
Mohammad, S. M., & Turney, P. D. (2010) · NPARC · Cited by 1,318
Self-citationSentiment Analysis of Short Informal Texts
Kiritchenko, S., Zhu, X., & Mohammad, S. M. (2014) · Journal of Artificial Intelligence Research · doi:10.1613/jair.4272 · Cited by 880 · cited 3×
“We built two lexicons, Yelp Restaurant Sentiment AffLex and Yelp Restaurant Sentiment NegLex, as described in (Kiritchenko et al., 2014).”
BackgroundMethodSelf-citationPart-of-Speech Tagging for Twitter: Annotation, Features, and Experiments
Gimpel, K., Schneider, N., O'Connor, B., Das, D., Mills, D. P., Eisenstein, J., Heilman, M., Yogatama, D., Flanigan, J., & Smith, N. A. (2018) · Figshare · doi:10.1184/r1/6473609.v1 · Cited by 800
Improved Part-of-Speech Tagging for Online Conversational Text with Word Clusters
Owoputi, O., O'Connor, B., Dyer, C., Gimpel, K., Schneider, N., & Smith, N. A. (2018) · Figshare · doi:10.1184/r1/6473408 · Cited by 662
(2004) · Cited by 610
NRC-Canada: Building the State-of-the-Art in Sentiment Analysis of Tweets
Mohammad, S. M., Kiritchenko, S., & Zhu, X. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1308.6242 · Cited by 451
Self-citationde Bruijn, B., Cherry, C., Kiritchenko, S., Martin, J., & Zhu, X. (2011) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000150 · Cited by 235
Self-citationAn Empirical Study on the Effect of Negation Words on Sentiment
Zhu, X., Guo, H., Mohammad, S. M., & Kiritchenko, S. (2014) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/v1/p14-1029 · Cited by 93 · cited 2×
“Negation words (e.g., not, never) can significantly affect the sentiment of an expression (Zhu et al., 2014).”
BackgroundSelf-citation
Research works printed but not linked: 2
- Sunita Sarawagi and William W Cohen. 2004. Semimarkov conditional random fields for information extraction. In Advances in Neural Information Processing Systems , volume 17, pages 1185-1192.
- Theresa Wilson, Janyce Wiebe, and Paul Hoffmann. 2005. Recognizing contextual polarity in phraselevel sentiment analysis. In Proceedings of the Conference on Human Language Technology and Empirical Methods in Natural Language Processing , HLT '05, pages 347-354, Stroudsburg, PA, USA.
Cited by
Aspect Level Sentiment Classification with Deep Memory Network
Tang, D., Qin, B., & Liu, T. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1021 · Cited by 1,041
“(2) Feature-based SVM performs state-of-the-art on aspect level sentiment classification.”
BackgroundMethodInfluentialRecurrent Attention Network on Memory for Aspect Sentiment Analysis
Chen, P., Sun, Z., Bing, L., & Yang, W. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1047 · Cited by 995
“The feature-based SVM, which needs labor-intensive feature engineering works and a mass of extra linguistic resources, doesn’t display its advantage, because the features for aspect sentiment analysis cannot be extracted as easily as for sentence or document level sentiment analysis.”
MethodSentiment Analysis of Short Informal Texts
Kiritchenko, S., Zhu, X., & Mohammad, S. M. (2014) · Journal of Artificial Intelligence Research · doi:10.1613/jair.4272 · Cited by 880
“In (Kiritchenko et al., 2014) we show how our approach can be adapted to identify the sentiment for a specified target.”
BackgroundMethodInfluentialSelf-citationSemEval-2016 Task 6: Detecting Stance in Tweets
Mohammad, S., Kiritchenko, S., Sobhani, P., Zhu, X., & Cherry, C. (2016) · Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016) · doi:10.18653/v1/s16-1003 · Cited by 844
“Most teams used standard text classification features such as n -grams and word embedding vectors, as well as standard sentiment analysis features such as those drawn from sentiment lexicons (Kiritchenko et al., 2014b).”
BackgroundMethodSelf-citationAspect-based Sentiment Classification with Aspect-specific Graph Convolutional Networks
Zhang, C., Li, Q., & Song, D. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1464 · Cited by 712
“In order to comprehensively evaluate the two variants of our model, namely, ASGCN-DG and ASGCN-DT, we compare them with a range of baselines and state-of-the-art models, as listed below: • SVM (Kiritchenko et al., 2014) is the model which has won SemEval 2014 task 4 with conventional feature extraction methods.”
Method
Metadata
- Authors
- Kiritchenko, S., Zhu, X., Cherry, C., & Mohammad, S.
- Affiliations
- National Research Council Canada (CA)
- Venue
- Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014)
- Published
- 2014
- Pages
- 437-442
- OpenAlex
- W2252057809
- Cited by
- 697
- References
- 21
- Citations by year
- 2026: 15 · 2025: 32 · 2024: 53 · 2023: 70 · 2022: 94 · 2021: 100 · 2020: 87 · 2019: 106 · 2018: 57 · 2017: 28 · 2016: 30 · 2015: 20 · 2014: 5
- FWCI
- 16.29
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- International Workshop on Semantic Evaluation (SemEval) (publishedVersion, cc-by)
- NPARC (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Keywords
- aspect-based sentiment analysis, aspect category detection, customer reviews, laptop review, restaurant reviews
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.