The DOI matches its Crossref record; 68 of the 68 references it cites were found in a registry or an index.
Sentiment Analysis of Short Informal Texts
Open accessAbstract
We describe a state-of-the-art sentiment analysis system that detects (a) the sentiment of short informal textual messages such as tweets and SMS (message-level task) and (b) the sentiment of a word or a phrase within a message (term-level task). The system is based on a supervised statistical text classification approach leveraging a variety of surface-form, semantic, and sentiment features. The sentiment features are primarily derived from novel high-coverage tweet-specific sentiment lexicons. These lexicons are automatically generated from tweets with sentiment-word hashtags and from tweets with emoticons. To adequately capture the sentiment of words in negated contexts, a separate sentiment lexicon is generated for negated words. The system ranked first in the SemEval-2013 shared task `Sentiment Analysis in Twitter' (Task 2), obtaining an F-score of 69.02 in the message-level task and 88.93 in the term-level task. Post-competition improvements boost the performance to an F-score of 70.45 (message-level task) and 89.50 (term-level task). The system also obtains state-of-the-art performance on two additional datasets: the SemEval-2013 SMS test set and a corpus of movie review excerpts. The ablation experiments demonstrate that the use of the automatically generated lexicons results in performance gains of up to 6.5 absolute percentage points.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LIBSVM: A library for support vector machines
Chang, C.-C., & Lin, C.-J. (2011) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/1961189.1961199 · Cited by 40,918
The meaning and use of the area under a receiver operating characteristic (ROC) curve.
Hanley, J. A., & McNeil, B. J. (1982) · Radiology · doi:10.1148/radiology.143.1.7063747 · Cited by 21,856
Libsvm : A library for support vector machines
Ferrari, V. (2008) · Medical Entomology and Zoology · Cited by 10,123
An algorithm for suffix stripping
Porter, M. F. (1980) · Program · doi:10.1108/eb046814 · Cited by 8,096
Mining and summarizing customer reviews
Hu, M., & Liu, B. (2004) · Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1014052.1014073 · Cited by 7,793
Thumbs up?: sentiment classification using machine learning techniques
Pang, B., Lee, L., & Vaithyanathan, S. (2002) · Proceedings of the ACL-02 conference on Empirical methods in natural language processing - EMNLP '02 · doi:10.3115/1118693.1118704 · Cited by 7,024
Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank
Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C. D., Ng, A., & Potts, C. (2013) · Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d13-1170 · Cited by 6,878
Opinion Mining and Sentiment Analysis
Pang, B., & Lee, L. (2008) · now publishers, Inc. eBooks · doi:10.1561/9781601981516 · Cited by 6,720
Recognizing contextual polarity in phrase-level sentiment analysis
Wilson, T., Wiebe, J., & Hoffmann, P. (2005) · Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing - HLT '05 · doi:10.3115/1220575.1220619 · Cited by 3,345
Lexicon-based methods for sentiment analysis
Taboada, M., Brooke, J., Tofiloski, M., Voll, K., & Stede, M. (2011) · Computational Linguistics · doi:10.1162/coli_a_00049 · Cited by 3,301
Twitter as a Corpus for Sentiment Analysis and Opinion Mining
Pak, A., & Paroubek, P. (2010) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/47akcj8foo4d · Cited by 2,704
SentiWordNet 3.0: An Enhanced Lexical Resource for Sentiment Analysis and Opinion Mining
Baccianella, S., Esuli, A., & Sebastiani, F. (2010) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/2jktnrykui9s · Cited by 2,666
SENTIWORDNET: A Publicly Available Lexical Resource for Opinion Mining
Esuli, A., & Sebastiani, F. (2006) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/54vo5wjya4cw · Cited by 2,480
Thumbs up? Sentiment Classification using Machine Learning Techniques
Pang, B., Lee, L., & Vaithyanathan, S. (2002) · arXiv (Cornell University) · doi:10.48550/arxiv.cs/0205070 · Cited by 2,191
Annotating Expressions of Opinions and Emotions in Language
Wiebe, J., Wilson, T., & Cardie, C. (2005) · Language Resources and Evaluation · doi:10.1007/s10579-005-7880-9 · Cited by 1,755
Sentiment Analysis of Twitter Data
Agarwal, A., Xie, B., Vovsha, I., Rambow, O., & Passonneau, R. J. (2011) · Cited by 1,565
Measuring praise and criticism: Inference of semantic orientation from association
Turney, P. D., & Littman, M. L. (2003) · ACM Transactions on Information Systems · doi:10.1145/944012.944013 · Cited by 1,502
Predicting the semantic orientation of adjectives
Hatzivassiloglou, V., & McKeown, K. R. (1997) · Proceedings of the 35th annual meeting on Association for Computational Linguistics - · doi:10.3115/976909.979640 · Cited by 1,439
A Survey of Opinion Mining and Sentiment Analysis
Liu, B., & Zhang, L. (2012) · Mining Text Data · doi:10.1007/978-1-4614-3223-4_13 · Cited by 1,323
Mining the Web for Synonyms: PMI-IR versus LSA on TOEFL
Turney, P. D. (2001) · Lecture Notes in Computer Science · doi:10.1007/3-540-44795-4_42 · Cited by 1,285
Semantic Compositionality through Recursive Matrix-Vector Spaces
Socher, R., Huval, B., Manning, C. D., & Ng, A. Y. (2012) · Cited by 1,259
Twitter Sentiment Analysis: The Good the Bad and the OMG!
Kouloumpis, E., Wilson, T., & Moore, J. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v5i1.14185 · Cited by 1,225
The General Inquirer: A Computer Approach to Content Analysis
Green, B. F., Stone, P. J., Dunphy, D. C., Smith, M. S., & Ogilvie, D. M. (1967) · American Educational Research Journal · doi:10.2307/1161774 · Cited by 1,013
SemEval-2016 Task 4: Sentiment Analysis in Twitter
Nakov, P., Ritter, A., Rosenthal, S., Sebastiani, F., & Stoyanov, V. (2016) · Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016) · doi:10.18653/v1/s16-1001 · Cited by 966
Target-dependent Twitter Sentiment Classification
Jiang, L., Yu, M., Zhou, M., Liu, X., & Zhao, T. (2011) · Cited by 897
Emotions Evoked by Common Words and Phrases: Using Mechanical Turk to Create an Emotion Lexicon
Mohammad, S. M., & Turney, P. D. (2010) · NPARC · Cited by 882
Self-citationThelwall, M., Buckley, K., & Paltoglou, G. (2011) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21462 · Cited by 822
Predicting the semantic orientation of adjectives
Hatzivassiloglou, V., & McKeown, K. R. (1997) · Proceedings of the eighth conference on European chapter of the Association for Computational Linguistics - · doi:10.3115/979617.979640 · Cited by 815
Part-of-Speech Tagging for Twitter: Annotation, Features, and Experiments
Gimpel, K., Schneider, N., O'Connor, B., Das, D., Mills, D. P., Eisenstein, J., Heilman, M., Yogatama, D., Flanigan, J., & Smith, N. A. (2018) · Figshare · doi:10.1184/r1/6473609.v1 · Cited by 800
SENTIMENT CLASSIFICATION of MOVIE REVIEWS USING CONTEXTUAL VALENCE SHIFTERS
Kennedy, A., & Inkpen, D. (2006) · Computational Intelligence · doi:10.1111/j.1467-8640.2006.00277.x · Cited by 751
Enhanced Sentiment Learning Using Twitter Hashtags and Smileys
Davidov, D., Tsur, O., & Rappoport, A. (2010) · Cited by 700
NRC-Canada-2014: Detecting Aspects and Sentiment in Customer Reviews
Kiritchenko, S., Zhu, X., Cherry, C., & Mohammad, S. (2014) · Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014) · doi:10.3115/v1/s14-2076 · Cited by 697
Self-citationSemEval-2014 Task 9: Sentiment Analysis in Twitter
Rosenthal, S., Ritter, A., Nakov, P., & Stoyanov, V. (2014) · Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014) · doi:10.3115/v1/s14-2009 · Cited by 617
Boucher, J., & Osgood, C. E. (1969) · Journal of Verbal Learning and Verbal Behavior · doi:10.1016/s0022-5371(69)80002-2 · Cited by 541
A model of textual affect sensing using real-world knowledge
Liu, H., Lieberman, H., & Selker, T. (2003) · Proceedings of the 8th international conference on Intelligent user interfaces · doi:10.1145/604045.604067 · Cited by 531
Polanyi, L., & Zaenen, A. (2006) · The Information Retrieval Series · doi:10.1007/1-4020-4102-0_1 · Cited by 521
Part-of-Speech Tagging for Twitter: Annotation, Features, and Experiments
Gimpel, K., Schneider, N., O'Connor, B., Das, D., Mills, D., Eisenstein, J., Heilman, M., Yogatama, D., Flanigan, J., & Smith, N. A. (2010) · doi:10.21236/ada547371 · Cited by 491
Using Hashtags to Capture Fine Emotion Categories from Tweets
Mohammad, S. M., & Kiritchenko, S. (2015) · Computational Intelligence · doi:10.1111/coin.12024 · Cited by 408
Self-citationMore accurate tests for the statistical significance of result differences
Yeh, A. (2000) · Proceedings of the 18th conference on Computational linguistics - · doi:10.3115/992730.992783 · Cited by 378
Mohammad, S. M. (2012) · Joint Conference on Lexical and Computational Semantics · Cited by 355
Self-citationMARTÍNEZ-CÁMARA, E., MARTÍN-VALDIVIA, M. T., UREÑA-LÓPEZ, L. A., & MONTEJO-RÁEZ, A. R. (2014) · Natural Language Engineering · doi:10.1017/s1351324912000332 · Cited by 347
The General Inquirer: a computer approach to content analysis
Stone, P. J., Dunphy, D. C., & Smith, M. S. (1966) · CERN Document Server (European Organization for Nuclear Research) · Cited by 344
Learning with compositional semantics as structural inference for subsentential sentiment analysis
Choi, Y., & Cardie, C. (2008) · Proceedings of the Conference on Empirical Methods in Natural Language Processing - EMNLP '08 · doi:10.3115/1613715.1613816 · Cited by 332
A survey on the role of negation in sentiment analysis
Wiegand, M., Balahur, A., Roth, B., Klakow, D., & Montoyo, A. (2010) · Publication Server of the Institute for German Language (Institute for German Language) · Cited by 248
Generating high-coverage semantic orientation lexicons from overtly marked words and a thesaurus
Mohammad, S., Dunne, C., & Dorr, B. (2009) · Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing Volume 2 - EMNLP '09 · doi:10.3115/1699571.1699591 · Cited by 233
Self-citationJoint Inference for Fine-grained Opinion Extraction
Yang, B., & Cardie, C. (2013) · Cited by 222
Brody, S., & Diakopoulos, N. A. (2011) · Cited by 214
The effect of negation on sentiment analysis and retrieval effectiveness
Jia, L., Yu, C., & Meng, W. (2009) · Proceedings of the 18th ACM conference on Information and knowledge management · doi:10.1145/1645953.1646241 · Cited by 209
Measuring Praise and Criticism: Inference of Semantic Orientation from Association
Turney, P. D., & Littman, M. L. (2003) · arXiv (Cornell University) · doi:10.48550/arxiv.cs/0309034 · Cited by 200
A Corpus-based Approach to Finding Happiness
Mihalcea, R. F., & Liu, H. (2006) · Cited by 199
SemEval-2012 Task 2 : measuring degrees of relational similarity
Jurgens, D., Turney, P. D., Mohammad, S. M., & Holyoak, K. J. (2012) · NPARC · Cited by 182
Self-citationSentiment Analysis on Twitter Data
Mohan Kumar, A. V., Suhas, M., & Fedrich, N. (2022) · Cognitive Science and Technology · doi:10.1007/978-981-19-2350-0_43 · Cited by 174
Tracking Sentiment in Mail: How Genders Differ on Emotional Axes
Mohammad, S. M., Tony, & Yang (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1309.6347 · Cited by 171
Self-citationHierarchical Sequential Learning for Extracting Opinions and Their Attributes
Choi, Y., & Cardie, C. (2010) · Cited by 126
Sentiment Classification of Movie and Product Reviews Using Contextual Valence Shifters
Kennedy, A., & Inkpen, D. Z. (2005) · Cited by 122
Affect Analysis Model: novel rule-based approach to affect sensing from text
Neviarouskaya, A., Prendinger, H., & Ishizuka, M. (2010) · Natural Language Engineering · doi:10.1017/s1351324910000239 · Cited by 112
Bakliwal, A., Arora, P., Madhappan, S., Kapre, N., Singh, M. P., & Varma, V. (2012) · Cited by 90
A model of textual affect sensing using real-world knowledge
Liu, H., Lieberman, H. A., & Selker, T. (2003) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/604050.604067 · Cited by 60
Learning the Scope of Negation via Shallow Semantic Parsing
Li, J., Zhou, G., Wang, H., & Zhu, Q. (2010) · Cited by 52
Representing and Resolving Negation for Sentiment Analysis
Lapponi, E., Read, J., & Øvrelid, L. (2012) · IEEE ... International Conference on Data Mining workshops · doi:10.1109/icdmw.2012.23 · Cited by 51
Emotion Analysis Using Latent Affective Folding and Embedding
Bellegarda, J. R. (2013) · Cited by 44
AVAYA: Sentiment Analysis on Twitter with Self-Training and Polarity Lexicon Expansion
Becker, L. A., Erhart, G., Skiba, D., & Matula, V. (2013) · Joint Conference on Lexical and Computational Semantics · Cited by 42
Automated Mark Up of Affective Information in English Texts
Francisco, V., & Gervás, P. (2006) · Lecture notes in computer science · doi:10.1007/11846406_47 · Cited by 26
teragram: Rule-based detection of sentiment phrases using SAS Sentiment Analysis
Reckman, H., Baird, C., Crawford, J., Crowell, R., Micciulla, L., Sethi, S., & Veress, F. (2013) · doi:10.13140/2.1.4108.0004 · Cited by 22
Representing emotional momentum within expressive internet communication
John, D., Boucouvalas, A. C., & Xu, Z. (2006) · Internet, Multimedia Systems and Applications · Cited by 20
Textual and contextual patterns for sentiment analysis over microblogs
Aisopos, F., Papadakis, G., Tserpes, K., & Varvarigou, T. A. (2012) · International World Wide Web Conference (WWW) · doi:10.1145/2187980.2188073 · Cited by 17
Distinguishing affective states in weblogs
Généreux, M., & Evans, R. (2006) · University of Brighton Repository (University of Brighton) · Cited by 16
Identifying purpose behind electoral tweets
Mohammad, S. M., Kiritchenko, S., & Martin, J. (2013) · International Workshop on Issues of Sentiment Discovery and Opinion Mining (WISDOM) · doi:10.1145/2502069.2502070 · Cited by 15
Self-citation
Cited by
Hierarchical Attention Networks for Document Classification
Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., & Hovy, E. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1174 · Cited by 4,712
A survey on sentiment analysis methods, applications, and challenges
Wankhade, M., Rao, A. C. S., & Kulkarni, C. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-022-10144-1 · Cited by 1,528
Document Modeling with Gated Recurrent Neural Network for Sentiment Classification
Tang, D., Qin, B., & Liu, T. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1167 · Cited by 1,519
SemEval-2016 Task 6: Detecting Stance in Tweets
Mohammad, S., Kiritchenko, S., Sobhani, P., Zhu, X., & Cherry, C. (2016) · Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016) · doi:10.18653/v1/s16-1003 · Cited by 844
Self-citationSentiment analysis using deep learning architectures: a review
Yadav, A., & Vishwakarma, D. K. (2020) · Artificial Intelligence Review · doi:10.1007/s10462-019-09794-5 · Cited by 841
Metadata
- Authors
- Kiritchenko, S., Zhu, X., & Mohammad, S. M.
- Venue
- Journal of Artificial Intelligence Research
- Publisher
- AI Access Foundation
- Published
- 2014-08-20
- Volume
- 50
- Pages
- 723-762
- ISSN
- 1076-9757 (ISSN-L)
- OpenAlex
- W2156413587
- Cited by
- 880
- References
- 68
- FWCI
- 93.09
- Type
- journal article
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.