SemEval-2014 Task 9: Sentiment Analysis in Twitter
Open accessThe DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
Abstract
We describe the Sentiment Analysis in Twitter task, ran as part of SemEval-2014.It is a continuation of the last year's task that ran successfully as part of SemEval-2013.As in 2013, this was the most popular SemEval task; a total of 46 teams contributed 27 submissions for subtask A (21 teams) and 50 submissions for subtask B (44 teams).This year, we introduced three new test sets: (i) regular tweets, (ii) sarcastic tweets, and (iii) LiveJournal sentences.We further tested on (iv) 2013 tweets, and (v) 2013 SMS messages.The highest F1score on (i) was achieved by NRC-Canada at 86.63 for subtask A and by TeamX at 70.96 for subtask B.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Thumbs up?: sentiment classification using machine learning techniques
Pang, B., Lee, L., & Vaithyanathan, S. (2002) · Proceedings of the ACL-02 conference on Empirical methods in natural language processing - EMNLP '02 · doi:10.3115/1118693.1118704 · Cited by 7,024
SentiWordNet 3.0: An Enhanced Lexical Resource for Sentiment Analysis and Opinion Mining
Baccianella, S., Esuli, A., & Sebastiani, F. (2010) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/2jktnrykui9s · Cited by 2,666
Predicting Elections with Twitter: What 140 Characters Reveal about Political Sentiment
Tumasjan, A., Sprenger, T., Sandner, P., & Welpe, I. (2010) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v4i1.14009 · Cited by 2,649
Thumbs up? Sentiment Classification using Machine Learning Techniques
Pang, B., Lee, L., & Vaithyanathan, S. (2002) · arXiv (Cornell University) · doi:10.48550/arxiv.cs/0205070 · Cited by 2,191
From Tweets to Polls: Linking Text Sentiment to Public Opinion Time Series
O'Connor, B., Balasubramanyan, R., Routledge, B., & Smith, N. (2010) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v4i1.14031 · Cited by 1,923
SemEval-2014 Task 4: Aspect Based Sentiment Analysis
Pontiki, M., Galanis, D., Pavlopoulos, J., Papageorgiou, H., Androutsopoulos, I., & Manandhar, S. (2014) · Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014) · doi:10.3115/v1/s14-2004 · Cited by 1,862
Annotating Expressions of Opinions and Emotions in Language
Wiebe, J., Wilson, T., & Cardie, C. (2005) · Language Resources and Evaluation · doi:10.1007/s10579-005-7880-9 · Cited by 1,755
Twitter power: Tweets as electronic word of mouth
Jansen, B. J., Zhang, M., Sobel, K., & Chowdury, A. (2009) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21149 · Cited by 1,642
Named Entity Recognition in Tweets: An Experimental Study
Ritter, A., Clark, S., & Etzioni, O. (2011) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · Cited by 1,496
Self-citationTwitter Sentiment Analysis: The Good the Bad and the OMG!
Kouloumpis, E., Wilson, T., & Moore, J. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v5i1.14185 · Cited by 1,225
Cited by
SemEval-2016 Task 5: Aspect Based Sentiment Analysis
Pontiki, M., Galanis, D., Papageorgiou, H., Androutsopoulos, I., Manandhar, S., AL-Smadi, M., Al-Ayyoub, M., Zhao, Y., Qin, B., De Clercq, O., et al. (2016) · Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016) · doi:10.18653/v1/s16-1002 · Cited by 1,281
Learning Sentiment-Specific Word Embedding for Twitter Sentiment Classification
Tang, D., Wei, F., Yang, N., Zhou, M., Liu, T., & Qin, B. (2014) · Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.3115/v1/p14-1146 · Cited by 1,181
Sentiment Analysis of Short Informal Texts
Kiritchenko, S., Zhu, X., & Mohammad, S. M. (2014) · Journal of Artificial Intelligence Research · doi:10.1613/jair.4272 · Cited by 880
SemEval-2017 Task 4: Sentiment Analysis in Twitter
Rosenthal, S., Farra, N., & Nakov, P. (2017) · Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017) · doi:10.18653/v1/s17-2088 · Cited by 870
Self-citationSemEval-2016 Task 6: Detecting Stance in Tweets
Mohammad, S., Kiritchenko, S., Sobhani, P., Zhu, X., & Cherry, C. (2016) · Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016) · doi:10.18653/v1/s16-1003 · Cited by 844
Like It or Not: A Survey of Twitter Sentiment Analysis Methods
Giachanou, A., & Crestani, F. (2017) · ACM Computing Surveys · doi:10.1145/2938640 · Cited by 547
Automatic Sarcasm Detection: A Survey
Joshi, A., Bhattacharyya, P., & Carman, M. J. (2018) · ACM Computing Surveys · doi:10.1145/3124420 · Cited by 411
ARBERT & MARBERT: Deep Bidirectional Transformers for Arabic
Abdul-Mageed, M., Elmadany, A., & Nagoudi, E. M. B. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.551 · Cited by 397
Mohammad, S. M. (2016) · Emotion Measurement · doi:10.1016/b978-0-08-100508-8.00009-6 · Cited by 391
“Notably, it used word and character ngrams, manually created and automatically generated sentiment lexicons, parts of speech, word clusters, and Twitter-specific encodings such as hashtags, creatively spelled words, and abbreviations (yummeee, lol, etc).”
InfluentialSemEval-2015 Task 10: Sentiment Analysis in Twitter
Rosenthal, S., Nakov, P., Kiritchenko, S., Mohammad, S., Ritter, A., & Stoyanov, V. (2015) · Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015) · doi:10.18653/v1/s15-2078 · Cited by 367
Self-citationAl-Smadi, M., Qawasmeh, O., Al-Ayyoub, M., Jararweh, Y., & Gupta, B. (2018) · Journal of Computational Science · doi:10.1016/j.jocs.2017.11.006 · Cited by 339
Sentiment Embeddings with Applications to Sentiment Analysis
Tang, D., Wei, F., Qin, B., Yang, N., Liu, T., & Zhou, M. (2016) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2015.2489653 · Cited by 335
Beneath the Tip of the Iceberg: Current Challenges and New Directions in Sentiment Analysis Research
Poria, S., Hazarika, D., Majumder, N., & Mihalcea, R. (2023) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2020.3038167 · Cited by 321
Gated Neural Networks for Targeted Sentiment Analysis
Zhang, M., Zhang, Y., & Vo, D.-T. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10380 · Cited by 304
Predicting Polarities of Tweets by Composing Word Embeddings with Long Short-Term Memory
Wang, X., Liu, Y., SUN, C., Wang, B., & Wang, X. (2015) · Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.3115/v1/p15-1130 · Cited by 297
A Deeper Look into Sarcastic Tweets Using Deep Convolutional Neural Networks
Poria, S., Cambria, E., Hazarika, D., & Vij, P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.08815 · Cited by 292
SemEval-2018 Task 3: Irony Detection in English Tweets
Van Hee, C., Lefever, E., & Hoste, V. (2018) · Proceedings of The 12th International Workshop on Semantic Evaluation · doi:10.18653/v1/s18-1005 · Cited by 292
A transformer-based approach to irony and sarcasm detection
Potamias, R. A., Siolas, G., & Stafylopatis, A. G. (2020) · Neural Computing and Applications · doi:10.1007/s00521-020-05102-3 · Cited by 278
Sentiment, emotion, purpose, and style in electoral tweets
Mohammad, S. M., Zhu, X., Kiritchenko, S., & Martin, J. (2015) · Information Processing & Management · doi:10.1016/j.ipm.2014.09.003 · Cited by 274
Code Mixing: A Challenge for Language Identification in the Language of Social Media
Barman, U., Das, A., Wagner, J., & Foster, J. (2014) · Proceedings of the First Workshop on Computational Approaches to Code Switching · doi:10.3115/v1/w14-3902 · Cited by 267
Opinion Mining and Information Fusion: A survey
Balazs, J. A., & Velásquez, J. D. (2016) · Information Fusion · doi:10.1016/j.inffus.2015.06.002 · Cited by 253
Unpaired Sentiment-to-Sentiment Translation: A Cycled Reinforcement Learning Approach
Xu, J., Sun, X., Zeng, Q., Zhang, X., Ren, X., Wang, H., & Li, W. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1090 · Cited by 183
Korkontzelos, I., Nikfarjam, A., Shardlow, M., Sarker, A., Ananiadou, S., & Gonzalez, G. H. (2016) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2016.06.007 · Cited by 173
Detecting Stance in Tweets And Analyzing its Interaction with Sentiment
Sobhani, P., Mohammad, S., & Kiritchenko, S. (2016) · Proceedings of the Fifth Joint Conference on Lexical and Computational Semantics · doi:10.18653/v1/s16-2021 · Cited by 130
Emo2Vec: Learning Generalized Emotion Representation by Multi-task Training
Xu, P., Madotto, A., Wu, C.-S., Park, J. H., & Fung, P. (2018) · Proceedings of the 9th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis · doi:10.18653/v1/w18-6243 · Cited by 83
From classification to quantification in tweet sentiment analysis
Gao, W., & Sebastiani, F. (2016) · Social Network Analysis and Mining · doi:10.1007/s13278-016-0327-z · Cited by 75
“The SemEval2013, SemEval2014, SemEval2015, and SemE-val2016 datasets are described more in detail in Nakov et al. (2013), Rosenthal et al. (2014, 2015), and Nakov et al. (2016), respectively, while all of the other datasets (Sanders, SST, OMD, HCR, GASP, WA, WB) are described in detail in Saif et…”
BackgroundLai, M., Tambuscio, M., Patti, V., Ruffo, G., & Rosso, P. (2019) · Data & Knowledge Engineering · doi:10.1016/j.datak.2019.101738 · Cited by 61
“Techniques such as sentiment analysis (SA) and opinion mining are usually exploited to monitor people’s mood, extracting information from users’ generated contents in social media, as in [52, 48, 47, 58, 57, 56, 39, 6].”
BackgroundA Close Look into the Calibration of Pre-trained Language Models
Chen, Y., Yuan, L., Cui, G., Liu, Z., & Ji, H. (2022) · arXiv · doi:10.48550/arxiv.2211.00151 · Cited by 18
A Close Look into the Calibration of Pre-trained Language Models
Chen, Y., Yuan, L., Cui, G., Liu, Z., & Ji, H. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.75 · Cited by 22
Metadata
- Authors
- Rosenthal, S., Ritter, A., Nakov, P., & Stoyanov, V.
- Venue
- Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014)
- Published
- 2014
- Pages
- 73-80
- OpenAlex
- W2250243742
- Cited by
- 617
- References
- 19
- FWCI
- 61.66
- Type
- conference paper
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.