CoChat
Sign Up
Cited by 130Sentiment Analysis and Opinion MiningHate Speech and Cyberbullying DetectionTopic Modeling

Detecting Stance in Tweets And Analyzing its Interaction with Sentiment

Open access
Sobhani, Parinaz, Mohammad, Saif, Kiritchenko, Svetlana·2016·Proceedings of the Fifth Joint Conference on Lexical and Computational Semantics
DOI 10.18653/v1/s16-2021
Open this paper in CoChat — free accountOpen PDF

Abstract

One may express favor (or disfavor) towards a target by using positive or negative language.Here for the first time we present a dataset of tweets annotated for whether the tweeter is in favor of or against pre-chosen targets, as well as for sentiment.These targets may or may not be referred to in the tweets, and they may or may not be the target of opinion in the tweets.We develop a simple stance detection system that outperforms all 19 teams that participated in a recent shared task competition on the same dataset (SemEval-2016 Task #6).Additionally, access to both stance and sentiment annotations allows us to conduct several experiments to tease out their interactions.We show that while sentiment features are useful for stance classification, they alone are not sufficient.We also show the impacts of various features on detecting stance and sentiment, respectively.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.18653/v1/s16-2021

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Sentiment and opinion contained in a message are two different characteristics of the message.

    “Therefore sentiment and opinion contained in a message are two different characteristics of the message.”

    From Credibility of climate change denial in social media (2019) · cites this paper as Sobhani et al., 2016

    Section 3 labels tweets for sentiment separately from the paper’s annotation of whether an opinion concerns the target, another entity, or no entity. This supports distinguishing sentiment from the target or presence of an opinion, but the paper does not establish the broader claim that sentiment and opinion themselves are two distinct characteristics. (LLM, conf=0.79)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 34

  1. Pranav Anand, Marilyn Walker, Rob Abbott, Jean E. Fox Tree, Robeson Bowmani, and Michael Mi- nor. 2011. Cats rule and dogs drool!: Classifying stance in online debate. In Proceedings of the Work- shop on Computational Approaches to Subjectivity and Sentiment Analysis, pages 1-9.
  2. Lingjia Deng and Janyce Wiebe. 2014. Sentiment propagation via implicature constraints. In Pro- ceedings of the Conference of the European Chap- ter of the Association for Computational Linguistics, pages 377-385, Sweden.
  3. Adam Faulkner. 2014. Automated classification of stance in student essays: An approach using stance target information and the Wikipedia link-based measure. In Proceedings of the Flairs Conference.
  4. Kevin Gimpel, Nathan Schneider, Brendan O'Connor, Dipanjan Das, Daniel Mills, Jacob Eisenstein, Michael Heilman, Dani Yogatama, Jeffrey Flanigan, and Noah A. Smith. 2011. Part-of-speech tagging for Twitter: Annotation, features, and experiments. In Proceedings of the Annual Meeting of the Associ- ation for Computational Linguistics.
  5. Kazi Saidul Hasan and Vincent Ng. 2013. Stance classification of ideological debates: Data, models, features, and constraints. In Proceedings of the In- ternational Joint Conference on Natural Language Processing, pages 1348-1356.
  6. Minqing Hu and Bing Liu. 2004. Mining and sum- marizing customer reviews. In Proceedings of the ACM SIGKDD International Conference on Knowl- edge Discovery and Data Mining, pages 168-177.
  7. Yoshikiyo Kato, Sadao Kurohashi, Kentaro Inui, Robert Malouf, and Tony Mullen. 2008. Taking sides: User classification for informal online politi- cal discourse. Internet Research, 18(2):177-190.
  8. Svetlana Kiritchenko, Xiaodan Zhu, Colin Cherry, and Saif M. Mohammad. 2014a. NRC-Canada-2014: Detecting aspects and sentiment in customer re- views. In Proceedings of the International Work- shop on Semantic Evaluation, Dublin, Ireland, Au- gust.
  9. Svetlana Kiritchenko, Xiaodan Zhu, and Saif M. Mo- hammad. 2014b. Sentiment analysis of short in- formal texts. Journal of Artificial Intelligence Re- search, 50:723-762.
  10. Wei-Hao Lin, Theresa Wilson, Janyce Wiebe, and Alexander Hauptmann. 2006. Which side are you on? Identifying perspectives at the document and sentence levels. In Proceedings of the Confer- ence on Computational Natural Language Learning, pages 109-116.
  11. Bing Liu and Lei Zhang. 2012. A survey of opin- ion mining and sentiment analysis. In Charu C. Ag- garwal and ChengXiang Zhai, editors, Mining Text Data, pages 415-463. Springer.
  12. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S. Cor- rado, and Jeff Dean. 2013. Distributed representa- tions of words and phrases and their compositional- ity. In Advances in Neural Information Processing Systems, pages 3111-3119.
  13. Saif M. Mohammad and Peter D. Turney. 2010. Emo- tions evoked by common words and phrases: Using Mechanical Turk to create an emotion lexicon. In Proceedings of the Workshop on Computational Ap- proaches to Analysis and Generation of Emotion in Text, pages 26-34.
  14. Saif M. Mohammad, Svetlana Kiritchenko, and Xiao- dan Zhu. 2013. NRC-Canada: Building the state- of-the-art in sentiment analysis of tweets. In Pro- ceedings of the International Workshop on Semantic Evaluation, Atlanta, Georgia, USA, June.
  15. Saif M. Mohammad, Svetlana Kiritchenko, Parinaz Sobhani, Xiaodan Zhu, and Colin Cherry. 2016a. A dataset for detecting stance in tweets. In Pro- ceedings of the Language Resources and Evaluation Conference, Portorož, Slovenia.
  16. Saif M. Mohammad, Svetlana Kiritchenko, Parinaz Sobhani, Xiaodan Zhu, and Colin Cherry. 2016b. SemEval-2016 Task 6: Detecting stance in tweets. In Proceedings of the International Workshop on Se- mantic Evaluation, San Diego, California.
  17. Akiko Murakami and Rudy Raymond. 2010. Sup- port or oppose? Classifying positions in online de- bates from reply activities and opinion expressions. In Proceedings of the International Conference on Computational Linguistics, pages 869-875, Beijing, China.
  18. Bo Pang and Lillian Lee. 2008. Opinion mining and sentiment analysis. Foundations and Trends in In- formation Retrieval, 2(1-2):1-135.
  19. Fabian Pedregosa, Gaël Varoquaux, et al. 2011. Scikit- learn: Machine learning in Python. Journal of Ma- chine Learning Research, 12:2825-2830.
  20. Maria Pontiki, Dimitrios Galanis, John Pavlopou- los, Harris Papageorgiou, Ion Androutsopoulos, and Suresh Manandhar. 2014. SemEval-2014 Task 4: Aspect based sentiment analysis. In Proceedings of the International Workshop on Semantic Evaluation, Dublin, Ireland, August.
  21. Maria Pontiki, Dimitrios Galanis, Haris Papageogiou, Suresh Manandhar, and Ion Androutsopoulos. 2015. SemEval-2015 Task 12: Aspect based sentiment analysis. In Proceedings of the International Work- shop on Semantic Evaluation, Denver, Colorado.
  22. Ashwin Rajadesingan and Huan Liu. 2014. Identify- ing users with opposing opinions in Twitter debates. In Proceedings of the Conference on Social Comput- ing, Behavioral-Cultural Modeling and Prediction, pages 153-160. Washington, DC, USA.
  23. Marta Recasens, Cristian Danescu-Niculescu-Mizil, and Dan Jurafsky. 2013. Linguistic models for an- alyzing and detecting biased language. In Proceed- ings of the Annual Meeting of the Association for Computational Linguistics, pages 1650-1659.
  24. Sara Rosenthal, Preslav Nakov, Svetlana Kiritchenko, Saif M. Mohammad, Alan Ritter, and Veselin Stoy- anov. 2015. SemEval-2015 Task 10: Sentiment analysis in Twitter. In Proceedings of the Interna- tional Workshop on Semantic Evaluations.
  25. Parinaz Sobhani, Diana Inkpen, and Stan Matwin. 2015. From argumentation mining to stance clas- sification. In Proceedings of the Workshop on Argu- mentation Mining, pages 67-77, Denver, Colorado, USA.
  26. Swapna Somasundaran and Janyce Wiebe. 2009. Rec- ognizing stances in online debates. In Proceedings of the Annual Meeting of the Association for Com- putational Linguistics, pages 226-234, Suntec, Sin- gapore.
  27. Swapna Somasundaran and Janyce Wiebe. 2010. Rec- ognizing stances in ideological on-line debates. In Proceedings of the NAACL HLT 2010 Workshop CAAGET, pages 116-124.
  28. Duyu Tang, Furu Wei, Nan Yang, Ming Zhou, Ting Liu, and Bing Qin. 2014. Learning sentiment- specific word embedding for Twitter sentiment clas- sification. In Proceedings of the Annual Meeting of the Association for Computational Linguistics, pages 1555-1565.
  29. Matt Thomas, Bo Pang, and Lillian Lee. 2006. Get out the vote: Determining support or opposition from congressional floor-debate transcripts. In Proceed- ings of the Conference on Empirical Methods in Nat- ural Language Processing, pages 327-335.
  30. Joseph Turian, Lev Ratinov, and Yoshua Bengio. 2010. Word representations: a simple and general method for semi-supervised learning. In Proceedings of the Annual Meeting of the Association for Computa- tional Linguistics, pages 384-394.
  31. Marilyn A. Walker, Pranav Anand, Robert Abbott, and Ricky Grant. 2012. Stance classification using di- alogic properties of persuasion. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Hu- man Language Technologies, pages 592-596.
  32. Theresa Wilson, Janyce Wiebe, and Paul Hoffmann. 2005. Recognizing contextual polarity in phrase- level sentiment analysis. In Proceedings of the Conference on Human Language Technology and Empirical Methods in Natural Language Process- ing, pages 347-354, Vancouver, British Columbia, Canada.
  33. Theresa Wilson, Zornitsa Kozareva, Preslav Nakov, Sara Rosenthal, Veselin Stoyanov, and Alan Ritter. 2013. SemEval-2013 Task 2: Sentiment analysis in Twitter. In Proceedings of the International Work- shop on Semantic Evaluation, Atlanta, USA, June.
  34. Tae Yano, Philip Resnik, and Noah A Smith. 2010. Shedding (a thousand points of) light on biased lan- guage. In Proceedings of the NAACL HLT 2010 Workshop on Creating Speech and Language Data with Amazon's Mechanical Turk, pages 152-158.

Cited by

Sorted by citations

Authors
Sobhani, P., Mohammad, S., & Kiritchenko, S.
Affiliations
  • University of Ottawa (CA)
  • National Research Council Canada (CA)
ORCID
Venue
Proceedings of the Fifth Joint Conference on Lexical and Computational Semantics
Publisher
Association for Computational Linguistics
Published
2016
Pages
159-169
DOI
10.18653/v1/s16-2021
OpenAlex
W2509222396
Cited by
130
References
1
Citations by year
2026: 4 · 2025: 6 · 2024: 6 · 2023: 10 · 2022: 16 · 2021: 19 · 2020: 21 · 2019: 20 · 2018: 11 · 2017: 16 · 2016: 1
FWCI
8.37
Type
conference paper
Language
en
Open access
Licence
cc-by
Locations
  • Joint Conference on Lexical and Computational Semantics (publishedVersion, cc-by)
  • NPARC (submittedVersion)
  • INFM-OAR (INFN Catania) (submittedVersion, other-oa)
Funders
Natural Sciences and Engineering Research Council of Canada
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
  • Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
Keywords
stance detection, sentiment analysis, tweets, annotated dataset

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review