CoChat
Sign Up
Cited by 61Sentiment Analysis and Opinion MiningTopic ModelingAdvanced Text Analysis Techniques

Measuring the Similarity of Sentential Arguments in Dialogue

Open access
Misra, Amita, Ecker, Brian, Walker, Marilyn·2016·Proceedings of the 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue
DOI 10.18653/v1/w16-3636
Open this paper in CoChat — free accountOpen PDF

Abstract

When people converse about social or political topics, similar arguments are often paraphrased by different speakers, across many different conversations.Debate websites produce curated summaries of arguments on such topics; these summaries typically consist of lists of sentences that represent frequently paraphrased propositions, or labels capturing the essence of one particular aspect of an argument, e.g.Morality or Second Amendment.We call these frequently paraphrased propositions ARGUMENT FACETS.Like these curated sites, our goal is to induce and identify argument facets across multiple conversations, and produce summaries.However, we aim to do this automatically.We frame the problem as consisting of two steps: we first extract sentences that express an argument from raw social media dialogs, and then rank the extracted arguments in terms of their similarity to one another.Sets of similar arguments are used to represent argument facets.We show here that we can predict ARGUMENT FACET SIMI-LARITY with a correlation averaging 0.63 compared to a human topline averaging 0.68 over three debate topics, easily beating several reasonable baselines.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.18653/v1/w16-3636

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • SBERT sets new state-of-the-art performance on a challenging argument similarity dataset (Misra et al., 2016).

    “It sets new state-of-the-art performance on a challenging argument similarity dataset (Misra et al., 2016) and on a triplet dataset to distinguish sentences from different sections of a Wikipedia article (Dor et al., 2018).”

    From Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks (2019) · cites this paper as Misra et al., 2016

  • The AFS corpus annotated 6,000 sentential argument pairs from social media dialogs on three controversial topics: gun control, gay marriage, and death penalty.

    “The AFS corpus annotated 6,000 sentential argument pairs from social media dialogs on three controversial topics: gun control , gay marriage , and death penalty .”

    From Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks (2019) · cites this paper as Misra et al. (2016)

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 42

  1. Robert Abbott, Brian Ecker, Pranav Anand, and Mari- lyn Walker. 2016. Internet argument corpus 2.0: An sql schema for dialogic social media and the corpora to go with it. In Language Resources and Evalua- tion Conference, LREC2016.
  2. Eneko Agirre, Mona Diab, Daniel Cer, and Aitor Gonzalez-Agirre. 2012. Semeval-2012 task 6: A pilot on semantic textual similarity. In Proc. of the First Joint Conference on Lexical and Compu- tational Semantics, volume 1, pages 385-393.
  3. Eneko Agirre, Daniel Cer, Mona Diab, Aitor Gonzalez- Agirre, and Weiwei Guo. 2013. Sem 2013 shared task: Semantic textual similarity, including a pilot on typed-similarity. In In* SEM 2013: The Second Joint Conference on Lexical and Computational Se- mantics.
  4. Oram Biran and Owen Rambow. 2011. Identifying justifications in written dialogs. In 2011 Fifth IEEE International Conference on Semantic Computing (ICSC), pages 162-168.
  5. Filip Boltuzic and Jan Šnajder. 2014. Back up your stance: Recognizing arguments in online discus- sions. In Proc. of the First Workshop on Argumen- tation Mining, pages 49-58.
  6. Filip Boltuzic and Jan Šnajder. 2015. Identifying prominent arguments in online debates using seman- tic textual similarity. In Proc. of the Second Work- shop on Argumentation Mining.
  7. Elena Cabrio and Serena Villata. 2012. Combining textual entailment and argumentation theory for sup- porting online debates interactions. In Proc. of the 50th Annual Meeting of the Association for Compu- tational Linguistics (Volume 2: Short Papers), pages 208-212.
  8. Alexander Conrad, Janyce Wiebe, et al. 2012. Rec- ognizing arguing subjectivity and argument tags. In Proc. of the Workshop on Extra-Propositional Aspects of Meaning in Computational Linguistics, pages 80-88.
  9. William B Dolan and Chris Brockett. 2005. Automati- cally constructing a corpus of sentential paraphrases. In Proc. of IWP.
  10. P.M. Dung. 1995. On the acceptability of arguments and its fundamental role in nonmonotonic reasoning, logic programming and n-person games* 1. Artifi- cial intelligence, 77(2):321-357.
  11. Debanjan Ghosh, Smaranda Muresan, Nina Wacholder, Mark Aakhus, and Matthew Mitsui. 2014. Analyz- ing argumentative discourse units in online interac- tions. ACL 2014, page 39.
  12. Theodosis Goudas, Christos Louizos, Georgios Petasis, and Vangelis Karkaletsis. 2014. Argument extrac- tion from news, blogs, and social media. In Artifi- cial Intelligence: Methods and Applications, pages 287-299. Springer.
  13. Ivan Habernal and Iryna Gurevych. 2015. Exploit- ing debate portals for semi-supervised argumenta- tion mining in user-generated web discourse. In Proc. of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2127-2137.
  14. Ivan Habernal and Iryna Gurevych. 2016. Argumenta- tion mining in user-generated web discourse. CoRR, abs/1601.02403. · arXiv
  15. Ivan Habernal, Judith Eckle-Kohler, and Iryna Gurevych. 2014. Argumentation mining on the web from information seeking perspective. In Proc. of the Workshop on Frontiers and Connections between Argumentation Theory and Natural Language Pro- cessing, pages 26-39.
  16. Lushan Han, Abhay Kashyap, Tim Finin, James May- field, and Jonathan Weese. 2013. Umbc ebiquity- core: Semantic textual similarity systems. In Pro- ceedings of the Second Joint Conference on Lexical and Computational Semantics, pages 44-52.
  17. Kazi Saidul Hasan and Vincent Ng. 2013. Frame se- mantics for stance classification. In CoNLL, pages 124-132.
  18. Kazi Saidul Hasan and Vincent Ng. 2014. Why are you taking this stance? identifying and classifying reasons in ideological debates. In Proc. of the Con- ference on Empirical Methods in Natural Language Processing.
  19. Sally Jackson and Scott Jacobs. 1980. Structure of conversational argument: Pragmatic bases for the enthymeme. Quarterly Journal of Speech, 66(3):251-265.
  20. M. Joshi and C. Penstein-Rosé. 2009. Generalizing dependency features for opinion mining. In Proc. of the ACL-IJCNLP 2009 Conference Short Papers, pages 313-316.
  21. Alon Lavie and Michael J. Denkowski. 2009. The meteor metric for automatic evaluation of machine translation. Machine Translation, 23(2-3):105-115, September.
  22. Quoc Le and Tomas Mikolov. 2014. Distributed repre- sentations of sentences and documents. In Tony Je- bara and Eric P. Xing, editors, Proc. of the 31st Inter- national Conference on Machine Learning (ICML- 14), pages 1188-1196.
  23. Jiwei Li, Xinlei Chen, Eduard Hovy, and Dan Jurafsky. 2015. Visualizing and understanding neural models in nlp. arXiv preprint arXiv:1506.01066. · arXiv
  24. C.-Y. Lin. 2004. Rouge: A package for automatic eval- uation of summaries rouge: A package for automatic evaluation of summaries. In Proc. of the Workshop on Text Summarization Branches Out (WAS 2004).
  25. Nitin Madnani, Joel Tetreault, and Martin Chodorow. 2012. Re-examining machine translation metrics for paraphrase identification. In Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT '12, pages 182-190, Stroudsburg, PA, USA. Association for Computational Linguistics.
  26. Christopher D. Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven J. Bethard, and David Mc- Closky. 2014. The Stanford CoreNLP natural lan- guage processing toolkit. In Proc. of 52nd Annual Meeting of the Association for Computational Lin- guistics: System Demonstrations, pages 55-60.
  27. Rada Mihalcea, Courtney Corley, and Carlo Strappa- rava. 2006. Corpus-based and knowledge-based measures of text semantic similarity. In AAAI, vol- ume 6, pages 775-780.
  28. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Cor- rado, and Jeff Dean. 2013. Distributed representa- tions of words and phrases and their compositional- ity. In Advances in Neural Information Processing Systems, pages 3111-3119.
  29. Amita Misra, Pranav Anand, Jean E. Fox Tree, and Marilyn Walker. 2015. Using summarization to dis- cover argument facets in dialog. In Proc. of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies.
  30. Nona Naderi and Graeme Hirst. 2015. Argumentation mining in parliamentary discourse. In CMNA 2015: 15th workshop on Computational Models of Natural Argument.
  31. Ani Nenkova and Rebecca Passonneau. 2004. Evalu- ating content selection in summarization: The pyra- mid method. In Proc. of Joint Annual Meeting of Human Language Technology and the North Amer- ican chapter of the Association for Computational Linguistics (HLT/NAACL), 6.
  32. F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Pretten- hofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Pas- sos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. 2011. Scikit-learn: Machine learn- ing in Python. Journal of Machine Learning Re- search, 12:2825-2830.
  33. Andreas Peldszus and Manfred Stede. 2013. Ranking the annotators: An agreement study on argumenta- tion structure. In Proc. of the 7th linguistic annota- tion workshop and interoperability with discourse, pages 196-204.
  34. Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. Glove: Global vectors for word representation. In EMNLP, volume 14, pages 1532- 1543.
  35. Chris Reed and Glenn Rowe. 2004. Araucaria: Soft- ware for argument analysis, diagramming and repre- sentation. International Journal on Artificial Intelli- gence Tools, 13(04):961-979.
  36. Radim Řehůřek and Petr Sojka. 2010. Software Framework for Topic Modelling with Large Cor- pora. In Proc. of the LREC 2010 Workshop on New Challenges for NLP Frameworks, pages 45-50.
  37. Matthew Snover, Nitin Madnani, Bonnie Dorr, and Richard Schwartz. 2009. Fluency, adequacy, or HTER? Exploring different human judgments with a tunable MT metric. In Proceedings of the Fourth Workshop on Statistical Machine Translation, pages 259-268, Athens, Greece, March. Association for Computational Linguistics.
  38. Swapna Somasundaran and Janyce Wiebe. 2009. Rec- ognizing stances in online debates. In Proc. of the 47th Annual Meeting of the ACL, pages 226-234.
  39. Christian Stab and Iryna Gurevych. 2014. Annotating argument components and relations in persuasive es- says. In Proc. of the 25th International Confer- ence on Computational Linguistics (COLING 2014), pages 1501-1510.
  40. Reid Swanson, Brian Ecker, and Marilyn Walker. 2015. Argument mining: Extracting arguments from online dialogue. In Proc. of the 16th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 217-226.
  41. Stephen E. Toulmin. 1958. The Uses of Argument. Cambridge University Press.
  42. Douglas Walton, Chris Reed, and Fabrizio Macagno. 2008. Argumentation Schemes. Cambridge Univer- sity Press.

Cited by

Sorted by citations

Authors
Misra, A., Ecker, B., & Walker, M.
Affiliations
University of California, Santa Cruz (US)
Venue
Proceedings of the 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue
Publisher
Association for Computational Linguistics
Published
2016
Pages
276-287
DOI
10.18653/v1/w16-3636
OpenAlex
W2564074212
Cited by
61
References
0
Citations by year
2026: 3 · 2025: 4 · 2024: 2 · 2023: 14 · 2022: 4 · 2021: 16 · 2020: 7 · 2019: 5 · 2018: 4 · 2017: 2
FWCI
1.98
Type
conference paper
Language
en
Open access
Licence
cc-by
Locations
  • Annual Meeting of the Special Interest Group on Discourse and Dialogue (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • National Science Foundation (1302668)
  • Directorate for Computer and Information Science and Engineering (1302668)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Advanced Text Analysis Techniques
Keywords
online political debate, argument extraction, dialogue summarization

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review