Learning extraction patterns for subjective expressions
Open accessAbstract
This paper presents a bootstrapping process that learns linguistically rich extraction patterns for subjective (opinionated) expressions. High-precision classifiers label unannotated data to automatically create a large training set, which is then given to an extraction pattern learning algorithm. The learned patterns are then used to identify more subjective sentences. The bootstrapping process learns many subjective patterns and increases recall while maintaining high precision.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Some studies divided sentiment analysis into two subtasks: classifying sentences into objective and subjective, then determining polarity of subjective sentences.
“Some studies followed a slightly different approach, whereby the sentiment analysis was divided into two subtasks, namely: (i) classifying sentences into objective and subjective sentences, and (ii) then determining the polarity of the subjective sentences (Marrese-Taylora et al., 2013;Riloff & Wiebe, 2003).”
From Sentiment Analysis in Tourism: Capitalizing on Big Data (2019) · cites this paper as Riloff & Wiebe, 2003
Section 2.1 (page unavailable) says the paper’s goal is to classify individual sentences as subjective or objective. It does not describe a second step that determines the polarity of those subjective sentences; its discussion of positive and negative classification concerns other work on reviews at the document level. (LLM, conf=0.88)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
No references could be resolved yet.
Research works printed but not linked: 25
- C. Baker, C. Fillmore, and J. Lowe. 1998. The Berkeley FrameNet Project. In Proceedings of the COLING-ACL-98.
- T. Ballmer and W. Brennenstuhl. 1981. Speech Act Classifi- cation: A Study in the Lexical Analysis of English Speech Activity Verbs. Springer-Verlag.
- A. Banfield. 1982. Unspeakable Sentences. Routledge and Kegan Paul, Boston.
- M. E. Califf. 1998. Relational Learning Techniques for Natural Language Information Extraction. Ph.D. thesis, Tech. Rept. AI98-276, Artificial Intelligence Laboratory, The University of Texas at Austin.
- Dayne Freitag. 1998. Toward General-Purpose Learning for Information Extraction. In Proceedings of the ACL-98.
- V. Hatzivassiloglou and K. McKeown. 1997. Predicting the Semantic Orientation of Adjectives. In Proceedings of the ACL-EACL-97.
- S. Huffman. 1996. Learning information extraction pat- terns from examples. In Stefan Wermter, Ellen Riloff, and Gabriele Scheler, editors, Connectionist, Statistical, and Symbolic Approaches to Learning for Natural Language Processing, pages 246-260. Springer-Verlag, Berlin.
- J. Karlgren and D. Cutting. 1994. Recognizing Text Genres with Simple Metrics Using Discriminant Analysis. In Pro- ceedings of the COLING-94.
- B. Kessler, G. Nunberg, and H. Schütze. 1997. Automatic De- tection of Text Genre. In Proceedings of the ACL-EACL-97.
- J. Kim and D. Moldovan. 1993. Acquisition of Semantic Pat- terns for Information Extraction from Corpora. In Proceed- ings of the Ninth IEEE Conference on Artificial Intelligence for Applications.
- Beth Levin. 1993. English Verb Classes and Alternations: A Preliminary Investigation. University of Chicago Press.
- B. Pang, L. Lee, and S. Vaithyanathan. 2002. Thumbs up? Sen- timent Classification Using Machine Learning Techniques. In Proceedings of the EMNLP-02.
- E. Riloff and R. Jones. 1999. Learning Dictionaries for In- formation Extraction by Multi-Level Bootstrapping. In Pro- ceedings of the AAAI-99.
- E. Riloff, J. Wiebe, and T. Wilson. 2003. Learning Subjective Nouns using Extraction Pattern Bootstrapping. In Proceed- ings of the Seventh Conference on Computational Natural Language Learning (CoNLL-03).
- E. Riloff. 1993. Automatically Constructing a Dictionary for Information Extraction Tasks. In Proceedings of the AAAI- 93.
- E. Riloff. 1996. Automatically Generating Extraction Patterns from Untagged Text. In Proceedings of the AAAI-96.
- S. Soderland, D. Fisher, J. Aseltine, and W. Lehnert. 1995. CRYSTAL: Inducing a Conceptual Dictionary. In Proceed- ings of the IJCAI-95.
- E. Spertus. 1997. Smokey: Automatic Recognition of Hostile Messages. In Proceedings of the IAAI-97.
- P. Turney. 2002. Thumbs Up or Thumbs Down? Semantic Ori- entation Applied to Unsupervised Classification of Reviews. In Proceedings of the ACL-02.
- J. Wiebe, R. Bruce, and T. O'Hara. 1999. Development and Use of a Gold Standard Data Set for Subjectivity Classifica- tions. In Proceedings of the ACL-99.
- J. Wiebe, T. Wilson, and M. Bell. 2001. Identifying Collo- cations for Recognizing Opinions. In Proceedings of the ACL-01 Workshop on Collocation: Computational Extrac- tion, Analysis, and Exploitation.
- J. Wiebe. 1990. Recognizing Subjective Sentences: A Compu- tational Investigation of Narrative Text. Ph.D. thesis, State University of New York at Buffalo.
- J. Wiebe. 2000. Learning Subjective Adjectives from Corpora. In Proceedings of the AAAI-00.
- T. Wilson and J. Wiebe. 2003. Annotating Opinions in the World Press. In Proceedings of the ACL SIGDIAL-03.
- R. Yangarber, R. Grishman, P. Tapanainen, and S. Huttunen. 2000. Automatic Acquisiton of Domain Knowledge for In- formation Extraction. In Proceedings of COLING 2000.
Cited by
Sorted by citations
Twitter mood predicts the stock market
Bollen, J., Mao, H., & Zeng, X. (2011) · Journal of Computational Science · doi:10.1016/j.jocs.2010.12.007 · Cited by 5,206 · see citing works
“We adopt OF’s subjective lexicon that has been established upon previous work [37], [38], [24].”
MethodSelf-Training With Noisy Student Improves ImageNet Classification
Xie, Q., Luong, M.-T., Hovy, E., & Le, Q. V. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01070 · Cited by 2,393 · see citing works
Sentiment strength detection in short informal text
Thelwall, M., Buckley, K., Paltoglou, G., Cai, D., & Kappas, A. (2010) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21416 · Cited by 1,449 · see citing works
Sentiment Analysis in Tourism: Capitalizing on Big Data
Alaei, A. R., Becken, S., & Stantic, B. (2019) · Journal of Travel Research · doi:10.1177/0047287517747753 · Cited by 622 · see citing works
Fusing audio, visual and textual clues for sentiment analysis from multimodal content
Poria, S., Cambria, E., Howard, N., Huang, G.-B., & Hussain, A. (2016) · Neurocomputing · doi:10.1016/j.neucom.2015.01.095 · Cited by 534 · see citing works
Good debt or bad debt: Detecting semantic orientations in economic texts
Malo, P., Sinha, A., Korhonen, P., Wallenius, J., & Takala, P. (2014) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23062 · Cited by 513 · see citing works
Mohammad, S. M. (2016) · Emotion Measurement · doi:10.1016/b978-0-08-100508-8.00009-6 · Cited by 391 · see citing works
Volkova, S., Shaffer, K., Jang, J. Y., & Hodas, N. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/p17-2102 · Cited by 381 · see citing works
Chung, Y.-A., Zhang, Y., Han, W., Chiu, C.-C., Qin, J., Pang, R., & Wu, Y. (2021) · 2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) · doi:10.1109/asru51503.2021.9688253 · Cited by 293 · see citing works
Towards an intelligent framework for multimodal affective data analysis
Poria, S., Cambria, E., Hussain, A., & Huang, G.-B. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.10.005 · Cited by 236 · see citing works
Naive-Student: Leveraging Semi-Supervised Learning in Video Sequences for Urban Scene Segmentation
Chen, L.-C., Lopes, R. G., Cheng, B., Collins, M. D., Cubuk, E. D., Zoph, B., Adam, H., & Shlens, J. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58545-7_40 · Cited by 165 · see citing works
Zhang, Y., Park, D. S., Han, W., Qin, J., Gulati, A., Shor, J., Jansen, A., Xu, Y., Huang, Y., Wang, S., et al. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3182537 · Cited by 161 · see citing works
Guo, W., & Caliskan, A. (2021) · Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3461702.3462536 · Cited by 180 · see citing works
Chen, A. T., Wu, S., Tomasino, K. N., Lattie, E. G., & Mohr, D. C. (2019) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2019.103187 · Cited by 58 · see citing works
“To facilitate a process of “discovery” of salient concepts from the text data, we employed a sentiment analysis lexicon comprised of over 6,000 words[46].”
MethodLearning Lexico-Functional Patterns for First-Person Affect
Reed, L., Wu, J., Oraby, S., Anand, P., & Walker, M. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/p17-2022 · Cited by 15 · see citing works
“Was it “stated” or was it “claimed”?: How linguistic bias affects generative language models
Patel, R., & Pavlick, E. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.790 · Cited by 11 · see citing works
- Authors
- Riloff, E., & Wiebe, J.
- Affiliations
- University of Utah (US)
- University of Pittsburgh (US)
- Venue
- Proceedings of the 2003 conference on Empirical methods in natural language processing -
- Published
- 2003
- Volume
- 10
- Pages
- 105-112
- OpenAlex
- W2088622183
- Cited by
- 1,059
- References
- 0
- Citations by year
- 2026: 56 · 2025: 16 · 2024: 16 · 2023: 18 · 2022: 35 · 2021: 49 · 2020: 68 · 2019: 34 · 2018: 47 · 2017: 53 · 2016: 57 · 2015: 81 · 2014: 79 · 2013: 72 · 2012: 69
- FWCI
- 14.94
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion)
- Funders
- National Science Foundation (9704240, 0208985, 0208798)
- Agricultural Research Development Agency
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Advanced Text Analysis Techniques
- Keywords
- subjective expression, opinion extraction, bootstrapping, semi-supervised learning, training set construction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
