Comments Mining With TF-IDF: The Inherent Bias and Its Removal
The DOI matches its Crossref record; 7 of the 7 references it cites were found in a registry or an index.
Abstract
Text mining have gained great momentum in recent years, with user-generated content becoming widely available. One key use is comment mining, with much attention being given to sentiment analysis and opinion mining. An essential step in the process of comment mining is text pre-processing; a step in which each linguistic term is assigned with a weight that commonly increases with its appearance in the studied text, yet is offset by the frequency of the term in the domain of interest. A common practice is to use the well-known tf-idf formula to compute these weights. This paper reveals the bias introduced by between-participants' discourse to the study of comments in social media, and proposes an adjustment. We find that content extracted from discourse is often highly correlated, resulting in dependency structures between observations in the study, thus introducing a statistical bias. Ignoring this bias can manifest in a non-robust analysis at best and can lead to an entirely wrong conclusion at worst. We propose an adjustment to tf-idf that accounts for this bias. We illustrate the effects of both the bias and correction with with seven Facebook fan pages data, covering different domains, including news, finance, politics, sport, shopping, and entertainment.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Thumbs up?: sentiment classification using machine learning techniques
Pang, B., Lee, L., & Vaithyanathan, S. (2002) · Proceedings of the ACL-02 conference on Empirical methods in natural language processing - EMNLP '02 · doi:10.3115/1118693.1118704 · Cited by 7,024
Opinion mining and sentiment analysis
Pang, B., & Lee, L. (2008) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000011 · Cited by 6,372
Business intelligence and analytics: from big data to big impact
Chen, H., Chiang, R. H. L., & Storey, V. C. (2012) · MIS Quarterly · doi:10.2307/41703503 · Cited by 6,250
An Introduction to Discourse Analysis
Gee, J. P. (2014) · doi:10.4324/9781315819679 · Cited by 1,036
Dhar, V. (2013) · Communications of the ACM · doi:10.1145/2500499 · Cited by 856 · cited 1×
“ment and Information Systems (IS) [3].”
Ghose, A., Ipeirotis, P. G., & Li, B. (2012) · Marketing Science · doi:10.1287/mksc.1110.0700 · Cited by 649
Effects of Online Comments on Smokers' Perception of Antismoking Public Service Announcements
Shi, R., Messaris, P., & Cappella, J. N. (2014) · Journal of Computer-Mediated Communication · doi:10.1111/jcc4.12057 · Cited by 117 · cited 1×
“Researchers have studied comments to posted pictures and YouTube videos [13], [14], [15], comments on blogs [16], [17], comments on press releases [18], [19], comments on corporate communication [20], comments on public service announcements [21], [22], and even comments on comments [18], [23], [24].”
Background
Cited by
Federated Unlearning via Class-Discriminative Pruning
Wang, J., Guo, S., Xie, X., & Qi, H. (2022) · Proceedings of the ACM Web Conference 2022 · doi:10.1145/3485447.3512222 · Cited by 157
Balayn, A., Lofi, C., & Houben, G.-J. (2021) · The VLDB Journal · doi:10.1007/s00778-021-00671-8 · Cited by 98
The design of reciprocal learning between human and artificial intelligence
Zagalsky, A., Te'eni, D., Yahav, I., Schwartz, D. G., Silverman, G., Cohen, D., Mann, Y., & Lewinsky, D. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479587 · Cited by 35
Self-citation
Metadata
- Authors
- Yahav, I., Shehory, O., & Schwartz, D.
- Venue
- IEEE Transactions on Knowledge and Data Engineering
- Published
- 2019-03-01
- Volume
- 31
- Issue
- 3
- Pages
- 437-450
- ISSN
- 1041-4347 (ISSN-L)
- 1558-2191
- 2326-3865
- OpenAlex
- W2804344463
- Cited by
- 104
- References
- 7
- FWCI
- 6.51
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.