The DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Turning from TF-IDF to TF-IGM for term weighting in text classification
Abstract
A new supervised term weighting scheme called TF-IGM is proposed.It adopts a new statistical model to measure a term's class distinguishing power.It makes full use of the fine-grained term distribution across different classes.It is adaptive to different text datasets by providing options or parameters.It outperforms TF-IDF and state-of-the-art supervised term weighting schemes. Massive textual data management and mining usually rely on automatic text classification technology. Term weighting is a basic problem in text classification and directly affects the classification accuracy. Since the traditional TF-IDF (term frequency & inverse document frequency) is not fully effective for text classification, various alternatives have been proposed by researchers. In this paper we make comparative studies on different term weighting schemes and propose a new term weighting scheme, TF-IGM (term frequency & inverse gravity moment), as well as its variants. TF-IGM incorporates a new statistical model to precisely measure the class distinguishing power of a term. Particularly, it makes full use of the fine-grained term distribution across different classes of text. The effectiveness of TF-IGM is validated by extensive experiments of text classification using SVM (support vector machine) and kNN (k nearest neighbors) classifiers on three commonly used corpora. The experimental results show that TF-IGM outperforms the famous TF-IDF and the state-of-the-art supervised term weighting schemes. In addition, some new findings different from previous studies are obtained and analyzed in depth in the paper.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LIBSVM: A library for support vector machines
Chang, C.-C., & Lin, C.-J. (2011) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/1961189.1961199 · Cited by 40,918
Quinlan, J. R. (1986) · Machine Learning · doi:10.1007/bf00116251 · Cited by 21,439
Term-weighting approaches in automatic text retrieval
Salton, G., & Buckley, C. (1988) · Information Processing & Management · doi:10.1016/0306-4573(88)90021-0 · Cited by 9,740
An algorithm for suffix stripping
Porter, M. F. (1980) · Program · doi:10.1108/eb046814 · Cited by 8,096 · cited 1×
“Meanwhile, the letters are converted to lowerase and words are stemmed using Porter’s stemmer ( Porter, 1980, 006 ), for example, removing the verb suffixes such as -s, -ed and ing .”
MethodMachine learning in automated text categorization
Sebastiani, F. (2002) · ACM Computing Surveys · doi:10.1145/505282.505283 · Cited by 7,926
A Comparative Study on Feature Selection in Text Categorization
Yang, Y., & Pedersen, J. (1997) · Cited by 4,790
A STATISTICAL INTERPRETATION OF TERM SPECIFICITY AND ITS APPLICATION IN RETRIEVAL
SPARCK JONES, K. (1972) · Journal of Documentation · doi:10.1108/eb026526 · Cited by 4,505
Improving the retrieval of information from external sources
Dumais, S. T. (1991) · Behavior Research Methods, Instruments, & Computers · doi:10.3758/bf03203370 · Cited by 579
Supervised and Traditional Term Weighting Methods for Automatic Text Categorization
Lan, M., Tan, C. L., Su, J., & Lu, Y. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.110 · Cited by 562
A statistical interpretation of term specificity and its application in retrieval
Spärck Jones, K. (2004) · Journal of Documentation · doi:10.1108/00220410410560573 · Cited by 500
Cited by
Text Classification Algorithms: A Survey
Kowsari, K., Jafari Meimandi, K., Heidarysafa, M., Mendu, S., Barnes, L., & Brown, D. (2019) · Information · doi:10.3390/info10040150 · Cited by 1,312
HDLTex: Hierarchical Deep Learning for Text Classification
Kowsari, K., Brown, D. E., Heidarysafa, M., Jafari Meimandi, K., Gerber, M. S., & Barnes, L. E. (2017) · 2017 16th IEEE International Conference on Machine Learning and Applications (ICMLA) · doi:10.1109/icmla.2017.0-134 · Cited by 454
A recent overview of the state-of-the-art elements of text classification
Mirończuk, M. M., & Protasiewicz, J. (2018) · Expert Systems with Applications · doi:10.1016/j.eswa.2018.03.058 · Cited by 317
Sentiment analysis: A survey on design framework, applications and future scopes
Bordoloi, M., & Biswas, S. K. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10442-2 · Cited by 183
Metadata
- Authors
- Chen, K., Zhang, Z., Long, J., & Zhang, H.
- Venue
- Expert Systems with Applications
- Publisher
- Elsevier BV
- Published
- 2016-12
- Volume
- 66
- Pages
- 245-260
- ISSN
- 0957-4174 (ISSN-L)
- OpenAlex
- W2520488855
- Cited by
- 271
- References
- 28
- FWCI
- 19.09
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Turning from TF-IDF to TF-IGM for term weighting in text classification. https://app.cochat.ai/papers/10.1016/j.eswa.2016.09.009
- The paper
- Chen, K., Zhang, Z., Long, J., & Zhang, H. (2016). Turning from TF-IDF to TF-IGM for term weighting in text classification. Expert Systems with Applications. https://doi.org/10.1016/j.eswa.2016.09.009
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.