The DOI matches its Crossref record; 45 of the 45 references it cites were found in a registry or an index.
Toward Optimal Feature Selection in Naive Bayes for Text Categorization
Open accessAbstract
Automated feature selection is important for text categorization to reduce feature size and to speed up learning process of classifiers. In this paper, we present a novel and efficient feature selection framework based on the Information Theory, which aims to rank the features with their discriminative capacity for classification. We first revisit two information measures: Kullback-Leibler divergence and Jeffreys divergence for binary hypothesis testing, and analyze their asymptotic properties relating to type I and type II errors of a Bayesian classifier. We then introduce a new divergence measure, called Jeffreys-Multi-Hypothesis (JMH) divergence, to measure multi-distribution divergence for multi-class classification. Based on the JMH-divergence, we develop two efficient feature selection methods, termed maximum discrimination ($MD$) and methods, for text categorization. The promising results of extensive experiments demonstrate the effectiveness of the proposed approaches.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
He, H., & Garcia, E. A. (2009) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2008.239 · Cited by 10,359
Self-citationTerm-weighting approaches in automatic text retrieval
Salton, G., & Buckley, C. (1988) · Information Processing & Management · doi:10.1016/0306-4573(88)90021-0 · Cited by 9,740
Wrappers for feature subset selection
Kohavi, R., & John, G. H. (1997) · Artificial Intelligence · doi:10.1016/s0004-3702(97)00043-x · Cited by 8,947
Text categorization with Support Vector Machines: Learning with many relevant features
Joachims, T. (1998) · Lecture Notes in Computer Science · doi:10.1007/bfb0026683 · Cited by 7,999
Machine learning in automated text categorization
Sebastiani, F. (2002) · ACM Computing Surveys · doi:10.1145/505282.505283 · Cited by 7,926
A vector space model for automatic indexing
Salton, G., Wong, A., & Yang, C. S. (1975) · Communications of the ACM · doi:10.1145/361219.361220 · Cited by 7,521
Information theory and statistics
(1959) · Journal of the Franklin Institute · doi:10.1016/0016-0032(59)90368-0 · Cited by 7,094
Divergence measures based on the Shannon entropy
Lin, J. (1991) · IEEE Transactions on Information Theory · doi:10.1109/18.61115 · Cited by 5,140
Distributed Representations of Sentences and Documents
Le, Q. V., & Mikolov, T. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1405.4053 · Cited by 4,934
A Comparative Study on Feature Selection in Text Categorization
Yang, Y., & Pedersen, J. (1997) · Cited by 4,790
A Measure of Asymptotic Efficiency for Tests of a Hypothesis Based on the sum of Observations
Chernoff, H. (1952) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177729330 · Cited by 3,550
A comparison of event models for naive bayes text classification
McCallum, A., & Nigam, K. (1998) · Cited by 3,289
Toward integrating feature selection algorithms for classification and clustering
Liu, H., & Yu, L. (2005) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2005.66 · Cited by 2,706
An extensive empirical study of feature selection metrics for text classification
Forman, G. (2003) · Journal of Machine Learning Research · doi:10.5555/944919.944974 · Cited by 2,394
An invariant form for the prior probability in estimation problems
Jeffreys, H. (1946) · Proceedings of the Royal Society of London. Series A. Mathematical and Physical Sciences · doi:10.1098/rspa.1946.0056 · Cited by 2,323
Estimation of Dependences Based on Empirical Data
Vapnik, V. (2006) · Information Science and Statistics · doi:10.1007/0-387-34239-7 · Cited by 2,238
Naive (Bayes) at forty: The independence assumption in information retrieval
Lewis, D. D. (1998) · Lecture Notes in Computer Science · doi:10.1007/bfb0026666 · Cited by 2,104
In Defense of One-Vs-All Classification
Rifkin, R. M., & Klautau, A. (2004) · Journal of Machine Learning Research · Cited by 2,056
(2010) · Cited by 1,906
Elements of Information Theory
(2013) · Wireless Communications · doi:10.1002/9781118576618.ch7 · Cited by 999
In Defense of One-Vs-All Classification
RifkinRyan, & KlautauAldebaro (2004) · Journal of Machine Learning Research · doi:10.5555/1005332.1005336 · Cited by 891
Hierarchically Classifying Documents Using Very Few Words
Koller, D., & Sahami, M. (1997) · Cited by 854
A Scalable Hierarchical Distributed Language Model
Mnih, A., & Hinton, G. E. (2008) · Cited by 838
Large-Scale Bayesian Logistic Regression for Text Categorization
Genkin, A., Lewis, D. D., & Madigan, D. (2007) · Technometrics · doi:10.1198/004017007000000245 · Cited by 808
Document clustering using locality preserving indexing
Cai, D., He, X., & Han, J. (2005) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2005.198 · Cited by 806
Estimation of Dependences Based on Empirical Data.
Shoesmith, E., Vapnik, V., & Kotz, S. (1984) · The Statistician · doi:10.2307/2988246 · Cited by 764
Information Theory and Statistics
(2005) · Elements of Information Theory · doi:10.1002/047174882x.ch11 · Cited by 729
Some Effective Techniques for Naive Bayes Text Classification
Kim, S.-B., Han, K.-S., Rim, H.-C., & Myaeng, S. H. (2006) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2006.180 · Cited by 576
Spam Filtering with Naive Bayes - Which Naive Bayes?
Metsis, V., Androutsopoulos, I., & Παλιούρας, Γ. (2006) · Conference on Email and Anti-Spam · Cited by 514
Feature Selection for Unbalanced Class Distribution and Naive Bayes
Mladenić, D., & Grobelnik, M. (1999) · Cited by 427
Classification of Text Documents
Li, Y. H. (1998) · The Computer Journal · doi:10.1093/comjnl/41.8.537 · Cited by 333
Experiments on the Use of Feature Selection and Negative Evidence in Automated Text Categorization
Galavotti, L., Sebastiani, F., & Simi, M. (2000) · Lecture Notes in Computer Science · doi:10.1007/3-540-45268-0_6 · Cited by 210
Caropreso, M. F., Matwin, S., & Sebastiani, F. (2001) · Cited by 179
On the Naive Bayes Model for Text Categorization.
Eyheramendy, S., Lewis, D. D., & Madigan, D. (2003) · Cited by 167
Modeling hidden topics on document manifold
Cai, D., Mei, Q., Han, J., & Zhai, C. (2008) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/1458082.1458202 · Cited by 159
Active Learning with Committees for Text Categorization
Liere, R., & Tadepalli, P. V. (1997) · Cited by 146
ENN: Extended Nearest Neighbor Method for Pattern Recognition [Research Frontier]
Tang, B., & He, H. (2015) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2015.2437512 · Cited by 128
Self-citationKernelADASYN: Kernel based adaptive synthetic data generation for imbalanced learning
Tang, B., & He, H. (2015) · IEEE Congress on Evolutionary Computation (CEC) · doi:10.1109/cec.2015.7256954 · Cited by 114
Self-citationA New Text Categorization Technique Using Distributional Clustering and Learning Logic
Al-Mubaid, H., & Umair, S. A. (2006) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2006.135 · Cited by 107
Introducing a family of linear measures for feature selection in text categorization
Combarro, E. F., Montañés, E., Dı́az, I., Ranilla, J., & Mones, R. (2005) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2005.149 · Cited by 88
Aphinyanaphongs, Y., Fu, L. D., Li, Z., Peskin, E. R., Efstathiadis, E., Aliferis, C., & Statnikov, A. (2014) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23110 · Cited by 68
A Parametric Classification Rule Based on the Exponentially Embedded Family
Tang, B., He, H., Ding, Q., & Kay, S. (2015) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2014.2383692 · Cited by 25
Self-citationEEF: Exponentially Embedded Families With Class-Specific Features for Classification
Tang, B., Kay, S., He, H., & Baggenstoss, P. M. (2016) · IEEE Signal Processing Letters · doi:10.1109/lsp.2016.2574327 · Cited by 16
Self-citationHybrid classification with partial models
Tang, B., Ding, Q., He, H., & Kay, S. (2014) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2014.6889782 · Cited by 3
Self-citation
Cited by
Feature selection for text classification: A review
Deng, X., Li, Y., Weng, J., & Zhang, J. (2019) · Multimedia Tools and Applications · doi:10.1007/s11042-018-6083-5 · Cited by 351
A recent overview of the state-of-the-art elements of text classification
Mirończuk, M. M., & Protasiewicz, J. (2018) · Expert Systems with Applications · doi:10.1016/j.eswa.2018.03.058 · Cited by 317
A local density-based approach for outlier detection
Tang, B., & He, H. (2017) · Neurocomputing · doi:10.1016/j.neucom.2017.02.039 · Cited by 301
Self-citationA Correlation-Based Feature Weighting Filter for Naive Bayes
Jiang, L., Zhang, L., Li, C., & Wu, J. (2019) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2018.2836440 · Cited by 268
Metadata
- Authors
- Tang, B., Kay, S., & He, H.
- Venue
- IEEE Transactions on Knowledge and Data Engineering
- Published
- 2016-09-01
- Volume
- 28
- Issue
- 9
- Pages
- 2508-2521
- ISSN
- 1041-4347 (ISSN-L)
- OpenAlex
- W2263270445
- Cited by
- 253
- References
- 45
- FWCI
- 27.39
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.