The DOI matches its Crossref record; 53 of the 53 references it cites were found in a registry or an index.
Combating the Small Sample Class Imbalance Problem Using Feature Selection
Abstract
The class imbalance problem is encountered in real-world applications of machine learning and results in a classifier's suboptimal performance. Researchers have rigorously studied the resampling, algorithms, and feature selection approaches to this problem. No systematic studies have been conducted to understand how well these methods combat the class imbalance problem and which of these methods best manage the different challenges posed by imbalanced data sets. In particular, feature selection has rarely been studied outside of text classification problems. Additionally, no studies have looked at the additional problem of learning from small samples. This paper presents a first systematic comparison of the three types of methods developed for imbalanced data classification problems and of seven feature selection metrics evaluated on small sample data sets from different applications. We evaluated the performance of these metrics using area under the receiver operating characteristic (AUC) and area under the precision-recall curve (PRC). We compared each metric on the average performance across all problems and on the likelihood of a metric yielding the best performance on a specific problem. We examined the performance of these metrics inside each problem domain. Finally, we evaluated the efficacy of these metrics to see which perform best across algorithms. Our results showed that signal-to-noise correlation coefficient (S2N) and Feature Assessment by Sliding Thresholds (FAST) are great candidates for feature selection in most applications, especially when selecting very small numbers of features.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
SMOTE synthetic minority over-sampling technique
Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002) · Journal of Artificial Intelligence Research · doi:10.1613/jair.953 · Cited by 32,865
Golub, T. R., Slonim, D. K., Tamayo, P., Huard, C., Gaasenbeek, M., Mesirov, J. P., Coller, H., Loh, M. L., Downing, J. R., Caligiuri, M. A., et al. (1999) · Science · doi:10.1126/science.286.5439.531 · Cited by 11,516
Peng, H., Long, F., & Ding, C. (2005) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2005.159 · Cited by 10,544
Gene selection for cancer classification using support vector machines
Guyon, I., Weston, J., Barnhill, S., & Vapnik, V. (2002) · Machine Learning · doi:10.1023/a:1012487302797 · Cited by 9,954
An Introduction to Variable and Feature Selection.
Guyon, I., & Elisseeff, A. (2003) · Cited by 6,576
The relationship between Precision-Recall and ROC curves
Davis, J., & Goadrich, M. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143874 · Cited by 6,559
(2000) · CrossRef Listing of Deleted DOIs · doi:10.1162/153244303322753616 · Cited by 3,502
Floating search methods in feature selection
Pudil, P., Novovičová, J., & Kittler, J. (1994) · Pattern Recognition Letters · doi:10.1016/0167-8655(94)90127-9 · Cited by 3,098
Estimating attributes: Analysis and extensions of RELIEF
Kononenko, I. (1994) · Lecture Notes in Computer Science · doi:10.1007/3-540-57868-4_57 · Cited by 2,958
An extensive empirical study of feature selection metrics for text classification
Forman, G. (2003) · Journal of Machine Learning Research · doi:10.5555/944919.944974 · Cited by 2,394
Addressing the Curse of Imbalanced Training Sets: One-Sided Selection.
Kubát, M., & Matwin, S. (1997) · Cited by 2,230
Editorial: special issue on learning from imbalanced data sets
Chawla, N. V., Japkowicz, N., & Kotcz, A. (2004) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1007730.1007733 · Cited by 2,063
Efficient Feature Selection via Analysis of Relevance and Redundancy
Yu, L., & Liu, H. (2004) · doi:10.5555/1005332.1044700 · Cited by 1,983
The foundations of cost-sensitive learning
Elkan, C. (2001) · Cited by 1,945
Training linear SVMs in linear time
Joachims, T. (2006) · Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1150402.1150429 · Cited by 1,920
The feature selection problem: traditional methods and a new algorithm
Kira, K., & Rendell, L. A. (1992) · National Conference on Artificial Intelligence · Cited by 1,897
SMOTEBoost: Improving Prediction of the Minority Class in Boosting
Chawla, N. V., Lazarevic, A., Hall, L. O., & Bowyer, K. W. (2003) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-39804-2_12 · Cited by 1,668
Toward optimal feature selection
Koller, D., & Sahami, M. (1996) · Cited by 1,482
Mining with rarity: a unifying framework
Weiss, G. M. (2004) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1007730.1007734 · Cited by 1,393
MetaCost: a general method for making classifiers cost-sensitive
Domingos, P. (1999) · Proceedings of the fifth ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/312129.312220 · Cited by 1,303
One-class svms for document classification
Manevitz, L. M., & Yousef, M. (2002) · doi:10.5555/944790.944808 · Cited by 1,207
Learning classifiers from only positive and unlabeled data
Elkan, C., & Noto, K. (2008) · Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1401890.1401920 · Cited by 1,028
A Multiple Resampling Method for Learning from Imbalanced Data Sets
Estabrooks, A., Jo, T., & Japkowicz, N. (2004) · Computational Intelligence · doi:10.1111/j.0824-7935.2004.t01-1-00228.x · Cited by 1,014
Learning when training data are costly: the effect of class distribution on tree induction
Weiss, G. M., & Provost, F. (2003) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1199 · Cited by 921
Fawcett, T., & Provost, F. (1997) · Data Mining and Knowledge Discovery · doi:10.1023/a:1009700419189 · Cited by 881
Experimental perspectives on learning from imbalanced data
Van Hulse, J., Khoshgoftaar, T. M., & Napolitano, A. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273614 · Cited by 786
Fast Binary Feature Selection with Conditional Mutual Information
FleuretFrançois (2004) · Journal of Machine Learning Research · doi:10.5555/1005332.1044711 · Cited by 593
Feature selection for text categorization on imbalanced data
Zheng, Z., Wu, X., & Srihari, R. (2004) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1007730.1007741 · Cited by 549
An Introduction to Machine Learning
Kubat, M. (2017) · doi:10.1007/978-3-319-63913-0 · Cited by 445
Feature Selection for Unbalanced Class Distribution and Naive Bayes
Mladenić, D., & Grobelnik, M. (1999) · Cited by 427
Confidence-weighted linear classification
Dredze, M., Crammer, K., & Pereira, F. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390190 · Cited by 375
Boosting for Learning Multiple Classes with Imbalanced Class Distribution
Sun, Y., Kamel, M., & Wang, Y. (2006) · Sixth International Conference on Data Mining (ICDM'06) · doi:10.1109/icdm.2006.29 · Cited by 307
Extreme re-balancing for SVMs: a case study
Raskutti, B., & Kowalczyk, A. (2004) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1007730.1007739 · Cited by 287
Learning when negative examples abound
Kubat, M., Holte, R., & Matwin, S. (1997) · Lecture Notes in Computer Science · doi:10.1007/3-540-62858-4_79 · Cited by 281
Chen, X.-W., & Wasikowski, M. (2008) · Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1401890.1401910 · Cited by 216
Self-citationExploiting the Cost (In)sensitivity of Decision Tree Splitting Criteria
Drummond, C., & Holte, R. C. (2000) · Cited by 198
(2000) · CrossRef Listing of Deleted DOIs · doi:10.1162/153244303322753670 · Cited by 189
Learning from Little: Comparison of Classifiers Given Little Training
Forman, G., & Cohen, I. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-30116-5_17 · Cited by 163
Supervised Versus Unsupervised Binary-Learning by Feedforward Neural Networks
Japkowicz, N. (2001) · Machine Learning · doi:10.1023/a:1007660820062 · Cited by 154
An iterative method for multi-class cost-sensitive learning
Abe, N., Zadrozny, B., & Langford, J. C. (2004) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/1014052.1014056 · Cited by 148
Feature Selection and the Class Imbalance Problem in Predicting Protein Function from Sequence
Al‐Shahib, A., Breitling, R., & Gilbert, D. (2005) · Applied Bioinformatics · doi:10.2165/00822942-200504030-00004 · Cited by 120
A Bias-Variance Analysis of a Real World Learning Problem: The CoIL Challenge 2000
van der Putten, P., & van Someren, M. (2004) · Machine Learning · doi:10.1023/b:mach.0000035476.95130.99 · Cited by 116
Overfitting in Wrapper-Based Feature Subset Selection: The Harder You Try the Worse it Gets
Loughrey, J., & Cunningham, P. (2007) · Springer eBooks · doi:10.1007/1-84628-102-4_3 · Cited by 113
Learning classifiers from imbalanced data based on biased minimax probability machine
Huang, K., Yang, H., King, I., & Lyu, M. R.-T. (2004) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2004.1315213 · Cited by 96
Learning a kernel function for classification with small training samples
Hertz, T., Bar-Hillel, A., & Weinshall, D. (2006) · International Conference on Machine Learning · doi:10.1145/1143844.1143895 · Cited by 93
Masnadi-Shirazi, H., & Vasconcelos, N. M. (2007) · International Conference on Machine Learning · doi:10.1145/1273496.1273573 · Cited by 82
Lund, O., Nielsen, M., Lundegaard, C., Keşmir, C., & Brunak, S. (2005) · The MIT Press eBooks · doi:10.7551/mitpress/3679.001.0001 · Cited by 74
Magical thinking in data mining: lessons from CoIL challenge 2000
Elkan, C. (2001) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/502512.502576 · Cited by 72
AUC Maximizing Support Vector Learning
Brefeld, U., & Scheffer, T. (2005) · Cited by 70
Kernel-based distance metric learning for microarray data classification
Xiong, H., & Chen, X.-W. (2006) · BMC Bioinformatics · doi:10.1186/1471-2105-7-299 · Cited by 58
Self-citationPruning support vectors for imbalanced data classification
Chen, X., Gerlach, B., & Casasent, D. P. (2006) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2005.1556167 · Cited by 50
Self-citationA study of sample size with neural network
Cui, Y.-J., Davis, S., Cheng, C., & Bai, X. (2005) · International Conference on Machine Learning and Cybernetics · doi:10.1109/icmlc.2004.1380382 · Cited by 21
Feature Selection for Support Vector Machines
Cao, L. J., & Tay, F. E. H. (2000) · Intelligent Data Engineering and Automated Learning · Cited by 20
Cited by
Learning from class-imbalanced data: Review of methods and applications
Haixiang, G., Yijing, L., Shang, J., Mingyun, G., Yuanyue, H., & Bing, G. (2017) · Expert Systems with Applications · doi:10.1016/j.eswa.2016.12.035 · Cited by 2,377
SMOTE for Learning from Imbalanced Data: Progress and Challenges, Marking the 15-year Anniversary
Fernandez, A., Garcia, S., Herrera, F., & Chawla, N. V. (2018) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.11192 · Cited by 2,219
López, V., Fernández, A., García, S., Palade, V., & Herrera, F. (2013) · Information Sciences · doi:10.1016/j.ins.2013.07.007 · Cited by 1,590
A Survey of Predictive Modeling on Imbalanced Domains
Branco, P., Torgo, L., & Ribeiro, R. P. (2017) · ACM Computing Surveys · doi:10.1145/2907070 · Cited by 1,129
Training and assessing classification rules with imbalanced data
Menardi, G., & Torelli, N. (2014) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-012-0295-5 · Cited by 765
Metadata
- Authors
- Wasikowski, M., & Chen, X.-W.
- Venue
- IEEE Transactions on Knowledge and Data Engineering
- Published
- 2010-10
- Volume
- 22
- Issue
- 10
- Pages
- 1388-1400
- ISSN
- 1041-4347 (ISSN-L)
- OpenAlex
- W2138776277
- Cited by
- 390
- References
- 53
- FWCI
- 12.29
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.