The DOI matches its Crossref record; 53 of the 53 references it cites were found in a registry or an index.
Online Feature Selection and Its Applications
Abstract
Feature selection is an important technique for data mining. Despite its importance, most studies of feature selection are restricted to batch learning. Unlike traditional batch learning methods, online learning represents a promising family of efficient and scalable machine learning algorithms for large-scale applications. Most existing studies of online learning require accessing all the attributes/features of training instances. Such a classical setting is not always appropriate for real-world applications when data instances are of high dimensionality or it is expensive to acquire the full set of attributes/features. To address this limitation, we investigate the problem of online feature selection (OFS) in which an online learner is only allowed to maintain a classifier involved only a small and fixed number of features. The key challenge of online feature selection is how to make accurate prediction for an instance using a small number of active features. This is in contrast to the classical setup of online learning where all the features can be used for prediction. We attempt to tackle this challenge by studying sparsity regularization and truncation techniques. Specifically, this article addresses two different tasks of online feature selection: 1) learning with full input, where an learner is allowed to access all the features to decide the subset of active features, and 2) learning with partial input, where only a limited number of features is allowed to be accessed for each instance by the learner. We present novel algorithms to solve each of the two problems and give their performance analysis. We evaluate the performance of the proposed algorithms for online feature selection on several public data sets, and demonstrate their applications to real-world problems including image classification in computer vision and microarray gene expression analysis in bioinformatics. The encouraging results of our experiments validate the efficacy and efficiency of the proposed techniques.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Donoho, D. L. (2006) · IEEE Transactions on Information Theory · doi:10.1109/tit.2006.871582 · Cited by 23,994
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,395
Exploration and Exploitation in Organizational Learning
March, J. G. (1991) · Organization Science · doi:10.1287/orsc.2.1.71 · Cited by 21,942
Donoho, D. L. (2004) · Cited by 16,703
The perceptron: A probabilistic model for information storage and organization in the brain.
Rosenblatt, F. (1958) · Psychological Review · doi:10.1037/h0042519 · Cited by 12,280
Peng, H., Long, F., & Ding, C. (2005) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2005.159 · Cited by 10,544
Wrappers for feature subset selection
Kohavi, R., & John, G. H. (1997) · Artificial Intelligence · doi:10.1016/s0004-3702(97)00043-x · Cited by 8,947
An Introduction to Variable and Feature Selection.
Guyon, I., & Elisseeff, A. (2003) · Cited by 6,576
A review of feature selection techniques in bioinformatics
Saeys, Y., Inza, I., & Larrañaga, P. (2007) · Bioinformatics · doi:10.1093/bioinformatics/btm344 · Cited by 5,244
Alon, U., Barkai, N., Notterman, D. A., Gish, K., Ybarra, S., Mack, D., & Levine, A. J. (1999) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.96.12.6745 · Cited by 4,167
Feature selection for classification
DASH, M., & LIU, H. (1997) · Intelligent Data Analysis · doi:10.1016/s1088-467x(97)00008-5 · Cited by 3,574
(2000) · CrossRef Listing of Deleted DOIs · doi:10.1162/153244303322753616 · Cited by 3,502
Prediction, Learning, and Games
Cesa-Bianchi, N., & Lugosi, G. (2006) · Cambridge University Press eBooks · doi:10.1017/cbo9780511546921 · Cited by 3,162
MINIMUM REDUNDANCY FEATURE SELECTION FROM MICROARRAY GENE EXPRESSION DATA
DING, C., & PENG, H. (2005) · Journal of Bioinformatics and Computational Biology · doi:10.1142/s0219720005001004 · Cited by 2,751
Toward integrating feature selection algorithms for classification and clustering
Liu, H., & Yu, L. (2005) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2005.66 · Cited by 2,706
Feature selection for high-dimensional data: a fast correlation-based filter solution
Yu, L., & Liu, H. (2003) · Cited by 2,305
Laplacian Score for Feature Selection
He, X., Cai, D., & Niyogi, P. (2005) · Cited by 1,962
Online Passive-Aggressive Algorithms
Crammer, K., Dekel, O., Keshet, J., Shalev‐Shwartz, S., & Singer, Y. (2003) · Cited by 1,682
Large margin classification using the perceptron algorithm
Freund, Y., & Schapire, R. E. (1998) · Proceedings of the eleventh annual conference on Computational learning theory · doi:10.1145/279943.279985 · Cited by 1,071
Spectral feature selection for supervised and unsupervised learning
Zhao, Z., & Liu, H. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273641 · Cited by 906
Online Passive-Aggressive Algorithms
CrammerKoby, DekelOfer, KeshetJoseph, Shalev-ShwartzShai, & SingerYoram (2006) · Journal of Machine Learning Research · doi:10.5555/1248547.1248566 · Cited by 894
Large Margin Classification Using the Perceptron Algorithm
Freund, Y., & Schapire, R. E. (1999) · Machine Learning · doi:10.1023/a:1007662407062 · Cited by 604
l 2,1 -norm regularized discriminative feature selection for unsupervised learning
Yang, Y. P., Shen, H. T., Ma, Z., Huang, Z. H., & Zhou, X. (2011) · Rare & Special e-Zone (The Hong Kong University of Science and Technology) · doi:10.5591/978-1-57735-516-8/ijcai11-267 · Cited by 549
Efficient Online and Batch Learning Using Forward Backward Splitting
Duchi, J. C., & Singer, Y. (2009) · Cited by 517
Dimensionality Reduction via Sparse Support Vector Machines
Bi, J., Bennett, K. P., Embrechts, M. J., Breneman, C. M., Song, M., Guyon, I., & Elisseeff, A. (2003) · Cited by 407
Discriminative Semi-Supervised Feature Selection Via Manifold Regularization
Xu, Z., King, I., Lyu, M. R.-T., & Jin, R. (2010) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2010.2047114 · Cited by 384
Self-citationConfidence-weighted linear classification
Dredze, M., Crammer, K., & Pereira, F. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390190 · Cited by 375
Adaptive regularization of weight vectors
Crammer, K., Kulesza, A., & Dredze, M. (2013) · Machine Learning · doi:10.1007/s10994-013-5327-x · Cited by 337
Semi-supervised Feature Selection via Spectral Analysis
Zhao, Z., & Liu, H. (2007) · SIAM International Conference on Data Mining · doi:10.1137/1.9781611972771.75 · Cited by 311
Distributional word clusters vs. words for text categorization
Bekkerman, R., El‐Yaniv, R., Tishby, N., & Winter, Y. (2003) · doi:10.5555/944919.944969 · Cited by 279
A New Approximate Maximal Margin Classification Algorithm
Gentile, C. (2000) · Cited by 225
Online feature selection using grafting
Perkins, S. J., & Theiler, J. P. (2003) · Cited by 206
Efficient Online and Batch Learning Using Forward Backward Splitting
DuchiJohn, & SingerYoram (2009) · Journal of Machine Learning Research · doi:10.5555/1577069.1755882 · Cited by 199
Exclusive Lasso for Multi-task Feature Selection
Zhou, Y., Jin, R., & Hoi, S. C. H. (2010) · Cited by 190
Self-citationLIBOL: a library for online learning algorithms
Hoi, S. C. H., Wang, J., & Zhao, P. (2014) · DR-NTU (Nanyang Technological University) · Cited by 189
Self-citationSparse Online Learning via Truncated Gradient
Langford, J. C., Li, L., & Zhang, T. (2008) · arXiv (Cornell University) · doi:10.48550/arxiv.0806.4686 · Cited by 184
Zhao, P., Jin, R., Yang, T., & Hoi, S. C. H. (2011) · Cited by 183
Self-citationThe Forgetron: A Kernel-Based Perceptron on a Budget
Dekel, O., Shalev-Shwartz, S., & Singer, Y. (2008) · SIAM Journal on Computing · doi:10.1137/060666998 · Cited by 176
Online Streaming Feature Selection
Wu, X., Yu, K., Wang, H., & Ding, W. (2010) · Adelaide Research & Scholarship (AR&S) (University of Adelaide) · Cited by 141
Feature grouping and selection over an undirected graph
Yang, S., Yuan, L., Lai, Y., Shen, X., Wonka, P., & Ye, J. (2012) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2339530.2339675 · Cited by 132
Exact Convex Confidence-Weighted Learning
Crammer, K., Dredze, M. H., & Pereira, F. M. B. (2008) · Cited by 125
Kivinen, J., Smola, A. J., & Williamson, R. C. (2002) · The MIT Press eBooks · doi:10.7551/mitpress/1120.003.0105 · Cited by 103
Orabona, F., Keshet, J., & Barbara Caputo, B. (2008) · International Conference on Machine Learning · doi:10.1145/1390156.1390247 · Cited by 102
Online feature selection for mining big data
Hoi, S. C. H., Wang, J., Zhao, P., & Jin, R. (2012) · International Workshop on Big Data, Streams and Heterogeneous Source Mining: Algorithms, Systems, Programming Models and Applications (BigMine) · doi:10.1145/2351316.2351329 · Cited by 102
Self-citationOTL: A Framework of Online Transfer Learning
Zhao, P., & Hoi, S. C. H. (2010) · Cited by 101
Self-citationOnline Multiple Kernel Classification
Hoi, S. C. H., Jin, R., Zhao, P., & Yang, T. (2012) · Machine Learning · doi:10.1007/s10994-012-5319-2 · Cited by 98
Self-citationDirect convex relaxations of sparse SVM
Chan, A. B., Vasconcelos, N. M., & Lanckriet, G. R. G. (2007) · International Conference on Machine Learning · doi:10.1145/1273496.1273515 · Cited by 97
Double Updating Online Learning
Zhao, P., Hoi, S. C. H., & Jin, R. (2011) · doi:10.5555/1953048.2021051 · Cited by 87
Self-citationNon-monotonic feature selection
Xu, Z., Jin, R., Ye, J., Lyu, M. R.-T., & King, I. (2009) · International Conference on Machine Learning · doi:10.1145/1553374.1553520 · Cited by 66
Self-citationFast Bounded Online Gradient Descent Algorithms for Scalable Kernel-Based Online LearningPreprint
Zhao, P., Wang, J., Wu, P., Jin, R., & Hoi, S. C. H. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1206.4633 · Cited by 64
Self-citationEfficient Learning with Partially Observed AttributesPreprint
Cesa‐Bianchi, N., Shalev‐Shwartz, S., & Shamir, O. (2010) · arXiv (Cornell University) · doi:10.48550/arxiv.1004.4421 · Cited by 59
Cost-Sensitive Online Classification
Wang, J., Zhao, P., & Hoi, S. C. H. (2012) · Proceedings · doi:10.1109/icdm.2012.116 · Cited by 22
Self-citationBDUOL: Double Updating Online Learning on a Fixed Budget
Zhao, P., & Hoi, S. C. H. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33460-3_57 · Cited by 8
Self-citation
Cited by
Feature Selection: A Data Perspective
Li, J., Cheng, K., Wang, S., Morstatter, F., Trevino, R. P., Tang, J., & Liu, H. (2018) · ACM Computing Surveys · doi:10.1145/3136625 · Cited by 2,442
A survey of machine learning for big data processing
Qiu, J., Wu, Q., Ding, G., Xu, Y., & Feng, S. (2016) · EURASIP Journal on Advances in Signal Processing · doi:10.1186/s13634-016-0355-x · Cited by 894
Feature selection for classification: A review
Tang, J., Alelyani, S., & Liu, H. (2014) · doi:10.1201/b17320-7 · Cited by 757
Online learning: A comprehensive survey
Hoi, S. C. H., Sahoo, D., Lu, J., & Zhao, P. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.04.112 · Cited by 746
Self-citationFeature Selection: A Data Perspective
Li, J., Cheng, K., Wang, S., Morstatter, F., Treviño, R. P., Tang, J., & Liu, H. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1601.07996 · Cited by 739
Metadata
- Authors
- Wang, J., Zhao, P., Hoi, S. C. H., & Jin, R.
- Venue
- IEEE Transactions on Knowledge and Data Engineering
- Published
- 2014-03
- Volume
- 26
- Issue
- 3
- Pages
- 698-710
- ISSN
- 1041-4347 (ISSN-L)
- OpenAlex
- W2136051823
- Cited by
- 275
- References
- 53
- FWCI
- 16.92
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.