Variational inference based automatic relevance determination kernel for embedded feature selection of noisy industrial data
Abstract
In this paper, an embedded feature selection based on variational relevance vector machines is proposed to simultaneously perform feature selection and model construction. With the settings of specific hierarchical priors over the parameters of an automatic relevance determination kernel (ARDK) function, an approximate posterior distribution over these parameters is here derived and expressed as a multivariate Gaussian distribution, in which a first-order Taylor expansion-based Laplace approximation with respect to the parameters is introduced into the variational inference procedure. The posterior distributions, rather than generic pointwise estimates, over the rest of parameters of the model are also derived. The proposed method can simultaneously select relevant features and samples by adjusting the parameters of ARDK and the weighting vector, respectively. To verify the effectiveness of the proposed method, a synthetic dataset and a number of benchmark datasets, as well as a practical industrial dataset, are employed to solve the regression and classification problems. These experimental results indicate that the proposed method supports the mechanisms of feature selection and model construction while maintaining prediction performance, particularly in an industrial environment.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
RVM provides performance comparable to SVM, while utilizing arbitrary kernel functions with high sparsity and also offering probabilistic predictions.
“RVM provides performance comparable to SVM, while utilizing arbitrary kernel functions with high sparsity and also offering probabilistic predictions [143].”
From Data-driven health estimation and lifetime prediction of lithium-ion batteries: A review (2019) · cites this paper as [143]
Cites
LIBSVM: A library for support vector machines
Chang, C.-C., & Lin, C.-J. (2011) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/1961189.1961199 · Cited by 40,918 · see citing works
The Nature of Statistical Learning Theory
Vapnik, V. N. (1995) · doi:10.1007/978-1-4757-2440-0 · Cited by 39,138 · see citing works
Gaussian Processes for Machine Learning
Rasmussen, C. E., & Williams, C. K. I. (2005) · doi:10.7551/mitpress/3206.001.0001 · Cited by 11,705 · see citing works
(2006) · Springer eBooks · Cited by 8,123 · see citing works
Weston, J., Mukherjee, S., Chapelle, O., Pontil, M., Poggio, T., & Vapnik, V. N. (2000) · Cited by 986 · see citing works
Joint Embedding Learning and Sparse Regression: A Framework for Unsupervised Feature Selection
Hou, C., Nie, F., Li, X., Yi, D., & Wu, Y. (2013) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2013.2272642 · Cited by 555 · see citing works
More generality in efficient multiple kernel learning
Varma, M., & Babu, B. R. (2009) · International Conference on Machine Learning · doi:10.1145/1553374.1553510 · Cited by 400 · see citing works
Bayesian Gaussian Process Latent Variable Model
Titsias, M. K., & Lawrence, N. D. (2010) · International Conference on Artificial Intelligence and Statistics · Cited by 378 · see citing works
Variational Relevance Vector MachinesPreprint
Bishop, C., & Tipping, M. E. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.3838 · Cited by 357 · see citing works
WPD-PCA-Based Laser Welding Process Monitoring and Defects Diagnosis by Using FNN and SVM
You, D., Gao, X., & Katayama, S. (2014) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2014.2319216 · Cited by 272 · see citing works
Simultaneous feature selection and classification using kernel-penalized support vector machines
Maldonado, S., Weber, R. W., & Basak, J. K. (2010) · Information Sciences · doi:10.1016/j.ins.2010.08.047 · Cited by 258 · see citing works
Optimal feature selection for support vector machines
Nguyen, M. H., & la Torre, F. D. (2009) · Pattern Recognition · doi:10.1016/j.patcog.2009.09.003 · Cited by 238 · see citing works
Bayesian automatic relevance determination algorithms for classifying gene expression data
Li, Y., Campbell, C., & Tipping, M. E. (2002) · Bioinformatics · doi:10.1093/bioinformatics/18.10.1332 · Cited by 175 · see citing works
Variational inference in nonconjugate models
Wang, C., & Blei, D. M. (2013) · doi:10.5555/2567709.2502613 · Cited by 174 · see citing works
A Bayesian approach to joint feature selection and classifier design
Krishnapuram, B., Harternink, A. J., Carin, L., & Figueiredo, M. A. T. (2004) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2004.55 · Cited by 158 · see citing works
Saxén, H., Gao, C., & Gao, Z. (2012) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2012.2226897 · Cited by 125 · see citing works
Variational inference for latent variables and uncertain inputs in Gaussian processes
Damianou, A., Titsias, M. K., & Lawrence, N. D. (2016) · White Rose Research Online (University of Leeds, The University of Sheffield, University of York) · doi:10.5555/2946645.2946687 · Cited by 109 · see citing works
MKBoost: A Framework of Multiple Kernel Boosting
Xia, H., & Hoi, S. C. H. (2012) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2012.89 · Cited by 73 · see citing works
Mohsenzadeh, Y., Sheikhzadeh, H., Reza, A. M., Bathaee, N. S., & Kalayeh, M. M. (2013) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2013.2260736 · Cited by 59 · see citing works
Spatio-Temporal Learning With the Online Finite and Infinite Echo-State Gaussian Processes
Soh, H., & Demiris, Y. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2014.2316291 · Cited by 58 · see citing works
Hassani, H., Zarei, J., Arefi, M. M., et al. (2017) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2017.2688963 · Cited by 57 · see citing works
Jacobs, J. P. (2012) · IEEE Transactions on Antennas and Propagation · doi:10.1109/tap.2012.2186252 · Cited by 55 · see citing works
Effective Noise Estimation-Based Online Prediction for Byproduct Gas System in Steel Industry
Zhao, J., Liu, Q., Pedrycz, W., et al. (2012) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2012.2205932 · Cited by 40 · see citing works
Self-citationFeature selection for clustering categorical data with an embedded modelling approach
Silvestre, C. M. V., Cardoso, M. G. M. S., & Figueiredo, M. A. T. (2014) · Expert Systems · doi:10.1111/exsy.12082 · Cited by 39 · see citing works
Chang, M., Kang, M., & PECHT, M. G. (2017) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2017.2677301 · Cited by 39 · see citing works
Sparse optimization in feature selection: application in neuroimaging
Kampa, K., Mehta, S., Chou, C., Chaovalitwongse, W. A., & Grabowski, T. J. (2014) · Journal of Global Optimization · doi:10.1007/s10898-013-0134-2 · Cited by 36 · see citing works
Variational Inference for Mahalanobis Distance Metrics in Gaussian Process Regression
Titsias, M. K., & Lázaro-Gredilla, M. (2013) · Cited by 22 · see citing works
Mohsenzadeh, Y., Sheikhzadeh, H., & Nazari, S. (2016) · Pattern Recognition · doi:10.1016/j.patcog.2016.06.028 · Cited by 22 · see citing works
Discovering Anomalies on Mixed-Type Data Using a Generalized Student- Based Approach
Lu, Y.-C., Chen, F., Wang, Y., & Lu, C. (2016) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2016.2583429 · Cited by 22 · see citing works
Primal explicit max margin feature selection for nonlinear support vector machines
Tayal, A., Coleman, T. F., & Li, Y. (2014) · Pattern Recognition · doi:10.1016/j.patcog.2014.01.003 · Cited by 19 · see citing works
A sparse Bayesian approach for joint feature selection and classifier learning
Lapedriza, À., Seguí, S., Masip, D., & Vitria, J. M. (2008) · Pattern Analysis and Applications · doi:10.1007/s10044-008-0130-1 · Cited by 13 · see citing works
Maldonado, S., & López, J. (2015) · Intelligent Data Analysis · doi:10.3233/ida-150781 · Cited by 13 · see citing works
Chen, L., Liu, Y., Zhao, J., et al. (2016) · Artificial Intelligence Review · doi:10.1007/s10462-016-9465-y · Cited by 11 · see citing works
Self-citationKernel learning at the first level of inference
Cawley, G. C., & Talbot, N. L. C. (2014) · Neural Networks · doi:10.1016/j.neunet.2014.01.011 · Cited by 4 · see citing works
Gaussian process classification using automatic relevance determination for SAR target recognition
Zhang, X., Gou, L., Hou, B., et al. (2010) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.864845 · Cited by 2 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Data-driven health estimation and lifetime prediction of lithium-ion batteries: A review
Li, Y., Liu, K., Foley, A. M., Zülke, A., Berecibar, M., Nanini-Maury, E., Van Mierlo, J., & Hoster, H. E. (2019) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2019.109254 · Cited by 1,193 · see citing works
“Hence, it is recommended to construct an isotropic kernel with an advanced structure such as automatic relevance determination [142].”
Background
- Authors
- Zhao, J., Chen, L., Pedrycz, W., & Wang, W.
- Affiliations
- Dalian University of Technology (CN)
- University of Alberta (CA)
- Venue
- IEEE Transactions on Industrial Electronics
- Published
- 2018
- Pages
- 1-1
- ISSN
- 0278-0046 (ISSN-L)
- 1557-9948
- OpenAlex
- W2789439234
- Cited by
- 50
- References
- 35
- Citations by year
- 2025: 3 · 2024: 4 · 2023: 2 · 2022: 10 · 2021: 5 · 2020: 9 · 2019: 13 · 2018: 4
- FWCI
- 4.34
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Transactions on Industrial Electronics (publishedVersion)
- Funders
- the National Natural Sciences Foundation of China (61522304, 61533005, U1560102)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Gaussian Processes and Bayesian Inference
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Inference
- Keywords
- embedded feature selection, variational inference, relevance vector machine, Laplace approximation, hierarchical priors, sample selection, regression analysis, classification, industrial data
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
