Incremental support vector learning for ordinal regression
Abstract
Support vector ordinal regression (SVOR) is a popular method to tackle ordinal regression problems. However, until now there were no effective algorithms proposed to address incremental SVOR learning due to the complicated formulations of SVOR. Recently, an interesting accurate on-line algorithm was proposed for training ν -support vector classification (ν-SVC), which can handle a quadratic formulation with a pair of equality constraints. In this paper, we first present a modified SVOR formulation based on a sum-of-margins strategy. The formulation has multiple constraints, and each constraint includes a mixture of an equality and an inequality. Then, we extend the accurate on-line ν-SVC algorithm to the modified formulation, and propose an effective incremental SVOR algorithm. The algorithm can handle a quadratic formulation with multiple constraints, where each constraint is constituted of an equality and an inequality. More importantly, it tackles the conflicts between the equality and inequality constraints. We also provide the finite convergence analysis for the algorithm. Numerical experiments on the several benchmark and real-world data sets show that the incremental algorithm can converge to the optimal solution in a finite number of steps, and is faster than the existing batch and incremental SVOR algorithms. Meanwhile, the modified formulation has better accuracy than the existing incremental SVOR algorithm, and is as accurate as the sum-of-margins based formulation of Shashua and Levin.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Facial expression detection has a fast and strong but fluctuating response, and EEG detection has a smooth but stable response over the trial time.
“The reason could be based on the fact that the facial expression detection has a fast and strong but fluctuating response, and the EEG detection had a smooth but stable response over the trial time [20].”
From Fusion of Facial Expressions and EEG for Multimodal Emotion Recognition (2017) · cites this paper as [20]
Decremental and incremental strategies have been applied to integrate relevant data samples for SVR training and ignore irrelevant parts.
“In order to improve the stability and robustness of SVR with largescale training samples, decremental and incremental strategies [136,137] have been applied to integrate the relevant data sample for SVR training and ignore the irrelevant part.”
From Data-driven health estimation and lifetime prediction of lithium-ion batteries: A review (2019) · cites this paper as [137]
Cites
Boyd, S., & Vandenberghe, L. (2004) · doi:10.1017/cbo9780511804441 · Cited by 56,601 · see citing works
Wu, Y., & Vapnik, V. N. (1999) · Technometrics · doi:10.2307/1271368 · Cited by 27,236 · see citing works
UCI Machine Learning Repository
Asuncion, A. (2007) · Medical Entomology and Zoology · Cited by 24,190 · see citing works
Schölkopf, B., & Smola, A. J. (2001) · The MIT Press eBooks · doi:10.7551/mitpress/4175.001.0001 · Cited by 11,934 · see citing works
Magnetic Resonance Classification of Lumbar Intervertebral Disc Degeneration
Pfirrmann, C. W. A., Metzdorf, A., Zanetti, M., Hodler, J., & Boos, N. (2001) · Spine · doi:10.1097/00007632-200109010-00011 · Cited by 4,032 · see citing works
(1999) · doi:10.1162/089976600300015565 · Cited by 2,799 · see citing works
Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond
Atiya, A. F. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2005.848998 · Cited by 2,288 · see citing works
The Theory of Matrices in Numerical Analysis.
Haynsworth, E. V., & Householder, A. S. (1966) · American Mathematical Monthly · doi:10.2307/2314680 · Cited by 1,370 · see citing works
THE THEORY OF MATRICES IN NUMERICAL ANALYSIS
Mirsky, L. (1965) · Journal of the London Mathematical Society · doi:10.1112/jlms/s1-40.1.378b · Cited by 1,170 · see citing works
Incremental and Decremental Support Vector Machine Learning
Cauwenberghs, G., & Poggio, T. (2000) · Cited by 1,156 · see citing works
Minima of Functions of Several Variables with Inequalities as Side Conditions
Karush, W. (2013) · Springer eBooks · doi:10.1007/978-3-0348-0439-4_10 · Cited by 863 · see citing works
The Entire Regularization Path for the Support Vector Machine
Hastie, T. J., Rosset, S., Tibshirani, R. J., & Zhu, J. (2004) · Cited by 617 · see citing works
Schölkopf, B. (1997) · Cited by 560 · see citing works
Support vector learning for ordinal regression
Herbrich, R. (1999) · doi:10.1049/cp:19991091 · Cited by 457 · see citing works
Incremental Support Vector Learning: Analysis, Implementation and Applications
Laskov, P., Gehl, C., Krüger, S., & Müller, K. (2006) · Cited by 322 · see citing works
Support Vector Ordinal Regression
Chu, W., & Keerthi, S. S. (2007) · Neural Computation · doi:10.1162/neco.2007.19.3.792 · Cited by 320 · see citing works
Ordinal Regression by Extended Binary Classification
Li, L., & Lin, H.-T. (2007) · The MIT Press eBooks · doi:10.7551/mitpress/7503.003.0113 · Cited by 253 · see citing works
On the convergence of the decomposition method for support vector machines
Lin, C. (2001) · IEEE Transactions on Neural Networks · doi:10.1109/72.963765 · Cited by 245 · see citing works
Incremental Support Vector Learning: Analysis, Implementation and Applications
LaskovPavel, GehlChristian, KrügerStefan, et al. (2006) · Journal of Machine Learning Research · doi:10.5555/1248547.1248616 · Cited by 188 · see citing works
Multiple Incremental Decremental Learning of Support Vector Machines
Karasuyama, M., & Takeuchi, I. (2009) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2010.2048039 · Cited by 176 · see citing works
Learning to Classify Ordinal Data: The Data Replication Method
Cardoso, J. S., & da Costa, J. P. (2007) · doi:10.5555/1314498.1314546 · Cited by 163 · see citing works
Soft Margin Multiple Kernel Learning
Xu, X., Tsang, I. W.-H., & Xu, D. (2013) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2012.2237183 · Cited by 140 · see citing works
A new family of online algorithms for category ranking
Crammer, K., & Singer, Y. (2002) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/564376.564404 · Cited by 74 · see citing works
Robust Support Vector Regression for Uncertain Input and Output Data
Huang, G. M., Song, S., Wu, C., et al. (2012) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2012.2212456 · Cited by 54 · see citing works
Taxonomy of Large Margin Principle Algorithms for Ordinal Regression Problems
Shashua, A., & Levin, A. (2002) · Cited by 52 · see citing works
On-Line Support Vector Machine Regression
Martín, M. (2002) · Lecture notes in computer science · doi:10.1007/3-540-36755-1_24 · Cited by 51 · see citing works
A kernel path algorithm for support vector machines
Wang, G., Yeung, D., & Lochovsky, F. H. (2007) · International Conference on Machine Learning · doi:10.1145/1273496.1273616 · Cited by 48 · see citing works
Feasibility and Finite Convergence Analysis for Accurate On-Line $\nu$-Support Vector Machine
Gu, B., & Sheng, V. S. (2013) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2013.2250300 · Cited by 47 · see citing works
Self-citationTransductive Ordinal Regression
Seah, C.-W., Tsang, I. W.-H., & Ong, Y.-S. (2012) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2012.2198240 · Cited by 45 · see citing works
Crammer, K., & Singer, Y. (2004) · Neural Computation · doi:10.1162/0899766052530848 · Cited by 45 · see citing works
McCrea, M. V., Sherali, H. D., & Trani, A. A. (2007) · Transportation Research Part C Emerging Technologies · doi:10.1016/j.trc.2007.09.001 · Cited by 41 · see citing works
A new family of online algorithms for category ranking
Crammer, K., & Singer, Y. (2002) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/564400.564404 · Cited by 39 · see citing works
On Parametric Linear and Quadratic Programming Problems.
Ritter, K. (1981) · doi:10.21236/ada100602 · Cited by 32 · see citing works
Computing the Solution Path for the Regularized Support Vector Regression
Gunter, L., & Zhu, J. (2005) · neural information processing systems · Cited by 28 · see citing works
Accurate on-line -support vector learning
Gu, B., Wang, J., Yu, Y., et al. (2011) · Neural Networks · doi:10.1016/j.neunet.2011.10.006 · Cited by 24 · see citing works
Self-citationGeneralization Bounds for Some Ordinal Regression Algorithms
Agarwal, S. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-87987-9_6 · Cited by 20 · see citing works
On-line off-line Ranking Support Vector Machine and analysis
Gu, B., Wang, J., & Chen, H. (2008) · IEEE International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2008.4633975 · Cited by 9 · see citing works
Self-citationOnline Learning of Complex Prediction Problems Using Simultaneous Projections
Amit, Y., Shalev‐Shwartz, S., & Singer, Y. (2008) · Cited by 5 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Data-driven health estimation and lifetime prediction of lithium-ion batteries: A review
Li, Y., Liu, K., Foley, A. M., Zülke, A., Berecibar, M., Nanini-Maury, E., Van Mierlo, J., & Hoster, H. E. (2019) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2019.109254 · Cited by 1,193 · see citing works
“In order to improve the stability and robustness of SVR with largescale training samples, decremental and incremental strategies [136,137] have been applied to integrate the relevant data sample for SVR training and ignore the irrelevant part.”
MethodA survey of depth and inertial sensor fusion for human action recognition
Chen, C., Jafari, R., & Kehtarnavaz, N. (2017) · Multimedia Tools and Applications · doi:10.1007/s11042-015-3177-1 · Cited by 356 · see citing works
Text classification based on deep belief network and softmax regression
Jiang, M., Liang, Y., Feng, X., Fan, X., Pei, Z., Xue, Y., & Guan, R. (2018) · Neural Computing and Applications · doi:10.1007/s00521-016-2401-x · Cited by 268 · see citing works
Fusion of Facial Expressions and EEG for Multimodal Emotion Recognition
Huang, Y., Yang, J., Liao, P., & Pan, J. (2017) · Computational Intelligence and Neuroscience · doi:10.1155/2017/2107451 · Cited by 115 · see citing works
Measures of uncertainty for neighborhood rough sets
Chen, Y., Xue, Y., Ma, Y., & Xu, F. (2017) · Knowledge-Based Systems · doi:10.1016/j.knosys.2017.01.008 · Cited by 87 · see citing works
Lü, Q., & Li, H. (2016) · arXiv · doi:10.48550/arxiv.1611.00990 · Cited by 54 · see citing works
- Authors
- Gu, B., Sheng, V. S., Tay, K. Y., Romano, W., & Li, S.
- Affiliations
- Western University (CA)
- University of Central Arkansas (US)
- London Health Sciences Centre (CA)
- St Joseph's Health Care (CA)
- Venue
- IEEE Transactions on Neural Networks and Learning Systems
- Published
- 2015-07
- Volume
- 26
- Issue
- 7
- Pages
- 1403-1416
- ISSN
- 2162-237X (ISSN-L)
- 2162-2388
- OpenAlex
- W2061879449
- Cited by
- 707
- References
- 38
- Citations by year
- 2026: 6 · 2025: 5 · 2024: 13 · 2023: 14 · 2022: 23 · 2021: 20 · 2020: 29 · 2019: 41 · 2018: 97 · 2017: 245 · 2016: 179 · 2015: 34 · 2013: 1
- FWCI
- 152.17
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Transactions on Neural Networks and Learning Systems (publishedVersion)
- PubMed (publishedVersion)
- Scholarship@Western (Western University) (submittedVersion)
- Funders
- Priority Academic Program Development, Jiangsu Higher Education Institutions
- U.S. National Science Foundation (IIS-1115417)
- National Natural Science Foundation of China (61232016, 61202137)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and ELM
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Keywords
- incremental learning, online learning, support vector machine classification, equality and inequality constraints, quadratic programming, ordinal classification
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
