The DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index.
Are Loss Functions All the Same?
Open accessAbstract
In this letter, we investigate the impact of choosing different loss functions from the viewpoint of statistical learning theory. We introduce a convexity assumption, which is met by all loss functions commonly used in the literature, and study how the bound on the estimation error changes with the loss. We also derive a general result on the minimizer of the expected risk for a convex loss function in the case of classification. The main outcome of our analysis is that for classification, the hinge loss appears to be the loss of choice. Other things being equal, the hinge loss leads to a convergence rate practically indistinguishable from the logistic loss rate and much better than the square loss rate. Furthermore, if the hypothesis space is sufficiently rich, the bounds obtained for the hinge loss are not loosened by the thresholding stage.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Nature of Statistical Learning Theory
Vapnik, V. N. (1995) · doi:10.1007/978-1-4757-2440-0 · Cited by 39,138
Wu, Y. (1999) · Technometrics · doi:10.1080/00401706.1999.10485951 · Cited by 27,221
The Elements of Statistical Learning
Hastie, T., Friedman, J., & Tibshirani, R. (2001) · Springer Series in Statistics · doi:10.1007/978-0-387-21606-5 · Cited by 24,162
An Introduction to Support Vector Machines and Other Kernel-based Learning Methods
Cristianini, N., & Shawe-Taylor, J. (2000) · doi:10.1017/cbo9780511801389 · Cited by 13,855
Aronszajn, N. (1950) · Transactions of the American Mathematical Society · doi:10.1090/s0002-9947-1950-0051437-7 · Cited by 5,380
An Introduction to Support Vector Machines
Cristianini, N., & Shawe-Taylor, J. S. (2000) · ePrints Soton (University of Southampton) · Cited by 4,115
On the mathematical foundations of learning
Cucker, F., & Smale, S. (2001) · Bulletin of the American Mathematical Society · doi:10.1090/s0273-0979-01-00923-5 · Cited by 1,545
Regularization Theory and Neural Networks Architectures
Girosi, F., Jones, M., & Poggio, T. (1995) · Neural Computation · doi:10.1162/neco.1995.7.2.219 · Cited by 1,349
Regularization Networks and Support Vector Machines
Evgeniou, T., Pontil, M., & Poggio, T. (2000) · Advances in Computational Mathematics · doi:10.1023/a:1018946025316 · Cited by 1,197
Statistical behavior and consistency of classification methods based on convex risk minimization
Zhang, T. (2004) · The Annals of Statistics · doi:10.1214/aos/1079120130 · Cited by 711
Aronszajn, N. (1950) · Transactions of the American Mathematical Society · doi:10.2307/1990404 · Cited by 495
Scale-sensitive dimensions, uniform convergence, and learnability
Alon, N., Ben-David, S., Cesa-Bianchi, N., & Haussler, D. (1997) · Journal of the ACM · doi:10.1145/263867.263927 · Cited by 340
The covering number in learning theory
Zhou, D.-X. (2002) · Journal of Complexity · doi:10.1006/jcom.2002.0635 · Cited by 316
Best Choices for Regularization Parameters in Learning Theory: On the Bias—Variance Problem
Cucker, F., & Smale, S. (2002) · Foundations of Computational Mathematics · doi:10.1007/s102080010030 · Cited by 261
On the Bayes-risk consistency of regularized boosting methods
Lugosi, G., & Vayatis, N. (2004) · The Annals of Statistics · doi:10.1214/aos/1079120129 · Cited by 185
SVM Soft Margin Classifiers: Linear Programming versus Quadratic Programming
Wu, Q., & Zhou, D.-X. (2005) · Neural Computation · doi:10.1162/0899766053491896 · Cited by 177
Statistical Properties and Adaptive Tuning of Support Vector Machines
Lin, Y., Wahba, G., Zhang, H., & Lee, Y. (2002) · Machine Learning · doi:10.1023/a:1013951620650 · Cited by 46
Cited by
Deep Learning for Anomaly Detection: A Review
Pang, G., Shen, C., Cao, L., & Hengel, A. V. D. (2022) · ACM Computing Surveys · doi:10.1145/3439950 · Cited by 2,837
“nstances randomly sampled from the dataset X, Aand Nrespectively be the pseudo-labeled anomaly and normal instance sets, with X= A∪Nand ∅= A∩N, its loss function is built upon the hinge loss function [131] and can be represented as L query = 1 |X| Õ x∈A,x′∈N max 0,m + f (x′,S;Θ)−f (x,S;Θ) , (25) wherem is a predefined…”
BackgroundRobust 1-Bit Compressive Sensing via Binary Stable Embeddings of Sparse Vectors
Jacques, L., Laska, J. N., Boufounos, P. T., & Baraniuk, R. G. (2013) · IEEE Transactions on Information Theory · doi:10.1109/tit.2012.2234823 · Cited by 653
Deep Learning and Its Applications in Biomedicine
Cao, C., Liu, F., Tan, H., Song, D., Shu, W., Li, W., Zhou, Y., Bo, X., & Xie, Z. (2018) · Genomics, Proteomics & Bioinformatics · doi:10.1016/j.gpb.2017.07.003 · Cited by 624
Sun, H., Burton, H. V., & Huang, H. (2021) · Journal of Building Engineering · doi:10.1016/j.jobe.2020.101816 · Cited by 544
Goldstein, B. A., Navar, A. M., & Carter, R. E. (2016) · European Heart Journal · doi:10.1093/eurheartj/ehw302 · Cited by 543
Metadata
- Authors
- Rosasco, L., Vito, E. D., Caponnetto, A., Piana, M., & Verri, A.
- Venue
- Neural Computation
- Publisher
- MIT Press - Journals
- Published
- 2004-05-01
- Volume
- 16
- Issue
- 5
- Pages
- 1063-1076
- ISSN
- 0899-7667 (ISSN-L)
- 1530-888X
- OpenAlex
- W2034365297
- Cited by
- 501
- References
- 17
- FWCI
- 1.02
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.