The DOI matches its OpenAlex record; 19 of the 19 references it cites were found in a registry or an index.
Deep Learning using Linear Support Vector Machines
Open accessAbstract
Recently, fully-connected and convolutional neural networks have been trained to achieve state-of-the-art performance on a wide variety of tasks such as speech recognition, image classification, natural language processing, and bioinformatics. For classification tasks, most of these "deep learning" models employ the softmax activation function for prediction and minimize cross-entropy loss. In this paper, we demonstrate a small but consistent advantage of replacing the softmax layer with a linear support vector machine. Learning minimizes a margin-based loss instead of the cross-entropy loss. While there have been various combinations of neural nets and SVMs in prior art, our results using L2-SVMs show that by simply replacing softmax with linear SVMs gives significant gains on popular deep learning datasets MNIST, CIFAR-10, and the ICML 2013 Representation Learning Workshop's face expression recognition challenge.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
The Nature of Statistical Learning Theory
Vapnik, V. N. (1995) · doi:10.1007/978-1-4757-2440-0 · Cited by 39,138
A training algorithm for optimal margin classifiers
Boser, B. E., Guyon, I. M., & Vapnik, V. N. (1992) · Proceedings of the fifth annual workshop on Computational learning theory · doi:10.1145/130385.130401 · Cited by 11,992
A comparison of methods for multiclass support vector machines
Hsu, C.-W., & Lin, C.-J. (2002) · IEEE Transactions on Neural Networks · doi:10.1109/72.991427 · Cited by 6,838
Dimensionality Reduction by Learning an Invariant Mapping
Hadsell, R., Chopra, S., & LeCun, Y. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.100 · Cited by 5,336
An analysis of single-layer networks in unsupervised feature learning
Coates, A., Ng, A. Y., & Lee, H. (2011) · International Conference on Artificial Intelligence and Statistics · Cited by 4,675
Lee, H., Grosse, R., Ranganath, R., & Ng, A. Y. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553453 · Cited by 2,412
What is the best multi-stage architecture for object recognition?
Jarrett, K., Kavukcuoglu, K., Ranzato, M. A., & LeCun, Y. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459469 · Cited by 2,153 · cited 1×
“These include (but not limited to) speech (Mohamed et al., 2009; Dahl et al., 2010) and vision (Jarrett et al., 2009; Ciresan et al., 2011; Rifai et al., 2011a; Krizhevsky et al., 2012).”
BackgroundHinton, G. E. (1990) · Machine Learning · doi:10.1016/b978-0-08-051055-2.50029-8 · Cited by 516
Learning a Nonlinear Embedding by Preserving Class Neighbourhood Structure
Salakhutdinov, R., & Hinton, G. E. (2007) · International Conference on Artificial Intelligence and Statistics · Cited by 440
Cited by
Recent advances in convolutional neural networks
Gu, J., Wang, Z., Kuen, J., Ma, L., Shahroudy, A., Shuai, B., Liu, T., Wang, X., Wang, G., Cai, J., et al. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2017.10.013 · Cited by 6,576
Deep Convolutional Neural Networks for Image Classification: A Comprehensive Review
Rawat, W., & Wang, Z. (2017) · Neural Computation · doi:10.1162/neco_a_00990 · Cited by 3,503
Deep Learning using Rectified Linear Units (ReLU)
Agarap, A. F. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.08375 · Cited by 2,399
AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild
Mollahosseini, A., Hasani, B., & Mahoor, M. H. (2019) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2017.2740923 · Cited by 2,338
BinaryConnect: Training Deep Neural Networks with binary weights during propagations
Courbariaux, M., Bengio, Y., & David, J.-P. (2015) · arXiv · doi:10.48550/arxiv.1511.00363 · Cited by 2,180
Courbariaux, M., Hubara, I., Soudry, D., El-Yaniv, R., & Bengio, Y. (2016) · arXiv · doi:10.48550/arxiv.1602.02830 · Cited by 2,106
Deep Facial Expression Recognition: A Survey
Li, S., & Deng, W. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2020.2981446 · Cited by 1,753
Challenges in Representation Learning: A Report on Three Machine Learning Contests
Goodfellow, I. J., Erhan, D., Carrier, P. L., Courville, A., Mirza, M., Hamner, B., Cukierski, W., Tang, Y., Thaler, D., Lee, D.-H., et al. (2013) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-42051-1_16 · Cited by 1,719
Self-citationApplications of Support Vector Machine (SVM) Learning in Cancer Genomics
(2018) · Cancer Genomics & Proteomics · doi:10.21873/cgp.20063 · Cited by 1,528
Reliable Crowdsourcing and Deep Locality-Preserving Learning for Expression Recognition in the Wild
Li, S., Deng, W., & Du, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.277 · Cited by 1,470
Zhang, W., Li, C., Peng, G., Chen, Y., & Zhang, Z. (2018) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2017.06.022 · Cited by 1,421
Automated Melanoma Recognition in Dermoscopy Images via Very Deep Residual Networks
Yu, L., Chen, H., Dou, Q., Qin, J., & Heng, P.-A. (2017) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2642839 · Cited by 1,140
Going deeper in facial expression recognition using deep neural networks
Mollahosseini, A., Chan, D., & Mahoor, M. H. (2016) · 2016 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv.2016.7477450 · Cited by 1,086
Lee, C., Xie, S., Gallagher, P., Zhang, Z., & Zhuowen Tu, A. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1409.5185 · Cited by 993
Region Attention Networks for Pose and Occlusion Robust Facial Expression Recognition
Wang, K., Peng, X., Yang, J., Meng, D., & Qiao, Y. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2956143 · Cited by 940
Suppressing Uncertainties for Large-Scale Facial Expression Recognition
Wang, K., Peng, X., Yang, J., Lu, S., & Qiao, Y. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00693 · Cited by 772
Challenges in representation learning: A report on three machine learning contests
Goodfellow, I. J., Erhan, D., Luc Carrier, P., Courville, A., Mirza, M., Hamner, B., Cukierski, W., Tang, Y., Thaler, D., Lee, D.-H., et al. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.09.005 · Cited by 697
Self-citationLi, S., & Deng, W. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2868382 · Cited by 660
Reducing the Hausdorff Distance in Medical Image Segmentation With Convolutional Neural Networks
Karimi, D., & Salcudean, S. E. (2020) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2019.2930068 · Cited by 635
Towards Deep Neural Network Architectures Robust to Adversarial Examples
Gu, S., & Rigazio, L. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.5068 · Cited by 622
Image based Static Facial Expression Recognition with Multiple Deep Network Learning
Yu, Z., & Zhang, C. (2015) · Proceedings of the 2015 ACM on International Conference on Multimodal Interaction · doi:10.1145/2818346.2830595 · Cited by 598
On Loss Functions for Deep Neural Networks in Classification
Janocha, K., & Czarnecki, W. M. (2017) · Schedae Informaticae · doi:10.4467/20838476si.16.004.6185 · Cited by 591
Gearbox fault diagnosis based on deep random forest fusion of acoustic and vibratory signals
Li, C., Sanchez, R.-V., Zurita, G., Cerrada, M., Cabrera, D., & Vásquez, R. E. (2016) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2016.02.007 · Cited by 470
Simple convolutional neural network on image classification
Guo, T., Dong, J., Li, H., & Gao, Y. (2017) · 2017 IEEE 2nd International Conference on Big Data Analysis (ICBDA)( · doi:10.1109/icbda.2017.8078730 · Cited by 408
Intelligent Waste Classification System Using Deep Learning Convolutional Neural Network
Adedeji, O., & Wang, Z. (2019) · Procedia Manufacturing · doi:10.1016/j.promfg.2019.05.086 · Cited by 400
Zhang, Y., Yuan, Y., Feng, Y., & Lu, X. (2019) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2019.2900302 · Cited by 354
Local Learning With Deep and Handcrafted Features for Facial Expression Recognition
Georgescu, M.-I., Ionescu, R. T., & Popescu, M. (2019) · IEEE Access · doi:10.1109/access.2019.2917266 · Cited by 331
Facial Expression Recognition With Visual Transformers and Attentional Selective Fusion
Ma, F., Sun, B., & Li, S. (2023) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2021.3122146 · Cited by 327
Facial Expression Recognition in the Wild via Deep Attentive Center Loss
Farzaneh, A. H., & Qi, X. (2021) · 2021 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv48630.2021.00245 · Cited by 326
Rudd, E. M., Jain, L. P., Scheirer, W. J., & Boult, T. E. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2707495 · Cited by 326
Island Loss for Learning Discriminative Features in Facial Expression Recognition
Cai, J., Meng, Z., Khan, A. S., Li, Z., OReilly, J., & Tong, Y. (2018) · 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018) · doi:10.1109/fg.2018.00051 · Cited by 321
Identity-Aware Convolutional Neural Network for Facial Expression Recognition
Meng, Z., Liu, P., Cai, J., Han, S., & Tong, Y. (2017) · 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017) · doi:10.1109/fg.2017.140 · Cited by 319
Li, C., Sanchez, R.-V., Zurita, G., Cerrada, M., Cabrera, D., & Vásquez, R. E. (2015) · Neurocomputing · doi:10.1016/j.neucom.2015.06.008 · Cited by 295
Sheng, T. J., Islam, M. S., Misran, N., Baharuddin, M. H., Arshad, H., Islam, M. R., Chowdhury, M. E. H., Rmili, H., & Islam, M. T. (2020) · IEEE Access · doi:10.1109/access.2020.3016255 · Cited by 295
A comprehensive survey on deep learning based malware detection techniques
M., G., & Sethuraman, S. C. (2023) · Computer Science Review · doi:10.1016/j.cosrev.2022.100529 · Cited by 277
Apple flower detection using deep convolutional networks
Dias, P. A., Tabb, A., & Medeiros, H. (2018) · Computers in Industry · doi:10.1016/j.compind.2018.03.010 · Cited by 265
Deep learning-based appearance features extraction for automated carp species identification
Banan, A., Nasiri, A., & Taheri-Garavand, A. (2020) · Aquacultural Engineering · doi:10.1016/j.aquaeng.2020.102053 · Cited by 262
Agarap, A. F. M. (2018) · Proceedings of the 2018 10th International Conference on Machine Learning and Computing · doi:10.1145/3195106.3195117 · Cited by 259
Mengu, D., Luo, Y., Rivenson, Y., & Ozcan, A. (2020) · IEEE Journal of Selected Topics in Quantum Electronics · doi:10.1109/jstqe.2019.2921376 · Cited by 255
Deep learning for human affect recognition: Insights and new developments
Rouast, P. V., Adam, M. T. P., & Chiong, R. (2021) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2018.2890471 · Cited by 243
Smirnov, E. A., Timoshenko, D. M., & Andrianov, S. N. (2014) · AASRI Procedia · doi:10.1016/j.aasri.2014.05.013 · Cited by 237
Multimodal Emotion Recognition Using Deep Neural Networks
Tang, H., Liu, W., Zheng, W.-L., & Lu, B.-L. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-70093-9_86 · Cited by 194
FERAtt: Facial Expression Recognition With Attention Net
Fernandez, P. D. M., Pena, F. A. G., Ren, T. I., & Cunha, A. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2019.00112 · Cited by 142
Mixed-Precision Deep Learning Based on Computational Memory
Nandakumar, S. R., Le Gallo, M., Piveteau, C., Joshi, V., Mariani, G., Boybat, I., Karunaratne, G., Khaddam-Aljameh, R., Egger, U., Petropoulos, A., et al. (2020) · Frontiers in Neuroscience · doi:10.3389/fnins.2020.00406 · Cited by 120
Recognizing Facial Expressions Using a Shallow Convolutional Neural Network
Miao, S., Xu, H., Han, Z., & Zhu, Y. (2019) · IEEE Access · doi:10.1109/access.2019.2921220 · Cited by 110
“Tang listed the top in the kaggle FER2013 competition, he used a simple neural network to extract features of static images and SVM hinge loss for classification, indicating that neural network is able to do better without handcraft features [27].”
MethodDeep Neural Networks with Relativity Learning for facial expression recognition
Guo, Y., Tao, D., Yu, J., Xiong, H., Li, Y., & Tao, D. (2016) · 2016 IEEE International Conference on Multimedia & Expo Workshops (ICMEW) · doi:10.1109/icmew.2016.7574736 · Cited by 93
(2014) · neural information processing systems · Cited by 66
Masoudnia, S., Mersa, O., Araabi, B. N., Vahabie, A.-H., Sadeghi, M. A., & Ahmadabadi, M. N. (2019) · Expert Systems with Applications · doi:10.1016/j.eswa.2019.03.040 · Cited by 57
Predicting Seizures from Electroencephalography Recordings: A Knowledge Transfer Strategy
Liang, J., Lu, R., Zhang, C., & Wang, F. (2016) · 2016 IEEE International Conference on Healthcare Informatics (ICHI) · doi:10.1109/ichi.2016.27 · Cited by 49
Human-in-the-loop AI in government: a case study
Benedikt, L., Joshi, C., Nolan, L., Henstra-Hill, R., Shaw, L., & Hook, S. (2020) · Proceedings of the 25th International Conference on Intelligent User Interfaces · doi:10.1145/3377325.3377489 · Cited by 24
Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations
Hubara, I., Courbariaux, M., Soudry, D., El-Yaniv, R., & Bengio, Y. (2016) · arXiv · doi:10.48550/arxiv.1609.07061
Metadata
- Authors
- Tang, Y.
- Venue
- arXiv (Cornell University)
- Publisher
- arXiv
- Published
- 2013
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1306.0239
- OpenAlex
- W1546411676
- Cited by
- 708
- References
- 19
- Type
- posted content
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.