The DOI matches its Crossref record; 61 of the 61 references it cites were found in a registry or an index.
Beyond one-hot encoding: Lower dimensional target embedding
Open accessAbstract
Abstract Target encoding plays a central role when learning Convolutional Neural Networks. In this realm, one-hot encoding is the most prevalent strategy due to its simplicity. However, this so widespread encoding schema assumes a flat label space, thus ignoring rich relationships existing among labels that can be exploited during training. In large-scale datasets, data does not span the full label space, but instead lies in a low-dimensional output manifold. Following this observation, we embed the targets into a low-dimensional space, drastically improving convergence speed while preserving accuracy. Our contribution is two fold: (i) We show that random projections of the label space are a valid tool to find such lower dimensional embeddings, boosting dramatically convergence rates at zero computational cost; and (ii) we propose a normalized eigenrepresentation of the class manifold that encodes the targets with minimal information loss, improving the accuracy of random projections encoding while enjoying the same convergence rates. Experiments on CIFAR-100, CUB200-2011, Imagenet, and MIT Places demonstrate that the proposed approach drastically improves convergence speed while reaching very competitive accuracy rates.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,395
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.123 · Cited by 19,741
Normalized cuts and image segmentation
Shi, J., & Malik, J. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.868688 · Cited by 15,502
Proceedings of IEEE Conference on Computer Vision and Pattern Recognition
(1993) · Proceedings of IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.1993.341010 · Cited by 14,265
Proceedings of IEEE International Conference on Computer Vision
(1995) · Proceedings of IEEE International Conference on Computer Vision · doi:10.1109/iccv.1995.466933 · Cited by 12,860
Advances in Neural Information Processing Systems 25
Zhang, Y., Sutton, C. A., Storkey, A., & Zoubin Ghahramani, A. (2012) · Cited by 12,653
Deep Learning Face Attributes in the Wild
Liu, Z., Luo, P., Wang, X., & Tang, X. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.425 · Cited by 7,747
The Pascal Visual Object Classes Challenge: A Retrospective
Everingham, M., Eslami, S. M. A., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-014-0733-5 · Cited by 7,139
Proceedings of the 25th international conference on Machine learning
Cohen, W. W., McCallum, A., & Roweis, S. T. (2008) · Cited by 5,691
Convolutional networks for images, speech, and time series
LeCun, Y., & Bengio, Y. (1998) · HAL (Le Centre pour la Communication Scientifique Directe) · doi:10.5555/303568.303704 · Cited by 4,231
The Handbook of brain theory and neural networks
(1996) · Choice Reviews Online · doi:10.5860/choice.33-4484 · Cited by 3,870
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. J. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1408.5093 · Cited by 3,752
Advances in Neural Information Processing Systems 20
Bonilla, E. V., Chai, K. M. A., & Williams, C. K. I. (2008) · neural information processing systems · Cited by 2,846
Solving Multiclass Learning Problems via Error-Correcting Output Codes
Dietterich, T. G., & Bakiri, G. (1995) · Journal of Artificial Intelligence Research · doi:10.1613/jair.105 · Cited by 2,739
(2014) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 2,534
DeViSE: A Deep Visual-Semantic Embedding Model
Frome, A., Corrado, G. S., Shlens, J., Samy Bengio, A., Dean, J., Ranzato, M., & Mikolov, T. (2013) · Cited by 2,308
Large Margin Methods for Structured and Interdependent Output Variables
Tsochantaridis, I., Joachims, T., Hofmann, T., & Altün, Y. (2005) · MPG.PuRe (Max Planck Society) · Cited by 1,879
Advances in Neural Information Processing Systems 21
Griffiths, T. L., Lucas, C. G., Williams, J. J., & Kalish, M. L. (2009) · neural information processing systems · Cited by 1,854
On a class of error correcting binary group codes
Bose, R. C., & Ray-Chaudhuri, D. K. (1960) · Information and Control · doi:10.1016/s0019-9958(60)90287-4 · Cited by 1,367
Reducing Multiclass to Binary: A Unifying Approach for Margin Classifiers
Allwein, E. L., Schapire, R. E., & Singer, Y. (2000) · International Conference on Machine Learning · Cited by 1,312
Classification by pairwise coupling
Hastie, T., & Tibshirani, R. (1998) · The Annals of Statistics · doi:10.1214/aos/1028144844 · Cited by 1,299
HD-CNN: Hierarchical Deep Convolutional Neural Networks for Large Scale Visual Recognition
Yan, Z., Zhang, H., Piramuthu, R., Jagadeesh, V., DeCoste, D., Di, W., & Yu, Y. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.314 · Cited by 936 · cited 1×
“In the field of large-scale recognition, hierarchical approaches such as using tree-based priors [38], label relational graphs [11], CNN hierarchies [46], and HD-CNNs [47] have been proposed.”
MethodLabel-Embedding for Image Classification
Akata, Z., Perronnin, F., Harchaoui, Z., & Schmid, C. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2487986 · Cited by 770
Proceedings of the 22nd ACM international conference on Multimedia
(2014) · doi:10.1145/2647868 · Cited by 725
Wsabie: Scaling up to large vocabulary image annotation
Weston, J., Samy Bengio, A., & Usunier, N. (2011) · Cited by 677
On the Learnability and Design of Output Codes for Multiclass Problems
Crammer, K., & Singer, Y. (2002) · Machine Learning · doi:10.1023/a:1013637720281 · Cited by 650
Label-Embedding for Attribute-Based Classification
Akata, Z., Perronnin, F., Harchaoui, Z., & Schmid, C. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.111 · Cited by 618
Deep learning of binary hash codes for fast image retrieval
Lin, K., Yang, H.-F., Hsiao, J.-H., & Chen, C.-S. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2015.7301269 · Cited by 614
The handbook of brain theory and neural networks
Rattay, F. (1996) · Simulation Practice and Theory · doi:10.1016/0928-4869(96)83761-x · Cited by 500
Large scale image annotation: learning to rank with joint word-image embeddings
Weston, J., Bengio, S., & Usunier, N. (2010) · Machine Learning · doi:10.1007/s10994-010-5198-3 · Cited by 399
Error-Correcting Output Coding Corrects Bias and Variance
Kong, E. B., & Dietterich, T. G. (1995) · Machine Learning Proceedings 1995 · doi:10.1016/b978-1-55860-377-6.50046-3 · Cited by 365
Large-Scale Object Classification Using Label Relation Graphs
Deng, J., Ding, N., Jia, Y., Frome, A., Murphy, K., Bengio, S., Li, Y., Neven, H., & Adam, H. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_4 · Cited by 347
Evaluating knowledge transfer and zero-shot learning in a large-scale setting
Rohrbach, M., Stark, M., & Schiele, B. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995627 · Cited by 303
Label Embedding Trees for Large Multi-Class Tasks
Samy Bengio, A., Weston, J., & Grangier, D. (2010) · Cited by 298
Uncovering shared structures in multiclass classification
Amit, Y., Fink, M., Srebro, N., & Ullman, S. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273499 · Cited by 278
On the Decoding Process in Ternary Error-Correcting Output Codes
Escalera, S., Pujol, O., & Radeva, P. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.266 · Cited by 276
Self-citationPujol, O., Radeva, P., & Vitria, J. (2006) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2006.116 · Cited by 258
Multi-Label Prediction via Compressed Sensing
Hsu, D., Kakade, S. M., Langford, J. C., & Zhang, T. (2009) · arXiv (Cornell University) · doi:10.48550/arxiv.0902.1284 · Cited by 245
Xiao, T., Zhang, J., Yang, K., Peng, Y., & Zhang, Z. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654926 · Cited by 234
Learning and using taxonomies for fast visual categorization
Griffin, G., & Perona, P. (2008) · 2008 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2008.4587410 · Cited by 208
Discriminative Transfer Learning with Tree-based Priors
Srivastava, N., & Salakhutdinov, R. (2013) · neural information processing systems · Cited by 199
Attribute-Based Transfer Learning for Object Categorization with Zero/One Training Example
Yu, X., & Aloimonos, Y. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15555-0_10 · Cited by 170
Pattern Recognition, 2008. ICPR 2008. 19th International Conference on
Belle, V., Deselaers, T., & Schiffer, S. (2008) · Cited by 145
IEE Conference Publication 210
(1982) · IEE Proceedings C Generation Transmission and Distribution · doi:10.1049/ip-c.1982.0050 · Cited by 140
On the Learnability and Design of Output Codes for Multiclass Problems
Boros, E. (2002) · Machine Learning · Cited by 125
Subclass Problem-Dependent Design for Error-Correcting Output Codes
Escalera, S., Tax, D. M. J., Pujol, O., Radeva, P. I., & Duin, R. P. W. (2008) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.38 · Cited by 121
Self-citationOnline incremental attribute-based zero-shot learning
Kankuekul, P., Kawewong, A., Tangruamsub, S., & Hasegawa, O. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6248112 · Cited by 114
Large Margin Taxonomy Embedding for Document Categorization
Weinberger, K. Q., & Chapelle, O. (2008) · neural information processing systems · Cited by 77
Deep Representation Learning with Target Coding
Yang, S., Luo, P., Loy, C. C., Shum, K. W., & Tang, X. (2015) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v29i1.9796 · Cited by 75
Learning ECOC Code Matrix for Multiclass Classification with Application to Glaucoma Diagnosis
Bai, X., Niwas, S. I., Lin, W., Ju, B., Kwoh, C. K., Wang, L., Sng, C. C. A., Aquino, M. C., & Chew, P. T. K. (2016) · Journal of Medical Systems · doi:10.1007/s10916-016-0436-2 · Cited by 55
HD-CNN: Hierarchical Deep Convolutional Neural Network for Large Scale Visual RecognitionPreprint
Yan, Z., Zhang, H., Piramuthu, R., Jagadeesh, V., DeCoste, D., Wei, D., & Yu, Y. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1410.0736 · Cited by 40
Boosted ECOC ensembles for face recognition
Windeatt, T. (2003) · doi:10.1049/cp:20030513 · Cited by 32
Deng, H., Stathopoulos, G., & Suen, C. Y. (2010) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2010.1043 · Cited by 28
Spectral error correcting output codes for efficient multiclass recognition
Zhang, X., Liang, L., & Shum, H. (2009) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2009.5459355 · Cited by 26
Learning Discriminative Features via Label Consistent Neural Network
Jiang, Z., Wang, Y., Davis, L. S., Andrews, W. D., & Rozgić, V. (2017) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2017.30 · Cited by 26
Facial action unit recognition using multi-class classification
Smith, R. S., & Windeatt, T. (2014) · Neurocomputing · doi:10.1016/j.neucom.2014.07.066 · Cited by 23
Circular ECOC. A theoretical and experimental analysis
Ghaderi, R., & Windeau, T. (2002) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2000.906048 · Cited by 12
Cited by
Al-Shehari, T., & Alsowail, R. A. (2021) · Entropy · doi:10.3390/e23101258 · Cited by 193
Danilevicz, M. F., Bayer, P. E., Boussaid, F., Bennamoun, M., & Edwards, D. (2021) · Remote Sensing · doi:10.3390/rs13193976 · Cited by 75
Bringing Machine Learning Classifiers Into Critical Cyber-Physical Systems: A Matter of Design
Sayin, B., Zoppi, T., Marchini, N., Khokhar, F. A., & Passerini, A. (2025) · IEEE Access · doi:10.1109/access.2025.3568501 · Cited by 7
“Categorical features usually require preprocessing to be used properly: a common preprocessing technique is to apply one-hot encoding or entity embedding, where each value (entity) of a categorical feature is represented by a vector of Boolean values [20].”
Hall, J. M., Xu, J., Walsh, M. G., Cho, H.-D., Harrell, G., Keshwani, S. A., Smith, S. M., & Staras, S. A. S. (2025) · PLOS One · doi:10.1371/journal.pone.0326776 · Cited by 4
Metadata
- Authors
- Rodríguez, P., Bautista, M. A., Gonzàlez, J., & Escalera, S.
- Venue
- Image and Vision Computing
- Publisher
- Elsevier BV
- Published
- 2018-07
- Volume
- 75
- Pages
- 21-31
- ISSN
- 0262-8856 (ISSN-L)
- OpenAlex
- W2799900537
- Cited by
- 480
- References
- 61
- FWCI
- 9.84
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.