FastXML: a fast, accurate and stable tree-classifier for extreme multi-label learning
The DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
Abstract
The objective in extreme multi-label classification is to learn a classifier that can automatically tag a data point with the most relevant subset of labels from a large label set. Extreme multi-label classification is an important research problem since not only does it enable the tackling of applications with many labels but it also allows the reformulation of ranking problems with certain advantages over existing formulations. Our objective, in this paper, is to develop an extreme multi-label classifier that is faster to train and more accurate at prediction than the state-of-the-art Multi-label Random Forest (MLRF) algorithm [2] and the Label Partitioning for Sub-linear Ranking (LPSR) algorithm [35]. MLRF and LPSR learn a hierarchy to deal with the large number of labels but optimize task independent measures, such as the Gini index or clustering error, in order to learn the hierarchy. Our proposed FastXML algorithm achieves significantly higher accuracies by directly optimizing an nDCG based ranking loss function. We also develop an alternating minimization algorithm for efficiently optimizing the proposed formulation. Experiments reveal that FastXML can be trained on problems with more than a million labels on a standard desktop in eight hours using a single core and in an hour using multiple cores.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Bertsekas, D. P. (1997) · Journal of the Operational Research Society · doi:10.1038/sj.jors.2600425 · Cited by 11,726
Geurts, P., Ernst, D., & Wehenkel, L. (2006) · Machine Learning · doi:10.1007/s10994-006-6226-1 · Cited by 9,300
Weik, M. H. (2000) · Computer Science and Communications Dictionary · doi:10.1007/1-4020-0613-6_12436 · Cited by 2,570
Wsabie: Scaling up to large vocabulary image annotation
Weston, J., Samy Bengio, A., & Usunier, N. (2011) · Cited by 677
The challenge problem for automated detection of 101 semantic concepts in multimedia
Snoek, C. G. M., Worring, M., van Gemert, J. C., Geusebroek, J.-M., & Smeulders, A. W. M. (2006) · Proceedings of the 14th ACM international conference on Multimedia · doi:10.1145/1180639.1180727 · Cited by 620
An Interior-Point Method for Large-Scale '1-Regularized Logistic Regression
Koh, K., Kim, S.-J., & Boyd, S. (2007) · Cited by 579
Scalable training of L1-regularized log-linear models
Andrew, G., & Gao, J. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273501 · Cited by 506
A Theoretical Analysis of NDCG Type Ranking Measures
Wang, Y., Wang, L., Li, Y., He, D., Liu, T.-Y., & Chen, W. (2013) · arXiv · doi:10.48550/arxiv.1304.6480 · Cited by 351
Large-scale Multi-label Learning with Missing LabelsPreprint
Yu, H., Jain, P., Kar, P., & Dhillon, I. S. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1307.5101 · Cited by 335
LIBLINEAR: A Library for Large Linear Classification
FanRong-En, ChangKai-Wei, HsiehCho-Jui, WangXiang-Rui, & LinChih-Jen (2008) · Journal of Machine Learning Research · doi:10.5555/1390681.1442794 · Cited by 328
Label Embedding Trees for Large Multi-Class Tasks
Bengio, S., Weston, J., & Grangier, D. (2010) · Cited by 317
Kernel-Based Learning of Hierarchical Multilabel Classification Models
Rousu, J., Saunders, C. J., Szedmák, S., & Shawe-Taylor, J. S. (2006) · Journal of Machine Learning Research · Cited by 294
Multilabel Text Classification for Automated Tag Suggestion
Katakis, I., Vlahavas, I. P., & Tsoumakas, G. (2008) · Cited by 277
Multilabel Classification with Principal Label Space Transformation
Tai, F., & Lin, H.-T. (2012) · Neural Computation · doi:10.1162/neco_a_00320 · Cited by 274
Multi-Label Prediction via Compressed Sensing
Hsu, D., Kakade, S. M., Langford, J. C., & Zhang, T. (2009) · arXiv (Cornell University) · doi:10.48550/arxiv.0902.1284 · Cited by 245
Multi-label learning with millions of labels: recommending advertiser bid phrases for web pages
Agrawal, R., Gupta, A., Prabhu, Y., & Varma, M. (2013) · Proceedings of the 22nd international conference on World Wide Web · doi:10.1145/2488388.2488391 · Cited by 239
Self-citationLearning to Rank by Optimizing NDCG Measure
Valizadegan, H., Jin, R., Zhang, R., & Mao, J. (2009) · neural information processing systems · Cited by 232
An Interior-Point Method for Large-Scale l1-Regularized Logistic Regression
KohKwangmoo, KimSeung-Jean, & BoydStephen (2007) · Journal of Machine Learning Research · doi:10.5555/1314498.1314550 · Cited by 231
Feature-aware Label Space Dimension Reduction for Multi-label Classification
Chen, Y.-N., & Lin, H.-T. (2012) · neural information processing systems · Cited by 214
Extracting shared subspace for multi-label classification
Ji, S., Tang, L., Yu, S., & Ye, J. (2008) · Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1401890.1401939 · Cited by 202
Efficient Multi-label Classification with Many Labels
Bi, W., & Kwok, J. T.-Y. (2013) · Rare & Special e-Zone (The Hong Kong University of Science and Technology) · Cited by 201
Fast and Balanced: Efficient Label Tree Learning for Large Scale Object Recognition
Deng, J., Satheesh, S., Berg, A. C., & Fei-Fei, L. (2011) · Cited by 144
Discriminative learning of relaxed hierarchy for large-scale visual recognition
Gao, T., & Koller, D. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126481 · Cited by 143
Incremental Algorithms for Hierarchical Classification
Cesa‐Bianchi, N., Gentile, C., Tironi, A. L., & Zaniboni, L. (2004) · neural information processing systems · Cited by 140
Multi-label output codes using canonical correlation analysis
Zhang, Y. (2025) · International Conference on Artificial Intelligence and Statistics · doi:10.57702/qayydmvx · Cited by 128
An improved GLMNET for L1-regularized logistic regression
Yuan, G.-X., Ho, C.-H., & Lin, C. (2012) · doi:10.5555/2188385.2343708 · Cited by 126
Multilabel Classification using Bayesian Compressed Sensing
Kapoor, A., Viswanathan, R., & Jain, P. (2012) · neural information processing systems · Cited by 124
BoltzRank: learning to maximize expected ranking gain
Volkovs, M., & Zemel, R. S. (2009) · International Conference on Machine Learning · doi:10.1145/1553374.1553513 · Cited by 123
Label Partitioning For Sublinear Ranking
Weston, J., Makadia, A., & Yee, H. Y. (2013) · International Conference on Machine Learning · Cited by 115
An improved GLMNET for l1-regularized logistic regression
Yuan, G.-X., Ho, C.-H., & Lin, C. (2011) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2020408.2020421 · Cited by 97
On NDCG Consistency of Listwise Ranking Methods
Ravikumar, P., Tewari, A., & Yang, E. (2011) · International Conference on Artificial Intelligence and Statistics · Cited by 94
Local Deep Kernel Learning for Efficient Non-linear SVM Prediction
Jose, C., Goyal, P., Aggrwal, P., & Varma, M. (2013) · International Conference on Machine Learning · Cited by 80
Self-citationRobust Bloom Filters for Large MultiLabel Classification Tasks
Cissé, M., Usunier, N., Artières, T., & Gallinari, P. (2013) · HAL (Le Centre pour la Communication Scientifique Directe) · Cited by 64
Efficient max-margin multi-label classification with applications to zero-shot learning
Hariharan, B., Vishwanathan, S. V. N., & Varma, M. (2012) · Machine Learning · doi:10.1007/s10994-012-5291-x · Cited by 54
Self-citationMulti-label Classification with Error-correcting Codes
Ferng, C.-S., & Lin, H.-T. (2011) · Asian Conference on Machine Learning · Cited by 39
Cited by
Deep Metric Learning via Lifted Structured Feature Embedding
Song, H. O., Xiang, Y., Jegelka, S., & Savarese, S. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.434 · Cited by 1,630
Deep Learning for Extreme Multi-label Text Classification
Liu, J., Chang, W.-C., Wu, Y., & Yang, Y. (2017) · Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3077136.3080834 · Cited by 676
The Emerging Trends of Multi-Label Learning
Liu, W., Wang, H., Shen, X., & Tsang, I. W. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3119334 · Cited by 400
Deep Metric Learning via Facility Location
Song, H. O., Jegelka, S., Rathod, V., & Murphy, K. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.237 · Cited by 287
Learning Tree-based Deep Model for Recommender Systems
Zhu, H., Li, X., Zhang, P., Li, G., He, J., Li, H., & Gai, K. (2018) · Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3219819.3219826 · Cited by 285
Metadata
- Authors
- Prabhu, Y., & Varma, M.
- Venue
- Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining
- Publisher
- ACM
- Published
- 2014-08-24
- Pages
- 263-272
- OpenAlex
- W2068074736
- Cited by
- 307
- References
- 35
- FWCI
- 10.28
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.