Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Sign Up
Cited by 3,321

The DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.

A Review on Multi-Label Learning Algorithms

Zhang, Min-Ling, Zhou, Zhi-Hua·2014·IEEE Transactions on Knowledge and Data Engineering
DOI 10.1109/tkde.2013.39
Open this paper in CoChat — free account

Abstract

Multi-label learning studies the problem where each example is represented by a single instance while associated with a set of labels simultaneously. During the past decade, significant amount of progresses have been made toward this emerging machine learning paradigm. This paper aims to provide a timely review on this area with emphasis on state-of-the-art multi-label learning algorithms. Firstly, fundamentals on multi-label learning including formal definition and evaluation metrics are given. Secondly and primarily, eight representative multi-label learning algorithms are scrutinized under common notations with relevant analyses and discussions. Thirdly, several related learning settings are briefly summarized. As a conclusion, online resources and open research problems on multi-label learning are outlined for reference purposes.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/tkde.2013.39

Cites

6 of 6 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 6 references

Research works printed but not linked: 108

  1. 10.1148/radiology.143.1.7063747 · DOI · Paper page
  2. 10.1145/1099554.1099591 · DOI · Paper page
  3. gao, 2011, On the consistency of multi-label learning, Proc 24th Annu Conf Learn Theory, 341
  4. 10.1007/978-0-387-47534-9_3 · DOI
  5. 10.1007/s10994-008-5064-8 · DOI · Paper page
  6. guo, 2011, Adaptive large margin training for multilabel classification, Proc 25th AAAI Conf Artif Intell, 374
  7. guo, 2011, Multi-label classification using conditional dependency networks, Proc 2nd IJCAI, 1300
  8. 10.1145/1835449.1835503 · doi:10.1145/1835449.1835503
  9. godbole, 2004, Discriminative methods for multi-labeled classification, Lecture Notes in Artificial Intelligence 3056, 22
  10. ferng, 2011, Multi-label classification with error-correcting codes, Proc Asian Conf Mach Learn, 281
  11. elisseeff, 2002, A kernel method for multi-labelled classification, Advances in Neural Information Processing Systems 14, 681
  12. dembczy?ski, 2010, Bayes optimal multilabel classification via probabilistic classifier chains, Proc 27th Int Conf Mach Learn, 279
  13. 10.1007/978-3-642-15880-3_24 · doi:10.1007/978-3-642-15880-3_24
  14. dembczy?ski, 2012, Consistent multilabel ranking through univariate loss minization, Proc 29th Int Conf Mach Learn, 1319
  15. 10.1007/s10994-012-5285-8 · DOI
  16. wang, 2010, Multi-label classification: Inconsistency and class balanced \(k\) -nearest neighbor, Proc 24th AAAI Conf Artif Intell, 1264
  17. elisseeff, 2001, Kernel methods for multi-labelled classification and categorical regression problems, Biowulf Technologies
  18. 10.1007/s10994-008-5077-3 · DOI · Paper page
  19. 10.1016/S0004-3702(96)00034-3 · DOI · Paper page
  20. katakis, 2008, Multilabel text classification for automated tag suggestion, Proceedings of ECML/PKDD Discovery Challenge, 75
  21. kazawa, 2005, Maximal margin labeling for multi-topic text categorization, In Advances in Neural Information Processing Systems 17, 649
  22. 10.1016/j.neucom.2009.11.024 · DOI
  23. loza menc?a, 2008, Pairwise learning of multilabel classifications with perceptrons, Proc IJCNN, 2899
  24. loza mencía, 2008, Efficient pairwise multilabel classification for large-scale problems in the legal domain, Lecture Notes in Artificial Intelligence 5212, 50
  25. loza mencía, 2010, Multilabel classification in parallel tasks, Proc Work Notes 2nd Int Workshop Learn MLD, 20
  26. kot?owski, 2011, Bipartite ranking through minimization of univariate loss, Proc 28th Int Conf Mach Learn, 1113
  27. 10.1007/s10115-011-0407-3 · DOI
  28. 10.4108/icst.collaboratecom.2011.247086 · doi:10.4108/icst.collaboratecom.2011.247086
  29. kocev, 2007, Ensembles of multi-objective decision trees, Proc 18th ECML, 624
  30. hariharan, 2010, Large scale max-margin multi-label classification with priors, Proc 27th Int Conf Mach Learn, 423
  31. 10.1016/j.artmed.2012.12.005 · DOI
  32. 10.1016/j.ijar.2011.01.007 · DOI
  33. brinker, 2006, A unified model for multilabel classification and ranking, Proc 17th Eur Conf Artif Intell, 489
  34. 10.1016/j.patcog.2004.03.009 · DOI · Paper page
  35. bucak, 2010, Multi-label multiple kernel learning by stochastic approximation: Application to visual object recognition, Advances in neural information processing systems, 325
  36. kang, 2006, Correlated label propagation with application to multi-label learning, Proc IEEE Comput Soc Conf Comput Vision Pattern Recognit, 1719
  37. brinker, 2007, Case-based multilabel ranking, Proc 20th Int Joint Conf Artif Intell, 702
  38. 10.1007/BF01582063 · DOI · Paper page
  39. ji, 2009, Multi-label multiple kernel learning, in Advances in Neural Information Processing Systems 21, 777
  40. jiang, 2008, Calibrated rank-SVM for multi-label image categorization, Proc IJCNN, 1450
  41. 10.1145/1401890.1401939 · DOI
  42. 10.1145/2339530.2339615 · doi:10.1145/2339530.2339615
  43. 10.1145/2019618.2019624 · DOI
  44. 10.1109/ICTAI.2010.65 · doi:10.1109/ictai.2010.65
  45. huang, 2012, Multi-label learning by exploiting label correlations locally, Proc 26th AAAI Conf Artif Intell, 949
  46. 10.1007/s10994-011-5256-5 · DOI · Paper page
  47. read, 2009, Classifier chains for multi-label classification, Lecture Notes in Artificial Intelligence 5782, 254
  48. 10.1109/ICDM.2008.74 · doi:10.1109/icdm.2008.74
  49. 10.1007/s10994-012-5279-6 · DOI
  50. 10.1007/978-3-642-23808-6_15 · doi:10.1007/978-3-642-23808-6_15
  51. 10.1145/1458082.1458098 · doi:10.1145/1458082.1458098
  52. 10.1145/2009916.2010011 · DOI
  53. 10.1023/A:1007649029923 · DOI · Paper page
  54. spyromitros-xioufis, 2011, Dealing with concept drift and class imbalance in multi-label stream classification, Proc 2nd IJCAI, 1583
  55. spyromitros-xioufis, 2008, An empirical study of lazy multilabel classification algorithms, Proc Hellenic Conf Artificial Intelligence, 401
  56. madjarov, 2011, Efficient two stage voting architecture for pairwise multi-label classification, Lecture Notes in Computer Science 6464, 164
  57. 10.1016/j.patcog.2012.03.004 · DOI · Paper page
  58. 10.1016/j.patcog.2011.08.011 · DOI
  59. mccallum, 1999, Multi-label text classification with a mixture model trained by EM, Proc Work Notes AAAI Workshop Text Learn
  60. montañés, 2011, Aggregating independent and dependent models to learn multi-label classifiers, Lecture Notes in Artificial Intelligence 6912, 484
  61. petterson, 2010, Reverse multi-label learning, Advances in Neural Information Processing Systems 23, 1912
  62. petterson, 2011, Submodular multi-label learning, in Advances in Neural Information Processing Systems 24, 1512
  63. 10.1145/1291233.1291245 · DOI · Paper page
  64. qu, 2009, Mining multi-label concept-drifting data streams using dynamic classifier ensemble, Lecture Notes in Artificial Intelligence 5828, 308
  65. bi, 2011, Multi-label classification on tree- and DAG-structured hierarchies, Proc 28th Int Conf Mach Learn, 17
  66. quevedo, 2012, Multilabel classifiers with a probabilistic thresholding strategy, Pattern Recognit, 45, 876, 10.1016/j.patcog.2011.08.007 · doi:10.1016/j.patcog.2011.08.007
  67. berger, 1996, A maximum entropy approach to natural language processing, Comput Linguist, 22, 39
  68. zhang, 2007, Multi-label learning by instance differentiation, Proc 22nd AAAI Conf Artif Intell, 669
  69. tsoumakas, 2009, Tutorial on learning from multi-label data, ECML PKDD
  70. 10.1016/j.patcog.2006.12.019 · DOI · Paper page
  71. tsoumakas, 2007, Random k-labelsets: An ensemble method for multilabel classification, Lecture Notes in Artificial Intelligence 4701, 406
  72. 10.1109/TKDE.2006.162 · DOI · Paper page
  73. tsoumakas, 2011, MULAN: A Java library for multi-label learning, J Mach Learn Res, 12, 2411
  74. 10.1145/1835804.1835930 · DOI · Paper page
  75. 10.1109/TKDE.2010.164 · DOI · Paper page
  76. zaragoza, 2011, Bayesian chain classifiers for multidimensional classification, Proc 2nd IJCAI, 2192
  77. 10.1007/978-0-387-09823-4_34 · DOI · Paper page
  78. younes, 2011, A dependent multilabel classification method derived from the \(k\) -nearest neighbor rule, EURASIP J Adv Sig Process, 2011
  79. tsoumakas, 2008, Effective and efficient multilabel classification in domains with large number of labels, Proc Work Notes ECML PKDD Workshop MMD
  80. 10.1145/1281192.1281281 · DOI
  81. 10.1145/1386352.1386359 · doi:10.1145/1386352.1386359
  82. zhang, 2012, Maximum margin output coding, Proc 29th Int Conf Mach Learn, 1575
  83. zhou, 2012, Ensemble Methods Foundations and Algorithms, 10.1201/b12207 · DOI · Paper page
  84. 10.1137/1.9781611972801.68 · doi:10.1137/1.9781611972801.68
  85. van der gaag, 2006, Multi-dimensional Bayesian network classifiers, Proc 3rd Eur Workshop Probab Graph Models, 107
  86. veloso, 2007, Multi-label lazy associative classification, Lecture Notes in Artificial Intelligence 4702, 605
  87. 10.1007/s10994-012-5292-9 · DOI
  88. ueda, 2003, Parametric mixture models for multi-label text, Advances in Neural Information Processing Systems 15, 721
  89. cheng, 2010, Graded multilabel classification: The ordinal case, Proc 27th Int Conf Mach Learn, 223
  90. 10.1007/s10994-009-5127-5 · DOI
  91. cheng, 2009, A simple instance-based approach to multilabel classification using the Mallows model, Proc Work Notes 1st Int Workshop Learn MLD, 28
  92. chiang, 2012, A ranking-based KNN approach for multi-label classification, Proc Asian Conf Mach Learn, 81
  93. clare, 2001, Knowledge discovery in multi-label phenotype data, Lecture Notes in Computer Science 2168, 42, 10.1007/3-540-44794-6_4 · DOI · Paper page
  94. 10.1145/1526709.1526738 · DOI
  95. de carvalho, 2009, A tutorial on multi-label classification techniques, Studies in Computational Intelligence 205, 177, 10.1007/978-3-642-01536-6_8 · DOI
  96. tang, 2009, On multiple kernel learning with multiple labels, Proc 21st IJCAI, 1255
  97. de waal, 2007, Inference and learning in multi-dimensional Bayesian network classifiers, Lecture Notes in Artificial Intelligence 4724, 501
  98. 10.1109/ICDM.2004.10117 · DOI
  99. delcroix, 2007, Bayesian networks versus other probabilistic models for the multiple diagnosis of large devices, Int J Artif Intell Tools, 16, 417, 10.1142/S0218213007003345 · doi:10.1142/s0218213007003345
  100. tenenboim-chekina, 2010, Identification of label dependencies for multi-label classification, Proc Work Notes 2nd Int Workshop Learn MLD, 53
  101. 10.1145/1076034.1076082 · DOI
  102. 10.1016/j.artint.2011.10.002 · DOI · Paper page
  103. streich, 2008, Classification of multi-labeled data: A generative approach, Lecture Notes in Artificial Intelligence 5212, 390
  104. 10.4018/jdwm.2007070101 · DOI · Paper page
  105. trohidis, 2008, Multilabel classification of music into emotions, Proc Int'l Conf on Music Information Retrieval, 325
  106. 10.1145/1015330.1015341 · DOI · Paper page
  107. tsochantaridis, 2005, Large margin methods for structured and interdependent output variables, J Mach Learn Res, 6, 1453
  108. tsoumakas, 2009, Correlation-based pruning of stacked binary relevance models for multi-label learning, Proc Work Notes 1st Int Workshop Learn MLD, 101

Cited by

See all 3,321

Metadata

Authors
Zhang, M.-L., & Zhou, Z.-H.
Venue
IEEE Transactions on Knowledge and Data Engineering
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Published
2014-08
Volume
26
Issue
8
Pages
1819-1837
ISSN
1041-4347 (ISSN-L)
DOI
10.1109/tkde.2013.39
OpenAlex
W2114315281
Cited by
3,321
References
6
FWCI
126.63
Type
journal article
Language
en
Open access
closed
Licence
ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html (vor)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: A Review on Multi-Label Learning Algorithms. https://app.cochat.ai/papers/10.1109/tkde.2013.39
The paper
Zhang, M.-L., & Zhou, Z.-H. (2014). A Review on Multi-Label Learning Algorithms. IEEE Transactions on Knowledge and Data Engineering. https://doi.org/10.1109/tkde.2013.39

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.