CoChat

Taxonomy of machine learning paradigms: A data‐centric perspective

Emmert‐Streib, Frank, Dehmer, Matthias · 2022 · WIREs Data Mining and Knowledge Discovery
Cited by 56

The DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.

Abstract

Machine learning is a field composed of various pillars. Traditionally, supervised learning (SL), unsupervised learning (UL), and reinforcement learning (RL) are the dominating learning paradigms that inspired the field since the 1950s. Based on these, thousands of different methods have been developed during the last seven decades used in nearly all application domains. However, recently, other learning paradigms are gaining momentum which complement and extend the above learning paradigms significantly. These are multi‐label learning (MLL), semi‐supervised learning (SSL), one‐class classification (OCC), positive‐unlabeled learning (PUL), transfer learning (TL), multi‐task learning (MTL), and one‐shot learning (OSL). The purpose of this article is a systematic discussion of these modern learning paradigms and their connection to the traditional ones. We discuss each of the learning paradigms formally by defining key constituents and paying particular attention to the data requirements for allowing an easy connection to applications. That means, we assume a data‐driven perspective. This perspective will also allow a systematic identification of relations between the individual learning paradigms in the form of a learning‐paradigm graph (LP‐graph). Overall, the LP‐graph establishes a taxonomy among 10 different learning paradigms. This article is categorized under: Technologies > Machine Learning Application Areas > Science and Technology Fundamental Concepts of Data and Knowledge > Key Design Issues in Data Mining

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1002/widm.1470

Cites

19 of 19 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 19 references

Research works printed but not linked: 100

  1. Alashwal H., 2006, One‐class support vector machines for protein‐protein interactions prediction, International Journal of Biological and Medical Sciences, 1
  2. Bart E. &Ullman S.(2005).Cross‐generalization: Learning novel classes from a single example by feature replacement. InProceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) June 20 2005 to June 26 2005. San Diego CA(Vol.1 pp.672–679).
  3. Bartkowiak A. M., 2011, Anomaly, novelty, one‐class classification: A comprehensive introduction, International Journal of Computer Information Systems and Industrial Management Applications, 3, 61
  4. 10.1016/j.ins.2021.11.061 · DOI
  5. Bi J. Xiong T. Yu S. Dundar M. &Rao R. B.(2008).An improved multi‐task learning approach with applications in medical diagnosis. InJoint European Conference on Machine Learning and Knowledge Discovery in Databases(pp.117–132). · DOI
  6. 10.1007/978-0-387-45528-0 · DOI · Paper page
  7. Capra F., 1996, The web of life: A new scientific understanding of living systems
  8. Carbonell J. G., 1983, Machine Learning. Symbolic Computation, 3
  9. 10.7551/mitpress/9780262033589.001.0001 · DOI · Paper page
  10. Clare A. &King R. D.(2001).Knowledge discovery in multi‐label phenotype data. InEuropean Conference on Principles of Data Mining and Knowledge Discovery(pp.42–53). · DOI · Paper page
  11. 10.1109/JSTSP.2016.2535972 · DOI
  12. Dayan P., 2001, Theoretical neuroscience: Computational and mathematical modelling of neural systems
  13. Do C. B., 2005, Transfer learning for text classification, Advances in Neural Information Processing Systems, 18, 299
  14. Elisseeff A., 2001, A kernel method for multi‐labelled classification, Advances in Neural Information Processing Systems, 1‐7
  15. Emmert‐Streib F., 2003, Aktive computation in offenen systemen. Lerndynamiken in biologischen systemen: Vom netzwerk zum organismus (Unpublished doctoral dissertation)
  16. 10.3390/make1030054 · DOI
  17. 10.1109/TPAMI.2006.79 · DOI · Paper page
  18. 10.1016/j.eswa.2019.06.070 · DOI
  19. 10.1007/s10994-008-5064-8 · DOI · Paper page
  20. Gammerman A. Vovk V. &Vapnik V.(2013).Learning by transduction.arXiv preprint arXiv:1301.7375. · arXiv
  21. Ghamrawi N. &McCallum A.(2005).Collective multi‐label classification. InProceedings of the 14th ACM International Conference on Information and Knowledge Management(pp.195–200). · DOI · Paper page
  22. 10.1002/widm.1139 · DOI · Paper page
  23. Guillaumin M. Verbeek J. &Schmid C.(2010).Multimodal semi‐supervised learning for image classification. In2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition(pp.902–909). · DOI · Paper page
  24. 10.1109/TPAMI.2017.2738004 · DOI · Paper page
  25. 10.1007/978-0-387-84858-7 · DOI · Paper page
  26. 10.1613/jair.678 · DOI · Paper page
  27. Hoaglin D., 1983, Understanding robust and exploratory data analysis
  28. Hou M. Chaib‐Draa B. Li C. &Zhao Q.(2017).Generative adversarial positive‐unlabelled learning.arXiv preprint arXiv:1711.08054. · DOI
  29. Hu D. &Yang Q.(2011).Transfer learning for activity recognition via sensor mapping. InProceedings of the Twenty‐Second International Joint Conference on Artificial Intelligence Barcelona Catalonia Spain(p.1962).
  30. Igl M. Zintgraf L. Le T. A. Wood F. &Whiteson S.(2018).Deep variational reinforcement learning for POMDPs. InInternational Conference on Machine Learning(pp.2117–2126).
  31. Jaakkola T., 1995, Advances in Neural Information Processing Systems, 345
  32. Japkowicz N., 1999, Concept‐learning in the absence of counter‐examples: An autoassociation based approach to classification (Unpublished doctoral dissertation)
  33. Jaskie K. &Spanias A.(2019).Positive and unlabeled learning algorithms and applications: A survey. In2019 10th International Conference on Information Intelligence Systems and Applications (IISA)(pp.1–8). · DOI
  34. Kadam S. &Vaidya V.(2018).Review and analysis of zero one and few shot learning approaches. InInternational Conference on Intelligent Systems Design and Applications(pp.100–112). · DOI
  35. 10.1613/jair.301 · DOI · Paper page
  36. 10.3390/socsci8090255 · DOI · Paper page
  37. 10.1007/s00521-017-2872-4 · DOI
  38. 10.1017/S026988891300043X · DOI · Paper page
  39. Kiryo R. Niu G. Plessis M. C.d. &Sugiyama M.(2017).Positive‐unlabeled learning with non‐negative risk estimator.arXiv preprint arXiv:1703.00593. · arXiv
  40. Koch G. Zemel R. &Salakhutdinov R.(2015).Siamese neural networks for one‐shot image recognition. InICML Deep Learning Workshop(Vol. 2).
  41. Koppel M. &Schler J.(2004).Authorship verification as a one‐class classification problem. InProceedings of the Twenty‐First International Conference on Machine Learning(p. 62). · DOI · Paper page
  42. Kuhn T. S., 1970, The structure of scientific revolutions
  43. Lake B. Salakhutdinov R. Gross J. &Tenenbaum J.(2011).One shot learning of simple visual concepts. InProceedings of the Annual Meeting of the Cognitive Science Society(Vol. 33).
  44. Li X. Sun Z. Xue J.‐H. &Ma Z.(2020).A concise review of recent few‐shot meta‐learning methods.arXiv preprint arXiv:2005.10953. · DOI
  45. Li X.‐L. Yu P. S. Liu B. &Ng S.‐K.(2009).Positive unlabeled learning for data stream classification. InProceedings of the 2009 SIAM International Conference on Data Mining(pp.259–270). · DOI · Paper page
  46. Li Y.‐X., 2011, Drosophila gene expression pattern annotation through multi‐instance multi‐label learning, IEEE/ACM Transactions on Computational Biology and Bioinformatics, 9, 98
  47. 10.1038/nrg3920 · DOI · Paper page
  48. Liu P. Qiu X. &Huang X.(2016).Recurrent neural network for text classification with multi‐task learning.arXiv preprint arXiv:1605.05101. · arXiv
  49. 10.1109/JPROC.2012.2197809 · DOI · Paper page
  50. Manevitz L. M. &Yousef M.(2000).Document classification on neural networks using only positive examples. InProceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval(pp.304–306). · doi:10.1145/345508.345608
  51. Mignone P., 2020, Exploiting transfer learning for the reconstruction of the human gene regulatory network, Bioinformatics, 36, 1553, 10.1093/bioinformatics/btz781 · DOI
  52. 10.1016/j.patrec.2013.06.010 · DOI · Paper page
  53. Moya M.(1993).One‐class classifier networks for target recognition applications. InProceedings of the World Congress on Neural Networks (pp.797–801).
  54. 10.1016/0893-6080(95)00120-4 · DOI · Paper page
  55. Niu G. duPlessis M. C. Sakai T. Ma Y. &Sugiyama M.(2016).Theoretical comparisons of positive‐unlabeled learning against positive‐negative learning. InAdvances in Neural Information Processing Systems(pp. 1199–1207).
  56. 10.1109/TKDE.2009.191 · DOI · Paper page
  57. 10.1016/S0749-2081(00)80008-X · DOI · Paper page
  58. Perera P. Oza P. &Patel V. M.(2021).One‐class classification: A survey.arXiv preprint arXiv:2101.03064. · arXiv
  59. Pulkkinen T. Roos T. &Myllymäki P.(2011).Semi‐supervised learning for wlan positioning. InInternational Conference on Artificial Neural Networks(pp.355–362). · DOI
  60. 10.1007/s10994-011-5256-5 · DOI · Paper page
  61. 10.1016/j.ibmed.2020.100005 · DOI
  62. 10.1016/j.chemolab.2016.10.002 · DOI
  63. Ruder S.(2017).An overview of multi‐task learning in deep neural networks.arXiv preprint arXiv:1706.05098. · arXiv
  64. Ruff L. Vandermeulen R. Goernitz N. Deecke L. Siddiqui S. A. Binder A. Müller E. &Kloft M.(2018).Deep one‐class classification. InInternational Conference on Machine Learning(pp.4393–4402).
  65. 10.1023/A:1007649029923 · DOI · Paper page
  66. Schmidhuber J., 1987, Evolutionary principles in self‐referential learning, or on learning how to learn: The meta‐meta‐… hook (Diploma thesis)
  67. Schölkopf B., 1999, Support vector method for novelty detection, Advances in Neural Information Processing Systems, 12, 582
  68. 10.1093/bioinformatics/btr502 · DOI · Paper page
  69. 10.1002/2211-5463.12652 · DOI · Paper page
  70. 10.1002/minf.201800108 · DOI · Paper page
  71. Sperry R. W., 1952, Neurology and the mind‐brain problem, American Scientist, 40, 291
  72. Sutton R., 1998, Reinforcement learning
  73. Tarassenko L. Hayton P. Cerneaz N. &Brady M.(1995).Novelty detection for the identification of masses in mammograms. InProceeding of the 4th International Conference on Artificial Neural Networks Vol. 1995 pp. 442–447. · DOI · Paper page
  74. Tax D. M. J., 2001, One‐class classification: Concept learning in the absence of counter‐examples (Doctoral dissertation)
  75. 10.1007/978-1-4615-5529-2 · DOI · Paper page
  76. 10.1007/s11042-018-6463-x · DOI · Paper page
  77. 10.4018/jdwm.2007070101 · DOI · Paper page
  78. 10.1007/978-0-387-09823-4_34 · DOI · Paper page
  79. 10.1109/TKDE.2010.164 · DOI · Paper page
  80. 10.1007/978-3-642-27645-3_1 · DOI · Paper page
  81. 10.1007/978-3-030-05318-5_2 · DOI · Paper page
  82. 10.1007/978-1-4757-2440-0 · DOI · Paper page
  83. Vinyals O. Blundell C. Lillicrap T. Kavukcuoglu K. &Wierstra D.(2016).Matching networks for one shot learning.arXiv preprint arXiv:1606.04080. · arXiv
  84. 10.1016/j.jmsy.2018.01.003 · DOI · Paper page
  85. 10.1007/BF00992698 · DOI · Paper page
  86. 10.1186/s40537-016-0043-6 · DOI · Paper page
  87. Wiering M. A. vanHasselt H. Pietersma A.‐D. &Schomaker L.(2011).Reinforcement learning algorithms for solving classification problems. In2011 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL)(pp.91–96). · DOI
  88. 10.1016/j.patcog.2015.01.022 · DOI
  89. Wu B., 2020, Landslide susceptibility modeling using bagging‐based positive‐unlabeled learning, IEEE Geoscience and Remote Sensing Letters, 18
  90. 10.1080/21693277.2016.1192517 · DOI · Paper page
  91. 10.1186/1752-0509-4-S2-S6 · DOI · Paper page
  92. 10.1007/s11042-018-5772-4 · DOI
  93. 10.1371/journal.pone.0097079 · DOI · Paper page
  94. Yu S. &Li C.(2007).Pe‐puc: A graph based pu‐learning approach for text classification. InInternational Workshop on Machine Learning and Data Mining in Pattern Recognition(pp.574–584). · DOI
  95. Zhang B. &Zuo W.(2008).Learning from positive and unlabeled examples: A survey. InProceedings of the 2008 International Symposiums on Information Processing(pp.650–654). · DOI · Paper page
  96. 10.1016/j.patcog.2006.12.019 · DOI · Paper page
  97. 10.1109/TKDE.2013.39 · DOI · Paper page
  98. 10.1093/nsr/nwx105 · DOI · Paper page
  99. 10.1007/978-3-031-01548-9 · DOI · Paper page
  100. Zhu Y. Chen Y. Lu Z. Pan S. Xue G.‐R. Yu Y. &Yang Q.(2011).Heterogeneous transfer learning for image classification. InProceedings of the AAAI Conference on Artificial Intelligence(Vol. 25). · DOI · Paper page

Cited by

See all 56

Metadata

Authors
Emmert‐Streib, F., & Dehmer, M.
Affiliations
  • Predictive Society and Data Analytics Lab, Faculty of Information Technology and Communication Sciences Tampere University Tampere Finland
  • Swiss Distance University of Applied Sciences Brig Switzerland
Venue
WIREs Data Mining and Knowledge Discovery
Publisher
Wiley
Published
2022-06-03
Volume
12
Issue
5
ISSN
  • 1942-4787
  • 1942-4795
Cited by
56
References
19
Language
en

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Taxonomy of machine learning paradigms: A data‐centric perspective. https://app.cochat.ai/papers/10.1002/widm.1470
The paper
Emmert‐Streib, F., & Dehmer, M. (2022). Taxonomy of machine learning paradigms: A data‐centric perspective. WIREs Data Mining and Knowledge Discovery. https://doi.org/10.1002/widm.1470

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.