CoChat
Sign Up
PreprintCited by 25Music and Audio ProcessingSpeech Recognition and SynthesisMusic Technology and Sound Studies

The DOI matches its DataCite record.

Semi-Supervised Music Tagging Transformer

Open access
Won, Minz, Choi, Keunwoo, Serra, Xavier·2021·arXiv
DOI 10.48550/arxiv.2111.13457arXiv 2111.13457
Open this paper in CoChat — free accountOpen PDF

Abstract

We present Music Tagging Transformer that is trained with a semi-supervised approach. The proposed model captures local acoustic characteristics in shallow convolutional layers, then temporally summarizes the sequence of the extracted features using stacked self-attention layers. Through a careful model assessment, we first show that the proposed architecture outperforms the previous state-of-the-art music tagging models that are based on convolutional neural networks under a supervised scheme. The Music Tagging Transformer is further improved by noisy student training, a semi-supervised approach that leverages both labeled and unlabeled data combined with data augmentation. To our best knowledge, this is the first attempt to utilize the entire audio of the million song dataset.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2111.13457

Cites

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Prevailing methodologies for contemporary musical descriptions typically lean on discriminative learning.

    “In the realm of Music Information Retrieval (MIR), prevailing methodologies for contemporary musical descriptions typically lean on discriminative learning.”

    From MusiLingo: Bridging Music and Text with Pre-trained Language Models for Music Captioning and Query Response (2024) · cites this paper as Won et al., 2021

    The Introduction (p. 1) describes music tagging as a classification task and says most previous tagging research used labeled data, consistent with a reliance on discriminative approaches. It does not establish that MIR methodologies for musical descriptions generally rely on discriminative learning; its discussion is limited to music tagging. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

No references could be resolved yet.

Research works printed but not linked: 38

  1. [1][1] A. Krizhevsky, I. Sutskever, and G. E. Hinton, 'Imagenet classification with deep convolutional neural networks,' Advances in neural information processing systems (NIPS) , 2012. -
  2. [2][2] K. Choi, G. Fazekas, and M. Sandler, 'Automatic tagging using deep convolutional neural networks,' in In Proc. of the 17th International Society for Music Information Retrieval Conference (ISMIR) , New York, USA, 2016. -
  3. [3][3] J. Pons, O. Nieto, M. Prockup, E. Schmidt, A. Ehmann, and X. Serra, 'End-to-end learning for music audio tagging at scale,' in In Proc. of the 19th International Society for Music Information Retrieval Conference (ISMIR) , Paris, France, 2018. -
  4. [4][4] J. Lee, J. Park, K. L. Kim, and J. Nam, 'Sample-level deep convolutional neural networks for music autotagging using raw waveforms,' In Proc. of the 14th Sound and music computing (SMC) , 2017. -
  5. [5][5] M. Won, S. Chun, , O. Nieto, and X. Serra, 'Datadriven harmonic filters for audio representation learning,' In Proc. of International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2020. -
  6. [6][6] M. Won, A. Ferraro, D. Bogdanov, and X. Serra, 'Evaluation of cnn-based automatic music tagging models,' In Proc. of Sound and Music Computing (SMC) , 2020. -
  7. [7][7] W. Brendel and M. Bethge, 'Approximating cnns with bag-of-local-features models works surprisingly well on imagenet,' International Conference on Learning Representations (ICLR) , 2019. -
  8. [8][8] T. Bertin-Mahieux, D. P. Ellis, B. Whitman, and P. Lamere, 'The million song dataset,' In Proc. of the 12th International Conference on Music Information Retrieval (ISMIR) , 2011. -
  9. [9][9] A. v. d. Oord, Y. Li, and O. Vinyals, 'Representation learning with contrastive predictive coding,' arXiv preprint arXiv:1807.03748 , 2018. - · arXiv
  10. [10][10] K. He, H. Fan, Y. Wu, S. Xie, and R. Girshick, 'Momentum contrast for unsupervised visual representation learning,' in Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 2020. -
  11. [11][11] T. Chen, S. Kornblith, M. Norouzi, and G. Hinton, 'A simple framework for contrastive learning of visual representations,' in International Conference on Machine Learning (ICML) , 2020. -
  12. [12][12] J.-B. Grill, F. Strub, F. Altché, C. Tallec, P. H. Richemond, E. Buchatskaya, C. Doersch, B. A. Pires, Z. D. Guo, M. G. Azar et al. , 'Bootstrap your own latent: A new approach to self-supervised learning,' arXiv preprint arXiv:2006.07733 , 2020. - · arXiv
  13. [13][13] J. Spijkervet and J. A. Burgoyne, 'Contrastive learning of musical representations,' In Proc. of International Society for Music Information Retrieval Conference (ISMIR) , 2021. -
  14. [14][14] O. Chapelle, B. Scholkopf, and A. Zien, 'Semisupervised learning,' IEEE Transactions on Neural Networks , vol. 20, no. 3, pp. 542-542, 2009. -
  15. [15][15] Q. Xie, M.-T. Luong, E. Hovy, and Q. V. Le, 'Selftraining with noisy student improves imagenet classification,' in Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 2020. -
  16. [16][16] T. Chen, S. Kornblith, K. Swersky, M. Norouzi, and G. Hinton, 'Big self-supervised models are strong semi-supervised learners,' in Conference on Neural Information Processing Systems (NeurIPS) , 2020. -
  17. [17][17] T. G. Dietterich, R. H. Lathrop, and T. Lozano-Pérez, 'Solving the multiple instance problem with axisparallel rectangles,' Artificial intelligence , vol. 89, no. 1-2, pp. 31-71, 1997. -
  18. [18][18] B. McFee, J. Salamon, and J. P. Bello, 'Adaptive pooling operators for weakly labeled sound event detection,' IEEE/ACM Transactions on Audio, Speech, and Language Processing , vol. 26, no. 11, pp. 2180-2193, 2018. -
  19. [19][19] K. Choi, G. Fazekas, M. Sandler, and K. Cho, 'Convolutional recurrent neural networks for music classification,' in Proc. of International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2017, pp. 2392-2396. -
  20. [20][20] M. Won, S. Chun, and X. Serra, 'Toward interpretable music tagging with self-attention,' arXiv preprint arXiv:1906.04972 , 2019. - · arXiv
  21. [21][21] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, 'BERT: Pre-training of deep bidirectional transformers for language understanding,' in Proc. of Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) , 2019. -
  22. [22][22] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, 'Attention is all you need,' Conference on Neural Information Processing Systems (NIPS) , 2017. -
  23. [23][23] C.-Z. A. Huang, A. Vaswani, J. Uszkoreit, N. Shazeer, I. Simon, C. Hawthorne, A. M. Dai, M. D. Hoffman, M. Dinculescu, and D. Eck, 'Music transformer,' International Conference on Learning Representations (ICLR) , 2019. -
  24. [24][24] A. Arnab, M. Dehghani, G. Heigold, C. Sun, M. Luˇ ci´ c, and C. Schmid, 'Vivit: A video vision transformer,' arXiv preprint arXiv:2103.15691 , 2021. - · arXiv
  25. [25][25] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly et al. , 'An image is worth 16x16 words: Transformers for image recognition at scale,' International Conference on Learning Representations (ICLR) , 2021. -
  26. [26][26] J. Li, M. L. Seltzer, X. Wang, R. Zhao, and Y. Gong, 'Large-scale domain adaptation via teacher-student learning,' arXiv preprint arXiv:1708.05466 , 2017. - · arXiv
  27. [27][27] S. Kum, J.-H. Lin, L. Su, and J. Nam, 'Semisupervised learning using teacher-student models for vocal melody extraction,' arXiv preprint arXiv:2008.06358 , 2020. - · arXiv
  28. [28][28] D.-H. Lee et al. , 'Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks,' in Workshop on challenges in representation learning, ICML , 2013. -
  29. [29][29] Y. Kim and A. M. Rush, 'Sequence-level knowledge distillation,' Proc. of the 2016 Conference on Empirical Methods in Natural Language Processing (EMNLP) , 2016. -
  30. [30][30] K. He, X. Zhang, S. Ren, and J. Sun, 'Deep residual learning for image recognition,' in Proc. of the IEEE conference on computer vision and pattern recognition (CVPR) , 2016. -
  31. [31][31] S. Ioffe and C. Szegedy, 'Batch normalization: Accelerating deep network training by reducing internal covariate shift,' in International conference on machine learning (ICML) , 2015. -
  32. [32][32] G. Hinton, O. Vinyals, and J. Dean, 'Distilling the knowledge in a neural network,' Advances in neural information processing systems (NIPS), Deep learning workshop , 2015. -
  33. [33][33] J. Spijkervet, 'Spijkervet/torchaudio-augmentations,' 2021. [Online]. Available: https://doi.org/10.5281/ zenodo.5042440 -
  34. [34][34] K. Choi, G. Fazekas, K. Cho, and M. Sandler, 'The effects of noisy labels on deep convolutional neural networks for music tagging,' IEEE Transactions on Emerging Topics in Computational Intelligence , 2018. -
  35. [35][35] M. Won, S. Oramas, O. Nieto, F. Gouyon, and X. Serra, 'Multimodal metric learning for tag-based music retrieval,' In Proc. of International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2021. -
  36. [36][36] D. P. Kingma and J. Ba, 'Adam: A method for stochastic optimization,' International Conference on Learning Representations (ICLR) , 2015. -
  37. [37][37] J. Davis and M. Goadrich, 'The relationship between precision-recall and roc curves,' in Proc. of international conference on Machine learning (ICML) , 2006. -
  38. [38][38] T. Kim, J. Lee, and J. Nam, 'Sample-level cnn architectures for music auto-tagging using raw waveforms,' in Proc. of International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2018.

Cited by

Sorted by citations

Metadata

Authors
Won, M., Choi, K., & Serra, X.
Affiliations
  • Universitat Pompeu Fabra (ES)
  • ByteDance (China) (CN)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2021
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2111.13457
arXiv
2111.13457
OpenAlex
W4286850225
Cited by
25
References
0
Citations by year
2025: 5 · 2024: 8 · 2023: 8 · 2022: 4
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Signal Processing › Music and Audio Processing
  • Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
  • Social Sciences › Arts and Humanities › Music › Music Technology and Sound Studies
Keywords
music tagging, semi-supervised learning, self-attention, audio feature extraction, convolutional neural networks, Million Song Dataset, audio classification

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review