CoChat
Sign Up
Cited by 42Music and Audio ProcessingMachine Learning and Data ClassificationAdvanced Neural Network Applications

The Effects of Noisy Labels on Deep Convolutional Neural Networks for Music Tagging

Open access
Choi, Keunwoo, Fazekas, Gyorgy, Cho, Kyunghyun, Sandler, Mark·2018·IEEE Transactions on Emerging Topics in Computational Intelligence
DOI 10.1109/tetci.2017.2771298
Open this paper in CoChat — free account

Abstract

Deep neural networks (DNNs) have been successfully applied to music classification including music tagging. However, there are several open questions regarding the training, evaluation, and analysis of DNNs. In this paper, we investigate specific aspects of neural networks, the effects of noisy labels, to deepen our understanding of their properties. We analyze and (re-)validate a large music tagging dataset to investigate the reliability of training and evaluation. Using a trained network, we compute label vector similarities, which are compared to groundtruth similarity. The results highlight several important aspects of music tagging and neural networks. We show that networks can be effective despite relatively large error rates in groundtruth datasets, while conjecturing that label noise can be the cause of varying tag-wise performance differences. Finally, the analysis of our trained network provides valuable insight into the relationships between music tags. These results highlight the benefit of using data-driven methods to address automatic music tagging.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/tetci.2017.2771298

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Tag-to-sentence approaches that rely on tagging datasets inherit the limitations of tagging datasets, such as high false-negative rates.

    “As relying on tagging datasets, however, the tag-to-sentence approaches would have the same limitation tagging datasets have.For example, high false-negative rates of tagging datasets [15].”

    From LP-MusicCaps: LLM-Based Pseudo Music Captioning (2023) · cites this paper as [15]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 36

  1. [1][1] K. Choi, G. Fazekas, and M. Sandler, 'Automatic tagging using deep convolutional neural networks,' in The 17th International Society of Music Information Retrieval Conference, New York, USA . International Society of Music Information Retrieval, 2016. -
  2. [2][2] P. Lamere, 'Social tagging and music information retrieval,' Journal of new music research , vol. 37, no. 2, pp. 101-114, 2008. -
  3. [3][3] D. Eck, P. Lamere, T. Bertin-Mahieux, and S. Green, 'Automatic generation of social tags for music recommendation,' in Advances in neural information processing systems , 2008, pp. 385-392. -
  4. [4][4] P. Saari, M. Barthet, G. Fazekas, T. Eerola, and M. Sandler, 'Semantic models of musical mood: Comparison between crowd-sourced and curated editorial tags,' in Multimedia and Expo Workshops (ICMEW), 2013 IEEE International Conference on . IEEE, 2013, pp. 1-6. -
  5. [5][5] K. Choi, G. Fazekas, M. Sandler, and K. Cho, 'Transfer learning for music classification and regression tasks,' in The 18th International Society of Music Information Retrieval (ISMIR) Conference 2017, Suzhou, China . International Society of Music Information Retrieval, 2017. -
  6. [6][6] J. Lee and J. Nam, 'Multi-level and multi-scale feature aggregation using pre-trained convolutional neural networks for music auto-tagging,' arXiv preprint arXiv:1703.01793 , 2017. - · arXiv
  7. [7][7] I. Cantador, I. Konstas, and J. M. Jose, 'Categorising social tags to improve folksonomy-based recommendations,' Web Semantics: Science, Services and Agents on the World Wide Web , vol. 9, no. 1, pp. 1 - 15, 2011. -
  8. [8][8] A. Chamberlain and A. Crabtree, 'Searching for music: understanding the discovery, acquisition, processing and organization of music in a domestic setting for design,' Personal and Ubiquitous Computing , vol. 20, no. 4, pp. 559-571, Aug 2016. -
  9. [9][9] F. Font, S. Oramas, G. Fazekas, and X. Serra, 'Extending tagging ontologies with domain specific knowledge,' in International Semantic Web Conference , Riva del Garda, Italy, October 2014, pp. 209-212. -
  10. [10][10] P. Saari, G. Fazekas, T. Eerola, M. Barthet, O. Lartillot, and M. Sandler, 'Genre-adaptive semantic computing and audio-based modelling for music mood annotation,' IEEE Transactions on Affective Computing , vol. 7, no. 2, pp. 122-135, 2016. -
  11. [11][11] Y. Freund, R. E. Schapire et al. , 'Experiments with a new boosting algorithm,' in icml , vol. 96, 1996, pp. 148-156. -
  12. [12][12] C. Baume, G. Fazekas, M. Barthet, D. Marston, and M. Sandler, 'Selection of audio features for music emotion recognition using production music,' in 53rd International Conference of the Audio Engineering Society on Semantic Audio , Jan 2014. -
  13. [13][13] M. A. Casey, R. Veltkamp, M. Goto, M. Leman, C. Rhodes, and M. Slaney, 'Content-based music information retrieval: Current directions and future challenges,' Proceedings of the IEEE , vol. 96, no. 4, pp. 668-696, April 2008. -
  14. [14][14] M. D. Hoffman, D. M. Blei, and P. R. Cook, 'Content-based musical similarity computation using the hierarchical dirichlet process.' in ISMIR , 2008, pp. 349-354. -
  15. [15][15] S. Dieleman and B. Schrauwen, 'End-to-end learning for music audio,' in Acoustics, Speech and Signal Processing (ICASSP), 2014 IEEE International Conference on . IEEE, 2014, pp. 6964-6968. -
  16. [16][16] J. Lorince, K. Joseph, and P. M. Todd, Analysis of Music Tagging and Listening Patterns: Do Tags Really Function as Retrieval Aids? Springer International Publishing, 2015, pp. 141-152. [Online]. Available: https://doi.org/10.1007/978-3-319-16268-3 15 - · DOI
  17. [17][17] T. Bertin-Mahieux, D. P. Ellis, B. Whitman, and P. Lamere, 'The million song dataset,' in ISMIR 2011: Proceedings of the 12th International Society for Music Information Retrieval Conference, October 24-28, 2011, Miami, Florida . University of Miami, 2011, pp. 591-596. -
  18. [18][18] D. Turnbull, L. Barrington, D. Torres, and G. Lanckriet, 'Towards musical query-by-semantic-description using the cal500 data set,' in Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval . ACM, 2007, pp. 439-446. -
  19. [20][20] B. Fr´ enay and M. Verleysen, 'Classification in the presence of label noise: a survey,' IEEE transactions on neural networks and learning systems , vol. 25, no. 5, pp. 845-869, 2014. -
  20. [21][21] V. Mnih and G. E. Hinton, 'Learning to label aerial images from noisy data,' in Proceedings of the 29th International Conference on Machine Learning (ICML-12) , 2012, pp. 567-574. -
  21. [22][22] S. Sukhbaatar and R. Fergus, 'Learning from noisy labels with deep neural networks,' arXiv preprint arXiv:1406.2080 , vol. 2, no. 3, p. 4, 2014. - · arXiv
  22. [23][23] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, 'Gradient-based learning applied to document recognition,' Proceedings of the IEEE , vol. 86, no. 11, pp. 2278-2324, 1998. -
  23. [24][24] E. J. Humphrey and J. P. Bello, 'Rethinking automatic chord recognition with convolutional neural networks,' in Machine Learning and Applications, 11th International Conference on , vol. 2. IEEE, 2012, pp. 357-362. -
  24. [26][26] J. Schl¨ uter and S. B¨ ock, 'Musical onset detection with convolutional neural networks,' in 6th International Workshop on Machine Learning and Music (MML), Prague, Czech Republic , 2013. -
  25. [27][27] A. Van den Oord, S. Dieleman, and B. Schrauwen, 'Deep content-based music recommendation,' in Advances in Neural Information Processing Systems , 2013, pp. 2643-2651. -
  26. [28][28] Y. Han, J. Kim, and K. Lee, 'Deep convolutional neural networks for predominant instrument recognition in polyphonic music,' IEEE/ACM Transactions on Audio, Speech, and Language Processing , vol. 25, no. 1, pp. 208-221, 2017. -
  27. [29][29] K. Choi, G. Fazekas, K. Cho, and M. Sandler, 'A tutorial on deep learning for music information retrieval,' arXiv preprint arXiv:1709.04396 , 2017. - · arXiv
  28. [30][30] J. Lee, J. Park, K. L. Kim, and J. Nam, 'Sample-level deep convolutional neural networks for music auto-tagging using raw waveforms,' arXiv preprint arXiv:1703.01789 , 2017. - · arXiv
  29. [31][31] K. Ullrich, J. Schl¨ uter, and T. Grill, 'Boundary detection in music structure analysis using convolutional neural networks,' in Proceedings of the 15th International Society for Music Information Retrieval Conference (ISMIR 2014), Taipei, Taiwan , 2014. -
  30. [32][32] S. B¨ ock, F. Krebs, and G. Widmer, 'Joint beat and downbeat tracking with recurrent neural networks,' in Proceedings of the International Society for Music Information Retrieval Conference (ISMIR) , 2016. -
  31. [33][33] A. Jansen, D. Ellis, D. Freedman, J. F. Gemmeke, W. Lawrence, and X. Liu, 'Large-scale audio event discovery in one million youtube videos,' in Proceedings of ICASSP , 2017. -
  32. [34][34] K. Choi, G. Fazekas, M. Sandler, and K. Cho, 'Convolutional recurrent neural networks for music classification,' in 2017 IEEE International Conference on Acoustics, Speech, and Signal Processing , 2016. -
  33. [35][35] S. Ioffe and C. Szegedy, 'Batch normalization: Accelerating deep network training by reducing internal covariate shift,' arXiv preprint arXiv:1502.03167 , 2015. - · arXiv
  34. [36][36] D.-A. Clevert, T. Unterthiner, and S. Hochreiter, 'Fast and accurate deep network learning by exponential linear units (elus),' arXiv preprint arXiv:1511.07289 , 2015. - · arXiv
  35. [38][38] L. Torresani, 'Weakly supervised learning,' in Computer Vision . Springer, 2014, pp. 883-885. -
  36. [39][39] K. Choi, G. Fazekas, K. Cho, and M. Sandler, 'A comparison on audio signal preprocessing methods for deep neural networks on music tagging,' arXiv preprint arXiv:1709.01922 , 2017. <!-- image --> <!-- image --> Keunwoo Choi received his B.Sc and M.Phil degrees in electrical engineering and computer science from Seoul National University, Seoul, South Korea, in 2009 and 2011, respectively. In 2011, He joined Electrical and Telecommunication Research Institute, Daejeon, South Korea, as a researcher. He is a PhD candidate at the Centre for Digital Music (C4DM), School of Electronic Engineering and Computer Science in Queen Mary University of London, London, UK, in 2014. Gy¨ orgy Fazekas is a lecturer at Queen Mary University of London (QMUL), working at the Centre for Digital Music (C4DM), School of Electronic Engineering and Computer Science. He received his BSc degree at Kand Klmn College of Electrical Engineering, Faculty of Electrical Engineering, buda University, Budapest, Hungary. He received his MSc and PhD degrees at QMUL, United Kingdom in 2012. His thesis titled Semantic Audio AnalysisUtilities and Applications explores novel applications of semantic audio analysis, semantic web tech- nologies and ontology based information management. His research interests include the development of semantic audio technologies and their application to creative music production. He is leading QMULs research on the EU funded Audio Commons project facilitating the use of open sound content in professional audio production. He collaborates on several other projects and he is a member of the IEEE, AES and ACM. <!-- image --> Kyunghyun Cho is an assistant professor of computer science and data science at New York University. He was a postdoctoral fellow at University of Montreal until summer 2015 under the supervision of Prof. Yoshua Bengio, and received PhD and MSc degrees from Aalto University early 2014 under the supervision of Prof. Juha Karhunen, Dr. Tapani Raiko and Dr. Alexander Ilin. <!-- image --> Mark Sandler was born in 1955. He received the BSc and PhD degrees from the University of Essex, UK, in 1978 and 1984 respectively. He is Professor of Signal Processing and Founding Director of the Centre for Digital Music in the School of Electronic Engineering and Computer Science at Queen Mary University of London, UK. He has published over 400 papers in journals and conferences. He is Fellow of the Royal Academy of Engineering, IEEE, AES and IET. · arXiv

Cited by

Sorted by citations

Authors
Choi, K., Fazekas, G., Cho, K., & Sandler, M.
Affiliations
  • Queen Mary University of London (GB)
  • New York University (US)
ORCID
Venue
IEEE Transactions on Emerging Topics in Computational Intelligence
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Published
2018-04
Volume
2
Issue
2
Pages
139-149
ISSN
2471-285X (ISSN-L)
DOI
10.1109/tetci.2017.2771298
OpenAlex
W2774685635
Cited by
42
References
1
Citations by year
2026: 2 · 2024: 2 · 2023: 4 · 2022: 9 · 2021: 10 · 2020: 7 · 2019: 4 · 2018: 1 · 2017: 3
FWCI
3.94
Type
journal article
Language
en
Open access
Licence
Locations
  • IEEE Transactions on Emerging Topics in Computational Intelligence (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • Queen Mary Research Online (Queen Mary University of London) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Funders
  • Engineering and Physical Sciences Research Council (EP/L019981/1)
  • Wolfson Research Merit Award
  • Queen Mary University of London
  • eBay
  • TenCent
  • Facebook
  • Google
  • NVIDIA
Topics
  • Physical Sciences › Computer Science › Signal Processing › Music and Audio Processing
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
Keywords
label noise, music tagging, deep convolutional neural networks, music classification, dataset reliability

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review