CoChat
Sign Up
PreprintCited by 2

The DOI matches its DataCite record; 2 of the 2 references it cites were found in a registry or an index.

End-to-end learning for music audio tagging at scale

Pons, Jordi, Nieto, Oriol, Prockup, Matthew, Schmidt, Erik, Ehmann, Andreas, Serra, Xavier·2017·arXiv
DOI 10.48550/arxiv.1711.02520arXiv 1711.02520
Open this paper in CoChat — free account

Abstract

The lack of data tends to limit the outcomes of deep learning research, particularly when dealing with end-to-end learning stacks processing raw data such as waveforms. In this study, 1.2M tracks annotated with musical labels are available to train our end-to-end models. This large amount of data allows us to unrestrictedly explore two different design paradigms for music auto-tagging: assumption-free models - using waveforms as input with very small convolutional filters; and models that rely on domain knowledge - log-mel spectrograms with a convolutional neural network designed to learn timbral and temporal features. Our work focuses on studying how these two types of deep architectures perform when datasets of variable size are available for training: the MagnaTagATune (25k songs), the Million Song Dataset (240k songs), and a private dataset of 1.2M songs. Our experiments suggest that music domain assumptions are relevant when not enough training data are available, thus showing how waveform-based models outperform spectrogram-based ones in large-scale data scenarios.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.1711.02520

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Flagged first, then by citations

Research works printed but not linked: 24

  1. Yann Bayle, Pierre Hanna, and Matthias Robine. Revisiting autotagging toward faultless instrumental playlists generation. arXiv preprint arXiv:1706.07613, 2017. · arXiv
  2. Judith C Brown. Calculation of a constant q spectral transform. The Journal of the Acoustical Society of America, 89(1):425-434, 1991.
  3. Ning Chen and Shijun Wang. High-level music de- scriptor extraction algorithm based on combination of multi-channel cnns and lstm. In International Society for Music Information Retrieval Conference (ISMIR), 2017.
  4. Keunwoo Choi, George Fazekas, and Mark Sandler. Automatic tagging using deep convolutional neural networks. International Society for Music Information Retrieval Conference (ISMIR), 2016.
  5. Keunwoo Choi, György Fazekas, Mark Sandler, and Kyunghyun Cho. Convolutional recurrent neural net- works for music classification. In IEEE International Conference on Acoustics, Speech and Signal Process- ing (ICASSP), 2017.
  6. Jesse Davis and Mark Goadrich. The relationship between precision-recall and roc curves. In Inter- national Conference on Machine Learning (ICML). ACM, 2006.
  7. Michaël Defferrard, Kirell Benzi, Pierre Van- dergheynst, and Xavier Bresson. Fma: A dataset for music analysis. In International Society for Music Information Retrieval Conference (ISMIR), 2017.
  8. Sander Dieleman and Benjamin Schrauwen. End-to- end learning for music audio. In IEEE International Conference on Acoustics, Speech and Signal Process- ing (ICASSP), 2014.
  9. Jort F Gemmeke, Daniel PW Ellis, Dylan Freedman, Aren Jansen, Wade Lawrence, R Channing Moore, Manoj Plakal, and Marvin Ritter. Audio set: An on- tology and human-labeled dataset for audio events. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2017.
  10. Edith Law, Kris West, Michael I Mandel, Mert Bay, and J Stephen Downie. Evaluation of algorithms using games: The case of music tagging. In International So- ciety for Music Information Retrieval Conference (IS- MIR), 2009.
  11. Honglak Lee, Peter Pham, Yan Largman, and An- drew Y Ng. Unsupervised feature learning for au- dio classification using convolutional deep belief net- works. In Advances in Neural Information Processing Systems, 2009.
  12. Jongpil Lee and Juhan Nam. Multi-level and multi- scale feature aggregation using pretrained convolu- tional neural networks for music auto-tagging. IEEE signal processing letters, 24(8):1208-1212, 2017.
  13. Jongpil Lee, Jiyoung Park, Keunhyoung Luke Kim, and Juhan Nam. Sample-level deep convolutional neural networks for music auto-tagging using raw waveforms. Sound and Music Computing Conference (SMC), 2017.
  14. Jongpil Lee, Jiyoung Park, Keunhyoung Luke Kim, and Juhan Nam. Samplecnn: End-to-end deep convolu- tional neural networks using very small filters for mu- sic classification. Applied Sciences, 8(1):150, 2018.
  15. Hao Li, Zheng Xu, Gavin Taylor, and Tom Goldstein. Visualizing the loss landscape of neural nets. arXiv preprint arXiv:1712.09913, 2017. · arXiv
  16. Huy Phan, Lars Hertel, Marco Maass, and Alfred Mertins. Robust audio event recognition with 1-max pooling convolutional neural networks. arXiv preprint arXiv:1604.06338, 2016. · arXiv
  17. Jordi Pons and Xavier Serra. Designing efficient archi- tectures for modeling temporal features with convo- lutional neural networks. In IEEE International Con- ference on Acoustics, Speech and Signal Processing (ICASSP), 2017.
  18. Jordi Pons, Olga Slizovskaia, Rong Gong, Emilia Gómez, and Xavier Serra. Timbre analysis of music au- dio signals with convolutional neural networks. Euro- pean Signal Processing Conference (EUSIPCO2017), 2017.
  19. Matthew Prockup, Andrew J. Asman, Fabian Gouyon, Erik M. Schmidt, Oscar Celma, and Youngmoo E. Kim. Modeling rhythm using tree ensembles and the music genome project. Machine Learning for Music Discovery Workshop at the International Conference on Machine Learning (ICML), 2015.
  20. Colin Raffel. Learning-Based Methods for Comparing Sequences, with Applications to Audio-to-MIDI Align- ment and Matching. PhD thesis, 2016.
  21. Tara N Sainath, Ron J Weiss, Andrew Senior, Kevin W Wilson, and Oriol Vinyals. Learning the speech front- end with raw waveform cldnns. In Sixteenth Annual Conference of the International Speech Communica- tion Association, 2015.
  22. Jan Schluter and Sebastian Bock. Improved musical onset detection with convolutional neural networks. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2014.
  23. Mohamed Sordo, Cyril Laurier, and Oscar Celma. An- notating music collections: How content-based simi- larity helps to propagate labels. In International So- ciety for Music Information Retrieval Conference (IS- MIR), 2007.
  24. Zhenyao Zhu, Jesse H Engel, and Awni Hannun. Learning multiscale features directly from waveforms. arXiv preprint arXiv:1603.09509, 2016. · arXiv

Cited by

Sorted by citations

Metadata

Authors
Pons, J., Nieto, O., Prockup, M., Schmidt, E., Ehmann, A., & Serra, X.
Venue
arXiv
Publisher
arXiv
Published
2017
DOI
10.48550/arxiv.1711.02520
arXiv
1711.02520
References
2
Type
Preprint
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review