The DOI matches its Crossref record; 75 of the 75 references it cites were found in a registry or an index.
A Survey of Audio-Based Music Classification and Annotation
Abstract
Music information retrieval (MIR) is an emerging research area that receives growing attention from both the research community and music industry. It addresses the problem of querying and retrieving certain types of music from large music data set. Classification is a fundamental problem in MIR. Many tasks in MIR can be naturally cast in a classification setting, such as genre classification, mood classification, artist recognition, instrument recognition, etc. Music annotation, a new research area in MIR that has attracted much attention in recent years, is also a classification problem in the general sense. Due to the importance of music classification in MIR research, rapid development of new methods, and lack of review papers on recent progress of the field, we provide a comprehensive review on audio-based classification in this paper and systematically summarize the state-of-the-art techniques for music classification. Specifically, we have stressed the difference in the features and the types of classifiers used for different classification tasks. This survey emphasizes on recent development of the techniques and discusses several open issues for future research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
A decision-theoretic generalization of on-line learning and an application to boosting
Freund, Y., & Schapire, R. E. (1997) · Journal of Computer and System Sciences · doi:10.1006/jcss.1997.1504 · Cited by 20,466
Nearest neighbor pattern classification
Cover, T., & Hart, P. (1967) · IEEE Transactions on Information Theory · doi:10.1109/tit.1967.1053964 · Cited by 16,936
A training algorithm for optimal margin classifiers
Boser, B. E., Guyon, I. M., & Vapnik, V. N. (1992) · Proceedings of the fifth annual workshop on Computational learning theory · doi:10.1145/130385.130401 · Cited by 11,992
Robust Face Recognition via Sparse Representation
Wright, J., Yang, A. Y., Ganesh, A., Sastry, S. S., & Ma, Y. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.79 · Cited by 9,388
Wolpert, D. H. (1992) · Neural Networks · doi:10.1016/s0893-6080(05)80023-1 · Cited by 7,951 · cited 3×
“stacked generalization (SG) [125], which provides a cascaded framework for classification by stacking classifiers on top of classifiers.”
MethodSemi-Supervised Learning Literature Survey
Zhu, X. (2005) · Minds at UW (University of Wisconsin) · Cited by 4,506
Abe, S. (2001) · Springer eBooks · doi:10.1007/978-1-4471-0285-4 · Cited by 3,721
Musical genre classification of audio signals
Tzanetakis, G., & Cook, P. (2002) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/tsa.2002.800560 · Cited by 2,798
Pattern Classification (2nd ed.)
Duda, R. O., Hart, P. E., & Stork, D. G. (1999) · Cited by 2,520
Learning multi-label scene classification
Boutell, M. R., Luo, J., Shen, X., & Brown, C. M. (2004) · Pattern Recognition · doi:10.1016/j.patcog.2004.03.009 · Cited by 2,296
A Kernel Method for Multi-Labelled Classification
Elisseeff, A., & Weston, J. (2002) · Advances in Neural Information Processing Systems 14 · doi:10.7551/mitpress/1120.003.0092 · Cited by 1,342
The biopsychology of mood and arousal
(1990) · Choice Reviews Online · doi:10.5860/choice.28-1830 · Cited by 1,240
(2009) · neural information processing systems · Cited by 962
Experimental Studies of the Elements of Expression in Music
Hevner, K. (1936) · The American Journal of Psychology · doi:10.2307/1415746 · Cited by 930
Calculation of a constant Q spectral transform
Brown, J. C. (1991) · The Journal of the Acoustical Society of America · doi:10.1121/1.400476 · Cited by 884
On the Dimensional and Hierarchical Structure of Affect
Tellegen, A., Watson, D., & Clark, L. A. (1999) · Psychological Science · doi:10.1111/1467-9280.00157 · Cited by 801
Content-Based Music Information Retrieval: Current Directions and Future Challenges
Casey, M. A., Veltkamp, R., Goto, M., Leman, M., Rhodes, C., & Slaney, M. (2008) · Proceedings of the IEEE · doi:10.1109/jproc.2008.916370 · Cited by 651
Multi-Label Classification Of Music Into Emotions.
Trohidis, K., Tsoumakas, G., Kalliris, G. M., & Vlahavas, I. P. (2008) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.1414899 · Cited by 607
A Regression Approach to Music Emotion Recognition
Yang, Y., Lin, Y., Su, Y.-F., & Chen, H. H. (2008) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2007.911513 · Cited by 541
Automatic mood detection and tracking of music audio signals
Lu, L., Liu, D., & Zhang, H.-J. (2006) · IEEE Transactions on Audio, Speech and Language Processing · doi:10.1109/tsa.2005.860344 · Cited by 518
The biopsychology of mood and arousal
Eysenck, H. J. (1990) · Personality and Individual Differences · doi:10.1016/0191-8869(90)90284-x · Cited by 515
Semantic Annotation and Retrieval of Music and Sound Effects
Turnbull, D., Barrington, L., Torres, D., & Lanckriet, G. (2008) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.1109/tasl.2007.913750 · Cited by 483
Online Learning: Theory, Algorithms, and Applications
Shalev‐Shwartz, S. (2007) · Cited by 418
A comparative study on content-based music genre classification
Li, T., Ogihara, M., & Li, Q. (2003) · Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval · doi:10.1145/860435.860487 · Cited by 410
Realtime Chord Recognition of Musical Sound : a System Using Common Lisp Music
Fujishima, T. (1999) · University of Michigan Library Repository · Cited by 404
Music type classification by spectral contrast feature
Jiang, D.-N., Lu, L., Zhang, H.-J., Tao, J.-H., & Cai, L.-H. (2003) · Proceedings. IEEE International Conference on Multimedia and Expo · doi:10.1109/icme.2002.1035731 · Cited by 339
A Discriminative Kernel-Based Approach to Rank Images from Text Queries
Grangier, D., & Bengio, S. (2008) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.70791 · Cited by 318
Multiple fundamental frequency estimation based on harmonicity and spectral smoothness
Klapuri, A. P. (2003) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/tsa.2003.815516 · Cited by 314
A music similarity function based on signal analysis
Logan, B., & Salomon, A. (2001) · IEEE International Conference on Multimedia and Expo, 2001. ICME 2001. · doi:10.1109/icme.2001.1237829 · Cited by 308
A Large-Scale Evaluation of Acoustic and Subjective Music-Similarity Measures
Berenzweig, A., Logan, B., Ellis, D. P. W., & Whitman, B. (2004) · Computer Music Journal · doi:10.1162/014892604323112257 · Cited by 289
Aggregate features and ADABOOST for music classification
Bergstra, J., Casagrande, N., Erhan, D., Eck, D., & Kégl, B. (2006) · Machine Learning · doi:10.1007/s10994-006-9019-7 · Cited by 279
Chroma Binary Similarity and Local Alignment Applied to Cover Song Identification
Serra, J., Gomez, E., Herrera, P., & Serra, X. (2008) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.1109/tasl.2008.924595 · Cited by 259
Brown, J. C. (1999) · The Journal of the Acoustical Society of America · doi:10.1121/1.426728 · Cited by 175
Musical Instrument Timbres Classification with Spectral Features
Agostini, G., Longari, M., & Pollastri, E. (2003) · EURASIP Journal on Advances in Signal Processing · doi:10.1155/s1110865703210118 · Cited by 171
Scanning the Dial: The Rapid Recognition of Music Genres
Gjerdingen, R. O., & Perrott, D. (2008) · Journal of New Music Research · doi:10.1080/09298210802479268 · Cited by 170
Shift-invariant sparse coding for audio classification
Grosse, R., Raina, R., Kwong, H., & Ng, A. Y. (2007) · Uncertainty in Artificial Intelligence · Cited by 168
Toward intelligent music information retrieval
Li, T., & Ogihara, M. (2006) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2006.870730 · Cited by 163
Content-Based Identification Of Audio Material Using Mpeg-7 Low Level Description.
Allamanche, E. (2001) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.1417852 · Cited by 162
Automatic Feature Extraction for Classifying Audio Data
Mierswa, I., & Morik, K. (2005) · Machine Learning · doi:10.1007/s10994-005-5824-7 · Cited by 158
Lee, C.-H., Shih, J.-L., Yu, K.-M., et al. (2009) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2009.2017635 · Cited by 148
Exploring Mood Metadata: Relationships with Genre, Artist and Usage Metadata.
Hu, X., & Downie, J. S. (2007) · International Symposium/Conference on Music Information Retrieval · Cited by 143
Audio classification and categorization based on wavelets and support vector Machine
Lin, C., Chen, S.-H., Truong, T., et al. (2005) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/tsa.2005.851880 · Cited by 136
A Benchmark Dataset For Audio Classification And Clustering.
Homburg, H., Mierswa, I., Möller, B., Morik, K. J., & Wurst, M. (2005) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.1417064 · Cited by 135
Support vector machine active learning for music retrieval
Mandel, M., Poliner, G. E., & Ellis, D. P. W. (2006) · Multimedia Systems · doi:10.1007/s00530-006-0032-2 · Cited by 133
Towards musical query-by-semantic-description using the CAL500 data set
Turnbull, D., Barrington, L., Torres, D., & Lanckriet, G. R. G. (2007) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/1277741.1277817 · Cited by 133
Classifying Music Audio with Timbral and Chroma Features
Ellis, D. P. W. (2007) · Columbia Academic Commons (Columbia University) · doi:10.7916/d8c255sd · Cited by 130
Spectral shape analysis in the central auditory system
Wang, K., & Shamma, S. (1995) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/89.466657 · Cited by 130
Autotagger: A Model for Predicting Social Tags from Acoustic Features on Large Music Databases
Bertin-Mahieux, T., Eck, D., Maillet, F., & Lamere, P. (2008) · Journal of New Music Research · doi:10.1080/09298210802479250 · Cited by 126
Audio Retrieval By Rhythmic Similarity.
Foote, J. T., Cooper, M. L., & Nam, U. (2002) · doi:10.5281/zenodo.1417602 · Cited by 120
Music Information Retrieval Using Social Tags and Audio
Levy, M., & Sandler, M. B. (2009) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2009.2012913 · Cited by 117
Shift-Invariance Sparse Coding for Audio ClassificationPreprint
Grosse, R., Raina, R., Kwong, H., & Ng, A. Y. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1206.5241 · Cited by 114
Musical Genre Classification Using Nonnegative Matrix Factorization-Based Features
Holzapfel, A., & Stylianou, Y. (2008) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2007.909434 · Cited by 107
Disambiguating Music Emotion Using Software Agents.
Dan, Y., & Lee, W. (2004) · Cited by 106
A comparison of human and automatic musical genre classification
Lippens, S., Martens, J.-P., & Mulder, T. D. (2004) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2004.1326806 · Cited by 103
Artist detection in music with Minnowmatch
Whitman, B., Flake, G. W., & Lawrence, S. (2002) · Machine learning for signal processing ... · doi:10.1109/nnsp.2001.943160 · Cited by 102
Slaney, M. (2002) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2002.5745561 · Cited by 100
Improving automatic music tag annotation using stacked generalization of probabilistic SVM outputs
Ness, S. R., Theocharis, A., Tzanetakis, G., & Martins, L. G. (2009) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/1631272.1631393 · Cited by 97
Fast recognition of musical genres using RBF networks
Turnbull, D., & Elkan, C. (2005) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2005.62 · Cited by 95
Audio Classification Based on MPEG-7 Spectral Basis Representations
Kim, H.-G., Moreau, N., & Sikora, T. (2004) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2004.826766 · Cited by 90
Easy As Cba: A Simple Probabilistic Model For Tagging Music.
Hoffman, M. D., Blei, D. M., & Cook, P. R. (2009) · doi:10.5281/zenodo.1417346 · Cited by 82
Bello, J. P. (2007) · doi:10.5281/zenodo.1417910 · Cited by 70
Combining audio content and social context for semantic music discovery
Turnbull, D., Barrington, L., Lanckriet, G. R. G., & Yazdani, M. (2009) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/1571941.1572009 · Cited by 69
Modeling timbre distance with temporal statistics from polyphonic music
Mörchen, F., Ultsch, A., Thies, M., et al. (2005) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tsa.2005.860352 · Cited by 68
Content-Based Musical Similarity Computation Using The Hierarchical Dirichlet Process.
Hoffman, M. D., Blei, D. M., & Cook, P. R. (2008) · doi:10.5281/zenodo.1417222 · Cited by 66
Musical instrument timbres classification with spectral features
Agostini, G., Longari, M., & Pollastri, E. (2002) · IEEE International Workshop on Multimedia Signal Processing (MMSP) · doi:10.1109/mmsp.2001.962718 · Cited by 61
Quantitative Analysis of a Common Audio Similarity Measure
Jensen, J. H., Christensen, M. G., Ellis, D. P. W., & Jensen, S. H. (2009) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2008.2012314 · Cited by 60
Score-Independent Audio Features for Description of Music Expression
Mion, L., & Poli, G. D. (2008) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2007.913743 · Cited by 59
Panagakis, Y., Kotropoulos, C. L., & Arce, G. R. (2009) · doi:10.5281/zenodo.1416413 · Cited by 58
Classification in music research
Weihs, C., Ligges, U., Mörchen, F., & Müllensiefen, D. (2007) · Advances in Data Analysis and Classification · doi:10.1007/s11634-007-0016-x · Cited by 57
FRAME LEVEL AUDIO SIMILARITY - A CODEBOOK APPROACH
Seyerlehner, K., Widmer, G., Kepler, J., & Knees, P. (2008) · Cited by 46
A Study On Music Genre Classification Based On Universal Acoustic Models.
Reed, J. T., & Lee, C. (2006) · doi:10.5281/zenodo.1417254 · Cited by 42
Improving Multilabel Analysis of Music Titles: A Large-Scale Validation of the Correction Approach
Francois Pachet, B., & Roy, P. (2009) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2008.2008734 · Cited by 32
Learning Musical Instruments From Mixtures Of Audio With Weak Labels.
Little, D., & Pardo, B. (2008) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.1417814 · Cited by 29
Using Regression To Combine Data Sources For Semantic Music Discovery.
Tomasik, B., Kim, J. H., Ladlow, M., Augat, M., Tingle, D., Wicentowski, R., & Turnbull, D. (2009) · doi:10.5281/zenodo.1415557 · Cited by 13
Cited by
Trends in audio signal feature extraction methods
Sharma, G., Umapathy, K., & Krishnan, S. (2020) · Applied Acoustics · doi:10.1016/j.apacoust.2019.107020 · Cited by 479
Gammatone Cepstral Coefficients: Biologically Inspired Features for Non-Speech Audio Classification
Valero, X., & Alias, F. (2012) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2012.2199972 · Cited by 281
Metadata
- Authors
- Fu, Z., Lu, G., Ting, K. M., & Zhang, D.
- Venue
- IEEE Transactions on Multimedia
- Published
- 2011-04
- Volume
- 13
- Issue
- 2
- Pages
- 303-319
- ISSN
- 1520-9210 (ISSN-L)
- 1941-0077
- OpenAlex
- W2107430826
- Cited by
- 435
- References
- 75
- FWCI
- 29.99
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.