A Survey of Audio Classification Using Deep Learning
Open accessThe DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.
Abstract
Deep learning can be used for audio signal classification in a variety of ways. It can be used to detect and classify various types of audio signals such as speech, music, and environmental sounds. Deep learning models are able to learn complex patterns of audio signals and can be trained on large datasets to achieve high accuracy. To employ deep learning for audio signal classification, the audio signal must first be represented in a suitable form. This can be done using signal representation techniques such as using spectrograms, Mel-frequency Cepstral coefficients, linear predictive coding, and wavelet decomposition. Once the audio signal is represented in a suitable form, it can then be fed into a deep learning model. Various deep learning models can be utilized for audio classification. We provide an extensive survey of current deep learning models used for a variety of audio classification tasks. In particular, we focus on works published under five different deep neural network architectures, namely Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), Autoencoders, Transformers and Hybrid Models (hybrid deep learning models and hybrid deep learning models with traditional classifiers). CNNs can be used to classify audio signals into different categories such as speech, music, and environmental sounds. They can also be used for speech recognition, speaker identification, and emotion recognition. RNNs are widely used for audio classification and audio segmentation. RNN models can capture temporal patterns of audio signals and be used to classify audio segments into different categories. Another approach is to use autoencoders for learning the features of audio signals and then classifying the signals into different categories. Transformers are also well-suited for audio classification. In particular, temporal and frequency features can be extracted to identify the characteristics of the audio signals. Finally, hybrid models for audio classification either combine various deep learning architectures (i.e. CNN-RNN) or combine deep learning models with traditional machine learning techniques (i.e. CNN-Support Vector Machine). These hybrid models take advantage of the strengths of different architectures while avoiding their weaknesses. Existing literature under different categories of deep learning are summarized and compared in detail.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Graves, A., Fernández, S., Gomez, F., & Schmidhuber, J. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143891 · Cited by 5,370
A survey of the recent architectures of deep convolutional neural networks
Khan, A., Sohail, A., Zahoora, U., & Qureshi, A. S. (2020) · Artificial Intelligence Review · doi:10.1007/s10462-020-09825-6 · Cited by 3,137
A database of German emotional speech
Burkhardt, F., Paeschke, A., Rolfes, M., Sendlmeier, W. F., & Weiss, B. (2005) · Interspeech 2005 · doi:10.21437/interspeech.2005-446 · Cited by 2,365 · cited 1×
“The EmoDB dataset is a collection of emotion-annotated audio recordings of German-speaking actors for emotion recognition from speech [168].”
BackgroundVoxCeleb: A Large-Scale Speaker Identification Dataset
Nagrani, A., Chung, J. S., & Zisserman, A. (2017) · Interspeech 2017 · doi:10.21437/interspeech.2017-950 · Cited by 2,210 · cited 1×
“The VoxCeleb dataset contains an extensive collection of over 100,000 utterances from 1,251 different celebrities [167].”
BackgroundAST: Audio Spectrogram Transformer
Gong, Y., Chung, Y.-A., & Glass, J. (2021) · Interspeech 2021 · doi:10.21437/interspeech.2021-698 · Cited by 1,159
Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition
Warden, P. (2018) · arXiv · doi:10.48550/arxiv.1804.03209 · Cited by 1,033 · cited 1×
“The Speech Commands dataset [173] contains spoken words and designed to assist keyword spotting systems.”
MethodSsast: Self-supervised audio spectrogram transformer
Gong, Y., Lai, C.-I., Chung, Y.-A., & Glass, J. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i10.21315 · Cited by 294
HTS-AT: A Hierarchical Token-Semantic Audio Transformer for Sound Classification and Detection
Chen, K., Du, X., Zhu, B., Ma, Z., Berg-Kirkpatrick, T., & Dubnov, S. (2022) · ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp43922.2022.9746312 · Cited by 294
Masked Autoencoders that Listen
Huang, P.-Y., Xu, H., Li, J., Baevski, A., Auli, M., Galuba, W., Metze, F., & Feichtenhofer, C. (2022) · arXiv · doi:10.48550/arxiv.2207.06405 · Cited by 206
Comparison of Pre-Trained CNNs for Audio Classification Using Transfer Learning
Tsalera, E., Papadakis, A., & Samarakou, M. (2021) · Journal of Sensor and Actuator Networks · doi:10.3390/jsan10040072 · Cited by 127 · cited 2×
“Moreover, training and evaluation of audio classification systems with deep neural networks (DNNs) is only possible with a large amount of audio data; without a large dataset, the system can be unsuccessful [35].”
Background
Cited by
The Tug-of-War Between Deepfake Generation and Detection
Lee, H., Lee, C., Farhat, K., Qiu, L., Geluso, S., Kim, A., & Etzioni, O. (2024) · arXiv · doi:10.48550/arxiv.2407.06174
Metadata
- Authors
- Zaman, K., Sah, M., Direkoglu, C., & Unoki, M.
- Venue
- IEEE Access
- Published
- 2023
- Volume
- 11
- Pages
- 106620-106649
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W4386918931
- Cited by
- 207
- References
- 10
- FWCI
- 25.46
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: A Survey of Audio Classification Using Deep Learning. https://app.cochat.ai/papers/10.1109/access.2023.3318015
- The paper
- Zaman, K., Sah, M., Direkoglu, C., & Unoki, M. (2023). A Survey of Audio Classification Using Deep Learning. IEEE Access. https://doi.org/10.1109/access.2023.3318015
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.