PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognition
Abstract
Audio pattern recognition is an important research topic in the machine learning area, and includes several tasks such as audio tagging, acoustic scene classification, music classification, speech emotion classification and sound event detection. Recently, neural networks have been applied to tackle audio pattern recognition problems. However, previous systems are built on specific datasets with limited durations. Recently, in computer vision and natural language processing, systems pretrained on large-scale datasets have generalized well to several tasks. However, there is limited research on pretraining systems on large-scale datasets for audio pattern recognition. In this paper, we propose pretrained audio neural networks (PANNs) trained on the large-scale AudioSet dataset. These PANNs are transferred to other audio related tasks. We investigate the performance and computational complexity of PANNs modeled by a variety of convolutional neural networks. We propose an architecture called Wavegram-Logmel-CNN using both log-mel spectrogram and waveform as input feature. Our best PANN system achieves a state-of-the-art mean average precision (mAP) of 0.439 on AudioSet tagging, outperforming the best previous system of 0.392. We transfer PANNs to six audio pattern recognition tasks, and demonstrate state-of-the-art performance in several of those tasks. We have released the source code and pretrained models of PANNs:https://github.com/qiuqiangkong/audioset_tagging_cnn.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
State-of-the-art audio event classification models may not perform well in detecting human vocal sounds.
“As a consequence, state-of-the-art audio event classification models may not perform well in detecting human vocal sounds.”
From Vocalsound: A Dataset for Improving Human Vocal Sounds Recognition (2022) · cites this paper as [5]
The average precision of state-of-the-art models on cough and sneeze classes is around 0.5 on the AudioSet evaluation set.
“the average precision of state-of-the-art models on cough and sneeze classes are only around 0.5 on the AudioSet evaluation set [5, 6].”
From Vocalsound: A Dataset for Improving Human Vocal Sounds Recognition (2022) · cites this paper as [5]
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818 · see citing works
musicnn: Pre-trained convolutional neural networks for music audio tagging
Pons, J., & Serra, X. (2019) · arXiv · doi:10.48550/arxiv.1909.06654 · Cited by 67 · cited 2× · see citing works
“Previous works on audio transfer learning [14], [21]–[23], [25] mainly focused on music tagging, and were limited to smaller datasets than AudioSet.”
BackgroundMethod
Flagged first, then by citations
Cited by
Sorted by citations
AST: Audio Spectrogram Transformer
Gong, Y., Chung, Y.-A., & Glass, J. (2021) · Interspeech 2021 · doi:10.21437/interspeech.2021-698 · Cited by 1,159 · see citing works
Wu, Y., Chen, K., Zhang, T., Hui, Y., Berg-Kirkpatrick, T., & Dubnov, S. (2023) · ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp49357.2023.10095969 · Cited by 459 · see citing works
“We select two models, PANN [10] and HTSAT [12], to construct the audio encoder.”
MethodInfluentialPerceiver: General Perception with Iterative Attention
Jaegle, A., Gimeno, F., Brock, A., Zisserman, A., Vinyals, O., & Carreira, J. (2021) · arXiv · doi:10.48550/arxiv.2103.03206 · Cited by 414 · see citing works
HTS-AT: A Hierarchical Token-Semantic Audio Transformer for Sound Classification and Detection
Chen, K., Du, X., Zhu, B., Ma, Z., Berg-Kirkpatrick, T., & Dubnov, S. (2022) · ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp43922.2022.9746312 · Cited by 294 · see citing works
Mei, X., Meng, C., Liu, H., Kong, Q., Ko, T., Zhao, C., Plumbley, M. D., Zou, Y., & Wang, W. (2024) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2024.3419446 · Cited by 150 · see citing works
Self-citationWhisper-AT: Noise-Robust Automatic Speech Recognizers are Also Strong General Audio Event Taggers
Gong, Y., Khurana, S., Karlinsky, L., & Glass, J. (2023) · INTERSPEECH 2023 · doi:10.21437/interspeech.2023-2193 · Cited by 65 · see citing works
Vocalsound: A Dataset for Improving Human Vocal Sounds Recognition
Gong, Y., Yu, J., & Glass, J. (2022) · ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp43922.2022.9746828 · Cited by 43 · see citing works
“Even the state-of-the-art audio classification models in [6, 5] cannot achieve satisfactory results for the vocal sound classes, e.”
Multimodal Emotion Recognition and Sentiment Analysis via Attention Enhanced Recurrent Model
Sun, L., Xu, M., Lian, Z., Liu, B., Tao, J., Wang, M., & Cheng, Y. (2021) · Proceedings of the 2nd on Multimodal Sentiment Analysis Challenge · doi:10.1145/3475957.3484456 · Cited by 36 · see citing works
Fonseca, E., Hershey, S., Plakal, M., Ellis, D. P. W., Jansen, A., & Moore, R. C. (2020) · IEEE Signal Processing Letters · doi:10.1109/lsp.2020.3006378 · Cited by 26 · see citing works
Ghani, B., Kalkman, V. J., Planqué, B., Vellinga, W.-P., Gill, L., & Stowell, D. (2025) · Scientific Reports · doi:10.1038/s41598-025-00996-2 · Cited by 22 · see citing works
Sparks of Large Audio Models: A Survey and Outlook
Latif, S., Shoukat, M., Shamshad, F., Usama, M., Ren, Y., Cuayáhuitl, H., Wang, W., Zhang, X., Togneri, R., Cambria, E., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.12792 · Cited by 17 · see citing works
“Large audio models dedicated to obtaining universal audio representations have also been widely applied in the field of sound event detection (SED), such as PANNs [238], AST [109], and BEATs [245].”
Self-citation
- Authors
- Kong, Q., Cao, Y., Iqbal, T., Wang, Y., Wang, W., & Plumbley, M. D.
- Affiliations
- University of Surrey (GB)
- ByteDance (China) (CN)
- Qingdao University of Science and Technology (CN)
- Venue
- IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Published
- 2020
- Volume
- 28
- Pages
- 2880-2894
- ISSN
- 2329-9290 (ISSN-L)
- 2329-9304
- OpenAlex
- W3094550259
- Cited by
- 1,235
- References
- 5
- Citations by year
- 2026: 234 · 2025: 268 · 2024: 245 · 2023: 205 · 2022: 164 · 2021: 103 · 2020: 16
- FWCI
- 66.06
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE/ACM Transactions on Audio Speech and Language Processing (publishedVersion)
- Surrey Research Insight Open Access (The University of Surrey) (submittedVersion)
- Funders
- Engineering and Physical Sciences Research Council (EP/N014111/1)
- China Scholarship Council (201406150082)
- EPSRC Doctoral Training Partnership (EP/N509772/1)
- National Natural Science Foundation of China (11804365)
- Topics
- Physical Sciences › Computer Science › Signal Processing › Music and Audio Processing
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Keywords
- AudioSet, audio tagging, transfer learning, convolutional neural networks, log-mel spectrogram, acoustic scene classification, music classification, speech emotion classification, sound event detection, computational complexity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
