AST: Audio Spectrogram Transformer
The DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
Abstract
In the past decade, convolutional neural networks (CNNs) have been widely adopted as the main building block for endto-end audio classification models, which aim to learn a direct mapping from audio spectrograms to corresponding labels.To better capture long-range global context, a recent trend is to add a self-attention mechanism on top of the CNN, forming a CNN-attention hybrid model.However, it is unclear whether the reliance on a CNN is necessary, and if neural networks purely based on attention are sufficient to obtain good performance in audio classification.In this paper, we answer the question by introducing the Audio Spectrogram Transformer (AST), the first convolution-free, purely attention-based model for audio classification.We evaluate AST on various audio classification benchmarks, where it achieves new state-of-the-art results of 0.485 mAP on AudioSet, 95.6% accuracy on ESC-50, and 98.1% accuracy on Speech Commands V2.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Goel et al. [143] used a self-supervised audio spectrogram Transformer (SSAST) [144,145], a ViT-based architecture, to develop a contrastive learning framework, SSAST-CL, for detecting audio deepfakes. Their approach integrates Siamese networks with self-attention and cross-attention mechanisms to enhance the ViT’s feature representation”
From Audio Deepfake Detection: What Has Been Achieved and What Lies Ahead (2025) · cites this paper as [144]
In the citing paper
“This change enables LTU-AS to recognize both speech and audio events.”
From Joint Audio and Speech Understanding (2023) · cites this paper as [6]
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
Tan, M., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1905.11946 · Cited by 17,072
Convolutional networks for images, speech, and time series
LeCun, Y., & Bengio, Y. (1998) · HAL (Le Centre pour la Communication Scientifique Directe) · doi:10.5555/303568.303704 · Cited by 4,231
SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition
Park, D. S., Chan, W., Zhang, Y., Chiu, C.-C., Zoph, B., Cubuk, E. D., & Le, Q. V. (2019) · Interspeech 2019 · doi:10.21437/interspeech.2019-2680 · Cited by 3,558
Conformer: Convolution-augmented Transformer for Speech Recognition
Gulati, A., Qin, J., Chiu, C.-C., Parmar, N., Zhang, Y., Yu, J., Han, W., Wang, S., Zhang, Z., Wu, Y., et al. (2020) · Interspeech 2020 · doi:10.21437/interspeech.2020-3015 · Cited by 3,150
Cited by
ImageBind One Embedding Space to Bind Them All
Girdhar, R., El-Nouby, A., Liu, Z., Singh, M., Alwala, K. V., Joulin, A., & Misra, I. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01457 · Cited by 863
A comprehensive survey on applications of transformers for deep learning tasks
Islam, S., Elmekki, H., Elsebai, A., Bentahar, J., Drawel, N., Rjoub, G., & Pedrycz, W. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.122666 · Cited by 487
A survey on LLM-based multi-agent systems: Workflow, infrastructure, and challenges
Li, X., Wang, S., Zeng, S., Wu, Y., & Yang, Y. (2024) · Vicinagearth · doi:10.1007/s44336-024-00009-2 · Cited by 404
Audioclip: Extending Clip to Image, Text and Audio
Guzhov, A., Raue, F., Hees, J., & Dengel, A. (2022) · ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp43922.2022.9747631 · Cited by 330
Ssast: Self-supervised audio spectrogram transformer
Gong, Y., Lai, C.-I., Chung, Y.-A., & Glass, J. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i10.21315 · Cited by 294
Self-citationHTS-AT: A Hierarchical Token-Semantic Audio Transformer for Sound Classification and Detection
Chen, K., Du, X., Zhu, B., Ma, Z., Berg-Kirkpatrick, T., & Dubnov, S. (2022) · ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp43922.2022.9746312 · Cited by 294
Wayformer: Motion Forecasting via Simple & Efficient Attention Networks
Nayakanti, N., Al-Rfou, R., Zhou, A., Goel, K., Refaat, K. S., & Sapp, B. (2023) · 2023 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra48891.2023.10160609 · Cited by 261
Efficient Training of Audio Transformers with Patchout
Koutini, K., Schlüter, J., Eghbal-zadeh, H., & Widmer, G. (2022) · Interspeech 2022 · doi:10.21437/interspeech.2022-227 · Cited by 225
β-Variational autoencoders and transformers for reduced-order modelling of fluid flows
Solera-Rico, A., Sanmiguel Vila, C., Gómez-López, M., Wang, Y., Almashjary, A., Dawson, S. T. M., & Vinuesa, R. (2024) · Nature Communications · doi:10.1038/s41467-024-45578-4 · Cited by 211
InternVideo2: Scaling Foundation Models for Multimodal Video Understanding
Wang, Y., Li, K., Li, X., Yu, J., He, Y., Chen, G., Pei, B., Zheng, R., Wang, Z., Shi, Y., et al. (2025) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-73013-9_23 · Cited by 205
A Survey of Audio Classification Using Deep Learning
Zaman, K., Sah, M., Direkoglu, C., & Unoki, M. (2023) · IEEE Access · doi:10.1109/access.2023.3318015 · Cited by 207
Selva, J., Johansen, A. S., Escalera, S., Nasrollahi, K., Moeslund, T. B., & Clapés, A. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3243465 · Cited by 169
MERLOT RESERVE: Neural Script Knowledge through Vision and Language and Sound
Zellers, R., Lu, J., Lu, X., Yu, Y., Zhao, Y., Salehi, M., Kusupati, A., Hessel, J., Farhadi, A., & Choi, Y. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01589 · Cited by 160
Mei, X., Meng, C., Liu, H., Kong, Q., Ko, T., Zhao, C., Plumbley, M. D., Zou, Y., & Wang, W. (2024) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2024.3419446 · Cited by 150
A Transformer-Based Deep Learning Network for Underwater Acoustic Target Recognition
Feng, S., & Zhu, X. (2022) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2022.3201396 · Cited by 181
Vision Transformers are Parameter-Efficient Audio-Visual Learners
Lin, Y.-B., Sung, Y.-L., Lei, J., Bansal, M., & Bertasius, G. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00228 · Cited by 184
Tursunov, A., Mustaqeem, M., Choeh, J. Y., & Kwon, S. (2021) · Sensors · doi:10.3390/s21175892 · Cited by 118
Rabbi, A. B. K., & Jeelani, I. (2024) · Automation in Construction · doi:10.1016/j.autcon.2024.105443 · Cited by 144
Transformers for Tabular Data Representation: A Survey of Models and Applications
Badaro, G., Saeed, M., & Papotti, P. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00544 · Cited by 96
Liu, H., Zhu, Z., Becherini, G., Peng, Y., Su, M., Zhou, Y., Zhe, X., Iwamoto, N., Zheng, B., & Black, M. J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00115 · Cited by 240
Fish school feeding behavior quantification using acoustic signal and improved Swin Transformer
Zeng, Y., Yang, X., Pan, L., Zhu, W., Wang, D., Zhao, Z., Liu, J., Sun, C., & Zhou, C. (2022) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2022.107580 · Cited by 104
MAE-AST: Masked Autoencoding Audio Spectrogram Transformer
Baade, A., Peng, P., & Harwath, D. F. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-10961 · Cited by 168
Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Action
Lu, J., Clark, C., Lee, S.-H., Zhang, Z., Khosla, S., Marten, R., Hoiem, D., & Kembhavi, A. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02497 · Cited by 384
ERANNs: Efficient residual audio neural networks for audio pattern recognition
Verbitskiy, S., Berikov, V., & Vyshegorodtsev, V. (2022) · Pattern Recognition Letters · doi:10.1016/j.patrec.2022.07.012 · Cited by 116
A surrogate gradient spiking baseline for speech command recognition
Bittar, A., & Garner, P. N. (2022) · Frontiers in Neuroscience · doi:10.3389/fnins.2022.865897 · Cited by 135
Vision-Language-Action Models for Robotics: A Review Towards Real-World Applications
Kawaharazuka, K., Oh, J., Yamada, J., Posner, I., & Zhu, Y. (2025) · IEEE Access · doi:10.1109/access.2025.3609980 · Cited by 252
FineDance: A Fine-grained Choreography Dataset for 3D Full Body Dance Generation
Li, R., Zhao, J., Zhang, Y., Su, M., Ren, Z., Zhang, H., Tang, Y., & Li, X. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00939 · Cited by 183
OmniVec: Learning robust representations with cross modal sharing
Srivastava, S., & Sharma, G. (2024) · 2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv57701.2024.00127 · Cited by 73
VALOR: Vision-Audio-Language Omni-Perception Pretraining Model and Dataset
Liu, J., Chen, S., He, X., Guo, L., Zhu, X., Wang, W., & Tang, J. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3479776 · Cited by 239
Bae, S., Kim, J.-W., Cho, W.-Y., Baek, H., Son, S., Lee, B., Ha, C., Tae, K., Kim, S., & Yun, S.-Y. (2023) · Interspeech (USB) · doi:10.21437/interspeech.2023-1426 · Cited by 114
Learning Audio-Video Modalities from Image Captions
Nagrani, A., Seo, P. H., Seybold, B., Hauth, A., Manén, S., Sun, C., & Schmid, C. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19781-9_24 · Cited by 133
Transformer: A General Framework from Machine Translation to Others
Zhao, Y., Zhang, J., & Zong, C. (2023) · Machine Intelligence Research · doi:10.1007/s11633-022-1393-5 · Cited by 93
A survey on machine learning in ship radiated noise
Hummel, H. I., van der Mei, R., & Bhulai, S. (2024) · Ocean Engineering · doi:10.1016/j.oceaneng.2024.117252 · Cited by 86
BYOL for Audio: Exploring Pre-Trained General-Purpose Audio Representations
Niizumi, D., Takeuchi, D., Ohishi, Y., Harada, N., & Kashino, K. (2022) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2022.3221007 · Cited by 120
STM: Spectrogram Transformer Model for Underwater Acoustic Target Recognition
Li, P., Wu, J., Wang, Y., Lan, Q., & Xiao, W. (2022) · Journal of Marine Science and Engineering · doi:10.3390/jmse10101428 · Cited by 76
Whisper-AT: Noise-Robust Automatic Speech Recognizers are Also Strong General Audio Event Taggers
Gong, Y., Khurana, S., Karlinsky, L., & Glass, J. (2023) · INTERSPEECH 2023 · doi:10.21437/interspeech.2023-2193 · Cited by 65
Self-citationA Historical Survey of Advances in Transformer Architectures
Sajun, A. R., Zualkernan, I. A., & Sankalpa, D. (2024) · Applied Sciences · doi:10.3390/app14104316 · Cited by 90
Efficient Large-Scale Audio Tagging Via Transformer-to-CNN Knowledge Distillation
Schmid, F. T., Koutini, K., & Widmer, G. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10096110 · Cited by 117
DeViT: Decomposing Vision Transformers for Collaborative Inference in Edge Devices
Xu, G., Hao, Z., Luo, Y., Hu, H., An, J., & Mao, S. (2023) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2023.3315138 · Cited by 83
Artificial Intelligence-Based Underwater Acoustic Target Recognition: A Survey
Feng, S., Ma, S., Zhu, X., & Yan, M. (2024) · Remote Sensing · doi:10.3390/rs16173333 · Cited by 73
Audio Deepfake Detection: What Has Been Achieved and What Lies Ahead
Zhang, B., Cui, H., Nguyen, V., & Whitty, M. (2025) · Sensors · doi:10.3390/s25071989 · Cited by 72
Dell, M. (2025) · Journal of Economic Literature · doi:10.1257/jel.20241733 · Cited by 107
Tuckute, G., Feather, J., Boebinger, D., & McDermott, J. H. (2023) · PLoS Biology · doi:10.1371/journal.pbio.3002366 · Cited by 124
Ng, D., Chen, Y., Tian, B., Fu, Q., & Chng, E. S. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747025 · Cited by 76
Multimodal fusion models for pulmonary embolism mortality prediction
Cahan, N., Klang, E., Marom, E. M., Soffer, S., Barash, Y., Burshtein, E., Konen, E., & Greenspan, H. (2023) · Scientific Reports · doi:10.1038/s41598-023-34303-8 · Cited by 58
Video2Music: Suitable music generation from videos using an Affective Multimodal Transformer model
Kang, J., Poria, S., & Herremans, D. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.123640 · Cited by 114
Audio Mamba: Bidirectional State Space Model for Audio Representation Learning
Erol, M. H., Senocak, A., Feng, J., & Chung, J. S. (2024) · IEEE Signal Processing Letters · doi:10.1109/lsp.2024.3483009 · Cited by 88
Acoustic scene classification: A comprehensive survey
Ding, B., Zhang, T., Wang, C., Liu, G., Liang, J., Hu, R., Wu, Y., & Guo, D. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121902 · Cited by 73
Multimodal Action Quality Assessment
Zeng, L.-A., & Zheng, W. (2024) · IEEE Transactions on Image Processing · doi:10.1109/tip.2024.3362135 · Cited by 84
Sound Classification and Processing of Urban Environments: A Systematic Literature Review
Nogueira, A. F. R., Oliveira, H. S., Machado, J. J. M., & Tavares, J. M. R. S. (2022) · Sensors · doi:10.3390/s22228608 · Cited by 63
CLIPPO: Image-and-Language Understanding from Pixels Only
Tschannen, M., Mustafa, B., & Houlsby, N. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01059 · Cited by 106
Towards Learning Universal Audio Representations
Wang, L., Luc, P., Wu, Y., Recasens, A., Smaira, L., Brock, A., Jaegle, A., Alayrac, J.-B., Dieleman, S., Carreira, J., et al. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746790 · Cited by 101
Xu, H., Tian, Y., Ren, H., & Liu, X. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.123768 · Cited by 59
A survey of video-based human action recognition in team sports
Yin, H., Sinnott, R., & Jayaputera, G. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10934-9 · Cited by 76
Masked Spectrogram Prediction for Self-Supervised Audio Pre-Training
Chong, D., Wang, H., Zhou, P., & Zeng, Q. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10095691 · Cited by 96
Audiovisual Masked Autoencoders
Georgescu, M.-I., Fonseca, E., Ionescu, R. T., Lucic, M., Schmid, C., & Arnab, A. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01479 · Cited by 44
Classify Respiratory Abnormality in Lung Sounds Using STFT and a Fine-Tuned ResNet18 Network
Chen, Z., Wang, H., Yeh, C.-H., & Liu, X. (2022) · IEEE Biomedical Circuits and Systems Conference (BioCAS) · doi:10.1109/biocas54905.2022.9948614 · Cited by 46
Chen, L., Luo, X., & Zhou, H. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.107320 · Cited by 52
Gong, Y., Luo, H., Liu, A. H., Karlinsky, L., & Glass, J. (2023) · arXiv · doi:10.48550/arxiv.2305.10790 · Cited by 43
Self-citationSelf-Supervised Pretraining via Multimodality Images With Transformer for Change Detection
Zhang, Y., Zhao, Y., Dong, Y., & Du, B. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3271024 · Cited by 52
Empirical Analysis of Honeybees Acoustics as Biosensors Signals for Swarm Prediction in Beehives
Iqbal, K., Alabdullah, B. I., Mudawi, N. A., Algarni, A. M., Jalal, A., & Park, J. (2024) · IEEE Access · doi:10.1109/access.2024.3471895 · Cited by 46
Multimodal Fish Feeding Intensity Assessment in Aquaculture
Cui, M., Liu, X., Liu, H., Du, Z., Chen, T., Lian, G., Li, D., & Wang, W. (2024) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2024.3507098 · Cited by 57
TRILLsson: Distilled Universal Paralinguistic Speech Representations
Shor, J., & Venugopalan, S. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-118 · Cited by 73
OmniVec2 - A Novel Transformer Based Network for Large Scale Multimodal and Multitask Learning
Srivastava, S., & Sharma, G. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02588 · Cited by 71
Deep Vision Multimodal Learning: Methodology, Benchmark, and Trend
Chai, W., & Wang, G. (2022) · Applied Sciences · doi:10.3390/app12136588 · Cited by 56
A Lightweight Network Model Based on an Attention Mechanism for Ship-Radiated Noise Classification
Yang, S., Lingzhi, X., Hong, X., & Zeng, X. (2023) · Journal of Marine Science and Engineering · doi:10.3390/jmse11020432 · Cited by 39
An efficient speech emotion recognition based on a dual-stream CNN-transformer fusion network
Tellai, M., Gao, L., & Mao, Q. (2023) · International Journal of Speech Technology · doi:10.1007/s10772-023-10035-y · Cited by 43
EclipSE: Efficient Long-Range Video Retrieval Using Sight and Sound
Lin, Y.-B., Lei, J., Bansal, M., & Bertasius, G. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19830-4_24 · Cited by 66
Audio-Enhanced Text-to-Video Retrieval using Text-Conditioned Feature Alignment
Ibrahimi, S., Sun, X., Wang, P., Garg, A., Sanan, A., & Omar, M. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01107 · Cited by 72
Masking Hierarchical Tokens for Underwater Acoustic Target Recognition With Self-Supervised Learning
Feng, S., Zhu, X., & Ma, S. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3358719 · Cited by 46
Kim, J.-W., Bae, S., Cho, W.-Y., Lee, B., & Jung, H. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10447734 · Cited by 67
MuseChat: A Conversational Music Recommendation System for Videos
Dong, Z., Liu, X., Chen, B., Polak, P., & Zhang, P. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01214 · Cited by 75
MuQ: Self-Supervised Music Representation Learning With Mel Residual Vector Quantization
Zhu, H., Zhou, Y., Chen, H., Yu, J., Ma, Z., Gu, R., Luo, Y., Tan, W., & Chen, X. (2025) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2025.3602320 · Cited by 146
InFi: end-to-end learnable input filter for resource-efficient mobile-centric inference
Yuan, M., Zhang, L., He, F., Tong, X., & Li, X. (2022) · ACM Annual International Conference on Mobile Computing and Networking (MobiCom) · doi:10.1145/3495243.3517016 · Cited by 53
Emotional Speech-Driven 3D Body Animation via Disentangled Latent Diffusion
Chhatre, K., Daněček, R., Athanasiou, N., Becherini, G., Peters, C., Black, M. J., & Bolkart, T. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00190 · Cited by 88
MetaAudio: A Few-Shot Audio Classification Benchmark
Heggan, C., Budgett, S., Hospedales, T. M., & Yaghoobi, M. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-15919-0_19 · Cited by 60
COVID-19 Detection From Respiratory Sounds With Hierarchical Spectrogram Transformers
Aytekin, I., Dalmaz, O., Gönç, K., Ankışhan, H., Sarıtaş, E. Ü., Bağcı, U., Celik, H., & Çukur, T. (2023) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2023.3339700 · Cited by 47
Self-Supervised Audio Teacher-Student Transformer for Both Clip-Level and Frame-Level Tasks
Li, X. W., Shao, N., & Li, X. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3352248 · Cited by 88
Masked Modeling Duo: Towards a Universal Audio Pre-Training Framework
Niizumi, D., Takeuchi, D., Ohishi, Y., Harada, N., & Kashino, K. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3389636 · Cited by 93
SSAMBA: Self-Supervised Audio Representation Learning With Mamba State Space Model
Shams, S., Dindar, S. S., Jiang, X., & Mesgarani, N. (2024) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt61566.2024.10832304 · Cited by 72
Multiscale Audio Spectrogram Transformer for Efficient Audio Classification
Zhu, W., & Omar, M. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10096513 · Cited by 50
CAT: Causal Audio Transformer for Audio Classification
Liu, X., Lu, H., Yuan, J., & Li, X. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10096787 · Cited by 50
It's Time for Artistic Correspondence in Music and Video
Surís, D., Vondrick, C., Russell, B. C., & Salamon, J. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01031 · Cited by 50
Alfasly, S., Lu, J., Xu, C., & Zou, Y. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01957 · Cited by 47
Skating-Mixer: Long-Term Sport Audio-Visual Modeling with MLPs
Xia, J., Zhuge, M., Geng, T., Fan, S., Wei, Y., He, Z., & Zheng, F. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i3.25392 · Cited by 61
Advancing Weakly-Supervised Audio-Visual Video Parsing via Segment-Wise Pseudo Labeling
Zhou, J., Guo, D., Zhong, Y., & Wang, M. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-024-02142-3 · Cited by 55
LoCoNet: Long-Short Context Network for Active Speaker Detection
Wang, X., Cheng, F., & Bertasius, G. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01747 · Cited by 57
Pruning vs XNOR-Net: A Comprehensive Study of Deep Learning for Audio Classification on Edge-Devices
Mohaimenuzzaman, M., Bergmeir, C., & Meyer, B. (2022) · IEEE Access · doi:10.1109/access.2022.3140807 · Cited by 40
Zero-Shot Audio Source Separation through Query-Based Learning from Weakly-Labeled Data
Chen, K., Du, X., Zhu, B., Ma, Z., Berg-Kirkpatrick, T., & Dubnov, S. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i4.20366 · Cited by 68
Fall detection from audios with Audio Transformers
Kaur, P., Wang, Q., & Shi, W. (2022) · Smart Health · doi:10.1016/j.smhl.2022.100340 · Cited by 39
Masked Modeling Duo: Learning Representations by Encouraging Both Networks to Model the Input
Niizumi, D., Takeuchi, D., Ohishi, Y., Harada, N., & Kashino, K. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10097236 · Cited by 67
Dynamic Convolutional Neural Networks as Efficient Pre-Trained Audio Models
Schmid, F. T., Koutini, K., & Widmer, G. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3376984 · Cited by 50
PolyViT: Co-training Vision Transformers on Images, Videos and Audio
Likhosherstov, V., Arnab, A., Choromański, K., Lučić, M., Tay, Y., Weller, A., & Dehghani, M. (2023) · Transactions on machine learning research · doi:10.48550/arxiv.2111.12993 · Cited by 86
AudioProtoPNet: An interpretable deep learning model for bird sound classification
Heinrich, R., Rauch, L., Sick, B., & Scholz, C. (2025) · Ecological Informatics · doi:10.1016/j.ecoinf.2025.103081 · Cited by 48
Chatbot technology as a means of forming foreign language grammatical competence in self-study
Avramenko, A. P., Akhmedova, A. S., & Робертовна, Б. Е. (2023) · Вестник Тамбовского университета. Серия: Гуманитарные науки · doi:10.20310/1810-0201-2023-28-2-386-394 · Cited by 31
EAV: EEG-Audio-Video Dataset for Emotion Recognition in Conversational Contexts
Lee, M.-H., Shomanov, A., Begim, B., Kabidenova, Z., Nyssanbay, A., Yazıcı, A., & Lee, S. (2024) · Scientific Data · doi:10.1038/s41597-024-03838-4 · Cited by 63
AudioProtoPNet: An interpretable deep learning model for bird sound classification
Heinrich, R., Rauch, L., Sick, B., & Scholz, C. (2024) · arXiv · doi:10.48550/arxiv.2404.10420 · Cited by 31
Si, X., Wu, X., Sheng, H., Zhu, J. J., & Li, Z. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3354456 · Cited by 62
Jeong, S.-M., Kim, S., Lee, E. C., & Kim, H. (2024) · Sensors · doi:10.3390/s24144625 · Cited by 39
Deep Learning in Heart Sound Analysis: From Techniques to Clinical Applications
Zhao, Q., Geng, S., Wang, B., Sun, Y., Nie, W., Bai, B., Yu, C., Zhang, F., Tang, G., Zhang, D., et al. (2024) · Health Data Science · doi:10.34133/hds.0182 · Cited by 58
Metadata
- Authors
- Gong, Y., Chung, Y.-A., & Glass, J.
- Affiliations
- MIT Computer Science and Artificial Intelligence Laboratory (US)
- Venue
- Interspeech 2021
- Publisher
- ISCA
- Published
- 2021-08-30
- Pages
- 571-575
- ISSN
- 2308-457X (ISSN-L)
- OpenAlex
- W3196974791
- Cited by
- 1,159
- References
- 33
- Citations by year
- 2026: 260 · 2025: 356 · 2024: 241 · 2023: 193 · 2022: 100 · 2021: 9
- FWCI
- 82.22
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Interspeech (USB) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Signal Processing › Music and Audio Processing
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Keywords
- Audio Spectrogram Transformer, audio classification, self-attention, spectrogram, AudioSet, ESC-50
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.