AST: Audio Spectrogram Transformer
The DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
Abstract
In the past decade, convolutional neural networks (CNNs) have been widely adopted as the main building block for endto-end audio classification models, which aim to learn a direct mapping from audio spectrograms to corresponding labels.To better capture long-range global context, a recent trend is to add a self-attention mechanism on top of the CNN, forming a CNN-attention hybrid model.However, it is unclear whether the reliance on a CNN is necessary, and if neural networks purely based on attention are sufficient to obtain good performance in audio classification.In this paper, we answer the question by introducing the Audio Spectrogram Transformer (AST), the first convolution-free, purely attention-based model for audio classification.We evaluate AST on various audio classification benchmarks, where it achieves new state-of-the-art results of 0.485 mAP on AudioSet, 95.6% accuracy on ESC-50, and 98.1% accuracy on Speech Commands V2.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Goel et al. [143] used a self-supervised audio spectrogram Transformer (SSAST) [144,145], a ViT-based architecture, to develop a contrastive learning framework, SSAST-CL, for detecting audio deepfakes. Their approach integrates Siamese networks with self-attention and cross-attention mechanisms to enhance the ViT’s feature representation”
From Audio Deepfake Detection: What Has Been Achieved and What Lies Ahead (2025) · cites this paper as [144]
In the citing paper
“This change enables LTU-AS to recognize both speech and audio events.”
From Joint Audio and Speech Understanding (2023) · cites this paper as [6]
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
Tan, M., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1905.11946 · Cited by 17,072
Convolutional networks for images, speech, and time series
LeCun, Y., & Bengio, Y. (1998) · HAL (Le Centre pour la Communication Scientifique Directe) · doi:10.5555/303568.303704 · Cited by 4,231
SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition
Park, D. S., Chan, W., Zhang, Y., Chiu, C.-C., Zoph, B., Cubuk, E. D., & Le, Q. V. (2019) · Interspeech 2019 · doi:10.21437/interspeech.2019-2680 · Cited by 3,558
Conformer: Convolution-augmented Transformer for Speech Recognition
Gulati, A., Qin, J., Chiu, C.-C., Parmar, N., Zhang, Y., Yu, J., Han, W., Wang, S., Zhang, Z., Wu, Y., et al. (2020) · Interspeech 2020 · doi:10.21437/interspeech.2020-3015 · Cited by 3,150
Cited by
Items 201–300 of 1,006
Basak, K., Mishra, N., & Chang, H.-T. (2023) · Sensors · doi:10.3390/s23198033 · Cited by 23
Sun, K., Xia, C., Zhang, X., Chen, H., & Zhang, J. (2024) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3643553 · Cited by 21
Training environmental sound classification models for real-world deployment in edge devices
Goulão, M., Bandeira, L., Martins, B., & Oliveira, A. L. (2024) · Discover Applied Sciences · doi:10.1007/s42452-024-05803-7 · Cited by 22
Learning incremental audio–visual representation for continual multimodal understanding
Zhu, B., Wang, C., Xu, K., Feng, D., Zhou, Z., & Zhu, X. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112513 · Cited by 18
Multilevel feature encoder for transfer learning-based fault detection on acoustic signal
Wang, D., & Chen, C. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103128 · Cited by 15
Zhao, J., Kuppuswamy, N. S., Feng, S., Burchfiel, B., & Adelson, E. H. (2025) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra55743.2025.11128816 · Cited by 50
Conformer-Based Self-Supervised Learning For Non-Speech Audio Tasks
Srivastava, S., Wang, Y., Tjandra, A., Kumar, A., Liu, C., Singh, K., & Saraf, Y. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746490 · Cited by 33
Effective Audio Classification Network Based on Paired Inverse Pyramid Structure and Dense MLP Block
Chen, Y., Zhu, Y., Yan, Z., Ren, Z., Huang, Y., Shen, J., & Chen, L. (2023) · Lecture notes in computer science · doi:10.1007/978-981-99-4742-3_6 · Cited by 20
Kang, J., & Herremans, D. (2025) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2025.3583505 · Cited by 37
Dataset Balancing Can Hurt Model Performance
Moore, R. C., Ellis, D. P. W., Fonseca, E., Hershey, S., Jansen, A., & Plakal, M. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10095255 · Cited by 27
Active few-shot learning for rare bioacoustic feature annotation
McEwen, B., Soltero, K., Gutschmidt, S., Bainbridge-Smith, A., Atlas, J., & Green, R. R. (2024) · Ecological Informatics · doi:10.1016/j.ecoinf.2024.102734 · Cited by 19
A Self-Supervised Learning Network for Student Engagement Recognition From Facial Expressions
Zhang, W.-L., Jia, R., Wang, H., Che, C.-Y., & Sun, H. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3436029 · Cited by 16
VIT-LENS: Towards Omni-modal Representations
Lei, W., Ge, Y., Yi, K., Zhang, J., Gao, D., Sun, D., Ge, Y., Shan, Y., & Shou, M. Z. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02516 · Cited by 49
CrossMAE: Cross-Modality Masked Autoencoders for Region-Aware Audio-Visual Pre-Training
Guo, Y., Sun, S., Ma, S., Zheng, K., Bao, X., Ma, S., Zou, W., & Zheng, Y. P. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02523 · Cited by 34
Classifying vocal responses of broilers to environmental stressors via artificial neural network
Lev-Ron, T., Yitzhak Yitzhaky, B., Halachmi, I., & Druyan, S. (2024) · animal · doi:10.1016/j.animal.2024.101378 · Cited by 18
Banchero, L., Vacalebri, F., Mossi, J. M., & López, J. J. (2025) · Sensors · doi:10.3390/s25030793 · Cited by 19
Effective Pre-Training of Audio Transformers for Sound Event Detection
Schmid, F., Morocutti, T., Foscarin, F., Schlüter, J., Primus, P., & Widmer, G. (2024) · IEEE International Conference on Acoustics, Speech, and Signal Processing · doi:10.1109/icassp49660.2025.10888942 · Cited by 27
Respiratory sounds classification by fusing the time-domain and 2D spectral features
Dong, G., Shen, Y., Wang, J., Zhang, M., Sun, P., & Zhang, M. (2025) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2025.107790 · Cited by 15
PACS: A Dataset for Physical Audiovisual CommonSense Reasoning
Yu, S., Wu, P., Liang, P. P., Salakhutdinov, R., & Morency, L. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19836-6_17 · Cited by 18
A hierarchical birdsong feature extraction architecture combining static and dynamic modeling
Wang, Y., Chen, A., Li, H., Zhou, G., Yi, J., & Zhang, Z. (2023) · Ecological Indicators · doi:10.1016/j.ecolind.2023.110258 · Cited by 16
Contrastive Speech Mixup for Low-Resource Keyword Spotting
Ng, D., Zhang, R., Yip, J. Q., Zhang, C., Ma, Y., Nguyen, T. H., Ni, C., Chng, E. S., & Ma, B. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10096976 · Cited by 13
Zimmermann, J., Demartsev, V., Averly, B., Dhanjal‐Adams, K. L., Duteil, M., Gall, G. E. C., Faiß, M., Johnson-Ulrich, L., Stowell, D., Manser, M. B., et al. (2025) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.70218 · Cited by 35
Spectrogram Features for Audio and Speech Analysis
McLoughlin, I. V., Pham, L. D., Song, Y., Miao, X., Phan, H., Cai, P., Gu, Q., Nan, J., Song, H., & Soh, D. C. L. (2026) · Applied Sciences · doi:10.3390/app16020572 · Cited by 26
Improving Misfire Fault Diagnosis with Cascading Architectures via Acoustic Vehicle Characterization
Terwilliger, A. M., & Siegel, J. E. (2022) · Sensors · doi:10.3390/s22207736 · Cited by 17
Transformer-based Automatic Music Mood Classification Using Multi-modal Framework
Sujeesha, A. S., & Rajan, R. (2023) · Journal of Computer Science and Technology · doi:10.24215/16666038.23.e02 · Cited by 16
Segment-Based Spotting of Bowel Sounds Using Pretrained Models in Continuous Data Streams
Baronetto, A., Graf, L. S., Fischer, S., Neurath, M. F., & Amft, O. (2023) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2023.3269910 · Cited by 14
Privileged representational axes in biological and artificial neural networks
Khosla, M., Williams, A. H., McDermott, J. H., & Kanwisher, N. (2023) · Conference on Cognitive Computational Neuroscience (CCN) · doi:10.32470/ccn.2023.1664-0 · Cited by 16
BLAT: Bootstrapping Language-Audio Pre-training based on AudioSet Tag-guided Synthetic Data
Xu, X., Zhang, Z., Zhou, Z., Zhang, P., Xie, Z., Wu, M., & Zhu, K. Q. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3613820 · Cited by 29
Learning from Taxonomy: Multi-Label Few-Shot Classification for Everyday Sound Recognition
Liang, J., Phan, H., & Benetos, E. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10446908 · Cited by 22
Priebe, D., Ghani, B., & Stowell, D. (2024) · Sensors · doi:10.3390/s24072046 · Cited by 14
FairSSD: Understanding Bias in Synthetic Speech Detectors
Singh Yadav, A. K., Bhagtani, K., Salvi, D., Bestagini, P., & Delp, E. J. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00445 · Cited by 12
Transformer Models improve the acoustic recognition of buzz-pollinating bee species
Ferreira, A. I. S., da Silva, N. F. F., Mesquita, F. N., Rosa, T. C., Buchmann, S. L., & Neto, J. N. M. (2025) · Ecological Informatics · doi:10.1016/j.ecoinf.2025.103010 · Cited by 15
Developing a multi-feature fusion model for exacerbation classification in asthma and COPD
Yan, Y., van Bemmel, L., Franssen, F. M. E., Simons, S. O., & Urovi, V. (2025) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2025.108796 · Cited by 16
Uncertain Multimodal Intention and Emotion Understanding in the Wild
Yang, Q., Shi, Q., Wang, T., & Ye, M. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02300 · Cited by 29
Sound Localization by Self-supervised Time Delay Estimation
Chen, Z., Fouhey, D. F., & Owens, A. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19809-0_28 · Cited by 26
Blind Room Acoustic Parameters Estimation Using Mobile Audio Transformer
Saini, S., & Peissig, J. (2023) · IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA) · doi:10.1109/waspaa58266.2023.10248186 · Cited by 13
Transformer-Based Speech Synthesizer Attribution in an Open Set Scenario
Bartusiak, E. R., & Delp, E. J. (2022) · International Conference on Machine Learning and Applications · doi:10.1109/icmla55696.2022.00054 · Cited by 22
Deep Learning for Heart Sound Analysis: A Literature ReviewPreprint
Zhao, Q.-H., Geng, S., Wang, B., Sun, Y., Nie, W., Bai, B., Yu, C., Zhang, F., Tang, G., Zhang, D.-Y., et al. (2023) · medRxiv · doi:10.1101/2023.09.16.23295653 · Cited by 7
Sparks of Large Audio Models: A Survey and Outlook
Latif, S., Shoukat, M., Shamshad, F., Usama, M., Ren, Y., Cuayáhuitl, H., Wang, W., Zhang, X., Togneri, R., Cambria, E., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.12792 · Cited by 17
“The architecture of the LTU model draws from various components, including an audio spectrogram transformer (AST) [109] as the audio encoder, LLaMA [93] as the large language model (LLM) enhanced with Vicuna [110] instructions, Low-rank Adapter [94], and specific generation settings.”
MethodSingle- and Cross-Lingual Speech Emotion Recognition Based on WavLM Domain Emotion Embedding
Yang, J., Liu, J., Huang, K., Xia, J., Zhu, Z., & Zhang, H. (2024) · Electronics · doi:10.3390/electronics13071380 · Cited by 16
Xu, H., Fan, Z., & Liu, X. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.108760 · Cited by 11
Mehra, S., Ranga, V., & Agarwal, R. (2024) · Computational Intelligence · doi:10.1111/coin.70012 · Cited by 16
Computer Audition: From Task-Specific Machine Learning to Foundation Models
Triantafyllopoulos, A., Tsangko, I., Gebhard, A., Mesaros, A., Virtanen, T. I., & Schuller, B. W. (2025) · Proceedings of the IEEE · doi:10.1109/jproc.2025.3593952 · Cited by 20
Composing General Audio Representation by Fusing Multilayer Features of a Pre-trained Model
Niizumi, D., Takeuchi, D., Ohishi, Y., Harada, N., & Kashino, K. (2022) · European Signal Processing Conference (EUSIPCO) · doi:10.23919/eusipco55093.2022.9909674 · Cited by 15
A Dataset for Audio-Visual Sound Event Detection in Movies
Hebbar, R., Bose, D., Somandepalli, K., Vijai, V., & Narayanan, S. S. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10094781 · Cited by 16
A differentiable short-time Fourier transform with respect to the window length
Leiber, M., Barrau, A., Marnissi, Y., & Abboud, D. (2022) · European Signal Processing Conference (EUSIPCO) · doi:10.23919/eusipco55093.2022.9909963 · Cited by 17
Multimodal recommender systems: A survey of representation, modeling, and optimization
Pan, Z., Pan, Z., Cai, F., & Chen, H. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103991 · Cited by 27
A Robust Pitch-Fusion Model for Speech Emotion Recognition in Tonal Languages
Thanh, P. V., Huyen, N. T. T., Quan, P. N., & Trang, N. T. T. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10448373 · Cited by 16
ALBERTA: ALgorithm-Based Error Resilience in Transformer Architectures
Liu, H., Singh, V., Filipiuk, M., & Hari, S. K. S. (2024) · IEEE Open Journal of the Computer Society · doi:10.1109/ojcs.2024.3400696 · Cited by 18
Auxiliary audio–textual modalities for better action recognition on vision-specific annotated videos
Alfasly, S., Lu, J. J., Chen, X., Yu, L., & Zou, Y. (2024) · Pattern Recognition · doi:10.1016/j.patcog.2024.110808 · Cited by 12
Wei, Y., Chen, W., Tuanmu, M., Lu, S.-S., & Shiao, M. (2024) · Ecological Informatics · doi:10.1016/j.ecoinf.2024.102927 · Cited by 14
Exploring Pre-trained General-purpose Audio Representations for Heart Murmur Detection
Niizumi, D., Takeuchi, D., Ohishi, Y., Harada, N., & Kashino, K. (2024) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc53108.2024.10782479 · Cited by 17
Temporal multi-modal knowledge graph generation for link prediction
Li, Y., Ji, H., Yu, F., Cheng, L., & Che, N. (2025) · Neural Networks · doi:10.1016/j.neunet.2024.107108 · Cited by 16
Resource-Efficient Federated Multimodal Learning via Layer-Wise and Progressive Training
Tun, Y. L., Thwal, C. M., Nguyen, M. N. H., & Zhang, C. (2025) · IEEE Internet of Things Journal · doi:10.1109/jiot.2025.3554541 · Cited by 12
Question-Aware Gaussian Experts for Audio-Visual Question Answering
Kim, H., Jung, I., Suh, D., Zhang, Y., Lee, S., & Hong, S. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01277 · Cited by 12
UniAlign: Scaling Multimodal Alignment within One Unified Model
Zhou, B., Li, L., Wang, Y., Liu, H., Yao, Y., & Wang, W. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02760 · Cited by 32
Music Classification Model Development Based on Audio Recognition using Transformer Model
Harryanto, A. A. A., Gunawan, K. W., Nagano, R., & Sutoyo, R. (2022) · International Conference on Artificial Intelligence and Data Sciences (AiDAS) · doi:10.1109/aidas56890.2022.9918787 · Cited by 11
Improved Zero-Shot Audio Tagging & Classification with Patchout Spectrogram Transformers
Primus, P., & Widmer, G. (2022) · European Signal Processing Conference (EUSIPCO) · doi:10.23919/eusipco55093.2022.9909760 · Cited by 11
Human-Like Modulation Sensitivity Emerging through Optimization to Natural Sound Recognition
Koumura, T., Terashima, H., & Furukawa, S. (2023) · Journal of Neuroscience · doi:10.1523/jneurosci.2002-22.2023 · Cited by 14
Energy-Efficient Audio Processing at the Edge for Biologging Applications
Miquel, J., Latorre, L., & Chamaillé‐Jammes, S. (2023) · Journal of Low Power Electronics and Applications · doi:10.3390/jlpea13020030 · Cited by 12
Guided Deep Embedded Clustering Regularization for Multifeature Medical Signal Classification
Vindas, Y., Roux, E., Guépié, B. K., Almar, M., & Delachartre, P. (2023) · Pattern Recognition · doi:10.1016/j.patcog.2023.109812 · Cited by 14
DeepLabV3+ Vision Transformer for Visual Bird Sound Denoising
Li, J., Wang, P., & Zhang, Y. (2023) · IEEE Access · doi:10.1109/access.2023.3294476 · Cited by 12
Transformers and Visual Transformers
Courant, R., Edberg, M., Dufour, N., & Kalogeiton, V. S. (2023) · Neuromethods · doi:10.1007/978-1-0716-3195-9_6 · Cited by 14
Chen, Y., Yan, Z., Zhu, Y., Ren, Z. X., Shen, J., & Huang, Y. (2023) · Lecture notes in computer science · doi:10.1007/978-981-99-4742-3_23 · Cited by 18
A High Accuracy and Low Power CNN-Based Environmental Sound Classification Processor
Peng, L., Yang, J., Chen, Z., Yan, L., Jiao, X., Xiao, J., Zhou, L., Chang, L., Long, Y., & Zhou, J. (2023) · IEEE Transactions on Circuits and Systems I Regular Papers · doi:10.1109/tcsi.2023.3299823 · Cited by 14
Study of Positional Encoding Approaches for Audio Spectrogram Transformers
Pepino, L., Riera, P., & Ferrer, L. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747742 · Cited by 14
Cough Classification Using Audio Spectrogram Transformer
Habashy, K., Valdés, J. J., Cohen-McFarlane, M., Xi, P., Wallace, B., Goubran, R. A., & Knoefel, F. D. (2022) · IEEE Sensors Applications Symposium (SAS) · doi:10.1109/sas54819.2022.9881344 · Cited by 13
Deep Neural Decision Forest for Acoustic Scene Classification
Sun, J., Liu, X., Mei, X., Zhao, J., Plumbley, M. D., Kılıç, V., & Wang, W. (2022) · European Signal Processing Conference (EUSIPCO) · doi:10.23919/eusipco55093.2022.9909575 · Cited by 12
MM-AU:Towards Multimodal Understanding of Advertisement Videos
Bose, D., Hebbar, R., Feng, T., Somandepalli, K., Xu, A., & Narayanan, S. S. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612371 · Cited by 17
Wu, C., Huang, D., Tao, X., Qiao, K., Lu, H., & Wang, W. (2023) · IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI ...) · doi:10.1109/bhi58575.2023.10313454 · Cited by 9
Multimodal Environmental Sensing Using AI & IoT Solutions: A Cognitive Sound Analysis Perspective
Emvoliadis, A., Vryzas, N., Stamatiadou, M. E., Vrysis, L., & Dimoulas, C. A. (2024) · Sensors · doi:10.3390/s24092755 · Cited by 13
Do Multimodal Large Language Models and Humans Ground Language Similarly?
Jones, C. R., Bergen, B. K., & Trott, S. (2024) · Computational Linguistics · doi:10.1162/coli_a_00531 · Cited by 23
Movie trailer genre classification using multimodal pretrained features
Sulun, S., Viana, P., & Davies, M. E. P. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.125209 · Cited by 16
Parameter-Efficient Transfer Learning of Audio Spectrogram Transformers
Cappellazzo, U., Falavigna, D., Brutti, A., & Ravanelli, M. (2024) · Machine learning for signal processing ... · doi:10.1109/mlsp58920.2024.10734776 · Cited by 19
Zheng, Y., Zhang, R., Atito, S., Yang, S., Wang, W., & Mei, Y. (2025) · Digital Signal Processing · doi:10.1016/j.dsp.2025.105055 · Cited by 9
A Flexible Template for Edge Generative AI With High-Accuracy Accelerated Softmax and GELU
Belano, A., Tortorella, Y., Garofalo, A., Benini, L., Rossi, D., & Conti, F. (2025) · IEEE Journal on Emerging and Selected Topics in Circuits and Systems · doi:10.1109/jetcas.2025.3562734 · Cited by 19
A Comparative Study of Deep Audio Models for Spectrogram- and Waveform-Based SingFake Detection
Nguyen-Duc, M., Nguyen, L. V., Nguyen-Ho-Nhat, H., Nguyen, T., & Lee, O. (2025) · IEEE Access · doi:10.1109/access.2025.3571728 · Cited by 12
Márquez‐Rodríguez, A., Mohedano-Muñoz, M. A., Marín-Jiménez, M. J., Santamaría-García, E., Bastianelli, G., Jordano, P., & Mendoza, I. (2025) · Ecological Informatics · doi:10.1016/j.ecoinf.2025.103254 · Cited by 15
Chen, Q., Lu, D., Feng, W., Ye, C., Qiu, L., Pan, J., & Li, J. (2025) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2025.3584138 · Cited by 14
Hu, J., Cao, Y. P., Wu, M., Kang, F., Yang, F., Wang, W., Plumbley, M. D., & Yang, J. J. (2025) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2025.3587446 · Cited by 34
Shen, Y., Cao, G., Dong, H.-Y., Dong, B., & Lee, C.-M. (2025) · Applied Sciences · doi:10.3390/app15158413 · Cited by 11
Jabbar, A., Huang, J., Jabbar, M. K., & Mahmood, T. (2025) · Results in Engineering · doi:10.1016/j.rineng.2025.107203 · Cited by 10
Connecting the Dots in Self-Supervised Learning: A Brief Survey for Beginners
Fang, P., Li, X., Yang, Y., Zhang, S., Kang, Q.-Y., Li, X., & Lan, Z. (2022) · Journal of Computer Science and Technology · doi:10.1007/s11390-022-2158-x · Cited by 11
Avoid Overfitting User Specific Information in Federated Keyword Spotting
Li, X., Tang, J.-L., Song, S., Li, B., Li, Y., Shao, Y., Gan, L., & Zhan, D. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-558 · Cited by 11
Hierarchical Multi-modal Transformer for Automatic Detection of COVID-19
Tang, S., Hu, X., Atlas, L., Khanzada, A., & Pilancı, M. (2022) · doi:10.1145/3556384.3556414 · Cited by 12
Bella, V., & Sanjaya, S. A. (2023) · International Conference on New Media Studies (CONMEDIA) · doi:10.1109/conmedia60526.2023.10428158 · Cited by 9
Lee, M., Lee, H., Kim, B., & Kim, S.-H. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00298 · Cited by 14
Paim, K. O., Rohweder, R., Recamonde‐Mendoza, M., Mansilha, R. B., & Cordeiro, W. L. D. C. (2024) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2024.106342 · Cited by 10
Traffic Intensity Detection Using General-Purpose Sensing
Myat, A. K., Minerva, R., Taparugssanagorn, A., Rajapaksha, P., & Crespi, N. (2023) · IEEE Sensors Letters · doi:10.1109/lsens.2023.3315251 · Cited by 9
Transformer-Based Network for Accurate Classification of Lung Auscultation Sounds
Sonali, C. S., Kiran, J., Chinmayi, B. S., Suma, K. V., & Easa, M. (2023) · Critical Reviews in Biomedical Engineering · doi:10.1615/critrevbiomedeng.2023048981 · Cited by 8
Cross-Modal Perception for Customer Service
Maj, M., Rymarczyk, T., Maciura, Ł., Cieplak, T., & Pliszczuk, D. (2023) · ACM Annual International Conference on Mobile Computing and Networking (MobiCom) · doi:10.1145/3570361.3615751 · Cited by 8
Assessing Audio-Based Transformer Models for Speech Emotion Recognition
Bayraktar, Ü., Kilimci, H., Kılınç, H. H., & Kilimci, Z. H. (2023) · International Symposium on Innovative Approaches in Smart Technologies (ISAS) · doi:10.1109/isas60782.2023.10391313 · Cited by 13
Audio-Visual Action Recognition Using Transformer Fusion Network
Kim, J.-H., & Won, C. S. (2024) · Applied Sciences · doi:10.3390/app14031190 · Cited by 14
Wu, F., Yao, H., Zhongda, Z., Zhao, X., Zang, Y., & Wang, H. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3520860 · Cited by 16
Acosta, L. H., & Reinhardt, D. (2025) · Computers & Security · doi:10.1016/j.cose.2024.104302 · Cited by 9
EEG-Based Multimodal Representation Learning for Emotion Recognition
Yin, K., Shin, H.-B., Li, D., & Lee, S.-W. (2025) · International Winter Conference on Brain-Computer Interface (BCI) · doi:10.1109/bci65088.2025.10931743 · Cited by 20
Saradopoulos, I. V., Potamitis, I., Ntalampiras, S., Rigakis, I. I., Manifavas, C., & Konstantaras, A. J. (2025) · Sensors · doi:10.3390/s25082597 · Cited by 15
Geldenhuys, C. M., & Niesler, T. (2025) · Bioacoustics · doi:10.1080/09524622.2025.2487099 · Cited by 14
Zuo, R., Zheng, C., Zhu, L., Lü, W., Si, J., & Zhang, W. (2025) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2025.3565973 · Cited by 8
EchoTraffic: Enhancing Traffic Anomaly Understanding with Audio-Visual Insights
Xing, Z., Chen, H., Xie, B., Xu, J., Guo, Z., Xu, X., Hao, J., Fu, C., Hu, X., & Heng, P. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01779 · Cited by 18
Metadata
- Authors
- Gong, Y., Chung, Y.-A., & Glass, J.
- Affiliations
- MIT Computer Science and Artificial Intelligence Laboratory (US)
- Venue
- Interspeech 2021
- Publisher
- ISCA
- Published
- 2021-08-30
- Pages
- 571-575
- ISSN
- 2308-457X (ISSN-L)
- OpenAlex
- W3196974791
- Cited by
- 1,159
- References
- 33
- Citations by year
- 2026: 260 · 2025: 356 · 2024: 241 · 2023: 193 · 2022: 100 · 2021: 9
- FWCI
- 82.22
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Interspeech (USB) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Signal Processing › Music and Audio Processing
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Keywords
- Audio Spectrogram Transformer, audio classification, self-attention, spectrogram, AudioSet, ESC-50
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.