SUPERB: Speech Processing Universal PERformance Benchmark
The DOI matches its Crossref record; 41 of the 41 references it cites were found in a registry or an index.
Abstract
Self-supervised learning (SSL) has proven vital for advancing research in natural language processing (NLP) and computer vision (CV).The paradigm pretrains a shared model on large volumes of unlabeled data and achieves state-of-the-art (SOTA) for various tasks with minimal adaptation.However, the speech processing community lacks a similar setup to systematically explore the paradigm.To bridge this gap, we introduce Speech processing Universal PERformance Benchmark (SUPERB).SUPERB is a leaderboard to benchmark the performance of a shared model across a wide range of speech processing tasks with minimal architecture changes and labeled data.Among multiple usages of the shared model, we especially focus on extracting the representation learned from SSL for its preferable re-usability.We present a simple framework to solve SUPERB tasks by learning task-specialized lightweight prediction heads on top of the frozen shared model.Our results demonstrate that the framework is promising as SSL representations show competitive generalizability and accessibility across SUPERB tasks.We release SUPERB as a challenge with a leaderboard 1 and a benchmark toolkit 2 to fuel the research in representation learning and general speech processing.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“According to [14], the authors have also indicated that by weighting the representations of different layers, self-supervised model has shown great potential in various speech downstream tasks.”
From Audio Deepfake Detection With Self-Supervised WavLM And Multi-Fusion Attentive Classifier (2024) · cites this paper as [14]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Self-citationBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Self-citationDeep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
Self-citationRepresentation Learning with Contrastive Predictive Coding
Oord, A. V. D., Li, Y., & Vinyals, O. (2018) · arXiv · doi:10.48550/arxiv.1807.03748 · Cited by 6,242
“Librispeech: an ASR corpus based on public domain audio books,”
Panayotov, V., Chen, G., Povey, D., & Khudanpur, S. (2015) · 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2015.7178964 · Cited by 6,072
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., & Bowman, S. (2018) · Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP · doi:10.18653/v1/w18-5446 · Cited by 4,222
IEMOCAP: Interactive emotional dyadic motion capture database
Busso, C., Bulut, M., Lee, C.-C., Kazemzadeh, A., Mower, E., Kim, S., Chang, J. N., Lee, S., & Narayanan, S. S. (2008) · Language Resources and Evaluation · doi:10.1007/s10579-008-9076-6 · Cited by 3,869
HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units
Hsu, W.-N., Bolte, B., Tsai, Y.-H. H., Lakhotia, K., Salakhutdinov, R., & Mohamed, A. (2021) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2021.3122291 · Cited by 2,983
Self-citationDeep Clustering for Unsupervised Learning of Visual Features
Caron, M., Bojanowski, P., Joulin, A., & Douze, M. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01264-9_9 · Cited by 2,674
X-Vectors: Robust DNN Embeddings for Speaker Recognition
Snyder, D., Garcia-Romero, D., Sell, G., Povey, D., & Khudanpur, S. (2018) · 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2018.8461375 · Cited by 2,620
Research works printed but not linked: 2
- [13][13] A. Baevski, S. Schneider, and M. Auli, 'vq-wav2vec: Selfsupervised learning of discrete speech representations,' in ICLR , 2020. -
- [25][25] X. Anguera, L. Rodriguez-Fuentes, A. Buzo, F. Metze, I. Sz¨ oke, and M. Penagarikano, 'Quesst2014: Evaluating query-byexample speech search in a zero-resource setting with real-life queries,' in ICASSP , 2015, pp. 5833-5837. -
Cited by
WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing
Chen, S., Wang, C., Chen, Z., Wu, Y., Liu, S., Chen, Z., Li, J., Kanda, N., Yoshioka, T., Xiao, X., et al. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3188113 · Cited by 1,926
XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale
Babu, A., Wang, C., Tjandra, A., Lakhotia, K., Xu, Q., Goyal, N., Singh, K., von Platen, P., Saraf, Y., Pino, J., et al. (2022) · Interspeech 2022 · doi:10.21437/interspeech.2022-143 · Cited by 586
Self-citationSpeechBrain: A General-Purpose Speech Toolkit
Ravanelli, M., Parcollet, T., Plantinga, P., Rouhe, A., Cornell, S., Lugosch, L., Subakan, C., Dawalatabad, N., Heba, A., Zhong, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2106.04624 · Cited by 512
Dawn of the Transformer Era in Speech Emotion Recognition: Closing the Valence Gap
Wagner, J., Triantafyllopoulos, A., Wierstorf, H., Schmitt, M., Burkhardt, F., Eyben, F., & Schuller, B. W. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3263585 · Cited by 395
A review of deep learning techniques for speech processing
Mehrish, A., Majumder, N., Bharadwaj, R., Mihalcea, R., & Poria, S. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101869 · Cited by 389
Self-supervised speech representation learning: A review
Mohamed, A., Lee, H.-Y., Borgholt, L., Havtorn, J. D., Edin, J., Igel, C., Kirchhoff, K., Li, S.-W., Livescu, K., Maaløe, L., et al. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3207050 · Cited by 372
“The WavLM model achieved substantial improvements on the speech separation, speaker verification and speaker diarization tasks in the SUPERB benchmark, while also performing well on many other tasks compared to HuBERT and wav2vec 2.0. d) data2vec: Motivated by the success of using an exponential moving average (EMA) t…”
BackgroundMethodInfluentialSelf-citationSsast: Self-supervised audio spectrogram transformer
Gong, Y., Lai, C.-I., Chung, Y.-A., & Glass, J. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i10.21315 · Cited by 294
Self-citationLayer-Wise Analysis of a Self-Supervised Speech Representation Model
Pasad, A., Chou, J.-C., & Livescu, K. (2021) · 2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) · doi:10.1109/asru51503.2021.9688093 · Cited by 262
Xu, C., Cao, B. T., Yuan, Y., & Meschke, G. (2023) · Computer Methods in Applied Mechanics and Engineering · doi:10.1016/j.cma.2022.115852 · Cited by 255
OpenFL: the open federated learning library
Foley, P., Sheller, M. J., Edwards, B., Pati, S., Riviera, W., Sharma, M., Narayana Moorthy, P., Wang, S.-H., Martin, J., Mirhaji, P., et al. (2022) · Physics in Medicine & Biology · doi:10.1088/1361-6560/ac97d9 · Cited by 132
SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processing
Ao, J., Wang, R., Zhou, L., Wang, C., Ren, S., Wu, Y., Liu, S., Ko, T., Li, Q., Zhang, Y., et al. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.393 · Cited by 144
Speech Emotion Recognition Using Self-Supervised Features
Morais, E. D. S., Hoory, R., Zhu, W., Gat, I., Damasceno, M., & Aronowitz, H. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747870 · Cited by 204
Generalization Ability of MOS Prediction Networks
Cooper, E., Huang, W.-C., Toda, T., & Yamagishi, J. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746395 · Cited by 233
Investigating Self-Supervised Front Ends for Speech Spoofing Countermeasures
Wang, X., & Yamagishi, J. (2022) · The Speaker and Language Recognition Workshop (Odyssey 2022) · doi:10.21437/odyssey.2022-14 · Cited by 138
Wang, Y., Boumadane, A., & Heba, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.02735 · Cited by 93
BEATs: Audio Pre-Training with Acoustic Tokenizers
Chen, S., Wu, Y., Wang, C., Liu, S., Tompkins, D., Chen, Z., & Wei, F. (2022) · arXiv · doi:10.48550/arxiv.2212.09058 · Cited by 131
Distilhubert: Speech Representation Learning by Layer-Wise Distillation of Hidden-Unit Bert
Chang, H.-J., Yang, S.-W., & Lee, H.-Y. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747490 · Cited by 246
Self-citationDirect Speech-to-Speech Translation With Discrete Units
Lee, A., Chen, P., Wang, C., Gu, J., Popuri, S., Ma, X., Polyak, A., Adi, Y., He, Q., Tang, Y., et al. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.235 · Cited by 226
Large-Scale Self-Supervised Speech Representation Learning for Automatic Speaker Verification
Chen, Z., Chen, S., Wu, Y., Qian, Y., Wang, C., Liu, S., Qian, Y., & Zeng, M. (2022) · ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp43922.2022.9747814 · Cited by 112
A Comparison of Discrete and Soft Speech Units for Improved Voice Conversion
van Niekerk, B., Carbonneau, M., Zaïdi, J., Baas, M., Seuté, H., & Kamper, H. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746484 · Cited by 189
Comparative Layer-Wise Analysis of Self-Supervised Speech Models
Pasad, A., Shi, B., & Livescu, K. (2023) · ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp49357.2023.10096149 · Cited by 105
Vesper: A Compact and Effective Pretrained Model for Speech Emotion Recognition
Chen, W., Xing, X., Chen, P., & Xu, X. (2024) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2024.3369726 · Cited by 135
Zhang, L., Wang, X., Cooper, E., Evans, N., & Yamagishi, J. (2023) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2022.3233236 · Cited by 100
MAE-AST: Masked Autoencoding Audio Spectrogram Transformer
Baade, A., Peng, P., & Harwath, D. F. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-10961 · Cited by 169
SpeechFormer++: A Hierarchical Efficient Framework for Paralinguistic Speech Processing
Chen, W., Xing, X., Xu, X., Pang, J., & Du, L. (2023) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2023.3235194 · Cited by 117
Audio Deepfake Detection With Self-Supervised WavLM And Multi-Fusion Attentive Classifier
Guo, Y., Huang, H., Chen, X., Zhao, H., & Wang, Y. (2024) · ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp48485.2024.10447923 · Cited by 95
Unispeech-Sat: Universal Speech Representation Learning With Speaker Aware Pre-Training
Chen, S., Wu, Y., Wang, C., Chen, Z., Chen, Z., Liu, S., Wu, J., Qian, Y., Wei, F., Li, J., et al. (2022) · ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp43922.2022.9747077 · Cited by 85
Tsai, H.-S., Chang, H.-J., Huang, W.-C., Huang, Z., Lakhotia, K., Yang, S., Dong, S., Liu, A. T., Lai, C.-I., Shi, J., et al. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.580 · Cited by 149
Self-citationZhao, J. S., & Zhang, W.-Q. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3184480 · Cited by 127
Jiang, Z., Seyedi, S. S., Griner, E. L., Abbasi, A., Rad, A. B., Kwon, H., Cotes, R. O., & Clifford, G. D. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3352075 · Cited by 103
AIR-Bench: Benchmarking Large Audio-Language Models via Generative Comprehension
Yang, Q., Xu, J., Liu, W., Chu, Y., Jiang, Z. K., Zhou, X., Leng, Y., Lv, Y., Zhao, Z., Zhou, C. J., et al. (2024) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2024.acl-long.109 · Cited by 368
Padi, S., Sadjadi, S. O., Manocha, D., & Sriram, R. D. (2022) · The Speaker and Language Recognition Workshop (Odyssey) · doi:10.21437/odyssey.2022-57 · Cited by 88
Speaker Normalization for Self-Supervised Speech Emotion Recognition
Gat, I., Aronowitz, H., Zhu, W., Morais, E. D. S., & Hoory, R. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747460 · Cited by 86
BYOL for Audio: Exploring Pre-Trained General-Purpose Audio Representations
Niizumi, D., Takeuchi, D., Ohishi, Y., Harada, N., & Kashino, K. (2022) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2022.3221007 · Cited by 120
Transformers in speech processing: Overcoming challenges and paving the future
Latif, S., Zaidi, S. A. M., Cuayáhuitl, H., Shamshad, F., Shoukat, M., Usama, M., & Qadir, J. (2025) · Computer Science Review · doi:10.1016/j.cosrev.2025.100768 · Cited by 126
Investigating Self-Supervised Learning for Speech Enhancement and Separation
Huang, Z., Watanabe, S., Yang, S.-W., Garcia, L. P., & Khudanpur, S. P. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746303 · Cited by 105
Self-citationGenerative Spoken Dialogue Language Modeling
Nguyen, T. A., Kharitonov, E., Copet, J., Adi, Y., Hsu, W.-N., Elkahky, A. M., Tomasello, P., Algayres, R., Sagot, B., Mohamed, A., et al. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00545 · Cited by 257
Self-citationThe CHiME-7 DASR Challenge: Distant Meeting Transcription with Multiple Devices in Diverse Scenarios
Cornell, S., Wiesner, M., Watanabe, S., Raj, D., Chang, X., Garcia, L. P., Masuyam, Y., Wang, Z.-Q., Squartini, S., & Khudanpur, S. P. (2023) · International Workshop on Speech Processing in Everyday Environments (CHiME) · doi:10.21437/chime.2023-1 · Cited by 109
Self-citationFake Audio Detection Based On Unsupervised Pretraining Models
Lv, Z., Zhang, S., Tang, K., & Hu, P. (2022) · ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp43922.2022.9747605 · Cited by 56
Sentiment Analysis and Emotion Recognition from Speech Using Universal Speech Representations
Atmaja, B. T., & Sasou, A. (2022) · Sensors · doi:10.3390/s22176369 · Cited by 76
Speech emotion recognition with light weight deep neural ensemble model using hand crafted features
Chowdhury, J. H., Ramanna, S., & Kotecha, K. V. (2025) · Scientific Reports · doi:10.1038/s41598-025-95734-z · Cited by 71
An Exploration of Self-Supervised Pretrained Representations for End-to-End Speech Recognition
Chang, X., Maekaku, T., Guo, P., Shi, J. Y., Lu, Y.-J., Subramanian, A. S., Wang, T., Yang, S.-W., Tsao, Y., Lee, H.-Y., et al. (2021) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru51503.2021.9688137 · Cited by 101
Self-citationAutomatic Voice Disorder Detection Using Self-Supervised Representations
Ribas, D., Pastor, M. A., Miguel, A., Martinez, D., Ortega, A., & Lleida, E. (2023) · IEEE Access · doi:10.1109/access.2023.3243986 · Cited by 52
SLUE: New Benchmark Tasks For Spoken Language Understanding Evaluation on Natural Speech
Shon, S., Pasad, A., Wu, F., Brusco, P., Artzi, Y., Livescu, K., & Han, K. J. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746137 · Cited by 121
Bias in Automated Speaker Recognition
Hutiri, W. T., & Ding, A. Y. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533089 · Cited by 47
SpeechLM: Enhanced Speech Pre-Training With Unpaired Textual Data
Zhang, Z., Chen, S., Zhou, L., Wu, Y., Ren, S., Liu, S., Yao, Z., Gong, X., Dai, L.-R., Li, J., et al. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3379877 · Cited by 90
Text-Free Prosody-Aware Generative Spoken Language Modeling
Kharitonov, E., Lee, A., Polyak, A., Adi, Y., Copet, J., Lakhotia, K., Nguyen, T. A., Rivière, M., Mohamed, A., Dupoux, E., et al. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.593 · Cited by 147
Self-citationFavaro, A., Tsai, Y.-T., Butala, A., Thebaud, T., Villalba, J., Dehak, N., & Moro-Velázquez, L. (2023) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2023.107559 · Cited by 80
The Singing Voice Conversion Challenge 2023
Huang, W.-C., Violeta, L. P., Liu, S., Shi, J., & Toda, T. (2023) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru57964.2023.10389671 · Cited by 97
Self-citationThe Codecfake Dataset and Countermeasures for the Universally Detection of Deepfake Audio
Xie, Y., Lu, Y., Fu, R., Wen, Z., Wang, Z., Tao, J., Qi, X., Wang, X., Liu, Y., Cheng, H., et al. (2025) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2025.3525966 · Cited by 111
A Survey of Reasoning with Foundation ModelsPreprint
Sun, J., Zheng, C., Xie, E., Liu, Z., Chu, R., Liu, J., Xu, J., Ding, M., Li, H., Geng, M., et al. (2023) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/ac4sp · Cited by 157
LEXTREME: A Multi-Lingual and Multi-Task Benchmark for the Legal Domain
Niklaus, J., Matoshi, V., Rani, P., Galassi, A., Stürmer, M., & Chalkidis, I. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.200 · Cited by 50
ESPnet-SLU: Advancing Spoken Language Understanding Through ESPnet
Arora, S., Dalmia, S., Denisov, P., Chang, X., Ueda, Y., Peng, Y., Zhang, Y., Kumar, S., Ganesan, K., Yan, B., et al. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747674 · Cited by 86
Self-citationMFGCN: Multimodal fusion graph convolutional network for speech emotion recognition
Qi, X., Wen, Y., Zhang, P., & Huang, H. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128646 · Cited by 61
Towards Learning Universal Audio Representations
Wang, L., Luc, P., Wu, Y., Recasens, A., Smaira, L., Brock, A., Jaegle, A., Alayrac, J.-B., Dieleman, S., Carreira, J., et al. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746790 · Cited by 105
A Large-Scale Evaluation of Speech Foundation Models
Yang, S.-W., Chang, H.-J., Huang, Z., Liu, A. T., Lai, C.-I., Wu, H., Shi, J., Chang, X., Tsai, H.-S., Huang, W.-C., et al. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3389631 · Cited by 102
Self-citationAutomatic Pronunciation Assessment using Self-Supervised Speech Representation Learning
Kim, E., Jeon, J.-J., Seo, H., & Kim, H. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-10245 · Cited by 99
Expresso: A Benchmark and Analysis of Discrete Expressive Speech Resynthesis
Nguyen, T. A., Hsu, W.-N., D’Avirro, A., Shi, B., Gat, I., Fazel-Zarani, M., Remez, T., Copet, J., Synnaeve, G., Hassid, M., et al. (2023) · Interspeech (USB) · doi:10.21437/interspeech.2023-1905 · Cited by 170
Alhussein, G., Ziogas, I., Saleem, S., & Hadjileontiadis, L. J. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11197-8 · Cited by 63
Evaluating Self-Supervised Speech Representations for Speech Emotion Recognition
Atmaja, B. T., & Sasou, A. (2022) · IEEE Access · doi:10.1109/access.2022.3225198 · Cited by 77
Self-Supervised Representations in Speech-Based Depression Detection
Wu, W., Zhang, C., & Woodland, P. C. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10094910 · Cited by 80
Effects of Data Augmentations on Speech Emotion Recognition
Atmaja, B. T., & Sasou, A. (2022) · Sensors · doi:10.3390/s22165941 · Cited by 55
Towards Afrocentric NLP for African Languages: Where We Are and Where We Can Go
Adebara, I., & Abdul-Mageed, M. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.265 · Cited by 101
Reproducing Whisper-Style Training Using An Open-Source Toolkit And Publicly Available Data
Peng, Y., Tian, J., Yan, B., Berrebbi, D., Chang, X., Li, X., Shi, J., Arora, S., Chen, W., Sharma, R., et al. (2023) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru57964.2023.10389676 · Cited by 136
Self-citationDeep neural networks for automatic speech processing: a survey from large corpora to limited data
Roger, V., Farinas, J., & Pinquier, J. (2022) · EURASIP Journal on Audio Speech and Music Processing · doi:10.1186/s13636-022-00251-w · Cited by 66
Feng, T., Peri, R., & Narayanan, S. S. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-10060 · Cited by 64
BEANS: The Benchmark of Animal Sounds
Hagiwara, M., Hoffman, B., Liu, J.-Y., Cusimano, M., Effenberger, F., & Zacarian, K. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10096686 · Cited by 102
Enhancing accuracy and privacy in speech-based depression detection through speaker disentanglement
Ravi, V., Wang, J., Flint, J., & Alwan, A. A. (2023) · Computer Speech & Language · doi:10.1016/j.csl.2023.101605 · Cited by 65
Boosting Self-Supervised Embeddings for Speech Enhancement
Hung, K.-H., Fu, S., Tseng, H.-H., Chiang, H.-T., Tsao, Y., & Lin, C. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-10002 · Cited by 55
Srinivasan, S., Huang, Z., & Kirchhoff, K. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747754 · Cited by 57
Self-citationSchu, G. G., Janbakhshi, P., & Kodrasi, I. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10095981 · Cited by 68
PARP: Prune, Adjust and Re-Prune for Self-Supervised Speech RecognitionPreprint
Lai, C.-I., Zhang, Y., Liu, A. H., Chang, S., Liao, Y.-L., Chuang, Y.-S., Qian, K., Khurana, S., Cox, D., & Glass, J. R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.05933 · Cited by 82
Self-citationWang, R., Bai, Q., Ao, J., Zhou, L., Xiong, Z., Wei, Z., Zhang, Y., Ko, T., & Li, H. (2022) · Interspeech 2022 · doi:10.21437/interspeech.2022-10269 · Cited by 40
AVES: Animal Vocalization Encoder Based on Self-Supervision
Hagiwara, M. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10095642 · Cited by 128
A Survey on Speech Large Language Models for Understanding
Peng, J., Wang, Y., Li, B., Guo, Y., Wang, H., Fang, Y., Xi, Y., Li, H., Li, X., Zhang, K., et al. (2025) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2025.3640535 · Cited by 153
Exploration of a Self-Supervised Speech Model: A Study on Emotional Corpora
Li, Y., Mohamied, Y., Bell, P., & Lai, C. (2023) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt54892.2023.10023428 · Cited by 73
Self-citationPeng, J., Plchot, O., Stafylakis, T., Mošner, L., Burget, L., & Černocký, J. (2023) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt54892.2023.10022775 · Cited by 82
Structured Pruning of Self-Supervised Pre-Trained Models for Speech Recognition and Understanding
Peng, Y., Kim, K., Wu, F., Sridhar, P., & Watanabe, S. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10095780 · Cited by 63
Self-citationMasked Modeling Duo: Towards a Universal Audio Pre-Training Framework
Niizumi, D., Takeuchi, D., Ohishi, Y., Harada, N., & Kashino, K. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3389636 · Cited by 93
Zhu, Q., Zhou, L., Zhang, Z., Liu, S., Jiao, B., Zhang, J., Dai, L.-R., Jiang, D., Li, J., & Wei, F. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3275873 · Cited by 68
Maiti, S., Ueda, Y., Watanabe, S., Zhang, C., Yu, M., Zhang, S.-X., & Xu, Y. (2023) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt54892.2023.10022924 · Cited by 62
Self-citationEmotion Neural Transducer for Fine-Grained Speech Emotion Recognition
Shen, S., Gao, Y. M., Liu, F., Wang, H., & Zhou, A. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10446974 · Cited by 53
What Do Self-Supervised Speech Models Know About Words?
Pasad, A., Chien, C.-M., Settle, S., & Livescu, K. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00656 · Cited by 104
Investigating Self-supervised Pretraining Frameworks for Pathological Speech Recognition
Violeta, L. P., Huang, W. C., & Toda, T. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-10043 · Cited by 55
Wav2Seq: Pre-Training Speech-to-Text Encoder-Decoder Models Using Pseudo Languages
Wu, F., Kim, K., Watanabe, S., Han, K. J., McDonald, R., Weinberger, K. Q., & Artzi, Y. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10096988 · Cited by 64
Self-citationNaderi, N., & Nasersharif, B. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.110814 · Cited by 42
Huang, C., Lu, K.-H., Wang, S.-H., Hsiao, C.-Y., Kuan, C.-Y., Wu, H., Arora, S., Chang, K., Shi, J., Peng, Y., et al. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10448257 · Cited by 143
Self-citationIGLUE: A Benchmark for Transfer Learning across Modalities, Tasks, and Languages
Bugliarello, E., Liu, F., Pfeiffer, J., Reddy, S., Elliott, D., Ponti, E. M., & Vulić, I. (2022) · TUbilio (Technical University of Darmstadt) · doi:10.48550/arxiv.2201.11732 · Cited by 89
Universal Paralinguistic Speech Representations Using self-Supervised Conformers
Shor, J., Jansen, A., Han, W., Park, D., & Zhang, Y. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747197 · Cited by 61
Designing and Evaluating Speech Emotion Recognition Systems: A Reality Check Case Study with IEMOCAP
Antoniou, N., Katsamanis, A., Γιαννακόπουλος, Θ., & Narayanan, S. S. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10096808 · Cited by 56
Hwang, J., Hira, M., Chen, C., Zhang, X., Ni, Z., Sun, G., Ma, P., Huang, R., Pratap, V., Zhang, Y., et al. (2023) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru57964.2023.10389648 · Cited by 83
Self-citationWhy does Self-Supervised Learning for Speech Recognition Benefit Speaker Recognition?
Chen, S., Wu, Y., Wang, C., Liu, S., Chen, Z., Wang, P., Liu, G., Li, J., Wu, J., Yu, X., et al. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-10019 · Cited by 58
Wang, S., Chen, Z., Lee, K. A., Qian, Y., & Li, H. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3492793 · Cited by 53
Self-citationIndicSUPERB: A Speech Processing Universal Performance Benchmark for Indian Languages
Javed, T., Bhogale, K. S., Raman, A., Kumar, P., Kunchukuttan, A., & Khapra, M. M. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i11.26521 · Cited by 110
S3PRL-VC: Open-Source Voice Conversion Framework with Self-Supervised Speech Representations
Huang, W.-C., Yang, S.-W., Hayashi, T., Lee, H.-Y., Watanabe, S., & Toda, T. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746430 · Cited by 45
Self-citationTowards Better Domain Adaptation for Self-Supervised Models: A Case Study of Child ASR
Fan, R., Zhu, Y., Wang, J., & Alwan, A. A. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3200910 · Cited by 65
On the Utility of Self-Supervised Models for Prosody-Related Tasks
Lin, G.-T., Feng, C.-L., Huang, W., Tseng, Y., Lin, T.-H., Li, C.-A., Lee, H.-Y., & Ward, N. (2023) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt54892.2023.10023234 · Cited by 77
Self-citationImproving Massively Multilingual ASR with Auxiliary CTC Objectives
Chen, W., Yan, B., Shi, J., Peng, Y., Maiti, S., & Watanabe, S. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10095326 · Cited by 55
Self-citationTextless Speech Emotion Conversion using Discrete & Decomposed Representations
Kreuk, F., Polyak, A., Copet, J., Kharitonov, E., Nguyen, T. A., Rivière, M., Hsu, W.-N., Mohamed, A., Dupoux, E., & Adi, Y. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.769 · Cited by 50
Self-citationLLM supervised Pre-training for Multimodal Emotion Recognition in Conversations
Dutta, S., & Ganapathy, S. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10889998 · Cited by 59
Metadata
- Authors
- Yang, S.-W., Chi, P.-H., Chuang, Y.-S., Lai, C.-I. J., Lakhotia, K., Lin, Y. Y., Liu, A. T., Shi, J., Chang, X., Lin, G.-T., Huang, T.-H., Tseng, W.-C., Lee, K.-T., Liu, D.-R., Huang, Z., Dong, S., Li, S.-W., Watanabe, S., Mohamed, A., & Lee, H.-Y.
- Affiliations
- National Taiwan University (TW)
- Meta (United States) (US)
- Johns Hopkins University (US)
- Carnegie Mellon University (US)
- Amazon (United States) (US)
- ORCID
- Venue
- Interspeech 2021
- Publisher
- ISCA
- Published
- 2021-08-30
- Pages
- 1194-1198
- ISSN
- 2308-457X (ISSN-L)
- OpenAlex
- W3197580070
- Cited by
- 681
- References
- 40
- Citations by year
- 2026: 67 · 2025: 138 · 2024: 157 · 2023: 205 · 2022: 108 · 2021: 6
- FWCI
- 54.87
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Interspeech (USB) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- self-supervised learning, speech processing, benchmark, shared models, representation learning, frozen pre-trained models, task generalization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.