SUPERB: Speech Processing Universal PERformance Benchmark
The DOI matches its Crossref record; 41 of the 41 references it cites were found in a registry or an index.
Abstract
Self-supervised learning (SSL) has proven vital for advancing research in natural language processing (NLP) and computer vision (CV).The paradigm pretrains a shared model on large volumes of unlabeled data and achieves state-of-the-art (SOTA) for various tasks with minimal adaptation.However, the speech processing community lacks a similar setup to systematically explore the paradigm.To bridge this gap, we introduce Speech processing Universal PERformance Benchmark (SUPERB).SUPERB is a leaderboard to benchmark the performance of a shared model across a wide range of speech processing tasks with minimal architecture changes and labeled data.Among multiple usages of the shared model, we especially focus on extracting the representation learned from SSL for its preferable re-usability.We present a simple framework to solve SUPERB tasks by learning task-specialized lightweight prediction heads on top of the frozen shared model.Our results demonstrate that the framework is promising as SSL representations show competitive generalizability and accessibility across SUPERB tasks.We release SUPERB as a challenge with a leaderboard 1 and a benchmark toolkit 2 to fuel the research in representation learning and general speech processing.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“According to [14], the authors have also indicated that by weighting the representations of different layers, self-supervised model has shown great potential in various speech downstream tasks.”
From Audio Deepfake Detection With Self-Supervised WavLM And Multi-Fusion Attentive Classifier (2024) · cites this paper as [14]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Self-citationBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Self-citationDeep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
Self-citationRepresentation Learning with Contrastive Predictive Coding
Oord, A. V. D., Li, Y., & Vinyals, O. (2018) · arXiv · doi:10.48550/arxiv.1807.03748 · Cited by 6,242
“Librispeech: an ASR corpus based on public domain audio books,”
Panayotov, V., Chen, G., Povey, D., & Khudanpur, S. (2015) · 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2015.7178964 · Cited by 6,072
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., & Bowman, S. (2018) · Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP · doi:10.18653/v1/w18-5446 · Cited by 4,222
IEMOCAP: Interactive emotional dyadic motion capture database
Busso, C., Bulut, M., Lee, C.-C., Kazemzadeh, A., Mower, E., Kim, S., Chang, J. N., Lee, S., & Narayanan, S. S. (2008) · Language Resources and Evaluation · doi:10.1007/s10579-008-9076-6 · Cited by 3,869
HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units
Hsu, W.-N., Bolte, B., Tsai, Y.-H. H., Lakhotia, K., Salakhutdinov, R., & Mohamed, A. (2021) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2021.3122291 · Cited by 2,983
Self-citationDeep Clustering for Unsupervised Learning of Visual Features
Caron, M., Bojanowski, P., Joulin, A., & Douze, M. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01264-9_9 · Cited by 2,674
X-Vectors: Robust DNN Embeddings for Speaker Recognition
Snyder, D., Garcia-Romero, D., Sell, G., Povey, D., & Khudanpur, S. (2018) · 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2018.8461375 · Cited by 2,620
Research works printed but not linked: 2
- [13][13] A. Baevski, S. Schneider, and M. Auli, 'vq-wav2vec: Selfsupervised learning of discrete speech representations,' in ICLR , 2020. -
- [25][25] X. Anguera, L. Rodriguez-Fuentes, A. Buzo, F. Metze, I. Sz¨ oke, and M. Penagarikano, 'Quesst2014: Evaluating query-byexample speech search in a zero-resource setting with real-life queries,' in ICASSP , 2015, pp. 5833-5837. -
Cited by
Items 101–200 of 756
Adapting Self-Supervised Models to Multi-Talker Speech Recognition Using Speaker Embeddings
Huang, Z., Raj, D., Garcia, L. P., & Khudanpur, S. P. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10097139 · Cited by 49
Self-citationSELM: Speech Enhancement using Discrete Tokens and Language Models
Wang, Z., Zhu, X., Zhang, Z., Lv, Y., Jiang, N., Zhao, G., & Xie, L. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10447464 · Cited by 95
Efficient Self-Supervised Learning Representations for Spoken Language Identification
Liu, H., Garcia, L. P., Khong, A. W. H., Chng, E. S., Styles, S. J., & Khudanpur, S. P. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3201445 · Cited by 40
Deep learning in marine bioacoustics: a benchmark for baleen whale detection
Schall, E., Kaya, İ. I., Debusschere, E., Devos, P., & Parcerisas, C. (2024) · Remote Sensing in Ecology and Conservation · doi:10.1002/rse2.392 · Cited by 49
Self-Supervised Language Learning From Raw Audio: Lessons From the Zero Resource Speech Challenge
Dunbar, E., Hamilakis, N., & Dupoux, E. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3206084 · Cited by 63
Exploring Efficient-Tuning Methods in Self-Supervised Speech Models
Chen, Z.-C., Fu, C.-L., Liu, C.-Y., Li, S.-W. D., & Lee, H.-Y. (2023) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt54892.2023.10023274 · Cited by 50
Self-citationZhu, Q., Zhou, L., Zhang, J., Liu, S., Hu, Y., & Dai, L.-R. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10095373 · Cited by 40
SLUE Phase-2: A Benchmark Suite of Diverse Spoken Language Understanding Tasks
Shon, S., Arora, S., Lin, C.-J., Pasad, A., Wu, F. F., Sharma, R. S., Wu, W.-L., Lee, H.-Y., Livescu, K., & Watanabe, S. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.496 · Cited by 77
Self-citationLanguage-Independent Speaker Anonymization Approach Using Self-Supervised Pre-Trained Models
Miao, X., Wang, X., Cooper, E., Yamagishi, J., & Tomashenko, N. (2022) · The Speaker and Language Recognition Workshop (Odyssey) · doi:10.21437/odyssey.2022-39 · Cited by 46
Feng, T., & Narayanan, S. S. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-141 · Cited by 36
Proficiency Assessment of L2 Spoken English Using Wav2Vec 2.0
Bannò, S., & Matassoni, M. (2023) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt54892.2023.10023019 · Cited by 56
Feng, T.-H., Dong, A., Yeh, C.-F., Yang, S.-W., Lin, T.-Q., Shi, J., Chang, K., Huang, Z., Wu, H., Chang, X., et al. (2023) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt54892.2023.10022770 · Cited by 41
Self-citationSpeechlmscore: Evaluating Speech Generation Using Speech Language Model
Maiti, S., Peng, Y., Saeki, T., & Watanabe, S. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10095710 · Cited by 56
Self-citationAnalysing Discrete Self Supervised Speech Representation For Spoken Language Modeling
Sicherman, A., & Adi, Y. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10097097 · Cited by 80
Hyeon, J., Oh, Y., Lee, Y., & Choi, H. (2023) · Data & Knowledge Engineering · doi:10.1016/j.datak.2023.102262 · Cited by 29
SpeechCLIP: Integrating Speech with Pre-Trained Vision and Language Model
Shih, Y.-J., Wang, H.-F., Chang, H.-J., Berry, L., Lee, H.-Y., & Harwath, D. F. (2023) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt54892.2023.10022954 · Cited by 54
Self-citationTowards Universal Speech Discrete Tokens: A Case Study for ASR and TTS
Yang, Y., Shen, F., Du, C., Ma, Z., Yu, K., Povey, D., & Chen, X. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10447751 · Cited by 49
Leveraging Self-Supervised Learning for Speaker Diarization
Han, J., Landini, F., Rohdin, J., Silnova, A., Diez, M., & Burget, L. (2025) · ICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp49660.2025.10889475 · Cited by 23
Multitask Transformer for Cross-Corpus Speech Emotion Recognition
Ahn, C.-S., Rana, R. K., Busso, C., & Rajapakse, J. C. (2025) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2025.3526592 · Cited by 26
Cross-Corpus Training Strategy for Speech Emotion Recognition Using Self-Supervised Representations
Pastor, M. A., Ribas, D., Ortega, A., Miguel, A., & Lleida, E. (2023) · Applied Sciences · doi:10.3390/app13169062 · Cited by 26
EAT: Self-Supervised Pre-Training with Efficient Audio Transformer
Chen, W., Liang, Y., Ma, Z., Zheng, Z., & Chen, X. (2024) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2024/421 · Cited by 164
XTREME-S: Evaluating Cross-lingual Speech Representations
Conneau, A., Bapna, A., Zhang, Y., Ma, M., von Platen, P., Lozhkov, A., Cherry, C., Ye, J., Rivera, C., Kale, M., et al. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-10007 · Cited by 44
Kakouros, S., Stafylakis, T., Mošner, L., & Burget, L. (2023) · ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp49357.2023.10094673 · Cited by 22
Kuatewo, M., Ebelin, W., Doegah, P. T., Kpodo, L., Kpordorlor, A. G., Lissah, S., Djokoto, S., Aberese-Ako, M., & Ansah, E. (2025) · PLoS ONE · doi:10.1371/journal.pone.0316969 · Cited by 28
A Comparative Study of Self-Supervised Speech Representation Based Voice Conversion
Huang, W.-C., Yang, S.-W., Hayashi, T., & Toda, T. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3193761 · Cited by 33
Self-citationPretext Tasks Selection for Multitask Self-Supervised Audio Representation Learning
Zaiem, S., Parcollet, T., Essid, S., & Heba, A. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3195430 · Cited by 31
Speaker-Independent Acoustic-to-Articulatory Speech Inversion
Wu, P., Chen, L., Cho, C. J., Watanabe, S., Goldstein, L. M., Black, A. W., & Anumanchipalli, G. K. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10096796 · Cited by 50
Self-citationFavaro, A., Butala, A., Thebaud, T., Villalba, J., Dehak, N., & Moro-Velázquez, L. (2024) · npj Parkinson s Disease · doi:10.1038/s41531-024-00817-9 · Cited by 35
Evidence of Vocal Tract Articulation in Self-Supervised Learning of Speech
Cho, C. J., Wu, P., Mohamed, A., & Anumanchipalli, G. K. (2023) · ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp49357.2023.10094711 · Cited by 21
Self-citationYodas: Youtube-Oriented Dataset for Audio and Speech
Li, X., Takamichi, S., Saeki, T., Chen, W., Shiota, S., & Watanabe, S. (2023) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru57964.2023.10389689 · Cited by 127
Self-citationRepAugment: Input-Agnostic Representation-Level Augmentation for Respiratory Sound Classification
Kim, J.-W., Toikkanen, M., Bae, S., Kim, M., & Jung, H.-Y. (2024) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc53108.2024.10782363 · Cited by 39
DUAL: Discrete Spoken Unit Adaptive Learning for Textless Spoken Question Answering
Lin, G.-T., Chuang, Y.-S., Chung, H.-L., Yang, S., Chen, H.-J., Dong, S. A., Li, S.-W. D., Mohamed, A., Lee, H.-Y., & Lee, L.-S. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-612 · Cited by 44
Self-citationRecent Advances in Discrete Speech Tokens: A Review
Guo, Y., Li, Z., Wang, H., Li, B., Shao, C., Zhang, H., Du, C., Chen, X., Liu, S., & Yu, K. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3643619 · Cited by 97
Self-Supervised Learning for Infant Cry Analysis
Gorin, A., Subakan, C., Abdoli, S., Wang, J., Latremouille, S., & Onu, C. C. (2023) · IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW) · doi:10.1109/icasspw59220.2023.10193421 · Cited by 30
Boosting Cross-Domain Speech Recognition With Self-Supervision
Zhu, H., Cheng, G., Wang, J., Hou, W., Zhang, P., & Yan, Y. (2023) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2023.3301230 · Cited by 31
Speech Self-Supervised Representation Benchmarking: Are We Doing it Right?
Zaiem, S., Kemiche, Y., Parcollet, T., Essid, S., & Ravanelli, M. (2023) · Interspeech (USB) · doi:10.21437/interspeech.2023-1087 · Cited by 49
Improving Self-Supervised Learning for Speech Recognition with Intermediate Layer Supervision
Wang, C., Wu, Y., Chen, S., Liu, S., Li, J., Qian, Y., & Yang, Z. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747022 · Cited by 30
Non-Contrastive Self-Supervised Learning for Utterance-Level Information Extraction From Speech
Cho, J., Villalba, J., Moro-Velázquez, L., & Dehak, N. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3197315 · Cited by 27
Lin, G.-T., Lian, J., Li, T., Wang, Q., Anumanchipalli, G. K., Liu, A. H., & Lee, H.-Y. (2025) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru65441.2025.11433838 · Cited by 169
Self-citationParcollet, T., Nguyen, H., Evain, S. V., Boito, M. Z., Pupier, A., Mdhaffar, S., Le, H., Alisamir, S., Tomashenko, N., Dinarelli, M., et al. (2024) · Computer Speech & Language · doi:10.1016/j.csl.2024.101622 · Cited by 60
Guragain, A., Liu, T., Pan, Z., Sailor, H. B., & Wang, Q. (2024) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt61566.2024.10832226 · Cited by 39
Masuyama, Y., Chang, X., Cornell, S., Watanabe, S., & Ono, N. (2023) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt54892.2023.10023199 · Cited by 22
Self-citationLi, Y., Mehrish, A., Bhardwaj, R., Majumder, N., Bo, C., Zhao, S., Zadeh, A., Mihalcea, R. F., & Poria, S. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10095656 · Cited by 29
LEXTREME: A Multi-Lingual and Multi-Task Benchmark for the Legal DomainPreprint
Niklaus, J., Matoshi, V., Rani, P., Galassi, A., Stuermer, M., & Chalkidis, I. (2023) · arXiv (Cornell University) · doi:10.24451/arbor.19688 · Cited by 51
Wang, W., Ren, S., Qian, Y., Liu, S., Shi, Y., Qian, Y., & Zeng, M. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747760 · Cited by 24
Self-Supervised Graphs for Audio Representation Learning With Limited Labeled Data
Shirian, A., Somandepalli, K., & Guha, T. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3190083 · Cited by 20
Arunkumar, A., Sukhadia, V. N., & Umesh, S. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-11376 · Cited by 31
Learning Lip-Based Audio-Visual Speaker Embeddings with AV-HuBERT
Shi, B., Mohamed, A., & Hsu, W.-N. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-885 · Cited by 26
Self-citationExploring WavLM on Speech Enhancement
Song, H., Chen, S., Chen, Z., Wu, Y., Yoshioka, T., Tang, M., Shin, J. W., & Liu, S. (2023) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt54892.2023.10023356 · Cited by 30
Parameter Efficient Transfer Learning for Various Speech Processing Tasks
Otake, S., Kawakami, R., & Inoue, N. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10096311 · Cited by 27
Target Speech Extraction with Pre-Trained Self-Supervised Learning Models
Peng, J., Delcroix, M., Ochiai, T., Plchot, O., Araki, S., & Černocký, J. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10448315 · Cited by 29
Improving Domain Generalization in Speech Emotion Recognition with Whisper
Goron, E., Asai, L., Rut, E., & Dinov, M. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10446997 · Cited by 42
Speech Emotion Recognition Using Dual-Stream Representation and Cross-Attention Fusion
Yu, S., Meng, J., Fan, W., Chen, Y., Zhu, B., Yu, H., Xie, Y., & Sun, Q. (2024) · Electronics · doi:10.3390/electronics13112191 · Cited by 24
ESPnet-Codec: Comprehensive Training and Evaluation of Neural Codecs For Audio, Music, and Speech
Shi, J., Tian, J., Wu, Y., Jung, J.-W., Yip, J. Q., Masuyama, Y., Chen, W., Wu, Y., Tang, Y., Baali, M., et al. (2024) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt61566.2024.10832289 · Cited by 63
Self-citationCross-Corpus Speech Emotion Recognition with HuBERT Self-Supervised Representation
Pastor, M., Ribas, D., Ortega, A., Miguel, A. D., & Lleida, E. (2022) · IberSPEECH · doi:10.21437/iberspeech.2022-16 · Cited by 28
Context-based emotion recognition: A survey
Abbas, R., Ni, B., Ma, R., Li, T., Lu, Y., & Li, X. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.129073 · Cited by 24
A Study of Gender Impact in Self-supervised Models for Speech-to-Text Systems
Boito, M. Z., Besacier, L., Tomashenko, N., & Estève, Y. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-353 · Cited by 37
Neural speech enhancement with unsupervised pre-training and mixture training
Hao, X., Xu, C., & Xie, L. (2022) · Neural Networks · doi:10.1016/j.neunet.2022.11.013 · Cited by 20
Multi-task Learning for Speech Emotion and Emotion Intensity Recognition
Yue, P., Qu, L., Zheng, S., & Li, T. (2022) · Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) · doi:10.23919/apsipaasc55919.2022.9979844 · Cited by 20
Self-Supervised Learning for Speech Enhancement Through Synthesis
Irvin, B., Stamenovic, M., Kegler, M., & Yang, L.-C. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10094705 · Cited by 30
Wav2vec-VC: Voice Conversion via Hidden Representations of Wav2vec 2.0
Lim, J., & Kim, K. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10447984 · Cited by 22
Text-Free Prosody-Aware Generative Spoken Language Modeling
Kharitonov, E., Lee, A., Polyak, A., Adi, Y., Copet, J., Lakhotia, K., Nguyen, T.-A., Rivière, M., Mohamed, A., Dupoux, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2109.03264 · Cited by 16
Self-citationComprehensive Layer-wise Analysis of SSL Models for Audio Deepfake Detection
El Kheir, Y., Samih, Y., Maharjan, S., Polzehl, T., & Möller, S. (2025) · Findings of the Association for Computational Linguistics: NAACL 2025 · doi:10.18653/v1/2025.findings-naacl.227 · Cited by 16
Peng, Y., Arora, S., Higuchi, Y., Ueda, Y., Kumar, S., Ganesan, K., Dalmia, S., Chang, X., & Watanabe, S. (2023) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt54892.2023.10022399 · Cited by 33
Self-citationTang, J., Chen, W., Chang, X., Watanabe, S., & MacWhinney, B. (2023) · Interspeech (USB) · doi:10.21437/interspeech.2023-2191 · Cited by 22
Self-citationLanSER: Language-Model Supported Speech Emotion Recognition
Gong, T., Belanich, J., Somandepalli, K., Nagrani, A., Eoff, B. D., & Jou, B. (2023) · Interspeech (USB) · doi:10.21437/interspeech.2023-1832 · Cited by 25
Attention-Based Speech Enhancement Using Human Quality Perception Modeling
Nayem, K. M., & Williamson, D. S. (2023) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2023.3328282 · Cited by 19
Toward Joint Language Modeling for Speech Units and Text
Chou, J.-C., Chien, C.-M., Hsu, W.-N., Livescu, K., Babu, A., Conneau, A., Baevski, A., & Auli, M. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.438 · Cited by 40
Open-Emotion: A Reproducible EMO-Superb For Speech Emotion Recognition Systems
Wu, H., Chou, H.-C., Chang, K.-W., Goncalves, L., Du, J., Jang, J. R., Lee, C.-C., & Lee, H.-Y. (2024) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt61566.2024.10832296 · Cited by 17
Self-citationLLMs for Low Resource Languages in Multilingual, Multimodal and Dialectal Settings
Alam, F., Chowdhury, S. A., Boughorbel, S., & Hasanain, M. (2024) · doi:10.18653/v1/2024.eacl-tutorials.5 · Cited by 44
Decorrelating Feature Spaces for Learning General-Purpose Audio Representations
Ghosh, S., Seth, A., & Umesh, S. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3202093 · Cited by 19
Phonetic Analysis of Self-supervised Representations of English Speech
Wells, D., Tang, H., & Richmond, K. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-10884 · Cited by 56
Lu, K.-H., Chen, Z., Fu, S., Yang, C.-H. H., Huang, S.-F., Yang, C.-K., Yu, C.-M., Chen, C., Chen, W.-C., Huang, C.-Y., et al. (2026) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2026.3675792 · Cited by 82
Self-citationGCF2-Net: global-aware cross-modal feature fusion network for speech emotion recognition
Li, F., Luo, J., Wang, L., Liu, W., & Sang, X. (2023) · Frontiers in Neuroscience · doi:10.3389/fnins.2023.1183132 · Cited by 15
MelHuBERT: A Simplified Hubert on Mel Spectrograms
Lin, T.-Q., Lee, H.-Y., & Tang, H. (2023) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru57964.2023.10389700 · Cited by 30
Self-citationParameter Efficient Finetuning for Speech Emotion Recognition and Domain Adaptation
Lashkarashvili, N., Wu, W., Sun, G., & Woodland, P. C. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10446272 · Cited by 19
AV-SUPERB: A Multi-Task Evaluation Benchmark for Audio-Visual Representation Models
Tseng, Y., Berry, L., Chen, Y.-T., Chiu, I.-H., Lin, H.-H., Liu, M., Peng, P., Shih, Y.-J., Wang, H., Wu, H., et al. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10445941 · Cited by 33
Self-citationELP-Adapters: Parameter Efficient Adapter Tuning for Various Speech Processing Tasks
Inoue, N., Otake, S., Hirose, T., Ohi, M., & Kawakami, R. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3434445 · Cited by 17
Don't Speak Too Fast: The Impact of Data Bias on Self-Supervised Speech Models
Meng, Y., Chou, Y.-H., Liu, A. T., & Lee, H.-Y. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747897 · Cited by 41
Self-citationMartín-Doñas, J. M., Álvarez, A., Rosello, E., Gomez, A. M., & Peinado, A. M. (2024) · Interspeech 2024 · doi:10.21437/interspeech.2024-942 · Cited by 18
“Following the SUPERB setup [23], we consider the hidden embed-dings from the SSL encoder layers to obtain a more discriminative representation for the target task, as proposed on [10] and other speech-related tasks [24, 25].”
MethodInfluentialWavLM model ensemble for audio deepfake detection
Combei, D., Stan, A., Oneata, D., & Cucu, H. (2024) · The Automatic Speaker Verification Spoofing Countermeasures Workshop (ASVspoof 2024) · doi:10.21437/asvspoof.2024-25 · Cited by 15
“The resulting representations can be successfully employed by multiple downstream tasks (e.g. speech recognition, keyword spotting, speaker identification) with limited data [13].”
BackgroundDisentangling Prosody Representations With Unsupervised Speech Reconstruction
Qu, L., Li, T., Weber, C., Rosin, T. P., Ren, F., & Wermter, S. (2023) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2023.3320864 · Cited by 30
DSTM: A transformer-based model with dynamic-static feature fusion in speech emotion recognition
Jin, G., Xu, Y., Kang, H., Wang, J., & Miao, B. (2024) · Computer Speech & Language · doi:10.1016/j.csl.2024.101733 · Cited by 20
Overlapped speech and gender detection with WavLM pre-trained features
Lebourdais, M., Tahon, M., Laurent, A., & Meignier, S. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-10825 · Cited by 30
MTI-Net: A Multi-Target Speech Intelligibility Prediction Model
Zezario, R. E., Fu, S., Chen, F., Fuh, C., Wang, H., & Tsao, Y. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-10828 · Cited by 17
Audio–visual keyword transformer for unconstrained sentence‐level keyword spotting
Li, Y., Ren, J., Wang, Y., Wang, G., Li, X., & Liu, H. (2023) · CAAI Transactions on Intelligence Technology · doi:10.1049/cit2.12212 · Cited by 17
Multitask Detection of Speaker Changes, Overlapping Speech and Voice Activity Using Wav2vec 2.0
Kunešová, M., & Zajíc, Z. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10094972 · Cited by 27
Chapter: Exploiting Convolutional Neural Network Adapters for Self-Supervised Speech Models
Chen, Z.-C., Sung, Y.-S., & Lee, H.-Y. (2023) · IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW) · doi:10.1109/icasspw59220.2023.10193427 · Cited by 24
Self-citationESPnet-ST-v2: Multipurpose Spoken Language Translation Toolkit
Yan, B., Shi, J., Tang, Y., Inaguma, H., Peng, Y., Dalmia, S., Polák, P., Fernandes, P., Berrebbi, D., Hayashi, T., et al. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-demo.38 · Cited by 13
Self-citationSpeech Enhancement: A Survey of Approaches and Applications
Chhetri, S., Joshi, M. S., Mahamuni, C. V., Sangeetha, R. N., & Roy, T. (2023) · International Conference on Edge Computing and Applications (ICECAA) · doi:10.1109/icecaa58104.2023.10212180 · Cited by 18
Ashihara, T., Delcroix, M., Moriya, T., Matsuura, K., Asami, T., & Ijima, Y. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10446422 · Cited by 30
UNIT-DSR: Dysarthric Speech Reconstruction System Using Speech Unit Normalization
Wang, Y., Wu, X., Wang, D., Meng, L., & Meng, H. M. L. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10446921 · Cited by 30
Comparison of wav2vec 2.0 models on three speech processing tasks
Kunešová, M., Zajíc, Z., Šmídl, L., & Karafiát, M. (2024) · International Journal of Speech Technology · doi:10.1007/s10772-024-10140-6 · Cited by 13
ON-TRAC Consortium Systems for the IWSLT 2022 Dialect and Low-resource Speech Translation Tasks
Boito, M. Z., Ortega, J., Riguidel, H., Laurent, A., Barrault, L., Bougares, F., Chaabani, F., Nguyen, H., Barbier, F., Gahbiche, S., et al. (2022) · International Conference on Spoken Language Translation (IWSLT) · doi:10.18653/v1/2022.iwslt-1.28 · Cited by 30
Context-Aware Fine-Tuning of Self-Supervised Speech Models
Shon, S., Wu, F., Kim, K., Sridhar, P., Livescu, K., & Watanabe, S. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10094687 · Cited by 13
Self-citationIntroducing Meta‐analysis in the Evaluation of Computational Models of Infant Language Development
Blandón, M. A. C., Cristià, A., & Räsänen, O. J. (2023) · Cognitive Science · doi:10.1111/cogs.13307 · Cited by 23
Self-Supervised Learning With Segmental Masking for Speech Representation
Yue, X., Lin, J., Ritter-Gutierrez, F., & Li, H. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3191845 · Cited by 13
ESB: A Benchmark For Multi-Domain End-to-End Speech Recognition
Gandhi, S., von Platen, P., & Rush, A. M. (2022) · arXiv · doi:10.48550/arxiv.2210.13352 · Cited by 13
Efficient and Robust Speaker Diarization via Structured Pruning of Self-Supervised Models
Han, J., Pálka, P., Delcroix, M., Landini, F., Rohdin, J., Černocký, J., & Burget, L. (2026) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2026.3675801 · Cited by 31
Disentangled Feature Learning for Real-Time Neural Speech Coding
Jiang, X., Peng, X., Zhang, Y., & Lu, Y. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10094723 · Cited by 12
Metadata
- Authors
- Yang, S.-W., Chi, P.-H., Chuang, Y.-S., Lai, C.-I. J., Lakhotia, K., Lin, Y. Y., Liu, A. T., Shi, J., Chang, X., Lin, G.-T., Huang, T.-H., Tseng, W.-C., Lee, K.-T., Liu, D.-R., Huang, Z., Dong, S., Li, S.-W., Watanabe, S., Mohamed, A., & Lee, H.-Y.
- Affiliations
- National Taiwan University (TW)
- Meta (United States) (US)
- Johns Hopkins University (US)
- Carnegie Mellon University (US)
- Amazon (United States) (US)
- ORCID
- Venue
- Interspeech 2021
- Publisher
- ISCA
- Published
- 2021-08-30
- Pages
- 1194-1198
- ISSN
- 2308-457X (ISSN-L)
- OpenAlex
- W3197580070
- Cited by
- 681
- References
- 40
- Citations by year
- 2026: 67 · 2025: 138 · 2024: 157 · 2023: 205 · 2022: 108 · 2021: 6
- FWCI
- 54.87
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Interspeech (USB) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- self-supervised learning, speech processing, benchmark, shared models, representation learning, frozen pre-trained models, task generalization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.