SUPERB: Speech processing Universal PERformance Benchmark
Open accessAbstract
Self-supervised learning (SSL) has proven vital for advancing research in natural language processing (NLP) and computer vision (CV). The paradigm pretrains a shared model on large volumes of unlabeled data and achieves state-of-the-art (SOTA) for various tasks with minimal adaptation. However, the speech processing community lacks a similar setup to systematically explore the paradigm. To bridge this gap, we introduce Speech processing Universal PERformance Benchmark (SUPERB). SUPERB is a leaderboard to benchmark the performance of a shared model across a wide range of speech processing tasks with minimal architecture changes and labeled data. Among multiple usages of the shared model, we especially focus on extracting the representation learned from SSL due to its preferable re-usability. We present a simple framework to solve SUPERB tasks by learning task-specialized lightweight prediction heads on top of the frozen shared model. Our results demonstrate that the framework is promising as SSL representations show competitive generalizability and accessibility across SUPERB tasks. We release SUPERB as a challenge with a leaderboard and a benchmark toolkit to fuel the research in representation learning and general speech processing.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Layer-wise analyses of self-supervised speech models show that acoustic and speaker-related cues concentrate in lower and intermediate layers while deeper layers drift toward lexical and semantic content, and that a learnable combination of all layers consistently outperforms the last-layer representation across diverse speech tasks.
“Layer-wise analyses of self-supervised speech models showthat acoustic and speaker-related cues concentrate in lower and intermediate layers while deeper layers drift toward lexical and semantic content [19, 20], and that a learnable combination of all layers consistently outperforms the last-layer representation across diverse speech tasks [21].”
From MOSS-Audio Technical Report (2026) · cites this paper as [21]
Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
Self-citationDeep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983 · see citing works
Self-citationRepresentation Learning with Contrastive Predictive Coding
Oord, A. V. D., Li, Y., & Vinyals, O. (2018) · arXiv · doi:10.48550/arxiv.1807.03748 · Cited by 6,242 · see citing works
“Librispeech: an ASR corpus based on public domain audio books,”
Panayotov, V., Chen, G., Povey, D., & Khudanpur, S. (2015) · 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2015.7178964 · Cited by 6,072 · see citing works
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., & Bowman, S. (2018) · Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP · doi:10.18653/v1/w18-5446 · Cited by 4,222 · see citing works
IEMOCAP: Interactive emotional dyadic motion capture database
Busso, C., Bulut, M., Lee, C.-C., Kazemzadeh, A., Mower, E., Kim, S., Chang, J. N., Lee, S., & Narayanan, S. S. (2008) · Language Resources and Evaluation · doi:10.1007/s10579-008-9076-6 · Cited by 3,869 · see citing works
HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units
Hsu, W.-N., Bolte, B., Tsai, Y.-H. H., Lakhotia, K., Salakhutdinov, R., & Mohamed, A. (2021) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2021.3122291 · Cited by 2,983 · see citing works
Self-citationX-Vectors: Robust DNN Embeddings for Speaker Recognition
Snyder, D., Garcia-Romero, D., Sell, G., Povey, D., & Khudanpur, S. P. (2018) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2018.8461375 · Cited by 2,698 · see citing works
Deep Clustering for Unsupervised Learning of Visual Features
Caron, M., Bojanowski, P., Joulin, A., & Douze, M. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01264-9_9 · Cited by 2,674 · see citing works
Don’t Stop Pretraining: Adapt Language Models to Domains and Tasks
Gururangan, S., Marasović, A., Swayamdipta, S., Lo, K., Beltagy, I., Downey, D., & Smith, N. A. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.740 · Cited by 2,103 · see citing works
wav2vec: Unsupervised Pre-Training for Speech Recognition
Schneider, S., Baevski, A., Collobert, R., & Auli, M. (2019) · Interspeech 2019 · doi:10.21437/interspeech.2019-1873 · Cited by 1,290 · see citing works
Computing and Visualizing Dynamic Time Warping Alignments inR: ThedtwPackage
Toni Giorgino, A. (2009) · Journal of Statistical Software · doi:10.18637/jss.v031.i07 · Cited by 1,263 · see citing works
KenLM: Faster and Smaller Language Model Queries
Heafield, K. (2011) · Cited by 1,145 · see citing works
Coucke, A., Saade, A., Ball, A., Bluche, T., Caulier, A., Leroy, D., Doumouro, C., Gisselbrecht, T., Caltagirone, F., Lavril, T., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.10190 · Cited by 690 · see citing works
Voxceleb: Large-scale speaker verification in the wild
Nagrani, A., Chung, J. S., Xie, W., & Zisserman, A. (2019) · Computer Speech & Language · doi:10.1016/j.csl.2019.101027 · Cited by 650 · see citing works
vq-wav2vec: Self-Supervised Learning of Discrete Speech RepresentationsPreprint
Baevski, A., Schneider, S., & Auli, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.05453 · Cited by 431 · see citing works
wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations
Baevski, A., Zhou, Y., Mohamed, A., & Auli, M. (2020) · neural information processing systems · Cited by 386 · see citing works
Self-citationAn Unsupervised Autoregressive Model for Speech Representation Learning
Chung, Y.-A., Hsu, W.-N., Tang, H., & Glass, J. R. (2019) · Interspeech (USB) · doi:10.21437/interspeech.2019-1473 · Cited by 356 · see citing works
Scaling and Benchmarking Self-Supervised Visual Representation Learning
Goyal, P., Mahajan, D. K., Gupta, A., & Misra, I. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00649 · Cited by 344 · see citing works
Mockingjay: Unsupervised Speech Representation Learning with Deep Bidirectional Transformer Encoders
Liu, A. T., Yang, S.-W., Chi, P.-H., Hsu, P.-C., & Lee, H.-Y. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9054458 · Cited by 335 · see citing works
Self-citationTERA: Self-Supervised Learning of Transformer Encoder Representation for Speech
Liu, A. T., Li, S.-W. D., & Lee, H.-Y. (2021) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2021.3095662 · Cited by 308 · see citing works
Self-citationSpeech Model Pre-Training for End-to-End Spoken Language Understanding
Lugosch, L., Ravanelli, M., Ignoto, P., Tomar, V. S., & Bengio, Y. (2019) · Interspeech (USB) · doi:10.21437/interspeech.2019-2396 · Cited by 280 · see citing works
Multi-Task Self-Supervised Learning for Robust Speech Recognition
Ravanelli, M., Zhong, J., Pascual, S., Świętojański, P., Monteiro, J., Trmal, J., & Bengio, Y. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9053569 · Cited by 276 · see citing works
End-to-End Neural Speaker Diarization with Permutation-Free Objectives
Fujita, Y., Kanda, N., Horiguchi, S., Nagamatsu, K., & Watanabe, S. (2019) · Interspeech (USB) · doi:10.21437/interspeech.2019-2899 · Cited by 247 · see citing works
Self-citationLearning Problem-Agnostic Speech Representations from Multiple Self-Supervised Tasks
Pascual, S., Ravanelli, M., Serrà, J., Bonafonte, A., & Bengio, Y. (2019) · Interspeech (USB) · doi:10.21437/interspeech.2019-2605 · Cited by 222 · see citing works
LibriMix: An Open-Source Dataset for Generalizable Speech SeparationPreprint
Cosentino, J., Pariente, M., Cornell, S., Deleforge, A., & Emmanuel Vincent, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.11262 · Cited by 217 · see citing works
Unsupervised Pretraining Transfers Well Across Languages
Riviere, M., Joulin, A., Mazare, P.-E., & Dupoux, E. (2020) · ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp40776.2020.9054548 · Cited by 186 · see citing works
Wav2Letter++: A Fast Open-source Speech Recognition System
Pratap, V., Hannun, A., Xu, Q., Cai, J., Kahn, J., Synnaeve, G., Liptchinsky, V., & Collobert, R. (2019) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2019.8683535 · Cited by 174 · see citing works
Exploring wav2vec 2.0 on Speaker Verification and Language Identification
Fan, Z., Li, M., Zhou, S., & Xu, B. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-1280 · Cited by 161 · see citing works
Towards Learning a Universal Non-Semantic Representation of Speech
Shor, J., Jansen, A., Maor, R., Lang, O., Tuval, O., Quitry, F. D. C., Tagliasacchi, M., Shavitt, I., Emanuel, D., & Haviv, Y. A. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-1242 · Cited by 128 · see citing works
How Useful Is Self-Supervised Pretraining for Visual Tasks?
Newell, A., & Deng, J. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00737 · Cited by 122 · see citing works
Vector-Quantized Autoregressive Predictive Coding
Chung, Y.-A., Tang, H., & Glass, J. (2020) · Interspeech 2020 · doi:10.21437/interspeech.2020-1228 · Cited by 97 · see citing works
Non-Autoregressive Predictive Coding for Learning Speech Representations from Local Dependencies
Liu, A. H., Chung, Y.-A., & Glass, J. R. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-349 · Cited by 81 · see citing works
Self-citationNguyen, T. A., de Seyssel, M., Rozé, P., Rivière, M., Kharitonov, E., Baevski, A., Dunbar, E., & Emmanuel Dupoux, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.11588 · Cited by 67 · see citing works
Lin, Y. Y., Chien, C.-M., Lin, J.-H., Lee, H.-Y., & Lee, L.-S. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9413699 · Cited by 65 · see citing works
Self-citationDeCoAR 2.0: Deep Contextualized Acoustic Representations with Vector QuantizationPreprint
Ling, S., & Liu, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.06659 · Cited by 60 · see citing works
Anguera, X., Rodríguez-Fuentes, L. J., Buzo, A., Metze, F., Szöke, I., & Peñagarikano, M. (2015) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2015.7179090 · Cited by 30 · see citing works
Recent Advances in End-to-End Spoken Language Understanding
Tomashenko, N., Caubrière, A., Estève, Y., Laurent, A., & Morin, E. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-31372-2_4 · Cited by 27 · see citing works
Problem-Agnostic Speech Embeddings for Multi-Speaker Text-to-Speech with SampleRNN
Álvarez, D., Pascual, S., & Bonafonte, A. (2019) · doi:10.21437/ssw.2019-7 · Cited by 13 · see citing works
GTTS-EHU Systems for QUESST at MediaEval 2014
Rodríguez-Fuentes, L. J., Varona, A., Peñagarikano, M., Bordel, G., & Díez, M. (2014) · MediaEval · Cited by 9 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- C.-I. Lai, Y.-S. Chuang, H.-Y. Lee, S.-W. Li, and J. Glass, "Semi- supervised spoken language understanding via self-supervised speech and language model pretraining," in ICASSP, 2021.
- T. Giorgino, "Computing and visualizing dynamic time warping alignments in R: The dtw package," Journal of Statistical Soft- ware, vol. 31, no. 7, pp. 1-24, 2009.
Cited by
Sorted by citations
WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing
Chen, S., Wang, C., Chen, Z., Wu, Y., Liu, S., Chen, Z., Li, J., Kanda, N., Yoshioka, T., Xiao, X., et al. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3188113 · Cited by 1,926 · see citing works
Ssast: Self-supervised audio spectrogram transformer
Gong, Y., Lai, C.-I., Chung, Y.-A., & Glass, J. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i10.21315 · Cited by 294 · see citing works
Self-citationSpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processing
Ao, J., Wang, R., Zhou, L., Wang, C., Ren, S., Wu, Y., Liu, S., Ko, T., Li, Q., Zhang, Y., et al. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.393 · Cited by 148 · see citing works
Distilhubert: Speech Representation Learning by Layer-Wise Distillation of Hidden-Unit Bert
Chang, H.-J., Yang, S.-W., & Lee, H.-Y. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747490 · Cited by 120 · see citing works
Self-citationDirect Speech-to-Speech Translation With Discrete Units
Lee, A., Chen, P., Wang, C., Gu, J., Popuri, S., Ma, X., Polyak, A., Adi, Y., He, Q., Tang, Y., et al. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.235 · Cited by 114 · see citing works
Large-Scale Self-Supervised Speech Representation Learning for Automatic Speaker Verification
Chen, Z., Chen, S., Wu, Y., Qian, Y., Wang, C., Liu, S., Qian, Y., & Zeng, M. (2022) · ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp43922.2022.9747814 · Cited by 112 · see citing works
Audio Deepfake Detection With Self-Supervised WavLM And Multi-Fusion Attentive Classifier
Guo, Y., Huang, H., Chen, X., Zhao, H., & Wang, Y. (2024) · ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp48485.2024.10447923 · Cited by 95 · see citing works
BYOL for Audio: Exploring Pre-Trained General-Purpose Audio Representations
Niizumi, D., Takeuchi, D., Ohishi, Y., Harada, N., & Kashino, K. (2022) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2022.3221007 · Cited by 68 · see citing works
The CHiME-7 DASR Challenge: Distant Meeting Transcription with Multiple Devices in Diverse Scenarios
Cornell, S., Wiesner, M., Watanabe, S., Raj, D., Chang, X., Garcia, L. P., Masuyam, Y., Wang, Z.-Q., Squartini, S., & Khudanpur, S. P. (2023) · International Workshop on Speech Processing in Everyday Environments (CHiME) · doi:10.21437/chime.2023-1 · Cited by 58 · see citing works
Self-citationText-Free Prosody-Aware Generative Spoken Language Modeling
Kharitonov, E., Lee, A., Polyak, A., Adi, Y., Copet, J., Lakhotia, K., Nguyen, T. A., Rivière, M., Mohamed, A., Emmanuel Dupoux, A., et al. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.593 · Cited by 53 · see citing works
Self-citationSrinivasan, S., Huang, Z., & Kirchhoff, K. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747754 · Cited by 42 · see citing works
Self-citationPARP: Prune, Adjust and Re-Prune for Self-Supervised Speech RecognitionPreprint
Lai, C.-I., Zhang, Y., Liu, A. H., Chang, S., Liao, Y.-L., Chuang, Y.-S., Qian, K., Khurana, S., Cox, D., & Glass, J. R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.05933 · Cited by 41 · see citing works
Self-citationNaderi, N., & Nasersharif, B. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.110814 · Cited by 35 · see citing works
IndicSUPERB: A Speech Processing Universal Performance Benchmark for Indian Languages
Javed, T., Bhogale, K. S., Raman, A., Kumar, P., Kunchukuttan, A., & Khapra, M. M. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i11.26521 · Cited by 31 · see citing works
Hyeon, J., Oh, Y., Lee, Y., & Choi, H. (2023) · Data & Knowledge Engineering · doi:10.1016/j.datak.2023.102262 · Cited by 20 · see citing works
Improving Distortion Robustness of Self-supervised Speech Processing Tasks with Domain Adaptation
Huang, K. P., Fu, Y.-K., Zhang, Y., & Lee, H.-Y. (2022) · arXiv · doi:10.48550/arxiv.2203.16104 · Cited by 19 · see citing works
Self-citationWang, W., Ren, S., Qian, Y., Liu, S., Shi, Y., Qian, Y., & Zeng, M. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747760 · Cited by 18 · see citing works
Parcollet, T., Nguyen, H., Evain, S. V., Boito, M. Z., Pupier, A., Mdhaffar, S., Le, H., Alisamir, S., Tomashenko, N., Dinarelli, M., et al. (2024) · Computer Speech & Language · doi:10.1016/j.csl.2024.101622 · Cited by 15 · see citing works
Introducing Meta‐analysis in the Evaluation of Computational Models of Infant Language Development
Blandón, M. A. C., Cristià, A., & Räsänen, O. J. (2023) · Cognitive Science · doi:10.1111/cogs.13307 · Cited by 13 · see citing works
Xie, Y., Arildsen, T., & Tan, Z. (2021) · Machine learning for signal processing ... · doi:10.1109/mlsp52302.2021.9596320 · Cited by 12 · see citing works
Textless Speech Emotion Conversion using Decomposed and Discrete RepresentationsPreprint
Kreuk, F., Polyak, A., Copet, J., Kharitonov, E., Nguyen, T. A., Rivière, M., Hsu, W.-N., Mohamed, A., Emmanuel Dupoux, A., & Adi, Y. (2021) · arXiv (Cornell University) · Cited by 12 · see citing works
Self-citationJiang, Z., Seyedi, S. S., Griner, E. L., Abbasi, A., Rad, A. B., Kwon, H., Cotes, R. O., & Clifford, G. D. (2023) · medRxiv · doi:10.1101/2023.09.11.23295212 · Cited by 11 · see citing works
Mohan, J., Reddy, M. S., Premjith, B., Jyothish Lal, G., & Chakravarthi, B. R. (2025) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3712260 · Cited by 9 · see citing works
Speech Tasks Relevant to Sleepiness Determined With Deep Transfer Learning
Tran, B., Zhu, Y., Liang, X., Schwoebel, J. W., & Warrenburg, L. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747000 · Cited by 9 · see citing works
Temporal Shift Module with Pretrained Representations for Speech Emotion Recognition
Shen, S., Liu, F., Wang, H., Wang, Y., & Zhou, A. (2024) · Intelligent Computing · doi:10.34133/icomputing.0073 · Cited by 10 · see citing works
Mandarin-English Code-switching Speech Recognition with Self-supervised Speech Representation ModelsPreprint
Tseng, L.-H., Fu, Y.-K., Chang, H.-J., & Lee, H.-Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.03504 · Cited by 11 · see citing works
Self-citationSVEva Fair: A Framework for Evaluating Fairness in Speaker VerificationPreprint
Toussaint, W., & Ding, A. Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.12049 · Cited by 9 · see citing works
CL-MASR: A Continual Learning Benchmark for Multilingual ASR
Libera, L. D., Mousavi, P., Zaiem, S., Subakan, C., & Ravanelli, M. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3487410 · Cited by 7 · see citing works
Masuyama, Y., Chang, X., Zhang, W., Cornell, S., Wang, Z.-Q., Ono, N., Qian, Y., & Watanabe, S. (2025) · Computer Speech & Language · doi:10.1016/j.csl.2025.101813 · Cited by 5 · see citing works
Self-citationPark, S., Mark, M., Park, B., & Hong, H. (2023) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2023.041332 · Cited by 8 · see citing works
Survey on Multimodal Transformers for RobotsPreprint
Miyazawa, K., & Nagai, T. (2023) · doi:10.36227/techrxiv.21993317 · Cited by 5 · see citing works
Let There Be Sound: Reconstructing High Quality Speech from Silent Videos
Kim, J.-H., Kim, J., & Chung, J. S. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i3.28055 · Cited by 4 · see citing works
Characterizing the Adversarial Vulnerability of Speech self-Supervised Learning
Wu, H., Zheng, B., Li, X., Wu, X., Lee, H.-Y., & Meng, H. M. L. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747242 · Cited by 3 · see citing works
Self-citationTransfer Learning for Multi-lingual Tasks -- a SurveyPreprint
Jafari, A. R., Heidary, B., Farahbakhsh, R., Salehi, M., & Jalili, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.02052 · Cited by 7 · see citing works
Do We Still Need Automatic Speech Recognition for Spoken Language Understanding?Preprint
Borgholt, L., Havtorn, J. D., Abdou, M., Edin, J., Maaløe, L., Søgaard, A., & Igel, C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.14842 · Cited by 3 · see citing works
Singing Beat Tracking With Self-supervised Front-end and Linear TransformersPreprint
Heydari, M., & Duan, Z. (2022) · doi:10.36227/techrxiv.20743012.v1 · Cited by 3 · see citing works
Deep Spectro-temporal Artifacts for Detecting Synthesized Speech
Liu, X., Liu, M., Zhang, L., Zhang, L., Zeng, C., Li, K., Li, N., Lee, K. A., Wang, L., & Dang, J. (2022) · doi:10.1145/3552466.3556527 · Cited by 2 · see citing works
Self-citationA Statistical WavLM Embedding Features with Auto-Encoder for Speech Emotion Recognition
Chakhtouna, A., Sekkate, S., & Adib, A. (2024) · Studies in computational intelligence · doi:10.1007/978-3-031-50381-8_19 · Cited by 2 · see citing works
Optimization of modular multi-speaker distant conversational speech recognition
Hu, Q., Sun, T., Chen, X., Rong, X., & Lu, J. Z. (2025) · Computer Speech & Language · doi:10.1016/j.csl.2025.101816 · Cited by 2 · see citing works
Towards Learning Universal Audio RepresentationsPreprint
Wang, L., Luc, P., Wu, Y. X., Recasens, A., Smaira, L., Brock, A., Jaegle, A., Alayrac, J.-B., Dieleman, S., Carreira, J., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.12124 · Cited by 2 · see citing works
Editorial Editorial of Special Issue on Self-Supervised Learning for Speech and Audio Processing
Lee, H.-Y., Watanabe, S., Livescu, K., Mohamed, A., & Sainath, T. N. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3205434 · Cited by 2 · see citing works
Self-citationCL-MASR: A Continual Learning Benchmark for Multilingual ASRPreprint
Libera, L. D., Mousavi, P., Zaiem, S., Subakan, C., & Ravanelli, M. (2023) · doi:10.36227/techrxiv.24430876.v1 · Cited by 2 · see citing works
Adversarial Input Ablation for Audio-Visual Learning
Xu, D., & Harwath, D. F. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746436 · Cited by 1 · see citing works
Wav2vec-S: Semi-Supervised Pre-Training for Speech RecognitionPreprint
Zhu, H., Wang, L., Hou, Y., Wang, J., Cheng, G., Zhang, P., & Yan, Y. (2021) · arXiv (Cornell University) · Cited by 1 · see citing works
Enhancing spoken language identification and diarization for multilingual speech
Liu, H. (2023) · doi:10.32657/10356/168498 · Cited by 1 · see citing works
Yang, C., Yu, C., Chen, H., Zhu, J., Chen, J., Chen, K., Wang, W., Wang, Y., Jiang, Y., Jiang, Y., et al. (2026) · arXiv · doi:10.48550/arxiv.2606.01802
Self-citationImproving information fusion in deep learning
Don, D. P. M. K. (2023) · Queensland University of Technology · doi:10.5204/thesis.eprints.241291
Apparent personality prediction from speech using expert features and wav2vec 2.0
Barchi, R. G., Pepino, L., Gauder, L., Estienne, L., Meza, M., Riera, P., & Ferrer, L. (2023) · Workshop on Speech, Music and Mind (SMM) · doi:10.21437/smm.2023-5
Speaker Group Encoding in Self-supervised Speech Recognition Models
Herron, F. E., Rossato, S., Allauzen, A., Favre, B., & Portet, F. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-02548-7_11
Pathirana, N., Htait, A., & Wanner, E. F. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-35390-0_24
Hu, K., Fan, X., Hu, Y., Li, J., Lee, B., Hasegawa‐Johnson, M., & McElwain, N. L. (2026) · Developmental Science · doi:10.1111/desc.70251
SpeechBrain: A General-Purpose Speech Toolkit
Ravanelli, M., Parcollet, T., Plantinga, P., Rouhe, A., Cornell, S., Lugosch, L., Subakan, C., Dawalatabad, N., Heba, A., Zhong, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2106.04624 · Cited by 512 · see citing works
- Authors
- Yang, S.-W., Chi, P.-H., Chuang, Y.-S., Lai, C.-I. J., Lakhotia, K., Lin, Y. Y., Liu, A. T., Shi, J., Chang, X., Lin, G.-T., Huang, T.-H., Tseng, W.-C., Lee, K.-T., Liu, D.-R., Huang, Z., Dong, S., Li, S.-W., Watanabe, S., Mohamed, A., & Lee, H.-Y.
- Affiliations
- National Taiwan University (TW)
- Meta (United States) (US)
- Johns Hopkins University (US)
- Carnegie Mellon University (US)
- Amazon (United States) (US)
- ORCID
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2021
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2105.01051
- OpenAlex
- W3157923770
- Cited by
- 52
- References
- 40
- Citations by year
- 2026: 2 · 2025: 4 · 2024: 5 · 2023: 14 · 2022: 16 · 2021: 11
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Keywords
- self-supervised learning, speech processing, benchmark, shared models, representation learning, frozen pre-trained models, task generalization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
