Cited by 264Speech Recognition and SynthesisSpeech and Audio ProcessingTopic Modeling

The DOI matches its Crossref record; 52 of the 55 references it cites were found in a registry or an index.

Layer-Wise Analysis of a Self-Supervised Speech Representation Model

Open access

Pasad, Ankita, Chou, Ju-Chieh, Livescu, Karen·2021·2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)

DOI 10.1109/asru51503.2021.9688093

Open this paper in CoChat — free account Open PDF

Abstract

Recently proposed self-supervised learning approaches have been successful for pre-training speech representation models. The utility of these learned representations has been observed empirically, but not much has been studied about the type or extent of information encoded in the pre-trained representations themselves. Developing such insights can help understand the capabilities and limits of these models and enable the research community to more efficiently develop their usage for downstream applications. In this work, we begin to fill this gap by examining one recent and successful pre-trained model (wav2vec 2.0), via its intermediate representation vectors, using a suite of analysis tools. We use the metrics of canonical correlation, mutual information, and performance on simple downstream tasks with non-parametric probes, in order to (i) query for acoustic and linguistic information content, (ii) characterize the evolution of information across model layers, and (iii) understand how fine-tuning the model for automatic speech recognition (ASR) affects these observations. Our findings motivate modifying the fine-tuning protocol for ASR, which produces improved word error rates in a low-resource setting.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1109/asru51503.2021.9688093

Cites

52 of 52 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533
A simple framework for contrastive learning of visual representations
Chen, T., Kornblith, S., Norouzi, M., & Hinton, G. (2020) · arXiv · doi:10.48550/arxiv.2002.05709 · Cited by 10,857
Representation Learning with Contrastive Predictive Coding
Oord, A. V. D., Li, Y., & Vinyals, O. (2018) · arXiv · doi:10.48550/arxiv.1807.03748 · Cited by 6,242
“Librispeech: an ASR corpus based on public domain audio books,”
Panayotov, V., Chen, G., Povey, D., & Khudanpur, S. (2015) · 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2015.7178964 · Cited by 6,072
RELATIONS BETWEEN TWO SETS OF VARIATES
Hotelling, H. (1936) · Biometrika · doi:10.1093/biomet/28.3-4.321 · Cited by 6,041
Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks
Graves, A., Fernández, S., Gomez, F., & Schmidhuber, J. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143891 · Cited by 5,370
Unsupervised Visual Representation Learning by Context Prediction
Doersch, C., Gupta, A., & Efros, A. A. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.167 · Cited by 2,629

Show all 55 references

Research works printed but not linked: 2

10.1109/SLT48900.2021.9383594 · DOI
radford, 2018, Improving language understanding by generative pretraining

Sorted by citations

Audiolm: A language modeling approach to audio generation
Borsos, Z., Marinier, R., Vincent, D., Kharitonov, E., Pietquin, O., Sharifi, M., Roblek, D., Teboul, O., Grangier, D., Tagliasacchi, M., et al. (2023) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2023.3288409 · Cited by 432
Self-supervised speech representation learning: A review
Mohamed, A., Lee, H.-Y., Borgholt, L., Havtorn, J. D., Edin, J., Igel, C., Kirchhoff, K., Li, S.-W., Livescu, K., Maaløe, L., et al. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3207050 · Cited by 372
“[68] further show that fine-tuning the pre-trained model with a character-level CTC training criterion changes the behavior of the last layers (especially the final two layers), breaking”
BackgroundSelf-citation
Investigating Self-Supervised Front Ends for Speech Spoofing Countermeasures
Wang, X., & Yamagishi, J. (2022) · The Speaker and Language Recognition Workshop (Odyssey 2022) · doi:10.21437/odyssey.2022-14 · Cited by 138
Distilhubert: Speech Representation Learning by Layer-Wise Distillation of Hidden-Unit Bert
Chang, H.-J., Yang, S.-W., & Lee, H.-Y. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747490 · Cited by 125
Large-Scale Self-Supervised Speech Representation Learning for Automatic Speaker Verification
Chen, Z., Chen, S., Wu, Y., Qian, Y., Wang, C., Liu, S., Qian, Y., & Zeng, M. (2022) · ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp43922.2022.9747814 · Cited by 112
Comparative Layer-Wise Analysis of Self-Supervised Speech Models
Pasad, A., Shi, B., & Livescu, K. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10096149 · Cited by 105
Self-citation
Improving Automatic Speech Recognition Performance for Low-Resource Languages With Self-Supervised Models
Zhao, J. S., & Zhang, W.-Q. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3184480 · Cited by 84
Fusing ASR Outputs in Joint Training for Speech Emotion Recognition
Li, Y., Bell, P., & Lai, C. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746289 · Cited by 59
End-to-End Integration of Speech Recognition, Speech Enhancement, and Self-Supervised Learning Representation
Chang, X., Maekaku, T., Fujita, Y., & Watanabe, S. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-10839 · Cited by 54
A Large-Scale Evaluation of Speech Foundation Models
Yang, S.-W., Chang, H.-J., Huang, Z., Liu, A. T., Lai, C.-I., Wu, H., Shi, J., Chang, X., Tsai, H.-S., Huang, W.-C., et al. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3389631 · Cited by 57
Automatic Pronunciation Assessment using Self-Supervised Speech Representation Learning
Kim, E., Jeon, J.-J., Seo, H., & Kim, H. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-10245 · Cited by 48
Representation Learning Through Cross-Modal Conditional Teacher-Student Training For Speech Emotion Recognition
Srinivasan, S., Huang, Z., & Kirchhoff, K. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747754 · Cited by 42
Boosting Self-Supervised Embeddings for Speech Enhancement
Hung, K.-H., Fu, S., Tseng, H.-H., Chiang, H.-T., Tsao, Y., & Lin, C. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-10002 · Cited by 43
Exploration of a Self-Supervised Speech Model: A Study on Emotional Corpora
Li, Y., Mohamied, Y., Bell, P., & Lai, C. (2023) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt54892.2023.10023428 · Cited by 43
Self-Supervised Representations in Speech-Based Depression Detection
Wu, W., Zhang, C., & Woodland, P. C. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10094910 · Cited by 47
What Do Self-Supervised Speech Models Know About Words?
Pasad, A., Chien, C.-M., Settle, S., & Livescu, K. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00656 · Cited by 54
Self-citation
Advancements in Speech Recognition: A Systematic Review of Deep Learning Transformer Models, Trends, Innovations, and Future Directions
Sharrab, Y. O., Attar, H. H., Eljinini, M. A. H., Al-Omary, Y., & Al-Momani, W. (2025) · IEEE Access · doi:10.1109/access.2025.3550855 · Cited by 35
Word Discovery in Visually Grounded, Self-Supervised Speech Models
Peng, P., & Harwath, D. F. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-10652 · Cited by 40
Cross Corpus Speech Emotion Recognition using transfer learning and attention-based fusion of Wav2Vec2 and prosody features
Naderi, N., & Nasersharif, B. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.110814 · Cited by 35
ZMM-TTS: Zero-Shot Multilingual and Multispeaker Speech Synthesis Conditioned on Self-Supervised Discrete Speech Representations
Cheng, G., Wang, X., Cooper, E., Wells, D., Wang, L., Dang, J., Richmond, K., & Yamagishi, J. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3451951 · Cited by 34
On the Utility of Self-Supervised Models for Prosody-Related Tasks
Lin, G.-T., Feng, C.-L., Huang, W., Tseng, Y., Lin, T.-H., Li, C.-A., Lee, H.-Y., & Ward, N. (2023) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt54892.2023.10023234 · Cited by 34
Probing speech emotion recognition transformers for linguistic knowledge
Triantafyllopoulos, A., Wagner, J., Wierstorf, H., Schmitt, M., Reichel, U. D., Eyben, F., Burkhardt, F., & Schuller, B. W. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-10371 · Cited by 28
Pseudo Label Is Better Than Human Label
Hwang, D., Sim, K. C., Huo, Z., & Strohman, T. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-11034 · Cited by 29
Exploring Efficient-Tuning Methods in Self-Supervised Speech Models
Chen, Z.-C., Fu, C.-L., Liu, C.-Y., Li, S.-W. D., & Lee, H.-Y. (2023) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt54892.2023.10023274 · Cited by 30
StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Models
Li, Y. A., Han, C., Raghavan, V., Mischler, G., & Mesgarani, N. (2023) · PubMed · doi:10.48550/arxiv.2306.07691 · Cited by 58
Language-Independent Speaker Anonymization Approach Using Self-Supervised Pre-Trained Models
Miao, X., Wang, X., Cooper, E., Yamagishi, J., & Tomashenko, N. (2022) · The Speaker and Language Recognition Workshop (Odyssey) · doi:10.21437/odyssey.2022-39 · Cited by 25
SpeechCLIP: Integrating Speech with Pre-Trained Vision and Language Model
Shih, Y.-J., Wang, H.-F., Chang, H.-J., Berry, L., Lee, H.-Y., & Harwath, D. F. (2023) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt54892.2023.10022954 · Cited by 24
Evidence of Vocal Tract Articulation in Self-Supervised Learning of Speech
Cho, C. J., Wu, P., Mohamed, A., & Anumanchipalli, G. K. (2023) · ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp49357.2023.10094711 · Cited by 21
Leveraging Multi-modal Interactions among the Intermediate Representations of Deep Transformers for Emotion Recognition
Wu, Y., Zhang, Z., Peng, P., Zhao, Y., & Qin, B. (2022) · doi:10.1145/3551876.3554813 · Cited by 21
Analyzing Acoustic Word Embeddings from Pre-Trained Self-Supervised Speech Models
Sanabria, R., Tang, H., & Goldwater, S. J. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10096099 · Cited by 20
Boosting Cross-Domain Speech Recognition With Self-Supervision
Zhu, H., Cheng, G., Wang, J., Hou, W., Zhang, P., & Yan, Y. (2023) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2023.3301230 · Cited by 21
Leveraging Pretrained Representations With Task-Related Keywords for Alzheimer’s Disease Detection
Li, J., Song, K., Li, J., Zheng, B., Li, D., Wu, X., Liu, X., & Meng, H. M. L. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10096205 · Cited by 24
Parameter Efficient Transfer Learning for Various Speech Processing Tasks
Otake, S., Kawakami, R., & Inoue, N. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10096311 · Cited by 19
Fine-Tuning Strategies for Faster Inference Using Speech Self-Supervised Models: A Comparative Study
Zaiem, S., Algayres, R., Parcollet, T., Essid, S., & Ravanelli, M. (2023) · IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW) · doi:10.1109/icasspw59220.2023.10193042 · Cited by 17
Useful blunders: Can automated speech recognition errors improve downstream dementia classification?
Li, C., Xu, W., Cohen, T., & Pakhomov, S. (2024) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2024.104598 · Cited by 25
Wav2vec-VC: Voice Conversion via Hidden Representations of Wav2vec 2.0
Lim, J., & Kim, K. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10447984 · Cited by 18
A Sidecar Separator Can Convert A Single-Talker Speech Recognition System to A Multi-Talker One
Meng, L., Kang, J., Cui, M., Wang, Y., Wu, X., & Meng, H. M. L. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10095295 · Cited by 16
MelHuBERT: A Simplified Hubert on Mel Spectrograms
Lin, T.-Q., Lee, H.-Y., & Tang, H. (2023) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru57964.2023.10389700 · Cited by 15
LeBenchmark 2.0: A standardized, replicable and enhanced framework for self-supervised representations of French speech
Parcollet, T., Nguyen, H., Evain, S. V., Boito, M. Z., Pupier, A., Mdhaffar, S., Le, H., Alisamir, S., Tomashenko, N., Dinarelli, M., et al. (2024) · Computer Speech & Language · doi:10.1016/j.csl.2024.101622 · Cited by 37
AV-SUPERB: A Multi-Task Evaluation Benchmark for Audio-Visual Representation Models
Tseng, Y., Berry, L., Chen, Y.-T., Chiu, I.-H., Lin, H.-H., Liu, M., Peng, P., Shih, Y.-J., Wang, H., Wu, H., et al. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10445941 · Cited by 15
Audio-visual representation learning via knowledge distillation from speech foundation models
Zhang, J.-X., Wan, G., Gao, J., & Ling, Z.-H. (2025) · Pattern Recognition · doi:10.1016/j.patcog.2025.111432 · Cited by 16
Procter: Pronunciation-Aware Contextual Adapter For Personalized Speech Recognition In Neural Transducers
Pandey, R., Ren, R., Luo, Q., Liu, J., Rastrow, A., Gandhe, A., Filimonov, D., Strimel, G. P., Stolcke, A., & Bulyko, I. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10096062 · Cited by 14
Deep-learning models reveal how context and listener attention shape electrophysiological correlates of speech-to-language transformation
Anderson, A. J., Davis, C., & Lalor, E. C. (2024) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1012537 · Cited by 17
ELP-Adapters: Parameter Efficient Adapter Tuning for Various Speech Processing Tasks
Inoue, N., Otake, S., Hirose, T., Ohi, M., & Kawakami, R. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3434445 · Cited by 16
Ultra-Low-Bitrate Speech Coding with Pretrained Transformers
Siahkoohi, A., Chinen, M., Denton, T., Kleijn, W. B., & Skoglund, J. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-10988 · Cited by 12
Transfer Learning for Personality Perception via Speech Emotion Recognition
Li, Y., Bell, P., & Lai, C. (2023) · Interspeech (USB) · doi:10.21437/interspeech.2023-2061 · Cited by 12
Self-supervised Predictive Coding Models Encode Speaker and Phonetic Information in Orthogonal Subspaces
Liu, O. D., Tang, H., & Goldwater, S. J. (2023) · Interspeech (USB) · doi:10.21437/interspeech.2023-871 · Cited by 12
Exploring the potential of Wav2vec 2.0 for speech emotion recognition using classifier combination and attention-based feature fusion
Nasersharif, B., & Namvarpour, M. (2024) · The Journal of Supercomputing · doi:10.1007/s11227-024-06158-x · Cited by 13
Layer-Wise Analysis of Self-Supervised Acoustic Word Embeddings: A Study on Speech Emotion Recognition
Saliba, A., Li, Y., Sanabria, R., & Lai, C. (2024) · IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW) · doi:10.1109/icasspw62465.2024.10627629 · Cited by 12
M-Adapter: Modality Adaptation for End-to-End Speech-to-Text Translation
Zhao, J., Yang, H., Haffari, G., & Shareghi, E. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-592 · Cited by 12
A Study of Gender Impact in Self-supervised Models for Speech-to-Text Systems
Boito, M. Z., Besacier, L., Tomashenko, N., & Estève, Y. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-353 · Cited by 23
Exploring emergent syllables in end-to-end automatic speech recognizers through model explainability technique
Vitale, V. N., Cutugno, F., Origlia, A., & Coro, G. (2024) · Neural Computing and Applications · doi:10.1007/s00521-024-09435-1 · Cited by 16
Speech Separation With Pretrained Frontend to Minimize Domain Mismatch
Wang, W., Pan, Z., Li, X., Wang, S., & Li, H. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3446242 · Cited by 13
Exploring Aggregated wav2vec 2.0 Features and Dual-Stream TDNN for Efficient Spoken Dialect Identification
Angra, A., Muralikrishna, H., Dinesh, D. A., & Thenkanidiyoor, V. (2024) · IEEE Access · doi:10.1109/access.2024.3523951 · Cited by 14
Wav2vec-S: Semi-Supervised Pre-Training for Low-Resource ASR
Zhu, H., Wang, L., Cheng, G., Wang, J., Zhang, P., & Yan, Y. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-909 · Cited by 12
Prosody Modelling With Pre-Trained Cross-Utterance Representations for Improved Speech Synthesis
Zhang, Y.-J., Zhang, C., Song, W. G., Zhang, Z., Wu, Y., & He, X. (2023) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2023.3278184 · Cited by 12
An Analysis of Semantically-Aligned Speech-Text Embeddings
Huzaifah, M., & Kukanov, I. (2023) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt54892.2023.10023147 · Cited by 11
DQ-Former: Querying Transformer with Dynamic Modality Priority for Cognitive-aligned Multimodal Emotion Recognition in Conversation
Jing, Y., & Zhao, X. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681599 · Cited by 10
Cross-Lingual Speech Emotion Recognition: Humans vs. Self-Supervised Models
Han, Z., Geng, T., Feng, H., Yuan, J., Richmond, K., & Li, Y. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10889008 · Cited by 10
ON-TRAC Consortium Systems for the IWSLT 2022 Dialect and Low-resource Speech Translation Tasks
Boito, M. Z., Ortega, J., Riguidel, H., Laurent, A., Barrault, L., Bougares, F., Chaabani, F., Nguyen, H., Barbier, F., Gahbiche, S., et al. (2022) · International Conference on Spoken Language Translation (IWSLT) · doi:10.18653/v1/2022.iwslt-1.28 · Cited by 15
Avoid Overthinking in Self-Supervised Models for Speech Recognition
Berrebbi, D., Yan, B., & Watanabe, S. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10095335 · Cited by 11
Speech self-supervised representations benchmarking: A case for larger probing heads
Zaiem, S., Kemiche, Y., Parcollet, T., Essid, S., & Ravanelli, M. (2024) · Computer Speech & Language · doi:10.1016/j.csl.2024.101695 · Cited by 12
PADA: Pruning Assisted Domain Adaptation for Self-Supervised Speech Representations
Lodagala, V. S., Ghosh, S., & Umesh, S. (2023) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt54892.2023.10022820 · Cited by 11
Large language model based hybrid framework for automatic vulnerability detection with explainable AI for cybersecurity enhancement
Basheer, N., Islam, S., Alwaheidi, M. K. S., Mouratidis, H., & Papastergiou, S. (2025) · Integrated Computer-Aided Engineering · doi:10.1177/10692509251368663 · Cited by 9
AdVerb: Visually Guided Audio Dereverberation
Chowdhury, S., Ghosh, S., Dasgupta, S., Ratnarajah, A. J., Tyagi, U., & Manocha, D. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00725 · Cited by 9
Mandarin-English Code-switching Speech Recognition with Self-supervised Speech Representation ModelsPreprint
Tseng, L.-H., Fu, Y.-K., Chang, H.-J., & Lee, H.-Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.03504 · Cited by 18
Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Tutorial Abstracts
(2022) · doi:10.18653/v1/2022.naacl-tutorials · Cited by 9
Analysis of Self-Supervised Speech Models on Children’s Speech and Infant Vocalizations
Li, J., Hasegawa‐Johnson, M., & McElwain, N. L. (2024) · IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW) · doi:10.1109/icasspw62465.2024.10626416 · Cited by 8
Wave to Syntax: Probing spoken language models for syntax
Shen, G., Alishahi, A., Bisazza, A., & Chrupała, G. (2023) · Interspeech (USB) · doi:10.21437/interspeech.2023-679 · Cited by 10
Contrastive representation learning with transformers for robust auditory EEG decoding
Bollens, L., Accou, B., hamme, H. V., & Francart, T. (2025) · Scientific Reports · doi:10.1038/s41598-025-13646-4 · Cited by 7
Leveraging Cross-Attention Transformer and Multifeature Fusion for Cross-Linguistic Speech Emotion Recognition
Zhao, R., Jiang, X., Yu, F. R., Leung, V. C. M., Wang, T., & Zhang, S. (2025) · IEEE Internet of Things Journal · doi:10.1109/jiot.2025.3613687 · Cited by 10
Accidental Learners: Spoken Language Identification in Multilingual Self-Supervised Models
Bartley, T., Jia, F., Puvvada, K. C., Kriman, S., & Ginsburg, B. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10096407 · Cited by 8
Hiformer: Sequence Modeling Networks With Hierarchical Attention Mechanisms
Wu, X., Lu, H., Li, K., Wu, Z., Liu, X., & Meng, H. M. L. (2023) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2023.3313428 · Cited by 9
Self-supervised Representation Learning for Speech Processing
Lee, H.-Y., Mohamed, A., Watanabe, S., Sainath, T. N., Livescu, K., Li, S.-W. D., Yang, S.-W., & Kirchhoff, K. (2022) · doi:10.18653/v1/2022.naacl-tutorials.2 · Cited by 9
Self-citation
How to Estimate Model Transferability of Pre-Trained Speech Models?
Chen, Z.-C., Yang, C.-H. H., Li, B., Zhang, Y., Chen, N., Chang, S.-Y., Prabhavalkar, R., Lee, H.-Y., & Sainath, T. N. (2023) · Interspeech (USB) · doi:10.21437/interspeech.2023-1079 · Cited by 14
Integrating Emotion Recognition with Speech Recognition and Speaker Diarisation for Conversations
Wu, W., Zhang, C., & Woodland, P. C. (2023) · Interspeech (USB) · doi:10.21437/interspeech.2023-293 · Cited by 7
Parameter-Efficient Cross-Language Transfer Learning for a Language-Modular Audiovisual Speech Recognition
Li, Z., Graave, T., Liu, J., Lohrenz, T., Kunzmann, S., & Fingscheidt, T. (2023) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru57964.2023.10389795 · Cited by 7
Do Self-Supervised Speech and Language Models Extract Similar Representations as Human Brain?
Chen, P., He, L., Fu, L., Fan, L., Chang, E. F., & Li, Y. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10446334 · Cited by 8
Advancing Spanish Speech Emotion Recognition: A Comprehensive Benchmark of Pre-Trained Models
Mares, A., Diaz-Arango, G. U., Perez-Jacome-Friscione, J., Vázquez-Leal, H., Hernández-Martínez, L., Huerta-Chua, J., Alvarado, A. F. J., & Dominguez-Chavez, J. A. (2025) · Applied Sciences · doi:10.3390/app15084340 · Cited by 6
Adversarial Continual Learning to Transfer Self-Supervised Speech Representations for Voice Pathology Detection
Park, D., Yu, Y., Katabi, D., & Kim, H. K. (2023) · IEEE Signal Processing Letters · doi:10.1109/lsp.2023.3298532 · Cited by 7
Zero-Shot Emotion Transfer for Cross-Lingual Speech Synthesis
Li, Y., Zhu, X., Lei, Y., Li, H. H., Liu, J., Xie, D., & Xie, L. (2023) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru57964.2023.10389638 · Cited by 6
PE-Wav2vec: A Prosody-Enhanced Speech Model for Self-Supervised Prosody Learning in TTS
Liu, Z.-C., Chen, L., Hu, Y.-J., Ling, Z.-H., & Pan, J. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3449148 · Cited by 6
Rethinking Mamba in Speech Processing by Self-Supervised Models
Zhang, X., Ma, J., Shahin, M. A., Ahmed, B., & Epps, J. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10889111 · Cited by 11
Automatic Rating of Spontaneous Speech for Low-Resource Languages
Al-Ghezi, R., Getman, Y., Voskoboinik, E., Singh, M., & Kurimo, M. (2023) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt54892.2023.10022381 · Cited by 6
Relative Dynamic Time Warping Comparison for Pronunciation Errors
Richter, C., & Guðnason, J. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10095033 · Cited by 6
Leveraging Semantic Information for Efficient Self-Supervised Emotion Recognition with Audio-Textual Distilled Models
de Oliveira, D., Prabhu, N. R., & Gerkmann, T. (2023) · Interspeech (USB) · doi:10.21437/interspeech.2023-1758 · Cited by 8
Transferring Speech-Generic and Depression-Specific Knowledge for Alzheimer’s Disease Detection
Cui, Z., Wu, W., Zhang, W.-Q., Wu, J. Y., & Zhang, C. (2023) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru57964.2023.10389785 · Cited by 6
Self-Supervised Models of Speech Infer Universal Articulatory Kinematics
Cho, C. J., Mohamed, A., Black, A. W., & Anumanchipalli, G. K. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10447345 · Cited by 7
Learning Music Representations with wav2vec 2.0
Ragano, A., Benetos, E., & Hines, A. (2023) · Irish Conference on Artificial Intelligence and Cognitive Science (AICS) · doi:10.1109/aics60730.2023.10470644 · Cited by 6
Perception of Phonological Assimilation by Neural Speech Recognition Models
Pouw, C., Kloots, M. D. H., Alishahi, A., & Zuidema, W. H. (2024) · Computational Linguistics · doi:10.1162/coli_a_00526 · Cited by 6
Uncertainty as a Predictor: Leveraging Self-Supervised Learning for Zero-Shot MOS Prediction
Ravuri, A., Cooper, E., & Yamagishi, J. (2024) · IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW) · doi:10.1109/icasspw62465.2024.10626267 · Cited by 6
A Study of Multi-Scale Feature Learning From Pre-Trained Models on Speaker Verification
Peng, S., Guo, W., Zhang, J., Li, Z., Yu, G., Gu, B., & Ai, Y. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10887949 · Cited by 5
Lightweight Feature Encoder for Wake-Up Word Detection Based on Self-Supervised Speech Representation
Lim, H., Kim, Y., Yeom, K., Seo, E.-J., Lee, H., Choi, S. J., & Lee, H. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10094623 · Cited by 5
Investigating Pre-trained Audio Encoders in the Low-Resource Condition
Yang, H., Zhao, J., Haffari, G., & Shareghi, E. (2023) · Interspeech (USB) · doi:10.21437/interspeech.2023-343 · Cited by 10
Efficient Finetuning for Dimensional Speech Emotion Recognition in the Age of Transformers
Sampath, A., Tavernor, J., & Provost, E. M. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10889878 · Cited by 8
A Comparative Study of Self-Supervised Speech Representations in Read and Spontaneous TTS
Wang, S., Henter, G. E., Gustafson, J., & Székely, É. (2023) · IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW) · doi:10.1109/icasspw59220.2023.10193157 · Cited by 5
Improved Decoding of Attentional Selection in Multi-Talker Environments with Self-Supervised Learned Speech Representation
Han, C., Choudhari, V., Li, Y. A., & Mesgarani, N. (2023) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc40787.2023.10340191 · Cited by 4
SCORE: Self-Supervised Correspondence Fine-Tuning for Improved Content Representations
Meghanani, A., & Hain, T. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10448060 · Cited by 5
Generic speech enhancement with self-supervised representation space loss
Satō, H., Ochiai, T., Delcroix, M., Moriya, T., Ashihara, T., & Masumura, R. (2025) · Frontiers in Signal Processing · doi:10.3389/frsip.2025.1587969 · Cited by 4
Efficient Utilization of Large Pre-Trained Models for Low Resource ASR
Vieting, P., Lüscher, C., Dierkes, J., Schlüter, R., & Ney, H. (2023) · IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW) · doi:10.1109/icasspw59220.2023.10193570 · Cited by 4

Authors: Pasad, A., Chou, J.-C., & Livescu, K.
Affiliations: Toyota Technological Institute at Chicago (US)
ORCID: Livescu, Karen
Venue: 2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)
Publisher: IEEE
Published: 2021-12-13
Pages: 914-921
DOI: 10.1109/asru51503.2021.9688093
OpenAlex: W3179803166
Cited by: 262
References: 55
Citations by year: 2026: 42 · 2025: 65 · 2024: 61 · 2023: 62 · 2022: 29 · 2021: 3
FWCI: 17.90
Type: conference paper
Language: en
Open access: green
arxiv.org
Licence: public-domain
doi.org/10.15223/policy-029 (stm-asf)
doi.org/10.15223/policy-037 (stm-asf)
Locations: IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) (publishedVersion)
arXiv (Cornell University) (submittedVersion, public-domain)
arXiv (Cornell University) (submittedVersion)
arXiv (Cornell University)
Funders: NSF (IIS-1816627)
Air Force Office of Scientific Research (FA9550-18-1-0166)
Topics: Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
Keywords: self-supervised speech representation learning, wav2vec 2.0, layer-wise analysis, canonical correlation analysis, mutual information

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Layer-Wise Analysis of a Self-Supervised Speech Representation Model

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks