Cited by 264Speech Recognition and SynthesisSpeech and Audio ProcessingTopic Modeling

The DOI matches its Crossref record; 52 of the 55 references it cites were found in a registry or an index.

Layer-Wise Analysis of a Self-Supervised Speech Representation Model

Open access

Pasad, Ankita, Chou, Ju-Chieh, Livescu, Karen·2021·2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)

DOI 10.1109/asru51503.2021.9688093

Open this paper in CoChat — free account Open PDF

Abstract

Recently proposed self-supervised learning approaches have been successful for pre-training speech representation models. The utility of these learned representations has been observed empirically, but not much has been studied about the type or extent of information encoded in the pre-trained representations themselves. Developing such insights can help understand the capabilities and limits of these models and enable the research community to more efficiently develop their usage for downstream applications. In this work, we begin to fill this gap by examining one recent and successful pre-trained model (wav2vec 2.0), via its intermediate representation vectors, using a suite of analysis tools. We use the metrics of canonical correlation, mutual information, and performance on simple downstream tasks with non-parametric probes, in order to (i) query for acoustic and linguistic information content, (ii) characterize the evolution of information across model layers, and (iii) understand how fine-tuning the model for automatic speech recognition (ASR) affects these observations. Our findings motivate modifying the fine-tuning protocol for ASR, which produces improved word error rates in a low-resource setting.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1109/asru51503.2021.9688093

Cites

52 of 52 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533
A simple framework for contrastive learning of visual representations
Chen, T., Kornblith, S., Norouzi, M., & Hinton, G. (2020) · arXiv · doi:10.48550/arxiv.2002.05709 · Cited by 10,857
Representation Learning with Contrastive Predictive Coding
Oord, A. V. D., Li, Y., & Vinyals, O. (2018) · arXiv · doi:10.48550/arxiv.1807.03748 · Cited by 6,242
“Librispeech: an ASR corpus based on public domain audio books,”
Panayotov, V., Chen, G., Povey, D., & Khudanpur, S. (2015) · 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2015.7178964 · Cited by 6,072
RELATIONS BETWEEN TWO SETS OF VARIATES
Hotelling, H. (1936) · Biometrika · doi:10.1093/biomet/28.3-4.321 · Cited by 6,041
Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks
Graves, A., Fernández, S., Gomez, F., & Schmidhuber, J. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143891 · Cited by 5,370
Unsupervised Visual Representation Learning by Context Prediction
Doersch, C., Gupta, A., & Efros, A. A. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.167 · Cited by 2,629

Show all 55 references

Research works printed but not linked: 2

10.1109/SLT48900.2021.9383594 · DOI
radford, 2018, Improving language understanding by generative pretraining

Sorted by citations

Items 101–200 of 264

Few-Shot Spoken Language Understanding Via Joint Speech-Text Models
Chien, C.-M., Zhang, M., Chou, J.-C., & Livescu, K. (2023) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru57964.2023.10389660 · Cited by 4
Self-citation
Revisiting Self-supervised Learning of Speech Representation from a Mutual Information Perspective
Liu, A. H., Yeh, S.-L., & Glass, J. R. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10447758 · Cited by 4
Statistics in Phonetics
Tavakoli, S., Matteo, B., Pigoli, D., Chodroff, E., Coleman, J., Gubian, M., Renwick, M. E. L., & Sonderegger, M. (2024) · Annual Review of Statistics and Its Application · doi:10.1146/annurev-statistics-112723-034642 · Cited by 9
Improving Dialect Identification in Indian Languages Using Multimodal Features from Dialect Informed ASR
Amartyaveer, Kumar, S., Sharma, S., Udupa, S., Badiger, S., Singh, A., Deekshitha, G., Bandekar, J., Murthy, S., & Ghosh, P. K. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10889099 · Cited by 4
Resource-Efficient Transfer Learning from Speech Foundation Model Using Hierarchical Feature Fusion
Huo, Z., Sim, K. C., Li, B., Hwang, D., Sainath, T. N., & Strohman, T. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10096799 · Cited by 5
Noise Robust Distillation of Self-Supervised Speech Models via Correlation Metrics
Ritter-Gutierrez, F., Huang, K.-P., Ng, D., Wong, J. H. M., Lee, H.-Y., Chng, E. S., & Chen, N. F. (2024) · IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW) · doi:10.1109/icasspw62465.2024.10627241 · Cited by 4
Head information bottleneck (HIB): leveraging information bottleneck for efficient transformer head attribution and pruning
Qian, Y., Zhuang, X., & Wang, M. (2025) · EURASIP Journal on Audio Speech and Music Processing · doi:10.1186/s13636-025-00411-8 · Cited by 4
Towards End-to-End Speech-to-Text Summarization
Monteiro, R., & Pernes, D. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-40498-6_27 · Cited by 3
WavRx: A Disease-Agnostic, Generalizable, and Privacy-Preserving Speech Health Diagnostic Model
Zhu, Y. M., & Falk, T. H. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3454550 · Cited by 3
Property Neurons in Self-Supervised Speech Transformers
Lin, T.-Q., Lin, G.-T., Lee, H.-Y., & Tang, H. (2024) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt61566.2024.10832151 · Cited by 3
EmoSDS: Unified Emotionally Adaptive Spoken Dialogue System Using Self-Supervised Speech Representations
Lee, J., Sim, Y., Kim, J., & Suh, Y.-J. (2025) · Future Internet · doi:10.3390/fi17040143 · Cited by 3
Multi-Modal Dataset Across Exertion Levels: Capturing Post-Exercise Speech, Breathing, and Phonocardiogram
Nie, J., Fan, Y., Zhao, M., Wan, R., Xuan, Z., Preindl, M., & Jiang, X. (2025) · ACM Conference on Embedded Networked Sensor Systems (SenSys) · doi:10.1145/3715014.3722065 · Cited by 3
Breathing and Semantic Pause Detection and Exertion-Level Classification in Post-Exercise Speech
Wang, Y., Xia, W., Yao, H., & Nie, J. (2025) · doi:10.1145/3737901.3768369 · Cited by 3
Speech-FT: Merging Pre-Trained and Fine-Tuned Speech Representation Models for Cross-Task Generalization
Lin, T.-Q., Huang, W.-P., Tang, H., & Lee, H.-Y. (2025) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2025.3635827 · Cited by 5
Analysing the Masked Predictive Coding Training Criterion for Pre-Training a Speech Representation Model
Yadav, H. N., Sitaram, S., & Shah, R. R. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10096167 · Cited by 3
NAIST Simultaneous Speech-to-speech Translation System for IWSLT 2023
Fukuda, R., Nishikawa, Y., Kano, Y., Ko, Y., Yanagita, T., Doi, K., Makinae, M., Sakti, S., Sudoh, K., & Nakamura, S. (2023) · International Conference on Spoken Language Translation (IWSLT) · doi:10.18653/v1/2023.iwslt-1.31 · Cited by 3
Introducing Semantics into Speech Encoders
Xu, D. C., Dong, S., Wang, C., Kim, S., Lin, Z., Liu, B., Shrivastava, A., Li, S.-W. D., Tseng, L.-H., Lin, G.-T., et al. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.639 · Cited by 7
SLMGAN: Exploiting Speech Language Model Representations for Unsupervised Zero-Shot Voice Conversion in GANs
Li, Y. A., Han, C., & Mesgarani, N. (2023) · IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA) · doi:10.1109/waspaa58266.2023.10248117 · Cited by 3
Homophone Disambiguation Reveals Patterns of Context Mixing in Speech Transformers
Mohebbi, H., Chrupała, G., Zuidema, W. H., & Alishahi, A. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.513 · Cited by 3
Distilling Hubert with LSTMs via Decoupled Knowledge Distillation
de Oliveira, D., & Gerkmann, T. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10446456 · Cited by 3
Enhancing Zero-Shot Multi-Speaker TTS with Negated Speaker Representations
Jeon, Y., Kim, Y., & Lee, G. G. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i16.29793 · Cited by 3
From Raw Speech to Fixed Representations: A Comprehensive Evaluation of Speech Embedding Techniques
Porjazovski, D., Grósz, T., & Kurimo, M. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3426301 · Cited by 4
Comparing human and machine's use of coarticulatory vowel nasalization for linguistic classification
Zellou, G., Kim, L., & Gendrot, C. (2024) · The Journal of the Acoustical Society of America · doi:10.1121/10.0027932 · Cited by 3
Zero-Shot Multi-Task Cough Sound Analysis with Speech Foundation Model Embeddings
Laska, B., Xi, P., Valdés, J. J., Wallace, B., & Goubran, R. A. (2024) · IEEE International Symposium on Medical Measurements and Applications (MeMeA) · doi:10.1109/memea60663.2024.10596816 · Cited by 3
PEFT for Speech: Unveiling Optimal Placement, Merging Strategies, and Ensemble Techniques
Lin, T.-H., Wang, H.-S., Weng, H.-Y., Peng, K.-C., Chen, Z.-C., & Lee, H.-Y. (2024) · IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW) · doi:10.1109/icasspw62465.2024.10625957 · Cited by 4
Speaker Recognition Based on Pre-Trained Model and Deep Clustering
He, L., Song, Z., Liu, S., Niu, M., Hu, Y., & Huang, H. M. (2024) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme57554.2024.10687367 · Cited by 3
Disentangled-Transformer: An Explainable End-to-End Automatic Speech Recognition Model with Speech Content-Context Separation
Wang, P., & hamme, H. V. (2025) · IEEE International Conference on Image Processing, Applications and Systems (IPAS) · doi:10.1109/ipas63548.2025.10924511 · Cited by 3
Discrete Speech Unit Extraction via Independent Component Analysis
Nakamura, T., Choi, K., Hojo, K., Bando, Y., Fukayama, S., & Watanabe, S. (2025) · IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW) · doi:10.1109/icasspw65056.2025.11011236 · Cited by 3
Applications of Artificial Intelligence for Cross-Language Intelligibility Assessment of Dysarthric Speech
Yeo, E., Liss, J., Berisha, V., & Mortensen, D. R. (2025) · Perspectives of the ASHA Special Interest Groups · doi:10.1044/2025_persp-25-00030 · Cited by 6
Quantifying Language Variation Acoustically with Few Resources
Bartelds, M., & Wieling, M. B. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.273 · Cited by 4
Self-supervised Rewiring of Pre-trained Speech Encoders:Towards Faster Fine-tuning with Less Labels in Speech Processing
Yang, H., Zhao, J., Haffari, G., & Shareghi, E. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.141 · Cited by 3
Hidden State Variability of Pretrained Language Models Can Guide Computation Reduction for Transfer Learning
Xie, S., Qiu, J., Pasad, A., Du, L., Qu, Q., & Mei, H. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.422 · Cited by 12
Self-citation
Do We Still Need Automatic Speech Recognition for Spoken Language Understanding?Preprint
Borgholt, L., Havtorn, J. D., Abdou, M., Edin, J., Maaløe, L., Søgaard, A., & Igel, C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.14842 · Cited by 4
Singing Beat Tracking With Self-supervised Front-end and Linear TransformersPreprint
Heydari, M., & Duan, Z. (2022) · doi:10.36227/techrxiv.20743012.v1 · Cited by 3
Exploring Effective Distillation of Self-Supervised Speech Models for Automatic Speech Recognition
Wang, Y., Tang, C., Ma, Z., Zheng, Z., Chen, X., & Zhang, W.-Q. (2023) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru57964.2023.10389746 · Cited by 3
Exploring ASR-Based WAV2VEC2 for Automated Speech Disorder Assessment: Insights and Analysis
Nguyen, T. A., Fredouille, C., Ghio, A., Balaguer, M., & Woisard, V. (2024) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt61566.2024.10832202 · Cited by 2
Towards Real-Time Audio Deepfake Detection in Resource-Limited Environments
Dinh-Xuan, H., Doan, T.-P., Han, S., Hong, K., & Jung, S. (2025) · Communications in computer and information science · doi:10.1007/978-981-96-7005-5_28 · Cited by 2
Gestural feature extraction and multi-feature co-activation for dysarthric speech recognition
Lin, Y., Wang, L., Dang, J., & Minematsu, N. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103490 · Cited by 2
Interpreting the Role of Visemes in Audio-Visual Speech Recognition
Papadopoulos, A., & Harte, N. (2025) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru65441.2025.11434749 · Cited by 2
A Ground-Truth-Free Framework for Validating Emotions in Generative AI Speech Synthesis
Özcan, A. R., & Figen, G. A. (2026) · IEEE Access · doi:10.1109/access.2026.3656800 · Cited by 2
Selection of Layers from Self-supervised Learning Models for Predicting Mean-Opinion-Score of Speech
Liang, X., Cumlin, F., Ungureanu, V., Reddy, C. K. A., Schüldt, C., & Chatterjee, S. (2025) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru65441.2025.11433842 · Cited by 7
AudioLens: A Closer Look at Auditory Attribute Perception of Large Audio-Language Models
Yang, C.-K., Ho, N., Lee, Y., & Lee, H.-Y. (2025) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru65441.2025.11434723 · Cited by 18
USAD: Universal Speech and Audio Representation via Distillation
Chang, H.-J., Bhati, S., Glass, J. R., & Liu, A. H. (2025) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru65441.2025.11434802 · Cited by 4
Toward Leveraging Pre-Trained Self-Supervised Frontends for Automatic Singing Voice Understanding Tasks: Three Case Studies
Yamamoto, Y. (2023) · Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) · doi:10.1109/apsipaasc58517.2023.10317286 · Cited by 2
An Experimental Comparison of Noise-Robust Text-To-Speech Synthesis Systems Based On Self-Supervised Representation
Zhao, X., Zhu, Q., & Hu, Y. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10446750 · Cited by 2
Unveiling the Linguistic Capabilities of a Self-Supervised Speech Model Through Cross-Lingual Benchmark and Layer- Wise Similarity Analysis
Ashihara, T., Delcroix, M., Ijima, Y., & Kashino, M. (2024) · IEEE Access · doi:10.1109/access.2024.3428364 · Cited by 3
Integrating Self-Supervised Speech Model with Pseudo Word-Level Targets from Visually-Grounded Speech Model
Fang, H.-C., Ye, N.-X., Shih, Y.-J., Peng, P., Wang, H.-F., Berry, L., Lee, H.-Y., & Harwath, D. F. (2024) · IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW) · doi:10.1109/icasspw62465.2024.10625802 · Cited by 2
Mitigation of gender bias in automatic facial non-verbal behaviors generation
Delbosc, A., Ochs, M., Sabouret, N., Ravenet, B., & Ayache, S. (2024) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3678957.3685732 · Cited by 2
Convexity Based Pruning of Speech Representation Models
Dorszewski, T., Tětková, L., & Hansen, L. K. (2024) · Machine learning for signal processing ... · doi:10.1109/mlsp58920.2024.10734716 · Cited by 2
Visinger2+: End-to-End Singing Voice Synthesis Augmented by Self-Supervised Learning Representation
Yu, Y., Shi, J., Wu, Y., Tang, Y., & Watanabe, S. (2024) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt61566.2024.10832313 · Cited by 3
Exploring Acoustic Similarity in Emotional Speech and Music via Self-Supervised Representations
Sun, Y., Zhao, Z., Richmond, K., & Li, Y. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10890304 · Cited by 2
Emotion information recovery potential of wav2vec2 network fine-tuned for speech recognition task
Purohit, T., & Magimai.-Doss, M. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10890800 · Cited by 2
Leveraging distance information for generalized spoofing speech detection
Lu, J., Zhang, Y., Li, Z., Shang, Z., Wang, W., & Zhang, P. (2025) · Computer Speech & Language · doi:10.1016/j.csl.2025.101804 · Cited by 3
Progressive Residual Extraction Based Pre-Training for Speech Representation Learning
Wang, T., Li, J., Ma, Z., Cao, R., Chen, X., Wang, L., Ge, M., Wang, X., Wang, Y. G., Dang, J., et al. (2025) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2025.3564118 · Cited by 3
Enhancing Code-switched Text-to-Speech Synthesis Capability in Large Language Models with only Monolingual Corpora
Xu, J., Tan, D., Wang, J., & Chen, X. (2025) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru65441.2025.11434663 · Cited by 2
Phone and Speaker Spatial Organization in Self-Supervised Speech Representations
Riera, P., Cerdeiro, M., Pepino, L., & Ferrer, L. (2023) · IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW) · doi:10.1109/icasspw59220.2023.10193460 · Cited by 2
Don’t Be So Sure! Boosting ASR Decoding via Confidence Relaxation
Wullach, T., & Chazan, S. E. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i11.26614 · Cited by 3
Synthetic Speech Detection with Wav2vec 2.0 in Various Language Settings
Dropuljić, B., Šuflaj, M., Jertec, A., & Obadić, L. (2024) · IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW) · doi:10.1109/icasspw62465.2024.10627750 · Cited by 1
Efficiency-oriented approaches for self-supervised speech representation learning
Lugo, L., & Vielzeuf, V. (2024) · International Journal of Speech Technology · doi:10.1007/s10772-024-10121-9 · Cited by 2
Systematic Literature Review and Bibliometric Analysis of Low-Resource Speech-to-Text Translation
Kponou, D. F., Laleye, F. A. A., & Ezin, E. C. (2024) · Lecture notes in networks and systems · doi:10.1007/978-3-031-73125-9_24 · Cited by 1
Exploring Foundation Model Fusion Effectiveness and Explainability for Stylistic Analysis of Emotional Podcast Data
Das, A., Franzreb, C., Polzehl, T., & Möller, S. (2025) · Lecture notes in networks and systems · doi:10.1007/978-3-031-84457-7_23 · Cited by 1
DQ-Data2vec: Decoupling Quantization for Multilingual Speech Recognition
Shao, Q., Dong, L., Wei, K., Sun, S., & Xie, L. (2025) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2025.3551074 · Cited by 1
Enhancing and Exploring Mild Cognitive Impairment Detection with W2V-BERT-2.0
Wang, Y., Matsushima, T., Matsushima, S., & Sakai, T. (2025) · IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW) · doi:10.1109/icasspw65056.2025.11011147 · Cited by 1
Minerva 2 for speech and language tasks
Mogridge, R., & Ragni, A. (2025) · Computer Speech & Language · doi:10.1016/j.csl.2025.101843 · Cited by 1
CSRef: Contrastive Semantic Alignment for Speech Referring Expression Comprehension
Huang, L., Zhong, S.-H., & Liu, Y. (2025) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3765520 · Cited by 1
Beyond Speech and More: Investigating the Emergent Ability of Speech Pre-Trained Models for Classifying Physiological Time-Series Signals
Phukan, O. C., Behera, S. R., Girish, Akhtar, M. M., Buduru, A. B., & Sharma, R. (2025) · Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) · doi:10.1109/apsipaasc65261.2025.11249005 · Cited by 1
Causal Self-Supervised Pretrained Frontend With Predictive Patterns for Speech Separation
Wang, W., Pan, Z., Li, X., Wang, S., & Li, H. (2025) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2025.3640969 · Cited by 1
Layer-wise Analysis for Quality of Multilingual Synthesized Speech
Cooper, E., Okamoto, T., Ohtani, Y., Toda, T., & Kawai, H. (2025) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru65441.2025.11434769 · Cited by 1
Transcribe, Translate, or Transliterate: An Investigation of Intermediate Representations in Spoken Language Models
Ògúnremí, T., Manning, C. D., Jurafsky, D., & Livescu, K. (2025) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru65441.2025.11434765 · Cited by 1
Self-citation
Time vs. Layer: Locating Predictive Cues for Dysarthric Speech Descriptors in Wav2vec 2.0
Engert, N., Wagner, D., Riedhammer, K., & Bocklet, T. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11461884 · Cited by 1
Layer-Aware Early Fusion of Acoustic and Linguistic Embeddings for Cognitive Status Classification
Novotny, K., Moro-Velázquez, L., & Mekyska, J. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11463687 · Cited by 1
Beyond Transcript Alignment: Diagnosing Paralinguistic Information Flow in Frozen Speech-to-LLM Adapters
Kadyrbek, N., & Мансурова, М. (2026) · Big Data and Cognitive Computing · doi:10.3390/bdcc10080251 · Cited by 1
Self-FiLM: Conditioning GANs with self-supervised representations for bandwidth extension based speaker recognition
Kataria, S., Villalba, J., Moro-Velázquez, L., Thebaud, T., & Dehak, N. (2023) · Interspeech (USB) · doi:10.21437/interspeech.2023-2031 · Cited by 1
Understanding Probe Behaviors Through Variational Bounds of Mutual Information
Choi, K., Jung, J.-W., & Watanabe, S. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10448065 · Cited by 3
Speech Representation Analysis Based on Inter- and Intra-Model Similarities
Kheir, Y. E., Ali, A. M. A., & Chowdhury, S. A. (2024) · IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW) · doi:10.1109/icasspw62465.2024.10669908 · Cited by 1
Domain Adaptation by Alternating Learning of Acoustic and Linguistic Information for Japanese Deaf and Hard-of-Hearing People
Takahashi, K., Wakabayashi, Y., Ohta, K., Kobayashi, A., & Kitaoka, N. (2024) · Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) · doi:10.1109/apsipaasc63619.2025.10849215 · Cited by 1
A correlation-permutation approach for speech-music encoders model merging
Ritter-Gutierrez, F., Lin, Y., Wong, J., Lee, H.-Y., Chng, E. S., & Chen, N. F. (2025) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru65441.2025.11433848 · Cited by 2
SCaLa: Supervised Contrastive Learning for End-to-End Speech Recognition
Fu, L., Li, X., Wang, R., Fan, L., Zhang, Z., Chen, M., Wu, Y., & He, X. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-412 · Cited by 2
Probing Statistical Representations for End-to-End ASR
Ollerenshaw, A., Jalal, M. A., & Hain, T. (2023) · European Signal Processing Conference (EUSIPCO) · doi:10.23919/eusipco58844.2023.10290070 · Cited by 1
Wav2vec-S: Semi-Supervised Pre-Training for Speech RecognitionPreprint
Zhu, H., Wang, L., Hou, Y., Wang, J., Cheng, G., Zhang, P., & Yan, Y. (2021) · arXiv (Cornell University) · Cited by 1
Singing Beat Tracking With Self-supervised Front-end and Linear TransformersPreprint
Heydari, M., & Duan, Z. (2022) · doi:10.36227/techrxiv.20743012 · Cited by 1
NCANet: Integrating Normalized Channel Attention for Enhanced Lightweight Image ClassificationPreprint
Qian, C., & Liu, B. (2024) · Research Square · doi:10.21203/rs.3.rs-4008005/v1 · Cited by 1
Convergent representations and spatiotemporal dynamics of speech and language in brain and deep neural networksPreprint
Chen, P., Xiang, S., He, L., Chang, E. F., & Li, Y. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.12.28.630582 · Cited by 1
Informed Speech Self-supervised Representation Learning
Zaiem, S. (2024) · theses.fr (ABES) · doi:10.70675/dc53ad72z177fz4bb4z91fbzd00576ba3f2e · Cited by 1
MOSS-Audio Technical Report
Yang, C., Yu, C., Chen, H., Zhu, J., Chen, J., Chen, K., Wang, W., Wang, Y., Jiang, Y., Jiang, Y., et al. (2026) · arXiv · doi:10.48550/arxiv.2606.01802
“…show that acoustic and speaker-related cues concentrate in lower and intermediate layers while deeper layers drift toward lexical and semantic content [19, 20], and that a learnable combination of all layers consistently outperforms the last-layer representation across diverse speech tasks [21].”
Masked Autoencoders with Multi-Window Local-Global Attention Are Better Audio LearnersPreprint
Yadav, S., Theodoridis, S., Hansen, L. K., & Tan, Z.-H. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.00561 · Cited by 1
Apparent personality prediction from speech using expert features and wav2vec 2.0
Barchi, R. G., Pepino, L., Gauder, L., Estienne, L., Meza, M., Riera, P., & Ferrer, L. (2023) · Workshop on Speech, Music and Mind (SMM) · doi:10.21437/smm.2023-5
Context and Attention Shape Electrophysiological Correlates of Speech-to-Language TransformationPreprint
Anderson, A. J., Davis, C., & Lalor, E. C. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.09.24.559177
Enhancing Whisper Model for Pronunciation Assessment with Multi-Adapters
Li, J., Li, R., Guo, S., & Wumaier, A. (2023) · Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) · doi:10.1109/apsipaasc58517.2023.10317374
Efficient Transfer Learning Methods Using Parameterization in Few-Shot Speech Recognition
Jin, H., Jin, X., Yu, M., & Wang, G. (2023) · International Congress on Image and Signal Processing (CISP) · doi:10.1109/cisp-bmei60920.2023.10373326
Text-Only Unsupervised Domain Adaptation for Neural Transducer-Based ASR Personalization Using Synthesized Data
Kim, D., Lee, J., & Chang, J. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10446454 · Cited by 1
Learning Semantic Information from Raw Audio Signal Using Both Contextual and Phonetic Representations
Kim, J., Hwang, I., & Lee, K. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10445745
Investigating Self-Supervised Features for Expressive, Multilingual Voice Conversion
Martín-Cortinas, Á., Sáez-Trigueros, D., Beringer, G., Vallés-Ṕerez, I., Barra-Chicote, R., Vecino, B. T., Gabryś, A., Merritt, T., Biliński, P., & Lorenzo-Trueba, J. (2024) · IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW) · doi:10.1109/icasspw62465.2024.10627128
Skew-pair fusion theory: An interpretable multimodal fusion frameworkPreprint
Zhi, H., Hong, H., Cai, X., Li, L., Ren, Z., Xiao, M., Jiang, H., & Wang, X. (2024) · Research Square · doi:10.21203/rs.3.rs-5208094/v1 · Cited by 1
CSRef: Contrastive Semantic Alignment for Speech Referring Expression Comprehension
Huang, L., & Zhong, S.-H. (2024) · doi:10.1145/3689089.3689706
Dual-Path Framework with Multi-Head Bayesian Coattention for Speech Emotion Recognition under Human-Machine Interaction
Wang, Q., Li, L., Li, F., Zhou, G., Chen, N., Meng, X., Li, X., Liu, Y., & Zhao, Z. (2024) · Procedia Computer Science · doi:10.1016/j.procs.2024.10.107
Advances in OpenASR21 Evaluation with Increased Temporal Resolution for Speech Self-supervised Learning Models
Gupta, V. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-77961-9_5
A Simple HMM with Self-Supervised Representations for Phone Segmentation
Yang, G.-P., & Tang, H. (2024) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt61566.2024.10832139
Language-Independent Prosody-Enhanced Speech Representations For Multilingual Speech Synthesis
Liu, C., Ling, Z.-H., & Hu, Y.-J. (2024) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt61566.2024.10832341
Measuring Sound Symbolism In Audio-Visual Models
Tseng, W.-C., Shih, Y.-J., Harwath, D. F., & Mooney, R. J. (2024) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt61566.2024.10832254 · Cited by 1

Authors: Pasad, A., Chou, J.-C., & Livescu, K.
Affiliations: Toyota Technological Institute at Chicago (US)
ORCID: Livescu, Karen
Venue: 2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)
Publisher: IEEE
Published: 2021-12-13
Pages: 914-921
DOI: 10.1109/asru51503.2021.9688093
OpenAlex: W3179803166
Cited by: 262
References: 55
Citations by year: 2026: 42 · 2025: 65 · 2024: 61 · 2023: 62 · 2022: 29 · 2021: 3
FWCI: 17.90
Type: conference paper
Language: en
Open access: green
arxiv.org
Licence: public-domain
doi.org/10.15223/policy-029 (stm-asf)
doi.org/10.15223/policy-037 (stm-asf)
Locations: IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) (publishedVersion)
arXiv (Cornell University) (submittedVersion, public-domain)
arXiv (Cornell University) (submittedVersion)
arXiv (Cornell University)
Funders: NSF (IIS-1816627)
Air Force Office of Scientific Research (FA9550-18-1-0166)
Topics: Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
Keywords: self-supervised speech representation learning, wav2vec 2.0, layer-wise analysis, canonical correlation analysis, mutual information

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Layer-Wise Analysis of a Self-Supervised Speech Representation Model

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks