The DOI matches its Crossref record; 43 of the 43 references it cites were found in a registry or an index; none of the 9 sources its claims cite is checked for support yet.
wav2vec: Unsupervised Pre-Training for Speech Recognition
Open accessAbstract
We explore unsupervised pre-training for speech recognition by learning representations of raw audio. wav2vec is trained on large amounts of unlabeled audio data and the resulting representations are then used to improve acoustic model training. We pre-train a simple multi-layer convolutional neural network optimized via a noise contrastive binary classification task. Our experiments on WSJ reduce WER of a strong character-based log-mel filterbank baseline by up to 36% when only a few hours of transcribed data is available. Our approach achieves 2.43% WER on the nov92 test set. This outperforms Deep Speech 2, the best reported character-based system in the literature while using two orders of magnitude less labeled training data.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
13 Own result · 7 Not checked yet · 3 Own interpretation · 2 No basis given · 1 Own method
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Thereafter, performance plateaus while training time increases.”
§ WAV2VEC: UNSUPERVISED PRE-TRAINING FOR SPEECH RECOGNITION > 4.3 ABLATIONS
plainn = 10- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Table 3 shows that increasing the number of negative samples only helps up to ten samples.”
§ WAV2VEC: UNSUPERVISED PRE-TRAINING FOR SPEECH RECOGNITION > 4.3 ABLATIONS
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Table 1 shows that pre-training on more data leads to better accuracy on the WSJ benchmark.”
§ WAV2VEC: UNSUPERVISED PRE-TRAINING FOR SPEECH RECOGNITION > 4.1 PRE-TRAINING FOR THE WSJ BENCHMARK
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“On TIMIT, pre-training enables us to match the best reported result in the literature.”
- Also stated in: § 4.2 PRE-TRAINING FOR TIMIT · plain
“Table 2 shows that wav2vec pre-training on Librispeech and WSJ audio data can lead to results matching the state of the art.”
§ WAV2VEC: UNSUPERVISED PRE-TRAINING FOR SPEECH RECOGNITION > 1 INTRODUCTION
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 4.2 PRE-TRAINING FOR TIMIT · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Pretrained representations can substantially improve performance over our character-based baseline which is trained on log-mel filterbank features.”
§ WAV2VEC: UNSUPERVISED PRE-TRAINING FOR SPEECH RECOGNITION > 4.1 PRE-TRAINING FOR THE WSJ BENCHMARK
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Table 4 shows that a crop size of 150k frames results in the best performance.”
§ WAV2VEC: UNSUPERVISED PRE-TRAINING FOR SPEECH RECOGNITION > 4.3 ABLATIONS
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We found it important to choose a normalization scheme that is invariant to the scaling and the offset of the input.This choice resulted in representations that generalize well across datasets.”
§ WAV2VEC: UNSUPERVISED PRE-TRAINING FOR SPEECH RECOGNITION > 2 PRE-TRAINING APPROACH
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Experimental results on the WSJ benchmark demonstrate that pre-trained representations estimated on about 1,000 hours of unlabeled speech can substantially improve a character-based ASR system and outperform the best character-based result in the literature, Deep Speech 2, improving WER from 3.1 % to 2.43 %.”
§ WAV2VEC: UNSUPERVISED PRE-TRAINING FOR SPEECH RECOGNITION > 1 INTRODUCTION
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: Unlabeled audio data is much easier to collect than labeled data.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Pre-training only on the audio data of WSJ (wav2vec WSJ) performs worse compared to the much larger Librispeech (wav2vec Libri).”
§ WAV2VEC: UNSUPERVISED PRE-TRAINING FOR SPEECH RECOGNITION > 4.1 PRE-TRAINING FOR THE WSJ BENCHMARK
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Similar tovan den Oord et al. (2018), we found that sampling negatives from different sequences and speakers yields inferior results.”
§ WAV2VEC: UNSUPERVISED PRE-TRAINING FOR SPEECH RECOGNITION > 5 CONCLUSIONS
What the sources say
- Not checked yetcites van den Oord et al., 2018
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- It may state two separate claims as one. 8
- The quote only partly states this claim. 1
- Probably not a substantial claim. 3
In the paper
“Our approach achieves 2.43 % WER on the nov92 test set.This outperforms Deep Speech 2, the best reported character-based system in the literature while using two orders of magnitude less labeled training data.”
- Also stated in: § 5 CONCLUSIONS · plain
“Our approach achieves 2.43 % WER on the test set of WSJ, a result that outperforms the next best known character-based speech recognition model in the literature (Amodei et al., 2016) while using two orders of magnitude less transcribed training data.”
- Also stated in: § 4.1 PRE-TRAINING FOR THE WSJ BENCHMARK · boosted
“This shows that pre-training on unlabeled audio data can improve over the best character-based approach, Deep Speech 2 (Amodei et al., 2016), by 0.67 WER on nov92.”
§ WAV2VEC: UNSUPERVISED PRE-TRAINING FOR SPEECH RECOGNITION > Abstract
What the sources say
- Not checked yetcites Amodei et al., 2016
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one. The quote only partly states this claim.
- Also stated in: § 5 CONCLUSIONS · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We found it important to introduce skip connections in the aggregator to help convergence in this case.”
§ WAV2VEC: UNSUPERVISED PRE-TRAINING FOR SPEECH RECOGNITION > 2 PRE-TRAINING APPROACH
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“In comparison to Hadian et al. (2018), wav2vec performs as well as their phoneme-based model and wav2vec large outperforms it by 0.37 WER.”
§ WAV2VEC: UNSUPERVISED PRE-TRAINING FOR SPEECH RECOGNITION > 4.1 PRE-TRAINING FOR THE WSJ BENCHMARK
What the sources say
- Not checked yetcites Hadian et al., 2018
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We show that more data for pre-training improves performance and that this approach not only improves resource-poor setups, but also settings where all WSJ training data is used.”
§ WAV2VEC: UNSUPERVISED PRE-TRAINING FOR SPEECH RECOGNITION > 5 CONCLUSIONS
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: Unlabeled audio data is much easier to collect than labeled data.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Table 5 also shows that predicting more than 12 steps ahead in the future does not result in better performance and increasing the number of steps increases training time.”
§ WAV2VEC: UNSUPERVISED PRE-TRAINING FOR SPEECH RECOGNITION > 5 CONCLUSIONS
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Similar to Hénaff et al. (2019), we noticed that fine-tuning the embedding network does not meaningfully improve performance while substantially increasing the acoustic model training time.”
§ WAV2VEC: UNSUPERVISED PRE-TRAINING FOR SPEECH RECOGNITION > 4.1 PRE-TRAINING FOR THE WSJ BENCHMARK
What the sources say
- Not checked yetcites Hénaff et al., 2019
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Accuracy steadily increases with more data for pre-training and the best accuracy is achieved with the largest amount of data for pre-training.”
§ WAV2VEC: UNSUPERVISED PRE-TRAINING FOR SPEECH RECOGNITION > 4.2 PRE-TRAINING FOR TIMIT
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Not restricting the maximum length (None) gives an average sequence length of about 207k frames and results in the worst accuracy.”
§ WAV2VEC: UNSUPERVISED PRE-TRAINING FOR SPEECH RECOGNITION > 4.3 ABLATIONS
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“wav2vec relies on a fully convolutional architecture which can be easily parallelized over time on modern hardware compared to recurrent models used in previous work ( §2).”
§ WAV2VEC: UNSUPERVISED PRE-TRAINING FOR SPEECH RECOGNITION > 1 INTRODUCTION
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“In this experiment, everything is kept equal except for the number of negative samples.”
§ WAV2VEC: UNSUPERVISED PRE-TRAINING FOR SPEECH RECOGNITION > 4.3 ABLATIONS
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
The paper relies on this without support.
Unlabeled audio data is much easier to collect than labeled data.
“This enables exploiting unlabeled audio data which is much easier to collect than labeled data.”
The paper relies on this without support.
Accurately modeling the data distribution p(x) is a common and challenging problem for these approaches.
“A common problem with these approaches is the requirement to accurately model the data distribution p(x), which is challenging.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“As in [15], utilizing the causal convolutional aggregator results in a fully convolutional model, increasing the parallelizability of the pre-training, resulting in faster training times.”
From Investigating Enhancements to Contrastive Predictive Coding for Human Activity Recognition (2023) · cites this paper as [15]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“we propose to instead utilize multiple causal convolution blocks to perform the autoregressive summarization, due to its simplicity, faster training times, and ease of parallelization that recurrent models do not enjoy.”
From Investigating Enhancements to Contrastive Predictive Coding for Human Activity Recognition (2023) · cites this paper as [15]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Note that wav2vec directly trains on speech’s raw waveform to extract deep features that represent speech signals.”
From A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face (2023) · cites this paper as [54]
The cited source says what the claim attributes to it.
In the citing paper
“For example, Schneider et al. [54] proposed an unsupervised speech pre-training method named wav2vec that only uses unlabeled speech data for self-supervised training to obtain generalized speech features that can be applied to multiple tasks.”
From A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face (2023) · cites this paper as [54]
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533
Natural Language Processing (almost) from ScratchPreprint
Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., & Kuksa, P. P. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1103.0398 · Cited by 6,920
Self-citationRepresentation Learning with Contrastive Predictive Coding
Oord, A. V. D., Li, Y., & Vinyals, O. (2018) · arXiv · doi:10.48550/arxiv.1807.03748 · Cited by 6,242
“Librispeech: an ASR corpus based on public domain audio books,”
Panayotov, V., Chen, G., Povey, D., & Khudanpur, S. (2015) · 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2015.7178964 · Cited by 6,072
SGDR: Stochastic Gradient Descent with Warm Restarts
Loshchilov, I., & Hutter, F. (2016) · arXiv · doi:10.48550/arxiv.1608.03983 · Cited by 5,309
Unsupervised Visual Representation Learning by Context Prediction
Doersch, C., Gupta, A., & Efros, A. A. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.167 · Cited by 2,629
Cited by
Sorted by citations
Items 201–300 of 1,003
Improved Language Identification Through Cross-Lingual Self-Supervised Learning
Tjandra, A., Choudhury, D. G., Zhang, F., Singh, K., Conneau, A., Baevski, A., Sela, A., Saraf, Y., & Auli, M. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747667 · Cited by 40
Self-citationPeng, J., Plchot, O., Stafylakis, T., Mošner, L., Burget, L., & Černocký, J. (2023) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt54892.2023.10022775 · Cited by 42
Zhu, Q., Zhou, L., Zhang, Z., Liu, S., Jiao, B., Zhang, J., Dai, L.-R., Jiang, D., Li, J., & Wei, F. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3275873 · Cited by 38
ATCSpeechNet: A multilingual end-to-end speech recognition framework for air traffic control systems
Lin, Y., Yang, B., Li, L., Guo, D., Zhang, J., Chen, H., & Zhang, Y. (2021) · Applied Soft Computing · doi:10.1016/j.asoc.2021.107847 · Cited by 38
A Virtual Simulation-Pilot Agent for Training of Air Traffic Controllers
Zuluaga-Gómez, J., Prasad, A., Nigmatulina, I., Motlíček, P., & Kleinert, M. (2023) · Aerospace · doi:10.3390/aerospace10050490 · Cited by 38
Multi-Format Contrastive Learning of Audio RepresentationsPreprint
Wang, L., & van den Oord, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.06508 · Cited by 57
Chang, X., Yan, B., Choi, K., Jung, J.-W., Lu, Y., Maiti, S., Sharma, R., Shi, J., Tian, J., Watanabe, S., et al. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10447929 · Cited by 37
A review on subjective and objective evaluation of synthetic speech
Cooper, E., Huang, W.-C., Tsao, Y., Wang, H., Toda, T., & Yamagishi, J. (2024) · Nippon Onkyo Gakkaishi/Acoustical science and technology/Nihon Onkyo Gakkaishi · doi:10.1250/ast.e24.12 · Cited by 37
A neuralized feature engineering method for entity relation extraction
Chen, Y., Yang, W., Wang, K., Qin, Y., Huang, R., & Zheng, Q. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.04.010 · Cited by 39
Do Infants Really Learn Phonetic Categories?
Feldman, N. H., Goldwater, S. J., Emmanuel Dupoux, A., & Schatz, T. (2021) · Open Mind · doi:10.1162/opmi_a_00046 · Cited by 41
G3SR: Global Graph Guided Session-Based Recommendation
Deng, Z.-H., Wang, C.-D., Huang, L., Lai, J., & Yu, P. S. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3159592 · Cited by 38
Multi-level Fusion of Wav2vec 2.0 and BERT for Multimodal Emotion Recognition
Zhao, Z., Wang, Y., & Wang, Y. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-10230 · Cited by 43
i-Code: An Integrative and Composable Multimodal Learning Framework
Yang, Z., Fang, Y., Zhu, C., Pryzant, R., Chen, D., Shi, Y., Yi‐chong, X., Qian, Y., Gao, M., Chen, Y., et al. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i9.26290 · Cited by 45
Voice Disorder Classification Using Wav2vec 2.0 Feature Extraction
Cai, J., Song, Y., Wu, J., & Chen, X. (2024) · Journal of Voice · doi:10.1016/j.jvoice.2024.09.002 · Cited by 36
SSAMBA: Self-Supervised Audio Representation Learning With Mamba State Space Model
Shams, S., Dindar, S. S., Jiang, X., & Mesgarani, N. (2024) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt61566.2024.10832304 · Cited by 36
Visually Guided Self Supervised Learning of Speech Representations
Shukla, A., Vougioukas, K., Ma, P., Petridis, S., & Pantić, M. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9053415 · Cited by 37
Probing Acoustic Representations for Phonetic Properties
Ma, D., Ryant, N., & Liberman, M. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414776 · Cited by 38
Momentum Pseudo-Labeling for Semi-Supervised Speech Recognition
Higuchi, Y., Moritz, N., Roux, J. L., & Hori, T. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-571 · Cited by 36
Wav2Seq: Pre-Training Speech-to-Text Encoder-Decoder Models Using Pseudo Languages
Wu, F., Kim, K., Watanabe, S., Han, K. J., McDonald, R., Weinberger, K. Q., & Artzi, Y. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10096988 · Cited by 40
Large-Scale Unsupervised Pre-Training for End-to-End Spoken Language Understanding
Wang, P., Wei, L., Cao, Y., Xie, J., & Nie, Z. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9053163 · Cited by 37
Zhu, Q., Zhang, J., Zhang, Z., Wu, M., Fang, X., & Dai, L.-R. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747379 · Cited by 36
Leveraging Speech PTM, Text LLM, And Emotional TTS For Speech Emotion Recognition
Ma, Z., Wu, W., Zheng, Z., Guo, Y., Chen, Q., Zhang, S., & Chen, X. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10445906 · Cited by 39
A Comprehensive Survey on Embodied Intelligence: Advancements, Challenges, and Future Perspectives
Sun, F., Chen, R., Ji, T., Luo, Y., Zhou, H., & Liu, H. (2024) · CAAI Artificial Intelligence Research · doi:10.26599/air.2024.9150042 · Cited by 45
Similarity Analysis of Self-Supervised Speech Representations
Chung, Y.-A., Belinkov, Y., & Glass, J. R. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414321 · Cited by 37
Deng, K., Cao, S., & Ma, L. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-1186 · Cited by 35
Contrastive Language-Image Pre-training for the Italian LanguagePreprint
Bianchi, F., Attanasio, G., Pisoni, R., Terragni, S., Sarti, G., & Lakshmi, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.08688 · Cited by 43
Feng, T., & Narayanan, S. (2023) · 2023 11th International Conference on Affective Computing and Intelligent Interaction (ACII) · doi:10.1109/acii59096.2023.10388152 · Cited by 34
Hallo3: Highly Dynamic and Realistic Portrait Image Animation with Video Diffusion Transformer
Cui, J., Li, H., Zhan, Y., Shang, H. L., Cheng, K., Ma, Y., Mu, S., Zhou, H., Wang, J., & Zhu, S. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01964 · Cited by 85
MERBench: A Unified Evaluation Benchmark for Multimodal Emotion Recognition
Lian, Z., Sun, L., Ren, Y., Gu, H., Sun, H., Chen, L., Liu, B., & Tao, J. (2026) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2026.3653457 · Cited by 71
Segmental Contrastive Predictive Coding for Unsupervised Word Segmentation
Bhati, S., Villalba, J., Żelasko, P., Moro-Velázquez, L., & Dehak, N. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-1874 · Cited by 34
Improving CTC-Based Speech Recognition Via Knowledge Transferring from Pre-Trained Language Models
Deng, K., Cao, S., Zhang, Y., Ma, L., Cheng, G., Ji, X., & Zhang, P. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747887 · Cited by 34
Fully Automated End-to-End Fake Audio Detection
Wang, C., Yi, J., Tao, J., Sun, H., Chen, X., Tian, Z., Ma, H., Fan, C., & Fu, R. (2022) · DDAM@MM · doi:10.1145/3552466.3556530 · Cited by 35
Successes and critical failures of neural networks in capturing human-like speech recognition
Adolfi, F., Bowers, J. S., & Poeppel, D. (2023) · Neural Networks · doi:10.1016/j.neunet.2023.02.032 · Cited by 34
OPT: Omni-Perception Pre-Trainer for Cross-Modal Understanding and GenerationPreprint
Liu, J., Zhu, X., Liu, F., Guo, L., Zhao, Z., Sun, M., Wang, N., Lu, H., Zhou, S., Zhang, J., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.00249 · Cited by 40
Voice Deepfake Detection Using the Self-Supervised Pre-Training Model HuBERT
Li, L., Lu, T., Ma, X., Yuan, M., & Wan, D. (2023) · Applied Sciences · doi:10.3390/app13148488 · Cited by 33
Designing and Evaluating Speech Emotion Recognition Systems: A Reality Check Case Study with IEMOCAP
Antoniou, N., Katsamanis, A., Γιαννακόπουλος, Θ., & Narayanan, S. S. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10096808 · Cited by 33
Towards interpretable learned representations for ecoacoustics using variational auto-encoding
Gibb, K., Eldridge, A., Sandom, C. J., & Simpson, I. (2023) · Ecological Informatics · doi:10.1016/j.ecoinf.2023.102449 · Cited by 33
Pušica, M., Kartali, A., Bojović, L., Gligorijević, I., Jovanović, J., Leva, M. C., & Mijović, B. (2024) · Brain Sciences · doi:10.3390/brainsci14020149 · Cited by 33
Li, M., Zhu, Z., Li, K., & Pei, H. (2024) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2024.3430045 · Cited by 33
Multimodal Speech Biomarkers for Remote Monitoring of ALS Disease Progression
Neumann, M., Kothare, H., & Ramanarayanan, V. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.108949 · Cited by 35
MusicID: A Brainwave-Based User Authentication System for Internet of Things
Sooriyaarachchi, J., Seneviratne, S., Thilakarathna, K., & Zomaya, A. Y. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3044726 · Cited by 33
Improving the Adversarial Robustness for Speaker Verification by Self-Supervised Learning
Wu, H., Xu, L., Liu, A. T., Wu, Z., Meng, H. M. L., & Lee, H.-Y. (2021) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2021.3133189 · Cited by 37
An Improved Model for Voicing Silent Speech
Gaddy, D., & Klein, D. (2021) · doi:10.18653/v1/2021.acl-short.23 · Cited by 42
Multi-Task Self-Supervised Pre-Training for Music Classification
Wu, H.-H., Kao, C.-C., Tang, Q., Sun, M., McFee, B., Bello, J. P., & Wang, C. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414405 · Cited by 32
Adversarial Meta Sampling for Multilingual Low-Resource Speech Recognition
Xiao, Y., Gong, K., Zhou, P., Zheng, G., Liang, X., & Lin, L. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i16.17661 · Cited by 37
PIRNet: Personality-Enhanced Iterative Refinement Network for Emotion Recognition in Conversation
Lian, Z., Liu, B., & Tao, J. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3192469 · Cited by 33
Self-Supervised Time Series Representation Learning by Inter-Intra Relational ReasoningPreprint
Fan, H., Zhang, F., & Gao, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.13548 · Cited by 38
Multi-Scale Permutation Entropy for Audio Deepfake Detection
Wang, C., He, J., Yi, J., Tao, J., Zhang, C. Y., & Zhang, X. (2024) · ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp48485.2024.10448095 · Cited by 31
Hallo3: Highly Dynamic and Realistic Portrait Image Animation with Video Diffusion Transformer
Cui, J., Li, H., Zhan, Y., Shang, H., Cheng, K., Ma, Y., Mu, S., Zhou, H., Wang, J., & Zhu, S. (2024) · arXiv · doi:10.48550/arxiv.2412.00733 · Cited by 32
Recent Advances in Direct Speech-to-text Translation
Xu, C., Ye, R., Dong, Q., Zhao, C., Ko, T., Wang, M., Xiao, T., & Zhu, J. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/761 · Cited by 31
Senane, Z., Cao, L., Buchner, V., Tashiro, Y., You, L., Herman, P. A., Nordahl, M. G., Tu, R., & von Ehrenheim, V. (2024) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3637528.3671673 · Cited by 38
Neural representations for modeling variation in speech
Bartelds, M., de Vries, W., Sanal, F., Richter, C., Liberman, M., & Wieling, M. B. (2022) · Journal of Phonetics · doi:10.1016/j.wocn.2022.101137 · Cited by 32
SAMU-XLSR: Semantically-Aligned Multimodal Utterance-Level Cross-Lingual Speech Representation
Khurana, S., Laurent, A., & Glass, J. R. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3192714 · Cited by 47
Towards Better Domain Adaptation for Self-Supervised Models: A Case Study of Child ASR
Fan, R., Zhu, Y., Wang, J., & Alwan, A. A. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3200910 · Cited by 37
On the Utility of Self-Supervised Models for Prosody-Related Tasks
Lin, G.-T., Feng, C.-L., Huang, W., Tseng, Y., Lin, T.-H., Li, C.-A., Lee, H.-Y., & Ward, N. (2023) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt54892.2023.10023234 · Cited by 34
Textless Speech Emotion Conversion using Discrete & Decomposed Representations
Kreuk, F., Polyak, A., Copet, J., Kharitonov, E., Nguyen, T. A., Rivière, M., Hsu, W.-N., Mohamed, A., Emmanuel Dupoux, A., & Adi, Y. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.769 · Cited by 33
Kanda, N., Ye, G., Wu, Y., Gaur, Y., Wang, X., Meng, Z., Chen, Z., & Yoshioka, T. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-102 · Cited by 31
Speech-Text Pre-training for Spoken Dialog Understanding with Explicit Cross-Modal Alignment
Yu, T., Gao, H., Lin, T.-E., Yang, M., Wu, Y., Ma, W., Wang, C., Huang, F., & Li, Y. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.438 · Cited by 33
Wang, S., Chen, Z., Lee, K. A., Qian, Y., & Li, H. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3492793 · Cited by 36
A Further Study of Unsupervised Pretraining for Transformer Based Speech Recognition
Jiang, D., Li, W., Zhang, R., Cao, M. Y., Luo, N., Yang, H., Zou, W., Han, K., & Li, X. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414539 · Cited by 34
Huang, W.-C., Wu, Y.-C., & Hayashi, T. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9415079 · Cited by 30
MRD-Net: Multi-Modal Residual Knowledge Distillation for Spoken Question Answering
You, C., Chen, N., & Zou, Y. (2021) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/549 · Cited by 32
Performance-Efficiency Trade-Offs in Unsupervised Pre-Training for Speech Recognition
Wu, F., Kim, K., Pan, J. S., Han, K. J., Weinberger, K. Q., & Artzi, Y. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747432 · Cited by 32
Vocal markers of autism: Assessing the generalizability of machine learning models
Rybner, A. S., Jessen, E. T., Mortensen, M. D., Larsen, S. N., Grossman, R. B., Bilenberg, N., Cantio, C., Jepsen, J. R. M., Weed, E., Simonsen, A., et al. (2022) · Autism Research · doi:10.1002/aur.2721 · Cited by 30
Nowakowski, K., Ptaszynski, M. E., Murasaki, K., & Nieuważny, J. (2022) · Information Processing & Management · doi:10.1016/j.ipm.2022.103148 · Cited by 32
Recognizing More Emotions with Less Data Using Self-supervised Transfer LearningPreprint
Boigne, J., Liyanage, B. N., & Östrem, T. (2020) · Preprints.org · doi:10.20944/preprints202008.0645.v1 · Cited by 32
Unsupervised Multimodal Learning for Dependency-Free Personality Recognition
Ghassemi, S., Zhang, T., van Breda, W., Koutsoumpis, A., Oostrom, J. K., Holtrop, D., & de Vries, R. E. (2023) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2023.3318367 · Cited by 32
A Survey of Cross-Modal Visual Content Generation
Nazarieh, F., Feng, Z., Awais, M., Wang, W., & Kittler, J. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3351601 · Cited by 32
He, J.-J., Shi, X.-H., Li, X.-F., & Toda, T. (2024) · IEEE International Conference on Acoustics, Speech, and Signal Processing · doi:10.1109/icassp48485.2024.10446548 · Cited by 40
Wav2Spk: A Simple DNN Architecture for Learning Speaker Embeddings from Waveforms
Lin, W., & Mak, M. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-1287 · Cited by 29
Pseudo Label Is Better Than Human Label
Hwang, D., Sim, K. C., Huo, Z., & Strohman, T. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-11034 · Cited by 29
Deep Learning Methods for Abstract Visual Reasoning: A Survey on Raven's Progressive Matrices
Małkiński, M., & Mańdziuk, J. (2025) · ACM Computing Surveys · doi:10.1145/3715093 · Cited by 47
Multimodal Self-Supervised Learning of General Audio RepresentationsPreprint
Wang, L., Luc, P., Recasens, A., Alayrac, J.-B., & van den Oord, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.12807 · Cited by 49
A Convolutional Deep Markov Model for Unsupervised Speech Representation Learning
Khurana, S., Laurent, A., Hsu, W.-N., Chorowski, J., Łańcucki, A., Marxer, R., & Glass, J. R. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-3084 · Cited by 32
Chou, H.-C., Lin, W.-C., Lee, C.-C., & Busso, C. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746990 · Cited by 28
textless-lib: a Library for Textless Spoken Language Processing
Kharitonov, E., Copet, J., Lakhotia, K., Nguyen, T. A., Tomasello, P., Lee, A., Elkahky, A. M., Hsu, W.-N., Mohamed, A., Emmanuel Dupoux, A., et al. (2022) · doi:10.18653/v1/2022.naacl-demo.1 · Cited by 42
Efficient Self-Supervised Learning Representations for Spoken Language Identification
Liu, H., Garcia, L. P., Khong, A. W. H., Chng, E. S., Styles, S. J., & Khudanpur, S. P. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3201445 · Cited by 28
Tell Me Where To Go: Voice-Controlled Hands-Free Locomotion for Virtual Reality Systems
Hombeck, J., Voigt, H., Heggemann, T., Datta, R. R., & Lawonn, K. (2023) · IEEE Conference on Virtual Reality and 3D User Interfaces (VR) · doi:10.1109/vr55154.2023.00028 · Cited by 30
Investigating Enhancements to Contrastive Predictive Coding for Human Activity Recognition
Haresamudram, H., Essa, I., & Plötz, T. (2023) · 2023 IEEE International Conference on Pervasive Computing and Communications (PerCom) · doi:10.1109/percom56429.2023.10099197 · Cited by 27
Applying machine learning to primate bioacoustics: Review and perspectives
Cauzinille, J., Favre, B., Marxer, R., & Rey, A. (2024) · American Journal of Primatology · doi:10.1002/ajp.23666 · Cited by 29
Ling, S., Salazar, J., Liu, Y., & Kirchhoff, K. (2020) · The Speaker and Language Recognition Workshop (Odyssey) · doi:10.21437/odyssey.2020-2 · Cited by 28
Discrete and continuous representations and processing in deep learning: Looking forward
Cartuyvels, R., Spinks, G., & Moens, M. (2021) · AI Open · doi:10.1016/j.aiopen.2021.07.002 · Cited by 32
Feng, T., & Narayanan, S. S. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10448130 · Cited by 27
Ong, K. L., Lee, C. P., Lim, H. S., Lim, K. M., & Alqahtani, A. (2024) · IEEE Access · doi:10.1109/access.2024.3360483 · Cited by 28
Large-Scale ASR Domain Adaptation Using Self- and Semi-Supervised Learning
Hwang, D., Misra, A., Huo, Z., Siddhartha, N., Garg, S., Qiu, D., Sim, K. C., Strohman, T., Beaufays, F., & He, Y. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746719 · Cited by 27
Deng, K., Cao, S., Zhang, Y., & Ma, L. (2021) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru51503.2021.9688009 · Cited by 30
Are Discrete Units Necessary for Spoken Language Modeling?
Nguyen, T. A., Sagot, B., & Emmanuel Dupoux, A. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3200909 · Cited by 29
Anti-Spoofing Using Transfer Learning with Variational Information Bottleneck
Eom, Y., Lee, Y., Um, J. S., & Kim, H. R. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-10200 · Cited by 27
Cross-Lingual Self-training to Learn Multilingual Representation for Low-Resource Speech Recognition
Zhang, Z., Song, Y., Wu, M., Fang, X., McLoughlin, I. V., & Dai, L.-R. (2022) · Circuits Systems and Signal Processing · doi:10.1007/s00034-022-02075-7 · Cited by 35
An Emotion Speech Synthesis Method Based on VITS
Zhao, W., & Yang, Z. (2023) · Applied Sciences · doi:10.3390/app13042225 · Cited by 26
Unsupervised Pre-Training Of Bidirectional Speech Encoders Via Masked ReconstructionPreprint
Wang, W., Tang, Q., & Livescu, K. (2020) · arXiv (Cornell University) · doi:10.17023/p6jq-6c54 · Cited by 26
CTAL: Pre-training Cross-modal Transformer for Audio-and-Language Representations
Li, H., Ding, W., Kang, Y., Liu, T., Wu, Z., & Liu, Z. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.323 · Cited by 29
End-to-End Speech-to-Text Translation: A Survey
Sethiya, N., & Maurya, C. K. (2024) · Computer Speech & Language · doi:10.1016/j.csl.2024.101751 · Cited by 33
Multimodal Semi-Supervised Learning Framework for Punctuation Prediction in Conversational Speech
Sunkara, M., Ronanki, S., Bekal, D., Bodapati, S., & Kirchhoff, K. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-3074 · Cited by 28
Brazilian Portuguese Speech Recognition Using Wav2vec 2.0
Gris, L. R. S., Casanova, E., de Oliveira, F. S., Soares, A. D. S., & Cândido, A. (2022) · Lecture notes in computer science · doi:10.1007/978-3-030-98305-5_31 · Cited by 25
Momentum Pseudo-Labeling: Semi-Supervised ASR With Continuously Improving Pseudo-Labels
Higuchi, Y., Moritz, N., Roux, J. L., & Hori, T. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3195367 · Cited by 30
Feng, T.-H., Dong, A., Yeh, C.-F., Yang, S.-W., Lin, T.-Q., Shi, J., Chang, K., Huang, Z., Wu, H., Chang, X., et al. (2023) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt54892.2023.10022770 · Cited by 26
Waveform Boundary Detection for Partially Spoofed Audio
Cai, Z., Wang, W., & Li, M. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10094774 · Cited by 25
Large-scale foundation models and generative AI for BigData neuroscience
Wang, R., & Chen, Z. S. (2024) · Neuroscience Research · doi:10.1016/j.neures.2024.06.003 · Cited by 29
SpeechCLIP: Integrating Speech with Pre-Trained Vision and Language Model
Shih, Y.-J., Wang, H.-F., Chang, H.-J., Berry, L., Lee, H.-Y., & Harwath, D. F. (2023) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt54892.2023.10022954 · Cited by 24
Metadata
- Authors
- Schneider, S., Baevski, A., Collobert, R., & Auli, M.
- Affiliations
- Technical University of Munich (DE)
- Meta (Israel) (IL)
- Venue
- Interspeech 2019
- Publisher
- ISCA
- Published
- 2019-09-15
- Pages
- 3465-3469
- ISSN
- 2308-457X (ISSN-L)
- OpenAlex
- W2939710050
- Cited by
- 1,290
- References
- 32
- Citations by year
- 2026: 86 · 2025: 169 · 2024: 222 · 2023: 262 · 2022: 221 · 2021: 225 · 2020: 97 · 2019: 7
- FWCI
- 52.61
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- public-domain
- Locations
- Interspeech (USB) (publishedVersion)
- arXiv (Cornell University) (submittedVersion, public-domain)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Physical Sciences › Computer Science › Signal Processing › Music and Audio Processing
- Keywords
- wav2vec, unsupervised pre-training, speech recognition, audio representation learning, noise contrastive estimation, acoustic model training, convolutional neural networks, Wall Street Journal
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-09-28, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-09-28 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
