The DOI matches its Crossref record; 43 of the 43 references it cites were found in a registry or an index; none of the 9 sources its claims cite is checked for support yet.
wav2vec: Unsupervised Pre-Training for Speech Recognition
Open accessAbstract
We explore unsupervised pre-training for speech recognition by learning representations of raw audio. wav2vec is trained on large amounts of unlabeled audio data and the resulting representations are then used to improve acoustic model training. We pre-train a simple multi-layer convolutional neural network optimized via a noise contrastive binary classification task. Our experiments on WSJ reduce WER of a strong character-based log-mel filterbank baseline by up to 36% when only a few hours of transcribed data is available. Our approach achieves 2.43% WER on the nov92 test set. This outperforms Deep Speech 2, the best reported character-based system in the literature while using two orders of magnitude less labeled training data.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 of 9 cited sources checked · 0 supported · 0 flagged
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Thereafter, performance plateaus while training time increases.”
§ WAV2VEC: UNSUPERVISED PRE-TRAINING FOR SPEECH RECOGNITION > 4.3 ABLATIONS
Strength:plain· not yet calibratedn = 10Tests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Table 3 shows that increasing the number of negative samples only helps up to ten samples.”
§ WAV2VEC: UNSUPERVISED PRE-TRAINING FOR SPEECH RECOGNITION > 4.3 ABLATIONS
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Table 1 shows that pre-training on more data leads to better accuracy on the WSJ benchmark.”
§ WAV2VEC: UNSUPERVISED PRE-TRAINING FOR SPEECH RECOGNITION > 4.1 PRE-TRAINING FOR THE WSJ BENCHMARK
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“On TIMIT, pre-training enables us to match the best reported result in the literature.”
- Also stated in: § 4.2 PRE-TRAINING FOR TIMIT · plain
“Table 2 shows that wav2vec pre-training on Librispeech and WSJ audio data can lead to results matching the state of the art.”
§ WAV2VEC: UNSUPERVISED PRE-TRAINING FOR SPEECH RECOGNITION > 1 INTRODUCTION
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes- Also stated in: § 4.2 PRE-TRAINING FOR TIMIT · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Pretrained representations can substantially improve performance over our character-based baseline which is trained on log-mel filterbank features.”
§ WAV2VEC: UNSUPERVISED PRE-TRAINING FOR SPEECH RECOGNITION > 4.1 PRE-TRAINING FOR THE WSJ BENCHMARK
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Table 4 shows that a crop size of 150k frames results in the best performance.”
§ WAV2VEC: UNSUPERVISED PRE-TRAINING FOR SPEECH RECOGNITION > 4.3 ABLATIONS
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We found it important to choose a normalization scheme that is invariant to the scaling and the offset of the input.This choice resulted in representations that generalize well across datasets.”
§ WAV2VEC: UNSUPERVISED PRE-TRAINING FOR SPEECH RECOGNITION > 2 PRE-TRAINING APPROACH
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Experimental results on the WSJ benchmark demonstrate that pre-trained representations estimated on about 1,000 hours of unlabeled speech can substantially improve a character-based ASR system and outperform the best character-based result in the literature, Deep Speech 2, improving WER from 3.1 % to 2.43 %.”
§ WAV2VEC: UNSUPERVISED PRE-TRAINING FOR SPEECH RECOGNITION > 1 INTRODUCTION
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes- Rests on an assumption: Unlabeled audio data is much easier to collect than labeled data.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Pre-training only on the audio data of WSJ (wav2vec WSJ) performs worse compared to the much larger Librispeech (wav2vec Libri).”
§ WAV2VEC: UNSUPERVISED PRE-TRAINING FOR SPEECH RECOGNITION > 4.1 PRE-TRAINING FOR THE WSJ BENCHMARK
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIn the paper
“Similar tovan den Oord et al. (2018), we found that sampling negatives from different sequences and speakers yields inferior results.”
§ WAV2VEC: UNSUPERVISED PRE-TRAINING FOR SPEECH RECOGNITION > 5 CONCLUSIONS
What the sources say
- Not checked yetcites van den Oord et al., 2018
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
- It may state two separate claims as one. 8
- The quote only partly states this claim. 1
- Probably not a substantial claim. 3
In the paper
“Our approach achieves 2.43 % WER on the nov92 test set.This outperforms Deep Speech 2, the best reported character-based system in the literature while using two orders of magnitude less labeled training data.”
- Also stated in: § 5 CONCLUSIONS · plain
“Our approach achieves 2.43 % WER on the test set of WSJ, a result that outperforms the next best known character-based speech recognition model in the literature (Amodei et al., 2016) while using two orders of magnitude less transcribed training data.”
- Also stated in: § 4.1 PRE-TRAINING FOR THE WSJ BENCHMARK · boosted
“This shows that pre-training on unlabeled audio data can improve over the best character-based approach, Deep Speech 2 (Amodei et al., 2016), by 0.67 WER on nov92.”
§ WAV2VEC: UNSUPERVISED PRE-TRAINING FOR SPEECH RECOGNITION > Abstract
What the sources say
- Not checked yetcites Amodei et al., 2016
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one. The quote only partly states this claim.
- Also stated in: § 5 CONCLUSIONS · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We found it important to introduce skip connections in the aggregator to help convergence in this case.”
§ WAV2VEC: UNSUPERVISED PRE-TRAINING FOR SPEECH RECOGNITION > 2 PRE-TRAINING APPROACH
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“In comparison to Hadian et al. (2018), wav2vec performs as well as their phoneme-based model and wav2vec large outperforms it by 0.37 WER.”
§ WAV2VEC: UNSUPERVISED PRE-TRAINING FOR SPEECH RECOGNITION > 4.1 PRE-TRAINING FOR THE WSJ BENCHMARK
What the sources say
- Not checked yetcites Hadian et al., 2018
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We show that more data for pre-training improves performance and that this approach not only improves resource-poor setups, but also settings where all WSJ training data is used.”
§ WAV2VEC: UNSUPERVISED PRE-TRAINING FOR SPEECH RECOGNITION > 5 CONCLUSIONS
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
- Rests on an assumption: Unlabeled audio data is much easier to collect than labeled data.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Table 5 also shows that predicting more than 12 steps ahead in the future does not result in better performance and increasing the number of steps increases training time.”
§ WAV2VEC: UNSUPERVISED PRE-TRAINING FOR SPEECH RECOGNITION > 5 CONCLUSIONS
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
In the paper
“Similar to Hénaff et al. (2019), we noticed that fine-tuning the embedding network does not meaningfully improve performance while substantially increasing the acoustic model training time.”
§ WAV2VEC: UNSUPERVISED PRE-TRAINING FOR SPEECH RECOGNITION > 4.1 PRE-TRAINING FOR THE WSJ BENCHMARK
What the sources say
- Not checked yetcites Hénaff et al., 2019
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Accuracy steadily increases with more data for pre-training and the best accuracy is achieved with the largest amount of data for pre-training.”
§ WAV2VEC: UNSUPERVISED PRE-TRAINING FOR SPEECH RECOGNITION > 4.2 PRE-TRAINING FOR TIMIT
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Not restricting the maximum length (None) gives an average sequence length of about 207k frames and results in the worst accuracy.”
§ WAV2VEC: UNSUPERVISED PRE-TRAINING FOR SPEECH RECOGNITION > 4.3 ABLATIONS
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“wav2vec relies on a fully convolutional architecture which can be easily parallelized over time on modern hardware compared to recurrent models used in previous work ( §2).”
§ WAV2VEC: UNSUPERVISED PRE-TRAINING FOR SPEECH RECOGNITION > 1 INTRODUCTION
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“In this experiment, everything is kept equal except for the number of negative samples.”
§ WAV2VEC: UNSUPERVISED PRE-TRAINING FOR SPEECH RECOGNITION > 4.3 ABLATIONS
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
The paper relies on this without support.
Unlabeled audio data is much easier to collect than labeled data.
“This enables exploiting unlabeled audio data which is much easier to collect than labeled data.”
The paper relies on this without support.
Accurately modeling the data distribution p(x) is a common and challenging problem for these approaches.
“A common problem with these approaches is the requirement to accurately model the data distribution p(x), which is challenging.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“As in [15], utilizing the causal convolutional aggregator results in a fully convolutional model, increasing the parallelizability of the pre-training, resulting in faster training times.”
From Investigating Enhancements to Contrastive Predictive Coding for Human Activity Recognition (2023) · cites this paper as [15]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“we propose to instead utilize multiple causal convolution blocks to perform the autoregressive summarization, due to its simplicity, faster training times, and ease of parallelization that recurrent models do not enjoy.”
From Investigating Enhancements to Contrastive Predictive Coding for Human Activity Recognition (2023) · cites this paper as [15]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Note that wav2vec directly trains on speech’s raw waveform to extract deep features that represent speech signals.”
From A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face (2023) · cites this paper as [54]
The cited source says what the claim attributes to it.
In the citing paper
“For example, Schneider et al. [54] proposed an unsupervised speech pre-training method named wav2vec that only uses unlabeled speech data for self-supervised training to obtain generalized speech features that can be applied to multiple tasks.”
From A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face (2023) · cites this paper as [54]
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533
Natural Language Processing (almost) from ScratchPreprint
Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., & Kuksa, P. P. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1103.0398 · Cited by 6,920
Self-citationRepresentation Learning with Contrastive Predictive Coding
Oord, A. V. D., Li, Y., & Vinyals, O. (2018) · arXiv · doi:10.48550/arxiv.1807.03748 · Cited by 6,242
“Librispeech: an ASR corpus based on public domain audio books,”
Panayotov, V., Chen, G., Povey, D., & Khudanpur, S. (2015) · 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2015.7178964 · Cited by 6,072
SGDR: Stochastic Gradient Descent with Warm Restarts
Loshchilov, I., & Hutter, F. (2016) · arXiv · doi:10.48550/arxiv.1608.03983 · Cited by 5,309
Unsupervised Visual Representation Learning by Context Prediction
Doersch, C., Gupta, A., & Efros, A. A. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.167 · Cited by 2,629
Flagged first, then by citations
Cited by
Sorted by citations
Items 101–200 of 1,003
Non-Autoregressive Predictive Coding for Learning Speech Representations from Local Dependencies
Liu, A. H., Chung, Y.-A., & Glass, J. R. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-349 · Cited by 93
Cross-modal Contrastive Learning for Speech Translation
Ye, R., Wang, M., & Li, L. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.376 · Cited by 86
Kotzen, K., Charlton, P. H., Salabi, S., Amar, L., Landesberg, A., & Behar, J. A. (2022) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2022.3225363 · Cited by 80
SpeechStew: Simply Mix All Available Speech Recognition Data to Train One Large Neural Network
Chan, W., Park, D., Lee, C., Zhang, Y., Le, Q., & Norouzi, M. (2021) · arXiv · doi:10.48550/arxiv.2104.02133 · Cited by 79
Zhao, J. S., & Zhang, W.-Q. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3184480 · Cited by 84
Jiang, D., Li, W., Cao, M. Y., Zou, W., & Li, X. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-391 · Cited by 77
Explainable Unsupervised Machine Learning for Cyber-Physical Systems
Wickramasinghe, C. S., Amarasinghe, K., Marino, D., Rieger, C. G., & Manic, M. (2021) · IEEE Access · doi:10.1109/access.2021.3112397 · Cited by 82
Liu, Z., Kang, X., & Ren, F. (2023) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2023.3282092 · Cited by 76
Self-Supervised Contrastive Learning for Unsupervised Phoneme Segmentation
Kreuk, F., Keshet, J., & Adi, Y. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-2398 · Cited by 82
Enabling Resource-Efficient AIoT System With Cross-Level Optimization: A Survey
Liu, S., Guo, B., Fang, C., Wang, Z., Luo, S., Zhou, Z., & Yu, Z. (2023) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2023.3319952 · Cited by 74
Speech Emotion Recognition via CNN-Transformer and multidimensional attention mechanism
Tang, X., Huang, J., Lin, Y., Dang, T., & Cheng, J. (2025) · Speech Communication · doi:10.1016/j.specom.2025.103242 · Cited by 76
SpeechFormer: A Hierarchical Efficient Framework Incorporating the Characteristics of Speech
Chen, W., Xing, X., Xu, X., Pang, J., & Du, L. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-74 · Cited by 82
UniSpeech: Unified Speech Representation Learning with Labeled and Unlabeled DataPreprint
Wang, C., Wu, Y., Qian, Y., Kumatani, K., Liu, S., Wei, F., Zeng, M., & Huang, X. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.07597 · Cited by 76
Automatic Speech Recognition: A survey of deep learning techniques and approaches
Ahlawat, H., Aggarwal, N., & Gupta, D. (2025) · International Journal of Cognitive Computing in Engineering · doi:10.1016/j.ijcce.2024.12.007 · Cited by 86
Speaker Normalization for Self-Supervised Speech Emotion Recognition
Gat, I., Aronowitz, H., Zhu, W., Morais, E. D. S., & Hoory, R. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747460 · Cited by 69
Evain, S. V., Nguyen, H., Le, H., Boito, M. Z., Mdhaffar, S., Alisamir, S., Tong, Z., Tomashenko, N., Dinarelli, M., Parcollet, T., et al. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-556 · Cited by 78
FINDINGS OF THE IWSLT 2021 EVALUATION CAMPAIGN
Anastasopoulos, A., Bojar, O., Bremerman, J., Cattoni, R., Elbayad, M., Federico, M., Ma, X., Nakamura, S., Negri, M., Niehues, J., et al. (2021) · International Conference on Spoken Language Translation (IWSLT) · doi:10.18653/v1/2021.iwslt-1.1 · Cited by 72
Transformers in speech processing: Overcoming challenges and paving the future
Latif, S., Zaidi, S. A. M., Cuayáhuitl, H., Shamshad, F., Shoukat, M., Usama, M., & Qadir, J. (2025) · Computer Science Review · doi:10.1016/j.cosrev.2025.100768 · Cited by 116
Investigating Self-Supervised Learning for Speech Enhancement and Separation
Huang, Z., Watanabe, S., Yang, S.-W., Garcia, L. P., & Khudanpur, S. P. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746303 · Cited by 72
Active Contrastive Learning of Audio-Visual Video RepresentationsPreprint
Ma, S., Zeng, Z., McDuff, D., & Song, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.09805 · Cited by 67
Nguyen, T. A., de Seyssel, M., Rozé, P., Rivière, M., Kharitonov, E., Baevski, A., Dunbar, E., & Emmanuel Dupoux, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.11588 · Cited by 94
Self-citationMultimodal Pretraining, Adaptation, and Generation for Recommendation: A Survey
Liu, Q., Zhu, J., Yang, Y., Dai, Q., Du, Z., Wu, X.-M., Zhao, Z., Zhang, R., & Dong, Z. (2024) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3637528.3671473 · Cited by 83
Efficient Adapter Transfer of Self-Supervised Speech Models for Automatic Speech Recognition
Thomas, B., Kessler, S., & Karout, S. A. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746223 · Cited by 68
Semi-Supervised Time Series Classification by Temporal Relation Prediction
Fan, H., Zhang, F., Wang, R., Huang, X., & Li, Z. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9413883 · Cited by 64
SLAM: A Unified Encoder for Speech and Language Modeling via Speech-Text Joint Pre-TrainingPreprint
Bapna, A., Chung, Y.-A., Wu, N., Gulati, A., Ye, J., Clark, J. H., Johnson, M., Riesa, J., Conneau, A., & Zhang, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.10329 · Cited by 95
Towards Domain-Agnostic Contrastive LearningPreprint
Verma, V., Luong, M.-T., Kawaguchi, K., Pham, H., & Le, Q. V. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.04419 · Cited by 61
SPLAT: Speech-Language Joint Pre-Training for Spoken Language Understanding
Chung, Y.-A., Zhu, C., & Zeng, M. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.152 · Cited by 66
Macary, M., Tahon, M., Estève, Y., & Rousseau, A. (2021) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt48900.2021.9383456 · Cited by 64
Wav2KWS: Transfer Learning From Speech Representations for Keyword Spotting
Seo, D., Oh, H., & Jung, Y. (2021) · IEEE Access · doi:10.1109/access.2021.3078715 · Cited by 65
DeCoAR 2.0: Deep Contextualized Acoustic Representations with Vector QuantizationPreprint
Ling, S., & Liu, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.06659 · Cited by 60
End-to-End Speech Translation via Cross-Modal Progressive Training
Ye, R., Wang, M., & Li, L. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-1065 · Cited by 59
The Dawn of the Human-Machine Era: A forecast of new and emerging language technologies
Sayers, D., Sousa‐Silva, R., Höhn, S., Ahmedi, L., Allkivi-Metsoja, K., Anastasiou, D., Beňuš, Š., Bowker, L., Bytyçi, E., Catalá, A., et al. (2021) · doi:10.17011/jyx/reports/20210518/1 · Cited by 64
Deep Multimodal Emotion Recognition on Human Speech: A Review
Koromilas, P., & Giannakopoulos, T. (2021) · Applied Sciences · doi:10.3390/app11177962 · Cited by 58
Speech-XLNet: Unsupervised Acoustic Model Pretraining for Self-Attention Networks
Song, X., Wang, G., Huang, Y., Wu, Z., Su, D., & Meng, H. M. L. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-1511 · Cited by 60
Hsieh, T.-A., Yu, C., Fu, S., Lu, X., & Tsao, Y. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-582 · Cited by 58
Spoken Moments: Learning Joint Audio-Visual Representations from Video Descriptions
Monfort, M., Jin, S., Liu, A. H., Harwath, D. F., Feris, R., Glass, J. R., & Oliva, A. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01463 · Cited by 63
Self-supervised Learning for Electroencephalogram: A Systematic Survey
Weng, W., Gu, Y., Guo, S., Ma, Y., Yang, Z., Liu, Y., & Chen, Y. (2025) · ACM Computing Surveys · doi:10.1145/3736574 · Cited by 76
Exploration of Whisper fine-tuning strategies for low-resource ASR
Liu, Y., Yang, X., & Qu, D. (2024) · EURASIP Journal on Audio Speech and Music Processing · doi:10.1186/s13636-024-00349-3 · Cited by 56
CDPAM: Contrastive Learning for Perceptual Audio Similarity
Manocha, P., Jin, Z., Zhang, R., & Finkelstein, A. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9413711 · Cited by 56
Divide and Contrast: Self-supervised Learning from Uncurated Data
Tian, Y., Hénaff, O. J., & van den Oord, A. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00991 · Cited by 69
Improved Speech Representations with Multi-Target Autoregressive Predictive Coding
Chung, Y.-A., & Glass, J. R. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.213 · Cited by 55
Wav2vec-Switch: Contrastive Learning from Original-Noisy Speech Pairs for Robust Speech Recognition
Wang, Y., Li, J., Wang, H., Qian, Y., Wang, C., & Wu, Y. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746929 · Cited by 55
An Exploration of Self-Supervised Pretrained Representations for End-to-End Speech Recognition
Chang, X., Maekaku, T., Guo, P., Shi, J. Y., Lu, Y.-J., Subramanian, A. S., Wang, T., Yang, S.-W., Tsao, Y., Lee, H.-Y., et al. (2021) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru51503.2021.9688137 · Cited by 64
Lai, C.-I., Chuang, Y.-S., Lee, H.-Y., Li, S.-W. D., & Glass, J. R. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414922 · Cited by 60
Combining cross-modal knowledge transfer and semi-supervised learning for speech emotion recognition
Zhang, S., Chen, M., Chen, J., Li, Y.-F., Wu, Y., Li, M., & Zhu, C. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.107340 · Cited by 54
Pinheiro, G. A., Silva, J. L. F. D., & Quiles, M. G. (2022) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.2c00521 · Cited by 54
Overview of Voice Conversion Methods Based on Deep Learning
Walczyna, T., & Piotrowski, Z. (2023) · Applied Sciences · doi:10.3390/app13053100 · Cited by 54
Privacy-preserving Voice Analysis via Disentangled Representations
Aloufi, R., Haddadi, H., & Boyle, D. (2020) · Cloud Computing Security Workshop · doi:10.1145/3411495.3421355 · Cited by 54
Text-Free Image-to-Speech Synthesis Using Learned Segmental Units
Hsu, W.-N., Harwath, D. F., Miller, T., Song, C., & Glass, J. R. (2021) · doi:10.18653/v1/2021.acl-long.411 · Cited by 65
SLUE: New Benchmark Tasks For Spoken Language Understanding Evaluation on Natural Speech
Shon, S., Pasad, A., Wu, F., Brusco, P., Artzi, Y., Livescu, K., & Han, K. J. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746137 · Cited by 60
Zhu, Q., Zhang, J., Zhang, Z., & Dai, L.-R. (2023) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2023.3275033 · Cited by 57
SpeechLM: Enhanced Speech Pre-Training With Unpaired Textual Data
Zhang, Z., Chen, S., Zhou, L., Wu, Y., Ren, S., Liu, S., Yao, Z., Gong, X., Dai, L.-R., Li, J., et al. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3379877 · Cited by 72
Text-Free Prosody-Aware Generative Spoken Language Modeling
Kharitonov, E., Lee, A., Polyak, A., Adi, Y., Copet, J., Lakhotia, K., Nguyen, T. A., Rivière, M., Mohamed, A., Emmanuel Dupoux, A., et al. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.593 · Cited by 53
SUPERB: Speech processing Universal PERformance Benchmark
Yang, S.-W., Chi, P.-H., Chuang, Y.-S., Lai, C.-I. J., Lakhotia, K., Lin, Y. Y., Liu, A. T., Shi, J., Chang, X., Lin, G.-T., et al. (2021) · arXiv · doi:10.48550/arxiv.2105.01051 · Cited by 52
Contrastive Learning Inverts the Data Generating Process
Zimmermann, R. S., Sharma, Y., Schneider, S., Bethge, M., & Brendel, W. (2021) · arXiv · doi:10.48550/arxiv.2102.08850 · Cited by 52
Self-citationSpeech Recognition and Multi-Speaker Diarization of Long Conversations
Mao, H. H., Li, S. X., McAuley, J., & Cottrell, G. W. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-3039 · Cited by 58
Multimodal Pre-Training Based on Graph Attention Network for Document Understanding
Zhang, Z., Ma, J., Du, J., Wang, L., & Zhang, J. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3214102 · Cited by 58
LiRA: Learning Visual Speech Representations from Audio Through Self-Supervision
Ma, P., Mira, R., Petridis, S., Schuller, B. W., & Pantić, M. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-1360 · Cited by 50
Leveraging Unimodal Self-Supervised Learning for Multimodal Audio-Visual Speech Recognition
Pan, X., Chen, P., Gong, Y., Zhou, H., Wang, X., & Lin, Z. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.308 · Cited by 53
Reitmaier, T., Wallington, E., Raju, D. K., Klejch, O., Pearson, J. S., Jones, M., Bell, P., & Robinson, S. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517639 · Cited by 56
Towards Learning Universal Audio Representations
Wang, L., Luc, P., Wu, Y., Recasens, A., Smaira, L., Brock, A., Jaegle, A., Alayrac, J.-B., Dieleman, S., Carreira, J., et al. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746790 · Cited by 50
Disentangled Graph Variational Auto-Encoder for Multimodal Recommendation With Interpretability
Zhou, X., & Miao, C. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2024.3369875 · Cited by 59
A Large-Scale Evaluation of Speech Foundation Models
Yang, S.-W., Chang, H.-J., Huang, Z., Liu, A. T., Lai, C.-I., Wu, H., Shi, J., Chang, X., Tsai, H.-S., Huang, W.-C., et al. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3389631 · Cited by 57
Unsupervised Concept Drift Detection From Deep Learning Representations in Real-Time
Greco, S., Vacchetti, B., Apiletti, D., & Cerquitelli, T. (2025) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2025.3593123 · Cited by 49
Introduction to latent variable energy-based models: a path toward autonomous machine intelligence
Dawid, A., & LeCun, Y. (2024) · Journal of Statistical Mechanics Theory and Experiment · doi:10.1088/1742-5468/ad292b · Cited by 82
Federated Learning for Audio Semantic Communication
Tong, H., Yang, Z., Wang, S., Hu, Y., Semiari, O., Saad, W., & Yin, C. (2021) · Frontiers in Communications and Networks · doi:10.3389/frcmn.2021.734402 · Cited by 49
A conformer-based classifier for variable-length utterance processing in anti-spoofing
Rosello, E., Gomez-Alanis, A., Gomez, A. M., & Peinado, A. (2023) · INTERSPEECH 2023 · doi:10.21437/interspeech.2023-1820 · Cited by 47
Bayesian Surprise Predicts Human Event Segmentation in Story Listening
Kumar, M., Goldstein, A., Michelmann, S., Zacks, J. M., Hasson, U., & Norman, K. A. (2023) · Cognitive Science · doi:10.1111/cogs.13343 · Cited by 51
Towards Multimodal Disinformation Detection by Vision-language Knowledge Interaction
Li, Q., Gao, M., Zhang, G., Zhai, W., Chen, J., & Jeon, G. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.102037 · Cited by 49
Self-Supervised Learning with Cross-Modal Transformers for Emotion Recognition
Khare, A., Parthasarathy, S., & Sundaram, S. (2021) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt48900.2021.9383618 · Cited by 47
wav2vec-C: A Self-Supervised Model for Speech Representation Learning
Sadhu, S., He, D., Huang, C.-W., Mallidi, S. H., Wu, M., Rastrow, A., Stolcke, A., Droppo, J., & Maas, R. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-717 · Cited by 46
Self-supervised Contrastive Cross-Modality Representation Learning for Spoken Question Answering
You, C., Chen, N., & Zou, Y. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.3 · Cited by 46
Transferable Multimodal Attack on Vision-Language Pre-training Models
Wang, H., Dong, K., Zhu, Z., Qin, H., Liu, A., Fang, X., Wang, J., & Liu, X. (2024) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp54263.2024.00102 · Cited by 53
Towards Unsupervised Speech Recognition and Synthesis with Quantized Speech Representation Learning
Liu, A. H., Tu, T., Lee, H.-Y., & Lee, L.-S. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9053571 · Cited by 45
Zuluaga-Gómez, J., Prasad, A., Nigmatulina, I., Sarfjoo, S. S., Motlíček, P., Kleinert, M., Helmke, H., Ohneiser, O., & Zhan, Q. (2023) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt54892.2023.10022724 · Cited by 45
A Survey on Self-supervised Pre-training for Sequential Transfer Learning in Neural NetworksPreprint
Mao, H. H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.00800 · Cited by 66
Deep neural networks for automatic speech processing: a survey from large corpora to limited data
Roger, V., Farinas, J., & Pinquier, J. (2022) · EURASIP Journal on Audio Speech and Music Processing · doi:10.1186/s13636-022-00251-w · Cited by 44
Huang, Y.-J., Lin, Y.-T., Liu, C.-C., Lee, L.-E., Hung, S.-H., Lo, J.-K., & Fu, L.-C. (2022) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2022.3163777 · Cited by 46
VQTTS: High-Fidelity Text-to-Speech Synthesis with Self-Supervised VQ Acoustic Feature
Du, C., Guo, Y., Chen, X., & Yu, K. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-489 · Cited by 56
AIA-Net: Adaptive Interactive Attention Network for Text–Audio Emotion Recognition
Zhang, T., Li, S., Chen, B., Yuan, H., & Chen, C. L. P. (2022) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2022.3195739 · Cited by 47
XLSR-Mamba: A Dual-Column Bidirectional State Space Model for Spoofing Attack Detection
Xiao, Y., & Das, R. K. (2025) · IEEE Signal Processing Letters · doi:10.1109/lsp.2025.3547861 · Cited by 54
Deep Vision Multimodal Learning: Methodology, Benchmark, and Trend
Chai, W., & Wang, G. (2022) · Applied Sciences · doi:10.3390/app12136588 · Cited by 48
Improving Speech Recognition Performance in Noisy Environments by Enhancing Lip Reading Accuracy
Li, D., Gao, Y., Zhu, C., Wang, Q., & Wang, R. (2023) · Sensors · doi:10.3390/s23042053 · Cited by 43
Hong, J. H., Kim, M., Choi, J., & Ro, Y. M. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01801 · Cited by 43
Sainburg, T., & Gentner, T. Q. (2021) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2021.811737 · Cited by 48
Boosting Self-Supervised Embeddings for Speech Enhancement
Hung, K.-H., Fu, S., Tseng, H.-H., Chiang, H.-T., Tsao, Y., & Lin, C. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-10002 · Cited by 43
Facial Emotion Recognition with Inter-Modality-Attention-Transformer-Based Self-Supervised Learning
Chaudhari, A., Bhatt, C., Krishna, A., & Travieso, C. M. (2023) · Electronics · doi:10.3390/electronics12020288 · Cited by 44
Pretraining Techniques for Sequence-to-Sequence Voice Conversion
Huang, W.-C., Hayashi, T., Wu, Y.-C., Kameoka, H., & Toda, T. (2021) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2021.3049336 · Cited by 46
Multilingual Speech Translation with Efficient Finetuning of Pretrained ModelsPreprint
Li, X., Wang, C., Tang, Y., Tran, C., Tang, Y., Pino, J., Baevski, A., Conneau, A., & Auli, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.12829 · Cited by 60
Self-citationZolnoori, M., Zolnour, A., & Topaz, M. (2023) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2023.102624 · Cited by 43
Self-Supervised Representations Improve End-to-End Speech Translation
Wu, A., Wang, C., Pino, J., & Gu, J. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-3094 · Cited by 45
Does Visual Self-Supervision Improve Learning of Speech Representations for Emotion Recognition?
Shukla, A., Petridis, S., & Pantić, M. (2021) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2021.3062406 · Cited by 42
Audio ALBERT: A Lite BERT for Self-supervised Learning of Audio RepresentationPreprint
Chi, P.-H., Chung, P.-H., Wu, T.-H., Hsieh, C.-C., Chen, Y.-H., Li, S.-W. D., & Lee, H.-Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.08575 · Cited by 41
Self-Supervised Learning for Time-Series Anomaly Detection in Industrial Internet of Things
Tran, D., Nguyen, V., Nguyen, H., & Jang, Y. (2022) · Electronics · doi:10.3390/electronics11142146 · Cited by 33
Deep Representation Learning: Fundamentals, Technologies, Applications, and Open Challenges
Payandeh, A., Baghaei, K. T., Fayyazsanavi, P., Ramezani, S. B., Chen, Z., & Rahimi, S. (2023) · IEEE Access · doi:10.1109/access.2023.3335196 · Cited by 49
Leveraging Knowledge of Modality Experts for Incomplete Multimodal Learning
Xu, W., Jiang, H., & Liang, X. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681683 · Cited by 44
Investigating Self-Supervised Pre-Training for End-to-End Speech Translation
Nguyen, H., Bougares, F., Tomashenko, N., Estève, Y., & Besacier, L. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-1835 · Cited by 41
Fusion-ConvBERT: Parallel Convolution and BERT Fusion for Speech Emotion Recognition
Lee, S., Han, D. K., & Ko, H. (2020) · Sensors · doi:10.3390/s20226688 · Cited by 39
Improving speech recognition models with small samples for air traffic control systems
Lin, Y., Li, Q., Yang, B., Yan, Z., Tan, H., & Chen, Z. (2021) · Neurocomputing · doi:10.1016/j.neucom.2020.08.092 · Cited by 41
An Iterative Framework for Self-Supervised Deep Speaker Representation Learning
Cai, D., Wang, W., & Li, M. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414713 · Cited by 43
Metadata
- Authors
- Schneider, S., Baevski, A., Collobert, R., & Auli, M.
- Affiliations
- Technical University of Munich (DE)
- Meta (Israel) (IL)
- Venue
- Interspeech 2019
- Publisher
- ISCA
- Published
- 2019-09-15
- Pages
- 3465-3469
- ISSN
- 2308-457X (ISSN-L)
- OpenAlex
- W2939710050
- Cited by
- 1,290
- References
- 32
- Citations by year
- 2026: 86 · 2025: 169 · 2024: 222 · 2023: 262 · 2022: 221 · 2021: 225 · 2020: 97 · 2019: 7
- FWCI
- 52.61
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- public-domain
- Locations
- Interspeech (USB) (publishedVersion)
- arXiv (Cornell University) (submittedVersion, public-domain)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Physical Sciences › Computer Science › Signal Processing › Music and Audio Processing
- Keywords
- wav2vec, unsupervised pre-training, speech recognition, audio representation learning, noise contrastive estimation, acoustic model training, convolutional neural networks, Wall Street Journal
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-09-28, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-09-28 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
