Conformer: Convolution-augmented Transformer for Speech Recognition
Open accessAbstract
Recently Transformer and Convolution neural network (CNN) based models have shown promising results in Automatic Speech Recognition (ASR), outperforming Recurrent neural networks (RNNs). Transformer models are good at capturing content-based global interactions, while CNNs exploit local features effectively. In this work, we achieve the best of both worlds by studying how to combine convolution neural networks and transformers to model both local and global dependencies of an audio sequence in a parameter-efficient way. To this regard, we propose the convolution-augmented transformer for speech recognition, named Conformer. Conformer significantly outperforms the previous Transformer and CNN based models achieving state-of-the-art accuracies. On the widely used LibriSpeech benchmark, our model achieves WER of 2.1%/4.3% without using a language model and 1.9%/3.9% with an external language model on test/testother. We also observe competitive performance of 2.7%/6.3% with a small model of only 10M parameters.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Transformer and recurrent neural-network transducer models containing up to one billion parameters have produced a marked decrease in word error rate for automated transcription of spoken English-language sentences.
“In particular, transformer1 and recurrent neural-network transducer (RNNT)12,13,16 models containing up to one billion parameters2 have produced a marked decrease in word error rate (WER) (and therefore much better accuracy) for the automated transcription of spoken English-language sentences, as shown in Fig. 1a for two widely used datasets, Librispeech17 and SwitchBoard18.”
From An analog-AI chip for energy-efficient speech recognition and transcription (2023) · cites this paper as [16]
The chosen baseline acoustic models (hybrid DNN, hybrid LF-MMI TDNN, and E2E Conformer) give state-of-the-art performance on the four tasks.
“A range of acoustic models that give state-of-the-art performance on these tasks are chosen as the baseline speech recognition systems, including hybrid DNN [41], hybrid lattice-free maximum mutual information (LF-MMI) trained time delay neural network (TDNN) [3] and end-to-end (E2E) Conformer [6] models.”
From Speaker adaptation using spectro-temporal deep features for dysarthric and elderly speech recognition (2022) · cites this paper as [6]
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
Self-citationAdam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
Squeeze-and-Excitation NetworksPreprint
Hu, J., Shen, L., & Sun, G. (2017) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.48550/arxiv.1709.01507 · Cited by 37,845 · see citing works
(2014) · Cited by 33,419 · see citing works
“Librispeech: an ASR corpus based on public domain audio books,”
Panayotov, V., Chen, G., Povey, D., & Khudanpur, S. (2015) · 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2015.7178964 · Cited by 6,072 · see citing works
SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition
Park, D. S., Chan, W., Zhang, Y., Chiu, C.-C., Zoph, B., Cubuk, E. D., & Le, Q. V. (2019) · Interspeech 2019 · doi:10.21437/interspeech.2019-2680 · Cited by 3,558 · see citing works
Self-citationTransformer-XL: Attentive language models beyond a fixed-length context
Dai, Z., Yang, Z., Yang, Y., Carbonell, J., Le, Q., & Salakhutdinov, R. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1285 · Cited by 3,169 · see citing works
Language Modeling with Gated Convolutional NetworksPreprint
Dauphin, Y., Fan, A., Auli, M., & Grangier, D. (2016) · International Conference on Machine Learning · doi:10.48550/arxiv.1612.08083 · Cited by 3,097 · see citing works
Searching for activation functions
Ramachandran, P., Zoph, B., & Le, Q. V. (2017) · arXiv · doi:10.48550/arxiv.1710.05941 · Cited by 2,335 · see citing works
Convolutional Neural Networks for Speech Recognition
Abdel‐Hamid, O., Mohamed, A., Jiang, H., Deng, L., Penn, G., & Yu, D. (2014) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2014.2339736 · Cited by 2,268 · see citing works
Sequence Transduction with Recurrent Neural NetworksPreprint
Graves, A. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1211.3711 · Cited by 1,320 · see citing works
Attention Augmented Convolutional Networks
Bello, I., Zoph, B., Le, Q. V., Vaswani, A., & Shlens, J. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00338 · Cited by 1,320 · see citing works
QANet: Combining Local Convolution with Global Self-Attention for Reading ComprehensionPreprint
Yu, A. W., Dohan, D., Luong, M.-T., Zhao, R., Chen, K., Norouzi, M., & Le, Q. V. (2018) · International Conference on Learning Representations · doi:10.48550/arxiv.1804.09541 · Cited by 1,169 · see citing works
Speech-Transformer: A No-Recurrence Sequence-to-Sequence Model for Speech Recognition
Dong, L., Xu, S., & Xu, B. (2018) · 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2018.8462506 · Cited by 1,100 · see citing works
Deep convolutional neural networks for LVCSR
Sainath, T. N., Mohamed, A., Kingsbury, B. E. D., & Ramabhadran, B. (2013) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2013.6639347 · Cited by 1,066 · see citing works
A Comparative Study on Transformer vs RNN in Speech Applications
Karita, S., Chen, N., Hayashi, T., Hori, T., Inaguma, H., Jiang, Z., Someki, M., Yalta, N., Yamamoto, R., Wang, X., et al. (2019) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru46091.2019.9003750 · Cited by 766 · see citing works
Learning Deep Transformer Models for Machine Translation
Wang, Q., Li, B., Xiao, T., Zhu, J., Li, C., Wong, D. F., & Chao, L. S. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1176 · Cited by 673 · see citing works
Pay Less Attention with Lightweight and Dynamic ConvolutionsPreprint
Wu, F., Fan, A., Baevski, A., Dauphin, Y., & Auli, M. (2019) · International Conference on Learning Representations · doi:10.48550/arxiv.1901.10430 · Cited by 669 · see citing works
Streaming End-to-end Speech Recognition for Mobile Devices
He, Y., Sainath, T. N., Prabhavalkar, R., McGraw, I., Alvarez, R., Zhao, D., Rybach, D., Kannan, A., Wu, Y., Pang, R., et al. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8682336 · Cited by 601 · see citing works
Self-citationZhang, Q., Han, L., Sak, H., Tripathi, A., McDermott, E., Koo, S., & Kumar, S. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9053896 · Cited by 462 · see citing works
Rao, K., Sak, H., & Prabhavalkar, R. (2017) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru.2017.8268935 · Cited by 351 · see citing works
Han, W., Zhang, Z., Zhang, Y., Yu, J., Chiu, C., Qin, J., Gulati, A., Pang, R., & Wu, Y. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-2059 · Cited by 290 · see citing works
Self-citationQuartznet: Deep Automatic Speech Recognition with 1D Time-Channel Separable Convolutions
Kriman, S., Beliaev, S., Ginsburg, B., Huang, J., Kuchaiev, O., Lavrukhin, V., Leary, R., Li, J., & Zhang, Y. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9053889 · Cited by 283 · see citing works
Self-citationJasper: An End-to-End Convolutional Neural Acoustic Model
Li, J., Lavrukhin, V., Ginsburg, B., Leary, R., Kuchaiev, O., Cohen, J. M., Nguyen, H., & Gadde, R. T. (2019) · Interspeech (USB) · doi:10.21437/interspeech.2019-1819 · Cited by 248 · see citing works
Transformer-Based Acoustic Modeling for Hybrid Speech Recognition
Wang, Y., Mohamed, A., Le, D., Liu, C., Xiao, A., Mahadeokar, J., Huang, H., Tjandra, A., Zhang, X., Zhang, F., et al. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9054345 · Cited by 237 · see citing works
A Streaming On-Device End-To-End Model Surpassing Server-Side Conventional Model Quality and Latency
Sainath, T. N., He, Y., Li, B., Narayanan, A., Pang, R., Bruguier, A., Chang, S.-Y., Li, W., Álvarez, R., Chen, Z., et al. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9054188 · Cited by 188 · see citing works
Self-citationLingvo: a Modular and Scalable Framework for Sequence-to-Sequence ModelingPreprint
Shen, J., Nguyen, P., Wu, Y., Chen, Z., Chen, M. X., Ye, J., Kannan, A., Sainath, T. N., Cao, Y. J., Chiu, C., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.08295 · Cited by 169 · see citing works
Self-citationUnderstanding and Improving Transformer From a Multi-Particle Dynamic System Point of ViewPreprint
Lu, Y., Li, Z., He, D., Sun, Z., Dong, B., Qin, T., Wang, L., & Liu, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.02762 · Cited by 146 · see citing works
Specaugment on Large Scale Datasets
Park, D., Zhang, Y., Chiu, C., Chen, Y., Li, B., Chan, W., Le, Q. V., & Wu, Y. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9053205 · Cited by 145 · see citing works
Self-citationAn analysis of noise in recurrent neural networks: convergence and generalization
Jim, K.-C., Giles, C. L., & Horne, B. G. (1996) · IEEE Transactions on Neural Networks · doi:10.1109/72.548170 · Cited by 133 · see citing works
Convolutional Self-Attention Networks
Yang, B., Wang, L., Wong, D. F., Chao, L. S., & Tu, Z. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1407 · Cited by 128 · see citing works
Transformers without Tears: Improving the Normalization of Self-AttentionPreprint
Nguyen, T. Q., & Salazar, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.05895 · Cited by 124 · see citing works
Flagged first, then by citations
1 of the 33 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units
Hsu, W.-N., Bolte, B., Tsai, Y.-H. H., Lakhotia, K., Salakhutdinov, R., & Mohamed, A. (2021) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2021.3122291 · Cited by 2,983 · see citing works
CvT: Introducing Convolutions to Vision Transformers
Wu, H., Xiao, B., Codella, N., Liu, M., Dai, X., Yuan, L., & Zhang, L. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00009 · Cited by 2,649 · see citing works
WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing
Chen, S., Wang, C., Chen, Z., Wu, Y., Liu, S., Chen, Z., Li, J., Kanda, N., Yoshioka, T., Xiao, X., et al. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3188113 · Cited by 1,926 · see citing works
Self-citationLin, T., Wang, Y., Liu, X., & Qiu, X. (2022) · AI Open · doi:10.1016/j.aiopen.2022.10.001 · Cited by 1,422 · see citing works
AST: Audio Spectrogram Transformer
Gong, Y., Chung, Y.-A., & Glass, J. (2021) · Interspeech 2021 · doi:10.21437/interspeech.2021-698 · Cited by 1,159 · see citing works
Rethinking Attention with Performers
Choromanski, K., Likhosherstov, V., Dohan, D., Song, X., Gane, A., Sarlos, T., Hawkins, P., Davis, J., Mohiuddin, A., Kaiser, L., et al. (2020) · arXiv · doi:10.48550/arxiv.2009.14794 · Cited by 695 · see citing works
“Several solutions have been proposed to address this issue (Beltagy et al., 2020; Gulati et al., 2020; Chan et al., 2020; Child et al., 2019; Bello et al., 2019).”
BackgroundEgo4D: Around the World in 3,000 Hours of Egocentric Video
Grauman, K., Westbury, A., Byrne, E., Chavis, Z., Furnari, A., Girdhar, R., Hamburger, J., Jiang, H., Liu, M., Liu, X., et al. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01842 · Cited by 630 · see citing works
Ding, Y., Jia, M., Miao, Q., & Cao, Y. (2022) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2021.108616 · Cited by 559 · see citing works
A comprehensive survey on applications of transformers for deep learning tasks
Islam, S., Elmekki, H., Elsebai, A., Bentahar, J., Drawel, N., Rjoub, G., & Witold Pedrycz, C. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.122666 · Cited by 522 · see citing works
Audiolm: A language modeling approach to audio generation
Borsos, Z., Marinier, R., Vincent, D., Kharitonov, E., Pietquin, O., Sharifi, M., Roblek, D., Teboul, O., Grangier, D., Tagliasacchi, M., et al. (2023) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2023.3288409 · Cited by 432 · see citing works
Recent Advances in End-to-End Automatic Speech Recognition
Li, J. (2022) · APSIPA Transactions on Signal and Information Processing · doi:10.1561/116.00000050 · Cited by 411 · see citing works
Emotion Recognition from Speech Using wav2vec 2.0 Embeddings
Pepino, L., Riera, P., & Ferrer, L. (2021) · Interspeech 2021 · doi:10.21437/interspeech.2021-703 · Cited by 389 · see citing works
Deep learning in food category recognition
Zhang, Y., Deng, L., Zhu, H., Wang, W., Ren, Z., Zhou, Q., Lu, S., Sun, S., Zhu, Z., Górriz, J. M., et al. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101859 · Cited by 393 · see citing works
Self-citationA review of deep learning techniques for speech processing
Mehrish, A., Majumder, N., Bharadwaj, R., Mihalcea, R., & Poria, S. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101869 · Cited by 389 · see citing works
A survey of sound source localization with deep learning methods
Grumiaux, P.-A., Kitić, S., Laurent Girin, B., & Guérin, A. (2022) · The Journal of the Acoustical Society of America · doi:10.1121/10.0011809 · Cited by 365 · see citing works
MetaFormer Baselines for Vision
Yu, W., Si, C., Zhou, P., Luo, M., Zhou, Y., Feng, J., Yan, S., & Wang, X. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3329173 · Cited by 355 · see citing works
Lin, K., Wang, L., & Liu, Z. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01270 · Cited by 339 · see citing works
Deep Learning Enabled Semantic Communications With Speech Recognition and Synthesis
Weng, Z., Qin, Z., Tao, X., Pan, C., Liu, G., & Li, G. Y. (2023) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2023.3240969 · Cited by 287 · see citing works
Chung, Y.-A., Zhang, Y., Han, W., Chiu, C.-C., Qin, J., Pang, R., & Wu, Y. (2021) · 2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) · doi:10.1109/asru51503.2021.9688253 · Cited by 293 · see citing works
Self-citationWayformer: Motion Forecasting via Simple & Efficient Attention Networks
Nayakanti, N., Al‐Rfou, R., Zhou, A., Goel, K., Refaat, K. S., & Sapp, B. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10160609 · Cited by 278 · see citing works
An analog-AI chip for energy-efficient speech recognition and transcription
Ambrogio, S., Narayanan, P., Okazaki, A., Fasoli, A., Mackin, C., Hosokawa, K., Nomura, A., Yasuda, T., Chen, A., Friz, A., et al. (2023) · Nature · doi:10.1038/s41586-023-06337-5 · Cited by 242 · see citing works
“In particular, transformer 1 and recurrent neural-network transducer (RNNT) 12,13,16 models containing up to one billion parameters 2 have produced a marked decrease in word error rate (WER) (and therefore much better accuracy) for the automated transcription of spoken English-language sentences,…”
BackgroundEnd-To-End Audio-Visual Speech Recognition with Conformers
Ma, P., Petridis, S., & Pantić, M. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414567 · Cited by 240 · see citing works
WeNet: Production Oriented Streaming and Non-Streaming End-to-End Speech Recognition Toolkit
Yao, Z., Wu, D., Wang, X., Zhang, B., Yu, F., Yang, C., Peng, Z., Chen, X., Xie, L., & Lei, X. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-1983 · Cited by 236 · see citing works
GigaSpeech: An Evolving, Multi-Domain ASR Corpus with 10,000 Hours of Transcribed Audio
Chen, G., Chai, S., Wang, G., Du, J., Zhang, W.-Q., Weng, C., Su, D., Povey, D., Trmal, J., Zhang, J., et al. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-1965 · Cited by 234 · see citing works
End-to-End Speech Recognition: A Survey
Prabhavalkar, R., Hori, T., Sainath, T., Schlűter, R., & Watanabe, S. (2023) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2023.3328283 · Cited by 348 · see citing works
Han, S.-Y., Shao, H., Cheng, J., Yang, X., & Cai, B. (2022) · IEEE/ASME Transactions on Mechatronics · doi:10.1109/tmech.2022.3199985 · Cited by 221 · see citing works
When LLMs meet cybersecurity: a systematic literature review
Zhang, J., Bu, H., Wen, H., Liu, Y., Fei, H., Xi, R., Li, L., Yang, Y., Zhu, H., & Meng, D. (2025) · Cybersecurity · doi:10.1186/s42400-025-00361-w · Cited by 220 · see citing works
“It compares the original code with LLM-generated code to flag notable differences as anomalies for further inspection.”
BrainGB: A Benchmark for Brain Network Analysis with Graph Neural Networks
Cui, H., Dai, W., Zhu, Y., Kan, X., Gu, A. A. C., Lukemire, J., Zhan, L., He, L., Guo, Y., & Yang, C. (2023) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2022.3218745 · Cited by 218 · see citing works
FLEURS: FEW-Shot Learning Evaluation of Universal Representations of Speech
Conneau, A., Ma, M., Khanuja, S., Zhang, Y., Axelrod, V., Dalmia, S., Riesa, J., Rivera, C. E., & Bapna, A. (2023) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt54892.2023.10023141 · Cited by 211 · see citing works
Self-citationPushing the Limits of Semi-Supervised Learning for Automatic Speech RecognitionPreprint
Zhang, Y., Qin, J., Park, D., Han, W., Chiu, C., Pang, R., Le, Q. V., & Wu, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.10504 · Cited by 233 · see citing works
Self-citationRecent Developments on Espnet Toolkit Boosted By Conformer
Guo, P., Boyer, F., Chang, X., Hayashi, T., Higuchi, Y., Inaguma, H., Kamo, N., Li, C., Garcia-Romero, D., Shi, J., et al. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414858 · Cited by 223 · see citing works
Self-citationImages Speak in Images: A Generalist Painter for In-Context Visual Learning
Wang, X., Wang, W., Cao, Y., Shen, C., & Huang, T. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00660 · Cited by 201 · see citing works
Listen, Denoise, Action! Audio-Driven Motion Synthesis with Diffusion Models
Alexanderson, S., Nagy, R., Beskow, J., & Henter, G. E. (2023) · ACM Transactions on Graphics · doi:10.1145/3592458 · Cited by 206 · see citing works
Time-series anomaly detection with stacked Transformer representations and 1D convolutional network
Kim, J., Kang, H., & Kang, P. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.105964 · Cited by 187 · see citing works
Unsupervised Speech RecognitionPreprint
Baevski, A., Hsu, W.-N., Conneau, A., & Auli, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.11084 · Cited by 191 · see citing works
Robust wav2vec 2.0: Analyzing Domain Shift in Self-Supervised Pre-Training
Hsu, W.-N., Sriram, A., Baevski, A., Likhomanenko, T., Xu, Q., Pratap, V., Kahn, J., Lee, A., Collobert, R., Synnaeve, G., et al. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-236 · Cited by 191 · see citing works
Pay attention to what you read: Non-recurrent handwritten text-Line recognition
Kang, L., Riba, P., Rusiñol, M., Fornés, A., & Villegas, M. (2022) · Pattern Recognition · doi:10.1016/j.patcog.2022.108766 · Cited by 196 · see citing works
Developing Real-Time Streaming Transformer Transducer for Speech Recognition on Large-Scale Dataset
Chen, X., Wu, Y., Wang, Z., Liu, S., & Li, J. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9413535 · Cited by 175 · see citing works
Self-citationVisual speech recognition for multiple languages in the wild
Ma, P., Petridis, S., & Pantić, M. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00550-z · Cited by 177 · see citing works
CMGAN: Conformer-Based Metric-GAN for Monaural Speech Enhancement
Abdulatif, S., Cao, R., & Yang, B. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3393718 · Cited by 170 · see citing works
Kang, H., & Kang, P. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.111507 · Cited by 174 · see citing works
Zhang, Y., Park, D. S., Han, W., Qin, J., Gulati, A., Shor, J., Jansen, A., Xu, Y., Huang, Y., Wang, S., et al. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3182537 · Cited by 161 · see citing works
Self-citationWENETSPEECH: A 10000+ Hours Multi-Domain Mandarin Corpus for Speech Recognition
Zhang, B., Lv, H., Guo, P., Shao, Q., Yang, C., Xie, L., Xu, X., Bu, H., Chen, X., Zeng, C., et al. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746682 · Cited by 156 · see citing works
Keyword Transformer: A Self-Attention Model for Keyword Spotting
Berg, A., O’Connor, M., & Cruz, M. T. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-1286 · Cited by 151 · see citing works
Do Transformers Really Perform Bad for Graph Representation?Preprint
Ying, C., Cai, T., Luo, S., Zheng, S., Ke, G., He, D., Shen, Y., & Liu, T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.05234 · Cited by 221 · see citing works
Wu, H., Triebe, M. J., & Sutherland, J. W. (2023) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2023.02.018 · Cited by 147 · see citing works
Transformer-based Image Compression
Lu, M., Guo, P., Shi, H., Cao, C., & Ma, Z. (2022) · DCC · doi:10.1109/dcc52660.2022.00080 · Cited by 142 · see citing works
Auto-AVSR: Audio-Visual Speech Recognition with Automatic Labels
Ma, P., Haliassos, A., Fernandez-Lopez, A., Chen, H., Petridis, S., & Pantić, M. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10096889 · Cited by 148 · see citing works
Wang, Y., Wang, L., & Siau, K. L. (2024) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2024.2316376 · Cited by 155 · see citing works
MFA-Conformer: Multi-scale Feature Aggregation Conformer for Automatic Speaker Verification
Zhang, Y., Lv, Z., Wu, H., Zhang, S., Hu, P., Wu, Z., Lee, H.-Y., & Meng, H. M. L. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-563 · Cited by 145 · see citing works
Self-citationOn the Comparison of Popular End-to-End Models for Large Scale Speech Recognition
Li, J., Wu, Y., Gaur, Y., Wang, C., Zhao, R., & Liu, S. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-2846 · Cited by 134 · see citing works
Self-citationShi, Y., Wang, Y., Wu, C., Yeh, C.-F., Chan, J., Zhang, F., Le, D., & Seltzer, M. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414560 · Cited by 139 · see citing works
Self-Training and Pre-Training are Complementary for Speech Recognition
Xu, Q., Baevski, A., Likhomanenko, T., Tomasello, P., Conneau, A., Collobert, R., Synnaeve, G., & Auli, M. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414641 · Cited by 139 · see citing works
SE-Conformer: Time-Domain Speech Enhancement Using Conformer
Kim, E., & Seo, H. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-2207 · Cited by 128 · see citing works
Intermediate Loss Regularization for CTC-Based Speech Recognition
Lee, J., & Watanabe, S. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414594 · Cited by 124 · see citing works
Using transformers for multimodal emotion recognition: Taxonomies and state of the art review
Hazmoune, S., & Bougamouza, F. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.108339 · Cited by 131 · see citing works
Gao, Z., Zhang, S., McLoughlin, I. V., & Yan, Z. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-9996 · Cited by 122 · see citing works
CMGAN: Conformer-based Metric GAN for Speech Enhancement
Cao, R., Abdulatif, S., & Yang, B. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-517 · Cited by 120 · see citing works
GenSLMs: Genome-scale language models reveal SARS-CoV-2 evolutionary dynamics
Zvyagin, M., Brace, A., Hippe, K., Deng, Y., Zhang, B., Bohorquez, C. O., Clyde, A., Kale, B., Perez-Rivera, D., Ma, H., et al. (2023) · The International Journal of High Performance Computing Applications · doi:10.1177/10943420231201154 · Cited by 120 · see citing works
Genomic language models: opportunities and challenges
Benegas, G., Ye, C., Albors, C., Li, J. C., & Song, Y. S. (2025) · Trends in Genetics · doi:10.1016/j.tig.2024.11.013 · Cited by 133 · see citing works
GenSLMs: Genome-scale language models reveal SARS-CoV-2 evolutionary dynamics
Zvyagin, M., Brace, A., Hippe, K., Deng, Y., Zhang, B., Bohorquez, C. O., Clyde, A., Kale, B., Perez-Rivera, D., Ma, H., et al. (2022) · doi:10.1101/2022.10.10.511571 · Cited by 120 · see citing works
Koluguri, N. R., Park, T., & Ginsburg, B. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746806 · Cited by 121 · see citing works
Wang, Q., Du, J., Wu, H., Pan, J., Ma, F., & Lee, C. (2023) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2023.3256088 · Cited by 121 · see citing works
Ding, Y., & Jia, M. (2022) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2022.3181933 · Cited by 120 · see citing works
Continuous Speech Separation with Conformer
Chen, S., Wu, Y., Chen, Z., Wu, J., Li, J., Yoshioka, T., Wang, C., Liu, S., & Zhou, M. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9413423 · Cited by 116 · see citing works
Self-citationSub-word Level Lip Reading With Visual Attention
R, P. K., Afouras, T., & Zisserman, A. (2021) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr52688.2022.00510 · Cited by 137 · see citing works
Chen, Z., Chen, J., Liu, S., Feng, Y., He, S., & Xu, E. (2022) · ISA Transactions · doi:10.1016/j.isatra.2022.04.043 · Cited by 114 · see citing works
Neural Architecture Search for Transformers: A Survey
Chitty-Venkata, K. T., Emani, M., Vishwanath, V., & Somani, A. K. (2022) · IEEE Access · doi:10.1109/access.2022.3212767 · Cited by 118 · see citing works
Audio self-supervised learning: A survey
Liu, S., Mallol-Ragolta, A., Parada‐Cabaleiro, E., Qian, K., Jing, X., Kathan, A., Hu, B., & Schuller, B. W. (2022) · Patterns · doi:10.1016/j.patter.2022.100616 · Cited by 116 · see citing works
FluentNet: End-to-End Detection of Stuttered Speech Disfluencies With Deep Learning
Kourkounakis, T., Hajavi, A., & Etemad, A. (2021) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2021.3110146 · Cited by 109 · see citing works
Prompting Large Language Models with Speech Recognition Abilities
Fathullah, Y., Wu, C., Lakomkin, E., Jia, J., Shangguan, Y., Li, K., Guo, J., Xiong, W., Mahadeokar, J., Kalinli, O., et al. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10447605 · Cited by 125 · see citing works
E-Branchformer: Branchformer with Enhanced Merging for Speech Recognition
Kim, K., Wu, F., Peng, Y., Pan, J. S., Sridhar, P., Han, K. J., & Watanabe, S. (2023) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt54892.2023.10022656 · Cited by 107 · see citing works
WeNet 2.0: More Productive End-to-End Speech Recognition Toolkit
Zhang, B., Wu, D., Peng, Z., Song, X., Yao, Z., Lv, H., Xie, L., Yang, C., Pan, F., & Niu, J. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-483 · Cited by 101 · see citing works
Multi-Modal Adaptive Fusion Transformer Network for the Estimation of Depression Level
Sun, H., Liu, J., Chai, S., Qiu, Z., Lin, L., Huang, X., & Chen, Y. (2021) · Sensors · doi:10.3390/s21144764 · Cited by 99 · see citing works
GLiT: Neural Architecture Search for Global and Local Image Transformer
Chen, B., Li, P., Li, C., Li, B., Bai, L., Chen, L., Sun, M., Yan, J., & Ouyang, W. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00008 · Cited by 100 · see citing works
M2Met: The Icassp 2022 Multi-Channel Multi-Party Meeting Transcription Challenge
Yu, F., Zhang, S., Fu, Y., Xie, L., Zheng, S., Du, Z., Huang, W., Guo, P., Yan, Z., Ma, B., et al. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746465 · Cited by 96 · see citing works
Fast Conformer With Linearly Scalable Attention For Efficient Speech Recognition
Rekesh, D., Koluguri, N. R., Kriman, S., Majumdar, S., Noroozi, V., Huang, H., Hrinchuk, O., Puvvada, K. C., Kumar, A., Balam, J., et al. (2023) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru57964.2023.10389701 · Cited by 96 · see citing works
A Better and Faster end-to-end Model for Streaming ASR
Li, B., Gulati, A., Yu, J., Sainath, T. N., Chiu, C., Narayanan, A., Chang, S.-Y., Pang, R., He, Y., Qin, J., et al. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9413899 · Cited by 94 · see citing works
Self-citationRouet‐Leduc, B., & Hulbert, C. (2024) · Nature Communications · doi:10.1038/s41467-024-47754-y · Cited by 94 · see citing works
SpeechFormer++: A Hierarchical Efficient Framework for Paralinguistic Speech Processing
Chen, W., Xing, X., Xu, X., Pang, J., & Du, L. (2023) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2023.3235194 · Cited by 96 · see citing works
Progressive graph convolution network for EEG emotion recognition
Zhou, Y., Li, F., Li, Y., Li, Y., Shi, G., Zheng, W., Zhang, L., Chen, Y., & Chen, Y. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126262 · Cited by 92 · see citing works
Ensemble Multifeatured Deep Learning Models and Applications: A Survey
Abimannan, S., El-Alfy, E.-S. M., Chang, Y., Hussain, S., Shukla, S., & Satheesh, D. (2023) · IEEE Access · doi:10.1109/access.2023.3320042 · Cited by 101 · see citing works
Frequency Dynamic Convolution: Frequency-Adaptive Pattern Recognition for Sound Event Detection
Nam, H., Kim, S.-H., Ko, B.-Y., & Park, Y. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-10127 · Cited by 87 · see citing works
Wang, L.-K., Zhang, M., Gao, X., & Shi, W.-Z. (2024) · Remote Sensing · doi:10.3390/rs16050804 · Cited by 91 · see citing works
MuIT: An End-to-End Multitask Learning Transformer
Bhattacharjee, D., Zhang, T., Susstrunk, S. E., & Salzmann, M. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01172 · Cited by 94 · see citing works
Efficient Conformer: Progressive Downsampling and Grouped Attention for Automatic Speech Recognition
Burchi, M., & Vielzeuf, V. (2021) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru51503.2021.9687874 · Cited by 83 · see citing works
Detecting Deepfake Voice Using Explainable Deep Learning Techniques
Lim, S.-Y., Chae, D.-K., & Lee, S.-C. (2022) · Applied Sciences · doi:10.3390/app12083926 · Cited by 82 · see citing works
Tang, J., Ma, Z., Gan, K., Zhang, J., & Yin, Z. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.102129 · Cited by 82 · see citing works
HINT: High-Quality INpainting Transformer With Mask-Aware Encoding and Enhanced Attention
Chen, S., Atapour–Abarghouei, A., & Shum, H. P. H. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2024.3369897 · Cited by 86 · see citing works
A generic non-invasive neuromotor interface for human-computer interaction
Kaifosh, P., Reardon, T. R., Labs, C.-L. A. R., Allen, B. D., Anderson, C., Arnoud, S., Arora, R., Atray, M., Awad, L., Ayerbe, F., et al. (2025) · Nature · doi:10.1038/s41586-025-09255-w · Cited by 138 · see citing works
Yang, Z., Zhou, D., Yang, Y., Zhang, J., & Chen, Z. (2022) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2022.3171973 · Cited by 80 · see citing works
Findings of the IWSLT 2022 Evaluation Campaign
Anastasopoulos, A., Barrault, L., Bentivogli, L., Boito, M. Z., Bojar, O., Cattoni, R., Currey, A., Dinu, G., Duh, K., Elbayad, M., et al. (2022) · International Conference on Spoken Language Translation (IWSLT) · doi:10.18653/v1/2022.iwslt-1.10 · Cited by 82 · see citing works
Mamba in Speech: Towards an Alternative to Self-Attention
Zhang, X., Zhang, Q., Liu, H., Xiao, T., Qian, X., Ahmed, B., Eliathamby Ambikairajah, A., Li, H., & Epps, J. (2025) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2025.3566210 · Cited by 96 · see citing works
SpeechStew: Simply Mix All Available Speech Recognition Data to Train One Large Neural Network
Chan, W., Park, D., Lee, C., Zhang, Y., Le, Q., & Norouzi, M. (2021) · arXiv · doi:10.48550/arxiv.2104.02133 · Cited by 79 · see citing works
Self-citationQuan, C., & Li, X. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3357036 · Cited by 78 · see citing works
Attention mechanisms and deep learning for machine vision: A survey of the state of the artPreprint
Hafiz, A. M., Parah, S. A., & Bhat, R. U. A. (2021) · Research Square · doi:10.21203/rs.3.rs-510910/v1 · Cited by 84 · see citing works
Zhao, S., & Ma, B. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10096646 · Cited by 77 · see citing works
Unsupervised Automatic Speech Recognition: A Review
Aldarmaki, H., Ullah, A., Ram, S., & Zaki, N. (2021) · Speech Communication · doi:10.1016/j.specom.2022.02.005 · Cited by 75 · see citing works
Zhao, J. S., & Zhang, W.-Q. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3184480 · Cited by 78 · see citing works
Shi, X. J., Yu, F., Lu, Y., Liang, Y., Feng, Q., Wang, D., Qian, Y., & Xie, L. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9413386 · Cited by 74 · see citing works
- Authors
- Gulati, A., Qin, J., Chiu, C.-C., Parmar, N., Zhang, Y., Yu, J., Han, W., Wang, S., Zhang, Z., Wu, Y., & Pang, R.
- Affiliations
- Google (United States) (US)
- ORCID
- Venue
- Interspeech 2020
- Publisher
- ISCA
- Published
- 2020-10-25
- Pages
- 5036-5040
- ISSN
- 2308-457X (ISSN-L)
- OpenAlex
- W3097777922
- Cited by
- 3,150
- References
- 33
- Citations by year
- 2026: 373 · 2025: 666 · 2024: 644 · 2023: 708 · 2022: 424 · 2021: 300 · 2020: 35
- FWCI
- 140.10
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- public-domain
- Locations
- Interspeech (USB) (publishedVersion)
- arXiv (Cornell University) (submittedVersion, public-domain)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Physical Sciences › Computer Science › Signal Processing › Music and Audio Processing
- Keywords
- conformational isomers, speech recognition, CNN-Transformer hybrid, parameter-efficient models, LibriSpeech, word error rate
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
