The DOI matches its OpenAlex record; 50 of the 50 references it cites were found in a registry or an index.
Deep Speech 2: End-to-End Speech Recognition in English and Mandarin
Open accessAbstract
We show that an end-to-end deep learning approach can be used to recognize either English or Mandarin Chinese speech--two vastly different languages. Because it replaces entire pipelines of hand-engineered components with neural networks, end-to-end learning allows us to handle a diverse variety of speech including noisy environments, accents and different languages. Key to our approach is our application of HPC techniques, resulting in a 7x speedup over our previous system. Because of this efficiency, experiments that previously took weeks now run in days. This enables us to iterate more quickly to identify superior architectures and algorithms. As a result, in several cases, our system is competitive with the transcription of human workers when benchmarked on standard datasets. Finally, using a technique called Batch Dispatch with GPUs in the data center, we show that our system can be inexpensively deployed in an online setting, delivering low latency when serving users at scale.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545
Bidirectional recurrent neural networks
Schuster, M., & Paliwal, K. K. (1997) · IEEE Transactions on Signal Processing · doi:10.1109/78.650093 · Cited by 10,172
Speech recognition with deep recurrent neural networks
Graves, A., Mohamed, A.-R., & Hinton, G. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638947 · Cited by 8,893
“Librispeech: an ASR corpus based on public domain audio books,”
Panayotov, V., Chen, G., Povey, D., & Khudanpur, S. (2015) · 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2015.7178964 · Cited by 6,072
Graves, A., Fernández, S., Gomez, F., & Schmidhuber, J. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143891 · Cited by 5,370
Bengio, Y., Louradour, J., Collobert, R., & Weston, J. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553380 · Cited by 5,366 · cited 1×
“In order to make training more stable, we experiment with a training curriculum (Bengio et al., 2009; Zaremba & Sutskever, 2014), which accelerates training and results in better generalization as well.”
MethodOn the difficulty of training recurrent neural networks
Pascanu, R., Mikolov, T., & Bengio, Y. (2012) · arXiv · doi:10.48550/arxiv.1211.5063 · Cited by 4,310
Cited by
Recent advances in convolutional neural networks
Gu, J., Wang, Z., Kuen, J., Ma, L., Shahroudy, A., Shuai, B., Liu, T., Wang, X., Wang, G., Cai, J., et al. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2017.10.013 · Cited by 6,576
Self-citationmixup: Beyond Empirical Risk Minimization
Zhang, H., Cisse, M., Dauphin, Y. N., & Lopez-Paz, D. (2017) · arXiv · doi:10.48550/arxiv.1710.09412 · Cited by 5,831
Hannun, A. Y., Rajpurkar, P., Haghpanahi, M., Tison, G. H., Bourn, C., Turakhia, M. P., & Ng, A. Y. (2019) · Nature Medicine · doi:10.1038/s41591-018-0268-3 · Cited by 3,148
Self-citationThreat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey
Akhtar, N., & Mian, A. (2018) · IEEE Access · doi:10.1109/access.2018.2807385 · Cited by 2,021
TabNet: Attentive Interpretable Tabular Learning
Arik, S. Ö., & Pfister, T. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i8.16826 · Cited by 1,771
Concrete Problems in AI Safety
Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., & Mané, D. (2016) · arXiv · doi:10.48550/arxiv.1606.06565 · Cited by 1,314
Self-citationwav2vec: Unsupervised Pre-Training for Speech Recognition
Schneider, S., Baevski, A., Collobert, R., & Auli, M. (2019) · Interspeech 2019 · doi:10.21437/interspeech.2019-1873 · Cited by 1,290
Micikevicius, P., Narang, S., Alben, J., Diamos, G., Elsen, E., Garcia, D., Ginsburg, B., Houston, M., Kuchaiev, O., Venkatesh, G., et al. (2017) · arXiv · doi:10.48550/arxiv.1710.03740 · Cited by 1,179
Self-citationLong Short-Term Memory Networks for Accurate State-of-Charge Estimation of Li-ion Batteries
Chemali, E., Kollmeyer, P. J., Preindl, M., Ahmed, R., & Emadi, A. (2018) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2017.2787586 · Cited by 906
Zhu, C., Han, S., Mao, H., & Dally, W. J. (2016) · arXiv · doi:10.48550/arxiv.1612.01064 · Cited by 865
ESE: Efficient Speech Recognition Engine with Sparse LSTM on FPGA
Han, S., Kang, J., Mao, H., Hu, Y., Li, X., Li, Y., Xie, D., Luo, H., Yao, S., Wang, Y., et al. (2017) · Proceedings of the 2017 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays · doi:10.1145/3020078.3021745 · Cited by 591
Self-citationFink, O., Wang, Q., Svensén, M., Dersin, P., Lee, W.-J., & Ducoffe, M. (2020) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2020.103678 · Cited by 549
The Computational Limits of Deep Learning
Thompson, N. C., Greenewald, K., Lee, K., & Manso, G. F. (2020) · arXiv · doi:10.48550/arxiv.2007.05558 · Cited by 538
Self-Supervised Representation Learning: Introduction, advances, and challenges
Ericsson, L., Gouk, H., Loy, C. C., & Hospedales, T. M. (2022) · IEEE Signal Processing Magazine · doi:10.1109/msp.2021.3134634 · Cited by 433
Efficient emotion recognition from speech using deep learning on spectrograms
Satt, A., Rozenberg, S., & Hoory, R. (2017) · Interspeech 2017 · doi:10.21437/interspeech.2017-200 · Cited by 422
She, C., Sun, C., Gu, Z., Li, Y., Yang, C., Poor, H. V., & Vucetic, B. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2021.3053601 · Cited by 405
AD-NeRF: Audio Driven Neural Radiance Fields for Talking Head Synthesis
Guo, Y., Chen, K., Liang, S., Liu, Y.-J., Bao, H., & Zhang, J. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00573 · Cited by 384
Towards Better Decoding and Language Model Integration in Sequence to Sequence Models
Chorowski, J., & Jaitly, N. (2017) · Interspeech 2017 · doi:10.21437/interspeech.2017-343 · Cited by 358
Compute Trends Across Three Eras of Machine Learning
Sevilla, J., Heim, L., Ho, A., Besiroglu, T., Hobbhahn, M., & Villalobos, P. (2022) · 2022 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn55064.2022.9891914 · Cited by 363
Murtaza, G., Shuib, L., Abdul Wahab, A. W., Mujtaba, G., Mujtaba, G., Nweke, H. F., Al-garadi, M. A., Zulfiqar, F., Raza, G., & Azmi, N. A. (2020) · Artificial Intelligence Review · doi:10.1007/s10462-019-09716-5 · Cited by 342
Rao, K., Sak, H., & Prabhavalkar, R. (2017) · 2017 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) · doi:10.1109/asru.2017.8268935 · Cited by 341
Shi, Z., Yao, W., Li, Z., Zeng, L., Zhao, Y., Zhang, R., Tang, Y., & Wen, J. (2020) · Applied Energy · doi:10.1016/j.apenergy.2020.115733 · Cited by 327
Triton: an intermediate language and compiler for tiled neural network computations
Tillet, P., Kung, H. T., & Cox, D. (2019) · Proceedings of the 3rd ACM SIGPLAN International Workshop on Machine Learning and Programming Languages · doi:10.1145/3315508.3329973 · Cited by 318
Neural Speech Recognizer: Acoustic-to-Word LSTM Model for Large Vocabulary Speech Recognition
Soltau, H., Liao, H., & Sak, H. (2017) · Interspeech 2017 · doi:10.21437/interspeech.2017-1566 · Cited by 304
OFF-ApexNet on micro-expression recognition system
Gan, Y. S., Liong, S.-T., Yau, W.-C., Huang, Y.-C., & Tan, L.-K. (2019) · Signal Processing: Image Communication · doi:10.1016/j.image.2019.02.005 · Cited by 303
[DL] A Survey of FPGA-based Neural Network Inference Accelerators
Guo, K., Zeng, S., Yu, J., Wang, Y., & Yang, H. (2019) · ACM Transactions on Reconfigurable Technology and Systems · doi:10.1145/3289185 · Cited by 292
Self-citationToward Human Parity in Conversational Speech Recognition
Xiong, W., Droppo, J., Huang, X., Seide, F., Seltzer, M. L., Stolcke, A., Yu, D., & Zweig, G. (2017) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2017.2756440 · Cited by 245
Deep Reinforcement Learning for Sequence-to-Sequence Models
Keneshloo, Y., Shi, T., Ramakrishnan, N., & Reddy, C. K. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2929141 · Cited by 230
Dentronics: Towards robotics and artificial intelligence in dentistry
Grischke, J., Johannsmeier, L., Eich, L., Griga, L., & Haddadin, S. (2020) · Dental Materials · doi:10.1016/j.dental.2020.03.021 · Cited by 221
Deep Residual Neural Networks for Audio Spoofing Detection
Alzantot, M., Wang, Z., & Srivastava, M. B. (2019) · Interspeech 2019 · doi:10.21437/interspeech.2019-3174 · Cited by 215
Self-citationQuantum Entanglement in Deep Learning Architectures
Levine, Y., Sharir, O., Cohen, N., & Shashua, A. (2019) · Physical Review Letters · doi:10.1103/physrevlett.122.065301 · Cited by 206
(2018) · doi:10.1145/3172944.3172977 · Cited by 186
Skin disease diagnosis with deep learning: a review
Li, H., Pan, Y., Zhao, J., & Zhang, L. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.08.096 · Cited by 178
A Study of BFLOAT16 for Deep Learning Training
Kalamkar, D., Mudigere, D., Mellempudi, N., Das, D., Banerjee, K., Avancha, S., Vooturi, D. T., Jammalamadaka, N., Huang, J., Yuen, H., et al. (2019) · arXiv · doi:10.48550/arxiv.1905.12322 · Cited by 169
Palaz, D., Magimai-Doss, M., & Collobert, R. (2019) · Speech Communication · doi:10.1016/j.specom.2019.01.004 · Cited by 168
A Survey on an Emerging Area: Deep Learning for Smart City Data
Chen, Q., Wang, W., Wu, F., De, S., Wang, R., Zhang, B., & Huang, X. (2019) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2019.2907718 · Cited by 167
$L1$ -Norm Batch Normalization for Efficient Training of Deep Neural Networks
Wu, S., Li, G., Deng, L., Liu, L., Wu, D., Xie, Y., & Shi, L. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2876179 · Cited by 166
Contrastive Predictive Coding for Human Activity Recognition
Haresamudram, H., Essa, I., & Plötz, T. (2021) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3463506 · Cited by 150
“The availability of large-scale annotated datasets has resulted in astonishing improvements in performance due to the application of deep learning to computer vision [34, 42], speech recognition [3, 26] and natural language tasks [18, 52].”
BackgroundBayesian Recurrent Neural Networks
Fortunato, M., Blundell, C., & Vinyals, O. (2017) · arXiv · doi:10.48550/arxiv.1704.02798 · Cited by 145
DeepHeart: Semi-Supervised Sequence Learning for Cardiovascular Risk Prediction
Ballinger, B., Hsieh, J., Singh, A., Sohoni, N., Wang, J., Tison, G., Marcus, G., Sanchez, J., Maguire, C., Olgin, J., et al. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11891 · Cited by 156
Multi-Modal Emotion recognition on IEMOCAP Dataset using Deep Learning
Tripathi, S., Tripathi, S., & Beigi, H. (2018) · arXiv · doi:10.48550/arxiv.1804.05788 · Cited by 132
Accelerating Deep Convolutional Networks using low-precision and sparsity
Venkatesh, G., Nurvitadhi, E., & Marr, D. (2017) · 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2017.7952679 · Cited by 123
Deep Learning: A Bayesian Perspective
Polson, N. G., & Sokolov, V. (2017) · Bayesian Analysis · doi:10.1214/17-ba1082 · Cited by 120
Machine learning powered ellipsometry
Liu, J., Zhang, D., Yu, D., Ren, M., & Xu, J. (2021) · Light: Science & Applications · doi:10.1038/s41377-021-00482-0 · Cited by 102
Adaptive deep learning model selection on embedded systems
Taylor, B., Marco, V. S., Wolff, W., Elkhatib, Y., & Wang, Z. (2018) · Proceedings of the 19th ACM SIGPLAN/SIGBED International Conference on Languages, Compilers, and Tools for Embedded Systems · doi:10.1145/3211332.3211336 · Cited by 90
Self-citation(2020) · neural information processing systems · Cited by 92
SearcHD: A Memory-Centric Hyperdimensional Computing With Stochastic Training
Imani, M., Yin, X., Messerly, J., Gupta, S., Niemier, M., Hu, X. S., & Rosing, T. (2020) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2019.2952544 · Cited by 87
End-to-End Federated Learning for Autonomous Driving Vehicles
Zhang, H., Bosch, J., & Olsson, H. H. (2021) · 2021 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9533808 · Cited by 84
Generative Language Modeling for Automated Theorem Proving
Polu, S., & Sutskever, I. (2020) · arXiv · doi:10.48550/arxiv.2009.03393 · Cited by 82
Lewis, J. K., Toubal, I. E., Chen, H., Sandesera, V., Lomnitz, M., Hampel-Arias, Z., Prasad, C., & Palaniappan, K. (2020) · 2020 IEEE Applied Imagery Pattern Recognition Workshop (AIPR) · doi:10.1109/aipr50011.2020.9425167 · Cited by 80
Towards relatable explainable AI with the perceptual process
Zhang, W., & Lim, B. Y. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501826 · Cited by 67
Rainey, S., Martin, S., Christen, A., Mégevand, P., & Fourneret, E. (2020) · Science and Engineering Ethics · doi:10.1007/s11948-020-00218-0 · Cited by 65
Low latency RNN inference with cellular batching
Gao, P., Yu, L., Wu, Y., & Li, J. (2018) · Proceedings of the Thirteenth EuroSys Conference · doi:10.1145/3190508.3190541 · Cited by 66
Demystifying differentiable programming: shift/reset the penultimate backpropagator
Wang, F., Zheng, D., Decker, J., Wu, X., Essertel, G. M., & Rompf, T. (2019) · Proceedings of the ACM on Programming Languages · doi:10.1145/3341700 · Cited by 59
Joshi, S., Villalba, J., Zelasko, P., Moro-Velazquez, L., & Dehak, N. (2021) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2021.3116438 · Cited by 53
Removing Backdoor-Based Watermarks in Neural Networks with Limited Data
Liu, X., Li, F., Wen, B., & Li, Q. (2021) · 2020 25th International Conference on Pattern Recognition (ICPR) · doi:10.1109/icpr48806.2021.9412684 · Cited by 49
Real-Time Machine Learning: The Missing Pieces
Nishihara, R., Moritz, P., Wang, S., Tumanov, A., Paul, W., Schleier-Smith, J., Liaw, R., Niknami, M., Jordan, M. I., & Stoica, I. (2017) · Proceedings of the 16th Workshop on Hot Topics in Operating Systems · doi:10.1145/3102980.3102998 · Cited by 49
On the Limit of English Conversational Speech Recognition
Tüske, Z., Saon, G., & Kingsbury, B. (2021) · Interspeech 2021 · doi:10.21437/interspeech.2021-211 · Cited by 41
QuClassi: A Hybrid Deep Neural Network Architecture based on Quantum State Fidelity
Stein, S. A., Baheri, B., Chen, D., Mao, Y., Guan, Q., Li, A., Xu, S., & Ding, C. (2021) · arXiv · doi:10.48550/arxiv.2103.11307 · Cited by 41
Deep learning for online harassment detection in tweets
Marwa, T., Salima, O., & Souham, M. (2018) · 2018 3rd International Conference on Pattern Analysis and Intelligent Systems (PAIS) · doi:10.1109/pais.2018.8598530 · Cited by 35
Adaptive e-Learning AI-Powered Chatbot based on Multimedia Indexing
Janati, S. E., Maach, A., & El, D. (2020) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2020.0111238 · Cited by 20
(2021) · doi:10.1145/3450268.3453521 · Cited by 19
E-BATCH: Energy-Efficient and High-Throughput RNN Batching
Silfa, F., Arnau, J. M., & González, A. (2022) · ACM Transactions on Architecture and Code Optimization · doi:10.1145/3499757 · Cited by 14
Joint Speech Recognition and Audio Captioning
Narisetty, C., Tsunoo, E., Chang, X., Kashiwagi, Y., Hentschel, M., & Watanabe, S. (2022) · ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp43922.2022.9746601 · Cited by 8
Metadata
- Authors
- Amodei, D., Anubhai, R., Battenberg, E., Case, C. T., Casper, J., Catanzaro, B., Chen, J., Chrzanowski, M., Coates, A., Diamos, G., Elsen, E., Engel, J., Fan, L., Fougner, C., Han, T. X., Hannun, A., Jun, B., LeGresley, P., Lin, L., Narang, S., Ng, A. Y., Ozair, S., Prenger, R., Raiman, J., Satheesh, S., Seetapun, D., Sengupta, S., Wang, Y., Wang, Z., Wang, C., Xiao, B. J., Yogatama, D., Zhan, J., & Zhu, Z.
- Venue
- arXiv (Cornell University)
- Publisher
- arXiv
- Published
- 2015
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1512.02595
- OpenAlex
- W2193413348
- Cited by
- 2,110
- References
- 50
- Type
- posted content
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
