The DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
Highway long short-term memory RNNS for distant speech recognition
Open accessAbstract
In this paper, we extend the deep long short-term memory (DL-STM) recurrent neural networks by introducing gated direct connections between memory cells in adjacent layers. These direct links, called highway connections, enable unimpeded information flow across different layers and thus alleviate the gradient vanishing problem when building deeper LSTMs. We further introduce the latency-controlled bidirectional LSTMs (BLSTMs) which can exploit the whole history while keeping the latency under control. Efficient algorithms are proposed to train these novel networks using both frame and sequence discriminative criteria. Experiments on the AMI distant speech recognition (DSR) task indicate that we can train deeper LSTMs and achieve better improvement from sequence training with highway LSTMs (HLSTMs). Our novel model obtains 43.9/47.7% WER on AMI (SDM) dev and eval sets, outperforming all previous works. It beats the strong DNN and DLSTM baselines with 15.7% and 5.3% relative improvement respectively.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Hinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A.-R., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., et al. (2012) · IEEE Signal Processing Magazine · doi:10.1109/msp.2012.2205597 · Cited by 10,260
Self-citationSpeech recognition with deep recurrent neural networks
Graves, A., Mohamed, A.-R., & Hinton, G. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638947 · Cited by 8,893 · cited 2×
“It is shown in [6, 7, 13] that bidirectional LSTN RNNs can indeed improve the speech recognition results.”
BackgroundMethodImproving neural networks by preventing co-adaptation of feature detectors
Hinton, G. E., Srivastava, N., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. R. (2012) · arXiv · doi:10.48550/arxiv.1207.0580 · Cited by 6,382
Kaldi Speech Recognition Toolkit
Povey, D. (2024) · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · doi:10.57702/jb3fvbn9 · Cited by 4,609
Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition
Dahl, G. E., Yu, D., Deng, L., & Acero, A. (2012) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2011.2134090 · Cited by 3,011 · cited 1×
“Recently the deep neural network (DNN)-based acoustic models (AMs) greatly improved automatic speech recognition (ASR) accuracy on many tasks [1, 2, 3, 4].”
BackgroundSelf-citationLong short-term memory recurrent neural network architectures for large scale acoustic modeling
Sak, H., Senior, A., & Beaufays, F. (2014) · Interspeech 2014 · doi:10.21437/interspeech.2014-80 · Cited by 2,973
Hybrid speech recognition with Deep Bidirectional LSTM
Graves, A., Jaitly, N., & Mohamed, A.-R. (2013) · 2013 IEEE Workshop on Automatic Speech Recognition and Understanding · doi:10.1109/asru.2013.6707742 · Cited by 1,736
Srivastava, R. K., Greff, K., & Schmidhuber, J. (2015) · arXiv · doi:10.48550/arxiv.1505.00387 · Cited by 912
Conversational speech transcription using context-dependent deep neural networks
Seide, F., Li, G., & Yu, D. (2011) · Interspeech 2011 · doi:10.21437/interspeech.2011-169 · Cited by 852
Self-citationAn investigation of deep neural networks for noise robust speech recognition
Seltzer, M. L., Yu, D., & Wang, Y. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6639100 · Cited by 640
Self-citationSeide, F., Li, G., Chen, X., & Yu, D. (2011) · 2011 IEEE Workshop on Automatic Speech Recognition & Understanding · doi:10.1109/asru.2011.6163899 · Cited by 632
Self-citationAn Efficient Gradient-Based Algorithm for On-Line Training of Recurrent Network Trajectories
Williams, R. J., & Peng, J. (1990) · Neural Computation · doi:10.1162/neco.1990.2.4.490 · Cited by 577
Unleashing the killer corpus: experiences in creating the multi-everything AMI Meeting Corpus
Carletta, J. (2007) · Language Resources and Evaluation · doi:10.1007/s10579-007-9040-x · Cited by 359
An Introduction to Computational Networks and the Computational Network Toolkit
Yu, D., Eversole, A., Seltzer, M., Yao, K., Kuchaiev, O., Zhang, Y., Seide, F. T. B., Huang, Z., Guenter, B. K., Wang, H., et al. (2014) · Cited by 332
Self-citationKalchbrenner, N., Danihelka, I., & Graves, A. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1507.01526 · Cited by 267
Convolutional Neural Networks for Distant Speech Recognition
Swietojanski, P., Ghoshal, A., & Renals, S. (2014) · IEEE Signal Processing Letters · doi:10.1109/lsp.2014.2325781 · Cited by 253
Depth-Gated LSTMPreprint
Yao, K., Cohn, T., Vylomova, K., Duh, K., & Dyer, C. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1508.03790 · Cited by 132
Self-citationConversational speech transcription using context-dependent deep neural networks
Yu, D., Seide, F. T. B., & Li, G. (2012) · International Conference on Machine Learning · Cited by 131
Self-citationKumatani, K., McDonough, J. W., & Raj, B. (2012) · IEEE Signal Processing Magazine · doi:10.1109/msp.2012.2205285 · Cited by 123
Transcribing Meetings With the AMIDA Systems
Hain, T., Burget, L., Dines, J., Garner, P. N., Grézl, F., Hannani, A. E., Huijbregts, M., Karafiát, M., Lincoln, M., & Wan, V. (2011) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2011.2163395 · Cited by 114
Hybrid acoustic models for distant and multichannel large vocabulary speech recognition
Świętojański, P., Ghoshal, A. K., & Renals, S. J. (2013) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru.2013.6707744 · Cited by 102
Training Deep Bidirectional LSTM Acoustic Model for LVCSR by a Context-Sensitive-Chunk BPTT Approach
Chen, K., & Huo, Q. (2016) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2016.2539499 · Cited by 101
Asynchronous stochastic optimization for sequence training of deep neural networks
Heigold, G., McDermott, E., Vanhoucke, V., Senior, A., & Bacchiani, M. (2014) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2014.6854672 · Cited by 64
Making themost from multiple microphones in meeting recognition
Stolcke, A. (2011) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2011.5947477 · Cited by 22
Cited by
Pixel Recurrent Neural Networks
van den Oord, A., Kalchbrenner, N., & Kavukcuoglu, K. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1601.06759 · Cited by 1,437
Neural Motifs: Scene Graph Parsing with Global Context
Zellers, R., Yatskar, M., Thomson, S., & Choi, Y. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00611 · Cited by 1,052
Training Deep Nets with Sublinear Memory Cost
Chen, T., Xu, B., Zhang, C., & Guestrin, C. (2016) · arXiv · doi:10.48550/arxiv.1604.06174 · Cited by 561
Semi-Orthogonal Low-Rank Matrix Factorization for Deep Neural Networks
Povey, D., Cheng, G., Wang, Y., Li, K., Xu, H., Yarmohammadi, M., & Khudanpur, S. (2018) · Interspeech 2018 · doi:10.21437/interspeech.2018-1417 · Cited by 446
Self-citationDeep Semantic Role Labeling: What Works and What’s Next
He, L., Lee, K., Lewis, M., & Zettlemoyer, L. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1044 · Cited by 420
Metadata
- Authors
- Zhang, Y., Chen, G., Yu, D., Yao, K., Khudanpur, S., & Glass, J.
- Venue
- 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
- Publisher
- IEEE
- Published
- 2016-03
- Pages
- 5755-5759
- ISSN
- 1520-6149 (ISSN-L)
- OpenAlex
- W2964084166
- Cited by
- 290
- References
- 25
- FWCI
- 29.04
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.