LSTM time and frequency recurrence for automatic speech recognition
The DOI matches its Crossref record; 30 of the 30 references it cites were found in a registry or an index.
Abstract
Long short-term memory (LSTM) recurrent neural networks (RNNs) have recently shown significant performance improvements over deep feed-forward neural networks (DNNs). A key aspect of these models is the use of time recurrence, combined with a gating architecture that ameliorates the vanishing gradient problem. Inspired by human spectrogram reading, in this paper we propose an extension to LSTMs that performs the recurrence in frequency as well as in time. This model first scans the frequency bands to generate a summary of the spectral information, and then uses the output layer activations as the input to a traditional time LSTM (T-LSTM). Evaluated on a Microsoft short message dictation task, the proposed model obtained a 3.6% relative word error rate reduction over the T-LSTM.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“neglecting to model the emotion-related correlations within frequency domain during the time-frequency joint learning”
From Speech Emotion Recognition via an Attentive Time–Frequency Neural Network (2023) · cites this paper as [17]
Experimental — accuracy not yet measuredIn the citing paper
“recent spectrogram-based works mainly focus on modeling the long-term dependency in time domain”
From Speech Emotion Recognition via an Attentive Time–Frequency Neural Network (2023) · cites this paper as [17]
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Speech recognition with deep recurrent neural networks
Graves, A., Mohamed, A.-R., & Hinton, G. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638947 · Cited by 8,893 · cited 1×
“These networks have been shown to outperform DNNs on a variety of ASR tasks [11][12][13] [14][15][16].”
BackgroundSelf-citationLearning long-term dependencies with gradient descent is difficult
Bengio, Y., Simard, P., & Frasconi, P. (1994) · IEEE Transactions on Neural Networks · doi:10.1109/72.279181 · Cited by 8,542 · cited 1×
“However, the simple RNN suffers from the vanishing/exploding gradient problem [21] when the error signal is back-propagated through time.”
BackgroundLearning to forget: Continual prediction with LSTM
Gers, F. A., Schmidhuber, J., & Cummins, F. (2000) · Neural Computation · doi:10.1162/089976600300015015 · Cited by 4,823
Long short-term memory recurrent neural network architectures for large scale acoustic modeling
Sak, H., Senior, A., & Beaufays, F. (2014) · Interspeech 2014 · doi:10.21437/interspeech.2014-80 · Cited by 2,973
Convolutional Neural Networks for Speech Recognition
Abdel-Hamid, O., Mohamed, A.-R., Jiang, H., Deng, L., Penn, G., & Yu, D. (2014) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2014.2339736 · Cited by 2,262
Self-citationHybrid speech recognition with Deep Bidirectional LSTM
Graves, A., Jaitly, N., & Mohamed, A.-R. (2013) · 2013 IEEE Workshop on Automatic Speech Recognition and Understanding · doi:10.1109/asru.2013.6707742 · Cited by 1,736
Self-citationAcoustic Modeling Using Deep Belief Networks
Mohamed, A.-R., Dahl, G. E., & Hinton, G. (2012) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.1109/tasl.2011.2109382 · Cited by 1,727
Self-citationConvolutional, Long Short-Term Memory, fully connected Deep Neural Networks
Sainath, T. N., Vinyals, O., Senior, A., & Sak, H. (2015) · 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2015.7178838 · Cited by 1,678
Connectionist Speech Recognition: A Hybrid Approach
Bourlard, H. A., & Morgan, N. H. (1993) · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · Cited by 1,115
Cited by
Deep Learning for Audio Signal Processing
Purwins, H., Li, B., Virtanen, T., Schluter, J., Chang, S.-Y., & Sainath, T. (2019) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2019.2908700 · Cited by 855
Recent Advances in Recurrent Neural Networks
Salehinejad, H., Sankar, S., Barfett, J., Colak, E., & Valaee, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.01078 · Cited by 562
An LSTM Short-Term Solar Irradiance Forecasting Under Complicated Weather Conditions
Yu, Y., Cao, J., & Zhu, J. (2019) · IEEE Access · doi:10.1109/access.2019.2946057 · Cited by 313
Music Source Separation With Band-Split RNN
Luo, Y., & Yu, J. (2023) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2023.3271145 · Cited by 282
Recent progresses in deep learning based acoustic models
Yu, D., & Li, J. (2017) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2017.7510508 · Cited by 171
Self-citation
Metadata
- Authors
- Li, J., Mohamed, A., Zweig, G., & Gong, Y.
- Affiliations
- Microsoft (United States) (US)
- ORCID
- Venue
- 2015 IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU)
- Publisher
- IEEE
- Published
- 2015-12
- Pages
- 187-191
- OpenAlex
- W2291522532
- Semantic Scholar
- 4795ad039abc56799f770a01a264df9141dd0b2f
- Cited by
- 119
- References
- 30
- Citations by year
- 2026: 3 · 2025: 5 · 2024: 10 · 2023: 8 · 2022: 20 · 2021: 20 · 2020: 17 · 2019: 18 · 2018: 6 · 2017: 8 · 2016: 4
- FWCI
- 3.38
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- long short-term memory, speech recognition, gating mechanism, gradient vanishing, word error rate reduction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: LSTM time and frequency recurrence for automatic speech recognition. https://app.cochat.ai/papers/10.1109/asru.2015.7404793
- The paper
- Li, J., Mohamed, A., Zweig, G., & Gong, Y. (2015). LSTM time and frequency recurrence for automatic speech recognition. 2015 IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU). https://doi.org/10.1109/asru.2015.7404793
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.