A time delay neural network architecture for efficient modeling of long temporal contexts
Abstract
Recurrent neural network architectures have been shown to efficiently model long term temporal dependencies between acoustic events. However the training time of recurrent networks is higher than feedforward networks due to the sequential nature of the learning algorithm. In this paper we propose a time delay neural network architecture which models long term temporal dependencies with training times comparable to standard feed-forward DNNs. The network uses sub-sampling to reduce computation during training. On the Switchboard task we show a relative improvement of 6% over the baseline DNN model. We present results on several LVCSR tasks with training data ranging from 3 to 1800 hours to show the effectiveness of the TDNN architecture in learning wider temporal dependencies in both small and large data scenarios.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The chosen baseline acoustic models (hybrid DNN, hybrid LF-MMI TDNN, and E2E Conformer) give state-of-the-art performance on the four tasks.
“A range of acoustic models that give state-of-the-art performance on these tasks are chosen as the baseline speech recognition systems, including hybrid DNN [41], hybrid lattice-free maximum mutual information (LF-MMI) trained time delay neural network (TDNN) [3] and end-to-end (E2E) Conformer [6] models.”
From Speaker adaptation using spectro-temporal deep features for dysarthric and elderly speech recognition (2022) · cites this paper as [3]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
“Librispeech: an ASR corpus based on public domain audio books,”
Panayotov, V., Chen, G., Povey, D., & Khudanpur, S. (2015) · 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2015.7178964 · Cited by 6,072 · see citing works
Self-citationPhoneme recognition using time-delay neural networks
Waibel, A., Hanazawa, T., Hinton, G., Shikano, K., & Lang, K. J. (1989) · IEEE Transactions on Acoustics, Speech, and Signal Processing · doi:10.1109/29.21701 · Cited by 2,643 · see citing works
SWITCHBOARD: telephone speech corpus for research and development
Godfrey, J. J., Holliman, E. C., & McDaniel, J. (1992) · [Proceedings] ICASSP-92: 1992 IEEE International Conference on Acoustics, Speech, and Signal Processing · doi:10.1109/icassp.1992.225858 · Cited by 2,134 · cited 5× · see citing works
“[7] J. Godfrey, E. Holliman, and J. McDaniel, “Switchboard: telephone speech corpus for research and development,” in Proceedings of the International Conference on Acoustics, Speech and Signal Processing (ICASSP), vol. 1, Mar 1992, pp. 517–520 vol.1.”
BackgroundMethodInfluential
Flagged first, then by citations
Cited by
Sorted by citations
Speaker recognition based on deep learning: An overview
Bai, Z., & Zhang, X.-L. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.03.004 · Cited by 386 · see citing works
Federated Learning for Keyword Spotting
Leroy, D., Coucke, A., Lavril, T., Gisselbrecht, T., & Dureau, J. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8683546 · Cited by 291 · see citing works
Geng, M., Xie, X., Ye, Z., Wang, T., Li, G., Hu, S., Liu, X., & Meng, H. (2022) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2022.3195113 · Cited by 48 · see citing works
Automatic Speech Assessment for People with Aphasia Using TDNN-BLSTM with Multi-Task Learning
Qin, Y., Lee, T., Feng, S., & Kong, A. P. H. (2018) · Interspeech 2018 · doi:10.21437/interspeech.2018-1630 · Cited by 30 · see citing works
Evaluating the Efficiency of Voice Control as Human Machine Interface in Production
Norda, M., Engel, C., Rennies, J., Appell, J.-E., Lange, S. C., & Hahn, A. (2024) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2023.3302951 · Cited by 26 · see citing works
Exploiting Cross Domain Acoustic-to-Articulatory Inverted Features for Disordered Speech Recognition
Hu, S., Liu, S., Xie, X., Geng, M., Wang, T., Hu, S., Cui, M., Liu, X., & Meng, H. (2022) · ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp43922.2022.9746989 · Cited by 14 · see citing works
Neural Architecture Search for LF-MMI Trained Time Delay Neural Networks
Hu, S., Xie, X., Liu, S., Cui, M., Geng, M., Liu, X., & Meng, H. (2021) · ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp39728.2021.9413630 · Cited by 6 · see citing works
- Authors
- Peddinti, V., Povey, D., & Khudanpur, S.
- Affiliations
- Johns Hopkins University (US)
- Venue
- Interspeech 2015
- Publisher
- ISCA
- Published
- 2015-09-06
- Pages
- 3214-3218
- ISSN
- 2308-457X (ISSN-L)
- OpenAlex
- W2402146185
- Cited by
- 986
- References
- 3
- Citations by year
- 2026: 13 · 2025: 36 · 2024: 67 · 2023: 89 · 2022: 101 · 2021: 171 · 2020: 142 · 2019: 128 · 2018: 139 · 2017: 64 · 2016: 27 · 2015: 9
- FWCI
- 44.89
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Interspeech (USB) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Keywords
- time-delay neural networks, long-range temporal dependencies, subsampling, training efficiency, large vocabulary continuous speech recognition, recurrent neural networks
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
