The DOI matches its Crossref record; 7 of the 7 references it cites were found in a registry or an index.
Deep Convolutional Neural Networks for Large-scale Speech Tasks
Abstract
Convolutional Neural Networks (CNNs) are an alternative type of neural network that can be used to reduce spectral variations and model spectral correlations which exist in signals. Since speech signals exhibit both of these properties, we hypothesize that CNNs are a more effective model for speech compared to Deep Neural Networks (DNNs). In this paper, we explore applying CNNs to large vocabulary continuous speech recognition (LVCSR) tasks. First, we determine the appropriate architecture to make CNNs effective compared to DNNs for LVCSR tasks. Specifically, we focus on how many convolutional layers are needed, what is an appropriate number of hidden units, what is the best pooling strategy. Second, investigate how to incorporate speaker-adapted features, which cannot directly be modeled by CNNs as they do not obey locality in frequency, into the CNN framework. Third, given the importance of sequence training for speech tasks, we introduce a strategy to use ReLU+dropout during Hessian-free sequence training of CNNs. Experiments on 3 LVCSR tasks indicate that a CNN with the proposed speaker-adapted and ReLU+dropout ideas allow for a 12%-14% relative improvement in WER over a strong DNN system, achieving state-of-the art results in these 3 tasks.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Hinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A.-R., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., et al. (2012) · IEEE Signal Processing Magazine · doi:10.1109/msp.2012.2205597 · Cited by 10,260
Self-citationImproving neural networks by preventing co-adaptation of feature detectors
Hinton, G. E., Srivastava, N., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. R. (2012) · arXiv · doi:10.48550/arxiv.1207.0580 · Cited by 6,382
Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition
Dahl, G. E., Yu, D., Deng, L., & Acero, A. (2012) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2011.2134090 · Cited by 3,011
Self-citationPhoneme recognition using time-delay neural networks
Waibel, A., Hanazawa, T., Hinton, G., Shikano, K., & Lang, K. J. (1989) · IEEE Transactions on Acoustics, Speech, and Signal Processing · doi:10.1109/29.21701 · Cited by 2,643
Learning methods for generic object recognition with invariance to pose and lighting
LeCun, Y., Huang, F. J., & Bottou, L. (2004) · Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004. · doi:10.1109/cvpr.2004.1315150 · Cited by 1,324
Research works printed but not linked: 24
- Abdel-Hamid, O., Mohamed, A., Jiang, H., & Penn, G. (2012). Applying convolutional neural network concepts to hybrid NN-HMM model for speech recognition, In Proc. ICASSP. · DOI · Paper page
- Dahl, G., Sainath, T., & Hinton, G. (2013), Improving deep neural networks for LVCSR using rectified linear units and dropout. In Proc. ICASSP. · DOI · Paper page
- Deng, L., Abdel-Hamid, O., & Yu, D. (2013). A deep convolutional neural network using heterogeneous pooling for trading acoustic invariance with phonetic confusion. In Proc. ICASSP. · DOI · Paper page
- Gales, 1998, Maximum likelihood linear transformations for HMM-based speech recognition, Computer Speech and Language, 12, 75, 10.1006/csla.1998.0043 · DOI · Paper page
- Gales, 1999, Semi-tied covariance matrices for hidden markov models, IEEE Transactions on Speech and Audio Processing, 7, 272, 10.1109/89.759034 · DOI · Paper page
- Glorot, X., & Bengio, Y. Understanding the difficulty of training deep feedforward neural networks. In Proc. AI Stats.
- Jaitly, N., Nguyen, P., Senior, A. W., & Vanhoucke, V. (2012). Application of pretrained deep neural networks to large vocabulary speech recognition, In Proc. Interspeech. · DOI · Paper page
- Kingsbury, B. (2009). Lattice-based optimization of sequence classification criteria for neural-network acoustic modeling. In Proc. ICASSP. · DOI · Paper page
- Kingsbury, B., Sainath, T. N., & Soltau, H. (2012). Scalable minimum bayes risk training of deep neural network acoustic models using distributed hessian-free optimization. In Proc. Interspeech. · DOI · Paper page
- Lawrence, 1997, Face recognition: A convolutional neural-network approach, IEEE Transactions on Neural Networks, 8, 98, 10.1109/72.554195 · DOI · Paper page
- Lee, L., & Rose, R. C. (1996). Speaker normalization using efficient frequency warping procedures, In Proc. ICASSP. · DOI · Paper page
- Martens, J. (2010) Deep learning via Hessian-free optimization, In Proc. intl. conf. on machine learnbeing (ICML).
- Mohamed, A., Hinton, G., & Penn, G. (2012). Understanding how deep belief networks perform acoustic modelling, In ICASSP. · DOI · Paper page
- Povey, D., Kanevsky, D., Kingsbury, B., Ramabhadran, B., Saon, G., & Visweswariah, K. (2008). Boosted MMI for model and feature-space discriminative training, In Proc. ICASSP (pp. 4057–4060). · DOI · Paper page
- Sainath, T., Mohamed, A., Kingsbury, B., & Ramabhadran, B. 2013 Deep Convolutional Neural Networks for LVCSR, In: Proc. ICASSP. · DOI · Paper page
- Sainath, T. N., Kingsbury, B., Mohamed, A., Dahl, G., Saon, G., Soltau, H., Beran, T., Aravkin, A. Y., & Ramabhadran, B. 2013 Improvements to Deep Convolutional Neural Networks for LVCSR, In: Proc. ASRU. · DOI · Paper page
- Sainath, T. N., Kingsbury, B., Ramabhadran, B., Fousek, P., Novak, P., & Mohamed, A. 2011 Making Deep Belief Networks Effective for Large Vocabulary Continuous Speech Recognition, In: Proc. ASRU. · DOI · Paper page
- Sainath, T. N., Ramabhadran, B., Picheny, M., Nahamoo, D., & Kanevsky, D. (2011). Exemplar-based sparse representation features: from TIMIT to LVCSR. · DOI
- Saon, G., Soltau, H., Picheny, M., & Nahamoo, D. 2013 Speaker adaptation of neural network acoustic models using I-vectors. In Proc. ASRU (in preperation). · DOI · Paper page
- Seide, F., Li, G., & Yu, D. 2011 Conversational Speech Transcription Using Context-Dependent Deep Neural Networks, In: Proc. Interspeech. · DOI · Paper page
- Sermanet, P., Chintala, S., & LeCun, Y. 2012 Convolutional neural networks applied to house numbers digit classification, In: Pattern Recognition (ICPR), 2012 21st International Conference on.
- Soltau, H., Saon, G., & Kingsbury, B. 2010 The IBM Attila Speech Recognition Toolkit, In: Proc. SLT. · DOI
- Young, S. J., Odell, J., & Woodland, P. 1994 Tree-based State Tying for High Accuracy Acoustic Modelling, In: Proc. HLT. pp. 307–312. · DOI · Paper page
- Zeiler, M., & Fergus, R. 2013 Stochastic Pooling for Regularization of Deep Convolutional Neural Networks, In: Proc. of the International Conference on Representaiton Learning (ICLR).
Cited by
Deep learning with convolutional neural networks for EEG decoding and visualization
Schirrmeister, R. T., Springenberg, J. T., Fiederer, L. D. J., Glasstetter, M., Eggensperger, K., Tangermann, M., Hutter, F., Burgard, W., & Ball, T. (2017) · Human Brain Mapping · doi:10.1002/hbm.23730 · Cited by 3,751
A survey of deep neural network architectures and their applications
Liu, W., Wang, Z., Liu, X., Zeng, N., Liu, Y., & Alsaadi, F. E. (2017) · Neurocomputing · doi:10.1016/j.neucom.2016.12.038 · Cited by 3,239
Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition
Ordóñez, F., & Roggen, D. (2016) · Sensors · doi:10.3390/s16010115 · Cited by 2,761
“Deep CNNs have had a major impact on fields, like content recommendation [24], speech recognition [25] and in computer vision [26–28], where they have become the de facto standard approach.”
BackgroundQuantum-chemical insights from deep tensor neural networks
Schütt, K. T., Arbabzadah, F., Chmiela, S., Müller, K. R., & Tkatchenko, A. (2017) · Nature Communications · doi:10.1038/ncomms13890 · Cited by 1,445
Review of Image Classification Algorithms Based on Convolutional Neural Networks
Chen, L., Li, S., Bai, Q., Yang, J., Jiang, S., & Miao, Y. (2021) · Remote Sensing · doi:10.3390/rs13224712 · Cited by 789
Metadata
- Authors
- Sainath, T. N., Kingsbury, B., Saon, G., Soltau, H., Mohamed, A.-R., Dahl, G., & Ramabhadran, B.
- Venue
- Neural Networks
- Publisher
- Elsevier BV
- Published
- 2015-04
- Volume
- 64
- Pages
- 39-48
- ISSN
- 0893-6080 (ISSN-L)
- OpenAlex
- W2033310064
- Cited by
- 496
- References
- 7
- FWCI
- 41.92
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Deep Convolutional Neural Networks for Large-scale Speech Tasks. https://app.cochat.ai/papers/10.1016/j.neunet.2014.08.005
- The paper
- Sainath, T. N., Kingsbury, B., Saon, G., Soltau, H., Mohamed, A.-R., Dahl, G., & Ramabhadran, B. (2015). Deep Convolutional Neural Networks for Large-scale Speech Tasks. Neural Networks. https://doi.org/10.1016/j.neunet.2014.08.005
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
