Ensemble deep learning for speech recognition
The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
Abstract
Deep learning systems have dramatically improved the accuracy of speech recognition, and various deep architectures and learning methods have been developed with distinct strengths and weaknesses in recent years. How can ensemble learning be applied to these varying deep learning systems to achieve greater recognition accuracy is the focus of this paper. We develop and report linear and log-linear stacking methods for ensemble learning with applications specifically to speechclass posterior probabilities as computed by the convolutional, recurrent, and fully-connected deep neural networks. Convex optimization problems are formulated and solved, with analytical formulas derived for training the ensemble-learning parameters. Experimental results demonstrate a significant increase in phone recognition accuracy after stacking the deep learning subsystems that use different mechanisms for computing high-level, hierarchical features from the raw acoustic signals in speech.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
Representation Learning: A Review and New Perspectives
Bengio, Y., Courville, A., & Vincent, P. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.50 · Cited by 13,001
Speech recognition with deep recurrent neural networks
Graves, A., Mohamed, A.-R., & Hinton, G. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638947 · Cited by 8,893 · cited 2×
“…include the use of locally-connected or convolutional deep neural networks (CNN) [30][9][10][31] and of temporally (deep) recurrent versions of neural networks (RNN) [14][15][8][6], also considerably outperforming the early neural networks with convolution in time [36] and the early RNN [28].”
BackgroundMethodRecurrent neural network based language model
Mikolov, T., Karafiát, M., Burget, L., Černocký, J., & Khudanpur, S. (2010) · Interspeech 2010 · doi:10.21437/interspeech.2010-343 · Cited by 5,390 · cited 1×
“Deep learning has gone a long way from the early small experiments [17][12][13][25][24][2] using the fully-connected DNN to increasingly larger experiments [16][39][11][29][33] [18] and to the exploitation of more advanced deep architectures such as the deep CNN [30][9][31] which takes into account invariant propertie…”
MethodOn the difficulty of training recurrent neural networks
Pascanu, R., Mikolov, T., & Bengio, Y. (2012) · arXiv · doi:10.48550/arxiv.1211.5063 · Cited by 4,310
Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition
Dahl, G. E., Yu, D., Deng, L., & Acero, A. (2012) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2011.2134090 · Cited by 3,011 · cited 2×
“The success of deep learning in speech recognition started with the fully-connected deep neural network (DNN) [24][40][7] [18][19][29][32][33][10][11][39].”
BackgroundMethodInfluentialSelf-citationPhoneme recognition using time-delay neural networks
Waibel, A., Hanazawa, T., Hinton, G., Shikano, K., & Lang, K. J. (1989) · IEEE Transactions on Acoustics, Speech, and Signal Processing · doi:10.1109/29.21701 · Cited by 2,643
Deep Neural Networks for Acoustic Modeling in Speech Recognition
Hinton, G. E., Deng, L., Yu, D., Dahl, G. E., Mohamed, A., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T. N., et al. (2012) · IEEE Signal Processing Magazine · Cited by 2,115
Self-citationBreiman, L. (1996) · Machine Learning · doi:10.1007/bf00117832 · Cited by 1,912
Generating Text with Recurrent Neural Networks
Sutskever, I., Martens, J., & Hinton, G. E. (2011) · International Conference on Machine Learning · Cited by 1,783
Cited by
Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition
Ordóñez, F., & Roggen, D. (2016) · Sensors · doi:10.3390/s16010115 · Cited by 2,761
“Further improvements in terms of time series classification have been obtained in the speech recognition domain, by combining convolutional and recurrent layers in a unified deep framework, which contains either standard recurrent units [29] or LSTM cells [16].”
BackgroundEnsemble deep learning: A review
Ganaie, M. A., Hu, M., Malik, A. K., Tanveer, M., & Suganthan, P. N. (2022) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2022.105151 · Cited by 2,143
Convolutional, Long Short-Term Memory, fully connected Deep Neural Networks
Sainath, T. N., Vinyals, O., Senior, A., & Sak, H. (2015) · 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2015.7178838 · Cited by 1,678
A State-of-the-Art Survey on Deep Learning Theory and Architectures
Alom, M. Z., Taha, T. M., Yakopcic, C., Westberg, S., Sidike, P., Nasrin, M. S., Hasan, M., Van Essen, B. C., Awwal, A. A. S., & Asari, V. K. (2019) · Electronics · doi:10.3390/electronics8030292 · Cited by 1,620
Speech Recognition Using Deep Neural Networks: A Systematic Review
Nassif, A. B., Shahin, I., Attili, I., Azzeh, M., & Shaalan, K. (2019) · IEEE Access · doi:10.1109/access.2019.2896880 · Cited by 1,152
A comprehensive review on ensemble deep learning: Opportunities and challenges
Mohammed, A., & Kora, R. (2023) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2023.01.014 · Cited by 1,127
Ensembles of Deep LSTM Learners for Activity Recognition using Wearables
Guan, Y., & Plötz, T. (2017) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3090076 · Cited by 511
“…for Ensemble learning of sets of deep neural networks have been applied successfully to speech recognition applications where class posterior probabilities as computed by convolutional, recurrent, and fully-connected deep neural networks are integrated into an Ensemble [Deng and Platt 2014].”
MethodRailway Track Circuit Fault Diagnosis Using Recurrent Neural Networks
de Bruin, T., Verbert, K., & Babuska, R. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2551940 · Cited by 322
A Data-Driven Soft Sensor Modeling Method Based on Deep Learning and its Application
Yan, W., Tang, D., & Lin, Y. (2017) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2016.2622668 · Cited by 294
Ansari, Y., Mourad, O., Qaraqe, K., & Serpedin, E. (2023) · Frontiers in Physiology · doi:10.3389/fphys.2023.1246746 · Cited by 291
Li, X., Li, C., Rahaman, M. M., Sun, H., Li, X., Wu, J., Yao, Y., & Grzegorzek, M. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-021-10121-0 · Cited by 278
Gong, Z., Zhong, P., & Hu, W. (2019) · IEEE Access · doi:10.1109/access.2019.2917620 · Cited by 277
Outlier Detection for Time Series with Recurrent Autoencoder Ensembles
Kieu, T., Yang, B., Guo, C., & Jensen, C. S. (2019) · Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/378 · Cited by 272
Fundamentals of speech recognition
Li, J., Deng, L., Haeb-Umbach, R., & Gong, Y. (2016) · Robust Automatic Speech Recognition · doi:10.1016/b978-0-12-802398-3.00002-7 · Cited by 266
Self-citationGuo, H., Chen, L., Peng, L., & Chen, G. (2016) · Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing · doi:10.1145/2971648.2971708 · Cited by 83
“Deng et al. [38] performed stacking for speech recognition at the most straightforward frame level and then fed the combined output to a separate HMM decoder.”
MethodEnsemble Deep Learning for Biomedical Time Series Classification
Jin, L.-P., & Dong, J. (2016) · Computational Intelligence and Neuroscience · doi:10.1155/2016/6212684 · Cited by 60
A systematic deep learning model selection for p300-based brain-computer interfaces
Abibullaev, B., & Zollanvari, A. (2022) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2021.3051136 · Cited by 53
FALCON: Feature driven selective classification for energy-efficient image recognition
Panda, P., Ankit, A., Wijesinghe, P., & Roy, K. (2017) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2017.2681075 · Cited by 21
“On the algorithmic front, substantial work for increasing accuracy in machine-learning classification has been done [8], [9].”
Accelerating the characterization of dynamic DNA origami devices with deep neural networks
Wang, Y., Jin, X., & Castro, C. (2023) · Scientific Reports · doi:10.1038/s41598-023-41459-w · Cited by 16
Metadata
- Authors
- Deng, L., & Platt, J. C.
- Affiliations
- Microsoft (United States) (US)
- Venue
- Interspeech 2014
- Publisher
- ISCA
- Published
- 2014-09-14
- Pages
- 1915-1919
- ISSN
- 2308-457X (ISSN-L)
- OpenAlex
- W2395106899
- Cited by
- 248
- References
- 38
- Citations by year
- 2026: 4 · 2025: 8 · 2024: 25 · 2023: 22 · 2022: 24 · 2021: 34 · 2020: 28 · 2019: 33 · 2018: 23 · 2017: 24 · 2016: 13 · 2015: 9 · 2014: 1
- FWCI
- 9.23
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Interspeech (USB) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Phonetics and Phonology Research
- Keywords
- ensemble learning, speech recognition, stacking, posterior probability, convolutional neural networks, recurrent neural networks, fully connected neural network, phone recognition, convex optimization, hierarchical features
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.