Deep Learning: Methods and Applications
Open accessThe DOI matches its Crossref record; 26 of the 166 references it cites were found in a registry or an index, and 140 are not checked yet.
Abstract
This monograph provides an overview of general deep learning methodology and its applications to a variety of signal and information processing tasks. The application areas are chosen with the following three criteria in mind: (1) expertise or knowledge of the authors; (2) the application areas that have already been transformed by the successful use of deep learning technology, such as speech recognition and computer vision; and (3) the application areas that have the potential to be impacted significantly by deep learning and that have been experiencing research growth, including natural language and text processing, information retrieval, and multimodal information processing empowered by multi-task deep learning.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Deep learning is successfully used as a tool for machine learning, where a neural network is capable of automatically learning features.”
From Medical image analysis using convolutional neural networks: a review (2018) · cites this paper as [47]
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Quantifying heterogeneity in a meta‐analysis
Higgins, J. P. T., & Thompson, S. G. (2002) · Statistics in Medicine · doi:10.1002/sim.1186 · Cited by 36,763
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
Representation Learning: A Review and New Perspectives
Bengio, Y., Courville, A., & Vincent, P. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.50 · Cited by 13,001
Hinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A.-R., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., et al. (2012) · IEEE Signal Processing Magazine · doi:10.1109/msp.2012.2205597 · Cited by 10,260
Self-citationLearning Deep Architectures for AI
Bengio, Y. (2009) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000006 · Cited by 9,239 · cited 6×
“Since 2006, deep structured learning, or more commonly called deep learning or hierarchical learning, has emerged as a new area of machine learning research [20, 163].”
BackgroundMethodInfluentialLearning long-term dependencies with gradient descent is difficult
Bengio, Y., Simard, P., & Frasconi, P. (1994) · IEEE Transactions on Neural Networks · doi:10.1109/72.279181 · Cited by 8,542 · cited 1×
“, speech or text), but until recently they had not been widely used partly because they are difficult to train to capture long-term dependencies, giving rise to gradient vanishing or gradient explosion problems which were known in early 1990s [29, 167].”
BackgroundWolpert, D. H. (1992) · Neural Networks · doi:10.1016/s0893-6080(05)80023-1 · Cited by 7,951
Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank
Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C. D., Ng, A., & Potts, C. (2013) · Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d13-1170 · Cited by 6,878 · cited 1×
“[347] developed the recursive neural tensor network, where each layer is constructed similarly to that of the neural tensor network described in [340] with an illustration shown in Figure 8.”
MethodCaruana, R. (1997) · Machine Learning · doi:10.1023/a:1007379606734 · Cited by 6,636 · cited 1×
“The several architectures discussed so far in this chapter for multimodal processing and learning can be regarded as special cases of more general multi-task learning and transfer learning [22, 47].”
BackgroundCommon SNPs explain a large proportion of the heritability for human height
Yang, J., Benyamin, B., McEvoy, B. P., Gordon, S., Henders, A. K., Nyholt, D. R., Madden, P. A., Heath, A. C., Martin, N. G., Montgomery, G. W., et al. (2010) · Nature Genetics · doi:10.1038/ng.608 · Cited by 4,446
(2010) · Journal of Machine Learning Research · doi:10.5555/1756006.1953039 · Cited by 4,051 · cited 5×
“An alternative unsupervised approach that performs equally well is to pretrain DNNs layer by layer by considering each pair of layers as a de-noising autoencoder regularized by setting a random subset of the input nodes to zero [20, 376].”
BackgroundMethodInfluentialMachine Learning Applications in Cancer Prognosis and Prediction
Kourou, K., Exarchos, T. P., Exarchos, K. P., Karamouzis, M. V., & Fotiadis, D. I. (2015) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2014.11.005 · Cited by 3,338
Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition
Dahl, G. E., Yu, D., Deng, L., & Acero, A. (2012) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2011.2134090 · Cited by 3,011 · cited 10×
“Over recent years, since the success of the (fully-connected) DNN– HMM hybrid system was demonstrated in [67, 68, 109, 161, 257, 258, 308, 309, 324, 429], many new architectures and nonlinear units have been proposed and evaluated for speech recognition.”
BackgroundMethodInfluentialSelf-citationPhoneme recognition using time-delay neural networks
Waibel, A., Hanazawa, T., Hinton, G., Shikano, K., & Lang, K. J. (1989) · IEEE Transactions on Acoustics, Speech, and Signal Processing · doi:10.1109/29.21701 · Cited by 2,643 · cited 3×
“It is useful to point out that the time-delay neural network (TDNN) [202, 382] developed for early speech recognition is a special case and predecessor of the CNN when weight sharing is limited to one of the two dimensions, i.”
BackgroundMethodMachine learning applications in genetics and genomics
Libbrecht, M. W., & Noble, W. S. (2015) · Nature Reviews Genetics · doi:10.1038/nrg3920 · Cited by 2,084
Breiman, L. (1996) · Machine Learning · doi:10.1007/bf00117832 · Cited by 1,912
Acoustic Modeling Using Deep Belief Networks
Mohamed, A.-R., Dahl, G. E., & Hinton, G. (2012) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.1109/tasl.2011.2109382 · Cited by 1,727 · cited 3×
“The collaborative work started in phone recognition tasks [89, 100, 135, 136, 257, 260, 258, 309, 311, 334], demonstrating the power of hybrid DNN architectures discussed in Section 5 and of subsequent new architectures with convolutional and recurrent structure.”
BackgroundBig Data: Astronomical or Genomical?
Stephens, Z. D., Lee, S. Y., Faghri, F., Campbell, R. H., Zhai, C., Efron, M. J., Iyer, R., Schatz, M. C., Sinha, S., & Robinson, G. E. (2015) · PLOS Biology · doi:10.1371/journal.pbio.1002195 · Cited by 1,340
Salakhutdinov, R., & Hinton, G. (2009) · International Journal of Approximate Reasoning · doi:10.1016/j.ijar.2008.11.006 · Cited by 1,246 · cited 2×
“1 above in this section based on the use of deep autoencoders [165, 314].”
BackgroundMethodAdaptive deconvolutional networks for mid and high level feature learning
Zeiler, M. D., Taylor, G. W., & Fergus, R. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126474 · Cited by 1,217 · cited 2×
“[437], Zeiler and Fergus [435, 436], and Zeiler ([434]).”
BackgroundMethodDeep Machine Learning - A New Frontier in Artificial Intelligence Research [Research Frontier]
Arel, I., Rose, D. C., & Karnowski, T. P. (2010) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2010.938364 · Cited by 1,121
Efficient Learning of Sparse Representations with an Energy-Based Model
Ranzato, M., Poultney, C., Chopra, S., & LeCun, Y. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0147 · Cited by 1,106 · cited 2×
“[297, 295] proposed an unsupervised learning algorithm called sparse encoding symmetric machine (SESM), which is quite similar to RBM.”
BackgroundResultApplications of Machine Learning in Cancer Prediction and Prognosis
Cruz, J. A., & Wishart, D. S. (2006) · Cancer Informatics · doi:10.1177/117693510600200030 · Cited by 1,100
Deep convolutional neural networks for LVCSR
Sainath, T. N., Mohamed, A.-R., Kingsbury, B., & Ramabhadran, B. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6639347 · Cited by 1,053
A Connection Between Score Matching and Denoising Autoencoders
Vincent, P. (2011) · Neural Computation · doi:10.1162/neco_a_00142 · Cited by 1,011
Smolensky, P. (1990) · Artificial Intelligence · doi:10.1016/0004-3702(90)90007-m · Cited by 959
An Experimental Study on Speech Enhancement Based on Deep Neural Networks
Xu, Y., Du, J., Dai, L.-R., & Lee, C.-H. (2014) · IEEE Signal Processing Letters · doi:10.1109/lsp.2013.2291240 · Cited by 924 · cited 1×
“to emotion recognition from speech in [207, 222], to spoken language understanding in [242, 366, 403], to speaker recognition in [351, 372], to language-type recognition in [112], to dialogue state tracking for spoken dialogue systems in [94, 152], to automatic voice activity detection in [442], to speech enhancement…”
BackgroundRecursive distributed representations
Pollack, J. B. (1990) · Artificial Intelligence · doi:10.1016/0004-3702(90)90005-k · Cited by 852 · cited 1×
“, they can potentially perform powerful reasoning over structures, as argued in various historical publications as well as recent essays [38, 156, 286, 288, 292, 336, 335].”
BackgroundTraining restricted Boltzmann machines using approximations to the likelihood gradient
Tieleman, T. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390290 · Cited by 837 · cited 1×
“There are other techniques for estimating the log-likelihood gradient of RBMs, in particular the stochastic maximum likelihood or persistent contrastive divergence (PCD) [363, 406].”
MethodSmall codes and large image databases for recognition
Torralba, A., Fergus, R., & Weiss, Y. (2008) · 2008 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2008.4587633 · Cited by 727
Modeling Human Motion Using Binary Latent Variables
Taylor, G. W., Hinton, G. E., & Roweis, S. T. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0173 · Cited by 689 · cited 3×
“Further, the use of a temporally conditional DBN for video sequence and human motion synthesis were reported in [361].”
BackgroundMethodLearning semantic representations using convolutional neural networks for web search
Shen, Y., He, X., Gao, J., Deng, L., & Mesnil, G. (2014) · Proceedings of the 23rd International Conference on World Wide Web · doi:10.1145/2567948.2577348 · Cited by 670 · cited 2×
“Most recently, the DSSM described above has been extended to its convolutional version, or C-DSSM [328].”
BackgroundSelf-citationHolographic reduced representations
Plate, T. A. (1995) · IEEE Transactions on Neural Networks · doi:10.1109/72.377968 · Cited by 657 · cited 1×
“, they can potentially perform powerful reasoning over structures, as argued in various historical publications as well as recent essays [38, 156, 286, 288, 292, 336, 335].”
BackgroundSequence-discriminative training of deep neural networks
Veselý, K., Ghoshal, A., Burget, L., & Povey, D. (2013) · Interspeech 2013 · doi:10.21437/interspeech.2013-548 · Cited by 657
An investigation of deep neural networks for noise robust speech recognition
Seltzer, M. L., Yu, D., & Wang, Y. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6639100 · Cited by 640 · cited 2×
“[325], who applied the C-MMSE [415] feature enhancement algorithm on the input feature”
BackgroundMethodSelf-citationMinimum classification error rate methods for speech recognition
Juang, B.-H., Hou, W., & Lee, C.-H. (1997) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/89.568732 · Cited by 618 · cited 1×
“Many of the discriminative techniques for supervised learning in signal and information processing are shallow architectures such as HMMs [52, 127, 147, 186, 188, 290, 394, 418] and conditional random fields (CRFs) [151, 155, 281, 400, 429, 446].”
BackgroundA time-delay neural network architecture for isolated word recognition
Lang, K. J., Waibel, A. H., & Hinton, G. E. (1990) · Neural Networks · doi:10.1016/0893-6080(90)90044-l · Cited by 596 · cited 2×
“It is useful to point out that the time-delay neural network (TDNN) [202, 382] developed for early speech recognition is a special case and predecessor of the CNN when weight sharing is limited to one of the two dimensions, i.”
BackgroundLow-rank matrix factorization for Deep Neural Network training with high-dimensional output targets
Sainath, T. N., Kingsbury, B., Sindhwani, V., Arisoy, E., & Ramabhadran, B. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638949 · Cited by 593 · cited 1×
“In [310] and [397], the DNN with high-dimensional output layer was trained first.”
MethodFrom HMM's to segment models: a unified view of stochastic modeling for speech recognition
Ostendorf, M., Digalakis, V. V., & Kimball, O. A. (1996) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/89.536930 · Cited by 587
Speaker adaptation of neural network acoustic models using i-vectors
Saon, G., Soltau, H., Nahamoo, D., & Picheny, M. (2013) · 2013 IEEE Workshop on Automatic Speech Recognition and Understanding · doi:10.1109/asru.2013.6707705 · Cited by 577 · cited 1×
“, 2013b), IBM (Sainath et al., 2011, 2013, 2013b; Kingsbury et al., 2012; Saon et al., 2013), Google (Jaitly et al.”
Three new graphical models for statistical language modelling
Mnih, A., & Hinton, G. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273577 · Cited by 563
From big data analysis to personalized medicine for all: challenges and opportunities
Alyass, A., Turcotte, M., & Meyre, D. (2015) · BMC Medical Genomics · doi:10.1186/s12920-015-0108-y · Cited by 557
Visualizing and Understanding Convolutional Networks
Zeiler, M. D., & Fergus, R. (2013) · arXiv · doi:10.48550/arxiv.1311.2901 · Cited by 522 · cited 3×
“[437], Zeiler and Fergus [435, 436], and Zeiler ([434]).”
BackgroundMethodBilingual Word Embeddings for Phrase-Based Machine Translation
Zou, W. Y., Socher, R., Cer, D., & Manning, C. D. (2013) · Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d13-1141 · Cited by 518
Strategies for training large scale neural network language models
Mikolov, T., Deoras, A., Povey, D., Burget, L., & Cernocky, J. (2011) · 2011 IEEE Workshop on Automatic Speech Recognition & Understanding · doi:10.1109/asru.2011.6163930 · Cited by 509 · cited 4×
“Empirical comparisons with other state-ofthe-art counting-based N -gram LMs show much better performance of RNNLM in the perplexity measure, as reported in [247, 248] and [245].”
BackgroundMethodInfluentialMachine Learning Paradigms for Speech Recognition: An Overview
Deng, L., & Li, X. (2013) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.1109/tasl.2013.2244083 · Cited by 456 · cited 2×
“, Chapter 28 in [264], and Reference [95], it may be natural to just classify deep learning techniques into deep discriminative models (e.”
BackgroundSelf-citationAn application of recurrent nets to phone probability estimation
Robinson, A. J. (1994) · IEEE Transactions on Neural Networks · doi:10.1109/72.279192 · Cited by 441 · cited 2×
“In [304], a separate HMM was used to segment the sequence during training, and the HMM was also used to transform the RNN classification results into label sequences.”
BackgroundMethodMesnil, G., He, X., Deng, L., & Bengio, Y. (2013) · Interspeech 2013 · doi:10.21437/interspeech.2013-596 · Cited by 413 · cited 1×
“, 2013; Le and Mower, 2013), to spoken language understanding in (Mesnil et al., 2013; Yao et al., 2013; Tur et al., 2012), to speaker recognition in (Vasilakakis et al.”
BackgroundSelf-citationYu, D., Yao, K., Su, H., Li, G., & Seide, F. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6639201 · Cited by 408 · cited 2×
“The most notable groups include major industrial speech labs worldwide: Microsoft [49, 89, 94, 324, 399, 430], IBM [195, 309, 311, 307, 317], Google [69, 150, 184, 223], iFlyTek, and Baidu.”
BackgroundMethodSelf-citationRestructuring of deep neural network acoustic models with singular value decomposition
Xue, J., Li, J., & Gong, Y. (2013) · Interspeech 2013 · doi:10.21437/interspeech.2013-552 · Cited by 407 · cited 1×
“In [310] and [397], the DNN with high-dimensional output layer was trained first.”
MethodSpeech Synthesis Based on Hidden Markov Models
Tokuda, K., Nankaku, Y., Toda, T., Zen, H., Yamagishi, J., & Oura, K. (2013) · Proceedings of the IEEE · doi:10.1109/jproc.2013.2251852 · Cited by 404
Large scale image annotation: learning to rank with joint word-image embeddings
Weston, J., Bengio, S., & Usunier, N. (2010) · Machine Learning · doi:10.1007/s10994-010-5198-3 · Cited by 399 · cited 1×
“The earlier WSABIE system as described in [388, 389] adopted a shallow architecture and trained a joint embedding model of both images and labels.”
MethodSupport Vector Machines (SVM) as a Technique for Solvency Analysis
Auria, L., & Moro, R. A. (2008) · SSRN Electronic Journal · doi:10.2139/ssrn.1424949 · Cited by 374
Abraham, A. (2005) · Handbook of Measuring System Design · doi:10.1002/0471497398.mm421 · Cited by 339
Optimality: From Neural Networks to Universal Grammar
Prince, A., & Smolensky, P. (1997) · Science · doi:10.1126/science.275.5306.1604 · Cited by 337 · cited 1×
“, they can potentially perform powerful reasoning over structures, as argued in various historical publications as well as recent essays [38, 156, 286, 288, 292, 336, 335].”
BackgroundRecurrent neural networks for noise reduction in robust ASR
Maas, A. L., Le, Q. V., O'Neil, T. M., Vinyals, O., Nguyen, P., & Ng, A. Y. (2012) · Interspeech 2012 · doi:10.21437/interspeech.2012-6 · Cited by 334 · cited 2×
“Learning algorithms for the RNN have been dramatically improved since then, and much better results have been obtained recently using the RNN [48, 134, 235], especially when the bi-directional LSTM (long short-term memory) is used [135, 136].”
MethodLayered representations for learning and inferring office activity from multiple sensory channels
Oliver, N., Garg, A., & Horvitz, E. (2004) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2004.02.004 · Cited by 316 · cited 1×
“Related models with greater technical depth and mathematical treatment can be found in [116] for HHMM and [271] for Layered HMM.”
MethodSum-product networks: A new deep architecture
Poon, H., & Domingos, P. (2011) · 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops) · doi:10.1109/iccvw.2011.6130310 · Cited by 315 · cited 2×
“Empirical solutions have been found to mitigate this difficulty as reported in [289].”
BackgroundRecurrent neural networks for language understanding
Yao, K., Zweig, G., Hwang, M.-Y., Shi, Y., & Yu, D. (2013) · Interspeech 2013 · doi:10.21437/interspeech.2013-569 · Cited by 315 · cited 2×
“[403] reported the success of RNNs in spoken language understanding.”
BackgroundSelf-citationDeep Belief Networks Based Voice Activity Detection
Zhang, X.-L., & Wu, J. (2013) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.1109/tasl.2012.2229986 · Cited by 313 · cited 1×
“to emotion recognition from speech in [207, 222], to spoken language understanding in [242, 366, 403], to speaker recognition in [351, 372], to language-type recognition in [112], to dialogue state tracking for spoken dialogue systems in [94, 152], to automatic voice activity detection in [442], to speech enhancement…”
BackgroundMapping part-whole hierarchies into connectionist networks
Hinton, G. E. (1990) · Artificial Intelligence · doi:10.1016/0004-3702(90)90004-j · Cited by 310 · cited 1×
“, they can potentially perform powerful reasoning over structures, as argued in various historical publications as well as recent essays [38, 156, 286, 288, 292, 336, 335].”
BackgroundImproving deep neural network acoustic models using generalized maxout networks
Zhang, X., Trmal, J., Povey, D., & Khudanpur, S. (2014) · 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2014.6853589 · Cited by 273 · cited 1×
“[441] generalize the above “maxout” units to two new types.”
BackgroundUnderstanding how Deep Belief Networks perform acoustic modelling
Mohamed, A.-R., Hinton, G., & Penn, G. (2012) · 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2012.6288863 · Cited by 265 · cited 3×
“For example, [259] applied a dimensionality reduction method to visualize the relationship among the feature vectors learned by the DNN.”
BackgroundMethodInfluentialImprovements to Deep Convolutional Neural Networks for LVCSR
Sainath, T. N., Kingsbury, B., Mohamed, A.-R., Dahl, G. E., Saon, G., Soltau, H., Beran, T., Aravkin, A. Y., & Ramabhadran, B. (2013) · 2013 IEEE Workshop on Automatic Speech Recognition and Understanding · doi:10.1109/asru.2013.6707749 · Cited by 243 · cited 3×
“[306] combined dropout with a number of novel techniques described in this section (including the use of deep CNNs, Hessian-free sequence learning, the use of ReLU units, and the use of joint fMLLR and filterbank features, etc.”
BackgroundMethodOn deep generative models with applications to recognition
Ranzato, M., Susskind, J., Mnih, V., & Hinton, G. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995710 · Cited by 237 · cited 2×
“[298], the deep generative model of DBN with gated Markov random field (MRF) at the lowest level is learned for feature extraction and then for recognition of difficult image classes including occlusions.”
BackgroundMaximum likelihood estimation for multivariate mixture observations of markov chains (Corresp.)
Juang, B.-H., Levinson, S., & Sondhi, M. (1986) · IEEE Transactions on Information Theory · doi:10.1109/tit.1986.1057145 · Cited by 235
Developments and directions in speech recognition and understanding, Part 1 [DSP Education]
Baker, J., Deng, L., Glass, J., Khudanpur, S., Lee, C.-H., Morgan, N., & O'Shaughnessy, D. (2009) · IEEE Signal Processing Magazine · doi:10.1109/msp.2009.932166 · Cited by 228
Self-citationSocher, R., & Fei-Fei, L. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5540112 · Cited by 217 · cited 1×
“For example, Socher and Fei-Fei [341] project words and images into the same space using kernelized canonical correlation analysis.”
MethodModeling pixel means and covariances using factorized third-order boltzmann machines
Ranzato, M., & Hinton, G. E. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5539962 · Cited by 215 · cited 1×
“The standard DBN has been extended to the factored higher-order Boltzmann machine in its bottom layer, with strong results obtained for phone recognition [64] and for computer vision [296].”
BackgroundLearning image representations from the pixel level via hierarchical sparse coding
Yu, K., Lin, Y., & Lafferty, J. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995732 · Cited by 212 · cited 1×
“In addition to unsupervised pretraining using greedy layer-wise procedures [28, 164, 295], the supervised pretraining, or sometimes called discriminative pretraining, has also been shown to be effective [28, 161, 324, 432] and in cases where labeled training data are abundant performs better than the unsupervised pret…”
MethodInvestigation of full-sequence training of deep belief networks for speech recognition
Mohamed, A.-R., Yu, D., & Deng, L. (2010) · Interspeech 2010 · doi:10.21437/interspeech.2010-304 · Cited by 210 · cited 8×
“In implementing the above full-sequence learning algorithm for the DNN system as described in [260], the DNN weights are initialized using the frame-level cross entropy as the objective.”
BackgroundMethodInfluentialSelf-citationGlobal optimization of a neural network-hidden Markov model hybrid
Bengio, Y., De Mori, R., Flammia, G., & Kompe, R. (1992) · IEEE Transactions on Neural Networks · doi:10.1109/72.125866 · Cited by 203
Adaptation of context-dependent deep neural networks for automatic speech recognition
Yao, K., Yu, D., Seide, F., Su, H., Deng, L., & Gong, Y. (2012) · 2012 IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt.2012.6424251 · Cited by 194 · cited 1×
“A number of exploratory studies on DNN adaptation made use of the same or related linear transformation methods [223, 401, 402].”
BackgroundSelf-citationSpeech translation: coupling of recognition and translation
Ney, H. (1999) · 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258) · doi:10.1109/icassp.1999.758176 · Cited by 189 · cited 1×
“The final example given here for hybrid deep networks is based on the idea and work of [144, 267], where one task of discrimination (e.”
BackgroundMaking Deep Belief Networks effective for large vocabulary continuous speech recognition
Sainath, T. N., Kingsbury, B., Ramabhadran, B., Fousek, P., Novak, P., & Mohamed, A.-R. (2011) · 2011 IEEE Workshop on Automatic Speech Recognition & Understanding · doi:10.1109/asru.2011.6163900 · Cited by 189 · cited 4×
“The success was quickly extended to large vocabulary speech recognition tasks with hundreds and even thousands of hours of training set and with thousands of tri-phone states, including the Switchboard and Broadcast News databases, and Google’s voice search and YouTube tasks [94, 161, 184, 309, 311, 324].”
BackgroundInfluentialEquivalence of Backpropagation and Contrastive Hebbian Learning in a Layered Network
Xie, X., & Seung, H. S. (2003) · Neural Computation · doi:10.1162/089976603762552988 · Cited by 184 · cited 1×
“How can computational neuroscience models about hierarchical brain structure help improve engineering deep learning architectures? How may the biologically feasible learning styles in the brain [158, 395] help design more effective and more robust deep learning algorithms? All these issues and those discussed earlier…”
BackgroundLearned-Norm Pooling for Deep Feedforward and Recurrent Neural Networks
Gulcehre, C., Cho, K., Pascanu, R., & Bengio, Y. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-662-44848-9_34 · Cited by 167
Deng, L., Aksmanovic, M., Sun, X., & Wu, C. (1994) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/89.326610 · Cited by 152 · cited 1×
“Specifically, the early work described in [71, 72, 83, 84, 99, 274] generalized and extended the conventional shallow and conditionally independent HMM structure by imposing dynamic constraints, in the form of polynomial trajectory, on the HMM parameters.”
BackgroundSelf-citationStatistical and Practical Considerations for Clinical Evaluation of Predictive Biomarkers
Polley, M.-Y. C., Freidlin, B., Korn, E. L., Conley, B. A., Abrams, J. S., & McShane, L. M. (2013) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/djt282 · Cited by 150
Ling, Z.-H., Deng, L., & Yu, D. (2013) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2013.2269291 · Cited by 150 · cited 1×
“[227, 229], the RBM and DBN as generative models are used to replace the traditional Gaussian models, achieving significant quality improvement, in both subjective and objective measures, of the synthesized voice.”
MethodSelf-citationDeng, L., & Sun, D. X. (1994) · The Journal of the Acoustical Society of America · doi:10.1121/1.409839 · Cited by 144
Self-citationDiscriminative learning in sequential pattern recognition
He, X., Deng, L., & Chou, W. (2008) · IEEE Signal Processing Magazine · doi:10.1109/msp.2008.926652 · Cited by 143 · cited 4×
“Here, it is useful to point out a connection between the above pretraining/fine-tuning strategy associated with hybrid deep networks and the highly popular minimum phone error (MPE) training technique for the HMM (see [147, 290] for an overview).”
BackgroundMethodInfluentialSelf-citationVoice conversion in high-order eigen space using deep belief nets
Nakashika, T., Takashima, R., Takiguchi, T., & Ariki, Y. (2013) · Interspeech (USB) · doi:10.21437/interspeech.2013-102 · Cited by 143 · cited 1×
“to emotion recognition from speech in [207, 222], to spoken language understanding in [242, 366, 403], to speaker recognition in [351, 372], to language-type recognition in [112], to dialogue state tracking for spoken dialogue systems in [94, 152], to automatic voice activity detection in [442], to speech enhancement…”
BackgroundRevisiting Recurrent Neural Networks for robust ASR
Vinyals, O., Ravuri, S. V., & Povey, D. (2012) · 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2012.6288816 · Cited by 143
Su, H., Li, G., Yu, D., & Seide, F. T. B. (2013) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2013.6638951 · Cited by 139 · cited 4×
“While the batch-mode, second-order Hessian-free techniques prove successful for full-sequence training of large-scale DNN-HMM systems, the success of the first-order stochastic gradient descent methods is also reported recently (Su et al., 2013).”
BackgroundMethodInfluentialSelf-citationExploiting sparseness in deep neural networks for large vocabulary speech recognition
Yu, D., Seide, F., Li, G., & Deng, L. (2012) · 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2012.6288897 · Cited by 137
Self-citationOn the convergence of markovian stochastic algorithms with rapidly decreasing ergodicity rates
Younes, L. (1999) · Stochastics and Stochastic Reports · doi:10.1080/17442509908834179 · Cited by 136 · cited 1×
“There are other techniques for estimating the log-likelihood gradient of RBMs, in particular the stochastic maximum likelihood or persistent contrastive divergence (PCD) [363, 406].”
MethodLearning filter banks within a deep neural network framework
Sainath, T. N., Kingsbury, B., Mohamed, A.-R., & Ramabhadran, B. (2013) · 2013 IEEE Workshop on Automatic Speech Recognition and Understanding · doi:10.1109/asru.2013.6707746 · Cited by 133 · cited 5×
“Substantial speech recognition error reduction is reported in [307].”
BackgroundMethodInfluentialDeep neural network features and semi-supervised training for low resource speech recognition
Thomas, S., Seltzer, M. L., Church, K., & Heřmanský, H. (2013) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2013.6638959 · Cited by 132
Deep and Wide: Multiple Layers in Automatic Speech Recognition
Morgan, N. (2012) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.1109/tasl.2011.2116010 · Cited by 128 · cited 3×
“Note the use of “hybrid” in (3) above is different from that used sometimes in the literature, which refers to the hybrid systems for speech recognition feeding the output probabilities of a neural network into an HMM [17, 25, 42, 261].”
BackgroundMethodTowards deeper understanding: Deep convex networks for semantic utterance classification
Tur, G., Deng, L., Hakkani-Tur, D., & He, X. (2012) · 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2012.6289054 · Cited by 122
Self-citationExploiting deep neural networks for detection-based speech recognition
Siniscalchi, S. M., Yu, D., Deng, L., & Lee, C. (2012) · Neurocomputing · doi:10.1016/j.neucom.2012.11.008 · Cited by 120 · cited 3×
“Finally, the learning architecture developed for bottom-up, detection-based speech recognition proposed in [214] and developed further since 2004, notably in [330, 332, 427] using the DBN–DNN technique, can also be categorized in the discriminative or supervisedlearning deep architecture category.”
BackgroundMethodSelf-citationDeep maxout networks for low-resource speech recognition
Miao, Y., Metze, F., & Rawat, S. (2013) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru.2013.6707763 · Cited by 117 · cited 1×
“Another new type of DNN units demonstrated more recently to be useful for speech recognition is the “maxout” units, which were used for forming the deep maxout network as described in [244].”
MethodGraphical model architectures for speech recognition
Bilmes, J. A., & Bartels, C. D. (2005) · IEEE Signal Processing Magazine · doi:10.1109/msp.2005.1511827 · Cited by 115 · cited 1×
“The above deepstructured generative models of speech can be shown as special cases of the more general dynamic network model and even more general dynamic graphical models [35, 34].”
BackgroundThe Deep Tensor Neural Network With Applications to Large Vocabulary Speech Recognition
Yu, D., Deng, L., & Seide, F. (2013) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.1109/tasl.2012.2227738 · Cited by 113 · cited 1×
“[421, 422], which extends the conventional DNN by replacing one or more of its layers with a double-projection layer and a tensor layer.”
BackgroundSelf-citationDeng, L. (1992) · Signal Processing · doi:10.1016/0165-1684(92)90112-a · Cited by 109 · cited 1×
“Specifically, the early work described in [71, 72, 83, 84, 99, 274] generalized and extended the conventional shallow and conditionally independent HMM structure by imposing dynamic constraints, in the form of polynomial trajectory, on the HMM parameters.”
BackgroundSelf-citationA segmental CRF approach to large vocabulary continuous speech recognition
Zweig, G., & Nguyen, P. (2009) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru.2009.5372916 · Cited by 106 · cited 1×
“Many of the discriminative techniques for supervised learning in signal and information processing are shallow architectures such as HMMs [52, 127, 147, 186, 188, 290, 394, 418] and conditional random fields (CRFs) [151, 155, 281, 400, 429, 446].”
BackgroundPapandreou, G., Katsamanis, A., Pitsikalis, V., & Maragos, P. (2009) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.1109/tasl.2008.2011515 · Cited by 101 · cited 1×
“However, the same paper also shows that a model of [278] in which a sophisticated signal processing technique for extracting visual features, together with the uncertainty-compensation method developed originally from robust speech recognition [104], gives the best classification accuracy in the cross-modal learning t…”
MethodPushing the envelope - aside [speech recognition
Morgan, N., Zhu, Q., Stolcke, A., Sonmez, K., Sivadas, S., Shinozaki, T., Ostendorf, M., Jain, P., Hermansky, H., Ellis, D., et al. (2005) · IEEE Signal Processing Magazine · doi:10.1109/msp.2005.1511826 · Cited by 95 · cited 2×
“In particular, a class of deep neural network models forms the basis of the popular “tandem” approach [262], where the output of the discriminatively learned neural network is treated as part”
BackgroundDeng, L., & Ma, J. (2000) · The Journal of the Acoustical Society of America · doi:10.1121/1.1315288 · Cited by 89 · cited 1×
“Subsequent work added a new hidden layer into the dynamic model to explicitly account for the target-directed, articulatory-like properties in human speech generation [45, 73, 74, 83, 96, 75, 90, 231, 232, 233, 251, 282].”
BackgroundSelf-citationDeng, L. (1998) · Speech Communication · doi:10.1016/s0167-6393(98)00023-5 · Cited by 88 · cited 2×
“It is important to note that the unique elasticity of temporal dynamics of speech as elaborated in [45, 73, 76, 83] would require temporally”
BackgroundSelf-citationSpeech Recognition Using Augmented Conditional Random Fields
Hifny, Y., & Renals, S. J. (2009) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2008.2010286 · Cited by 87 · cited 1×
“Many of the discriminative techniques for supervised learning in signal and information processing are shallow architectures such as HMMs [52, 127, 147, 186, 188, 290, 394, 418] and conditional random fields (CRFs) [151, 155, 281, 400, 429, 446].”
BackgroundLearning emotion-based acoustic features with deep belief networks
Schmidt, E. M., & Kim, Y. E. (2011) · 2011 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA) · doi:10.1109/aspaa.2011.6082328 · Cited by 87 · cited 1×
“In the general field of audio and music processing, the impacted areas by deep learning include mainly music signal processing and music information retrieval [15, 22, 141, 177, 178, 179, 319].”
BackgroundAwwalu, J., Garba, A. G., Ghazvini, A., & Atuah, R. (2014) · International Journal of Computer Theory and Engineering · doi:10.7763/ijcte.2015.v7.999 · Cited by 86
Production models as a structural basis for automatic speech recognition
Deng, L., Ramsay, G., & Sun, D. (1997) · Speech Communication · doi:10.1016/s0167-6393(97)00018-6 · Cited by 84 · cited 1×
“It is important to note that the unique elasticity of temporal dynamics of speech as elaborated in (Deng et al., 1997; Bridle et al., 1998; Deng, 1998, 2006) would require temporally-correlated”
BackgroundSelf-citationLi, D., Yu, D., & Acero, A. (2006) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2006.878265 · Cited by 84 · cited 1×
“Further, the DNN results were found to be slightly superior to the then-best-performing single system based on the generative hidden trajectory model (HTM) in the literature [105, 108] evaluated on the same, commonly used TIMIT task by many speech researchers [107, 108, 274, 313].”
MethodSelf-citationAnalysis of MLP-Based Hierarchical Phoneme Posterior Probability Estimator
Pinto, J., Garimella, S., Magimai.-Doss, M., Hynek Hermansky, B., & Bourlard, H. A. (2010) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2010.2045943 · Cited by 80
Acoustic Modeling With Hierarchical Reservoirs
Triefenbach, F., Jalalvand, A., Demuynck, K., & Martens, J. (2013) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2013.2280209 · Cited by 80 · cited 1×
“[365] applied such models to phone recognition, with reasonably good accuracy obtained.”
MethodAn application of discriminative feature extraction to filter-bank-based speech recognition
Biem, A., Katagiri, S., McDermott, E., & Juang, B.-H. F. (2001) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/89.902277 · Cited by 78
On parallelizability of stochastic gradient descent for speech DNNS
Seide, F. T. B., Fu, H., Droppo, J., Li, G., & Yu, D. (2014) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2014.6853593 · Cited by 78 · cited 1×
“To make deep learning techniques scalable to very large training data, theoretically sound parallel learning and optimization algorithms together with novel architectures need to be further developed [31, 39, 49, 69, 181, 322, 356].”
MethodSelf-citationImproving low-resource CD-DNN-HMM using dropout and multilingual DNN training
Miao, Y., & Metze, F. (2013) · Interspeech (USB) · doi:10.21437/interspeech.2013-526 · Cited by 72 · cited 1×
“Subsequent work on applying dropout includes the study by Miao and Metze [243], where DNN-based speech recognition is constrained by low resources with sparse training data.”
MethodSun, J., & Deng, L. (2002) · The Journal of the Acoustical Society of America · doi:10.1121/1.1420380 · Cited by 71 · cited 2×
“One intermediate neural network layer in the implementation of this detection-based framework explicitly represents the speech attributes, which are simplified entities from the “atomic” units of speech developed in the early work of [101, 355].”
BackgroundSelf-citationContinuous Stochastic Feature Mapping Based on Trajectory HMMs
Zen, H., Nankaku, Y., & Tokuda, K. (2010) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2010.2049685 · Cited by 70 · cited 1×
“Similar trajectory HMMs also form the basis for parametric speech synthesis [228, 326, 439, 438].”
BackgroundHermitian Polynomial for Speaker Adaptation of Connectionist Speech Recognition Systems
Siniscalchi, S. M., Li, J., & Lee, C.-H. (2013) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.1109/tasl.2013.2270370 · Cited by 68 · cited 3×
“Finally, the learning architecture developed for bottom-up, detection-based speech recognition proposed in [214] and developed further since 2004, notably in [330, 332, 427] using the DBN–DNN technique, can also be categorized in the discriminative or supervisedlearning deep architecture category.”
BackgroundMethodEfficient and effective algorithms for training single-hidden-layer neural networks
Yu, D., & Deng, L. (2011) · Pattern Recognition Letters · doi:10.1016/j.patrec.2011.12.002 · Cited by 67 · cited 1×
“we can derive batch-mode gradient descent learning algorithm where the gradient takes the following form (Deng and Yu, 2011; Yu and Deng, 2012):”
MethodSelf-citationCalibration of Confidence Measures in Speech Recognition
Yu, D., Li, J., & Deng, L. (2011) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2011.2141988 · Cited by 67 · cited 1×
“For example, the deep-structured CRF, which stacks many layers of CRFs, have been usefully applied to the task of language identification [429], phone recognition [410], sequential labeling in natural language processing [428], and confidence calibration in speech recognition [423].”
MethodSelf-citationYu, D., Siniscalchi, S. M., Deng, L., & Lee, C. (2012) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2012.6288837 · Cited by 66
Self-citationExemplar-Based Sparse Representation Features: From TIMIT to LVCSR
Sainath, T. N., Ramabhadran, B., Picheny, M., Nahamoo, D., & Kanevsky, D. (2011) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2011.2155060 · Cited by 65 · cited 1×
“Further, the DNN results were found to be slightly superior to the then-best-performing single system based on the generative hidden trajectory model (HTM) in the literature [105, 108] evaluated on the same, commonly used TIMIT task by many speech researchers [107, 108, 274, 313].”
MethodSparse Multilayer Perceptron for Phoneme Recognition
Sivaram, G. S. V. S., & Hermansky, H. (2012) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.1109/tasl.2011.2129510 · Cited by 65
A Probabilistic Interaction Model for Multipitch Tracking With Factorial Hidden Markov Models
Wohlmayr, M., Stark, M. M., & Pernkopf, F. (2010) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2010.2064309 · Cited by 63
Autoregressive Models for Statistical Parametric Speech Synthesis
Shannon, M., Zen, H., & Byrne, B. (2012) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2012.2227740 · Cited by 61 · cited 1×
“Similar trajectory HMMs also form the basis for parametric speech synthesis [228, 326, 439, 438].”
BackgroundUpdated MINDS report on speech recognition and understanding, Part 2 [DSP Education]
Baker, J. M., Deng, L., Khudanpur, S. P., Lee, C., Glass, J. R., Morgan, N., & O’Shaughnessy, D. D. (2009) · IEEE Signal Processing Magazine · doi:10.1109/msp.2009.932707 · Cited by 59
Self-citationOptimization Techniques to Improve Training Speed of Deep Neural Networks for Large Speech Tasks
Sainath, T. N., Kingsbury, B., Soltau, H., & Ramabhadran, B. (2013) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.1109/tasl.2013.2284378 · Cited by 59 · cited 4×
“The success was quickly extended to large vocabulary speech recognition tasks with hundreds and even thousands of hours of training set and with thousands of tri-phone states, including the Switchboard and Broadcast News databases, and Google’s voice search and YouTube tasks [94, 161, 184, 309, 311, 324].”
BackgroundInfluentialSheikhzadeh, H., & Deng, L. (1994) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/89.260337 · Cited by 56 · cited 2×
“The work also showed the importance of raw speech features of spectrogram — back from the long-popular MFCC features toward but not yet reaching the raw speech-waveform level [183, 327].”
BackgroundMethodSelf-citationComparing multilayer perceptron to Deep Belief Network Tandem features for robust ASR
Vinyals, O., & Ravuri, S. V. (2011) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2011.5947378 · Cited by 55 · cited 2×
“Vinyals and Ravuri [379] investigated the tandem approaches to noise-robust speech recognition, where DNNs were trained directly with noisy speech to generate posterior features.”
BackgroundMethodLarge-Margin Minimum Classification Error Training for Large-Scale Speech Recognition Tasks
Yu, D., Deng, L., He, X., & Acero, A. (2007) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2007.367275 · Cited by 55 · cited 2×
“Many of the discriminative techniques for supervised learning in signal and information processing are shallow architectures such as HMMs [52, 127, 147, 186, 188, 290, 394, 418] and conditional random fields (CRFs) [151, 155, 281, 400, 429, 446].”
BackgroundMethodSelf-citationDisease Risk Factors Identified Through Shared Genetic Architecture and Electronic Medical Records
Li, L., Ruau, D. J., Patel, C. J., Weber, S. C., Chen, R., Tatonetti, N. P., Dudley, J. T., & Butte, A. J. (2014) · Science Translational Medicine · doi:10.1126/scitranslmed.3007191 · Cited by 54
Computational Models for Speech Production
Deng, L. (1999) · Springer eBooks · doi:10.1007/978-3-642-60087-6_20 · Cited by 51
Self-citationDeng, L., Kenny, P. J., Lennig, M., Gupta, V., Seitz, F., & Mermelstein, P. G. (1991) · IEEE Transactions on Signal Processing · doi:10.1109/78.134406 · Cited by 51
Self-citationDeng, L. F., & Aksmanovic, M. (1997) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/89.593305 · Cited by 50
Self-citationProduct of Experts for Statistical Parametric Speech Synthesis
Zen, H., Gales, M., Nankaku, Y., & Tokuda, K. (2011) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2011.2165280 · Cited by 50 · cited 1×
“Similar trajectory HMMs also form the basis for parametric speech synthesis [228, 326, 439, 438].”
BackgroundA scalable approach to using DNN-derived features in GMM-HMM based acoustic modeling for LVCSR
Yan, Z., Huo, Q., & Xu, J. F. (2013) · Interspeech (USB) · doi:10.21437/interspeech.2013-47 · Cited by 46 · cited 3×
“The collaboration continued to large vocabulary tasks with more convincing, highly positive results [67, 68, 94, 89, 161, 199, 195, 223, 323, 353, 399, 414].”
BackgroundMethodSequential Labeling Using Deep-Structured Conditional Random Fields
Yu, D., Wang, S., & Deng, L. (2010) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2010.2075990 · Cited by 46 · cited 3×
“For example, the deep-structured CRF, which stacks many layers of CRFs, have been usefully applied to the task of language identification [429], phone recognition [410], sequential labeling in natural language processing [428], and confidence calibration in speech recognition [423].”
BackgroundMethodSelf-citationHierarchical Bayesian Language Models for Conversational Speech Recognition
Huang, S., & Renals, S. J. (2010) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2010.2040782 · Cited by 44 · cited 1×
“Before we discuss neural network based LMs, we note the use of hierarchical Bayesian priors in building up deep and recursive structure for LMs [174].”
MethodA Novel Framework and Training Algorithm for Variable-Parameter Hidden Markov Models
Yu, D., Deng, L., Gong, Y., & Acero, A. (2009) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2009.2020890 · Cited by 41
Self-citationLarge vocabulary word recognition using context-dependent allophonic hidden Markov models
Deng, L., Lennig, M., Seitz, F., & Mermelstein, P. (1990) · Computer Speech & Language · doi:10.1016/0885-2308(90)90015-x · Cited by 40
Self-citationDeng, L., Hassanein, K., & Elmasry, M. I. (1994) · Neural Networks · doi:10.1016/0893-6080(94)90027-2 · Cited by 36
Self-citationTarget-Directed Mixture Dynamic Models for Spontaneous Speech Recognition
Ma, J., & Deng, L. (2004) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/tsa.2003.818074 · Cited by 36 · cited 1×
“Subsequent work added a new hidden layer into the dynamic model to explicitly account for the target-directed, articulatory-like properties in human speech generation [45, 73, 74, 83, 96, 75, 90, 231, 232, 233, 251, 282].”
BackgroundSelf-citationDeep-structured hidden conditional random fields for phonetic recognition
Yu, D., & Deng, L. (2010) · Interspeech (USB) · doi:10.21437/interspeech.2010-35 · Cited by 36 · cited 2×
“For example, the deep-structured CRF, which stacks many layers of CRFs, have been usefully applied to the task of language identification [429], phone recognition [410], sequential labeling in natural language processing [428], and confidence calibration in speech recognition [423].”
BackgroundMethodSelf-citationLarge-margin minimum classification error training: A theoretical risk minimization perspective
Yu, D., Deng, L., He, X., & Acero, A. (2008) · Computer Speech & Language · doi:10.1016/j.csl.2008.03.002 · Cited by 35 · cited 1×
“In the implementation of DSSM learning described in [172], a subset of the negative candidates are used, following the common practice adopted in MCE (Minimum Classification Error) training in speech recognition [52, 118, 417, 418].”
MethodSelf-citationDynamic Speech Models: Theory, Algorithms, and Applications
Deng, L. (2006) · Synthesis lectures on speech and audio processing · doi:10.1007/978-3-031-02555-6 · Cited by 34 · cited 4×
“It is important to note that the unique elasticity of temporal dynamics of speech as elaborated in [45, 73, 76, 83] would require temporally”
BackgroundMethodInfluentialSelf-citationEquivalence of Generative and Log-Linear Models
Heigold, G., Ney, H., Lehnen, P., Gass, T., & Schlüter, R. (2010) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2010.2082532 · Cited by 34
Single-channel mixed speech recognition using deep neural networks
Weng, C., Yu, D., Seltzer, M. L., & Droppo, J. (2014) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2014.6854681 · Cited by 33 · cited 1×
“to emotion recognition from speech in [207, 222], to spoken language understanding in [242, 366, 403], to speaker recognition in [351, 372], to language-type recognition in [112], to dialogue state tracking for spoken dialogue systems in [94, 152], to automatic voice activity detection in [442], to speech enhancement…”
BackgroundSelf-citationSpeech Recognition Using Long-Span Temporal Patterns in a Deep Network Model
Siniscalchi, S. M., Yu, D., Deng, L., & Lee, C. (2013) · IEEE Signal Processing Letters · doi:10.1109/lsp.2013.2237901 · Cited by 32
Self-citationAnthes, G. (2013) · Communications of the ACM · doi:10.1145/2461256.2461262 · Cited by 31 · cited 1×
“During the past several years, the techniques developed from deep learning research have already been impacting a wide range of signal and information processing work within the traditional and the new, widened scopes including key aspects of machine learning and artificial intelligence; see overview articles in [7, 2…”
BackgroundBackpropagation training for multilayer conditional random field based phone recognition
Prabhavalkar, R., & Fosler‐Lussier, E. (2010) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2010.5495222 · Cited by 30 · cited 1×
“Also, the benefit of using the full-sequence classification criteria was shown earlier on shallow neural networks in [194, 291].”
Deng, L., Yu, D., & Acero, A. (2005) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tsa.2005.854107 · Cited by 28
Self-citationInvestigations on hessian-free optimization for cross-entropy training of deep neural networks
Wiesler, S., Li, J., & Xue, J. (2013) · Interspeech (USB) · doi:10.21437/interspeech.2013-734 · Cited by 28 · cited 1×
“[390] investigated the Hessian-free optimization method for training the DNN with the cross-entropy objective and empirically analyzed the properties of the method.”
MethodImproved pre-training of Deep Belief Networks using Sparse Encoding Symmetric Machines
Plahl, C., Sainath, T. N., Ramabhadran, B., & Nahamoo, D. (2012) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2012.6288836 · Cited by 27 · cited 1×
“The use of SESM in pre-training DNNs for speech recognition can be found in [284].”
MethodMa, J., & Deng, L. (2003) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/tsa.2003.818075 · Cited by 26 · cited 1×
“Subsequent work added a new hidden layer into the dynamic model to explicitly account for the target-directed, articulatory-like properties in human speech generation [45, 73, 74, 83, 96, 75, 90, 231, 232, 233, 251, 282].”
BackgroundSelf-citationBurkhard, R. J. (2010) · International Journal of E-Health and Medical Communications · doi:10.4018/jehmc.2010010106 · Cited by 25
Optimization Algorithms and Applications for Speech and Language Processing
Wright, S. J., Kanevsky, D., Deng, L., He, X., Heigold, G., & Li, H. (2013) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2013.2283777 · Cited by 25 · cited 2×
“Optimization methods specific to speech recognition problems may need to be taken into account in order to push speech recognition advances to the next level [46, 149, 393].”
BackgroundSelf-citationLarge vocabulary speech recognition using deep tensor neural networks
Yu, D., Deng, L., & Seide, F. T. B. (2012) · Interspeech (USB) · doi:10.21437/interspeech.2012-2 · Cited by 25 · cited 1×
“[421, 422], which extends the conventional DNN by replacing one or more of its layers with a double-projection layer and a tensor layer.”
BackgroundSelf-citationIntroduction to Computational Intelligence Techniques and Areas of Their Applications in Medicine
Niknejad, A., & Petrov́ić, D. (2013) · CRC Press eBooks · doi:10.1201/b15618-8 · Cited by 21
Accelerated parallelizable neural network learning algorithm for speech recognition
Yu, D., & Deng, L. (2011) · Interspeech (USB) · doi:10.21437/interspeech.2011-606 · Cited by 21
Self-citationParameter Estimation of Statistical Models Using Convex Optimization
Jiang, H., & Li, X. (2010) · IEEE Signal Processing Magazine · doi:10.1109/msp.2010.936018 · Cited by 19 · cited 1×
“Many of the discriminative techniques for supervised learning in signal and information processing are shallow architectures such as HMMs [52, 127, 147, 186, 188, 290, 394, 418] and conditional random fields (CRFs) [151, 155, 281, 400, 429, 446].”
BackgroundA Bottom-Up Modular Search Approach to Large Vocabulary Continuous Speech Recognition
Siniscalchi, S. M., Svendsen, T. K., & Lee, C. (2012) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2012.2234115 · Cited by 19
Unsupervised Adaptation With Discriminative Mapping Transforms
Yu, K., Gales, M., & Woodland, P. C. (2009) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2008.2011535 · Cited by 17 · cited 2×
“See a comprehensive review in [220] and some additional review literature or original work in [4, 82, 119, 140, 230, 370, 404, 431, 444].”
BackgroundMethodJang, K. J., Balakrishnan, G., Syed, Z., & Verma, N. (2011) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/iembs.2011.6090411 · Cited by 13
E-Health and Telemedicine: Current State and Future Steps
Кобринский, Б. А. (2014) · E-Health Telecommunication Systems and Networks · doi:10.4236/etsn.2014.34007 · Cited by 12
Nonlinear Compensation Using the Gauss–Newton Method for Noise-Robust Speech Recognition
Zhao, Y. H., & Juang, B.-H. F. (2012) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2012.2199107 · Cited by 11 · cited 1×
“See a comprehensive review in [220] and some additional review literature or original work in [4, 82, 119, 140, 230, 370, 404, 431, 444].”
BackgroundFactorial Hidden Restricted Boltzmann Machines for noise robust speech recognition
Rennie, S. J., Fousek, P., & Dognin, P. (2012) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2012.6288869 · Cited by 8 · cited 1×
“[300] explored the use of a version of the RBM, called the factorial hidden RBM, for noise-robust speech recognition.”
MethodA Multifunctional System of the National Genetic Register
Кобринский, Б. А., Tester, I. B., Demikova, N. S., Yu., S., Marjanchik, B., Николаевна, Т. Л., Yu., G., & Podolnaja, M. (1998) · Studies in health technology and informatics · doi:10.3233/978-1-60750-896-0-121 · Cited by 6
MEDICAL APPLICATIONS OF ARTIFICIAL INTELLIGENCE
Yang, N. (2009) · Xiandai yufang yixue · Cited by 4
Research works printed but not linked: 278
- Abdel-Hamid, 2013, Proceedings of Interspeech,
- Abdel-Hamid, 2013, Proceedings of Interspeech.
- Abdel-Hamid, 2012, Proceedings of International Conference on Acoustics Speech and Signal Processing (ICASSP).
- Acero, 2000, Proceedings of Interspeech
- Alain, 2013, Proceedings of International Conference on Learning Representations (ICLR).
- Arisoy, 2012, Proceedings of the Joint Human Language Technology Conference and the North American Chapter of the Association of Computational Linguistics (HLT-NAACL) Workshop.
- Aslan, 2013, Proceedings of Neural Information Processing Systems (NIPS).
- Ba, 2013, Proceedings of Neural Information Processing Systems (NIPS
- Baldi, 2013, Proceedings of Neural Information Processing Systems (NIPS).
- E. Battenberg , E.Schmidt, and J.Bello. Deep learning for music, special session at International Conference on Acoustics Speech and Signal Processing (ICASSP) (http://www.icassp2014.org/ special_sections.html#ss8), 2014.
- Batternberg, 2012, Proceedings of International Symposium on Music Information Retrieval (ISMIR).
- Bell, 2013, Proceedings of International Conference on Acoustics Speech and Signal Processing (ICASSP).
- Bengio, 2008, Scholarpedia,
- Bengio, 2012, Deep learning of representations for unsupervised and transfer learning, Journal of Machine Learning Research Workshop and Conference Proceedings, 27, 17
- Bengio, 2013, Statistical Language and Speech Processing,, 1
- Bengio, 2013, Proceedings of International Conference on Acoustics Speech and Signal Processing (ICASSP).
- Bengio, 2000, Proceedings of Neural Information Processing Systems (NIPS
- Bengio, 2003, A neural probabilistic language model, Journal of Machine Learning Research,, 3, 1137
- Bengio, 2006, Proceedings of Neural Information Processing Systems (NIPS
- Bengio, 2013, Proceedings of International Conference on Machine Learning (ICML), 2014.
- Bengio, 2013, Proceedings of Neural Information Processing Systems (NIPS).
- Bergstra, 2012, Random search for hyper-parameter optimization, Journal on Machine Learning Research,, 3, 281
- Bilmes, 2010, Dynamic graphical models, IEEE Signal Processing Magazine,, 33, 29
- Bordes, 2013, Machine Learning,
- Bordes, 2011, Proceedings of Association for the Advancement of Artificial Intelligence (AAAI).
- Bottou, 2013, From machine learning to machine reasoning: An essay, Journal of Machine Learning Research,, 14, 3207
- Bottou, 2004, Proceedings of Neural Information Processing Systems (NIPS
- Boulanger-Lewandowski, 2012, Proceedings of International Conference on Machine Learning (ICML).
- Boulanger-Lewandowski, 2013, Proceedings of International Symposium on Music Information Retrieval (ISMIR).
- Bourlard, 1993, Connectionist Speech Recognition: A Hybrid Approach.
- Breiman, 1996, Stacked regression, Machine Learning,, 24, 49, 10.1023/a:1018046112532 · doi:10.1023/a:1018046112532 · Paper page
- Cardinal, 2013, Large vocabulary speech recognition on parallel architectures, IEEE Transactions on Audio, Speech, and Language Processing,, 21, 2290, 10.1109/tasl.2013.2271591 · doi:10.1109/tasl.2013.2271591
- Chen, 2014, Proceedings of International Conference on Learning Representations.
- Chen, 2012, Proceedings of Interspeech.
- Chengalvarayan, 1997, IEEE Transactions on Speech and Audio Processing, 243
- Chengalvarayan, 1997, IEEE Transactions on Speech and Audio Processing,, 232
- Chengalvarayan, 1998, Speech trajectory discrimination using the minimum classification error learning, IEEE Transactions on Speech and Audio Processing,, 6, 505, 10.1109/89.725317 · doi:10.1109/89.725317
- Cho, 2009, Proceedings of Neural Information Processing Systems (NIPS, 342
- Ciresan, 2012, Proceedings of Neural Information Processing Systems (NIPS
- Ciresan, 2010, Neural Computation
- Ciresan, 2011, Proceedings of International Joint Conference on Neural Networks (IJCNN).
- Ciresan, 2012, Proceedings of Computer Vision and Pattern Recognition (CVPR).
- Ciresan, 2012, Proceedings of International Joint Conference on Neural Networks (IJCNN).
- Coates, 2013, Proceedings of International Conference on Machine Learning (ICML).
- Cohen, 2005, Proceedings of International Joint Conference on Artificial Intelligence (IJCAI),, 671
- Collobert, 2011, Proceedings of Artificial Intelligence and Statistics (AISTATS
- Collobert, 2008, Proceedings of International Conference on Machine Learning (ICML).
- Collobert, 2011, Natural language processing (almost) from scratch, Journal on Machine Learning Research,, 12, 2493
- Dahl, 2010, Phone recognition with the mean-covariance restricted boltzmann machine, Proceedings of Neural Information Processing Systems (NIPS, 23, 469
- Dahl, 2013, Proceedings of International Conference on Acoustics Speech and Signal Processing (ICASSP).
- Dahl, 2013, Proceedings of International Conference on Acoustics Speech and Signal Processing (ICASSP).
- Dahl, 2011, Proceedings of International Conference on Acoustics Speech and Signal Processing (ICASSP).
- Dean, 2012, Proceedings of Neural Information Processing Systems (NIPS
- Demuynck, 2013, Proceedings of the Automatic Speech Recognition and Understanding Workshop (ASRU).
- Deng, 1993, IEEE Transactions on Speech and Audio Processing, 471
- Deng, 2003, Mathematical Foundations of Speech and Language Processing,, 115
- Deng, 2011, Proceedings of Asian-Pacific Signal & Information Processing Annual Summit and Conference (APSIPA-ASC).
- Deng, 2012, The MNIST database of handwritten digit images for machine learning research, IEEE Signal Processing Magazine,, 29
- Deng, 2013, Neural Information Processing Systems (NIPS) Workshop on Learning Output Representations.
- Deng, 2013, Asian-Pacific Signal & Information Processing Association Transactions on Signal and Information Processing.
- Deng, 2013, Proceedings of International Conference on Acoustics Speech and Signal Processing (ICASSP).
- Deng, 2001, Proceedings of International Conference on Acoustics Speech and Signal Processing (ICASSP).
- Deng, 2014, Proceedings of International Conference on Acoustics Speech and Signal Processing (ICASSP).
- Deng, 1992, Structural design of a hidden Markov model based speech recognizer using multi-valued phonetic features: Comparison with segmental speech units, Journal of the Acoustical Society of America,, 92, 3058, 10.1121/1.404202 · doi:10.1121/1.404202
- Deng, 2013, Proceedings of International Conference on Acoustics Speech and Signal Processing (ICASSP).
- Deng, 2013, Proceedings of International Conference on Acoustics Speech and Signal Processing (ICASSP).
- Deng, 2004, Challenges in adopting speech recognition, Communications of the Association for Computing Machinery (ACM),, 47, 11
- Deng, 2012, Proceedings of Interspeech.
- Deng, 2013, Proceedings of International Conference on Acoustics Speech and Signal Processing (ICASSP).
- Deng, 2003, Speech Processing — A Dynamic and Optimization-Oriented Approach
- Deng, 1996, Transitional speech units and their representation by regressive Markov states: Applications to speech recognition, IEEE Transactions on speech and audio processing,, 4, 301, 10.1109/89.506934 · doi:10.1109/89.506934
- Deng, 2010, Proceedings of Interspeech.
- Deng, 2012, Proceedings of IEEE Workshop on Spoken Language Technologies
- Deng, 2002, Distributed speech processing in mipad’s multimodal user interface, IEEE Transactions on Speech and Audio Processing,, 10, 605, 10.1109/tsa.2002.804538 · doi:10.1109/tsa.2002.804538
- Deng, 2005, Dynamic compensation of HMM variances using the feature enhancement uncertainty computed from a parametric model of speech distortion, IEEE Transactions on Speech and Audio Processing,, 13, 412, 10.1109/tsa.2005.845814 · doi:10.1109/tsa.2005.845814
- Deng, 2007, Proceedings of International Conference on Acoustics Speech and Signal Processing (ICASSP).
- Deng, 2011, Proceedings of Interspeech.
- Deng, 2009, Neural Information Processing Systems (NIPS) Workshop
- Deng, 2012, Proceedings of International Conference on Acoustics Speech and Signal Processing (ICASSP).
- Deselaers, 2009, Proceedings of 4th Workshop on Statistical Machine Translation,, 233
- Dognin, 2013, Proceedings of the Automatic Speech Recognition and Understanding Workshop (ASRU).
- Erhan, Journal on Machine Learning Research,, 201
- Fernandez, 2013, Proceedings of International Conference on Acoustics Speech and Signal Processing (ICASSP),, 6885
- Fine, 1998, Machine Learning, 41
- Frome, 2013, Proceedings of Neural Information Processing Systems (NIPS).
- Fu, 2007, Proceedings of Interspeech
- Gales, 2011, Robust Speech Recognition of Uncertain or Missing Data: Theory and Application, 101, 10.1007/978-3-642-21317-5_5 · doi:10.1007/978-3-642-21317-5_5
- Gao, 2010, Proceedings of Conference on Information and Knowledge Management (CIKM).
- Gao, 2013, Proceedings of Neural Information Processing Systems (NIPS) Workshop on Deep Learning
- Gao, 2014, Proceedings of Association for Computational Linguistics (ACL).
- Gao, 2011, Proceedings of Special Interest Group on Information Retrieval (SIGIR).
- Gens, 2012, Neural Information Processing Systems (NIPS),
- Gibson, 2010, Error approximation and minimum phone error acoustic model estimation, IEEE Transactions on Audio, Speech, and Language Processing,, 18, 1269, 10.1109/tasl.2009.2032607 · doi:10.1109/tasl.2009.2032607
- Glorot, 2010, Proceedings of Artificial Intelligence and Statistics (AISTATS).
- Glorot, 2011, Proceedings of Artificial Intelligence and Statistics (AIS-TATS
- Goodfellow, 2013, Proceedings of Neural Information Processing Systems (NIPS).
- Graves, 2012, Representation Learning Workshop, International Conference on Machine Learning (ICML),
- Graves, 2006, Proceedings of International Conference on Machine Learning (ICML).
- Graves, 2013, Proceedings of the Automatic Speech Recognition and Understanding Workshop (ASRU).
- Graves, 2013, Proceedings of International Conference on Acoustics Speech and Signal Processing (ICASSP).
- Grezl, 2008, Proceedings of International Conference on Acoustics Speech and Signal Processing (ICASSP).
- Gutmann, 2012, Noise-contrastive estimation of unnormalized statistical models, with applications to natural image statistics, Journal of Machine Learning Research,, 13, 307
- Hain, 2012, IEEE Transactions on Audio, Speech, and Language Processing,, 486
- Hamel, 2010, Proceedings of International Symposium on Music Information Retrieval (ISMIR).
- Hawkins, 2004, On Intelligence: How a New Understanding of the Brain will lead to the Creation of Truly Intelligent Machines
- He, 2011, IEEE Signal Processing Magazine,
- He, 2012, Proceedings of International Conference on Acoustics Speech and Signal Processing (ICASSP).
- He, 2013, Proceedings of the IEEE.
- Heigold, 2013, Investigations on an EM-style optimization algorithm for discriminative training of HMMs, IEEE Transactions on Audio, Speech, and Language Processing,, 21, 2616, 10.1109/tasl.2013.2280234 · doi:10.1109/tasl.2013.2280234
- Heigold, 2013, Proceedings of International Conference on Acoustics Speech and Signal Processing (ICASSP).
- Heintz, 2009, Discriminative input stream combination for conditional random field phone recognition, IEEE Transactions on Audio, Speech, and Language Processing,, 17, 1533, 10.1109/tasl.2009.2022204 · doi:10.1109/tasl.2009.2022204
- Henderson, 2013, Proceedings of Special Interest Group on Disclosure and Dialogue (SIGDIAL).
- Hermans, 2013, Proceedings of Neural Information Processing Systems (NIPS).
- Hermansky, 2000, Proceedings of International Conference on Acoustics Speech and Signal Processing (ICASSP).
- Hinton, 1990, Artificial Intelligence,, 1
- Hinton, 2003, Canadian Psychology,, 10
- Hinton, 2011, A better way to learn features, Communications of the Association for Computing Machinery (ACM),, 54
- Hinton, 2006, Neural Computation,, 1527
- Hinton, 2010, Topics in Cognitive Science,, 1
- Huang, 2012, Proceedings of Association for Computational Linguistics (ACL).
- Huang, 2013, Proceedings of International Conference on Acoustics Speech and Signal Processing (ICASSP).
- Huang, 2013, Proceedings of International Conference on Acoustics Speech and Signal Processing (ICASSP).
- Huang, 2013, Association for Computing Machinery (ACM) International Conference Information and Knowledge Management (CIKM),
- Huang, 2013, Proceedings of International Conference on Acoustics Speech and Signal Processing (ICASSP).
- Huang, 2001, Proceedings of International Conference on Acoustics Speech and Signal Processing (ICASSP).
- Huang, 2013, Proceedings of Interspeech,, 2360
- Humphrey, 2012, Proceedings of International Conference on Machine Learning and Application (ICMLA).
- Humphrey, 2012, Proceedings of International Symposium on Music Information Retrieval (ISMIR).
- Humphrey, 2013, Journal of Intelligent Information Systems,
- Hutchinson, 2012, Proceedings of International Conference on Acoustics Speech and Signal Processing (ICASSP).
- Hutchinson, 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence,, 1944
- Imseng, 2013, Proceedings of the Automatic Speech Recognition and Understanding Workshop (ASRU).
- Jaitly, 2011, Proceedings of International Conference on Acoustics Speech and Signal Processing (ICASSP).
- Jaitly, 2012, Proceedings of Interspeech.
- Jarrett, 2009, Proceedings of International Conference on Computer Vision,, 2146
- Kahou, 2013, Proceedings of International Conference on Multimodal Interaction (ICMI).
- Kang, 2013, Proceedings of International Conference on Acoustics Speech and Signal Processing (ICASSP),, 8012
- Kashiwagi, 2013, Proceedings of the Automatic Speech Recognition and Understanding Workshop (ASRU).
- Kavukcuoglu, 2010, Proceedings of Neural Information Processing Systems (NIPS
- Ketabdar, 2010, IEEE Transactions on Audio, Speech, and Language Processing,, 1094
- Kingsbury, 2009, Proceedings of International Conference on Acoustics Speech and Signal Processing (ICASSP).
- Kingsbury, 2012, Proceedings of Interspeech.
- Kiros, 2013, Proceedings of Neural Information Processing Systems (NIPS) Deep Learning Workshop.
- Ko, 2013, IEEE Transactions on Audio, Speech, and Language Processing, 1285
- Krizhevsky, 2012, Proceedings of Neural Information Processing Systems (NIPS).
- Kubo, 2012, Proceedings of Interspeech.
- Kurzweil, 2012, How to Create a Mind.
- Lal, 2013, IEEE Transactions on Audio, Speech, and Language Processing,, 2506
- Larochelle, 2008, Proceedings of International Conference on Machine Learning (ICML).
- Le, 2013, Proceedings of the Automatic Speech Recognition and Understanding Workshop (ASRU).
- Le, 2010, Proceedings of Empirical Methods in Natural Language Processing (EMNLP, 778
- Le, 2011, Proceedings of International Conference on Acoustics Speech and Signal Processing (ICASSP).
- Le, 2013, IEEE Transactions on Audio, Speech, and Language Processing, 197
- Le, 2011, Proceedings of International Conference on Machine Learning (ICML).
- Le, 2012, Proceedings of International Conference on Machine Learning (ICML).
- LeCun, 2012, Proceedings of European Conference on Computer Vision (ECCV).
- LeCun, 1995, The Handbook of Brain Theory and Neural Networks,, 255
- LeCun, 1998, Proceedings of the IEEE,, 2278
- LeCun, 2007, Proceedings of International Conference on Document Analysis and Recognition (ICDAR).
- Lee, 2004, Proceedings of International Conference on Spoken Language Processing (ICSLP),, 109
- Lee, 2009, Proceedings of International Conference on Machine Learning (ICML).
- Lee, 2011, Communications of the Association for Computing Machinery (ACM),, 95
- Lee, 2010, Proceedings of Neural Information Processing Systems (NIPS
- Lena, 2012, Proceedings of Neural Information Processing Systems (NIPS).
- Li, 2014, IEEE/Association for Computing Machinery (ACM) Transactions on Audio, Speech, and Language Processing, 1
- Li, 2012, Proceedings of IEEE Spoken Language Technology (SLT
- Li, 2013, Proceedings Conference on Affective Computing and Intelligent Interaction (ACII),, 312
- Liao, 2013, Proceedings of International Conference on Acoustics Speech and Signal Processing (ICASSP).
- Liao, 2013, Proceedings of the Automatic Speech Recognition and Understanding Workshop (ASRU).
- Lin, 2009, Proceedings of International Conference on Acoustics Speech and Signal Processing (ICASSP).
- Lin, 2011, Proceedings of Computer Vision and Pattern Recognition (CVPR
- Ling, 2013, Modeling spectral envelopes using restricted boltzmann machines for statistical parametric speech synthesis, International Conference on Acoustics Speech and Signal Processing (ICASSP),, 7825
- Ling, 2013, Articulatory control of HMM-based parametric speech synthesis using feature-space-switched multiple regression
- Lu, 2013, Joint uncertainty decoding for noise robust subspace gaussian mixture models, IEEE Transactions on Audio, Speech, and Language Processing,, 21, 1791, 10.1109/tasl.2013.2248718 · doi:10.1109/tasl.2013.2248718
- Ma, 2000, Computer, Speech and Language
- Maas, 2013, Rectifier nonlinearities improve neural network acoustic models
- Manning, 2009, Introduction to Information Retrieval.
- J.Markoff, 2012, Scientists see promise in deep-learning programs
- Martens, 2010, Deep learning with hessian-free optimization
- Martens, 2011, Learning recurrent neural networks with hessian-free optimization
- McGraw, 2013, Learning lexicons from speech using a pronunciation mixture model, IEEE Transactions on Audio, Speech, and Language Processing,, 21, 357, 10.1109/tasl.2012.2226158 · doi:10.1109/tasl.2012.2226158
- Mikolov, 2013, Efficient estimation of word representations in vector space
- Mikolov, 2010, Proceedings of International Conference on Acoustics Speech and Signal Processing (ICASSP),, 1045
- Mikolov, 2013, Distributed representations of words and phrases and their compositionality
- Minami, 2002, A recognition method with parametric trajectory synthesized using direct relations between static and dynamic feature vector time series, 957
- Mnih, 2008, A scalable hierarchical distributed language model, 1081
- Mnih, 2013, Learning word embeddings efficiently with noise-contrastive estimation
- Mnih, 2012, A fast and simple algorithm for training neural probabilistic language models, 1751
- Mnih, 2013, laying arari with deep reinforcement learning
- Mohamed, 2009, Deep belief networks for phone recognition
- Morin, 2005, Hierarchical probabilistic neural network language models
- Murphy, 2012, Machine Learning
- Nair, 2009, 3-d object recognition with deep belief nets
- Ngiam, 2011, Learning deep energy models
- Ngiam, 2011, Multimodal deep learning
- Olshausen, 2012, Can ‘deep learning’ offer deep insights about visual representation?
- Ostendorf, 1999, Moving beyond the ‘beads-on-a-string’ model of speech
- Oudre, 2011, Probabilistic template-based chord recognition, IEEE Transactions on Audio, Speech, and Language Processing, 19, 2249, 10.1109/tasl.2010.2098870 · doi:10.1109/tasl.2010.2098870
- Palangi, 2013, Learning input and recurrent weight matrices in echo state networks
- Palangi, 2013, Using deep stacking network to improve structured compressive sensing with multiple measurement vectors
200 of 278 shown
Cited by
Deep learning in neural networks: an overview
Schmidhuber, J. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.09.003 · Cited by 18,570
“5 Deep Hierarchical RL (HRL) and Subgoal Learning with FNNs and RNNs Multiple learnable levels of abstraction (Fu, 1977; Lenat and Brown, 1984; Ring, 1994; Bengio et al., 2013; Deng and Yu, 2014) seem as important for RL as for SL.”
BackgroundInfluentialDeep learning in agriculture: A survey
Kamilaris, A., & Prenafeta-Boldú, F. X. (2018) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2018.02.016 · Cited by 5,133
“To the authors’ knowledge, this is the first such survey in the agricultural domain, while a small number of more general surveys do exist (Deng & Yu, 2014), (Wan, et al., 2014), (Najafabadi, et al., 2015), covering related work in DL in other domains.”
BackgroundMethodInfluentialGeometric Deep Learning: Going beyond Euclidean data
Bronstein, M. M., Bruna, J., LeCun, Y., Szlam, A., & Vandergheynst, P. (2017) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2693418 · Cited by 3,661
“and computer vision [5], [6], [7], [8], [9], [10], [11] (the reader is referred to [12], [13] for many additional examples of successful applications of deep learning).”
BackgroundDeep Convolutional Neural Networks for Image Classification: A Comprehensive Review
Rawat, W., & Wang, Z. (2017) · Neural Computation · doi:10.1162/neco_a_00990 · Cited by 3,503
“Prior to this, it was taught that deep multilayered networks (including DCNNs) were too hard to train due to issues with gradient descent and thus were not popular (Bengio et al., 2006; Bengio, 2009; Deng & Yu, 2014; Schmidhuber, 2015; Goodfellow et al., 2016).”
BackgroundMethodInfluentialDeep Learning Based Recommender System: A Survey and New Perspectives
Zhang, S., Yao, L., Sun, A., & Tay, Y. (2020) · ACM Computing Surveys · doi:10.1145/3285029 · Cited by 3,485
“Note that the deep composite models here is dierent from the hybrid deep networks in [21] which refer to the deep architectures that make use of both generative and discriminative components.”
Background
Metadata
- Authors
- Deng, L., & Yu, D.
- Affiliations
- Microsoft (United States) (US)
- Venue
- Foundations and Trends® in Signal Processing
- Publisher
- Emerald
- Published
- 2014-06-30
- Volume
- 7
- Issue
- 3-4
- Pages
- 197-387
- ISSN
- 1932-8346 (ISSN-L)
- 1932-8354
- OpenAlex
- W4205947740
- Cited by
- 3,480
- References
- 21
- Citations by year
- 2026: 113 · 2025: 206 · 2024: 298 · 2023: 410 · 2022: 469 · 2021: 453 · 2020: 404 · 2019: 426 · 2018: 304 · 2017: 188 · 2016: 146 · 2015: 57 · 2014: 3
- FWCI
- 117.48
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Foundations and Trends® in Signal Processing (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- deep learning, signal processing, information processing, speech recognition, computer vision, natural language processing, text processing, information retrieval, multimodal information processing, multi-task learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.