The DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
Learning a better representation of speech soundwaves using restricted boltzmann machines
Abstract
State of the art speech recognition systems rely on preprocessed speech features such as Mel cepstrum or linear predictive coding coefficients that collapse high dimensional speech sound waves into low dimensional encodings. While these have been successfully applied in speech recognition systems, such low dimensional encodings may lose some relevant information and express other information in a way that makes it difficult to use for discrimination. Higher dimensional encodings could both improve performance in recognition tasks, and also be applied to speech synthesis by better modeling the statistical structure of the sound waves. In this paper we present a novel approach for modeling speech sound waves using a Restricted Boltzmann machine (RBM) with a novel type of hidden variable and we report initial results demonstrating phoneme recognition performance better than the current state-of-the-art for methods based on Mel cepstrum coefficients.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
(2010) · International Conference on Machine Learning · Cited by 13,995
Self-citationAn Information-Maximization Approach to Blind Separation and Blind Deconvolution
Bell, A. J., & Sejnowski, T. J. (1995) · Neural Computation · doi:10.1162/neco.1995.7.6.1129 · Cited by 8,986
Training Products of Experts by Minimizing Contrastive Divergence
Hinton, G. E. (2002) · Neural Computation · doi:10.1162/089976602760128018 · Cited by 4,879 · cited 3×
“a weight parameter, wij ∈ W is as follows [6]:”
BackgroundMethodSelf-citationA Practical Guide to Training Restricted Boltzmann Machines
Hinton, G. E. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-35289-8_32 · Cited by 2,852
Self-citationInformation processing in dynamical systems: foundations of harmony theory
Smolensky, P. (1986) · The MIT Press eBooks · Cited by 1,673
Advances in Neural Information Processing Systems 19 (NIPS 2006)
Simonotto, E., Whalley, H. C., Lawrie, S. M., Murray, L. M., McGonigle, D. J., & Storkey, A. (2006) · Cited by 1,651
(2009) · neural information processing systems · Cited by 962
Efficient coding of natural sounds
Lewicki, M. S. (2002) · Nature Neuroscience · doi:10.1038/nn831 · Cited by 697
An application of recurrent nets to phone probability estimation
Robinson, A. J. (1994) · IEEE Transactions on Neural Networks · doi:10.1109/72.279192 · Cited by 441
Phone Recognition with the Mean-Covariance Restricted Boltzmann Machine
Dahl, G. E., Ranzato, M., Mohamed, A., & Hinton, G. E. (2010) · Cited by 266
Self-citation
Research works printed but not linked: 1
- Mohamed, Deep belief networks for phone recognition
Cited by
Sorted by citations
A survey of deep neural network architectures and their applications
Liu, W., Wang, Z., Liu, X., Zeng, N., Liu, Y., & Alsaadi, F. E. (2017) · Neurocomputing · doi:10.1016/j.neucom.2016.12.038 · Cited by 3,239
“During the past few decades, machine learning algorithms have been widely used in areas such as automatic speech recognition (ASR) and acoustic modeling [76,116,118,126].”
MethodZheng, W.-L., & Lu, B.-L. (2015) · IEEE Transactions on Autonomous Mental Development · doi:10.1109/tamd.2015.2431497 · Cited by 2,477
“MLP, SVMs, CRFs) in many challenge tasks, especially in speech and image domains [29]–[31].”
BackgroundTowards End-To-End Speech Recognition with Recurrent Neural Networks
Graves, A., & Jaitly, N. (2014) · Cited by 1,861
Self-citationDeep Learning: Methods and Applications
Deng, L. (2014) · now publishers, Inc. eBooks · doi:10.1561/9781601988157 · Cited by 1,789
Improving deep neural networks for LVCSR using rectified linear units and dropout
Dahl, G. E., Sainath, T. N., & Hinton, G. E. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6639346 · Cited by 1,310
“the only use of these units in speech to date was in [10] where Gaussian-ReLU RBMs were used to learn features from raw, unlabeled acoustic data to later use in phone recognition experiments on TIMIT.”
MethodSelf-citationA review of unsupervised feature learning and deep learning for time-series modeling
Längkvist, M., Karlsson, L., & Loutfi, A. (2014) · Pattern Recognition Letters · doi:10.1016/j.patrec.2014.01.008 · Cited by 1,276
“Only a few works have attempted this with speech recognition (Jaitly and Hinton, 2011; Smith and Lewicki, 2005) and physiological data (Wulsin et al.”
BackgroundLei, Y., Jia, F., Lin, J., Xing, S., & Ding, S. X. (2016) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2016.2519325 · Cited by 1,191
“Traditionally, the learned features f i of x is obtained by combining these local features f i j in an aggregate way [38].”
BackgroundSpeech Recognition Using Deep Neural Networks: A Systematic Review
Nassif, A. B., Shahin, I. M. A., Attili, I. B., Azzeh, M., & Shaalan, K. F. (2019) · IEEE Access · doi:10.1109/access.2019.2896880 · Cited by 1,215
AST: Audio Spectrogram Transformer
Gong, Y., Chung, Y.-A., & Glass, J. (2021) · Interspeech 2021 · doi:10.21437/interspeech.2021-698 · Cited by 1,159
“E[1] E[2] E[3] E[4] E[5] E[6] E[7] E[8] P[1] P[2] P[3] P[4] P[5] P[6] P[7] P[8] Linear Output”
BackgroundAdieu features? End-to-end speech emotion recognition using a deep convolutional recurrent network
Trigeorgis, G., Ringeval, F., Brueckner, R., Marchi, E., Nicolaou, M. A., Schuller, B., & Zafeiriou, S. (2016) · 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2016.7472669 · Cited by 904
“In one of the first studies that suggested learning better features for automatic speech recognition (ASR) that used directly the speech waveform was Jaitly and Hinton [6].”
Deep Learning for Audio Signal Processing
Purwins, H., Li, B., Virtanen, T. I., Schlüter, J., Chang, S.-Y., & Sainath, T. N. (2019) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2019.2908700 · Cited by 929
“[18] and [19] use a full-resolution magnitude spectrum, [20]–[23] directly use a raw waveform representation of the audio signals as inputs and learn datadriven filters jointly with the rest of the network for the target tasks.”
MethodA tutorial survey of architectures, algorithms, and applications for deep learning
Deng, L. (2014) · APSIPA Transactions on Signal and Information Processing · doi:10.1017/atsip.2013.9 · Cited by 747
“The work also showed the importance of raw speech features of spectrogram --- back from the longpopular MFCC features, but not yet reaching the raw speechwaveform level (Sheikhzadeh and Deng, 1994; Jaitly and Hinton, 2011).”
BackgroundLearning the speech front-end with raw waveform CLDNNs
Sainath, T. N., Weiss, R. J., Senior, A., Wilson, K., & Vinyals, O. (2015) · Interspeech (USB) · doi:10.21437/interspeech.2015-1 · Cited by 456
“Past work has achieved phase invariance using convolutional layers which pool in time [4, 5, 7] or DNN layers with large, potentially overcomplete, hidden units [6], which can capture the same filter shape at a variety of phases.”
BackgroundMethodInfluentialA comprehensive survey and analysis of generative models in machine learning
Harshvardhan, G. M., Gourisaria, M. K., Pandey, M., & Rautaray, S. S. (2020) · Computer Science Review · doi:10.1016/j.cosrev.2020.100285 · Cited by 477
End-to-end learning for music audio
Dieleman, S., & Schrauwen, B. (2014) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2014.6854950 · Cited by 392
“Although there has been some work about learning features directly from speech signal fragments [9, 10, 11], to our knowledge feature learning has not been applied directly to raw music audio signals in literature.”
BackgroundDeep learning-based clustering approaches for bioinformatics
Karim, M. R., Beyan, O., Zappa, A., Costa, I. G., Rebholz‐Schuhmann, D., Cochez, M., & Decker, S. (2019) · Briefings in Bioinformatics · doi:10.1093/bib/bbz170 · Cited by 319
“NMMC and UMMC) DBN is employed as the feature extractor in which Restricted Boltzmann Machines (RBM) [60] formed the basic building block.”
MethodAnalysis of CNN-based speech recognition system using raw speech as input
Palaz, D., Magimai-Doss, M., & Collobert, R. (2015) · Interspeech 2015 · doi:10.21437/interspeech.2015-3 · Cited by 288
“Features learning from the raw speech signal using neural networks-based systems has also been investigated in [12].”
BackgroundSpeech Emotion Classification Using Attention-Based LSTM
Xie, Y., Liang, R., Liang, Z., Huang, C., Zou, C., & Schuller, B. W. (2019) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2019.2925934 · Cited by 286
“(2) Hinton [45] believed that deep learning has the ability to automatically learn feature changes, and can learn deep features related to tasks from the underlying speech features.”
BackgroundFundamentals of speech recognition
Li, J., Deng, L., Haeb‐Umbach, R., & Gong, Y. (2015) · Elsevier eBooks · doi:10.1016/b978-0-12-802398-3.00002-7 · Cited by 273
Intelligent Fault Diagnosis Method Based on Full 1-D Convolutional Generative Adversarial Network
Guo, Q., Li, Y., Song, Y., Wang, D., & Chen, W. (2019) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2019.2934901 · Cited by 262
“Deep learning, which implies a breakthrough in artificial intelligence, exhibits excellent performance in speech recognition [3], [4], image restoration [5], [6], natural language processing [7], multimodal image-text [8], and games [9].”
BackgroundSleep Stage Classification Using Unsupervised Feature Learning
Längkvist, M., Karlsson, L., & Loutfi, A. (2012) · Advances in Artificial Neural Systems · doi:10.1155/2012/107046 · Cited by 248
“This has proven to give good results in other areas such as vision tasks [10], object recognition [16], motion capture data [17], speech recognition [18], and bacteria identification [19].”
BackgroundLiu, H., Zhou, J., Xu, Y., Zheng, Y., Peng, X., & Jiang, W. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.07.034 · Cited by 244
Deep learning for human affect recognition: Insights and new developments
Rouast, P. V., Adam, M. T. P., & Chiong, R. (2021) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2018.2890471 · Cited by 243
“Feature learning directly from the raw waveform was proposed in 2011 [133].”
BackgroundSpeech acoustic modeling from raw multichannel waveforms
Hoshen, Y., Weiss, R. J., & Wilson, K. (2015) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2015.7178847 · Cited by 232
“This is in line with previously reported results [2, 3, 5].”
MethodResultMultichannel Signal Processing With Deep Neural Networks for Automatic Speech Recognition
Sainath, T. N., Weiss, R. J., Wilson, K., Li, B., Narayanan, A., Variani, E., Bacchiani, M., Izhak Shafran, J., Senior, A., Chin, K. K., et al. (2017) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2017.2672401 · Cited by 231
“The mic array is placed at the center of the room and closer to the wall, respectively, to capture reasonably different reverberation characteristics.”
BackgroundUnsupervised speech representation learning using WaveNet autoencoders
Chorowski, J., Weiss, R. J., Bengio, S., & van den Oord, A. (2019) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2019.2938863 · Cited by 325
“to learn auditory-like frequency decompositions on music [23] and speech [24]–[27] in their lower layers.”
BackgroundConvolutional Neural Networks-based continuous speech recognition using raw speech signal
Palaz, D., Magimai.-Doss, M., & Collobert, R. (2015) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2015.7178781 · Cited by 194
“Features learning from raw speech using neural networks-based systems has been investigated in [17].”
BackgroundRecent progresses in deep learning based acoustic models
Yu, D., & Li, J. (2017) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2017.7510508 · Cited by 170
“Inspired by the end-to-end processing in the machine learning community, there are always efforts [51]–[54] trying to replace the Mel-filter-bank extraction by directly learning filters using a network to process the raw speech waveforms and training it jointly with the recognizer network.”
MethodXing, S., Lei, Y., Wang, S., & Jia, F. (2020) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2020.2972461 · Cited by 172
“RBM is a kind of stochastic neural network with two layers [25].”
BackgroundPalaz, D., Magimai-Doss, M., & Collobert, R. (2019) · Speech Communication · doi:10.1016/j.specom.2019.01.004 · Cited by 168
“Feature learning from raw speech using neural networks-based systems has been investigated in [10].”
Background3D CNN-Based Speech Emotion Recognition Using K-Means Clustering and Spectrograms
Hajarolasvadi, N., & Demirel, H. (2019) · Entropy · doi:10.3390/e21050479 · Cited by 157
“[15] proposed a limited-weight-sharing scheme that models the speech features for speech recognition systems while [11] proposed a new method for modeling speech signals using Restricted Boltzmann Machine.”
MethodA Survey of Automatic Text Summarization: Progress, Process and Challenges
Mridha, M. F., Lima, A. A., Nur, K. M., Das, S. C., Hasan, M., & Kabir, M. M. (2021) · IEEE Access · doi:10.1109/access.2021.3129786 · Cited by 173
“[293], [294], [295], [296], [297], [298] are the research that focused on RBM method for text summarization.”
BackgroundDeep convex net: a scalable architecture for speech pattern classification
Deng, L., & Yu, D. (2011) · Interspeech (USB) · doi:10.21437/interspeech.2011-607 · Cited by 152
“And many years later the use of more powerful Restricted Boltzmann Machine (RBM) overcomes some difficulty encountered earlier [10].”
MethodDeep learning methods for underwater target feature extraction and recognition
Hu, G., Wang, K., Peng, Y., Qiu, M., Shi, J., & Liu, L. (2018) · Computational Intelligence and Neuroscience · doi:10.1155/2018/1214301 · Cited by 151
“In 2012, four major international scientific research institutions summed up the progress made by deep learning in sound recognition task [15] and pointed out in the paper that the experiments show the effect of deep neural network is better than the one of traditional Gaussian mixed model.”
BackgroundTowards deeper understanding: Deep convex networks for semantic utterance classification
Tür, G., Deng, L., Hakkani-Tür, D. Z., & He, X. (2012) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2012.6289054 · Cited by 123
“Following the introduction of this RBM learning and layer-by-layer construction of deep architectures, DBNs have been successfully used for numerous tasks in speech and language processing, including acoustic modeling [8, 9, 10] and intent determination [6].”
MethodJamalipour, A., & Murali, S. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3126811 · Cited by 143
Aldwairi, T., Perera, D., & Novotny, M. A. (2018) · Computer Networks · doi:10.1016/j.comnet.2018.07.025 · Cited by 125
An Ensemble Stacked Convolutional Neural Network Model for Environmental Event Sound Recognition
Li, S., Yao, Y. F., Hu, J., Liu, G., Yao, X., & Hu, J. (2018) · Applied Sciences · doi:10.3390/app8071152 · Cited by 118
EOG-based drowsiness detection using convolutional neural networks
Zhu, X., Zheng, W., Lu, B., Chen, X., Chen, S., & Wang, C. (2014) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2014.6889642 · Cited by 117
“Such unsupervised training method has been proved to reach awesome results in areas such as object recognition [15] and speech recognition [16].”
MethodKoçak, B., Durmaz, E. Ş., Kuş, E. A., & Ulusan, M. B. (2019) · American Journal of Roentgenology · doi:10.2214/ajr.18.20443 · Cited by 120
A first look into a Convolutional Neural Network for speech emotion detection
Bertero, D., & Fung, P. (2017) · 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2017.7953131 · Cited by 117
“Deep learning from raw audio is now replacing traditional feature-based learning in all speech-related tasks, with Automatic Speech Recognition the most prominent field [6, 7, 8].”
BackgroundThree Classes of Deep Learning Architectures and Their Applications: A Tutorial Survey
Deng, L. (2012) · APSIPA Transactions on Signal and Information Processing · Cited by 111
LEAF: A Learnable Frontend for Audio ClassificationPreprint
Zeghidour, N., Teboul, O., Quitry, F. D. C., & Tagliasacchi, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.08596 · Cited by 116
“The first notable contributions in this field emerged for ASR, with Jaitly & Hinton (2011) pretraining Restricted Boltzmann Machines from the waveform, and Palaz et al. (2013) training a hybrid DNN-HMM model, replacing mel-filterbanks by several layers of convolution.”
MethodEmotion Recognition from Variable-Length Speech Segments Using Deep Learning on Spectrograms
Ma, X., Wu, Z., Jia, J., Xu, M., Meng, H. M. L., & Cai, L. (2018) · Interspeech (USB) · doi:10.21437/interspeech.2018-2228 · Cited by 102
“In speech recognition, one of the first studies that suggested learning better features for automatic speech recognition (ASR) that used directly the speech waveform was Jaitly and Hinton [10].”
Barushka, A., & Hájek, P. (2018) · Applied Intelligence · doi:10.1007/s10489-018-1161-y · Cited by 106
“The combination of dropout regularization and ReL units has shown promising synergistic effects [38].”
BackgroundDeep learning: from speech recognition to language and multimodal processing
Deng, L. (2016) · APSIPA Transactions on Signal and Information Processing · doi:10.1017/atsip.2015.22 · Cited by 97
“As an initial attempt toward this goal, the study carried out by Jaitly and Hinton [95] made use of speech sound waves as the raw input feature to a deep learning system.”
Learning Invariant Representations of Molecules for Atomization Energy Prediction
Montavon, G., Hansen, K., Fazli, S., Biegler, F., Ziehe, A., Tkatchenko, A., von Lilienfeld, O. A., & Müller, K.-R. (2014) · Cited by 97
Comparison and Analysis of SampleCNN Architectures for Audio Classification
Kim, T., Lee, J., & Nam, J. (2019) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2019.2909479 · Cited by 89
“waveforms to uncover hearing mechanism or improve audio classification date back to more than a decade [18]–[21].”
BackgroundGas Recognition under Sensor Drift by Using Deep Learning
Liu, Q., Hu, X., Ye, M., Cheng, X., & Li, F. (2015) · International Journal of Intelligent Systems · doi:10.1002/int.21731 · Cited by 87
Unsupervised Fault Diagnosis of a Gear Transmission Chain Using a Deep Belief Network
He, J., Yang, S., & Gan, C. (2017) · Sensors · doi:10.3390/s17071564 · Cited by 74
“Deep learning [26–29] has recently proven its capability for unsupervised feature learning in various fields such as speech recognition [30,31], motion capture [32], visual recognition [33,34], and physiology [35,36].”
BackgroundUsmani, U. A., Happonen, A., & Watada, J. (2022) · Lecture notes in networks and systems · doi:10.1007/978-3-031-10464-0_11 · Cited by 69
Deep Representation Learning in Speech Processing: Challenges, Recent Advances, and Future TrendsPreprint
Latif, S., Rana, R. K., Khalifa, S., Jurdak, R., Qadir, J., & Schuller, B. W. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.00378 · Cited by 108
“For instance, Jaitly and Hinton used RBMs for phoneme recognition in [52], and showed that RBMs can learn more discriminative features that achieved better performance compared to MFCCs. Interestingly, RBMs can also learn filterbanks from raw speech.”
BackgroundCombining bag-of-words and sentiment features of annual reports to predict abnormal stock returns
Hájek, P. (2017) · Neural Computing and Applications · doi:10.1007/s00521-017-3194-2 · Cited by 65
“The synergistic effects of combining rectified linear units with dropout regularization have been demonstrated by [69].”
BackgroundMachine Learning for Speaker Recognition
Mak, M., & Chien, J. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108552332 · Cited by 55
Deep Learning in Paralinguistic Recognition Tasks: Are Hand-crafted Features Still Relevant?
Wagner, J., Schiller, D., Seiderer, A., & André, E. (2018) · Interspeech (USB) · doi:10.21437/interspeech.2018-1238 · Cited by 54
“The finding that DNNs have the potential to learn robust feature representations [13, 14] has led to breakthroughs in various audio classification tasks such as speech recognition [15], sound event classification [16] and automatic music transcriptions [17].”
BackgroundWaldmann, I. P. (2016) · The Astrophysical Journal · doi:10.3847/0004-637x/820/2/107 · Cited by 54
“…(e.g., Krizhevsky et al. 2012; Liu et al. 2014; Wang et al. 2016; Shen et al. 2015), speech and music recognition (e.g., Hung et al. 2005; Jaitly & Hinton 2011; Zhang & Wu 2013; Pradeep & Kumaraswamy 2014), biology (e.g., Head-Gordon & Stillinger 1993; Plebe 2007; Wu & McLarty 2012;…”
BackgroundBrain–computer interfacing: more than the sum of its parts
Scherer, R., Faller, J., Balderas, D., Friedrich, E. V. C., Pröll, M., Allison, B. Z., & Muller-Putz, G. R. (2012) · Soft Computing · doi:10.1007/s00500-012-0895-4 · Cited by 52
“, speech representation (Navdeep and Hinton 2011), or features extracted from EEG signals (Balderas et al.”
BackgroundVásquez-Correa, J. C., Klumpp, P., Orozco‐Arroyave, J. R., & Nöth, E. (2019) · Interspeech (USB) · doi:10.21437/interspeech.2019-1405 · Cited by 50
“Recently, deep learning methods have also been considered to extract features from speech for specific tasks [3, 4].”
BackgroundKoçak, B., Durmaz, E. Ş., Kaya, O. K., & Kılıçkesmez, Ö. (2019) · Acta Radiologica · doi:10.1177/0284185119881742 · Cited by 50
Towards Deep Developmental Learning
Sigaud, O., & Droniou, A. (2015) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tamd.2015.2496248 · Cited by 48
Attention-Based Dense LSTM for Speech Emotion Recognition
Xie, Y., Liang, R., LIANG, Z., & Zhao, L. (2019) · IEICE Transactions on Information and Systems · doi:10.1587/transinf.2019edl8019 · Cited by 48
“[14] N. Jaitly and G. Hinton, “Learning a better representation of speech soundwaves using restricted boltzmann machines,” in IEEE International Conference on Acoustics, Speech and Signal Processing, pp.5884–5887, 2011.”
BackgroundKim, J., Urbano, J., Liem, C. C. S., & Hanjalić, A. (2019) · Neural Computing and Applications · doi:10.1007/s00521-019-04076-1 · Cited by 49
“Although variants of CNN using 1-dimensional filters also were suggested by [12, 57–59] to learn features directly from a raw audio signal in an end-to-end manner, not many works managed to use them on music classification tasks successfully [60].”
MethodAn Overview of Deep-Structured Learning for Information Processing
Deng, L. (2011) · Cited by 47
Novel Unsupervised Auditory Filterbank Learning Using Convolutional RBM for Speech Recognition
Sailor, H. B., & Patil, H. A. (2016) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2016.2607341 · Cited by 47
“This may be due to the fact that RBM was trained on randomly selected smaller windows of the speech signal and hence, they were in any random temporal phase [19].”
BackgroundMethodLooking into Your Speech: Learning Cross-modal Affinity for Audio-visual Speech Separation
Lee, J., Chung, S.-W., Kim, S., Kang, H.-G., & Sohn, K. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00139 · Cited by 47
“However, reliably separating a target speech signal for humancomputer interaction (HCI) systems such as speech recognition [3, 4, 5], speaker recognition [6, 7, 8], and emotion recognition [9, 10] is still a challenging task because it is an ill-posed problem.”
BackgroundA Deep Neural Network for Acoustic-Articulatory Speech Inversion
Uría, B., Renals, S. J., & Richmond, K. (2011) · Cited by 45
Gao, S., Wang, X., Miao, X., Su, C., & Li, Y. (2019) · Journal of Signal Processing Systems · doi:10.1007/s11265-019-01463-8 · Cited by 47
“As a major breakthrough in the domain of data science, deep learning has shown excellent performance in speech recognition [6, 7], NLP [8], multi-mode data processing [9, 10], image restoration [11, 12] and other applications.”
BackgroundUnsupervised feature learning for electronic nose data applied to Bacteria Identification in Blood.
Längkvist, M., & Loutfi, A. (2011) · Cited by 43
Deep-Sparse-Representation-Based Features for Speech Recognition
Sharma, P., Abrol, V., & Sao, A. K. (2017) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2017.2748240 · Cited by 41
“[41] H. B. Sailor and H. A. Patil, “Novel unsupervised auditory filterbank learning using convolutional RBM for speech recognition,” IEEE/ACM Trans.”
MethodConvolutional Neural Networks for Raw Speech Recognition
Passricha, V., & Aggarwal, R. K. (2018) · IntechOpen eBooks · doi:10.5772/intechopen.80026 · Cited by 40
“Some efforts were also made to model the raw speech signal with little or no preprocessing [34].”
BackgroundSpeech Quality Assessment Over Lossy Transmission Channels Using Deep Belief Networks
Affonso, E. T., Rosa, R. L., & Rodríguez, D. Z. (2017) · IEEE Signal Processing Letters · doi:10.1109/lsp.2017.2773536 · Cited by 38
“In [42], the RBM is used for a better representation of speech sound waves, their results reported that the phoneme recognition performance with their proposed RBM model overcomes MFCC-based solutions.”
MethodCLUSTERING-BASED FEATURE LEARNING ON VARIABLE STARS
Mackenzie, C., Pichara, K., & Protopapas, P. (2016) · The Astrophysical Journal · doi:10.3847/0004-637x/820/2/138 · Cited by 37
“Raw time series data has been used with success in a limited number of problems, most notably by Jaitly & Hinton (2011).”
MethodEnd-to-end Phoneme Sequence Recognition using Convolutional Neural NetworksPreprint
Palaz, D., Collobert, R., & Magimai.-Doss, M. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1312.2137 · Cited by 42
“The work presented in [19] successfully investigates features learning from raw speech for phoneme recognition.”
BackgroundImproving robustness of deep neural networks via spectral masking for automatic speech recognition
Li, B., & Sim, K. C. (2013) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru.2013.6707743 · Cited by 37
“The log Mel filterbank (FBank) features [15] or the waveform signals [16] have been shown to outperform the human engineered Mel frequency cepstral coefficient (MFCC) features as inputs for DNNs.”
MethodAST: Audio Spectrogram TransformerPreprint
Gong, Y., Chung, Y.-A., & Glass, J. R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.01778 · Cited by 447
“E[1] E[2] E[3] E[4] E[5] E[6] E[7] E[8] P[1] P[2] P[3] P[4] P[5] P[6] P[7] P[8] Linear Output”
BackgroundMultiscale Audio Spectrogram Transformer for Efficient Audio Classification
Zhu, W., & Omar, M. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10096513 · Cited by 34
Li, B., & Sim, K. C. (2013) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2013.6639102 · Cited by 33
“With a powerful model, less pre-processed input representations, such as FBank features [1] and waveforms [16], could yield better”
MethodMRI-based radiomics analysis for differentiating phyllodes tumors of the breast from fibroadenomas
Tsuchiya, M., Masui, T., Terauchi, K., Yamada, T., Katyayama, M., Ichikawa, S., Noda, Y., & Goshima, S. (2022) · European Radiology · doi:10.1007/s00330-021-08510-8 · Cited by 33
“Fourth, MR image data were collected from three different scanners with different magnetic field strengths, which can affect the extracted features [26, 27].”
MethodDeep Neural Network Acoustic Models for ASR
Mohamed, A. (2014) · TSpace (University of Toronto) · Cited by 32
Filterbank learning using Convolutional Restricted Boltzmann Machine for speech recognition
Sailor, H. B., & Patil, H. A. (2016) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2016.7472808 · Cited by 31
“Recently, Restricted Boltzmann Machine (RBM) with rectified linear units (ReLUs) was used for representation learning in ASR task [9].”
BackgroundMethodInfluentialVariani, E., Sainath, T. N., Izhak Shafran, J., & Bacchiani, M. (2016) · Interspeech (USB) · doi:10.21437/interspeech.2016-1459 · Cited by 31
“The emergence of deep neural networks for acoustic modeling [7] has recently sparked a resurgence in data-driven techniques which jointly estimate filter bank and AM parameters [8, 9, 10, 11, 12, 13, 14, 15].”
MethodQian, Y., Ubale, R., Ramanaryanan, V., Lange, P., Suendermann-Oeft, D., Evanini, K., & Tsuprun, E. (2017) · 2017 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) · doi:10.1109/asru.2017.8268987 · Cited by 62
“, using filter-bank features instead of MFCC [17] or directly using speech waveform [18].”
MethodAcoustic Analysis for Automatic Speech Recognition
O’Shaughnessy, D. D. (2013) · Proceedings of the IEEE · doi:10.1109/jproc.2013.2251592 · Cited by 28
“den units and one layer of stochastic observable units [46].”
BackgroundDeep Neural Network Frontend for Continuous EMG-Based Speech Recognition
Wand, M., & Schmidhuber, J. (2016) · Interspeech (USB) · doi:10.21437/interspeech.2016-340 · Cited by 27
“Current research targets two main directions: First, the feature preprocessing frontend is gradually replaced by networks trained on raw waveforms [24], and second, the HMM backend is replaced by a recurrent neural network architecture [25] usually implemented as LSTM ( Long Short-Term Memory…”
MethodEvaluating raw waveforms with deep learning frameworks for speech emotion recognition
Kilimci, Z. H., Bayraktar, Ü., & Küçükmanіşa, A. (2025) · Multimedia Tools and Applications · doi:10.1007/s11042-025-20930-y · Cited by 35
“…deep neural network-based acoustic method which fed with raw multichannel waveforms as input by inspiring studies Dai et al. [55], Hoshen et al. [56 , 57], Tüske et al. [58], Jaitly and Hinton [59] with - out performing feature extraction stage by constructing high-level representative features.”
Interval Type-2 Fuzzy Restricted Boltzmann Machine
Hosseini-Pozveh, M. S., Safayani, M., & Mirzaei, A. (2020) · IEEE Transactions on Fuzzy Systems · doi:10.1109/tfuzz.2020.2969120 · Cited by 23
“RestrictedBoltzmannmachine(RBM)[1]is aspecialtypeofneuralnetworkwithmanyapplicationslike imageprocessing[2],[3],topicmodeling[4],classification,[5] andspeechrecognition[6],[7].”
End-to-End Speech Recognition in Agglutinative Languages
Mamyrbayev, O. J., Alimhan, K., Жумажанов, Б., Turdalykyzy, T., & Gusmanova, F. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-42058-1_33 · Cited by 24
Kim, S. K., & Kang, H.-B. (2017) · Neurocomputing · doi:10.1016/j.neucom.2017.09.081 · Cited by 25
Seven neurons memorizing sequences of alphabetical images via spike-timing dependent plasticity
Osogami, T., & Otsuka, M. (2015) · Scientific Reports · doi:10.1038/srep14149 · Cited by 22
“Similar to standard recurrent neural networks 40,49,50 , the DyBM can be unfolded though time 51,52 .”
Speech classification using SIFT features on spectrogram images
Nguyen, Q. T., & Bui, T. D. (2016) · Vietnam Journal of Computer Science · doi:10.1007/s40595-016-0071-3 · Cited by 22
Spam Filtering Using Regularized Neural Networks with Rectified Linear Units
Barushka, A., & Hájek, P. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-49130-1_6 · Cited by 22
Sertolli, B., Ren, Z. X., Schuller, B. W., & Cummins, N. (2021) · Computer Speech & Language · doi:10.1016/j.csl.2021.101204 · Cited by 22
Cognitive Speech Coding: Examining the Impact of Cognitive Speech Processing on Speech Compression
Cerňak, M., Asaei, A., & Hyafil, A. (2018) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2761895 · Cited by 21
“At present, speech recognition and synthesis are highly dependent on machine learning tools and big data, and speech coding can benefit from it, for example by learning of better speech representation [76].”
Neural Networks and Deep Learning
Theodoridis, S. (2020) · Elsevier eBooks · doi:10.1016/b978-0-12-818803-3.00030-1 · Cited by 22
A survey on preprocessing and classification techniques for acoustic scene
Singh, V. K., Sharma, K., & Sur, S. N. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.120520 · Cited by 23
Multi-level region-of-interest CNNs for end to end speech recognition
Singhal, S., Passricha, V., Sharma, P., & Aggarwal, R. K. (2018) · Journal of Ambient Intelligence and Humanized Computing · doi:10.1007/s12652-018-1146-z · Cited by 21
“Earlier, Jaitly and Hinton (2011) used restricted Boltzmann machines, Tüske et al. (2014) used DNN and Sainath et al. (2015b) used the combination of convolutional, long short-term memory, and fully connected DNN to process the raw waveforms.”
MethodQuantum Inspired Training for Boltzmann MachinesPreprint
Wiebe, N., Kapoor, A., Granade, C., & Svore, K. M. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1507.02642 · Cited by 21
“Due to the emergence of CD, Boltzmann machines, specifically restricted Boltzmann machines (RBMs) and their deep layered counterparts (DBMs), have become standard tools for solving problems in vision and speech recognition [3, 4].”
Mamyrbayev, O. J., Alimhan, K., Оралбекова, Д., Bekarystankyzy, A., & Жумажанов, Б. (2022) · Eastern-European Journal of Enterprise Technologies · doi:10.15587/1729-4061.2022.252801 · Cited by 20
“Some studies used deep neural networks [2] to build acoustic models, and others used RNNs [3] to build language models with excellent results.”
BackgroundUsing an autoencoder with deformable templates to discover features for automated speech recognition
Jaitly, N., & Hinton, G. E. (2013) · Interspeech (USB) · doi:10.21437/interspeech.2013-432 · Cited by 19
“The at values were outputs of rectified linear units (ReLU) [3, 4].”
BackgroundSelf-citationHindi speech recognition using time delay neural network acoustic modeling with i-vector adaptation
Kumar, A., & Aggarwal, R. K. (2020) · International Journal of Speech Technology · doi:10.1007/s10772-020-09757-0 · Cited by 19
“DNN/HMM hybrid models were also used to train the acoustic model in various work to improve the ASR system’s performance (Jaitly and Hinton 2011; Dahl et al. 2011a; Seide et al. 2011).”
Method
Metadata
- Authors
- Jaitly, N., & Hinton, G.
- Affiliations
- University of Toronto (CA)
- Venue
- 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
- Publisher
- IEEE
- Published
- 2011-05
- Pages
- 5884-5887
- ISSN
- 1520-6149 (ISSN-L)
- OpenAlex
- W2132037657
- Cited by
- 231
- References
- 19
- Citations by year
- 2026: 3 · 2025: 8 · 2024: 3 · 2023: 11 · 2022: 8 · 2021: 24 · 2020: 34 · 2019: 29 · 2018: 21 · 2017: 16 · 2016: 22 · 2015: 16 · 2014: 17 · 2013: 9 · 2012: 5
- FWCI
- 9.01
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE International Conference on Acoustics Speech and Signal Processing (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Phonetics and Phonology Research
- Keywords
- restricted Boltzmann machine, phoneme recognition, hidden variables, speech feature extraction, speech synthesis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
