The DOI matches its Crossref record; 4 of the 4 references it cites were found in a registry or an index.
Acoustic Modeling Using Deep Belief Networks
Abstract
Gaussian mixture models are currently the dominant technique for modeling the emission distribution of hidden Markov models for speech recognition. We show that better phone recognition on the TIMIT dataset can be achieved by replacing Gaussian mixture models by deep neural networks that contain many layers of features and a very large number of parameters. These networks are first pre-trained as a multi-layer generative model of a window of spectral feature vectors without making use of any discriminative information. Once the generative pre-training has designed the features, we perform discriminative fine-tuning using backpropagation to adjust the features slightly to make them better at predicting a probability distribution over the states of monophone hidden Markov models.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336
Self-citationTraining Products of Experts by Minimizing Contrastive Divergence
Hinton, G. E. (2002) · Neural Computation · doi:10.1162/089976602760128018 · Cited by 4,879 · cited 4×
“gradient ascent in the log likelihood quite well by just addi ng the derivatives for the first two layers [10].”
BackgroundMethodInfluentialSelf-citationGreedy layer-wise training of deep networks
Bengio, Y., Lamblin, P., Popovici, D., & Larochelle, H. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0024 · Cited by 4,841
Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition
Dahl, G. E., Yu, D., Deng, L., & Acero, A. (2012) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2011.2134090 · Cited by 3,011
Self-citation
Cited by
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Self-citation(2014) · Cited by 33,419
Representation Learning: A Review and New Perspectives
Bengio, Y., Courville, A., & Vincent, P. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.50 · Cited by 13,001
Hinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A.-R., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., et al. (2012) · IEEE Signal Processing Magazine · doi:10.1109/msp.2012.2205597 · Cited by 10,260
Self-citationSpeech recognition with deep recurrent neural networks
Graves, A., Mohamed, A.-R., & Hinton, G. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638947 · Cited by 8,893
Self-citationImproving neural networks by preventing co-adaptation of feature detectors
Hinton, G. E., Srivastava, N., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. R. (2012) · arXiv · doi:10.48550/arxiv.1207.0580 · Cited by 6,382
Self-citation(2013) · Cited by 3,740
Gaussian Error Linear Units (GELUs)
Hendrycks, D., & Gimpel, K. (2016) · arXiv · doi:10.48550/arxiv.1606.08415 · Cited by 3,537
A survey of deep neural network architectures and their applications
Liu, W., Wang, Z., Liu, X., Zeng, N., Liu, Y., & Alsaadi, F. E. (2017) · Neurocomputing · doi:10.1016/j.neucom.2016.12.038 · Cited by 3,239
Supervised Sequence Labelling with Recurrent Neural Networks
Graves, A. (2012) · Studies in Computational Intelligence · doi:10.1007/978-3-642-24797-2 · Cited by 3,120
A survey of the recent architectures of deep convolutional neural networks
Khan, A., Sohail, A., Zahoora, U., & Qureshi, A. S. (2020) · Artificial Intelligence Review · doi:10.1007/s10462-020-09825-6 · Cited by 3,137
Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition
Dahl, G. E., Yu, D., Deng, L., & Acero, A. (2012) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2011.2134090 · Cited by 3,011
Self-citationLong short-term memory recurrent neural network architectures for large scale acoustic modeling
Sak, H., Senior, A., & Beaufays, F. (2014) · Interspeech 2014 · doi:10.21437/interspeech.2014-80 · Cited by 2,973
Deep learning and its applications to machine health monitoring
Zhao, R., Yan, R., Chen, Z., Mao, K., Wang, P., & Gao, R. X. (2019) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2018.05.050 · Cited by 2,894
Deep learning applications and challenges in big data analytics
Najafabadi, M. M., Villanustre, F., Khoshgoftaar, T. M., Seliya, N., Wald, R., & Muharemagic, E. (2015) · Journal of Big Data · doi:10.1186/s40537-014-0007-7 · Cited by 2,594
Improving deep neural networks for LVCSR using rectified linear units and dropout
Dahl, G. E., Sainath, T. N., & Hinton, G. E. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6639346 · Cited by 1,310
Self-citationA survey on deep learning for big data
Zhang, Q., Yang, L. T., Chen, Z., & Li, P. (2018) · Information Fusion · doi:10.1016/j.inffus.2017.10.006 · Cited by 1,226
Hierarchical Feature Representation and Multimodal Fusion with Deep Learning for AD/MCI Diagnosis
Suk, H.-I., Lee, S.-W., & Shen, D. (2014) · NeuroImage · doi:10.1016/j.neuroimage.2014.06.077 · Cited by 968
“The rationale of using DBM for feature representation is as follows: (1) it can learn internal latent representations that capture non-linear complicatedpatterns and/or statistical structures in a hierarchicalmanner (Bengio, 2009; Bengio et al., 2007; Hinton et al., 2006; Mohamed et al., 2012).”
Speech Emotion Recognition Using Deep Learning Techniques: A Review
Khalil, R. A., Jones, E., Babar, M. I., Jan, T., Zafar, M. H., & Alhussain, T. (2019) · IEEE Access · doi:10.1109/access.2019.2936124 · Cited by 702
Regularizing Neural Networks by Penalizing Confident Output Distributions
Pereyra, G., Tucker, G., Chorowski, J., Kaiser, Ł., & Hinton, G. (2017) · arXiv · doi:10.48550/arxiv.1701.06548 · Cited by 655
Self-citationArtificial intelligence: a survey on evolution, models, applications and future trends
Lu, Y. (2019) · Journal of Management Analytics · doi:10.1080/23270012.2019.1570365 · Cited by 641
A Survey of Deep Learning Methods for Cyber Security
Berman, D. S., Buczak, A. L., Chavis, J. S., & Corbett, C. L. (2019) · Information · doi:10.3390/info10040122 · Cited by 569
Evaluating deep learning architectures for Speech Emotion Recognition
Fayek, H. M., Lech, M., & Cavedon, L. (2017) · Neural Networks · doi:10.1016/j.neunet.2017.02.013 · Cited by 554
Brain plasticity through the life span: Learning to learn and action video games
Bavelier, D., Green, C. S., Pouget, A., & Schrater, P. (2012) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-060909-152832 · Cited by 541
Deep Learning for Sensor-based Human Activity Recognition
Chen, K., Zhang, D., Yao, L., Guo, B., Yu, Z., & Liu, Y. (2022) · ACM Computing Surveys · doi:10.1145/3447744 · Cited by 690
“Also, deep generative models are more robust against overfitting problems as compared to discriminative models [93].”
BackgroundPredicting Parameters in Deep Learning
Shakibi, B. (2014) · University of British Columbia · doi:10.14288/1.0165555 · Cited by 462
Deep learning for robust feature generation in audiovisual emotion recognition
Kim, Y., Lee, H., & Provost, E. M. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638346 · Cited by 401
Big Data: Deep Learning for financial sentiment analysis
Sohangir, S., Wang, D., Pomeranets, A., & Khoshgoftaar, T. M. (2018) · Journal of Big Data · doi:10.1186/s40537-017-0111-6 · Cited by 390
Lebovitz, S., Levina, N., & Lifshitz-Assaf, H. (2021) · MIS Quarterly · doi:10.25300/misq/2021/16564 · Cited by 49
Theoretical Insights Into the Optimization Landscape of Over-Parameterized Shallow Neural Networks
Soltanolkotabi, M., Javanmard, A., & Lee, J. D. (2019) · IEEE Transactions on Information Theory · doi:10.1109/tit.2018.2854560 · Cited by 356
Multi-task Self-Supervised Learning for Human Activity Detection
Saeed, A., Ozcelebi, T., & Lukkien, J. (2019) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3328932 · Cited by 346
Deep Learning for Sensor-based Human Activity Recognition: Overview, Challenges and Opportunities
Chen, K., Zhang, D., Yao, L., Guo, B., Yu, Z., & Liu, Y. (2020) · arXiv · doi:10.48550/arxiv.2001.07416 · Cited by 314
“Also, deep generative models are more robust against overfitting problems as compared to discriminative models [93].”
BackgroundAnalysis of CNN-based speech recognition system using raw speech as input
Palaz, D., Magimai-Doss, M., & Collobert, R. (2015) · Interspeech 2015 · doi:10.21437/interspeech.2015-3 · Cited by 288
Wheel Defect Detection With Machine Learning
Krummenacher, G., Ong, C. S., Koller, S., Kobayashi, S., & Buhmann, J. M. (2018) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2017.2720721 · Cited by 258
Ensemble deep learning for speech recognition
Deng, L., & Platt, J. C. (2014) · Interspeech 2014 · doi:10.21437/interspeech.2014-433 · Cited by 248
Multimodal emotion recognition using deep learning architectures
Ranganathan, H., Chakraborty, S., & Panchanathan, S. (2016) · 2016 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv.2016.7477679 · Cited by 249
Ahmad, S., Shakeel, I., Mehfuz, S., & Ahmad, J. (2023) · Computer Science Review · doi:10.1016/j.cosrev.2023.100568 · Cited by 197
Palaz, D., Magimai-Doss, M., & Collobert, R. (2019) · Speech Communication · doi:10.1016/j.specom.2019.01.004 · Cited by 168
Deep learning methods for underwater target feature extraction and recognition
Hu, G., Wang, K., Peng, Y., Qiu, M., Shi, J., & Liu, L. (2018) · Computational Intelligence and Neuroscience · doi:10.1155/2018/1214301 · Cited by 151
The Rhetoric and Reality of Anthropomorphism in Artificial Intelligence
Watson, D. (2019) · Minds and Machines · doi:10.1007/s11023-019-09506-6 · Cited by 148
Hybrid state of charge estimation for lithium-ion battery under dynamic operating conditions
Liu, D., Li, L., Song, Y., Wu, L., & Peng, Y. (2019) · International Journal of Electrical Power & Energy Systems · doi:10.1016/j.ijepes.2019.02.046 · Cited by 126
Deep Learning for Detecting Robotic Grasps
Lenz, I., Lee, H., & Saxena, A. (2013) · Robotics: Science and Systems IX · doi:10.15607/rss.2013.ix.012 · Cited by 126
Deep Activity Recognition Models with Triaxial Accelerometers
Alsheikh, M. A., Selim, A., Niyato, D., Doyle, L., Lin, S., & Tan, H.-P. (2015) · arXiv · doi:10.48550/arxiv.1511.04664 · Cited by 126
LSTM time and frequency recurrence for automatic speech recognition
Li, J., Mohamed, A., Zweig, G., & Gong, Y. (2015) · 2015 IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru.2015.7404793 · Cited by 119
Self-citationTowards real-time Speech Emotion Recognition using deep neural networks
Fayek, H. M., Lech, M., & Cavedon, L. (2015) · 2015 9th International Conference on Signal Processing and Communication Systems (ICSPCS) · doi:10.1109/icspcs.2015.7391796 · Cited by 103
Deep learning-based recognition of underwater target
Cao, X., Zhang, X., Yu, Y., & Niu, L. (2016) · 2016 IEEE International Conference on Digital Signal Processing (DSP) · doi:10.1109/icdsp.2016.7868522 · Cited by 78
Siamese-GAN: Learning Invariant Representations for Aerial Vehicle Image Categorization
Bashmal, L., Bazi, Y., AlHichri, H., AlRahhal, M., Ammour, N., & Alajlan, N. (2018) · Remote Sensing · doi:10.3390/rs10020351 · Cited by 60
Automatic Assessment of Speech Intelligibility for Individuals with Aphasia
Le, D., Licata, K., Persad, C., & Provost, E. M. (2016) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2016.2598428 · Cited by 59
Qu, R., Xu, G., Ding, C., Jia, W., & Sun, M. (2020) · IEEE Access · doi:10.1109/access.2020.2991845 · Cited by 59
Cloud-based scalable object detection and classification in video streams
Yaseen, M. U., Anjum, A., Rana, O., & Hill, R. (2018) · Future Generation Computer Systems · doi:10.1016/j.future.2017.02.003 · Cited by 55
Yildiz, I. B., von Kriegstein, K., & Kiebel, S. J. (2013) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003219 · Cited by 55
Emotion recognition from thermal infrared images using deep Boltzmann machine
Wang, S., He, M., Gao, Z., He, S., & Ji, Q. (2014) · Frontiers of Computer Science · doi:10.1007/s11704-014-3295-3 · Cited by 51
Shahin, M., Ahmed, B., McKechnie, J., Ballard, K., & Gutierrez-Osuna, R. (2014) · Interspeech 2014 · doi:10.21437/interspeech.2014-377 · Cited by 34
Kazmi, S. M. K. A., Aafaq, N., Khan, M. A., Khalil, M., & Saleem, A. (2023) · IEEE Access · doi:10.1109/access.2023.3299878 · Cited by 13
Cyberbullying Detection using Recursive Neural Network through Offline Repository
Chandra, N., Khatri, S. K., & Som, S. (2018) · 2018 7th International Conference on Reliability, Infocom Technologies and Optimization (Trends and Future Directions) (ICRITO) · doi:10.1109/icrito.2018.8748570 · Cited by 9
Metadata
- Authors
- Mohamed, A.-R., Dahl, G. E., & Hinton, G.
- Venue
- IEEE Transactions on Audio, Speech, and Language Processing
- Published
- 2012-01
- Volume
- 20
- Issue
- 1
- Pages
- 14-22
- ISSN
- 1558-7916 (ISSN-L)
- 1558-7924
- OpenAlex
- W1993882792
- Cited by
- 1,727
- References
- 4
- FWCI
- 104.96
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
