The DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.
Binary coding of speech spectrograms using a deep auto-encoder
Abstract
This paper reports our recent exploration of the layer-by-layer learning strategy for training a multi-layer generative model of patches of speech spectrograms. The top layer of the generative model learns binary codes that can be used for efficient compression of speech and could also be used for scalable speech recognition or rapid speech content retrieval. Each layer of the generative model is fully connected to the layer below and the weights on these connections are pretrained efficiently by using the contrastive divergence approximation to the log likelihood gradient. After layer-bylayer pre-training we “unroll” the generative model to form a deep auto-encoder, whose parameters are then fine-tuned using back-propagation. To reconstruct the full-length speech spectrogram, individual spectrogram segments predicted by their respective binary codes are combined using an overlapand-add method. Experimental results on speech spectrogram coding demonstrate that the binary codes produce a logspectral distortion that is approximately 2 dB lower than a subband vector quantization technique over the entire frequency range of wide-band speech. Index Terms: deep learning, speech feature extraction, neural networks, auto-encoder, binary codes, Boltzmann machine
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
Self-citationA fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336
Self-citationSalakhutdinov, R., & Hinton, G. (2009) · International Journal of Approximate Reasoning · doi:10.1016/j.ijar.2008.11.006 · Cited by 1,246
Self-citationDevelopments and directions in speech recognition and understanding, Part 1 [DSP Education]
Baker, J., Deng, L., Glass, J., Khudanpur, S., Lee, C.-H., Morgan, N., & O'Shaughnessy, D. (2009) · IEEE Signal Processing Magazine · doi:10.1109/msp.2009.932166 · Cited by 228
Self-citationInvestigation of full-sequence training of deep belief networks for speech recognition
Mohamed, A.-R., Yu, D., & Deng, L. (2010) · Interspeech 2010 · doi:10.21437/interspeech.2010-304 · Cited by 210
Self-citationLi, D., Yu, D., & Acero, A. (2006) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2006.878265 · Cited by 84
Self-citationComputational Models for Speech Production
Deng, L. (1999) · Springer eBooks · doi:10.1007/978-3-642-60087-6_20 · Cited by 51
Self-citationA Novel Framework and Training Algorithm for Variable-Parameter Hidden Markov Models
Yu, D., Deng, L., Gong, Y., & Acero, A. (2009) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2009.2020890 · Cited by 41
Self-citationLanguage recognition using deep-structured conditional random fields
Yu, D., Wang, S., Karam, Z. N., & Deng, L. (2010) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2010.5495072 · Cited by 38
Self-citationDeep-structured hidden conditional random fields for phonetic recognition
Yu, D., & Deng, L. (2010) · Interspeech (USB) · doi:10.21437/interspeech.2010-35 · Cited by 36
Self-citation
Cited by
Representation Learning: A Review and New Perspectives
Bengio, Y., Courville, A., & Vincent, P. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.50 · Cited by 13,001
Object Detection With Deep Learning: A Review
Zhao, Z.-Q., Zheng, P., Xu, S.-T., & Wu, X. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2876865 · Cited by 5,545
Convolutional Neural Networks for Speech Recognition
Abdel-Hamid, O., Mohamed, A.-R., Jiang, H., Deng, L., Penn, G., & Yu, D. (2014) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2014.2339736 · Cited by 2,262
Self-citationDeep Learning: Methods and Applications
Deng, L. (2014) · now publishers, Inc. eBooks · doi:10.1561/9781601988157 · Cited by 1,759
Self-citationDeep learning for smart manufacturing: Methods and applications
Wang, J., Ma, Y., Zhang, L., Gao, R. X., & Wu, D. (2018) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2018.01.003 · Cited by 1,740
A Regression Approach to Speech Enhancement Based on Deep Neural Networks
Xu, Y., Du, J., Dai, L.-R., & Lee, C.-H. (2015) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2014.2364452 · Cited by 1,366
Deng, L., Hinton, G., & Kingsbury, B. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6639344 · Cited by 1,178
Self-citationSpeech Recognition Using Deep Neural Networks: A Systematic Review
Nassif, A. B., Shahin, I., Attili, I., Azzeh, M., & Shaalan, K. (2019) · IEEE Access · doi:10.1109/access.2019.2896880 · Cited by 1,152
An Experimental Study on Speech Enhancement Based on Deep Neural Networks
Xu, Y., Du, J., Dai, L.-R., & Lee, C.-H. (2014) · IEEE Signal Processing Letters · doi:10.1109/lsp.2013.2291240 · Cited by 924
Speech enhancement based on deep denoising autoencoder
Lu, X., Tsao, Y., Matsuda, S., & Hori, C. (2013) · Interspeech 2013 · doi:10.21437/interspeech.2013-130 · Cited by 834
Recent advances in deep learning for speech research at Microsoft
Deng, L., Li, J., Huang, J.-T., Yao, K., Yu, D., Seide, F., Seltzer, M., Zweig, G., He, X., Williams, J., et al. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6639345 · Cited by 801
Self-citationA tutorial survey of architectures, algorithms, and applications for deep learning
Deng, L. (2014) · APSIPA Transactions on Signal and Information Processing · doi:10.1017/atsip.2013.9 · Cited by 725
Self-citationAtzori, M., Cognolato, M., & Müller, H. (2016) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2016.00009 · Cited by 684
“Since 2010, the application of deep learning techniques to speech recognition has allowed a quick and impressive reduction of error rate (Dahl et al., 2010; Deng et al., 2010b, 2013; Hinton et al., 2012; Goodfellow et al., 2016).”
BackgroundArtificial intelligence: a survey on evolution, models, applications and future trends
Lu, Y. (2019) · Journal of Management Analytics · doi:10.1080/23270012.2019.1570365 · Cited by 641
A Brief Review of Domain Adaptation
Farahani, A., Voghoei, S., Rasheed, K., & Arabnia, H. R. (2021) · Transactions on Computational Science and Computational Intelligence · doi:10.1007/978-3-030-71704-9_65 · Cited by 615
Mathematics for Machine Learning
Deisenroth, M. P., Faisal, A. A., & Ong, C. S. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108679930 · Cited by 573
Deep Learning of Representations: Looking Forward
Bengio, Y. (2013) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-39593-2_1 · Cited by 513
Multimodal intelligence: Representation learning, information fusion, and applications
Zhang, C., Yang, Z., He, X., & Deng, L. (2020) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2020.2987728 · Cited by 511
Self-citationMachine Learning Paradigms for Speech Recognition: An Overview
Deng, L., & Li, X. (2013) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.1109/tasl.2013.2244083 · Cited by 456
Self-citationDeep Learning and Its Applications to Signal and Information Processing [Exploratory DSP
Yu, D., & Deng, L. (2011) · IEEE Signal Processing Magazine · doi:10.1109/msp.2010.939038 · Cited by 455
Self-citationA review of deep learning with special emphasis on architectures, applications and recent trends
Sengupta, S., Basak, S., Saikia, P., Paul, S., Tsalavoutis, V., Atiah, F., Ravi, V., & Peters, A. (2020) · Knowledge-Based Systems · doi:10.1016/j.knosys.2020.105596 · Cited by 454
Artificial intelligence and internet of things in small and medium-sized enterprises: A survey
Hansen, E. B., & Bøgh, S. (2021) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2020.08.009 · Cited by 407
Noise Reduction in ECG Signals Using Fully Convolutional Denoising Autoencoders
Chiang, H.-T., Hsieh, Y.-Y., Fu, S.-W., Hung, K.-H., Tsao, Y., & Chien, S.-Y. (2019) · IEEE Access · doi:10.1109/access.2019.2912036 · Cited by 363
Zhu, Z., Peng, G., Chen, Y., & Gao, H. (2019) · Neurocomputing · doi:10.1016/j.neucom.2018.09.050 · Cited by 357
Rough Deep Neural Architecture for Short-Term Wind Speed Forecasting
Khodayar, M., Kaynak, O., & Khodayar, M. E. (2017) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2017.2730846 · Cited by 348
Deep Generative Stochastic Networks Trainable by Backprop
Bengio, Y., Thibodeau-Laufer, É., Alain, G., & Yosinski, J. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1306.1091 · Cited by 290
Fundamentals of speech recognition
Li, J., Deng, L., Haeb-Umbach, R., & Gong, Y. (2016) · Robust Automatic Speech Recognition · doi:10.1016/b978-0-12-802398-3.00002-7 · Cited by 266
Self-citationTemporal Multimodal Learning in Audiovisual Speech Recognition
Hu, D., Li, X., & Lu, X. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.389 · Cited by 108
Densely connected convolutional networks for vibration based structural damage identification
Wang, R., Li, J., Chencho, An, S., Hao, H., Liu, W., & Li, L. (2021) · Engineering Structures · doi:10.1016/j.engstruct.2021.112871 · Cited by 62
Metadata
- Authors
- Deng, L., Seltzer, M. L., Yu, D., Acero, A., Mohamed, A.-R., & Hinton, G.
- Venue
- Interspeech 2010
- Publisher
- ISCA
- Published
- 2010-09-26
- Pages
- 1692-1695
- ISSN
- 2308-457X (ISSN-L)
- OpenAlex
- W2168013545
- Cited by
- 375
- References
- 12
- FWCI
- 10.40
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.