The DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.
Application of Deep Belief Networks for Natural Language Understanding
Abstract
Applications of Deep Belief Nets (DBN) to various problems have been the subject of a number of recent studies ranging from image classification and speech recognition to audio classification. In this study we apply DBNs to a natural language understanding problem. The recent surge of activity in this area was largely spurred by the development of a greedy layer-wise pretraining method that uses an efficient learning algorithm called Contrastive Divergence (CD). CD allows DBNs to learn a multi-layer generative model from unlabeled data and the features discovered by this model are then used to initialize a feed-forward neural network which is fine-tuned with backpropagation. We compare a DBN-initialized neural network to three widely used text classification algorithms: Support Vector Machines (SVM), boosting and Maximum Entropy (MaxEnt). The plain DBN-based model gives a call-routing classification accuracy that is equal to the best of the other models. However, using additional unlabeled data for DBN pre-training and combining DBN-based learned features with the original features provides significant gains over SVMs, which, in turn, performed better than both MaxEnt and Boosting.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Nature of Statistical Learning Theory
Vapnik, V. N. (1995) · doi:10.1007/978-1-4757-2440-0 · Cited by 39,138
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336
Self-citationTraining Products of Experts by Minimizing Contrastive Divergence
Hinton, G. E. (2002) · Neural Computation · doi:10.1162/089976602760128018 · Cited by 4,879 · cited 1×
“A restricted Boltzmann machine [16] is a two-layer, undirected, bipartite graphical model where the first layer consists of observed data variables (or visible units), and the second layer consists of latent variables (or hidden units).”
Self-citation
Cited by
A survey on deep learning for big data
Zhang, Q., Yang, L. T., Chen, Z., & Li, P. (2018) · Information Fusion · doi:10.1016/j.inffus.2017.10.006 · Cited by 1,226
Fault diagnosis for rotating machinery using multiple sensors and convolutional neural networks
Xia, M., Li, T., Xu, L., Liu, L., & de Silva, C. W. (2018) · IEEE/ASME Transactions on Mechatronics · doi:10.1109/tmech.2017.2728371 · Cited by 723
A Survey of Deep Learning Methods for Cyber Security
Berman, D. S., Buczak, A. L., Chavis, J. S., & Corbett, C. L. (2019) · Information · doi:10.3390/info10040122 · Cited by 569
Sagheer, A., & Kotb, M. (2019) · Scientific Reports · doi:10.1038/s41598-019-55320-6 · Cited by 351
Text classification based on deep belief network and softmax regression
Jiang, M., Liang, Y., Feng, X., Fan, X., Pei, Z., Xue, Y., & Guan, R. (2018) · Neural Computing and Applications · doi:10.1007/s00521-016-2401-x · Cited by 268
A survey of swarm and evolutionary computing approaches for deep learning
Darwish, A., Hassanien, A. E., & Das, S. (2020) · Artificial Intelligence Review · doi:10.1007/s10462-019-09719-2 · Cited by 200
End-to-end joint learning of natural language understanding and dialogue manager
Yang, X., Chen, Y.-N., Hakkani-Tur, D., Crook, P., Li, X., Gao, J., & Deng, L. (2017) · 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2017.7953246 · Cited by 80
Arabic Cyberbullying Detection: Using Deep Learning
Haidar, B., Chamoun, M., & Serhrouchni, A. (2018) · 2018 7th International Conference on Computer and Communication Engineering (ICCCE) · doi:10.1109/iccce.2018.8539303 · Cited by 58
Deep learning framework for detection of hypoglycemic episodes in children with type 1 diabetes
San, P. P., Ling, S. H., & Nguyen, H. T. (2016) · 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2016.7591483 · Cited by 42
An Instruction Set Architecture for Machine Learning
Chen, Y., Lan, H., Du, Z., Liu, S., Tao, J., Han, D., Luo, T., Guo, Q., Li, L., Xie, Y., et al. (2018) · ACM Transactions on Computer Systems · doi:10.1145/3331469 · Cited by 14
Deep Learning for Enterprise Systems Implementation Lifecycle Challenges: Research Directions
Hassanien, H. E.-D., & Elragal, A. (2021) · Informatics · doi:10.3390/informatics8010011 · Cited by 5
Metadata
- Authors
- Sarikaya, R., Hinton, G. E., & Deoras, A.
- Venue
- IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Published
- 2014-04
- Volume
- 22
- Issue
- 4
- Pages
- 778-784
- ISSN
- 2329-9290 (ISSN-L)
- 2329-9304
- OpenAlex
- W1964812476
- Cited by
- 460
- References
- 3
- FWCI
- 36.97
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
