The DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
Deep belief networks
Open accessAbstract
Recently, Hinton et al.[6] derived a way to perform fast, greedy learning of deep belief networks (DBN) one layer at a time, with the top two layers forming an undirected bipartite graph (associate memory). The learning procedure consists of training a stack of Restricted Boltzmann Machines (RBM’s) each having only one layer of latent (hidden) feature detectors. The learned feature activations of one RBM are used as the “data” for training the next RBM in the stack. The important aspect of this layer-wise training procedure is that, provided the number of features per layer does not decrease, [6] showed that each extra layer increases a variational lower bound on the log probability of data. So layer-by-layer training can be repeated several times 1 to learn a deep, hierarchical model in which each layer of features captures strong high-order correlations between the activities of features in the layer below. We will discuss three ideas based on greedily learning a hierarchy of features: Nonlinear Dimensionality Reduction. The DBN framework allows us to make nonlinear autoencoders work considerably better [7] than widely used methods such as PCA, SVD, and LLE. The standard way to train autoencoders is to use backpropagation to reduce the reconstruction error. It is difficult, however, to optimize the weights in non-linear autoencoders that have multiple hidden layers with many million parameters [3, 5]. We use our greedy learning algorithm to pretrain autoencoders. This pretraining stage discovers useful features efficiently. After the pretraining stage, the model is “unfolded” to produce encoder and decoder networks that initially use the same weights. The global fine-tuning stage then uses backpropagation through the whole autoencoder to fine-tune the weights for optimal reconstruction. The key idea is that the greedy learning algorithm
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336
Self-citationLocality-sensitive hashing scheme based on p-stable distributions
Datar, M., Immorlica, N., Indyk, P., & Mirrokni, V. S. (2004) · Proceedings of the twentieth annual symposium on Computational geometry · doi:10.1145/997817.997857 · Cited by 2,936
Nonlinear Dimensionality Reduction
(2007) · Information Science and Statistics · doi:10.1007/978-0-387-39351-3 · Cited by 1,517
Training Invariant Support Vector Machines
Decoste, D., & Schölkopf, B. (2002) · Machine Learning · doi:10.1023/a:1012454411458 · Cited by 588
Learning a Nonlinear Embedding by Preserving Class Neighbourhood Structure
Salakhutdinov, R., & Hinton, G. E. (2007) · International Conference on Artificial Intelligence and Statistics · Cited by 440
Self-citationReplicator Neural Networks for Universal Optimal Source Coding
Hecht-Nielsen, R. (1995) · Science · doi:10.1126/science.269.5232.1860 · Cited by 134
Cited by
Sarker, I. H. (2021) · SN Computer Science · doi:10.1007/s42979-021-00815-1 · Cited by 2,678
EEG Emotion Recognition Using Dynamical Graph Convolutional Neural Networks
Song, T., Zheng, W., Song, P., & Cui, Z. (2020) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2018.2817622 · Cited by 1,597
A Survey on Deep Learning: Algorithms, Techniques, and Applications
Pouyanfar, S., Sadiq, S., Yan, Y., Tian, H., Tao, Y., Reyes, M. P., Shyu, M.-L., Chen, S.-C., & Iyengar, S. S. (2019) · ACM Computing Surveys · doi:10.1145/3234150 · Cited by 1,403
Machine Learning and Deep Learning Methods for Cybersecurity
Xin, Y., Kong, L., Liu, Z., Chen, Y., Li, Y., Zhu, H., Gao, M., Hou, H., & Wang, C. (2018) · IEEE Access · doi:10.1109/access.2018.2836950 · Cited by 1,228
Sarker, I. H. (2022) · SN Computer Science · doi:10.1007/s42979-022-01043-x · Cited by 1,208
Deep learning for cyber security intrusion detection: Approaches, datasets, and comparative study
Ferrag, M. A., Maglaras, L., Moschoyiannis, S., & Janicke, H. (2020) · Journal of Information Security and Applications · doi:10.1016/j.jisa.2019.102419 · Cited by 1,130
Deep learning modelling techniques: Current progress, applications, advantages, and challenges
Ahmed, S. F., Alam, M. S. B., Hassan, M., Rozbu, M. R., Ishtiak, T., Rafa, N., Mofijur, M., Shawkat Ali, A. B. M., & Gandomi, A. H. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10466-8 · Cited by 1,079
Taye, M. M. (2023) · Computers · doi:10.3390/computers12050091 · Cited by 1,057
A survey on deep learning based bearing fault diagnosis
Hoang, D.-T., & Kang, H.-J. (2019) · Neurocomputing · doi:10.1016/j.neucom.2018.06.078 · Cited by 866
Feature dimensionality reduction: a review
Jia, W., Sun, M., Lian, J., & Hou, S. (2022) · Complex & Intelligent Systems · doi:10.1007/s40747-021-00637-x · Cited by 797
Deep learning based ensemble approach for probabilistic wind power forecasting
Wang, H.-Z., Li, G.-Q., Wang, G.-B., Peng, J.-C., Jiang, H., & Liu, Y.-T. (2017) · Applied Energy · doi:10.1016/j.apenergy.2016.11.111 · Cited by 753
MolTrans: Molecular Interaction Transformer for drug–target interaction prediction
Huang, K., Xiao, C., Glass, L. M., & Sun, J. (2021) · Bioinformatics · doi:10.1093/bioinformatics/btaa880 · Cited by 653
SySeVR: A Framework for Using Deep Learning to Detect Software Vulnerabilities
Li, Z., Zou, D., Xu, S., Jin, H., Zhu, Y., & Chen, Z. (2022) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2021.3051525 · Cited by 639
Deep Learning and Its Applications in Biomedicine
Cao, C., Liu, F., Tan, H., Song, D., Shu, W., Li, W., Zhou, Y., Bo, X., & Xie, Z. (2018) · Genomics, Proteomics & Bioinformatics · doi:10.1016/j.gpb.2017.07.003 · Cited by 624
A comprehensive review of deep learning applications in hydrology and water resources
Sit, M., Demiray, B. Z., Xiang, Z., Ewing, G. J., Sermet, Y., & Demir, I. (2020) · Water Science and Technology · doi:10.2166/wst.2020.369 · Cited by 618
Darko, A., Chan, A. P. C., Adabre, M. A., Edwards, D. J., Hosseini, M. R., & Ameyaw, E. E. (2020) · Automation in Construction · doi:10.1016/j.autcon.2020.103081 · Cited by 603
Survey on semantic segmentation using deep learning techniques
Lateef, F., & Ruichek, Y. (2019) · Neurocomputing · doi:10.1016/j.neucom.2019.02.003 · Cited by 559
Deep learning for image-based cancer detection and diagnosis − A survey
Hu, Z., Tang, J., Wang, Z., Zhang, K., Zhang, L., & Sun, Q. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2018.05.014 · Cited by 509
A review on deep learning for recommender systems: challenges and remedies
Batmaz, Z., Yurekli, A., Bilge, A., & Kaleli, C. (2019) · Artificial Intelligence Review · doi:10.1007/s10462-018-9654-y · Cited by 504
Breast cancer classification using deep belief networks
Abdel-Zaher, A. M., & Eldeib, A. M. (2016) · Expert Systems with Applications · doi:10.1016/j.eswa.2015.10.015 · Cited by 493
Thirty Years of Machine Learning: The Road to Pareto-Optimal Wireless Networks
Wang, J., Jiang, C., Zhang, H., Ren, Y., Chen, K.-C., & Hanzo, L. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2965856 · Cited by 486
A Review on a Deep Learning Perspective in Brain Cancer Classification
Tandel, G. S., Biswas, M., Kakde, O. G., Tiwari, A., Suri, H. S., Turk, M., Laird, J., Asare, C., Ankrah, A. A., Khanna, N. N., et al. (2019) · Cancers · doi:10.3390/cancers11010111 · Cited by 466
Cancer Diagnosis Using Deep Learning: A Bibliographic Review
Munir, K., Elahi, H., Ayub, A., Frezza, F., & Rizzi, A. (2019) · Cancers · doi:10.3390/cancers11091235 · Cited by 422
Simple convolutional neural network on image classification
Guo, T., Dong, J., Li, H., & Gao, Y. (2017) · 2017 IEEE 2nd International Conference on Big Data Analysis (ICBDA)( · doi:10.1109/icbda.2017.8078730 · Cited by 408
Deep Learning for Medical Anomaly Detection – A Survey
Fernando, T., Gammulle, H., Denman, S., Sridharan, S., & Fookes, C. (2022) · ACM Computing Surveys · doi:10.1145/3464423 · Cited by 399
Thakkar, A., & Lohiya, R. (2021) · Archives of Computational Methods in Engineering · doi:10.1007/s11831-020-09496-0 · Cited by 366
Deep learning and big data technologies for IoT security
Amanullah, M. A., Habeeb, R. A. A., Nasaruddin, F. H., Gani, A., Ahmed, E., Nainar, A. S. M., Akim, N. M., & Imran, M. (2020) · Computer Communications · doi:10.1016/j.comcom.2020.01.016 · Cited by 358
Machine learning in aerodynamic shape optimization
Li, J., Du, X., & Martins, J. R. R. A. (2022) · Progress in Aerospace Sciences · doi:10.1016/j.paerosci.2022.100849 · Cited by 354
The present and future of deep learning in radiology
Saba, L., Biswas, M., Kuppili, V., Cuadrado Godia, E., Suri, H. S., Edla, D. R., Omerzu, T., Laird, J. R., Khanna, N. N., Mavrogeni, S., et al. (2019) · European Journal of Radiology · doi:10.1016/j.ejrad.2019.02.038 · Cited by 353
Tama, B. A., Vania, M., Lee, S., & Lim, S. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-022-10293-3 · Cited by 345
Deep Learning for Medical Image-Based Cancer Diagnosis
Jiang, X., Hu, Z., Wang, S., & Zhang, Y. (2023) · Cancers · doi:10.3390/cancers15143608 · Cited by 345
A deep learning approach for detecting traffic accidents from social media data
Zhang, Z., He, Q., Gao, J., & Ni, M. (2018) · Transportation Research Part C: Emerging Technologies · doi:10.1016/j.trc.2017.11.027 · Cited by 343
Machine Learning and Deep Learning in Energy Systems: A Review
Forootan, M. M., Larki, I., Zahedi, R., & Ahmadi, A. (2022) · Sustainability · doi:10.3390/su14084832 · Cited by 343
Flexible Piezoelectric Acoustic Sensors and Machine Learning for Speech Processing
Jung, Y. H., Hong, S. K., Wang, H. S., Han, J. H., Pham, T. X., Park, H., Kim, J., Kang, S., Yoo, C. D., & Lee, K. J. (2020) · Advanced Materials · doi:10.1002/adma.201904020 · Cited by 338
Hierarchical Convolutional Neural Networks for EEG-Based Emotion Recognition
Li, J., Zhang, Z., & He, H. (2018) · Cognitive Computation · doi:10.1007/s12559-017-9533-x · Cited by 337
Du, Y., Fu, Z., & Calhoun, V. D. (2018) · Frontiers in Neuroscience · doi:10.3389/fnins.2018.00525 · Cited by 332
Application of deep learning to cybersecurity: A survey
Mahdavifar, S., & Ghorbani, A. A. (2019) · Neurocomputing · doi:10.1016/j.neucom.2019.02.056 · Cited by 325
Pan, D., Zeng, A., Jia, L., Huang, Y., Frizzell, T., & Song, X. (2020) · Frontiers in Neuroscience · doi:10.3389/fnins.2020.00259 · Cited by 322
Auto-Encoders in Deep Learning—A Review with New Perspectives
Chen, S., & Guo, W. (2023) · Mathematics · doi:10.3390/math11081777 · Cited by 318
A data-driven lane-changing model based on deep learning
Xie, D.-F., Fang, Z.-Z., Jia, B., & He, Z. (2019) · Transportation Research Part C: Emerging Technologies · doi:10.1016/j.trc.2019.07.002 · Cited by 314
Performance Comparison and Current Challenges of Using Machine Learning Techniques in Cybersecurity
Shaukat, K., Luo, S., Varadharajan, V., Hameed, I., Chen, S., Liu, D., & Li, J. (2020) · Energies · doi:10.3390/en13102509 · Cited by 303
Miglani, A., & Kumar, N. (2019) · Vehicular Communications · doi:10.1016/j.vehcom.2019.100184 · Cited by 295
Deep Cybersecurity: A Comprehensive Overview from Neural Network and Deep Learning Perspective
Sarker, I. H. (2021) · doi:10.20944/preprints202102.0340.v1 · Cited by 295
Robertson, S., Azizpour, H., Smith, K., & Hartman, J. (2018) · Translational Research · doi:10.1016/j.trsl.2017.10.010 · Cited by 295
Prognostics and health management of Lithium-ion battery using deep learning methods: A review
Zhang, Y., & Li, Y.-F. (2022) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2022.112282 · Cited by 295
Deep Cybersecurity: A Comprehensive Overview from Neural Network and Deep Learning Perspective
Sarker, I. H. (2021) · SN Computer Science · doi:10.1007/s42979-021-00535-6 · Cited by 289
A survey on deep learning-based non-invasive brain signals: recent advances and new frontiers
Zhang, X., Yao, L., Wang, X., Monaghan, J., McAlpine, D., & Zhang, Y. (2021) · Journal of Neural Engineering · doi:10.1088/1741-2552/abc902 · Cited by 296
Deep Learning for fault detection in wind turbines
Helbing, G., & Ritter, M. (2018) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2018.09.012 · Cited by 278
A comprehensive survey on deep learning based malware detection techniques
M., G., & Sethuraman, S. C. (2023) · Computer Science Review · doi:10.1016/j.cosrev.2022.100529 · Cited by 277
Charte, D., Charte, F., García, S., del Jesus, M. J., & Herrera, F. (2018) · Information Fusion · doi:10.1016/j.inffus.2017.12.007 · Cited by 273
Deep Learning for Diabetes: A Systematic Review
Zhu, T., Li, K., Herrero, P., & Georgiou, P. (2021) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2020.3040225 · Cited by 264
A novel deep learning method for detection and classification of plant diseases
Albattah, W., Nawaz, M., Javed, A., Masood, M., & Albahli, S. (2022) · Complex & Intelligent Systems · doi:10.1007/s40747-021-00536-1 · Cited by 263
Ahmed, A. A., Sayed, S., Abdoulhalik, A., Moutari, S., & Oyedele, L. (2024) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2024.140715 · Cited by 260
A Systematic Review of Deep Learning Approaches to Educational Data Mining
Hernández-Blanco, A., Herrera-Flores, B., Tomás, D., & Navarro-Colorado, B. (2019) · Complexity · doi:10.1155/2019/1306039 · Cited by 230
eToxPred: a machine learning-based approach to estimate the toxicity of drug candidates
Pu, L., Naderi, M., Liu, T., Wu, H.-C., Mukhopadhyay, S., & Brylinski, M. (2019) · BMC Pharmacology and Toxicology · doi:10.1186/s40360-018-0282-6 · Cited by 212
“1C) [39].”
Computational Health Informatics in the Big Data Age: A Survey
Fang, R., Pouyanfar, S., Yang, Y., Chen, S.-C., & Iyengar, S. S. (2017) · ACM Computing Surveys · doi:10.1145/2932707 · Cited by 210
Innovative Materials Science via Machine Learning
Gao, C., Min, X., Fang, M., Tao, T., Zheng, X., Liu, Y., Wu, X., & Huang, Z. (2022) · Advanced Functional Materials · doi:10.1002/adfm.202108044 · Cited by 198
Deep learning-enabled anomaly detection for IoT systems
Abusitta, A., de Carvalho, G. H. S., Wahab, O. A., Halabi, T., Fung, B. C. M., & Mamoori, S. A. (2023) · Internet of Things · doi:10.1016/j.iot.2022.100656 · Cited by 235
Facial Sentiment Analysis Using AI Techniques: State-of-the-Art, Taxonomies, and Challenges
Patel, K., Mehta, D., Mistry, C., Gupta, R., Tanwar, S., Kumar, N., & Alazab, M. (2020) · IEEE Access · doi:10.1109/access.2020.2993803 · Cited by 157
Context-Aware Generative Adversarial Privacy
Huang, C., Kairouz, P., Chen, X., Sankar, L., & Rajagopal, R. (2017) · Entropy · doi:10.3390/e19120656 · Cited by 156
Learning for Personalized Medicine: A Comprehensive Review From a Deep Learning Perspective
Zhang, S., Bamakan, S. M. H., Qu, Q., & Li, S. (2019) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2018.2864254 · Cited by 125
A survey of multimodal deep generative models
Suzuki, M., & Matsuo, Y. (2022) · Advanced Robotics · doi:10.1080/01691864.2022.2035253 · Cited by 109
“Xing et al. [116] build a joint model of images and texts using a linear RBM. Srivastava et al. [82] propose a multimodal generative model based on deep belief networks [28, 29], which has since been used in various multimodal applications [34, 41].”
A Review on Deep-Learning-Based Cyberbullying Detection
Hasan, M. T., Hossain, M. A. E., Mukta, M. S. H., Akter, A., Ahmed, M., & Islam, S. (2023) · Future Internet · doi:10.3390/fi15050179 · Cited by 103
Survey on Explainable AI: Techniques, challenges and open issues
Abusitta, A., Li, M. Q., & Fung, B. C. M. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.124710 · Cited by 98
Core-Brain-Network-Based Multilayer Convolutional Neural Network for Emotion Recognition
Gao, Z., Li, R., Ma, C., Rui, L., & Sun, X. (2021) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2021.3090164 · Cited by 61
Implementation of EEG Emotion Recognition System Based on Hierarchical Convolutional Neural Networks
Li, J., Zhang, Z., & He, H. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-49685-6_3 · Cited by 28
Metadata
- Authors
- Hinton, G.
- Venue
- Scholarpedia
- Publisher
- Scholarpedia
- Published
- 2009
- Volume
- 4
- Issue
- 5
- Pages
- 5947
- ISSN
- 1941-6016 (ISSN-L)
- OpenAlex
- W1973445088
- Cited by
- 885
- References
- 6
- FWCI
- 1.45
- Type
- journal article
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.