Deep Generative Stochastic Networks Trainable by Backprop
Open accessThe DOI matches its OpenAlex record; 36 of the 36 references it cites were found in a registry or an index.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336
(2012) · Cited by 9,062
(2010) · Cited by 8,261
Extracting and composing robust features with denoising autoencoders
Vincent, P., Larochelle, H., Bengio, Y., & Manzagol, P.-A. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390294 · Cited by 7,436
Self-citationImproving neural networks by preventing co-adaptation of feature detectors
Hinton, G. E., Srivastava, N., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. R. (2012) · arXiv · doi:10.48550/arxiv.1207.0580 · Cited by 6,382
A unified architecture for natural language processing: deep neural networks with multitask learning
Collobert, R., & Weston, J. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390177 · Cited by 5,058
Learning Deep Architectures for AI
Bengio, Y. (2009) · now publishers, Inc. eBooks · doi:10.1561/9781601982957 · Cited by 4,941
Self-citationGreedy layer-wise training of deep networks
Bengio, Y., Lamblin, P., Popovici, D., & Larochelle, H. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0024 · Cited by 4,841 · cited 1×
“…for reviews) grew from breakthroughs in unsupervised learning of representations, based mostly on the Restricted Boltzmann Machine (RBM) (Hinton et al., 2006), auto-encoder variants (Bengio et al., 2007; Vin-cent et al., 2008), and sparse coding variants (Lee et al., 2007; Ranzato et al., 2007).”
BackgroundSelf-citationNoise-contrastive estimation: A new estimation principle for unnormalized statistical models
Gutmann, M. U., & Hyvärinen, A. (2010) · Edinburgh Research Explorer (University of Edinburgh) · Cited by 3,010
Cited by
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067
GAN(Generative Adversarial Nets)
(2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,270
Generative adversarial networks
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., & Bengio, Y. (2020) · Communications of the ACM · doi:10.1145/3422622 · Cited by 18,384
Self-citationConditional Generative Adversarial Nets
Mirza, M., & Osindero, S. (2014) · arXiv · doi:10.48550/arxiv.1411.1784 · Cited by 8,863
Deep Convolutional Neural Networks for Image Classification: A Comprehensive Review
Rawat, W., & Wang, Z. (2017) · Neural Computation · doi:10.1162/neco_a_00990 · Cited by 3,503
Deep neural networks are easily fooled: High confidence predictions for unrecognizable images
Nguyen, A., Yosinski, J., & Clune, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298640 · Cited by 3,258
Self-citationDeep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images
Nguyen, A., Yosinski, J., & Clune, J. (2014) · arXiv · doi:10.48550/arxiv.1412.1897 · Cited by 3,258
Self-citationDeep Unsupervised Learning using Nonequilibrium Thermodynamics
Sohl-Dickstein, J., Weiss, E. A., Maheswaranathan, N., & Ganguli, S. (2015) · arXiv · doi:10.48550/arxiv.1503.03585 · Cited by 2,637
“Multiplying distributions is costly and difficult for many techniques, including variational autoencoders, GSNs, NADEs, and most graphical models.”
MethodInfluentialUnsupervised Visual Representation Learning by Context Prediction
Doersch, C., Gupta, A., & Efros, A. A. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.167 · Cited by 2,629
“For example, denoising autoencoders [46, 3] use reconstruction from noisy data as a pretext task; to tell the difference between noise and signal, the algorithm must connect images to other images with similar objects, in order to remove the parts of the image that don’t fit the shared appearance…”
MethodAn Introduction to Variational Autoencoders
Kingma, D. P., & Welling, M. (2019) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000056 · Cited by 2,609
Denoising Diffusion Implicit Models
Song, J., Meng, C., & Ermon, S. (2020) · arXiv · doi:10.48550/arxiv.2010.02502 · Cited by 2,507
“Recent works on iterative generative models (Bengio et al., 2014), such as denoising diffusion probabilistic models (DDPM, Ho et al. (2020)) and noise conditional score networks (NCSN, Song & Ermon (2019)) have demonstrated the ability to produce samples comparable to that of GANs, without having…”
BackgroundMethodGenerative Adversarial Networks
(2022) · The Science of Deep Learning · doi:10.1017/9781108891530.013 · Cited by 2,402
Deep Learning: Methods and Applications
Deng, L. (2014) · now publishers, Inc. eBooks · doi:10.1561/9781601988157 · Cited by 1,759
Makhzani, A., Shlens, J., Jaitly, N., Goodfellow, I., & Frey, B. (2015) · arXiv · doi:10.48550/arxiv.1511.05644 · Cited by 1,579
Tutorial on Variational Autoencoders
Doersch, C. (2016) · arXiv · doi:10.48550/arxiv.1606.05908 · Cited by 1,470
Learning Implicit Fields for Generative Shape Modeling
Chen, Z., & Zhang, H. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00609 · Cited by 1,443
A Survey on Deep Learning: Algorithms, Techniques, and Applications
Pouyanfar, S., Sadiq, S., Yan, Y., Tian, H., Tao, Y., Reyes, M. P., Shyu, M.-L., Chen, S.-C., & Iyengar, S. S. (2019) · ACM Computing Surveys · doi:10.1145/3234150 · Cited by 1,403
A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications
Gui, J., Sun, Z., Wen, Y., Tao, D., & Ye, J. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3130191 · Cited by 1,231
“It produces data instances from the distribution without an explicit hypothesis [101] and utilizes the produced examples to modify the model.”
MethodNIPS 2016 Tutorial: Generative Adversarial Networks
Goodfellow, I. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1701.00160 · Cited by 1,229
Generative Visual Manipulation on the Natural Image Manifold
Zhu, J.-Y., Krähenbühl, P., Shechtman, E., & Efros, A. A. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46454-1_36 · Cited by 1,171
DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agents
Lee, N., Choi, W., Vernaza, P., Choy, C. B., Torr, P. H. S., & Chandraker, M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.233 · Cited by 1,067
A note on the evaluation of generative models
Theis, L., Oord, A. V. D., & Bethge, M. (2015) · arXiv · doi:10.48550/arxiv.1511.01844 · Cited by 1,026
“Following Bengio et al. (2013a), Parzen windows are in practice commonly fit to only 10,000 samples (e.g., Bengio et al., 2013b; Goodfellow et al., 2014; Li et al., 2015; Sohl-Dickstein et al., 2015).”
A survey of unsupervised deep domain adaptation
Wilson, G., & Cook, D. J. (2020) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3400066 · Cited by 987
Generative adversarial networks: introduction and outlook
Wang, K., Gou, C., Duan, Y., Lin, Y., Zheng, X., & Wang, F.-Y. (2017) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2017.7510583 · Cited by 806
Deep learning in optical metrology: a review
Zuo, C., Qian, J., Feng, S., Yin, W., Li, Y., Fan, P., Han, J., Qian, K., & Chen, Q. (2022) · Light: Science & Applications · doi:10.1038/s41377-022-00714-x · Cited by 711
Training Deep Neural Networks on Noisy Labels with Bootstrapping
Reed, S., Lee, H., Anguelov, D., Szegedy, C., Erhan, D., & Rabinovich, A. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.6596 · Cited by 667
Learning to generate chairs with convolutional neural networks
Dosovitskiy, A., Springenberg, J. T., & Brox, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298761 · Cited by 640
“This includes a wide variety of methods ranging from Gaussian mixture models [9, 31] to autoregressive models [19] and stochastic variations of neural networks [2, 11, 25, 17, 30].”
MethodAdversarially Learned Inference
Dumoulin, V., Belghazi, M. I., Poole, B., Mastropietro, O., Lamb, A., Arjovsky, M., & Courville, A. C. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.00704 · Cited by 579
Generative Moment Matching Networks
Li, Y., Swersky, K., & Zemel, R. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1502.02761 · Cited by 517
Imaging Time-Series to Improve Classification and Imputation
Wang, Z., & Oates, T. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.00327 · Cited by 470
A review of deep learning with special emphasis on architectures, applications and recent trends
Sengupta, S., Basak, S., Saikia, P., Paul, S., Tsalavoutis, V., Atiah, F., Ravi, V., & Peters, A. (2020) · Knowledge-Based Systems · doi:10.1016/j.knosys.2020.105596 · Cited by 454
MADE: Masked Autoencoder for Distribution Estimation
Germain, M., Gregor, K., Murray, I., & Larochelle, H. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1502.03509 · Cited by 448
A comprehensive survey and analysis of generative models in machine learning
GM, H., Gourisaria, M. K., Pandey, M., & Rautaray, S. S. (2020) · Computer Science Review · doi:10.1016/j.cosrev.2020.100285 · Cited by 432
Learning with Pseudo-Ensembles
Bachman, P., Alsharif, O., & Precup, D. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.4864 · Cited by 329
Generalized Denoising Auto-Encoders as Generative Models
Bengio, Y., Yao, L., Alain, G., & Vincent, P. (2013) · arXiv · doi:10.48550/arxiv.1305.6663 · Cited by 324
Self-citationUnsupervised and Semi-supervised Learning with Categorical Generative Adversarial Networks
Springenberg, J. T. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.06390 · Cited by 300
Wang, T., Huang, J., Zhang, H., & Sun, Q. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01077 · Cited by 291
Towards Biologically Plausible Deep Learning
Bengio, Y., Lee, D.-H., Bornschein, J., Mesnard, T., & Lin, Z. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1502.04156 · Cited by 276
Self-citationOpportunities and challenges for quantum-assisted machine learning in near-term quantum computers
Perdomo-Ortiz, A., Benedetti, M., Realpe-Gómez, J., & Biswas, R. (2018) · Quantum Science and Technology · doi:10.1088/2058-9565/aab859 · Cited by 274
“Implicit density estimators achieve the generative task indirectly; for example, by casting the problem into a classification task [50] or finding the Nash equilibrium of a game [51].”
BackgroundDeep digital twins for detection, diagnostics and prognostics
Booyse, W., Wilke, D. N., & Heyns, S. (2020) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2019.106612 · Cited by 274
A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications
Gui, J., Sun, Z., Wen, Y., Tao, D., & Ye, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.06937 · Cited by 253
Unsupervised Visual Representation Learning by Context Prediction
Doersch, C., Gupta, A., & Efros, A. A. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.05192 · Cited by 242
“For example, denoising autoencoders [46, 3] use reconstruction from noisy data as a pretext task; to tell the difference between noise and signal, the algorithm must connect images to other images with similar objects, in order to remove the parts of the image that don’t fit the shared appearance…”
MethodLee, D.-H., Zhang, S., Fischer, A., & Bengio, Y. (2015) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-23528-8_31 · Cited by 216
“Like in the work on denoising auto-encoders (Vincent et al., 2010) and Generative Stochastic Networks (Bengio et al., 2014), we consider the denoising auto-encoder like a stochastic network with noise injected in input and hidden units, trained to minimize a reconstruction loss. h = f(x) = sigm(Wx…”
BackgroundMethodSelf-citationFrom neural PCA to deep unsupervised learning
Valpola, H. (2015) · Advances in Independent Component Analysis and Learning Machines · doi:10.1016/b978-0-12-802806-3.00008-7 · Cited by 181
(2017) · doi:10.1109/ijcnn.2017.7965870 · Cited by 57
Metadata
- Authors
- Bengio, Y., Thibodeau-Laufer, É., Alain, G., & Yosinski, J.
- Venue
- arXiv (Cornell University)
- Publisher
- Cornell University
- Published
- 2013-06-05
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1306.1091
- OpenAlex
- W2126398289
- Cited by
- 290
- References
- 36
- Type
- posted content
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.