The DOI matches its Crossref record; 75 of the 75 references it cites were found in a registry or an index.
Towards Bayesian Deep Learning: A Framework and Some Existing Methods
Open accessAbstract
While perception tasks such as visual object recognition and text understanding play an important role in human intelligence, subsequent tasks that involve inference, reasoning, and planning require an even higher level of intelligence. The past few years have seen major advances in many perception tasks using deep learning models. For higher-level inference, however, probabilistic graphical models with their Bayesian nature are still more powerful and flexible. To achieve integrated intelligence that involves both perception and inference, it is naturally desirable to tightly integrate deep learning and Bayesian models within a principled probabilistic framework, which we call Bayesian deep learning. In this unified framework, the perception of text or images using deep learning can boost the performance of higher-level inference and in return, the feedback from the inference process is able to enhance the perception of text or images. This paper proposes a general framework for Bayesian deep learning and reviews its recent applications on recommender systems, topic models, and control. In this paper, we also discuss the relationship and differences between Bayesian deep learning and other related topics such as the Bayesian treatment of neural networks.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
(2014) · Cited by 33,419
(2003) · Journal of Machine Learning Research · doi:10.5555/944919.944937 · Cited by 32,337
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545
Book Review: Pattern Recognition and Machine Learning
Nasrabadi, N. M. (2007) · Journal of Electronic Imaging · doi:10.1117/1.2819119 · Cited by 22,066
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067
Blei, D. M., Ng, A. Y., & Jordan, M. I. (2002) · Advances in Neural Information Processing Systems 14 · doi:10.7551/mitpress/1120.003.0082 · Cited by 12,223
Pattern Recognition and Machine Learning
(2007) · Kybernetes · doi:10.1108/03684920710743466 · Cited by 8,265
Receptive fields and functional architecture of monkey striate cortex
Hubel, D. H., & Wiesel, T. N. (1968) · The Journal of Physiology · doi:10.1113/jphysiol.1968.sp008455 · Cited by 6,568
Training Products of Experts by Minimizing Contrastive Divergence
Hinton, G. E. (2002) · Neural Computation · doi:10.1162/089976602760128018 · Cited by 4,879
Bayesian Learning for Neural Networks
Neal, R. M. (1996) · Lecture Notes in Statistics · doi:10.1007/978-1-4612-0745-0 · Cited by 4,276
(2010) · Journal of Machine Learning Research · doi:10.5555/1756006.1953039 · Cited by 4,051
Probabilistic Matrix Factorization
Mnih, A., & Salakhutdinov, R. (2007) · neural information processing systems · Cited by 3,798
An Introduction to Variational Methods for Graphical Models
Jordan, M. I., Ghahramani, Z., Jaakkola, T. S., & Saul, L. K. (1999) · Machine Learning · doi:10.1023/a:1007665907178 · Cited by 3,791
(2011) · Springer eBooks · doi:10.1007/978-0-387-85820-3 · Cited by 3,761
A Convolutional Neural Network for Modelling Sentences
Kalchbrenner, N., Grefenstette, E., & Blunsom, P. (2014) · Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.3115/v1/p14-1062 · Cited by 3,601
Collaborative Filtering for Implicit Feedback Datasets
Hu, Y., Koren, Y., & Volinsky, C. (2008) · 2008 Eighth IEEE International Conference on Data Mining · doi:10.1109/icdm.2008.22 · Cited by 3,170
A practical Bayesian framework for backpropagation networks
MacKay, D. J. C. (1992) · Neural Computation · doi:10.1162/neco.1992.4.3.448 · Cited by 2,917
Blei, D. M., & Lafferty, J. D. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143859 · Cited by 2,386
Introduction to Recommender Systems Handbook
Ricci, F., Rokach, L., & Shapira, B. (2011) · Recommender Systems Handbook · doi:10.1007/978-0-387-85820-3_1 · Cited by 2,378
An Introduction to the Conjugate Gradient Method Without the Agonizing Pain
Shewchuk, J. R. (1994) · Cited by 2,173
Restricted Boltzmann machines for collaborative filtering
Salakhutdinov, R., Mnih, A., & Hinton, G. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273596 · Cited by 1,885
Collaborative Knowledge Base Embedding for Recommender Systems
Zhang, F., Yuan, N. J., Lian, D., Xie, X., & Ma, W.-Y. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939673 · Cited by 1,749
Collaborative Deep Learning for Recommender Systems
Wang, H., Wang, N., & Yeung, D.-Y. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2783273 · Cited by 1,657
Self-citationCollaborative topic modeling for recommending scientific articles
Wang, C., & Blei, D. M. (2011) · Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/2020408.2020480 · Cited by 1,527
Online Learning for Latent Dirichlet Allocation
Hoffman, M. D., Blei, D. M., & Bach, F. (2010) · Cited by 1,363
Auto-association by multilayer perceptrons and singular value decomposition
Bourlard, H., & Kamp, Y. (1988) · Biological Cybernetics · doi:10.1007/bf00332918 · Cited by 1,345
Weight Uncertainty in Neural Network
Blundell, C., Cornebise, J., Kavukcuoglu, K., & Wierstra, D. (2015) · International Conference on Machine Learning · Cited by 1,335
Stochastic variational inference
Hoffman, M. D., Blei, D. M., Wang, C., & Paisley, J. W. (2013) · Journal of Machine Learning Research · doi:10.5555/2567709.2502622 · Cited by 1,333
Relational learning via collective matrix factorization
Singh, A. P., & Gordon, G. J. (2008) · Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1401890.1401969 · Cited by 1,204
Contractive Auto-Encoders: Explicit Invariance During Feature Extraction
Rifai, S., Vincent, P., Muller, X., Glorot, X., & Bengio, Y. (2011) · International Conference on Machine Learning · Cited by 1,190
Efficient Learning of Sparse Representations with an Energy-Based Model
Ranzato, M., Poultney, C., Chopra, S., & LeCun, Y. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0147 · Cited by 1,106
Probabilistic Matrix Factorization
Salakhutdinov, R., & Mnih, A. (2007) · Cited by 1,075
Keeping the neural networks simple by minimizing the description length of the weights
Hinton, G. E., & van Camp, D. (1993) · Proceedings of the sixth annual conference on Computational learning theory - COLT '93 · doi:10.1145/168304.168306 · Cited by 994
Deep content-based music recommendation
van den Oord, A., Dieleman, S., & Schrauwen, B. (2013) · Ghent University Academic Bibliography (Ghent University) · Cited by 962
Lafferty, J., & Blei, D. M. (2005) · neural information processing systems · Cited by 917
Improving Aggregate Recommendation Diversity Using Ranking-Based Techniques
Adomavicius, G., & Kwon, Y. (2012) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2011.15 · Cited by 675
Low-rank matrix factorization for Deep Neural Network training with high-dimensional output targets
Sainath, T. N., Kingsbury, B., Sindhwani, V., Arisoy, E., & Ramabhadran, B. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638949 · Cited by 593
Fast collapsed gibbs sampling for latent dirichlet allocation
Porteous, I., Newman, D., Ihler, A., Asuncion, A., Smyth, P., & Welling, M. (2008) · Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1401890.1401960 · Cited by 560
Connectionist learning of belief networks
Neal, R. M. (1992) · Artificial Intelligence · doi:10.1016/0004-3702(92)90065-6 · Cited by 513
Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks
Hernández-Lobato, J. M., & Adams, R. P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1502.05336 · Cited by 504
Marginalized Denoising Autoencoders for Domain Adaptation
Chen, M., Xu, Z., Weinberger, K. Q., & Sha, F. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1206.4683 · Cited by 498
Deep Collaborative Filtering via Marginalized Denoising Auto-encoder
Li, S., Kawale, J., & Fu, Y. (2015) · Proceedings of the 24th ACM International on Conference on Information and Knowledge Management · doi:10.1145/2806416.2806527 · Cited by 422
(2021) · The Multivariate Normal Distribution · doi:10.1142/9789811235290_0012 · Cited by 379
Improving Content-based and Hybrid Music Recommendation using Deep Learning
Wang, X., & Wang, Y. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654940 · Cited by 367
A Guide to Distribution Theory and Fourier Transforms.
Synowiec, J. A., & Strichartz, R. (1996) · The American Mathematical Monthly · doi:10.2307/2974944 · Cited by 339
Generalized Denoising Auto-Encoders as Generative Models
Bengio, Y., Yao, L., Alain, G., & Vincent, P. (2013) · arXiv · doi:10.48550/arxiv.1305.6663 · Cited by 324
Collaborative Filtering Meets Mobile Recommendation: A User-Centered Approach
Zheng, V., Cao, B., Zheng, Y., Xie, X., & Yang, Q. (2010) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v24i1.7577 · Cited by 321
Continuous Time Dynamic Topic Models
Wang, C., Blei, D. M., & Heckerman, D. E. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1206.3298 · Cited by 311
A Guide to Distribution Theory and Fourier Transforms
Strichartz, R. S. (2003) · WORLD SCIENTIFIC eBooks · doi:10.1142/5314 · Cited by 242
Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Images
Watter, M., Springenberg, J. T., Boedecker, J., & Riedmiller, M. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.07365 · Cited by 212
Typicality-Based Collaborative Filtering Recommendation
Cai, Y., Leung, H.-F., Li, Q., Min, H., Tang, J., & Li, J. (2014) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2013.7 · Cited by 210
SVDFeature: a toolkit for feature-based collaborative filtering
Chen, T., Zhang, W., Lu, Q., Chen, K., Zhao, Z., & Yu, Y. H. (2012) · Cited by 205
Negative Binomial Process Count and Mixture Modeling
Zhou, M., & Carin, L. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.211 · Cited by 191
Korattikara, A., Rathod, V., Murphy, K., & Welling, M. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.04416 · Cited by 182
Collaborative Topic Regression with Social Matrix Factorization for Recommendation SystemsPreprint
Purushotham, S., Liu, Y., & Kuo, C. ‐. J. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1206.4684 · Cited by 177
A non-IID Framework for Collaborative Filtering with Restricted Boltzmann Machines
Georgiev, K. B., & Nakov, P. (2013) · Cited by 128
Relational Stacked Denoising Autoencoder for Tag Recommendation
Wang, H., Shi, X., & Yeung, D. (2015) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v29i1.9548 · Cited by 125
Self-citationMarginalized Denoising Auto-encoders for Nonlinear Representations
Chen, M., Weinberger, K. Q., Sha, F., & Bengio, Y. (2014) · International Conference on Machine Learning · Cited by 115
Collaborative Topic Regression with Social Regularization for Tag Recommendation
Wang, H., Chen, B., & Li, W.-J. (2013) · doi:10.13140/2.1.3004.7689 · Cited by 107
Self-citationRelation regularized matrix factorization
Li, W.-J., & Yeung, D. (2009) · Cited by 99
Self-citationLiu, N., Meng, X., Liu, C., & Yang, Q. (2011) · ACM Conference on Recommender Systems (RecSys) · doi:10.1145/2043932.2043943 · Cited by 91
Learning Deep Sigmoid Belief Networks with Data Augmentation
Gan, Z., Henao, R., Carlson, D., & Carin, L. (2015) · International Conference on Artificial Intelligence and Statistics · Cited by 83
Collaborative Recurrent Autoencoder: Recommend while Learning to Fill in the BlanksPreprint
Wang, H., Shi, X., & Yeung, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.00454 · Cited by 82
Self-citationThe Adaptive Clustering Method for the Long Tail Problem of Recommender Systems
Park, Y.-J. (2012) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2012.119 · Cited by 75
Collaborative Deep Ranking: A Hybrid Pair-Wise Recommendation Algorithm with Implicit Feedback
Ying, H., Chen, L., Xiong, Y., & Wu, J. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-31750-2_44 · Cited by 72
Relational Collaborative Topic Regression for Recommender Systems
Wang, H., & Li, W.-J. (2014) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2014.2365789 · Cited by 69
Self-citationScalable Deep Poisson Factor Analysis for Topic Modeling
Gan, Z., Chen, C., Henao, R., et al. (2015) · Cited by 57
Natural-Parameter Networks: A Class of Probabilistic Neural NetworksPreprint
Wang, H., Shi, X., & Yeung, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.00448 · Cited by 53
Self-citationProduct of Gaussians for speech recognition
Gales, M., & Airey, S. S. (2005) · Computer Speech & Language · doi:10.1016/j.csl.2004.12.002 · Cited by 49
Collaborative Filtering with Personalized Skylines
Ilaria Bartolini, A., Zhang, Z., & Dimitris Papadias, C. (2010) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2010.86 · Cited by 47
(1987) · Cited by 47
Learning users' interests by quality classification in market-based recommender systems
Wei, Y. Z., Moreau, L., & Jennings, N. R. (2005) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2005.200 · Cited by 35
Extending Recommender Systems for Disjoint User/Item Sets: The Conference Recommendation Problem
Hornick, M. F., & Tamayo, P. (2012) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2011.90 · Cited by 34
Cited by
A review of uncertainty quantification in deep learning: Techniques, applications and challenges
Abdar, M., Pourpanah, F., Hussain, S., Rezazadegan, D., Liu, L., Ghavamzadeh, M., Fieguth, P., Cao, X., Khosravi, A., Acharya, U. R., et al. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.05.008 · Cited by 3,111
Deep Learning for Anomaly Detection: A Review
Pang, G., Shen, C., Cao, L., & Hengel, A. V. D. (2022) · ACM Computing Surveys · doi:10.1145/3439950 · Cited by 2,837
Deep Learning Based Recommender System: A Survey and New Perspectives
Zhang, S., Yao, L., Sun, A., & Tay, Y. (2020) · ACM Computing Surveys · doi:10.1145/3285029 · Cited by 1,511
Explainable artificial intelligence: a comprehensive review
Minh, D., Wang, H. X., Li, Y. F., & Nguyen, T. N. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-021-10088-y · Cited by 983
Self-citationDeep learning in cancer diagnosis, prognosis and treatment selection
Tran, K. A., Kondrashova, O., Bradley, A., Williams, E. D., Pearson, J. V., & Waddell, N. (2021) · Genome Medicine · doi:10.1186/s13073-021-00968-x · Cited by 932
Metadata
- Authors
- Wang, H., & Yeung, D.-Y.
- Venue
- IEEE Transactions on Knowledge and Data Engineering
- Published
- 2016-12-01
- Volume
- 28
- Issue
- 12
- Pages
- 3395-3408
- ISSN
- 1041-4347 (ISSN-L)
- OpenAlex
- W2963058055
- Cited by
- 253
- References
- 75
- FWCI
- 16.04
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.