Bayesian Methods for Neural Networks and Related Models

Titterington, D. M.

doi:10.1214/088342304000000099

Cited by 162Markov Chains and Monte Carlo MethodsGaussian Processes and Bayesian InferenceNeural Networks and Applications

Bayesian Methods for Neural Networks and Related Models

Titterington, D. M.·2004·Statistical Science

DOI 10.1214/088342304000000099

Open this paper in CoChat — free account

Abstract

Models such as feed-forward neural networks and certain other structures investigated in the computer science literature are not amenable to closed-form Bayesian analysis. The paper reviews the various approaches taken to overcome this difficulty, involving the use of Gaussian approximations, Markov chain Monte Carlo simulation routines and a class of non-Gaussian but “deterministic” approximations called variational approximations.

Integrity signals

Registry

Crossref · 10.1214/088342304000000099

Cites

102 of 102 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Bayesian Neural Networks are stochastic artificial neural networks trained with Bayesian inference.
“Bayesian Neural Networks (BNNs) [68,71,72] are stochastic ANNs trained using Bayesian inference.”
From Explainable AI in medical imaging: An overview for clinical practitioners – Beyond saliency-based XAI approaches (2023) · cites this paper as [72]

The Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970 · see citing works
Neural networks for pattern recognition
Bishop, C. (1994) · Choice Reviews Online · doi:10.5860/choice.31-5500 · Cited by 18,537 · see citing works
Statistical Decision Theory and Bayesian Analysis
Berger, J. O. (1985) · Springer Series in Statistics · doi:10.1007/978-1-4757-4286-2 · Cited by 7,722 · see citing works
Pattern Recognition and Neural Networks
Ripley, B. D. (1996) · Cambridge University Press eBooks · doi:10.1017/cbo9780511812651 · Cited by 6,446 · see citing works
Reversible Jump Markov Chain Monte Carlo Computation and Bayesian Model Determination
Green, P. J. (1995) · Biometrika · doi:10.1093/biomet/82.4.711 · Cited by 5,941 · see citing works
Bayesian Interpolation
Mackay, D. (1992) · Neural Computation · doi:10.1162/neco.1992.4.3.415 · Cited by 4,440 · see citing works
Bayesian Learning for Neural Networks
Neal, R. M. (1996) · Lecture Notes in Statistics · doi:10.1007/978-1-4612-0745-0 · Cited by 4,276 · see citing works
Hybrid Monte Carlo
Duane, S., Kennedy, A. D., Pendleton, B. J., & Roweth, D. (1987) · Physics Letters B · doi:10.1016/0370-2693(87)91197-x · Cited by 3,920 · see citing works
An Introduction to Variational Methods for Graphical Models
Jordan, M. I., Zoubin Ghahramani, A., Jaakkola, T., & Saul, L. K. (1999) · Machine Learning · doi:10.1023/a:1007665907178 · Cited by 3,665 · see citing works
A practical Bayesian framework for backpropagation networks
MacKay, D. J. C. (1992) · Neural Computation · doi:10.1162/neco.1992.4.3.448 · Cited by 2,917 · see citing works
Statistical Analysis of Finite Mixture Distributions.
Lindsay, B. G., Titterington, D. M., Smith, A. F. M., & Makov, U. E. (1987) · Journal of the American Statistical Association · doi:10.2307/2289504 · Cited by 2,906 · see citing works
Hierarchical Mixtures of Experts and the EM Algorithm
Jordan, M. I., & Jacobs, R. A. (1994) · Neural Computation · doi:10.1162/neco.1994.6.2.181 · Cited by 2,730 · see citing works
Learning in Graphical Models
Jordan, M. I. (1998) · Springer eBooks · doi:10.1007/978-94-011-5014-9 · Cited by 2,449 · see citing works
A View of the Em Algorithm that Justifies Incremental, Sparse, and other Variants
Neal, R. M., & Hinton, G. E. (1998) · Learning in Graphical Models · doi:10.1007/978-94-011-5014-9_12 · Cited by 2,187 · see citing works
10.1162/15324430152748236
(2000) · Applied Physics Letters · doi:10.1162/15324430152748236 · Cited by 1,819 · see citing works
A Tutorial on Learning with Bayesian Networks
Heckerman, D. E. (2008) · Studies in computational intelligence · doi:10.1007/978-3-540-85066-3_3 · Cited by 1,712 · see citing works
Neural Networks
Bishop, C. (1996) · Taylor & Francis eBooks · doi:10.1201/9781420050646.ptb6 · Cited by 1,607 · see citing works
A Tutorial on Learning with Bayesian Networks
Heckerman, D. E. (1998) · Springer eBooks · doi:10.1007/978-94-011-5014-9_11 · Cited by 1,242 · see citing works
Factorial Hidden Markov Models
Ghahramani, Z., & Jordan, M. I. (1997) · Machine Learning · doi:10.1023/a:1007425814087 · Cited by 1,157 · see citing works
Neural Networks: A Review from a Statistical Perspective
Cheng, B., & Titterington, D. M. (1994) · Statistical Science · doi:10.1214/ss/1177010638 · Cited by 1,054 · see citing works
Keeping the neural networks simple by minimizing the description length of the weights
Hinton, G. E., & van Camp, D. (1993) · Proceedings of the sixth annual conference on Computational learning theory - COLT '93 · doi:10.1145/168304.168306 · Cited by 994 · see citing works
Probable networks and plausible predictions — a review of practical Bayesian methods for supervised neural networks
Mackay, D. J. C. (1995) · Network Computation in Neural Systems · doi:10.1088/0954-898x/6/3/011 · Cited by 915 · see citing works
The Relevance Vector Machine
Tipping, M. E. (1999) · Cited by 869 · see citing works
The EM Algorithm—an Old Folk-song Sung to a Fast New Tune
Meng, X.-L., & Van Dyk, D. (1997) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/1467-9868.00082 · Cited by 802 · see citing works
The Evidence Framework Applied to Classification Networks
Mackay, D. (1992) · Neural Computation · doi:10.1162/neco.1992.4.5.720 · Cited by 724 · see citing works
A Variational Baysian Framework for Graphical Models
Attias, H. T. (1999) · neural information processing systems · Cited by 689 · see citing works
Neural Networks and Related Methods for Classification
Ripley, B. D. (1994) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.2517-6161.1994.tb01990.x · Cited by 627 · see citing works
Information geometry and alternating minimization procedures
Csiszár, I. (1984) · Medical Entomology and Zoology · Cited by 596 · see citing works
Bayesian parameter estimation via variational methods
Jaakkola, T., & Jordan, M. I. (2000) · Statistics and Computing · doi:10.1023/a:1008932416310 · Cited by 594 · see citing works
An Introduction to Variational Methods for Graphical Models
Jordan, M. I., Zoubin Ghahramani, A., Jaakkola, T., & Saul, L. K. (1998) · Springer eBooks · doi:10.1007/978-94-011-5014-9_5 · Cited by 551 · see citing works
Inferring Parameters and Structure of Latent Variable Models by Variational BayesPreprint
Attias, H. T. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.6676 · Cited by 525 · see citing works
Reversible Jump Markov Chain Monte Carlo Computation and Bayesian Model Determination
Green, P. J. (1995) · Biometrika · doi:10.2307/2337340 · Cited by 503 · see citing works
Independent Factor Analysis
Attias, H. T. (1999) · Neural Computation · doi:10.1162/089976699300016458 · Cited by 501 · see citing works
The Effective Number of Parameters: An Analysis of Generalization and Regularization in Nonlinear Learning Systems
Moody, J. E. (1991) · Cited by 437 · see citing works
Bayesian approach for neural networks—review and case studies
Lampinen, J., & Vehtari, A. (2001) · Neural Networks · doi:10.1016/s0893-6080(00)00098-8 · Cited by 392 · see citing works
Advanced mean field methods: theory and practice
Opper, M., & Saad, D. (2001) · Cited by 385 · see citing works
Variational Inference for Bayesian Mixtures of Factor Analysers
Ghahramani, Z., & Beal, M. J. (1999) · Cited by 380 · see citing works
Online Model Selection Based on the Variational Bayes
Sato, M.-A. (2001) · Neural Computation · doi:10.1162/089976601750265045 · Cited by 378 · see citing works
Variational Relevance Vector MachinesPreprint
Bishop, C., & Tipping, M. E. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.3838 · Cited by 357 · see citing works
Bayesian Back-Propagation.
Buntine, W., & Weigend, A. S. (1991) · Complex Systems · Cited by 357 · see citing works
The mean field theory in EM procedures for Markov random fields
Zhang, J. (1992) · IEEE Transactions on Signal Processing · doi:10.1109/78.157297 · Cited by 347 · see citing works
Variational Bayesian Model Selection for Mixture Distributions
Corduneanu, A., & Bishop, C. (2001) · Cited by 345 · see citing works
Advanced Mean Field Methods
(2001) · The MIT Press eBooks · doi:10.7551/mitpress/1100.001.0001 · Cited by 335 · see citing works
Bayesian Framework for Least-Squares Support Vector Machine Classifiers, Gaussian Processes, and Kernel Fisher Discriminant Analysis
Gestel, T. V., Suykens, J. A. K., Lanckriet, G. R. G., Lambrechts, A., Moor, B. D., & Vandewalle, J. P. L. (2002) · Neural Computation · doi:10.1162/089976602753633411 · Cited by 291 · see citing works
Propagation Algorithms for Variational Bayesian Learning
Ghahramani, Z., & Beal, M. J. (2000) · Cited by 281 · see citing works
Variational principal components
Bishop, C. (1999) · doi:10.1049/cp:19991160 · Cited by 274 · see citing works
Tutorial on Variational Approximation Methods
Jaakkola, T. (2001) · The MIT Press eBooks · doi:10.7551/mitpress/1100.003.0014 · Cited by 270 · see citing works
Comparison of Approximate Methods for Handling Hyperparameters
Mackay, D. (1999) · Neural Computation · doi:10.1162/089976699300016331 · Cited by 246 · see citing works
Variational Gaussian process classifiers
Mackay, D., & Gibbs, M. (2000) · IEEE Transactions on Neural Networks · doi:10.1109/72.883477 · Cited by 224 · see citing works
Sequential Monte Carlo Methods to Train Neural Network Models
Freitas, J., Niranjan, M., Gee, A. H., & Arnaud Doucet, B. (2000) · Neural Computation · doi:10.1162/089976600300015664 · Cited by 213 · see citing works
Developments of the generative topographic mapping
Bishop, C., Svensén, M., & Williams, C. K. I. (1998) · Neurocomputing · doi:10.1016/s0925-2312(98)00043-5 · Cited by 199 · see citing works
A review of Bayesian neural networks with an application to near infrared spectroscopy
Thodberg, H. H. (1996) · IEEE Transactions on Neural Networks · doi:10.1109/72.478392 · Cited by 184 · see citing works
Bayesian Methods for Mixtures of Experts
Waterhouse, S. R., Mackay, D., & Robinson, A. J. (1995) · Cited by 175 · see citing works
Pricing and hedging derivative securities with neural networks: Bayesian regularization, early stopping, and bagging
Gençay, R., & Qi, M. (2001) · IEEE Transactions on Neural Networks · doi:10.1109/72.935086 · Cited by 174 · see citing works
Ensemble Learning for Blind Image Separation and Deconvolution
Miskin, J. W., & Mackay, D. (2000) · Perspectives in neural computing · doi:10.1007/978-1-4471-0443-8_7 · Cited by 164 · see citing works
Bethe free energy, Kikuchi approximations, and belief propagation algorithms
Yedidia, J. S., Freeman, W. T., & Weiss, Y. (2001) · Cited by 162 · see citing works
Moderating the outputs of support vector machine classifiers
Kwok, J. T.-Y. (1999) · IEEE Transactions on Neural Networks · doi:10.1109/72.788642 · Cited by 155 · see citing works
The evidence framework applied to support vector machines
Kwok, J. T.-Y. (2000) · IEEE Transactions on Neural Networks · doi:10.1109/72.870047 · Cited by 155 · see citing works
Bayesian Measures of Model Complexity and Fit
Spiegelhalter, D. J., Best, N. G., Carlin, B. P., & Van Der Linde, A. (2002) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/1467-9868.00353 · Cited by 151 · see citing works
A Bayesian Analysis of Self-Organizing Maps
Luttrell, S. P. (1994) · Neural Computation · doi:10.1162/neco.1994.6.5.767 · Cited by 149 · see citing works
Computation with Infinite Neural Networks
Williams, C. K. I. (1998) · Neural Computation · doi:10.1162/089976698300017412 · Cited by 149 · see citing works
The mean field theory in EM procedures for blind Markov random field image restoration
Zhang, J. (1993) · IEEE Transactions on Image Processing · doi:10.1109/83.210863 · Cited by 149 · see citing works
Bayesian Inference in Mixtures-of-Experts and Hierarchical Mixtures-of-Experts Models with an Application to Speech Recognition
Peng, F., Jacobs, R. A., & Tanner, M. A. (1996) · Journal of the American Statistical Association · doi:10.1080/01621459.1996.10476965 · Cited by 149 · see citing works
Multiresolution forecasting for futures trading using wavelet decompositions
Zhang, B., Coggins, R., Jabri, M. A., Dersch, D. R., & Flower, B. (2001) · IEEE Transactions on Neural Networks · doi:10.1109/72.935090 · Cited by 137 · see citing works
Bayesian information criteria and smoothing parameter selection in radial basis function networks
Konishi, S. (2004) · Biometrika · doi:10.1093/biomet/91.1.27 · Cited by 135 · see citing works
Bayesian model search for mixture models based on optimizing variational bounds
Ueda, N., & Ghahramani, Z. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00040-0 · Cited by 133 · see citing works
[Neural Networks: A Review from Statistical Perspective]: Rejoinder
Cheng, B., & Titterington, D. M. (1994) · Statistical Science · doi:10.1214/ss/1177010646 · Cited by 131 · see citing works
Issues in Bayesian Analysis of Neural Network Models
Müller, P., & Insua, D. R. (1998) · Neural Computation · doi:10.1162/089976698300017737 · Cited by 115 · see citing works
Bayesian neural networks for classification: how useful is the evidence framework?
Penny, W. D., & Roberts, S. (1999) · Neural Networks · doi:10.1016/s0893-6080(99)00040-4 · Cited by 104 · see citing works
Bayesian Model Selection for Support Vector Machines, Gaussian Processes and Other Kernel Classifiers
Seeger, M. (1999) · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · Cited by 104 · see citing works
Robust Full Bayesian Learning for Radial Basis Networks
Andrieu, C., de Freitas, N., & Doucet, A. (2001) · Neural Computation · doi:10.1162/089976601750541831 · Cited by 94 · see citing works
Bayesian Radial Basis Functions of Variable Dimension
Holmes, C. C., & Mallick, B. K. (1998) · Neural Computation · doi:10.1162/089976698300017421 · Cited by 93 · see citing works
VIBES: A Variational Inference Engine for Bayesian Networks
Bishop, C., Spiegelhalter, D. J., & Winn, J. M. (2002) · neural information processing systems · Cited by 84 · see citing works
Asymptotic Model Selection for Directed Networks with Hidden Variables
Geiger, D., Heckerman, D. E., & Meek, C. (1998) · Springer eBooks · doi:10.1007/978-94-011-5014-9_16 · Cited by 77 · see citing works
Bayesian approach to neural-network modeling with input uncertainty
Wright, W. A. (1999) · IEEE Transactions on Neural Networks · doi:10.1109/72.809073 · Cited by 74 · see citing works
Approximating Posterior Distributions in Belief Networks Using Mixtures
Bishop, C., Lawrence, N. D., Jaakkola, T., & Jordan, M. I. (1997) · Cited by 67 · see citing works
The relevance vector machine technique for channel equalization application
Chen, S., Gunn, S. R., & Harris, C. J. (2001) · IEEE Transactions on Neural Networks · doi:10.1109/72.963792 · Cited by 62 · see citing works
Hyperparameter Selection for Self-Organizing Maps
Utsugi, A. (1997) · Neural Computation · doi:10.1162/neco.1997.9.3.623 · Cited by 58 · see citing works
Variational Bayesian Learning of ICA with Missing Data
Chan, K., Lee, T.-W., & Sejnowski, T. J. (2003) · Neural Computation · doi:10.1162/08997660360675116 · Cited by 56 · see citing works
Variational MCMC
de Freitas, N., Højen-Sørensen, P. A. D. F. R., Jordan, M. I., & Russell, S. (2001) · Uncertainty in Artificial Intelligence · Cited by 56 · see citing works
Statistics and neural networks: advances at the interface
Kay, J., & Titterington, D. M. (2000) · Cited by 51 · see citing works
A Bayesian Approach to Model Selection in Hierarchical Mixtures-of-Experts Architectures
Jacobs, R. A., Peng, F., & Tanner, M. A. (1997) · Neural Networks · doi:10.1016/s0893-6080(96)00050-0 · Cited by 50 · see citing works
The application of neural networks to the papermaking industry
Edwards, P. J., Murray, A., Papadopoulos, G. T., Wallace, A. R., Barnard, J., & Smith, G. (1999) · IEEE Transactions on Neural Networks · doi:10.1109/72.809090 · Cited by 47 · see citing works
Modeling exchange rates: smooth transitions, neural networks, and linear models
Medeiros, M. C., Veiga, A., & Pedreira, C. E. (2001) · IEEE Transactions on Neural Networks · doi:10.1109/72.935089 · Cited by 46 · see citing works
Bayesian retrieval in associative memories with storage errors
Sommer, F. T., & Dayan, P. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/72.701183 · Cited by 41 · see citing works
Bayesian nonlinear model selection and neural networks: a conjugate prior approach
Vila, J.-P., Wagner, V., & Neveu, P. (2000) · IEEE Transactions on Neural Networks · doi:10.1109/72.838999 · Cited by 41 · see citing works
Parameter estimation for hidden Markov chains
Archer, G. E., & Titterington, D. M. (2002) · Journal of Statistical Planning and Inference · doi:10.1016/s0378-3758(02)00318-x · Cited by 39 · see citing works
An empirical evaluation of Bayesian sampling with hybrid Monte Carlo for training neural network classifiers
Husmeier, D., Penny, W. D., & Roberts, S. (1999) · Neural Networks · doi:10.1016/s0893-6080(99)00020-9 · Cited by 35 · see citing works
Model Selection for Neural Network Classification
Lee, H. K. H. (2001) · Journal of Classification · doi:10.1007/s00357-001-0017-y · Cited by 33 · see citing works
Approximate Bayesian inference for simple mixtures
Humphreys, K., & Titterington, D. M. (2000) · COMPSTAT · doi:10.1007/978-3-642-57678-2_42 · Cited by 30 · see citing works
The Bayesian Evidence Scheme for Regularizing Probability-Density Estimating Neural Networks
Husmeier, D. (2000) · Neural Computation · doi:10.1162/089976600300014890 · Cited by 28 · see citing works
On the Adequacy of Variational Lower Bound Functions for Likelihood-based Inference in Markovian Models with Missing Values
Hall, P., Humphreys, K., & Titterington, D. M. (2002) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/1467-9868.00350 · Cited by 25 · see citing works
Comparing Bayesian neural network algorithms for classifying segmented outdoor images
Vivarelli, F., & Williams, C. K. I. (2001) · Neural Networks · doi:10.1016/s0893-6080(01)00024-7 · Cited by 23 · see citing works
A Noninformative Prior for Neural Networks
Lee, H. K. H. (2003) · Machine Learning · doi:10.1023/a:1020258113913 · Cited by 23 · see citing works
Probabilistic image processing by means of the Bethe approximation for theQ-Ising model
Tanaka, K., Inoue, J.-I., & Titterington, D. M. (2003) · Journal of Physics A Mathematical and General · doi:10.1088/0305-4470/36/43/025 · Cited by 23 · see citing works
A Tighter Bound for Graphical Models
Leisink, M. A. R., & Kappen, H. J. (2001) · Neural Computation · doi:10.1162/089976601750399344 · Cited by 22 · see citing works
Density Estimation by Mixture Models with Smoothing Priors
Utsugi, A. (1998) · Neural Computation · doi:10.1162/089976698300016990 · Cited by 20 · see citing works
Robust Full Bayesian Methods for Neural Networks
Andrieu, C., de Freitas, J. F. G., & Doucet, A. (1999) · Bristol Research (University of Bristol) · Cited by 12 · see citing works
Bayesian inference in neural networks
Paige, R. L. (2001) · Biometrika · doi:10.1093/biomet/88.3.623 · Cited by 11 · see citing works
Asymptotic Model Selection for Directed Networks with Hidden VariablesPreprint
Geiger, D., Heckerman, D. E., & Meek, C. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1302.3580 · Cited by 10 · see citing works
Improving the Mean-Field Approximation in Belief Networks Using Bahadur's Reparameterisation of the Multivariate Binary Distribution
Humphreys, K., & Titterington, D. M. (2000) · Neural Processing Letters · doi:10.1023/a:1009617914949 · Cited by 6 · see citing works
Some examples of recursive variational approximations
Humphreys, K., & Titterington, M. (2001) · Cited by 3 · see citing works

Flagged first, then by citations

1 of the 103 references couldn’t be matched to a known paper.

Sorted by citations

Approximate Bayesian Inference for Latent Gaussian models by using Integrated Nested Laplace Approximations
Rue, H., Martino, S., & Chopin, N. (2009) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2008.00700.x · Cited by 5,279 · see citing works
Bioinformatics and Computational Biology Solutions Using R and Bioconductor
Gentleman, R. C., Carey, V. J., Huber, W. K., Irizarry, R. A., & Dudoit, S. (2005) · Statistics in the health sciences · doi:10.1007/0-387-29362-0 · Cited by 3,470 · see citing works
“A review of Bayesian inference methods with neural network models is given in Titterington (2004).”
Method
Hands-On Bayesian Neural Networks—A Tutorial for Deep Learning Users
Jospin, L. V., Laga, H., Boussaid, F., Buntine, W., & Bennamoun, M. (2022) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2022.3155327 · Cited by 879 · see citing works
“Bayesian neural networks (BNNs) [8, 9, 10] are stochastic neural networks trained using a Bayesian approach.”
Method
Confronting the Challenge of Modeling Cloud and Precipitation Microphysics
Morrison, H., van Lier‐Walqui, M., Fridlind, A. M., Grabowski, W. W., Harrington, J. Y., Hoose, C., Korolev, A. V., Kumjian, M. R., Milbrandt, J. A., Pawłowska, H., et al. (2020) · Journal of Advances in Modeling Earth Systems · doi:10.1029/2019ms001689 · Cited by 672 · see citing works
Graphical Models
Jordan, M. I. (2004) · Statistical Science · doi:10.1214/088342304000000026 · Cited by 440 · see citing works
“…were formulated in terms of the minimization of a Kullback–Leibler (KL) divergence , and the space over which the optimization was performed was a set of " simplified " probability distributions , generally obtained by removing edges from a graphical model (see Titterington, 2004, this volume ).”
Statistical challenges of high-dimensional data
Johnstone, I. M., & Titterington, D. M. (2009) · Philosophical Transactions of the Royal Society A Mathematical Physical and Engineering Sciences · doi:10.1098/rsta.2009.0159 · Cited by 420 · see citing works
“Its recent application to many implementations of the Bayesian approach has mainly developed in machine learning, see Bishop (2006) and references therein, rather than in the mainstream statistics literature, although not exclusively so (Titterington 2004; Beal & Ghahramani 2006).”
Background
Predictive Abilities of Bayesian Regularization and Levenberg–Marquardt Algorithms in Artificial Neural Networks: A Comparative Empirical Study on Social Data
Kayri, M. (2016) · Mathematical and Computational Applications · doi:10.3390/mca21020020 · Cited by 405 · see citing works
“In this case, a gradient-based optimization algorithm is preferred for minimizing the objective [15,18,23,24]; In Equation (5), w 2 j , the sum of squares of network weights, α and β , are hyperparameters that need to be estimated function parameters.”
Background
Explaining Variational Approximations
Ormerod, J. T., & WAND, M. P. (2010) · The American Statistician · doi:10.1198/tast.2010.09058 · Cited by 403 · see citing works
“The expositions by Jordan (2004) and Titterington (2004) provide access to some of the literature on variational approximation accuracy.”
BackgroundInfluential
Predicting complex quantitative traits with Bayesian neural networks: a case study with Jersey cows and wheat
Daniel Gianola, B., Okut, H., WEIGEL, K. A., & Rosa, G. J. M. (2011) · BMC Genetics · doi:10.1186/1471-2156-12-87 · Cited by 285 · see citing works
“In this section, an approach used often for Bayesian regularization in neural networks [18,19] is presented along the lines of the hierarchical models employed by quantitative geneticists [15].”
Method
Nuclear mass predictions for the crustal composition of neutron stars: A Bayesian neural network approach
Utama, R., Piekarewicz, J., & Prosper, H. B. (2016) · Physical Review C · doi:10.1103/physrevc.93.014311 · Cited by 238 · see citing works
Explainable AI in medical imaging: An overview for clinical practitioners – Beyond saliency-based XAI approaches
Borys, K., Schmitt, Y. A., Nauta, M., Seifert, C., Krämer, N., Friedrich, C. M., & Nensa, F. (2023) · European Journal of Radiology · doi:10.1016/j.ejrad.2023.110786 · Cited by 233 · see citing works
Parametric and Nonparametric Statistical Methods for Genomic Selection of Traits with Additive and Epistatic Genetic Architectures
Howard, R., Carriquiry, A. L., & Beavis, W. D. (2014) · G3 Genes Genomes Genetics · doi:10.1534/g3.114.010298 · Cited by 216 · see citing works
A review of predictive uncertainty estimation with machine learning
Tyralis, H., & Papacharalampous, G. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-023-10698-8 · Cited by 198 · see citing works
Variational approximations in Bayesian model selection for finite mixture distributions
McGrory, C. A., & Titterington, D. M. (2006) · Computational Statistics & Data Analysis · doi:10.1016/j.csda.2006.07.020 · Cited by 160 · see citing works
Applied Bayesian Modeling And Causal Inference From Incomplete-Data Perspectives
Gelman, A., & Xiaoli, M. (2014) · Cited by 152 · see citing works
Nuclear charge radii: density functional theory meets Bayesian neural networks
Utama, R., Chen, W.-C., & Piekarewicz, J. (2016) · Journal of Physics G Nuclear and Particle Physics · doi:10.1088/0954-3899/43/11/114002 · Cited by 148 · see citing works
Bayesian regularized artificial neural networks for the estimation of the probability of default
Sariev, E., & Germano, G. (2019) · Quantitative Finance · doi:10.1080/14697688.2019.1633014 · Cited by 135 · see citing works
“Titterington (2004) reviewed the various approaches taken to determine the network architecture, involving the use of Gaussian approximations and of non-Gaussian but deterministic approximations called variational approximations.”
Background
Convergence properties of a general algorithm for calculating variational Bayesian estimates for a normal mixture model
Titterington, D. M., & Wang, B. Y. (2006) · Bayesian Analysis · doi:10.1214/06-ba121 · Cited by 118 · see citing works
“Keywords: Mixture model, Variational Bayes, Local convergence, Laplace approximation”
BackgroundMethodInfluential
Bayesian Regularized Neural Networks for Small n Big p Data
Okut, H. (2016) · InTech eBooks · doi:10.5772/63256 · Cited by 109 · see citing works
“An alternative approach [12, 14] involves iteration (updating is from right to left, with wMAP evaluated at the “old” values of the tuning parameters)”
Method
Bayesian training of artificial neural networks used for water resources modeling
Kingston, G. B., Lambert, M. F., & Maier, H. R. (2005) · Water Resources Research · doi:10.1029/2005wr004152 · Cited by 103 · see citing works
“These approaches have since been reviewed by MacKay [1995], Bishop [1995], Neal [1996a], Lampinen and Vehtari [2001], and Titterington [2004].”
Method
Gaussian Variational Approximate Inference for Generalized Linear Mixed Models
Ormerod, J. T., & WAND, M. P. (2011) · Journal of Computational and Graphical Statistics · doi:10.1198/jcgs.2011.09118 · Cited by 101 · see citing works
“The lower-bound expression (3.1) can also be derived using the ideas of Kullback– Leibler divergence, which underpins much of variational approximation methodology (e.g., Titterington 2004; Bishop 2006, chap.”
MethodInfluential
An evaluation of Bayesian techniques for controlling model complexity and selecting inputs in a neural network for short-term load forecasting
Hippert, H. S., & Taylor, J. W. (2009) · Neural Networks · doi:10.1016/j.neunet.2009.11.016 · Cited by 98 · see citing works
Prediction of body mass index in mice using dense molecular markers and a regularized neural network
Okut, H., Daniel Gianola, B., Rosa, G. J. M., & WEIGEL, K. A. (2011) · Genetics Research · doi:10.1017/s0016672310000662 · Cited by 85 · see citing works
“Alternatively,BRgaugesanobjective functionconsistingofaresidualsumofsquaresplus thesumofsquaredweights;thefunctionisminimized with respect to the weights, and the aim is to produce a network that generalizes well (Bishop & Tipping, 1998; Titterington, 2004; Marwala, 2007; Ripley, 2007).”
A Review of Parameter Learning Methods in Bayesian Network
Ji, Z., Xia, Q., & Meng, G. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-22053-6_1 · Cited by 81 · see citing works
Lessons in uncertainty quantification for turbulent dynamical systems
Majda, A. J., & Branicki, M. (2012) · Discrete and Continuous Dynamical Systems · doi:10.3934/dcds.2012.32.3133 · Cited by 77 · see citing works
“…(UQ) is undoubtedly an important issue in many physics, engineering, neural science and geoscience applications where complex nonlinear interactions between the resolved and unresolved processes, as well as various parametric uncertainties, need to be properly treated in order to reliably…”
Background
Predicting expected progeny difference for marbling score in Angus cattle using artificial neural networks and Bayesian regression models
Okut, H., Wu, X.-L., Rosa, G. J. M., Bauck, S. W., Woodward, B. W., Schnabel, R. D., Taylor, J. F., & Daniel Gianola, B. (2013) · Genetics Selection Evolution · doi:10.1186/1297-9686-45-34 · Cited by 71 · see citing works
“Large values of α lead to posterior densities of weights that are highly concentrated around zero, so that the weights effectively disappear, the model discounts connections in the network [24,25], and complex models are automatically self-penalized.”
Background
Deep Learning in Diverse Intelligent Sensor Based Systems
Zhu, Y., Wang, M., Yin, X., Zhang, J., Meijering, E. H. W., & Hu, J. (2022) · Sensors · doi:10.3390/s23010062 · Cited by 67 · see citing works
Assessing Systematic Effects of Stroke on Motor Control by Using Hierarchical Function-On-Scalar Regression
Goldsmith, J., & Kitago, T. (2015) · Journal of the Royal Statistical Society Series C (Applied Statistics) · doi:10.1111/rssc.12115 · Cited by 65 · see citing works
“Variational Bayes methods are regularly used in the computer science literature, and to a more limited extent in the statistics literature, to provide approximate solutions to intractable inference problems (Jordan, 2004; Jordan et al., 1999; Titterington, 2004; Ormerod and Wand, 2012).”
Computational Advances for and from Bayesian Analysis
Andrieu, C., Arnaud Doucet, B., & Robert, C. P. (2004) · Statistical Science · doi:10.1214/088342304000000071 · Cited by 62 · see citing works
“Some are mentioned in other sections (sequential Monte Carlo methods, Section 4) or in other papers in this volume (like variational methods, Jordan, 2004; Titterington, 2004).”
Method
A methodology for data gap filling in wave records using Artificial Neural Networks
Vieira, F., Cavalcante, G. H., Campos, E. J. D., & Pinto, F. T. (2020) · Applied Ocean Research · doi:10.1016/j.apor.2020.102109 · Cited by 58 · see citing works
Challenges in Markov Chain Monte Carlo for Bayesian Neural Networks
Papamarkou, T., Hinkle, J. D., Young, M. T., & Womble, D. E. (2022) · Statistical Science · doi:10.1214/21-sts840 · Cited by 60 · see citing works
“For a review of Bayesian approaches to neural networks, see Titterington (2004).”
Background
Explainable Artificial Intelligence for Bayesian Neural Networks: Toward Trustworthy Predictions of Ocean Dynamics
Clare, M., Sonnewald, M., Lguensat, R., Deshayes, J., & Balaji, V. (2022) · Journal of Advances in Modeling Earth Systems · doi:10.1029/2022ms003162 · Cited by 56 · see citing works
“A number of methods have been proposed to approximate the denominator term including Markov Chain Monte Carlo sampling (Titterington, 2004) and variational inference (Osawa et al., 2019).”
Backpropagation Neural Network-Based Machine Learning Model for Prediction of Soil Friction Angle
Nguyen, T., Ly, H., & Pham, B. T. (2020) · Mathematical Problems in Engineering · doi:10.1155/2020/8845768 · Cited by 53 · see citing works
“For the purpose of improving the generalization of the model, the gradient-based optimization algorithm is preferred to minimize the target [48, 49].”
Method
Spatial Root Zone Soil Water Content Estimation in Agricultural Lands Using Bayesian-Based Artificial Neural Networks and High- Resolution Visual, NIR, and Thermal Imagery
Hassan-Esfahani, L., Torres‐Rua, A. F., Jensen, A. M., & McKee, M. (2017) · Irrigation and Drainage · doi:10.1002/ird.2098 · Cited by 49 · see citing works
Reliability and Interpretability in Science and Deep Learning
Scorzato, L. (2024) · Minds and Machines · doi:10.1007/s11023-024-09682-0 · Cited by 47 · see citing works
“…cost, which leads to a very active search for approximations that offer the best trade-off between accuracy and computational efficiency (Titterington, 2004; Jospin et al., 2020; MacKay, 1992; Gal and Ghahramani, 2016; Blundell et al., 2015; Sensoy et al., 2018; Hartmann and Richter,…”
Predicting methane emission in Canadian Holstein dairy cattle using milk mid-infrared reflectance spectroscopy and other commonly available predictors via artificial neural networks
Shadpour, S., Chud, T. C. S., Hailemariam, D., Plastow, G. S., de Oliveira, H. R., Stothard, P., Lassen, J., Miglior, F., Baes, C. F., Tulpan, D., et al. (2022) · Journal of Dairy Science · doi:10.3168/jds.2021-21176 · Cited by 43 · see citing works
Correlation between molecular features and electrochemical properties using an artificial neural network
Chen, F. F., Breedon, M., White, P., Chu, C., Mallick, D., Thomas, S., Sapper, E. D., & Cole, I. S. (2016) · Materials & Design · doi:10.1016/j.matdes.2016.09.084 · Cited by 43 · see citing works
Functional regression via variational Bayes
Goldsmith, J., WAND, M. P., & Crainiceanu, C. M. (2011) · Electronic Journal of Statistics · doi:10.1214/11-ejs619 · Cited by 42 · see citing works
“Variational approximations, now regularly used in computer science, are a collection of techniques for deriving approximate solutions to inference problems [10, 11, 25].”
Background
Error-correction learning for artificial neural networks using the Bayesian paradigm. Application to automated medical diagnosis
Belciug, S., & Gorunescu, F. (2014) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2014.07.013 · Cited by 41 · see citing works
“Once the training data is presented to the NN, the posterior probabilities provide the measure that different weights are consistent with the observed data [10–12].”
Background
Gaussian variational estimation for multidimensional item response theory
Cho, A. E., Chun, W., Zhang, X., & Xu, G. (2020) · British Journal of Mathematical and Statistical Psychology · doi:10.1111/bmsp.12219 · Cited by 41 · see citing works
“Variational approximation methods are mainstream methodology in computer science and statistical learning, and they have been applied to diverse areas including speech recognition, genetic linkage analysis, and document retrieval (Blei & Jordan, 2004; Titterington, 2004).”
Method
Theory of Gaussian variational approximation for a Poisson mixed model
Hall, P., Ormerod, J. T., & WAND, M. P. (2011) · Research Online (University of Wollongong) · Cited by 40 · see citing works
Slope stability analysis of heavy-haul freight corridor using novel machine learning approach
Sabri, M. S., Ahmad, F., & Samui, P. (2023) · Modeling Earth Systems and Environment · doi:10.1007/s40808-023-01774-7 · Cited by 34 · see citing works
A deep learning‒genetic algorithm approach for aerodynamic inverse design via optimization of pressure distribution
Shirvani, A., Nili‐Ahmadabadi, M., & Ha, M. Y. (2024) · Computer Methods in Applied Mechanics and Engineering · doi:10.1016/j.cma.2024.117187 · Cited by 33 · see citing works
Neural networks
Titterington, M. (2009) · Wiley Interdisciplinary Reviews Computational Statistics · doi:10.1002/wics.50 · Cited by 33 · see citing works
“Another recent approach reviewed by Titterington, 22 called the variational method, creates deterministic approximations, to posterior distributions, that are competitors both to (stochastic) empirical approximations created by Markov chain Monte Carlo and to Laplace approximations, especially with a view to approxima…”
Influential
Streamlined mean field variational Bayes for longitudinal and multilevel data analysis
Lee, C. Y. Y., & WAND, M. P. (2016) · Biometrical Journal · doi:10.1002/bimj.201500007 · Cited by 32 · see citing works
“However, since the early 2000s there has been an increasing recognition of their usefulness in mainstream Statistics (e.g. Titterington, 2004).”
Background
SHAP-powered insights into spatiotemporal effects: Unlocking explainable Bayesian-neural-network urban flood forecasting
Chu, W. P., Zhang, C., Li, H., Zhang, L., Shen, D., & Li, R. (2024) · International Journal of Applied Earth Observation and Geoinformation · doi:10.1016/j.jag.2024.103972 · Cited by 31 · see citing works
Machine learning-accelerated aerodynamic inverse design
Shirvani, A., Nili‐Ahmadabadi, M., & Ha, M. Y. (2023) · Engineering Applications of Computational Fluid Mechanics · doi:10.1080/19942060.2023.2237611 · Cited by 29 · see citing works
“This principle indicates that a NN with a lower complexity of weights corresponds to a higher probability in the Bayesian view (Neal, 1996; Titterington, 2004) and a higher performance with more generalizability (Baum & Haussler, 1989).”
Background
Rapid mechanical evaluation of quadrangular steel plates subjected to localised blast loadings
Neto, L. B., Saleh, M., Pickerd, V., Yiannakopoulos, G., Mathys, Z., & Reid, W. D. (2019) · International Journal of Impact Engineering · doi:10.1016/j.ijimpeng.2019.103461 · Cited by 30 · see citing works
Estimation of groundwater recharge using simulation-optimization model and cascade forward ANN at East Nile Delta aquifer, Egypt
Abd-Elmaboud, M. E., Abdel-Gawad, H. A. A., El-Alfy, K. S., & Ezzeldin, M. M. (2021) · Journal of Hydrology Regional Studies · doi:10.1016/j.ejrh.2021.100784 · Cited by 30 · see citing works
Efficient prediction of wind and wave induced long-term extreme load effects of floating suspension bridges using artificial neural networks and support vector machines
Xu, Y., Fenerci, A., Øiseth, O. A., & Moan, T. (2020) · Ocean Engineering · doi:10.1016/j.oceaneng.2020.107888 · Cited by 29 · see citing works
SEMIPARAMETRIC REGRESSION AND GRAPHICAL MODELS
WAND, M. P. (2009) · Australian & New Zealand Journal of Statistics · doi:10.1111/j.1467-842x.2009.00538.x · Cited by 25 · see citing works
“In the Statistics literature, pioneering work (2002) and Wang & Titterington (2004, 2006) age in years s p i na l bone m i ne r a l den s i t y ( g / c m 2 ) 0.6 0.8 1.0 1.2 1.4 10 15 20 25 l l l l l l l l l l l l l l l ll l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l…”
BackgroundMethodInfluential
Explicit and explainable artificial intelligent model for prediction of CO2 molecular diffusion coefficient in heavy crude oils and bitumen
Alatefi, S., Agwu, O. E., & Alkouh, A. B. (2024) · Results in Engineering · doi:10.1016/j.rineng.2024.103328 · Cited by 24 · see citing works
Predicting dry matter intake in Canadian Holstein dairy cattle using milk mid-infrared reflectance spectroscopy and other commonly available predictors via artificial neural networks
Shadpour, S., Chud, T. C. S., Hailemariam, D., de Oliveira, H. R., Plastow, G. S., Stothard, P., Lassen, J., Baldwin, R. L., Miglior, F., Baes, C. F., et al. (2022) · Journal of Dairy Science · doi:10.3168/jds.2021-21297 · Cited by 23 · see citing works
Coupling self‐organizing maps with a Naïve Bayesian classifier: Stream classification studies using multiple assessment data
Fytilis, N., & Rizzo, D. M. (2013) · Water Resources Research · doi:10.1002/2012wr013422 · Cited by 23 · see citing works
Machine Learning-Aided Monte Carlo Simulation and Subset Simulation
Sabri, M. S., Ahmad, F., & Samui, P. (2024) · Transportation Research Record Journal of the Transportation Research Board · doi:10.1177/03611981241248166 · Cited by 22 · see citing works
“A BNN is a stochastic ANN that utilizes Bayesian inference in its development ( 76 – 78 ).”
Integrated Computational Alloy Design of Nickel-Base Superalloys
Montakhab, M., & Balikci, E. (2019) · Metallurgical and Materials Transactions A · doi:10.1007/s11661-019-05252-7 · Cited by 22 · see citing works
Mixture of Clustered Bayesian Neural Networks for Modeling Friction Processes at the Nanoscale
Zaidan, M. A., Canova, F. F., Laurson, L., & Foster, A. S. (2016) · Journal of Chemical Theory and Computation · doi:10.1021/acs.jctc.6b00830 · Cited by 22 · see citing works
A practical tutorial on Variational BayesPreprint
Tran, M., Nguyễn, T. N., & Dao, V. H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.01327 · Cited by 21 · see citing works
“Early work on MFVB in machine learning and statistics can be found in Waterhouse et al. (1996); Jordan et al. (1999) and Titterington (2004), and tutorial-style introductions to MFVB can be found in Bishop (2006) and Ormerod and Wand (2010).”
Background
Bayesian neural networks for macroeconomic analysis
Hauzenberger, N., Huber, F., Klieber, K., & Marcellino, M. G. (2024) · Journal of Econometrics · doi:10.1016/j.jeconom.2024.105843 · Cited by 20 · see citing works
“Empirically, we often face a situation where most ϕ ′ ℓ,ij s are large and thus many elements in W ℓ are shrunk to zero, reducing the effective dimension of the state space considerably (Titterington, 2004).”
Background
Modeling Motor Learning Using Heteroscedastic Functional Principal Components Analysis
Backenroth, D., Goldsmith, J., Harran, M. D., Cortés, J. C., Krakauer, J. W., & Kitago, T. (2017) · Journal of the American Statistical Association · doi:10.1080/01621459.2017.1379403 · Cited by 20 · see citing works
Selection in sugarcane families with artificial neural networks
Brasileiro, B. P., Marinho, C. D., Costa, P. M. D. A., Cruz, C. D., Peternelli, L. A., & Barbosa, M. H. P. (2015) · Crop Breeding and Applied Biotechnology · doi:10.1590/1984-70332015v15n2a14 · Cited by 19 · see citing works
Development of Conversational Artificial Intelligence for Pandemic Healthcare Query Support
Woo, W. L., Gao, B., Al-Nima, R. R. O., & Ling, B. W. (2020) · International Journal of Automation Artificial Intelligence and Machine Learning · doi:10.61797/ijaaiml.v1i1.35 · Cited by 19 · see citing works
“The regularization parameter can be determined using the Bayesian evidence framework or Variational Bayesian approaches [34-38].”
Method
Predicting the maturity and organic richness using artificial neural networks (ANNs): A case study of Montney Formation, NE British Columbia, Canada
Barham, A., Ismail, M. S. B., Hermana, M., Padmanabhan, E., Baashar, Y. M., & Sabir, O. (2021) · Alexandria Engineering Journal · doi:10.1016/j.aej.2021.01.036 · Cited by 18 · see citing works
The Effectiveness of the Game of Dice Task in Predicting At-Risk and Problem Gambling Among Adolescents: The Contribution of the Neural Networks
Donati, M. A., Frosini, A., Izzo, V. A., & Primi, C. (2018) · Journal of Gambling Studies · doi:10.1007/s10899-018-9796-5 · Cited by 18 · see citing works
“…interpolation that, based on the work of MacKay (1992), considers a probabilistic interpretation of the network parameters that leads to predictions that can be regarded as probability distributions (see Foresee and Hagan 1997, for a deeper insight, and Titterington 2004, for a recent survey).”
Background
Fitting Structural Equation Models via Variational Approximations
Dang, K., & Maestrini, L. (2022) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705511.2022.2053857 · Cited by 16 · see citing works
“This is a well-known issue of MFVB, which is related to the mean field restriction imposed to the approximating density (Titterington, 2004; Wang and Titterington, 2005).”
Background
A variational Bayesian approach for inverse problems with skew-t error distributions
Guha, N., Xiaoqing, W., Efendiev, Y. R., Jin, B., & Mallick, B. K. (2015) · Journal of Computational Physics · doi:10.1016/j.jcp.2015.07.062 · Cited by 16 · see citing works
“In this paper we investigate an alternative approach based on the variational method [20, 19, 24].”
Method
Gradient conjugate priors and multi-layer neural networks
Gurevich, P., & Stuke, H. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2019.103184 · Cited by 16 · see citing works
“We refer to [19, 41] and the recent works [6, 10, 22, 23, 33] for a comprehensive comparison of the above methods and further references to research on the Bayesian learning of neural networks.”
Result
Variational methods for fitting complex Bayesian mixed effects models to health data
Lee, C. Y. Y., & WAND, M. P. (2015) · Statistics in Medicine · doi:10.1002/sim.6737 · Cited by 14 · see citing works
“, [25–28]).”
Parameter probabilistic prediction for satellite power system based on unsupervised multi-dimensional sequence segmentation
Kang, S., Gao, Y., Song, Y., Zhou, R., & Pang, J. (2024) · Aerospace Science and Technology · doi:10.1016/j.ast.2024.108933 · Cited by 14 · see citing works
Incorporating modelling uncertainty and prior knowledge into landslide susceptibility mapping using Bayesian neural networks
Chen, C., Shen, Z., Fang, L., Lin, J., Li, S., & Wang, K. (2024) · Georisk Assessment and Management of Risk for Engineered Systems and Geohazards · doi:10.1080/17499518.2024.2422498 · Cited by 14 · see citing works
Predicting Flavonoid UGT Regioselectivity
Jackson, R., Knisley, D. J., McIntosh, C. A., & Pfeiffer, P. E. (2011) · Advances in Bioinformatics · doi:10.1155/2011/506583 · Cited by 13 · see citing works
“[60–62] This learning technique treats network weights as a multidimensional random variable with some prior distribution.”
Method
Quantile Regression Neural Networks: A Bayesian Approach
Jantre, S. R., Bhattacharya, S., & Maiti, T. (2021) · Journal of Statistical Theory and Practice · doi:10.1007/s42519-021-00189-w · Cited by 13 · see citing works
Gradient and uncertainty enhanced sequential sampling for global fit
Lämmle, S., Bogoclu, C., Cremanns, K., & Roos, D. (2023) · Computer Methods in Applied Mechanics and Engineering · doi:10.1016/j.cma.2023.116226 · Cited by 13 · see citing works
Active and transfer learning with partially Bayesian neural networks for materials and chemicals
Allec, S. I., & Ziatdinov, M. A. (2025) · Digital Discovery · doi:10.1039/d5dd00027k · Cited by 16 · see citing works
“Bayesian neural networks (BNNs), where all network weights are treated as probability distributions rather than scalar values [40, 41], offer a promising approach that combines powerful representation learning capabilities with reliable UQ.”
Comparison of Genomic Best Linear Unbiased Prediction and Bayesian Regularization Neural Networks for Genomic Selection
Sınecen, M. (2019) · IEEE Access · doi:10.1109/access.2019.2922006 · Cited by 12 · see citing works
The EM Algorithm, Variational Approximations and Expectation Propagation for Mixtures
Titterington, D. M. (2011) · Wiley series in probability and statistics · doi:10.1002/9781119995678.ch1 · Cited by 12 · see citing works
“Wang and Titterington (2004b) considered the more general context of data from exponential family models with missing values, establishing local convergence of the iterative algorithm for calculating the variational approximation and proving asymptotic normality of the estimator.”
BackgroundInfluential
Gaussian Scale Mixture Models for Robust Linear Multivariate Regression with Missing Data
Ala-Luhtala, J., & Piché, R. (2014) · Communications in Statistics - Simulation and Computation · doi:10.1080/03610918.2013.875565 · Cited by 10 · see citing works
“Titterington (2004) provides more discussion about the VB-EM algorithm, especially in the neural networks point of view.”
Background
Learning neural impulsive MPC for tailoring therapies in viral infections
Magdaleno, G. D. V., Alanís, A. Y., & Hernandez-Vargas, E. A. (2019) · Applied Soft Computing · doi:10.1016/j.asoc.2019.105767 · Cited by 7 · see citing works
An efficient variational Bayesian algorithm for calibrating fundamental diagrams and its probabilistic sensitivity analysis
Jin, X., Ma, W. F., Zhong, R., & Jiang, G. (2023) · Transportmetrica B Transport Dynamics · doi:10.1080/21680566.2023.2231159 · Cited by 7 · see citing works
“…Inference (VBI), provides an efficient method to approximate the posterior by using a simpler distribution (Bishop and Nasrabadi 2006; Blei, Kucukelbir, and McAuliffe 2017; Guo et al. 2016; Jordan et al. 1999; Titterington 2004; Tran, Nguyen, and Dao 2021; Waterhouse, MacKay, and Robinson 1995).”
Method
Simultaneous clustering and feature selection via nonparametric Pitman–Yor process mixture models
Fan, W., & Bouguila, N. (2019) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-018-0900-z · Cited by 7 · see citing works
“Bayesian inference is often approximated via variational learning which the main goal is to find and evaluate variational lower bounds, generally via Jensen’s inequality, instead of these expressions [44].”
Background
Approximate blocked Gibbs sampling for Bayesian neural networks
Papamarkou, T. (2023) · Statistics and Computing · doi:10.1007/s11222-023-10285-5 · Cited by 6 · see citing works
“Several other reviews of the topic exist, see for instance Titterington (2004); Wenzel et al. (2020); Izmailov et al. (2021); Papamarkou et al. (2022).”
Background
Variational Estimation for Multidimensional Generalized Partial Credit Model
Cui, C., Wang, C., & Xu, G. (2024) · Psychometrika · doi:10.1007/s11336-024-09955-8 · Cited by 6 · see citing works
“The variational method, ﬁrst proposed in computer science and statistical learning, has since become an efﬁcient approach for large-scale computation in ﬁelds such as pattern recognition and document retrieval (Titterington, 2004; Blei & Jordan, 2006).”
RVD2: an ultra-sensitive variant detection model for low-depth heterogeneous next-generation sequencing data
He, Y., Zhang, F., & Flaherty, P. (2015) · Bioinformatics · doi:10.1093/bioinformatics/btv275 · Cited by 6 · see citing works
“Variational methods can be generalized as the process of minimizing the Kullback-Leibler (KL) divergence between exact posterior distributions and the approximate distributions, which are generally obtained by decoupling distributions from a graphical model [69, 70].”
Isoprenylcysteine carboxyl methyltransferase inhibitors: QSAR, docking and molecular dynamics studies
Das, N. R., Sharma, T., Goyal, N., Singh, N., Toropov, A. A., Toropova, A. P., & Achary, P. G. R. (2023) · Journal of Molecular Structure · doi:10.1016/j.molstruc.2023.135966 · Cited by 6 · see citing works
The missing radial velocities of Gaia: a catalogue of Bayesian estimates for DR3
Naik, A. P., & Widmark, A. (2023) · Monthly Notices of the Royal Astronomical Society · doi:10.1093/mnras/stad3822 · Cited by 6 · see citing works
Gaussian mixture models for training Bayesian convolutional neural networks
Bakhouya, M., Ramchoun, H., Mohammed, H., & Masrour, T. (2024) · Evolutionary Intelligence · doi:10.1007/s12065-023-00900-9 · Cited by 6 · see citing works
“To overcome this problem, we can infer this distribution using statistical estimates known as "approximate inferences" [19, 27, 27, 48, 60, 62].”
The missing radial velocities of Gaia: Blind predictions for DR3
Naik, A. P., & Widmark, A. (2022) · Monthly Notices of the Royal Astronomical Society · doi:10.1093/mnras/stac2425 · Cited by 6 · see citing works
Efficient Global Optimization Algorithm Using Neural Network-based Prediction and Uncertainty
Koratikere, P., Leifsson, L., Barnet, L., & Bryden, K. M. (2023) · AIAA SciTech Forum · doi:10.2514/6.2023-2683 · Cited by 7 · see citing works
Bayesian Inference ForgettingPreprint
Fu, S., He, F., Xu, Y., & Tao, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.06417 · Cited by 6 · see citing works
“Bayesian inference enables a wide spectrum of machine learning algorithms, such as probabilistic matrix factorization [46, 62, 72], topic models [30, 8, 74], probabilistic graphic models [39, 6, 71] and Bayesian neural networks [10, 37, 52, 66, 58, 61, 75].”
Background
A comprehensive guide to selecting the right modeling strategy for explanatory and predictive data analysis
Niazy, M., Murphy, H. M., Nadeem, K., & Ricker, N. (2025) · Canadian Journal of Microbiology · doi:10.1139/cjm-2025-0038 · Cited by 5 · see citing works
Practical Guide to Estimation and Uncertainty Quantification of Aerodynamic Flows
Eldredge, J. D., & Mousavi, H. (2026) · AIAA Journal · doi:10.2514/1.j066257 · Cited by 8 · see citing works
Multiscale Bayesian state-space model for Granger causality analysis of brain signal
Cekic, S., Grandjean, D., & Renaud, O. (2018) · Journal of Applied Statistics · doi:10.1080/02664763.2018.1455814 · Cited by 5 · see citing works
“This approximation methodology has been extensively explored and used during the past years; see for example Titterington [40], Beal [4], Ormerod and Wand [33] and Fox and Roberts [13].”
BackgroundMethod
Spectral enhancement of Landsat OLI images by using Hyperion data: a comparison between multilayer perceptron and radial basis function networks
Mokhtari, M. H., Deilami, K., & Moosavi, V. (2020) · Earth Science Informatics · doi:10.1007/s12145-020-00451-y · Cited by 5 · see citing works
“The BR is also a method that automatically adjusts ideal values for the parameters of the objective function (Burden and Winkler 2008; Marwala 2007; Okut et al. 2011; Titterington 2004).”
Method
Joint segmentation and image reconstruction with error prediction in photoacoustic imaging using deep learning
Shang, R., Luke, G. P., & O’Donnell, M. (2024) · Photoacoustics · doi:10.1016/j.pacs.2024.100645 · Cited by 4 · see citing works
Constrained Aerodynamic Shape Optimization Using Neural Networks and Sequential Sampling
Koratikere, P., Leifsson, L., Koziel, S. M., & Pietrenko‐Dabrowska, A. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-36024-4_33 · Cited by 4 · see citing works
High Resolution Multi-Spectral Imagery and Learning Machines in Precision Irrigation Water Management
Hassan-Esfahani, L. (2021) · Digital Commons - USU (Utah State University) · doi:10.26076/d3c7-f268 · Cited by 4 · see citing works
Bayesian neural networks for the probabilistic forecasting of wind direction and speed using ocean data
Clare, M., & Piggott, M. (2022) · CRC Press eBooks · doi:10.1201/9781003360773-61 · Cited by 4 · see citing works
Variational bayesian method of estimating variance components
Arakawa, A., Taniguchi, M., Hayashi, T., & Mikawa, S. (2016) · Animal Science Journal · doi:10.1111/asj.12514 · Cited by 4 · see citing works
fMRI activation detection with EEG priors
Kalus, S., Sämann, P. G., Czisch, M., & Fahrmeir, L. (2013) · Open access LMU (Ludwid Maxmilian's Universitat Munchen) · doi:10.5282/ubm/epub.15725 · Cited by 4 · see citing works
“Another popular approach in neuroimaging is a variational Bayes approach to Bayesian inference (see e.g. Titterington, 2004).”
A Statistical Theory of Deep Learning via Proximal SplittingPreprint
Polson, N. G., Willard, B. T., & Heidari, M. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1509.06061 · Cited by 4 · see citing works
“Our methods can therefore be seen as related to sparse Bayes MAP techniques; see Titterington [2004], Windle et al. [2013], Polson et al. [2015] for high dimensional data reduction.”
Method

Authors: Titterington, D. M.
Corresponding author: Titterington, D. M.
Venue: Statistical Science
Publisher: Institute of Mathematical Statistics
Published: 2004-02-01
Volume: 19
Issue: 1
Pages: 128-139
ISSN: 0883-4237 (ISSN-L)
2168-8745
DOI: 10.1214/088342304000000099
OpenAlex: W2019449965
Cited by: 162
References: 103
Citations by year: 2026: 2 · 2025: 9 · 2024: 20 · 2023: 16 · 2022: 17 · 2021: 9 · 2020: 9 · 2019: 7 · 2018: 6 · 2017: 7 · 2016: 10 · 2015: 9 · 2014: 8 · 2013: 3 · 2012: 1
FWCI: 3.05
Type: journal article
Language: en
Open access: closed
Locations: Statistical Science (publishedVersion)
Project Euclid (Cornell University) (submittedVersion)
Topics: Physical Sciences › Mathematics › Statistics and Probability › Markov Chains and Monte Carlo Methods
Physical Sciences › Computer Science › Artificial Intelligence › Gaussian Processes and Bayesian Inference
Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
Keywords: Bayesian neural networks, feedforward neural networks, Gaussian approximation, Markov chain Monte Carlo, variational approximation, computational intractability

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Bayesian Methods for Neural Networks and Related Models

Abstract

Integrity signals

Claims the paper makes

Claims other papers make about this one

Cites

Cited by

Related

Record

About these checks