The DOI matches its Crossref record; 102 of the 103 references it cites were found in a registry or an index.
Bayesian Methods for Neural Networks and Related Models
Abstract
Models such as feed-forward neural networks and certain other structures investigated in the computer science literature are not amenable to closed-form Bayesian analysis. The paper reviews the various approaches taken to overcome this difficulty, involving the use of Gaussian approximations, Markov chain Monte Carlo simulation routines and a class of non-Gaussian but “deterministic” approximations called variational approximations.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Bayesian Neural Networks (BNNs) [68,71,72] are stochastic ANNs trained using Bayesian inference.”
From Explainable AI in medical imaging: An overview for clinical practitioners – Beyond saliency-based XAI approaches (2023) · cites this paper as [72]
Citations
Cites
The Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970
Neural networks for pattern recognition
Bishop, C. (1994) · Choice Reviews Online · doi:10.5860/choice.31-5500 · Cited by 18,537
Statistical Decision Theory and Bayesian Analysis
Berger, J. O. (1985) · Springer Series in Statistics · doi:10.1007/978-1-4757-4286-2 · Cited by 7,722
Pattern Recognition and Neural Networks
Ripley, B. D. (1996) · Cambridge University Press eBooks · doi:10.1017/cbo9780511812651 · Cited by 6,500
Reversible Jump Markov Chain Monte Carlo Computation and Bayesian Model Determination
Green, P. J. (1995) · Biometrika · doi:10.1093/biomet/82.4.711 · Cited by 5,941
Mackay, D. (1992) · Neural Computation · doi:10.1162/neco.1992.4.3.415 · Cited by 4,440
Bayesian Learning for Neural Networks
Neal, R. M. (1996) · Lecture Notes in Statistics · doi:10.1007/978-1-4612-0745-0 · Cited by 4,276
Duane, S., Kennedy, A. D., Pendleton, B. J., & Roweth, D. (1987) · Physics Letters B · doi:10.1016/0370-2693(87)91197-x · Cited by 3,920
An Introduction to Variational Methods for Graphical Models
Jordan, M. I., Ghahramani, Z., Jaakkola, T. S., & Saul, L. K. (1999) · Machine Learning · doi:10.1023/a:1007665907178 · Cited by 3,791
A practical Bayesian framework for backpropagation networks
MacKay, D. J. C. (1992) · Neural Computation · doi:10.1162/neco.1992.4.3.448 · Cited by 2,917
Research works printed but not linked: 3
- Choudrey, R. and Roberts, S. J. (2001). Variational Bayesian independent component analysis with flexible sources. Technical Report PARG-01-03, Dept. Engineering Science, Univ. Oxford.
- MacKay, D. J. C. (1997). Ensemble learning for hidden Markov models. Technical report, Cavendish Lab., Univ. Cambridge.
- Ripley, B. D. (1995). Choosing network complexity. In Probabilistic Reasoning and Bayesian Belief Networks (A. Gammerman, ed.) 97--108. Waller, Henley-on-Thames, UK.
Cited by
Sorted by citations
Rue, H., Martino, S., & Chopin, N. (2009) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2008.00700.x · Cited by 5,279
Bioinformatics and Computational Biology Solutions Using R and Bioconductor
Gentleman, R. C., Carey, V. J., Huber, W. K., Irizarry, R. A., & Dudoit, S. (2005) · Statistics in the health sciences · doi:10.1007/0-387-29362-0 · Cited by 3,470
“A review of Bayesian inference methods with neural network models is given in Titterington (2004).”
MethodHands-On Bayesian Neural Networks—A Tutorial for Deep Learning Users
Jospin, L. V., Laga, H., Boussaid, F., Buntine, W., & Bennamoun, M. (2022) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2022.3155327 · Cited by 879
“Bayesian neural networks (BNNs) [8, 9, 10] are stochastic neural networks trained using a Bayesian approach.”
MethodConfronting the Challenge of Modeling Cloud and Precipitation Microphysics
Morrison, H., van Lier‐Walqui, M., Fridlind, A. M., Grabowski, W. W., Harrington, J. Y., Hoose, C., Korolev, A. V., Kumjian, M. R., Milbrandt, J. A., Pawłowska, H., et al. (2020) · Journal of Advances in Modeling Earth Systems · doi:10.1029/2019ms001689 · Cited by 672
Jordan, M. I. (2004) · Statistical Science · doi:10.1214/088342304000000026 · Cited by 440
“…were formulated in terms of the minimization of a Kullback–Leibler (KL) divergence , and the space over which the optimization was performed was a set of " simplified " probability distributions , generally obtained by removing edges from a graphical model (see Titterington, 2004, this volume ).”
Statistical challenges of high-dimensional data
Johnstone, I. M., & Titterington, D. M. (2009) · Philosophical Transactions of the Royal Society A Mathematical Physical and Engineering Sciences · doi:10.1098/rsta.2009.0159 · Cited by 420
“Its recent application to many implementations of the Bayesian approach has mainly developed in machine learning, see Bishop (2006) and references therein, rather than in the mainstream statistics literature, although not exclusively so (Titterington 2004; Beal & Ghahramani 2006).”
BackgroundKayri, M. (2016) · Mathematical and Computational Applications · doi:10.3390/mca21020020 · Cited by 405
“In this case, a gradient-based optimization algorithm is preferred for minimizing the objective [15,18,23,24]; In Equation (5), w 2 j , the sum of squares of network weights, α and β , are hyperparameters that need to be estimated function parameters.”
BackgroundExplaining Variational Approximations
Ormerod, J. T., & WAND, M. P. (2010) · The American Statistician · doi:10.1198/tast.2010.09058 · Cited by 403
“The expositions by Jordan (2004) and Titterington (2004) provide access to some of the literature on variational approximation accuracy.”
BackgroundInfluentialGianola, D., Okut, H., WEIGEL, K. A., & Rosa, G. J. M. (2011) · BMC Genetics · doi:10.1186/1471-2156-12-87 · Cited by 290
“In this section, an approach used often for Bayesian regularization in neural networks [18,19] is presented along the lines of the hierarchical models employed by quantitative geneticists [15].”
MethodUtama, R., Piekarewicz, J., & Prosper, H. B. (2016) · Physical Review C · doi:10.1103/physrevc.93.014311 · Cited by 238
Borys, K., Schmitt, Y. A., Nauta, M., Seifert, C., Krämer, N., Friedrich, C. M., & Nensa, F. (2023) · European Journal of Radiology · doi:10.1016/j.ejrad.2023.110786 · Cited by 233
Howard, R., Carriquiry, A. L., & Beavis, W. D. (2014) · G3 Genes Genomes Genetics · doi:10.1534/g3.114.010298 · Cited by 226
A review of predictive uncertainty estimation with machine learning
Tyralis, H., & Papacharalampous, G. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-023-10698-8 · Cited by 220
Variational approximations in Bayesian model selection for finite mixture distributions
McGrory, C. A., & Titterington, D. M. (2006) · Computational Statistics & Data Analysis · doi:10.1016/j.csda.2006.07.020 · Cited by 160
Applied Bayesian Modeling And Causal Inference From Incomplete-Data Perspectives
Gelman, A., & Xiaoli, M. (2014) · Cited by 152
Nuclear charge radii: density functional theory meets Bayesian neural networks
Utama, R., Chen, W.-C., & Piekarewicz, J. (2016) · Journal of Physics G Nuclear and Particle Physics · doi:10.1088/0954-3899/43/11/114002 · Cited by 148
Bayesian regularized artificial neural networks for the estimation of the probability of default
Sariev, E., & Germano, G. (2019) · Quantitative Finance · doi:10.1080/14697688.2019.1633014 · Cited by 135
“Titterington (2004) reviewed the various approaches taken to determine the network architecture, involving the use of Gaussian approximations and of non-Gaussian but deterministic approximations called variational approximations.”
BackgroundTitterington, D. M., & Wang, B. Y. (2006) · Bayesian Analysis · doi:10.1214/06-ba121 · Cited by 118
“Keywords: Mixture model, Variational Bayes, Local convergence, Laplace approximation”
BackgroundMethodInfluentialBayesian Regularized Neural Networks for Small n Big p Data
Okut, H. (2016) · InTech eBooks · doi:10.5772/63256 · Cited by 109
“An alternative approach [12, 14] involves iteration (updating is from right to left, with wMAP evaluated at the “old” values of the tuning parameters)”
MethodBayesian training of artificial neural networks used for water resources modeling
Kingston, G. B., Lambert, M. F., & Maier, H. R. (2005) · Water Resources Research · doi:10.1029/2005wr004152 · Cited by 103
“These approaches have since been reviewed by MacKay [1995], Bishop [1995], Neal [1996a], Lampinen and Vehtari [2001], and Titterington [2004].”
MethodGaussian Variational Approximate Inference for Generalized Linear Mixed Models
Ormerod, J. T., & WAND, M. P. (2011) · Journal of Computational and Graphical Statistics · doi:10.1198/jcgs.2011.09118 · Cited by 101
“The lower-bound expression (3.1) can also be derived using the ideas of Kullback– Leibler divergence, which underpins much of variational approximation methodology (e.g., Titterington 2004; Bishop 2006, chap.”
MethodInfluentialHippert, H. S., & Taylor, J. W. (2009) · Neural Networks · doi:10.1016/j.neunet.2009.11.016 · Cited by 98
Prediction of body mass index in mice using dense molecular markers and a regularized neural network
Okut, H., Daniel Gianola, B., Rosa, G. J. M., & WEIGEL, K. A. (2011) · Genetics Research · doi:10.1017/s0016672310000662 · Cited by 85
“Alternatively,BRgaugesanobjective functionconsistingofaresidualsumofsquaresplus thesumofsquaredweights;thefunctionisminimized with respect to the weights, and the aim is to produce a network that generalizes well (Bishop & Tipping, 1998; Titterington, 2004; Marwala, 2007; Ripley, 2007).”
A Review of Parameter Learning Methods in Bayesian Network
Ji, Z., Xia, Q., & Meng, G. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-22053-6_1 · Cited by 81
Lessons in uncertainty quantification for turbulent dynamical systems
Majda, A. J., & Branicki, M. (2012) · Discrete and Continuous Dynamical Systems · doi:10.3934/dcds.2012.32.3133 · Cited by 77
“…(UQ) is undoubtedly an important issue in many physics, engineering, neural science and geoscience applications where complex nonlinear interactions between the resolved and unresolved processes, as well as various parametric uncertainties, need to be properly treated in order to reliably…”
BackgroundOkut, H., Wu, X.-L., Rosa, G. J. M., Bauck, S. W., Woodward, B. W., Schnabel, R. D., Taylor, J. F., & Daniel Gianola, B. (2013) · Genetics Selection Evolution · doi:10.1186/1297-9686-45-34 · Cited by 71
“Large values of α lead to posterior densities of weights that are highly concentrated around zero, so that the weights effectively disappear, the model discounts connections in the network [24,25], and complex models are automatically self-penalized.”
BackgroundDeep Learning in Diverse Intelligent Sensor Based Systems
Zhu, Y., Wang, M., Yin, X., Zhang, J., Meijering, E. H. W., & Hu, J. (2022) · Sensors · doi:10.3390/s23010062 · Cited by 71
Goldsmith, J., & Kitago, T. (2015) · Journal of the Royal Statistical Society Series C (Applied Statistics) · doi:10.1111/rssc.12115 · Cited by 65
“Variational Bayes methods are regularly used in the computer science literature, and to a more limited extent in the statistics literature, to provide approximate solutions to intractable inference problems (Jordan, 2004; Jordan et al., 1999; Titterington, 2004; Ormerod and Wand, 2012).”
Computational Advances for and from Bayesian Analysis
Andrieu, C., Arnaud Doucet, B., & Robert, C. P. (2004) · Statistical Science · doi:10.1214/088342304000000071 · Cited by 62
“Some are mentioned in other sections (sequential Monte Carlo methods, Section 4) or in other papers in this volume (like variational methods, Jordan, 2004; Titterington, 2004).”
MethodA methodology for data gap filling in wave records using Artificial Neural Networks
Vieira, F., Cavalcante, G. H., Campos, E. J. D., & Pinto, F. T. (2020) · Applied Ocean Research · doi:10.1016/j.apor.2020.102109 · Cited by 58
Challenges in Markov Chain Monte Carlo for Bayesian Neural Networks
Papamarkou, T., Hinkle, J. D., Young, M. T., & Womble, D. E. (2022) · Statistical Science · doi:10.1214/21-sts840 · Cited by 82
“For a review of Bayesian approaches to neural networks, see Titterington (2004).”
BackgroundClare, M., Sonnewald, M., Lguensat, R., Deshayes, J., & Balaji, V. (2022) · Journal of Advances in Modeling Earth Systems · doi:10.1029/2022ms003162 · Cited by 63
“A number of methods have been proposed to approximate the denominator term including Markov Chain Monte Carlo sampling (Titterington, 2004) and variational inference (Osawa et al., 2019).”
Backpropagation Neural Network-Based Machine Learning Model for Prediction of Soil Friction Angle
Nguyen, T., Ly, H., & Pham, B. T. (2020) · Mathematical Problems in Engineering · doi:10.1155/2020/8845768 · Cited by 53
“For the purpose of improving the generalization of the model, the gradient-based optimization algorithm is preferred to minimize the target [48, 49].”
MethodHassan-Esfahani, L., Torres‐Rua, A. F., Jensen, A. M., & McKee, M. (2017) · Irrigation and Drainage · doi:10.1002/ird.2098 · Cited by 49
Reliability and Interpretability in Science and Deep Learning
Scorzato, L. (2024) · Minds and Machines · doi:10.1007/s11023-024-09682-0 · Cited by 51
“…cost, which leads to a very active search for approximations that offer the best trade-off between accuracy and computational efficiency (Titterington, 2004; Jospin et al., 2020; MacKay, 1992; Gal and Ghahramani, 2016; Blundell et al., 2015; Sensoy et al., 2018; Hartmann and Richter,…”
Shadpour, S., Chud, T. C. S., Hailemariam, D., Plastow, G. S., de Oliveira, H. R., Stothard, P., Lassen, J., Miglior, F., Baes, C. F., Tulpan, D., et al. (2022) · Journal of Dairy Science · doi:10.3168/jds.2021-21176 · Cited by 43
Chen, F. F., Breedon, M., White, P., Chu, C., Mallick, D., Thomas, S., Sapper, E. D., & Cole, I. S. (2016) · Materials & Design · doi:10.1016/j.matdes.2016.09.084 · Cited by 43
Functional regression via variational Bayes
Goldsmith, J., WAND, M. P., & Crainiceanu, C. M. (2011) · Electronic Journal of Statistics · doi:10.1214/11-ejs619 · Cited by 42
“Variational approximations, now regularly used in computer science, are a collection of techniques for deriving approximate solutions to inference problems [10, 11, 25].”
BackgroundBelciug, S., & Gorunescu, F. (2014) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2014.07.013 · Cited by 41
“Once the training data is presented to the NN, the posterior probabilities provide the measure that different weights are consistent with the observed data [10–12].”
BackgroundGaussian variational estimation for multidimensional item response theory
Cho, A. E., Chun, W., Zhang, X., & Xu, G. (2020) · British Journal of Mathematical and Statistical Psychology · doi:10.1111/bmsp.12219 · Cited by 41
“Variational approximation methods are mainstream methodology in computer science and statistical learning, and they have been applied to diverse areas including speech recognition, genetic linkage analysis, and document retrieval (Blei & Jordan, 2004; Titterington, 2004).”
MethodTheory of Gaussian variational approximation for a Poisson mixed model
Hall, P., Ormerod, J. T., & WAND, M. P. (2011) · Research Online (University of Wollongong) · Cited by 40
Slope stability analysis of heavy-haul freight corridor using novel machine learning approach
Sabri, M. S., Ahmad, F., & Samui, P. (2023) · Modeling Earth Systems and Environment · doi:10.1007/s40808-023-01774-7 · Cited by 34
Shirvani, A., Nili‐Ahmadabadi, M., & Ha, M. Y. (2024) · Computer Methods in Applied Mechanics and Engineering · doi:10.1016/j.cma.2024.117187 · Cited by 33
Titterington, M. (2009) · Wiley Interdisciplinary Reviews Computational Statistics · doi:10.1002/wics.50 · Cited by 37
“Another recent approach reviewed by Titterington, 22 called the variational method, creates deterministic approximations, to posterior distributions, that are competitors both to (stochastic) empirical approximations created by Markov chain Monte Carlo and to Laplace approximations, especially with a view to approxima…”
InfluentialStreamlined mean field variational Bayes for longitudinal and multilevel data analysis
Lee, C. Y. Y., & WAND, M. P. (2016) · Biometrical Journal · doi:10.1002/bimj.201500007 · Cited by 32
“However, since the early 2000s there has been an increasing recognition of their usefulness in mainstream Statistics (e.g. Titterington, 2004).”
BackgroundChu, W. P., Zhang, C., Li, H., Zhang, L., Shen, D., & Li, R. (2024) · International Journal of Applied Earth Observation and Geoinformation · doi:10.1016/j.jag.2024.103972 · Cited by 31
Machine learning-accelerated aerodynamic inverse design
Shirvani, A., Nili‐Ahmadabadi, M., & Ha, M. Y. (2023) · Engineering Applications of Computational Fluid Mechanics · doi:10.1080/19942060.2023.2237611 · Cited by 30
“This principle indicates that a NN with a lower complexity of weights corresponds to a higher probability in the Bayesian view (Neal, 1996; Titterington, 2004) and a higher performance with more generalizability (Baum & Haussler, 1989).”
BackgroundRapid mechanical evaluation of quadrangular steel plates subjected to localised blast loadings
Neto, L. B., Saleh, M., Pickerd, V., Yiannakopoulos, G., Mathys, Z., & Reid, W. D. (2019) · International Journal of Impact Engineering · doi:10.1016/j.ijimpeng.2019.103461 · Cited by 30
Abd-Elmaboud, M. E., Abdel-Gawad, H. A. A., El-Alfy, K. S., & Ezzeldin, M. M. (2021) · Journal of Hydrology Regional Studies · doi:10.1016/j.ejrh.2021.100784 · Cited by 30
Xu, Y., Fenerci, A., Øiseth, O. A., & Moan, T. (2020) · Ocean Engineering · doi:10.1016/j.oceaneng.2020.107888 · Cited by 29
SEMIPARAMETRIC REGRESSION AND GRAPHICAL MODELS
WAND, M. P. (2009) · Australian & New Zealand Journal of Statistics · doi:10.1111/j.1467-842x.2009.00538.x · Cited by 25
“In the Statistics literature, pioneering work (2002) and Wang & Titterington (2004, 2006) age in years s p i na l bone m i ne r a l den s i t y ( g / c m 2 ) 0.6 0.8 1.0 1.2 1.4 10 15 20 25 l l l l l l l l l l l l l l l ll l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l l…”
BackgroundMethodInfluentialAlatefi, S., Agwu, O. E., & Alkouh, A. B. (2024) · Results in Engineering · doi:10.1016/j.rineng.2024.103328 · Cited by 23
Shadpour, S., Chud, T. C. S., Hailemariam, D., de Oliveira, H. R., Plastow, G. S., Stothard, P., Lassen, J., Baldwin, R. L., Miglior, F., Baes, C. F., et al. (2022) · Journal of Dairy Science · doi:10.3168/jds.2021-21297 · Cited by 23
Fytilis, N., & Rizzo, D. M. (2013) · Water Resources Research · doi:10.1002/2012wr013422 · Cited by 23
Machine Learning-Aided Monte Carlo Simulation and Subset Simulation
Sabri, M. S., Ahmad, F., & Samui, P. (2024) · Transportation Research Record Journal of the Transportation Research Board · doi:10.1177/03611981241248166 · Cited by 24
“A BNN is a stochastic ANN that utilizes Bayesian inference in its development ( 76 – 78 ).”
Integrated Computational Alloy Design of Nickel-Base Superalloys
Montakhab, M., & Balikci, E. (2019) · Metallurgical and Materials Transactions A · doi:10.1007/s11661-019-05252-7 · Cited by 22
Mixture of Clustered Bayesian Neural Networks for Modeling Friction Processes at the Nanoscale
Zaidan, M. A., Canova, F. F., Laurson, L., & Foster, A. S. (2016) · Journal of Chemical Theory and Computation · doi:10.1021/acs.jctc.6b00830 · Cited by 22
A practical tutorial on Variational BayesPreprint
Tran, M., Nguyễn, T. N., & Dao, V. H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.01327 · Cited by 21
“Early work on MFVB in machine learning and statistics can be found in Waterhouse et al. (1996); Jordan et al. (1999) and Titterington (2004), and tutorial-style introductions to MFVB can be found in Bishop (2006) and Ormerod and Wand (2010).”
BackgroundBayesian neural networks for macroeconomic analysis
Hauzenberger, N., Huber, F., Klieber, K., & Marcellino, M. G. (2024) · Journal of Econometrics · doi:10.1016/j.jeconom.2024.105843 · Cited by 24
“Empirically, we often face a situation where most ϕ ′ ℓ,ij s are large and thus many elements in W ℓ are shrunk to zero, reducing the effective dimension of the state space considerably (Titterington, 2004).”
BackgroundModeling Motor Learning Using Heteroscedastic Functional Principal Components Analysis
Backenroth, D., Goldsmith, J., Harran, M. D., Cortés, J. C., Krakauer, J. W., & Kitago, T. (2017) · Journal of the American Statistical Association · doi:10.1080/01621459.2017.1379403 · Cited by 20
Selection in sugarcane families with artificial neural networks
Brasileiro, B. P., Marinho, C. D., Costa, P. M. D. A., Cruz, C. D., Peternelli, L. A., & Barbosa, M. H. P. (2015) · Crop Breeding and Applied Biotechnology · doi:10.1590/1984-70332015v15n2a14 · Cited by 19
Development of Conversational Artificial Intelligence for Pandemic Healthcare Query Support
Woo, W. L., Gao, B., Al-Nima, R. R. O., & Ling, B. W. (2020) · International Journal of Automation Artificial Intelligence and Machine Learning · doi:10.61797/ijaaiml.v1i1.35 · Cited by 20
“The regularization parameter can be determined using the Bayesian evidence framework or Variational Bayesian approaches [34-38].”
MethodBarham, A., Ismail, M. S. B., Hermana, M., Padmanabhan, E., Baashar, Y. M., & Sabir, O. (2021) · Alexandria Engineering Journal · doi:10.1016/j.aej.2021.01.036 · Cited by 18
Donati, M. A., Frosini, A., Izzo, V. A., & Primi, C. (2018) · Journal of Gambling Studies · doi:10.1007/s10899-018-9796-5 · Cited by 18
“…interpolation that, based on the work of MacKay (1992), considers a probabilistic interpretation of the network parameters that leads to predictions that can be regarded as probability distributions (see Foresee and Hagan 1997, for a deeper insight, and Titterington 2004, for a recent survey).”
BackgroundFitting Structural Equation Models via Variational Approximations
Dang, K., & Maestrini, L. (2022) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705511.2022.2053857 · Cited by 16
“This is a well-known issue of MFVB, which is related to the mean field restriction imposed to the approximating density (Titterington, 2004; Wang and Titterington, 2005).”
BackgroundA variational Bayesian approach for inverse problems with skew-t error distributions
Guha, N., Xiaoqing, W., Efendiev, Y. R., Jin, B., & Mallick, B. K. (2015) · Journal of Computational Physics · doi:10.1016/j.jcp.2015.07.062 · Cited by 16
“In this paper we investigate an alternative approach based on the variational method [20, 19, 24].”
MethodGradient conjugate priors and multi-layer neural networks
Gurevich, P., & Stuke, H. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2019.103184 · Cited by 17
“We refer to [19, 41] and the recent works [6, 10, 22, 23, 33] for a comprehensive comparison of the above methods and further references to research on the Bayesian learning of neural networks.”
ResultVariational methods for fitting complex Bayesian mixed effects models to health data
Lee, C. Y. Y., & WAND, M. P. (2015) · Statistics in Medicine · doi:10.1002/sim.6737 · Cited by 14
“, [25–28]).”
Kang, S., Gao, Y., Song, Y., Zhou, R., & Pang, J. (2024) · Aerospace Science and Technology · doi:10.1016/j.ast.2024.108933 · Cited by 14
Chen, C., Shen, Z., Fang, L., Lin, J., Li, S., & Wang, K. (2024) · Georisk Assessment and Management of Risk for Engineered Systems and Geohazards · doi:10.1080/17499518.2024.2422498 · Cited by 14
Predicting Flavonoid UGT Regioselectivity
Jackson, R., Knisley, D. J., McIntosh, C. A., & Pfeiffer, P. E. (2011) · Advances in Bioinformatics · doi:10.1155/2011/506583 · Cited by 13
“[60–62] This learning technique treats network weights as a multidimensional random variable with some prior distribution.”
MethodQuantile Regression Neural Networks: A Bayesian Approach
Jantre, S. R., Bhattacharya, S., & Maiti, T. (2021) · Journal of Statistical Theory and Practice · doi:10.1007/s42519-021-00189-w · Cited by 13
Gradient and uncertainty enhanced sequential sampling for global fit
Lämmle, S., Bogoclu, C., Cremanns, K., & Roos, D. (2023) · Computer Methods in Applied Mechanics and Engineering · doi:10.1016/j.cma.2023.116226 · Cited by 13
Active and transfer learning with partially Bayesian neural networks for materials and chemicals
Allec, S. I., & Ziatdinov, M. A. (2025) · Digital Discovery · doi:10.1039/d5dd00027k · Cited by 16
“Bayesian neural networks (BNNs), where all network weights are treated as probability distributions rather than scalar values [40, 41], offer a promising approach that combines powerful representation learning capabilities with reliable UQ.”
Sınecen, M. (2019) · IEEE Access · doi:10.1109/access.2019.2922006 · Cited by 12
The EM Algorithm, Variational Approximations and Expectation Propagation for Mixtures
Titterington, D. M. (2011) · Wiley series in probability and statistics · doi:10.1002/9781119995678.ch1 · Cited by 12
“Wang and Titterington (2004b) considered the more general context of data from exponential family models with missing values, establishing local convergence of the iterative algorithm for calculating the variational approximation and proving asymptotic normality of the estimator.”
BackgroundInfluentialGaussian Scale Mixture Models for Robust Linear Multivariate Regression with Missing Data
Ala-Luhtala, J., & Piché, R. (2014) · Communications in Statistics - Simulation and Computation · doi:10.1080/03610918.2013.875565 · Cited by 10
“Titterington (2004) provides more discussion about the VB-EM algorithm, especially in the neural networks point of view.”
BackgroundLearning neural impulsive MPC for tailoring therapies in viral infections
Magdaleno, G. D. V., Alanís, A. Y., & Hernandez-Vargas, E. A. (2019) · Applied Soft Computing · doi:10.1016/j.asoc.2019.105767 · Cited by 7
Jin, X., Ma, W. F., Zhong, R., & Jiang, G. (2023) · Transportmetrica B Transport Dynamics · doi:10.1080/21680566.2023.2231159 · Cited by 7
“…Inference (VBI), provides an efficient method to approximate the posterior by using a simpler distribution (Bishop and Nasrabadi 2006; Blei, Kucukelbir, and McAuliffe 2017; Guo et al. 2016; Jordan et al. 1999; Titterington 2004; Tran, Nguyen, and Dao 2021; Waterhouse, MacKay, and Robinson 1995).”
MethodSimultaneous clustering and feature selection via nonparametric Pitman–Yor process mixture models
Fan, W., & Bouguila, N. (2019) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-018-0900-z · Cited by 7
“Bayesian inference is often approximated via variational learning which the main goal is to find and evaluate variational lower bounds, generally via Jensen’s inequality, instead of these expressions [44].”
BackgroundApproximate blocked Gibbs sampling for Bayesian neural networks
Papamarkou, T. (2023) · Statistics and Computing · doi:10.1007/s11222-023-10285-5 · Cited by 7
“Several other reviews of the topic exist, see for instance Titterington (2004); Wenzel et al. (2020); Izmailov et al. (2021); Papamarkou et al. (2022).”
BackgroundVariational Estimation for Multidimensional Generalized Partial Credit Model
Cui, C., Wang, C., & Xu, G. (2024) · Psychometrika · doi:10.1007/s11336-024-09955-8 · Cited by 6
“The variational method, first proposed in computer science and statistical learning, has since become an efficient approach for large-scale computation in fields such as pattern recognition and document retrieval (Titterington, 2004; Blei & Jordan, 2006).”
He, Y., Zhang, F., & Flaherty, P. (2015) · Bioinformatics · doi:10.1093/bioinformatics/btv275 · Cited by 6
“Variational methods can be generalized as the process of minimizing the Kullback-Leibler (KL) divergence between exact posterior distributions and the approximate distributions, which are generally obtained by decoupling distributions from a graphical model [69, 70].”
Das, N. R., Sharma, T., Goyal, N., Singh, N., Toropov, A. A., Toropova, A. P., & Achary, P. G. R. (2023) · Journal of Molecular Structure · doi:10.1016/j.molstruc.2023.135966 · Cited by 6
The missing radial velocities of Gaia: a catalogue of Bayesian estimates for DR3
Naik, A. P., & Widmark, A. (2023) · Monthly Notices of the Royal Astronomical Society · doi:10.1093/mnras/stad3822 · Cited by 6
Gaussian mixture models for training Bayesian convolutional neural networks
Bakhouya, M., Ramchoun, H., Mohammed, H., & Masrour, T. (2024) · Evolutionary Intelligence · doi:10.1007/s12065-023-00900-9 · Cited by 7
“To overcome this problem, we can infer this distribution using statistical estimates known as "approximate inferences" [19, 27, 27, 48, 60, 62].”
The missing radial velocities of Gaia: Blind predictions for DR3
Naik, A. P., & Widmark, A. (2022) · Monthly Notices of the Royal Astronomical Society · doi:10.1093/mnras/stac2425 · Cited by 6
Efficient Global Optimization Algorithm Using Neural Network-based Prediction and Uncertainty
Koratikere, P., Leifsson, L., Barnet, L., & Bryden, K. M. (2023) · AIAA SciTech Forum · doi:10.2514/6.2023-2683 · Cited by 7
Bayesian Inference ForgettingPreprint
Fu, S., He, F., Xu, Y., & Tao, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.06417 · Cited by 13
“Bayesian inference enables a wide spectrum of machine learning algorithms, such as probabilistic matrix factorization [46, 62, 72], topic models [30, 8, 74], probabilistic graphic models [39, 6, 71] and Bayesian neural networks [10, 37, 52, 66, 58, 61, 75].”
BackgroundNiazy, M., Murphy, H. M., Nadeem, K., & Ricker, N. (2025) · Canadian Journal of Microbiology · doi:10.1139/cjm-2025-0038 · Cited by 5
Practical Guide to Estimation and Uncertainty Quantification of Aerodynamic Flows
Eldredge, J. D., & Mousavi, H. (2026) · AIAA Journal · doi:10.2514/1.j066257 · Cited by 8
Multiscale Bayesian state-space model for Granger causality analysis of brain signal
Cekic, S., Grandjean, D., & Renaud, O. (2018) · Journal of Applied Statistics · doi:10.1080/02664763.2018.1455814 · Cited by 6
“This approximation methodology has been extensively explored and used during the past years; see for example Titterington [40], Beal [4], Ormerod and Wand [33] and Fox and Roberts [13].”
BackgroundMethodMokhtari, M. H., Deilami, K., & Moosavi, V. (2020) · Earth Science Informatics · doi:10.1007/s12145-020-00451-y · Cited by 5
“The BR is also a method that automatically adjusts ideal values for the parameters of the objective function (Burden and Winkler 2008; Marwala 2007; Okut et al. 2011; Titterington 2004).”
MethodShang, R., Luke, G. P., & O’Donnell, M. (2024) · Photoacoustics · doi:10.1016/j.pacs.2024.100645 · Cited by 5
Constrained Aerodynamic Shape Optimization Using Neural Networks and Sequential Sampling
Koratikere, P., Leifsson, L., Koziel, S. M., & Pietrenko‐Dabrowska, A. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-36024-4_33 · Cited by 4
Hassan-Esfahani, L. (2021) · Digital Commons - USU (Utah State University) · doi:10.26076/d3c7-f268 · Cited by 4
Clare, M., & Piggott, M. (2022) · CRC Press eBooks · doi:10.1201/9781003360773-61 · Cited by 4
Variational bayesian method of estimating variance components
Arakawa, A., Taniguchi, M., Hayashi, T., & Mikawa, S. (2016) · Animal Science Journal · doi:10.1111/asj.12514 · Cited by 4
fMRI activation detection with EEG priors
Kalus, S., Sämann, P. G., Czisch, M., & Fahrmeir, L. (2013) · Open access LMU (Ludwid Maxmilian's Universitat Munchen) · doi:10.5282/ubm/epub.15725 · Cited by 4
“Another popular approach in neuroimaging is a variational Bayes approach to Bayesian inference (see e.g. Titterington, 2004).”
A Statistical Theory of Deep Learning via Proximal SplittingPreprint
Polson, N. G., Willard, B. T., & Heidari, M. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1509.06061 · Cited by 4
“Our methods can therefore be seen as related to sparse Bayes MAP techniques; see Titterington [2004], Windle et al. [2013], Polson et al. [2015] for high dimensional data reduction.”
Method
Metadata
- Authors
- Titterington, D. M.
- Corresponding author
- Titterington, D. M.
- Venue
- Statistical Science
- Publisher
- Institute of Mathematical Statistics
- Published
- 2004-02-01
- Volume
- 19
- Issue
- 1
- Pages
- 128-139
- ISSN
- 0883-4237 (ISSN-L)
- 2168-8745
- OpenAlex
- W2019449965
- Cited by
- 162
- References
- 103
- Citations by year
- 2026: 2 · 2025: 9 · 2024: 20 · 2023: 16 · 2022: 17 · 2021: 9 · 2020: 9 · 2019: 7 · 2018: 6 · 2017: 7 · 2016: 10 · 2015: 9 · 2014: 8 · 2013: 3 · 2012: 1
- FWCI
- 3.05
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Statistical Science (publishedVersion)
- Project Euclid (Cornell University) (submittedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Markov Chains and Monte Carlo Methods
- Physical Sciences › Computer Science › Artificial Intelligence › Gaussian Processes and Bayesian Inference
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Keywords
- Bayesian neural networks, feedforward neural networks, Gaussian approximation, Markov chain Monte Carlo, variational approximation, computational intractability
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
