Probable networks and plausible predictions — a review of practical Bayesian methods for supervised neural networks
Abstract
Bayesian probability theory provides a unifying framework for data modelling. In this framework the overall aims are to find models that are well-matched to the data, and to use these models to make optimal predictions. Neural network learning is interpreted as an inference of the most probable parameters for the model, given the training data. The search in model space (i.e., the space of architectures, noise models, preprocessings, regularizers and weight decay constants) can then also be treated as an inference problem, in which we infer the relative probability of alternative models, given the data. This review describes practical techniques based on Gaussian approximations for implementation of these powerful methods for controlling, comparing and using adaptive networks.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
This form of model selection implements automatic relevance determination by optimising the model evidence with respect to model hyperparameters.
“This reflects the fact that this form of model selection implements automatic relevance determination (MacKay 1995), by virtue of optimising the model evidence with respect to model hyperparameters; in this instance, the shrinkage priors prescribed by an adjacency matrix.”
From Network discovery with DCM (2011) · cites this paper as MacKay 1995
Cites
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892 · see citing works
Parallel Distributed Processing: Explorations in the Microstructure of Cognition: Foundations
Rumelhart, D. E., McClelland, J. L., & AU (1986) · doi:10.7551/mitpress/5236.001.0001 · Cited by 15,686 · see citing works
Jeffreys, H., Moran, P. A. P., & Chatfield, C. (1922) · Nature · doi:10.1038/109132a0 · Cited by 7,837 · see citing works
Statistical Decision Theory and Bayesian Analysis
Berger, J. O. (1985) · Springer Series in Statistics · doi:10.1007/978-1-4757-4286-2 · Cited by 7,722 · see citing works
Mackay, D. (1992) · Neural Computation · doi:10.1162/neco.1992.4.3.415 · Cited by 4,440 · see citing works
Bayesian Learning for Neural Networks
Neal, R. M. (1996) · Lecture Notes in Statistics · doi:10.1007/978-1-4612-0745-0 · Cited by 4,276 · see citing works
Bayesian Inference in Statistical Analysis.
Kadane, J. B., Box, G. E. P., & Tiao, G. C. (1975) · Journal of the American Statistical Association · doi:10.2307/2285412 · Cited by 3,869 · see citing works
(1989) · Cited by 3,214 · see citing works
A practical Bayesian framework for backpropagation networks
MacKay, D. J. C. (1992) · Neural Computation · doi:10.1162/neco.1992.4.3.448 · Cited by 2,917 · see citing works
Stastical Decision Theory and Bayesian Analysis.
Ghosh, M., & Berger, J. O. (1988) · Journal of the American Statistical Association · doi:10.2307/2288950 · Cited by 2,901 · see citing works
Arithmetic coding for data compression
Witten, I. H., Neal, R. M., & Cleary, J. G. (1987) · Communications of the ACM · doi:10.1145/214762.214771 · Cited by 2,894 · see citing works
Advances in Neural Information Processing Systems 37
Touretzky, D. S. (2024) · doi:10.52202/079017 · Cited by 2,874 · see citing works
Breiman, L. (1996) · Machine Learning · doi:10.1023/a:1018046112532 · Cited by 1,964 · see citing works
Advances in Neural Information Processing Systems 5
Zemel, R. S., Williams, C. K. I., & Mozer, M. C. (1993) · Cited by 1,896 · see citing works
(1993) · CaltechAUTHORS (California Institute of Technology) · Cited by 1,498 · see citing works
Bayesian Inference in Statistical Analysis
Hill, B. M., Box, G. E. P., & Tiao, G. C. (1974) · Technometrics · doi:10.2307/1267683 · Cited by 1,465 · see citing works
Probability, Frequency and Reasonable Expectation
Cox, R. T. (1946) · American Journal of Physics · doi:10.1119/1.1990764 · Cited by 1,290 · see citing works
Bridle, J. S. (1990) · Springer eBooks · doi:10.1007/978-3-642-76153-9_28 · Cited by 1,278 · see citing works
(1993) · Cited by 1,173 · see citing works
Learning and relearning in Boltzmann machines
Hinton, G. E., & Sejnowski, T. J. (2001) · Graphical Models · doi:10.7551/mitpress/3349.003.0005 · Cited by 1,037 · see citing works
Keeping the neural networks simple by minimizing the description length of the weights
Hinton, G. E., & van Camp, D. (1993) · Proceedings of the sixth annual conference on Computational learning theory - COLT '93 · doi:10.1145/168304.168306 · Cited by 994 · see citing works
An Information Measure for Classification
Wallace, C. S., & Boulton, D. M. (1968) · The Computer Journal · doi:10.1093/comjnl/11.2.185 · Cited by 929 · see citing works
Advances in Neural Information Processing Systems 7
Tesauro, G., Touretzky, D. S., & Leen, T. K. (1995) · Cited by 869 · see citing works
The Evidence Framework Applied to Classification Networks
Mackay, D. (1992) · Neural Computation · doi:10.1162/neco.1992.4.5.720 · Cited by 724 · see citing works
Bayesian methods for adaptive models
Mackay, D. (1992) · doi:10.7907/h3a1-wm07 · Cited by 620 · see citing works
Regression, Prediction and Shrinkage
Copas, J. B. (1983) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1983.tb01258.x · Cited by 612 · see citing works
Fast Exact Multiplication by the Hessian
Pearlmutter, B. A. (1994) · Neural Computation · doi:10.1162/neco.1994.6.1.147 · Cited by 553 · see citing works
Sequential updating of conditional probabilities on directed graphical structures
Spiegelhalter, D. J., & Lauritzen, S. L. (1990) · Networks · doi:10.1002/net.3230200507 · Cited by 521 · see citing works
Estimation and Inference by Compact Coding
Wallace, C. S., & Freeman, P. R. (1987) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.2517-6161.1987.tb01695.x · Cited by 521 · see citing works
Bayesian Spectrum Analysis and Parameter Estimation
Guber, M. H. J. (1990) · Technometrics · doi:10.1080/00401706.1990.10484646 · Cited by 443 · see citing works
Moody, J. E. (1991) · Cited by 437 · see citing works
Bayesian Spectrum Analysis and Parameter Estimation
Bretthorst, G. L. (1988) · Lecture notes in statistics · doi:10.1007/978-1-4684-9399-3 · Cited by 435 · see citing works
Buntine, W., & Weigend, A. S. (1991) · Complex Systems · Cited by 357 · see citing works
E. T. Jaynes: Papers on Probability, Statistics and Statistical Physics
Jaynes, E. T., & Rosenkrantz, R. D. (1983) · Springer eBooks · doi:10.1007/978-94-009-6581-2 · Cited by 309 · see citing works
Neurocomputing : algorithms, architectures and applications
Fogelman‐Soulié, F., & Hérault, J. (1990) · Springer eBooks · Cited by 275 · see citing works
Developments in Maximum Entropy Data Analysis
Gull, S. F. (1989) · Springer eBooks · doi:10.1007/978-94-015-7860-8_4 · Cited by 262 · see citing works
Bayesian Inductive Inference and Maximum Entropy
Gull, S. F. (1988) · Springer eBooks · doi:10.1007/978-94-009-3049-0_4 · Cited by 253 · see citing works
Diaconis, P. W. (1988) · Springer eBooks · doi:10.1007/978-1-4613-8768-8_20 · Cited by 244 · see citing works
Bayesian Learning via Stochastic Dynamics
Neal, R. M. (1992) · neural information processing systems · Cited by 180 · see citing works
Bayesian neural networks and density networks
Mackay, D. (1995) · Nuclear Instruments and Methods in Physics Research Section A Accelerators Spectrometers Detectors and Associated Equipment · doi:10.1016/0168-9002(94)00931-7 · Cited by 179 · see citing works
From Laplace to Supernova SN 1987A: Bayesian Inference in Astrophysics
Loredo, T. J. (1990) · Maximum Entropy and Bayesian Methods · doi:10.1007/978-94-009-0683-9_6 · Cited by 168 · see citing works
The Vapnik-Chervonenkis Dimension: Information versus Complexity in Learning
Abu‐Mostafa, Y. S. (1989) · Neural Computation · doi:10.1162/neco.1989.1.3.312 · Cited by 151 · see citing works
Exact Calculation of the Hessian Matrix for the Multilayer Perceptron
Bishop, C. (1992) · Neural Computation · doi:10.1162/neco.1992.4.4.494 · Cited by 146 · see citing works
Structural Risk Minimization for Character Recognition
Guyon, I., Vapnik, V. N., Boser, B. E., Bottou, L., & Solla, S. A. (1991) · neural information processing systems · Cited by 105 · see citing works
Hyperparameters: Optimize, or Integrate Out?
Mackay, D. (1996) · Springer eBooks · doi:10.1007/978-94-015-8729-7_2 · Cited by 90 · see citing works
Bayesian classification with correlation and inheritance
Hanson, R., Stutz, J., & Cheeseman, P. (1991) · Cited by 53 · see citing works
Skilling, J. (1993) · Cambridge University Press eBooks · doi:10.1017/cbo9780511524448.020 · Cited by 17 · see citing works
Applications of Maximum Entropy Techniques to HST Data
Weir, N. (1991) · European Southern Observatory Conference and Workshop Proceedings · Cited by 16 · see citing works
STONE CIRCLE GEOMETRIES: AN INFORMATION THEORY APPROACH
Patrick, J. D., & Wallace, C. (1982) · Cambridge University Press eBooks · doi:10.1017/cbo9780511898310.016 · Cited by 12 · see citing works
Skilling, J., Robinson, D. R., & Gull, S. F. (1991) · Springer eBooks · doi:10.1007/978-94-011-3460-6_37 · Cited by 8 · see citing works
Flagged first, then by citations
2 of the 52 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Bayesian Measures of Model Complexity and Fit
Spiegelhalter, D. J., Best, N. G., Carlin, B. P., & Van Der Linde, A. (2002) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/1467-9868.00353 · Cited by 151 · see citing works
Probabilistic diffusion tractography with multiple fibre orientations: What can we gain?
Behrens, T. E. J., Berg, H. J., Jbabdi, S., Rushworth, M. F. S., & Woolrich, M. W. (2007) · NeuroImage · doi:10.1016/j.neuroimage.2006.09.018 · Cited by 3,471 · see citing works
Supervised Sequence Labelling with Recurrent Neural Networks
Graves, A. (2012) · Studies in Computational Intelligence · doi:10.1007/978-3-642-24797-2 · Cited by 3,120 · see citing works
Dynamic bayesian networks: representation, inference and learning
Murphy, K. P., & Russell, S. (2002) · Cited by 2,563 · see citing works
Weight Uncertainty in Neural Network
Blundell, C., Cornebise, J., Kavukcuoglu, K., & Wierstra, D. (2015) · International Conference on Machine Learning · Cited by 1,280 · see citing works
Stegle, O., Parts, L., Piipari, M., Winn, J., & Durbin, R. (2012) · Nature Protocols · doi:10.1038/nprot.2011.457 · Cited by 1,049 · see citing works
Practical Variational Inference for Neural Networks
Graves, A. (2011) · Cited by 1,079 · see citing works
Gigerenzer, G. (2004) · The Journal of Socio-Economics · doi:10.1016/j.socec.2004.09.033 · Cited by 890 · see citing works
A family of algorithms for approximate bayesian inference
Minka, T. P., & Picard, R. W. (2001) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 877 · see citing works
Variational free energy and the Laplace approximation
Friston, K., Mattout, J., Trujillo-Barreto, N., Ashburner, J., & Penny, W. (2007) · NeuroImage · doi:10.1016/j.neuroimage.2006.08.035 · Cited by 880 · see citing works
Benchmarking Least Squares Support Vector Machine Classifiers
Gestel, T. V., Suykens, J. A. K., Baesens, B., Viaene, S., Vanthienen, J., Dedene, G., Moor, B. D., & Vandewalle, J. P. L. (2003) · Machine Learning · doi:10.1023/b:mach.0000008082.80494.e0 · Cited by 746 · see citing works
The priority heuristic: Making choices without trade-offs.
Brandstätter, E., Gigerenzer, G., & Hertwig, R. (2006) · Psychological Review · doi:10.1037/0033-295x.113.2.409 · Cited by 744 · see citing works
Machine Learning for Survival Analysis
Wang, P., Li, Y., & Reddy, C. K. (2019) · ACM Computing Surveys · doi:10.1145/3214306 · Cited by 473 · see citing works
Big-Data Science in Porous Materials: Materials Genomics and Machine Learning
Jablonka, K. M., Ongari, D., Moosavi, S. M., & Smit, B. (2020) · Chemical Reviews · doi:10.1021/acs.chemrev.0c00004 · Cited by 636 · see citing works
Stegle, O., Parts, L., Durbin, R., & Winn, J. (2010) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000770 · Cited by 497 · see citing works
Probabilistic models of cognition: exploring representations and inductive biases
Griffiths, T. L., Chater, N., Kemp, C., Perfors, A., & Tenenbaum, J. B. (2010) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2010.05.004 · Cited by 568 · see citing works
Machine-learned potentials for next-generation matter simulations
Friederich, P., Häse, F., Proppe, J., & Aspuru‐Guzik, A. (2021) · Nature Materials · doi:10.1038/s41563-020-0777-6 · Cited by 558 · see citing works
Learning dynamic Bayesian networks
Zoubin Ghahramani, A. (1998) · Lecture notes in computer science · doi:10.1007/bfb0053999 · Cited by 561 · see citing works
Structural risk minimization over data-dependent hierarchies
Shawe-Taylor, J. S., Bartlett, P. L., Williamson, R. C., & Anthony, M. (1998) · IEEE Transactions on Information Theory · doi:10.1109/18.705570 · Cited by 528 · see citing works
Gestel, T. V., Suykens, J. A. K., Baestaens, D.-E., Lambrechts, A., Lanckriet, G. R. G., Vandaele, B., Moor, B. D., & Vandewalle, J. P. L. (2001) · IEEE Transactions on Neural Networks · doi:10.1109/72.935093 · Cited by 499 · see citing works
Automatic Choice of Dimensionality for PCA
Minka, T. P. (2000) · Cited by 499 · see citing works
Bayesian Neural Networks for Internet Traffic Classification
Auld, T., Moore, A. W., & Gull, S. F. (2007) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2006.883010 · Cited by 494 · see citing works
Kwok, T.-Y., & Yeung, D. (1997) · IEEE Transactions on Neural Networks · doi:10.1109/72.572102 · Cited by 490 · see citing works
Variational Bayesian Inference for a Nonlinear Forward Model
Chappell, M. A., Groves, A. R., Whitcher, B. J., & Woolrich, M. W. (2008) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2008.2005752 · Cited by 468 · see citing works
Ghahramani, Z. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-28650-9_5 · Cited by 476 · see citing works
Gradient-Based Optimization of Hyperparameters
Bengio, Y. (2000) · Neural Computation · doi:10.1162/089976600300015187 · Cited by 467 · see citing works
A review of evidence of health benefit from artificial neural networks in medical intervention
Lisboa, P. J. G. (2002) · Neural Networks · doi:10.1016/s0893-6080(01)00111-3 · Cited by 459 · see citing works
Beal, M. J., & Ghahramani, Z. (2003) · Bayesian Statistics 7 · doi:10.1093/oso/9780198526155.003.0025 · Cited by 458 · see citing works
Nonrigid Structure-from-Motion: Estimating Shape and Motion with Hierarchical Priors
Torresani, L., Hertzmann, A., & Bregler, C. (2008) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.70752 · Cited by 456 · see citing works
Uncertainty quantification in scientific machine learning: Methods, metrics, and comparisons
Psaros, A. F., Meng, X., Zou, Z., Guo, L., & Karniadakis, G. E. (2023) · Journal of Computational Physics · doi:10.1016/j.jcp.2022.111902 · Cited by 464 · see citing works
Accurate freeway travel time prediction with state-space neural networks under missing data
van Lint, H., Hoogendoorn, S. P., & van Zuylen, H. J. (2005) · Transportation Research Part C Emerging Technologies · doi:10.1016/j.trc.2005.03.001 · Cited by 442 · see citing works
Neural networks in business: a survey of applications (1992–1998)
Vellido, A. (1999) · Expert Systems with Applications · doi:10.1016/s0957-4174(99)00016-0 · Cited by 428 · see citing works
The graphical brain: Belief propagation and active inference
Friston, K. J., Parr, T., & de Vries, B. (2017) · Network Neuroscience · doi:10.1162/netn_a_00018 · Cited by 415 · see citing works
Dynamic modeling of genetic networks using genetic algorithm and S-system
Kikuchi, S., Tominaga, D., Arita, M., Takahashi, K., & Tomita, M. (2003) · Bioinformatics · doi:10.1093/bioinformatics/btg027 · Cited by 401 · see citing works
Overlapping community detection using Bayesian non-negative matrix factorization
Psorakis, I., Roberts, S., Ebden, M., & Sheldon, B. C. (2011) · Physical Review E · doi:10.1103/physreve.83.066114 · Cited by 399 · see citing works
Language Evolution by Iterated Learning With Bayesian Agents
Griffiths, T. L., & Kalish, M. L. (2007) · Cognitive Science · doi:10.1080/15326900701326576 · Cited by 387 · see citing works
Bayesian approach for neural networks—review and case studies
Lampinen, J., & Vehtari, A. (2001) · Neural Networks · doi:10.1016/s0893-6080(00)00098-8 · Cited by 392 · see citing works
Analytical estimates of limited sampling biases in different information measures
Panzeri, S., & Treves, A. (1996) · Network Computation in Neural Systems · doi:10.1080/0954898x.1996.11978656 · Cited by 383 · see citing works
Modelling Gene Expression Data using Dynamic Bayesian Networks
Murphy, K. J. (2006) · Cited by 353 · see citing works
Bayesian Compression for Deep LearningPreprint
Louizos, C., Ullrich, K., & Welling, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.08665 · Cited by 363 · see citing works
Bayesian Deep Learning and a Probabilistic Perspective of GeneralizationPreprint
Wilson, A. G., & Izmailov, P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.08791 · Cited by 349 · see citing works
Blind image deconvolution: theory and applications
Campisi, P., & Egiazarian, K. (2007) · Cited by 348 · see citing works
Variational Bayesian Model Selection for Mixture Distributions
Corduneanu, A., & Bishop, C. (2001) · Cited by 345 · see citing works
COMPLETE FUNCTIONAL CHARACTERIZATION OF SENSORY NEURONS BY SYSTEM IDENTIFICATION
Wu, M. C., David, S. V., & Gallant, J. L. (2006) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.29.051605.113024 · Cited by 332 · see citing works
A small number of abnormal brain connections predicts adult autism spectrum disorder
Yahata, N., Morimoto, J., Hashimoto, R., Lisi, G., Shibata, K., Kawakubo, Y., Kuwabara, H., Kuroda, M., Yamada, T., Megumi, F., et al. (2016) · Nature Communications · doi:10.1038/ncomms11254 · Cited by 305 · see citing works
A tutorial on support vector machine-based methods for classification problems in chemometrics
Luts, J., Ojeda, F., de Plas, R. V., Moor, B. D., Huffel, S. V., & Suykens, J. A. K. (2010) · Analytica Chimica Acta · doi:10.1016/j.aca.2010.03.030 · Cited by 331 · see citing works
Soft sensing modeling based on support vector machine and Bayesian model selection
Yan, W., Shao, H., & Wang, X. (2004) · Computers & Chemical Engineering · doi:10.1016/j.compchemeng.2003.11.004 · Cited by 326 · see citing works
Estimation of bainite plate-thickness in low-alloy steels
Singh, S. B., & Bhadeshia, H. K. D. H. (1998) · Materials Science and Engineering A · doi:10.1016/s0921-5093(97)00701-6 · Cited by 320 · see citing works
Chu, C., Hsu, A.-L., Chou, K.-H., Bandettini, P., & Lin, C. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2011.11.066 · Cited by 319 · see citing works
Towards dropout training for convolutional neural networks
Wu, H., & Gu, X. (2015) · Neural Networks · doi:10.1016/j.neunet.2015.07.007 · Cited by 315 · see citing works
A tutorial introduction to Bayesian models of cognitive development
Perfors, A., Tenenbaum, J. B., Griffiths, T. L., & Xu, F. (2011) · Cognition · doi:10.1016/j.cognition.2010.11.015 · Cited by 300 · see citing works
Structural learning with forgetting
Ishikawa, M. (1996) · Neural Networks · doi:10.1016/0893-6080(96)83696-3 · Cited by 300 · see citing works
Gestel, T. V., Suykens, J. A. K., Lanckriet, G. R. G., Lambrechts, A., Moor, B. D., & Vandewalle, J. P. L. (2002) · Neural Computation · doi:10.1162/089976602753633411 · Cited by 291 · see citing works
Natural computing for mechanical systems research: A tutorial overview
Worden, K., Staszewski, W. J., & Hensman, J. J. (2010) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2010.07.013 · Cited by 290 · see citing works
Waking and dreaming consciousness: neurobiological and functional considerations.
Hobson, J. A., & Friston, K. J. (2012) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2012.05.003 · Cited by 287 · see citing works
Friston, K. J., Li, B., Daunizeau, J., & Stephan, K. E. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2010.12.039 · Cited by 283 · see citing works
A pre-trained convolutional neural network based method for thyroid nodule diagnosis
Ma, J., Wu, F., Zhu, J., Xu, D. M., & Kong, D. (2016) · Ultrasonics · doi:10.1016/j.ultras.2016.09.011 · Cited by 275 · see citing works
Automatic Relevance Determination in Nonnegative Matrix Factorization with the /spl beta/-Divergence
Tan, V. Y. F., & Févotte, C. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.240 · Cited by 271 · see citing works
Design of inferential sensors in the process industry: A review of Bayesian methods
Khatibisepehr, S., Huang, B., & Khare, S. (2013) · Journal of Process Control · doi:10.1016/j.jprocont.2013.05.007 · Cited by 261 · see citing works
Predicting clinical scores from magnetic resonance scans in Alzheimer's disease
Stonnington, C. M., Chu, C., Klöppel, S., Jack, C. R., Ashburner, J., & Frackowiak, R. S. J. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2010.03.051 · Cited by 256 · see citing works
Machine Theory of MindPreprint
Rabinowitz, N. C., Perbet, F., Song, H. F., Zhang, C., Eslami, S. M. A., & Botvinick, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.07740 · Cited by 259 · see citing works
Fully Bayesian Spatio-Temporal Modeling of FMRI Data
Woolrich, M. W., Jenkinson, M., Brady, J. M., & Smith, S. M. (2004) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2003.823065 · Cited by 248 · see citing works
Bayesian Neural Networks: An Introduction and Survey
Goan, E., & Fookes, C. (2020) · Lecture notes in mathematics · doi:10.1007/978-3-030-42553-1_3 · Cited by 248 · see citing works
Bittig, H. C., Steinhoff, T., Claustre, H., Fiedler, B., Williams, N. L., Sauzède, R., Körtzinger, A., & Gattuso, J. (2018) · Frontiers in Marine Science · doi:10.3389/fmars.2018.00328 · Cited by 246 · see citing works
The Principles of Deep Learning Theory
Roberts, D. A., Yaida, S., & Hanin, B. (2022) · Cambridge University Press eBooks · doi:10.1017/9781009023405 · Cited by 250 · see citing works
Variational Gaussian process classifiers
Mackay, D., & Gibbs, M. (2000) · IEEE Transactions on Neural Networks · doi:10.1109/72.883477 · Cited by 224 · see citing works
Bishop, C. (1998) · neural information processing systems · Cited by 224 · see citing works
Pyzer‐Knapp, E. O., Li, K., & Aspuru‐Guzik, A. (2015) · Advanced Functional Materials · doi:10.1002/adfm.201501919 · Cited by 223 · see citing works
Analytical estimates of limited sampling biases in different information measures
Panzeri, S., & Treves, A. (1996) · Network Computation in Neural Systems · doi:10.1088/0954-898x/7/1/006 · Cited by 207 · see citing works
Online Learning Solutions for Freeway Travel Time Prediction
van Lint, H. (2008) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2008.915649 · Cited by 200 · see citing works
Griffiths, T. L., Chater, N., Norris, D., & Pouget, A. (2012) · Psychological Bulletin · doi:10.1037/a0026884 · Cited by 192 · see citing works
Automatic recognition of alertness and drowsiness from EEG by an artificial neural network
Vučković, A., Radivojević, V., Chen, A. C. N., & Popović, D. B. (2002) · Medical Engineering & Physics · doi:10.1016/s1350-4533(02)00030-9 · Cited by 198 · see citing works
Deo, R. C., & Şahin, M. (2017) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2017.01.114 · Cited by 193 · see citing works
Intrinsic dimension estimation: Advances and open problems
Camastra, F., & Staiano, A. (2015) · Information Sciences · doi:10.1016/j.ins.2015.08.029 · Cited by 192 · see citing works
Artificial neural networks and genetic algorithms in QSAR
Niculescu, Ş. P. (2003) · Journal of Molecular Structure THEOCHEM · doi:10.1016/s0166-1280(02)00619-x · Cited by 190 · see citing works
Ultrasound image-based thyroid nodule automatic segmentation using convolutional neural networks
Ma, J., Wu, F., Jiang, T., Zhao, Q., & Kong, D. (2017) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-017-1649-7 · Cited by 187 · see citing works
Lancashire, L., Lemetre, C., & Ball, G. (2008) · Briefings in Bioinformatics · doi:10.1093/bib/bbp012 · Cited by 177 · see citing works
Bayesian kernel based classification for financial distress detection
Gestel, T. V., Baesens, B., Suykens, J. A. K., den Poel, D. V., Baestaens, D.-E., & Willekens, M. (2005) · European Journal of Operational Research · doi:10.1016/j.ejor.2004.11.009 · Cited by 185 · see citing works
Low-Rank Modeling and Its Applications in Image Analysis
Zhou, X., Yang, C., Zhao, H., & Yu, W. (2014) · ACM Computing Surveys · doi:10.1145/2674559 · Cited by 174 · see citing works
A PAC analysis of a Bayesian estimator
Shawe-Taylor, J. S., & Williamson, R. C. (1997) · Conference on Learning Theory · doi:10.1145/267460.267466 · Cited by 168 · see citing works
Ching, J., & Phoon, K. (2018) · Journal of Engineering Mechanics · doi:10.1061/(asce)em.1943-7889.0001537 · Cited by 165 · see citing works
BELM: Bayesian Extreme Learning Machine
Soria‐Olivas, E., Gómez‐Sanchís, J., Martín‐Guerrero, J. D., Vila‐Francés, J., Martinez, M., Magdalena‐Benedito, R., & Serrano-López, A. J. (2011) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2010.2103956 · Cited by 164 · see citing works
Bayesian Methods for Neural Networks and Related Models
Titterington, D. M. (2004) · Statistical Science · doi:10.1214/088342304000000099 · Cited by 162 · see citing works
Safe Visual Navigation via Deep Learning and Novelty Detection
Richter, C., & Roy, N. (2017) · Robotics: Science and Systems · doi:10.15607/rss.2017.xiii.064 · Cited by 160 · see citing works
Bayesian learning for neural networks: an algorithmic survey
Magris, M., & Iosifidis, A. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10443-1 · Cited by 166 · see citing works
Ensemble learning in Bayesian neural networks
Barber, D., & Bishop, C. (1998) · Cited by 160 · see citing works
Moderating the outputs of support vector machine classifiers
Kwok, J. T.-Y. (1999) · IEEE Transactions on Neural Networks · doi:10.1109/72.788642 · Cited by 155 · see citing works
Bayesian neural networks: An introduction and survey
Goan, E., & Fookes, C. (2020) · QUT ePrints (Queensland University of Technology) · Cited by 162 · see citing works
Revisiting Stacking Fault Energy of Steels
Das, A. (2015) · Metallurgical and Materials Transactions A · doi:10.1007/s11661-015-3266-9 · Cited by 155 · see citing works
DropConnect is effective in modeling uncertainty of Bayesian deep networks
Mobiny, A., Yuan, P., Moulik, S. K., Garg, N., Wu, C. C., & Nguyen, H. V. (2021) · Scientific Reports · doi:10.1038/s41598-021-84854-x · Cited by 154 · see citing works
Bayesian committee of neural networks to predict travel times with confidence intervals
van Hinsbergen, C. P. I., van Lint, H., & van Zuylen, H. J. (2009) · Transportation Research Part C Emerging Technologies · doi:10.1016/j.trc.2009.04.007 · Cited by 150 · see citing works
Probabilistic neural networks for fluid flow surrogate modeling and data recovery
Maulik, R., Fukami, K., Ramachandra, N., Fukagata, K., & Taira, K. (2020) · Physical Review Fluids · doi:10.1103/physrevfluids.5.104401 · Cited by 148 · see citing works
Lisboa, P. J. G., Wong, H., Harris, P., & Swindell, R. (2003) · Artificial Intelligence in Medicine · doi:10.1016/s0933-3657(03)00033-2 · Cited by 149 · see citing works
Bayesian Support Vector Regression Using a Unified Loss Function
Chu, W., Keerthi, S. S., & Ong, C. J. (2004) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2003.820830 · Cited by 144 · see citing works
Deo, R. C., Samui, P., & Kim, D. (2015) · Stochastic Environmental Research and Risk Assessment · doi:10.1007/s00477-015-1153-y · Cited by 142 · see citing works
Super Resolution of Images and Video
Katsaggelos, A. K., Molina, R., & Javier Mateos, A. (2007) · Synthesis lectures on image, video, and multimedia processing · doi:10.1007/978-3-031-02243-2 · Cited by 142 · see citing works
Bayesian Gaussian Process Models: PAC-Bayesian Generalisation Error Bounds and Sparse Approximations
Seeger, M. (2003) · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · Cited by 142 · see citing works
Ma, J., Wu, F., Jiang, T., Zhu, J., & Kong, D. (2017) · Medical Physics · doi:10.1002/mp.12134 · Cited by 142 · see citing works
Vong, C., Wong, P.-K., & Li, Y. (2005) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2005.09.001 · Cited by 139 · see citing works
Learning Non-Rigid 3D Shape from 2D Motion
Torresani, L., Hertzmann, A., & Bregler, C. (2003) · Cited by 139 · see citing works
- Authors
- Mackay, D. J. C.
- Affiliations
- University of Cambridge (GB)
- Corresponding author
- Mackay, David J C
- ORCID
- Mackay, David J C
- Venue
- Network Computation in Neural Systems
- Publisher
- Informa UK Limited
- Published
- 1995-01
- Volume
- 6
- Issue
- 3
- Pages
- 469-505
- ISSN
- 0954-898X (ISSN-L)
- 1361-6536
- OpenAlex
- W2139701068
- Cited by
- 915
- References
- 52
- Citations by year
- 2026: 14 · 2025: 33 · 2024: 28 · 2023: 40 · 2022: 50 · 2021: 39 · 2020: 58 · 2019: 40 · 2018: 36 · 2017: 27 · 2016: 17 · 2015: 36 · 2014: 24 · 2013: 29 · 2012: 39
- FWCI
- 13.31
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Network Computation in Neural Systems (publishedVersion)
- Cambridge University Engineering Department Publications Database (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Gaussian Processes and Bayesian Inference
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Bayesian Modeling and Causal Inference
- Keywords
- Bayesian neural networks, supervised learning, Gaussian approximation, model selection, model space search, parameter inference, adaptive networks, noise models, regularization, weight decay
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review