Cited by 915Gaussian Processes and Bayesian InferenceNeural Networks and ApplicationsBayesian Modeling and Causal Inference

The DOI matches its Crossref record; 50 of the 52 references it cites were found in a registry or an index.

Probable networks and plausible predictions — a review of practical Bayesian methods for supervised neural networks

Mackay, David J C·1995·Network Computation in Neural Systems

DOI 10.1088/0954-898x/6/3/011

Open this paper in CoChat — free account

Abstract

Bayesian probability theory provides a unifying framework for data modelling. In this framework the overall aims are to find models that are well-matched to the data, and to use these models to make optimal predictions. Neural network learning is interpreted as an inference of the most probable parameters for the model, given the training data. The search in model space (i.e., the space of architectures, noise models, preprocessings, regularizers and weight decay constants) can then also be treated as an inference problem, in which we infer the relative probability of alternative models, given the data. This review describes practical techniques based on Gaussian approximations for implementation of these powerful methods for controlling, comparing and using adaptive networks.

Integrity signals

Registry

Crossref · 10.1088/0954-898x_6_3_011

Cites

50 of 50 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892
Parallel Distributed Processing: Explorations in the Microstructure of Cognition: Foundations
Rumelhart, D. E., McClelland, J. L., & AU (1986) · doi:10.7551/mitpress/5236.001.0001 · Cited by 15,686
The Theory of Probability
Jeffreys, H., Moran, P. A. P., & Chatfield, C. (1922) · Nature · doi:10.1038/109132a0 · Cited by 7,877
Statistical Decision Theory and Bayesian Analysis
Berger, J. O. (1985) · Springer Series in Statistics · doi:10.1007/978-1-4757-4286-2 · Cited by 7,722
Bayesian Interpolation
Mackay, D. (1992) · Neural Computation · doi:10.1162/neco.1992.4.3.415 · Cited by 4,440
Bayesian Learning for Neural Networks
Neal, R. M. (1996) · Lecture Notes in Statistics · doi:10.1007/978-1-4612-0745-0 · Cited by 4,276
Bayesian Inference in Statistical Analysis.
Kadane, J. B., Box, G. E. P., & Tiao, G. C. (1975) · Journal of the American Statistical Association · doi:10.2307/2285412 · Cited by 3,853
Untitled work
(1989) · Cited by 3,214
A practical Bayesian framework for backpropagation networks
MacKay, D. J. C. (1992) · Neural Computation · doi:10.1162/neco.1992.4.3.448 · Cited by 2,917
Stastical Decision Theory and Bayesian Analysis.
Ghosh, M., & Berger, J. O. (1988) · Journal of the American Statistical Association · doi:10.2307/2288950 · Cited by 2,901

Show all 52 references

Research works printed but not linked: 11

Breiman L, 1992, Technical Report 367
Buntine W, 1991, Complex Systems, 5, 603
Guyon I, 1992, Advances in Neural Information Processing Systems 4, 471
Hinton G E, 1986, Parallel Distributed Processing, 282
Jeffreys H, 1939, Theory of Probability
LeCun Y, 1990, Advances in Neural Information Processing Systems 2, 598
MacKay D J C, 1994, ASHRAE Trans. Vol 100, part 2
MacKay D J C, 1995, Nucl. Instrum. Methods Phys. Res. A
Moody J E, 1992, Advances in Neural Information Processing Systems 4, 847
Thodberg H H, 1993, Technical Report 1132 E
Wolpert D H, 1993, Advances in Neural Information Processing Systems 5, 539

Sorted by citations

Items 801–900 of 912

Continuous simulation of hypothetical physics processes with multiple free parameters
Zhong, J., Huang, R.-S., & Lee, S. C. (2012) · Nuclear Instruments and Methods in Physics Research Section A Accelerators Spectrometers Detectors and Associated Equipment · doi:10.1016/j.nima.2012.03.026
Iterative optimal control based on support vector machine modeling within the Bayesian evidence framework
Li, G.-P., Yan, W., & Hui-he, S. (2007) · Journal of Shanghai University (English Edition) · doi:10.1007/s11741-007-0613-2
Visual object categorization with indefinite kernels in discriminant analysis framework
Kosinov, S., & Marchand‐Maillet, S. (2006) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.642378
Application of a MLP neural network for compensation of distortions in strain gauges measurements due to temperature variations in offshore operation of a flexible pipe-lay vessel
Barbosa, C. R. H., Kuhner, G. S., Lima, E. A., Vellasco, M. M. B. R., & Pacheco, M. A. C. (2003) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2002.1005449
High-Dimensional Non-Gaussian Data Clustering using Variational Learning of Mixture Models
Fan, W. (2013) · Spectrum Research Repository (Concordia University)
Automatic image classification from Cherenkov telescopes using Bayesian ensemble of neural networks
Malagón, C., Barrio, J. A., & Nieto, D. (2009) · International Workshop on Soft Computing Applications (SOFA) · doi:10.1109/sofa.2009.5254880
Power Load Modeling Based on Wide-Area Measurements and Support Vector Machine
Wang, Z., Li, L., & Niu, L. (2009) · Asia-Pacific Power and Energy Engineering Conference (APPEEC) · doi:10.1109/appeec.2009.4918609
Modelli del funzionamento cerebrale
Meraviglia, M. V. (2012) · Springer eBooks · doi:10.1007/978-88-470-1995-9_2
Inference of distillation column products quality using Bayesian networks
Barbosa, C. R. H., Melo, B., Vellasco, M. M. B. R., Pacheco, M. A. C., & Vasconcellos, L. P. (2003) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2002.1005448
Variational Bayesian speaker change detection
Valente, F., & Wellekens, C. J. (2005) · Interspeech (USB) · doi:10.21437/interspeech.2005-199
Bayesian evolution of rich neural networks
Matteucci, M., & Spadoni, D. (2005) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2004.1379904
Bayesian models for unsupervised feature selection
Guan, Y. (2012) · doi:10.17760/d20002991
Bayesian Inference in Nonlinear and Relational Latent Variable Models
Raiko, T. (2006) · Aaltodoc (Aalto University) · Cited by 1
Variational Approximate Inference in Latent Linear Models
Challis, E. (2013) · UCL Discovery (University College London)
Model Selection and Model Complexity: Identifying Truth Within A Space Saturated with Random Models
Helman, P. (2005)
ASYMPTOTIC THEORY OF EMPIRICAL AND VARIATIONAL
Nakajima, S. (2006) · Tokyo Tech Research Repository (Tokyo Institute of Technology) · Cited by 2
Nonlinear Modeling of the Early and Mid-Level Visual System
Oliver, M. (2014) · eScholarship (California Digital Library)
On Evidence Weighted Mixture Classification
Everson, R., Krzanowski, W. J., Bailey, T., & Fieldsend, J. E. (2005) · Open Research Exeter (University of Exeter)
CATEGORY-LEVEL VISUAL OBJECT RECOGNITION USING NOVEL MACHINE LEARNING TECHNIQUES
Márquez, P., & Mark., B. (2013)
An Analysis of Bayesian Bagging Prediction for Improving Generalization Performance
Kurogi, S., Harashima, K., & Shibata, N. (2010) · International Conference on Intelligent Information Processing
Selection in neural information processing
van de Laar, P. J. L. J. (1999) · Radboud Repository (Radboud University)
A New Statistical Model for Radar HRRP Target Recognition
Hou, Q., Chen, F., Liu, H., & Bao, Z. (2009) · Advances in intelligent and soft computing · doi:10.1007/978-3-642-01216-7_42
Ambiguous Data in Connectionist Language Processing Models
Bullinaria, J. A. (1996)
Advanced source separation methods with applications to spatio-temporal datasets
Ilin, A. (2006) · Aaltodoc (Aalto University)
Error Bounds for Steel Tensile Strength Prediction—Comparison of Bayesian and Ensemble Model Approaches
Yang, Y. Y., & Linkens, D. A. (2001) · IFAC Proceedings Volumes · doi:10.1016/s1474-6670(17)33207-x
Improving the Generalization Ability of Restricted Boltzmann Machines via Theta Pure Dependency
Xu, Q., & Hou, Y. (2017) · International Conference on Deep Learning Technologies (ICDLT) · doi:10.1145/3094243.3094246
Component-level aggregation of probabilistic PCA mixtures using variational-Bayes
Bruneau, P., Gelgon, M., & Picarougne, F. (2011) · HAL
Structural Learning with Forgetting of Neural Networks
Ishikawa, M. (1997) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jfuzzy.9.1_2
An interoperable framework for a clinical decision support system
Bilykh, I. (2004)
Latent Dimensionality Estimation for Probabilistic Canonical Correlation Analysis Using Normalized Maximum Likelihood Code-Length
Nakmaura, T., Iwata, T., & Yamanishi, K. (2017) · IEEE International Conference on Data Science and Advanced Analytics (DSAA) · doi:10.1109/dsaa.2017.39
Continuous simulation of hypothetical physics processes with multiple free parameters
Zhong, J., & Lee, S. C. (2012) · Journal of Physics Conference Series · doi:10.1088/1742-6596/368/1/012042
Laplace Exponential Family PCA
Li, F., & Ren, X. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-95930-6_30
Study on the Pattern Recognition Enhancement for Matrix Factorizations with Automatic Relevance Determination
Tao, H. (2018) · CSUSB ScholarWorks (California State University, San Bernardino)
An Optimized Regularization Method to Enhance Low-Resource MT
Ji, Y., Hou, H., Lei, Y., & Ren, Z. (2019) · Communications in computer and information science · doi:10.1007/978-981-13-5907-1_30
Sample-Efficient Optimization Using Bayesian Neural Networks
Aryan, M., Frean, M. R., & Lewis, J. P. (2018) · International Conference on Image and Vision Computing New Zealand (IVCNZ) · doi:10.1109/ivcnz.2018.8634788
Bayesian Approach to the Classification of BMI Time Series Data from Babyhood to Junior High School Age of Japanese Children
Aida, T., & Haga, C. (2019) · IEEE International Conference on Big Data Analytics (ICBDA) · doi:10.1109/icbda.2019.8713200
Modelowanie niepewności krótkoterminowego popytu na energię elektryczną z wykorzystaniem sieci neuronowych i neuronowo-rozmytych
Bartkiewicz, W. (2013) · Wydawnictwo Uniwersytetu Łódzkiego eBooks · doi:10.18778/7525-926-1
Repulsive Mixture Models of Exponential Family PCA for ClusteringPreprint
Qiao, M., Liu, T., Yu, J., Bian, W., & Tao, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.03112
Implementation of a Neural Network for retrieving atmospheric parameters from remote sensing
Baron, P., Mendrok, J., Kasai, Y. J., & 康子, 笠. (2007)
Assessing Local Generalization Capability in Deep Models.
Wang, H., Keskar, N. S., Xiong, C., & Socher, R. (2020) · International Conference on Artificial Intelligence and Statistics
Investigation of the process in the construction of 3D models from 2D sketches
Sun, Y. (2006) · doi:10.32657/10356/6418
Making Decisions Under High Stakes: Trustworthy and Expressive Bayesian Deep Learning
Yang, W. (2020) · Digital Access to Scholarship at Harvard (DASH) (Harvard University)
Parsimonious InferencePreprint
Duersch, J. A., & Catanach, T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.02165
Bayesian Deep Learning Hyperparameter Search for Robust Function Mapping to Polynomials with NoisePreprint
Harilal, N., Bhatia, U., & Ganguly, A. R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.12532
Hybrid Statistical and Deep Learning Models for Diagnosis and Prognosis in Manufacturing Systems
Ansari, M. S. A. M. I. (2020) · Spectrum Research Repository (Concordia University)
Evaluating and Diagnosing Convergence for Stochastic Gradient Langevin Dynamics
Hernández, S., & López, J. L. (2021) · International Conference of the Chilean Computer Science Society (SCCC) · doi:10.1109/sccc54552.2021.9650355
Report on 1909.03081v3
Nachman, B. (2020) · SciPost Physics · doi:10.21468/scipost.report.1725
References
(2011) · Wiley eBooks · doi:10.1002/9781119952954.refs
Kernel Methods
Signoretto, M., & Suykens, J. A. K. (2015) · Springer eBooks · doi:10.1007/978-3-662-43505-2_32
References
(2009) · CRC Press eBooks · doi:10.1201/9781439803707.bmatt
Report on 1909.03081v1
Nachman, B. (2020) · SciPost Physics · doi:10.21468/scipost.report.1438
References
(2017) · Elsevier eBooks · doi:10.1016/b978-0-08-098235-9.16001-x
Bibliography
(2018) · Elsevier eBooks · doi:10.1016/b978-0-12-804566-4.00023-1
Report on 1909.03081v1
Nachman, B. (2020) · SciPost Physics · doi:10.21468/scipost.report.1443
Referências
(2014) · Elsevier eBooks · doi:10.1016/b978-85-352-7197-3.00006-6
The Priority Heuristic: Making Choices Without Trade-offs
Brandstätter, E., Gigerenzer, G., & Hertwig, R. (2011) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780199744282.003.0007 · Cited by 3
Neural Networks
Titterington, M. (2014) · Wiley eBooks · doi:10.1002/9781118445112.stat00448
References
(2017) · Elsevier eBooks · doi:10.1016/b978-0-12-813526-6.09998-0
Neural Networks
Titterington, M. (2005) · Wiley eBooks · doi:10.1002/0471667196.ess1112.pub3
Report on 1909.03081v1
Nachman, B. (2019) · SciPost Physics · doi:10.21468/scipost.report.1294
Deep Gaussian processes and infinite neural networks for the analysis of EEG signals in Alzheimer’s diseases
Román, K., Cumbicus, A., Infante, S., & Fonseca-Delgado, R. (2022) · Revista de Matemática Teoría y Aplicaciones · doi:10.15517/rmta.v29i2.48885
Report on 2104.04543v1
(2021) · SciPost Physics · doi:10.21468/scipost.report.3020
Development of A Comprehensive Quality Measure for Identification of Nonlinear Hybrid Systems
Madary, A., & Momeni, H. (2022) · Journal of Control · doi:10.52547/joc.16.1.49
References
(2023) · Elsevier eBooks · doi:10.1016/b978-0-44-319035-3.00033-1
Report on 2110.13632v2
(2022) · SciPost Physics · doi:10.21468/scipost.report.4594
Report on 2110.13632v2
(2022) · SciPost Physics · doi:10.21468/scipost.report.5008
Report on 2206.14831v1
(2022) · SciPost Physics Core · doi:10.21468/scipost.report.5675
Report on 2206.14831v1
(2022) · SciPost Physics Core · doi:10.21468/scipost.report.5787
Law of Large Numbers for Bayesian two-layer Neural Network trained with Variational InferencePreprint
Descours, A., Huix, T., Guillin, A., Michel, M., Moulines, É., & Nectoux, B. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2307.04779
Report on 2210.00019v3
(2023) · SciPost Physics · doi:10.21468/scipost.report.6919
Report on 2210.00019v2
(2022) · SciPost Physics · doi:10.21468/scipost.report.6234
Report on 2210.00019v2
(2022) · SciPost Physics · doi:10.21468/scipost.report.6217
Adaptive Feature Fusion Based Motion-Aware Network For Moving Object SegmentationPreprint
Wu, Q., Yu, J., Xu, B., Zhou, H., & Guo, G. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4606904
Layer Ensembles
Oleksiienko, I., & Iosifidis, A. (2023) · Machine learning for signal processing ... · doi:10.1109/mlsp55844.2023.10286005
Report on scipost_202302_00018v1
(2023) · SciPost Physics Core · doi:10.21468/scipost.report.7102
Report on scipost_202302_00018v1
(2023) · SciPost Physics Core · doi:10.21468/scipost.report.7069
Simultaneous Bayesian Clustering and Model Selection with Mixture of Robust Factor Analyzers
Feng, S., Xie, W., & Nie, Y. (2024) · Mathematics · doi:10.3390/math12071091
Dual automatic relevance determination for linear latent variable models and its application to calcium imaging data analysisPreprint
Li, Y., Funamizu, A., Kühn, B., & Doya, K. (2024) · Research Square · doi:10.21203/rs.3.rs-4242219/v1
Implicit generative prior for Bayesian neural networks
Liu, Y., & Wang, X. (2024) · The Annals of Applied Statistics · doi:10.1214/24-aoas1908 · Cited by 2
Learning in parametric modeling: basic concepts and directions
Theodoridis, S. (2025) · Elsevier eBooks · doi:10.1016/b978-0-44-329238-5.00009-3
Extensions of the Neural Network Models into Applications and Comparisons with General Linear Models
Wang, E. Y. (2021) · doi:10.12794/metadc1833526
A Bayesian Investment Analyst on the Johannesburg Stock Exchange
Mongwe, W. T., Mbuvha, R., & Marwala, T. (2025) · Springer eBooks · doi:10.1007/978-3-031-88431-3_13
Bayesian Detection of Unauthorized Expenditure Using Langevin and Hamiltonian Monte Carlo
Mongwe, W. T., Mbuvha, R., & Marwala, T. (2025) · Springer eBooks · doi:10.1007/978-3-031-88431-3_9
Static and Dynamic Nested Sampling for Yield Curve Model Selection
Mongwe, W. T., Mbuvha, R., & Marwala, T. (2025) · Springer eBooks · doi:10.1007/978-3-031-88431-3_12
Bayesian Detection of Recovery on Charged-Off Loan Accounts
Mongwe, W. T., Mbuvha, R., & Marwala, T. (2025) · Springer eBooks · doi:10.1007/978-3-031-88431-3_7
Bayesian Neural Network Inference of Motor Insurance Claims
Mongwe, W. T., Mbuvha, R., & Marwala, T. (2025) · Springer eBooks · doi:10.1007/978-3-031-88431-3_10
BaLNet: A probabilistic Bayesian neural network for solar wind speed forecasting
Cobos, M., Soriano, M. F., & Barrero, D. F. (2025) · Journal of Space Weather and Space Climate · doi:10.1051/swsc/2025029 · Cited by 1
Bayesian Federated Learning for Continual Training
Milasheuski, U., Barbieri, L., Kianoush, S., Nicoli, M., & Savazzi, S. (2025) · IEEE Statistical Signal Processing Workshop (SSP) · doi:10.1109/ssp64130.2025.11073336
HEAL: Brain-Inspired H yperdimensional E fficient A ctive L earning
Ni, Y., Zou, Z., Huang, W., Chen, H., Chung, W. Y., Cho, S., Krishnan, R., Mercati, P., & Imani, M. (2025) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2025.3617929 · Cited by 7
Decoding Mouse Visual Tasks via Hierarchical Neural-Information Gradients
Feng, J., Feng, X., Luo, Y., & Li, J. (2025) · Mathematics · doi:10.3390/math14010031
Predictive Science: A Quest for the Holy
Patra, A., Faghihi, D., Jha, P. K., Cao, L., & Maupin, K. A. (2025) · Computing in Science & Engineering · doi:10.1109/mcse.2025.3622975
Active reward learning and iterative trajectory improvement from comparative language feedback
Hirota, E., Yang, Z., Hiranaka, A., Jun, M., Tien, J., Russell, S., Dragan, A. D., & Bıyık, E. (2025) · The International Journal of Robotics Research · doi:10.1177/02783649251394742 · Cited by 9
Joint Range-Angle Estimation for FDA–MIMO Radar: A Probabilistic Tensor Factorization Approach
Zhang, X., Yan, J. X., Fu, Y., Guo, J., & Wang, W.-Q. (2026) · IEEE Transactions on Aerospace and Electronic Systems · doi:10.1109/taes.2026.3664841
Bayesian Smoothing and Variable Selection Using Variational Automatic Relevance Determination
Liu, Z., Saha, D., & Liang, F. (2026) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.2026.2619519
Central limit theorem for bayesian neural network trained with variational inference
Descours, A., Huix, T., Moulines, É., Guillin, A., Michel, M., & Nectoux, B. (2026) · Stochastic Partial Differential Equations Analysis and Computations · doi:10.1007/s40072-025-00399-4
Structural Learning of Neural Networks
Wolinski, P. (2020) · theses.fr (ABES) · doi:10.70675/c3a1f1e1zbb69z4628zb95fz3433a1c80828 · Cited by 3
Improving Scalability and Inference in Probabilistic Deep Models
Rossi, S. (2022) · theses.fr (ABES) · doi:10.70675/807ea578zd70dz4898zb642zb1cdc00dcd86
Advancing Bayesian Deep Learning : Sensible Priors and Accelerated Inference
Tran, B.-H. (2023) · theses.fr (ABES) · doi:10.70675/760dcd03z2d6ez44a9zbe4fzd88426b2744b
Variational Inference : theory and large scale applications.
Huix, T. (2024) · doi:10.70675/71d30b2bz5caaz4652z9588z24718408fd9a
How to trust learned loop amplitudes
Bahl, H., Braun, J., Heinrich, G., Plehn, T., & Revelli, R. (2026) · SciPost Physics · doi:10.21468/scipostphys.20.6.165

Authors: Mackay, D. J. C.
Affiliations: University of Cambridge (GB)
Corresponding author: Mackay, David J C
ORCID: Mackay, David J C
Venue: Network Computation in Neural Systems
Publisher: Informa UK Limited
Published: 1995-01
Volume: 6
Issue: 3
Pages: 469-505
ISSN: 0954-898X (ISSN-L)
1361-6536
DOI: 10.1088/0954-898x/6/3/011
OpenAlex: W2139701068
Cited by: 915
References: 52
Citations by year: 2026: 14 · 2025: 33 · 2024: 28 · 2023: 40 · 2022: 50 · 2021: 39 · 2020: 58 · 2019: 40 · 2018: 36 · 2017: 27 · 2016: 17 · 2015: 36 · 2014: 24 · 2013: 29 · 2012: 39
FWCI: 13.31
Type: journal article
Language: en
Open access: closed
Locations: Network Computation in Neural Systems (publishedVersion)
Cambridge University Engineering Department Publications Database (submittedVersion)
Topics: Physical Sciences › Computer Science › Artificial Intelligence › Gaussian Processes and Bayesian Inference
Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
Physical Sciences › Computer Science › Artificial Intelligence › Bayesian Modeling and Causal Inference
Keywords: Bayesian neural networks, supervised learning, Gaussian approximation, model selection, model space search, parameter inference, adaptive networks, noise models, regularization, weight decay

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Probable networks and plausible predictions — a review of practical Bayesian methods for supervised neural networks

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks