The DOI matches its Crossref record; 63 of the 63 references it cites were found in a registry or an index.
Bayesian approach for neural networks—review and case studies
Abstract
We give a short review on the Bayesian approach for neural network learning and demonstrate the advantages of the approach in three real applications. We discuss the Bayesian approach with emphasis on the role of prior knowledge in Bayesian models and in classical error minimization approaches. The generalization capability of a statistical model, classical or Bayesian, is ultimately based on the prior assumptions. The Bayesian approach permits propagation of uncertainty in quantities which are unknown to other assumptions in the model, which may be more generally valid or easier to guess in the problem. The case problem studied in this paper include a regression, a classification, and an inverse problem. In the most thoroughly analyzed regression problem, the best models were those with less restrictive priors. This emphasizes the major advantage of the Bayesian approach, that we are not forced to guess attributes that are unknown, such as the number of degrees of freedom in the model, non-linearity of the model with respect to each input variable, or the exact form for the distribution of the model residuals.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Bayesian Neural Networks (BNNs) [68,71,72] are stochastic ANNs trained using Bayesian inference.”
From Explainable AI in medical imaging: An overview for clinical practitioners – Beyond saliency-based XAI approaches (2023) · cites this paper as [71]
In the citing paper
“Requires complex modifications to training process”
From Explainable AI in medical imaging: An overview for clinical practitioners – Beyond saliency-based XAI approaches (2023) · cites this paper as [71]
In the citing paper
“Intrinsically interpretable”
From Explainable AI in medical imaging: An overview for clinical practitioners – Beyond saliency-based XAI approaches (2023) · cites this paper as [71]
Citations
Cites
Gelman, A., Carlin, J. B., Stern, H. S., & Rubin, D. B. (2003) · doi:10.1201/9780429258480 · Cited by 27,276
Self-citationClassification and Regression Trees.
Gordon, A. D., Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1984) · Biometrics · doi:10.2307/2530946 · Cited by 23,867
Classification and Regression Trees.
Ryzin, J. V., Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1986) · Journal of the American Statistical Association · doi:10.2307/2288003 · Cited by 21,030
Neural networks for pattern recognition
Bishop, C. (1994) · Choice Reviews Online · doi:10.5860/choice.31-5500 · Cited by 18,537
Stochastic relaxation, gibbs distributions, and the bayesian restoration of images.
Geman, S., & Geman, D. (1984) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.1984.4767596 · Cited by 17,949
Monte Carlo sampling methods using Markov chains and their applications.
Hastings, W. K. (1970) · Biometrika · doi:10.1093/biomet/57.1.97 · Cited by 15,195
Advances in neural information processing systems 7
(1996) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(96)90463-0 · Cited by 14,759
Markov Chain Monte Carlo in Practice
Gilks, W. R., Richardson, S. T., & Spiegelhalter, D. J. (1995) · Chapman & Hall/CRC eBooks · doi:10.1201/b14835 · Cited by 7,927
Jeffreys, H., Moran, P. A. P., & Chatfield, C. (1922) · Nature · doi:10.1038/109132a0 · Cited by 7,877
Statistical Decision Theory and Bayesian Analysis
Berger, J. O. (1985) · Springer Series in Statistics · doi:10.1007/978-1-4757-4286-2 · Cited by 7,722
Monte Carlo Statistical Methods
Kim, H., Robert, C. P., & Casella, G. (2000) · Technometrics · doi:10.2307/1270959 · Cited by 5,596
Lohrli, A. (1985) · Notes and Queries · doi:10.1093/notesj/32.3.377-a · Cited by 4,372
Bayesian Learning for Neural Networks
Neal, R. M. (1996) · Lecture Notes in Statistics · doi:10.1007/978-1-4612-0745-0 · Cited by 4,276
Duane, S., Kennedy, A. D., Pendleton, B. J., & Roweth, D. (1987) · Physics Letters B · doi:10.1016/0370-2693(87)91197-x · Cited by 4,081
Approximate Statistical Tests for Comparing Supervised Classification Learning Algorithms
Dietterich, T. G. (1998) · Neural Computation · doi:10.1162/089976698300017197 · Cited by 3,840
An Introduction to Variational Methods for Graphical Models
Jordan, M. I., Ghahramani, Z., Jaakkola, T. S., & Saul, L. K. (1999) · Machine Learning · doi:10.1023/a:1007665907178 · Cited by 3,791
A practical Bayesian framework for backpropagation networks
MacKay, D. J. C. (1992) · Neural Computation · doi:10.1162/neco.1992.4.3.448 · Cited by 2,917
Stastical Decision Theory and Bayesian Analysis.
Ghosh, M., & Berger, J. O. (1988) · Journal of the American Statistical Association · doi:10.2307/2288950 · Cited by 2,901
The Predictive Sample Reuse Method with Applications
Geisser, S. (1975) · Journal of the American Statistical Association · doi:10.1080/01621459.1975.10479865 · Cited by 2,748
Jordan, M. I. (1998) · Springer eBooks · doi:10.1007/978-94-011-5014-9 · Cited by 2,449
Kass, R. E., & Raftery, A. E. (1995) · Journal of the American Statistical Association · doi:10.2307/2291091 · Cited by 2,359
Computing Bayes Factors by Combining Simulation and Asymptotic Approximations
DiCiccio, T., Kass, R., Raftery, A., & Wasserman, L. (1997) · Journal of the American Statistical Association · doi:10.1080/01621459.1997.10474045 · Cited by 2,029
Neural networks for pattern recognition
Marshall, J. A. (1995) · Neural Networks · doi:10.1016/0893-6080(95)90002-0 · Cited by 1,888
Markov Chain Monte Carlo in Practice
Gasparini, M., Gilks, W. R., Richardson, S. T., & Spiegelhalter, D. J. (1997) · Technometrics · doi:10.2307/1271145 · Cited by 1,881
Neural Network Ensembles, Cross Validation, and Active Learning
Krogh, A., & Vedelsby, J. (1994) · Cited by 1,838
The Lack of A Priori Distinctions Between Learning Algorithms
Wolpert, D. H. (1996) · Neural Computation · doi:10.1162/neco.1996.8.7.1341 · Cited by 1,652
The Selection of Prior Distributions by Formal Rules
Kass, R. E., & Wasserman, L. (1996) · Journal of the American Statistical Association · doi:10.1080/01621459.1996.10477003 · Cited by 1,291
Smith, J. Q., Bernardo, J. M., Berger, J. O., Dawid, A. P., & Smith, A. F. M. (1993) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.2307/2982740 · Cited by 1,098
No Free Lunch Theorems for SearchPreprint
Wolpert, D. H., & Macready, W. G. (1995) · RePEc: Research Papers in Economics · Cited by 927
Mackay, D. J. C. (1995) · Network Computation in Neural Systems · doi:10.1088/0954-898x/6/3/011 · Cited by 915
Bayesian treatment of the independent student-t linear model
Geweke, J. F. (1993) · Journal of Applied Econometrics · doi:10.1002/jae.3950080504 · Cited by 527
Convergence assessment techniques for Markov chain Monte Carlo
Brooks, S. P., & Roberts, G. O. (1998) · Statistics and Computing · doi:10.1023/a:1008820505350 · Cited by 473
Evaluation of gaussian processes and other methods for non-linear regression
Hinton, G. E., & Rasmussen, C. E. (1997) · Library and Archives Canada (Government of Canada) · Cited by 430
Inference and monitoring convergence
Gilks, W. R., Richardson, S. T., & Spiegelhalter, D. J. (1995) · Chapman & Hall/CRC eBooks · doi:10.1201/b14835-13 · Cited by 412
The Predictive Sample Reuse Method with Applications
Geisser, S. (1975) · Journal of the American Statistical Association · doi:10.2307/2285815 · Cited by 386
Buntine, W., & Weigend, A. S. (1991) · Complex Systems · Cited by 357
Neural networks and machine learning
Bishop, C. (1998) · Springer eBooks · Cited by 316
On the Development of Reference Priors*
Berger, J. O., & Bernardo, J. M. (1992) · doi:10.1093/oso/9780198522669.003.0003 · Cited by 290
The Selection of Prior Distributions by Formal Rules
Kass, R. E., & Wasserman, L. (1996) · Journal of the American Statistical Association · doi:10.2307/2291752 · Cited by 233
Sequential Monte Carlo Methods to Train Neural Network Models
Freitas, J., Niranjan, M., Gee, A. H., & Arnaud Doucet, B. (2000) · Neural Computation · doi:10.1162/089976600300015664 · Cited by 213
Model determination using sampling-based methods
Gilks, W. R., Richardson, S. T., & Spiegelhalter, D. (1995) · Chapman & Hall/CRC eBooks · doi:10.1201/b14835-14 · Cited by 196
A review of Bayesian neural networks with an application to near infrared spectroscopy
Thodberg, H. H. (1996) · IEEE Transactions on Neural Networks · doi:10.1109/72.478392 · Cited by 184
Ensemble learning in Bayesian neural networks
Barber, D., & Bishop, C. (1998) · Cited by 166
Issues in Bayesian Analysis of Neural Network Models
Müller, P., & Insua, D. R. (1998) · Neural Computation · doi:10.1162/089976698300017737 · Cited by 115
Electrical impedance tomography with basis constraints
Vauhkonen, M. J., Kaipio, J. P., Somersalo, E., & Karjalainen, P. A. (1997) · Inverse Problems · doi:10.1088/0266-5611/13/2/020 · Cited by 112
The Existence of A Priori Distinctions Between Learning Algorithms
Wolpert, D. H. (1996) · Neural Computation · doi:10.1162/neco.1996.8.7.1391 · Cited by 109
Bayesian neural networks for classification: how useful is the evidence framework?
Penny, W. D., & Roberts, S. (1999) · Neural Networks · doi:10.1016/s0893-6080(99)00040-4 · Cited by 104
Curvature-driven smoothing: a learning algorithm for feedforward networks
Bishop, C. M. (1993) · IEEE Transactions on Neural Networks · doi:10.1109/72.248466 · Cited by 91
Evaluation Of Gaussian Processes And Other Methods For Non-Linear Regression
Rasmussen, C. E. (1996) · Library and Archives Canada (Government of Canada) · Cited by 65
Assessing Relevance Determination Methods Using DELVE
Neal, R. M. (1998) · Cited by 55
Information about Hyperparameters in Hierarchical Models
Goel, P. K., & DeGroot, M. H. (1981) · Journal of the American Statistical Association · doi:10.1080/01621459.1981.10477620 · Cited by 54
Regression with Input-Dependent Noise: A Bayesian Treatment
Bishop, C., & Quazaz, C. S. (1996) · Aston Publications Explorer (Aston University) · Cited by 41
Lemm, J. C. (2003) · Johns Hopkins University Press eBooks · doi:10.1353/book.21310 · Cited by 31
Bayesian Neural Network to Solve the Inverse Problem in Electrical Impedance Tomography
Lampinen, J., Vehtari, A., & Leinonen, K. (1999) · Cited by 20
Self-citationInformation About Hyperparamters in Hierarchical Models
Goel, P. K., & DeGroot, M. H. (1981) · Journal of the American Statistical Association · doi:10.2307/2287059 · Cited by 20
On MCMC sampling in Bayesian MLP neural networks
Vehtari, A., Särkkä, S., & Lampinen, J. (2000) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2000.857855 · Cited by 19
Self-citationUsing Bayesian neural networks to classify forest scenes
Vehtari, A., Heikkonen, J., Lampinen, J., & Juujärvi, J. (1998) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.325800 · Cited by 16
Self-citationPrior Information and Generalized Questions
Lemm, J. C. (1996) · Cited by 11
Using Bayesian neural networks to classify segmented images
Vivarelli, F. (1997) · doi:10.1049/cp:19970738 · Cited by 9
On Bayesian Model Asessment and Choice Using Cross-Validation Predictive Densities
Vehtari, A., & Lampinen, J. (2001) · Cited by 8
Self-citationBayesian neural networks with correlating residuals
Vehtari, A., & Lampinen, J. (2003) · 2022 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.1999.832623 · Cited by 6
Self-citationUsing Background Knowledge in Multilayer Perceptron Learning
Lampinen, J., & Selonen, A. (1997) · Cited by 5
Self-citationEmpirical Evaluation of Bayesian Sampling for Neural Classifiers
Husmeier, D., Penny, W. D., & Roberts, S. (1998) · Perspectives in neural computing · doi:10.1007/978-1-4471-1599-1_47 · Cited by 3
Research works printed but not linked: 1
- MacKay, 1994, Bayesian non-linear modelling for the prediction competition, ASHRAE Transactions, 100, 1053
Cited by
Sorted by citations
A review on lithium-ion battery ageing mechanisms and estimations for automotive applications
Barré, A., Deguilhem, B., Grolleau, S., Gérard, M., Suard, F., & Riu, D. (2013) · Journal of Power Sources · doi:10.1016/j.jpowsour.2013.05.040 · Cited by 1,758
Jaguemont, J., Boulon, L., & Dubé, Y. (2016) · Applied Energy · doi:10.1016/j.apenergy.2015.11.034 · Cited by 1,188
Hands-On Bayesian Neural Networks—A Tutorial for Deep Learning Users
Jospin, L. V., Laga, H., Boussaid, F., Buntine, W., & Bennamoun, M. (2022) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2022.3155327 · Cited by 879
Bio inspired computing – A review of algorithms and scope of applications
Kar, A. K. (2016) · Expert Systems with Applications · doi:10.1016/j.eswa.2016.04.018 · Cited by 578
Measurement of the properties of a Higgs boson in the four-lepton final state
Chatrchyan, S., Khachatryan, V., Sirunyan, A. M., Tumasyan, A., Adam, W., Bergauer, T., Dragicevic, M., Erö, J., Fabjan, C. W., Friedl, M., et al. (2014) · Physical review. D. Particles, fields, gravitation, and cosmology/Physical review. D. Particles and fields · doi:10.1103/physrevd.89.092007 · Cited by 542
Metadata
- Authors
- Lampinen, J., & Vehtari, A.
- Corresponding author
- Lampinen, Jouko
- ORCID
- Vehtari, Aki
- Venue
- Neural Networks
- Publisher
- Elsevier BV
- Published
- 2001-04
- Volume
- 14
- Issue
- 3
- Pages
- 257-274
- ISSN
- 0893-6080 (ISSN-L)
- OpenAlex
- W2040355924
- Cited by
- 392
- References
- 62
- Citations by year
- 2026: 27 · 2025: 38 · 2024: 35 · 2023: 32 · 2022: 21 · 2021: 15 · 2020: 16 · 2019: 12 · 2018: 14 · 2017: 8 · 2016: 15 · 2015: 9 · 2014: 12 · 2013: 7 · 2012: 3
- FWCI
- 10.39
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Neural Networks (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Tekes
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Gaussian Processes and Bayesian Inference
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Keywords
- Bayesian neural networks, prior knowledge, error propagation, generalization, regression analysis, classification, inverse problems, model complexity, residual distribution, error reduction
- MeSH terms
- Algorithms, Bayes Theorem, Neural Networks, Computer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
