The DOI matches its OpenAlex record; 68 of the 68 references it cites were found in a registry or an index.
Bayesian methods for adaptive models
Abstract
The Bayesian framework for model comparison and regularisation is demonstrated by studying interpolation and classification problems modelled with both linear and non-linear models. This framework quantitatively embodies `Occam's razor'. Over-complex and under-regularised models are automatically inferred to be less probable, even though their flexibility allows them to fit the data better. When applied to `neural networks', the Bayesian framework makes possible (1) objective comparison of solutions using alternative network architectures; (2) objective stopping rules for network pruning or growing procedures; (3) objective choice of type of weight decay terms (or regularisers); (4) on--line techniques for optimising weight decay (or regularisation constant) magnitude; (5) a measure of the effective number of well--determined parameters in a model; (6) quantified estimates of the error bars on network parameters and on network output. In the case of classification models, it is sho...
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Estimating the Dimension of a Model
Schwarz, G. (1978) · The Annals of Statistics · doi:10.1214/aos/1176344136 · Cited by 47,033
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892
(1973) · CERN Document Server (European Organization for Nuclear Research) · Cited by 12,895
Introduction to Statistical Pattern Recognition
(1990) · Elsevier eBooks · doi:10.1016/c2009-0-27872-x · Cited by 11,031
Jeffreys, H., Moran, P. A. P., & Chatfield, C. (1922) · Nature · doi:10.1093/oso/9780198503682.001.0001 · Cited by 7,878
Statistical Decision Theory and Bayesian Analysis
Berger, J. O. (1985) · Springer Series in Statistics · doi:10.1007/978-1-4757-4286-2 · Cited by 7,722
Modeling by shortest data description
Rissanen, J. (1978) · Automatica · doi:10.1016/0005-1098(78)90005-5 · Cited by 6,011
MacKay, D. J. C. (1992) · Neural Computation · doi:10.1162/neco.1992.4.3.415 · Cited by 4,388
Self-citationBayesian Inference in Statistical Analysis.
Kadane, J. B., Box, G. E. P., & Tiao, G. C. (1975) · Journal of the American Statistical Association · doi:10.2307/2285412 · Cited by 3,824
AN ESSAY TOWARDS SOLVING A PROBLEM IN THE DOCTRINE OF CHANCES
BAYES (1958) · Biometrika · doi:10.1093/biomet/45.3-4.296 · Cited by 2,911
Stastical Decision Theory and Bayesian Analysis.
Ghosh, M., & Berger, J. (1988) · Journal of the American Statistical Association · doi:10.2307/2288950 · Cited by 2,905
Fedorov, V. V., Studden, W. J., & Klimko, E. M. (1972) · Cited by 2,092
On a measure of the information provided by an experiment
Lindley, D. V. (1956) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177728069 · Cited by 1,509
Computational vision and regularization theory
Poggio, T., Torre, V., & Koch, C. (1985) · Nature · doi:10.1038/317314a0 · Cited by 1,412
Probability, Frequency and Reasonable Expectation
Cox, R. T. (1946) · American Journal of Physics · doi:10.1119/1.1990764 · Cited by 1,308
Bridle, J. S. (1990) · Neurocomputing · doi:10.1007/978-3-642-76153-9_28 · Cited by 1,237
Information-based objective functions for active data selection
MacKay, D. J. C. (1992) · Neural Computation · doi:10.1162/neco.1992.4.4.590 · Cited by 1,221
Self-citationStatistical Predictor Identification
Akaike, H. (1998) · Springer Series in Statistics · doi:10.1007/978-1-4612-1694-0_11 · Cited by 1,213
Learning and relearning in Boltzmann machines
Hinton, G. E., & Sejnowski, T. J. (2001) · Graphical Models · doi:10.7551/mitpress/3349.003.0005 · Cited by 1,037
An Information Measure for Classification
Wallace, C. S., & Boulton, D. M. (1968) · The Computer Journal · doi:10.1093/comjnl/11.2.185 · Cited by 933
The Evidence Framework Applied to Classification Networks
MacKay, D. J. C. (1992) · Neural Computation · doi:10.1162/neco.1992.4.5.720 · Cited by 722
Self-citationOckham's Razor and Bayesian Analysis
Jefferys, W. H., & Berger, J. O. (1992) · American Scientist · Cited by 706
Statistical mechanics of learning from examples
Seung, H. S., Sompolinsky, H., & Tishby, N. (1992) · Physical Review A · doi:10.1103/physreva.45.6056 · Cited by 532
Estimation and Inference by Compact Coding
Wallace, C. S., & Freeman, P. R. (1987) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1987.tb01695.x · Cited by 523
Sequential updating of conditional probabilities on directed graphical structures
Spiegelhalter, D. J., & Lauritzen, S. L. (1990) · Networks · doi:10.1002/net.3230200507 · Cited by 521
Bayes Factors and Choice Criteria for Linear Models
Smith, A. F. M., & Spiegelhalter, D. J. (1980) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1980.tb01122.x · Cited by 333
On the Asymptotic Behaviour of Posterior Distributions
Walker, A. M. (1969) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1969.tb00767.x · Cited by 332
Improving the convergence of back-propagation learning with second-order methods
Becker, S., & LeCun, Y. (1989) · Cited by 331
Bayesian modeling of uncertainty in low-level vision
Szeliski, R. (1990) · International Journal of Computer Vision · doi:10.1007/bf00126502 · Cited by 303
Developments in Maximum Entropy Data Analysis
Gull, S. F. (1989) · Maximum Entropy and Bayesian Methods · doi:10.1007/978-94-015-7860-8_4 · Cited by 262
A Bayesian approach to some outlier problems
BOX, G. E. P., & TIAO, G. C. (1968) · Biometrika · doi:10.1093/biomet/55.1.119 · Cited by 252
Bayesian Inductive Inference and Maximum Entropy
Gull, S. F. (1988) · Maximum-Entropy and Bayesian Methods in Science and Engineering · doi:10.1007/978-94-009-3049-0_4 · Cited by 246
Learning from hints in neural networks
Abu-Mostafa, Y. S. (1990) · Journal of Complexity · doi:10.1016/0885-064x(90)90006-y · Cited by 241
Transforming Neural-Net Output Levels to Probability Distributions
Denker, J. S., & LeCun, Y. (1990) · Cited by 241
A further look at robustness via Bayes's theorem
BOX, G. E. P., & TIAO, G. C. (1962) · Biometrika · doi:10.1093/biomet/49.3-4.419 · Cited by 234
Common Structure of Smoothing Techniques in Statistics
Titterington, D. M. (1985) · International Statistical Review / Revue Internationale de Statistique · doi:10.2307/1402932 · Cited by 219
Accelerated learning in layered neural networks
Solla, S. A., Levin, E., & Fleisher, M. (1988) · Complex Systems · Cited by 213
Learning algorithms and probability distributions in feed-forward and feed-back networks
Hopfield, J. J. (1987) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.84.23.8429 · Cited by 189
Bayesian Methods: General Background
Jaynes, E. T. (1986) · Maximum Entropy and Bayesian Methods in Applied Statistics · doi:10.1017/cbo9780511569678.003 · Cited by 185
From Laplace to Supernova SN 1987A: Bayesian Inference in Astrophysics
Loredo, T. J. (1990) · Maximum Entropy and Bayesian Methods · doi:10.1007/978-94-009-0683-9_6 · Cited by 168
Neural net algorithms that learn in polynomial time from examples and queries
Baum, E. B. (1991) · IEEE Transactions on Neural Networks · doi:10.1109/72.80287 · Cited by 168
Consistent inference of probabilities in layered networks: predictions and generalizations
Tishby, Levin, & Solla (1989) · International Joint Conference on Neural Networks · doi:10.1109/ijcnn.1989.118274 · Cited by 161
Query-based learning applied to partially trained multilayer perceptrons
Hwang, J., Choi, J. J., Oh, S., & Marks, R. J. (1991) · IEEE Transactions on Neural Networks · doi:10.1109/72.80299 · Cited by 152
The Vapnik-Chervonenkis Dimension: Information versus Complexity in Learning
Abu‐Mostafa, Y. S. (1989) · Neural Computation · doi:10.1162/neco.1989.1.3.312 · Cited by 151
Adaptive optics for array telescopes using neural-network techniques
Angel, J. R. P., Wizinowich, P. L., Lloyd‐Hart, M., & Sandler, D. G. (1990) · Nature · doi:10.1038/348221a0 · Cited by 150
Exact Calculation of the Hessian Matrix for the Multilayer Perceptron
Bishop, C. (1992) · Neural Computation · doi:10.1162/neco.1992.4.4.494 · Cited by 146
Note on generalization, regularization and architecture selection in nonlinear learning systems
Moody, J. E. (2002) · Machine learning for signal processing ... · doi:10.1109/nnsp.1991.239541 · Cited by 138
Nowlan, S. J. (1991) · Cited by 124
Bretthorst, G. L. (1990) · Journal of Magnetic Resonance (1969) · doi:10.1016/0022-2364(90)90289-l · Cited by 117
Bayesian Statistics, A Review.
Dickey, J., & Lindley, D. V. (1973) · Journal of the American Statistical Association · doi:10.2307/2284178 · Cited by 117
Bayesian classification theory
Hanson, R., Stutz, J., & Cheeseman, P. C. (1991) · NASA Technical Reports Server (NASA) · Cited by 112
Structural Risk Minimization for Character Recognition
Guyon, I., Vapnik, V. N., Boser, B. E., Bottou, L., & Solla, S. A. (1991) · neural information processing systems · Cited by 105
Skilling, J. (1990) · Springer eBooks · doi:10.1007/978-94-009-0683-9_21 · Cited by 93
Information, Weight of Evidence, the Singularity between Probability Measures and Signal Detection
Osteyee, D. B., & Good, I. J. (1974) · Lecture notes in mathematics · doi:10.1007/bfb0064126 · Cited by 92
Bayesian Solution of Ordinary Differential Equations
Skilling, J. (1992) · Springer eBooks · doi:10.1007/978-94-017-2219-3_2 · Cited by 75
The Eigenvalues of Mega-dimensional Matrices
Skilling, J. (1989) · Maximum Entropy and Bayesian Methods · doi:10.1007/978-94-015-7860-8_48 · Cited by 66
Generalizing Smoothness Constraints from Discrete Samples
Ji, C., Snapp, R. R., & Psaltis, D. (1990) · Neural Computation · doi:10.1162/neco.1990.2.2.188 · Cited by 63
Bayesian Data Analysis: Straight-line fitting
Gull, S. F. (1989) · Springer eBooks · doi:10.1007/978-94-015-7860-8_55 · Cited by 58
A Bayesian Approach to the Importance of Assumptions Applied to the Comparison of Variances
Box, G. E. P., & Tiao, G. C. (1964) · Biometrika · doi:10.1093/biomet/51.1-2.153 · Cited by 52
The use of transinformation in the design of data sampling schemes for inverse problems
Luttrell, S. P. (1985) · Inverse Problems · doi:10.1088/0266-5611/1/3/006 · Cited by 48
Optimisation in the regularisation ill-posed problems
Davies, A. R., & Anderssen, R. (1986) · ANZIAM Journal · doi:10.1017/s0334270000005221 · Cited by 30
Bayesian model selection and minimum description length estimation of auditory-nerve discharge rates
Mark, K. E., & Miller, M. I. (1992) · The Journal of the Acoustical Society of America · doi:10.1121/1.402504 · Cited by 18
Applications of Maximum Entropy Techniques to HST Data
Weir, N. (1991) · European Southern Observatory Conference and Workshop Proceedings · Cited by 16
A statistical approach to learning and generalization in layered neural networks
Levin, E., Tishby, N., & Solla, S. A. (1989) · Conference on Learning Theory · doi:10.5555/93335.93361 · Cited by 16
STONE CIRCLE GEOMETRIES: AN INFORMATION THEORY APPROACH
Patrick, J. D., & Wallace, C. (1982) · Cambridge University Press eBooks · doi:10.1017/cbo9780511898310.016 · Cited by 12
The Role of Priors in Active Bayesian Learning in the Sequential Statistical Decision Framework
El-Gamal, M. A. (1991) · Springer eBooks · doi:10.1007/978-94-011-3460-6_3 · Cited by 9
Lee, W. T., & Tenorio, M. F. (1991) · Purdue e-Pubs (Purdue University System) · Cited by 5
Cited by
A tutorial on support vector regression
Smola, A. J., & Schölkopf, B. (2004) · Statistics and Computing · doi:10.1023/b:stco.0000035301.49549.88 · Cited by 13,387
Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles
Lakshminarayanan, B., Pritzel, A., & Blundell, C. (2016) · arXiv · doi:10.48550/arxiv.1612.01474 · Cited by 3,666
Support Vector Data Description
Tax, D. M. J., & Duin, R. P. W. (2004) · Machine Learning · doi:10.1023/b:mach.0000008084.60811.49 · Cited by 3,512
Nonlinear black-box modeling in system identification: a unified overview
Sjöberg, J., Zhang, Q., Ljung, L., Benveniste, A., Delyon, B., Glorennec, P.-Y., Hjalmarsson, H., & Juditsky, A. (1995) · Automatica · doi:10.1016/0005-1098(95)00120-8 · Cited by 1,942
GAUSSIAN PROCESSES FOR MACHINE LEARNING
SEEGER, M. (2004) · International Journal of Neural Systems · doi:10.1142/s0129065704001899 · Cited by 1,488
Metadata
- Authors
- Mackay, D.
- Published
- 1992-01-01
- OpenAlex
- W2117670920
- Cited by
- 610
- References
- 68
- FWCI
- 11.63
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.