The DOI matches its Crossref record; 74 of the 74 references it cites were found in a registry or an index.
GAUSSIAN PROCESSES FOR MACHINE LEARNING
Open accessAbstract
Gaussian processes (GPs) are natural generalisations of multivariate Gaussian random variables to infinite (countably or continuous) index sets. GPs have been applied in a large number of fields to a diverse range of ends, and very many deep theoretical analyses of various properties are available. This paper gives an introduction to Gaussian processes on a fairly elementary level with special emphasis on characteristics relevant in machine learning. It draws explicit connections to branches such as spline smoothing models and support vector machines in which similar ideas have been investigated. Gaussian process models are routinely used to solve hard machine learning problems. They are attractive because of their flexible non-parametric nature and computational simplicity. Treated within a Bayesian framework, very powerful statistical methods can be implemented which offer valid estimates of uncertainties in our predictions and generic model selection procedures cast as nonlinear optimization problems. Their main drawback of heavy computational scaling has recently been alleviated by the introduction of generic sparse approximations.13,78,31 The mathematical literature on GPs is large and often uses deep concepts which are not required to fully understand most machine learning applications. In this tutorial paper, we aim to present characteristics of GPs relevant to machine learning and to show up precise connections to other "kernel machines" popular in the community. Our focus is on a simple presentation, but references to more detailed sources are provided.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Boyd, S., & Vandenberghe, L. (2004) · doi:10.1017/cbo9780511804441 · Cited by 56,601
(2005) · Statistics for Biology and Health · doi:10.1007/0-387-27255-0_9 · Cited by 20,133
A Tutorial on Support Vector Machines for Pattern Recognition
Burges, C. J. C. (1998) · Data Mining and Knowledge Discovery · doi:10.1023/a:1009715923555 · Cited by 16,429
Press, W. H., Teukolsky, S. A., Vetterling, W. T., & Flannery, B. P. (1994) · Cited by 15,411
Nonlinear Dimensionality Reduction by Locally Linear Embedding
Roweis, S. T., & Saul, L. K. (2000) · Science · doi:10.1126/science.290.5500.2323 · Cited by 14,965
A Global Geometric Framework for Nonlinear Dimensionality Reduction
Tenenbaum, J. B., Silva, V. D., & Langford, J. C. (2000) · Science · doi:10.1126/science.290.5500.2319 · Cited by 13,585
The perceptron: A probabilistic model for information storage and organization in the brain.
Rosenblatt, F. (1958) · Psychological Review · doi:10.1037/h0042519 · Cited by 12,280
Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond
Schölkopf, B., & Smola, A. J. (2001) · The MIT Press eBooks · doi:10.7551/mitpress/4175.001.0001 · Cited by 11,703
Advances in Neural Information Processing Systems 14: Proceedings of the 2001 Conference
(2002) · The MIT Press eBooks · doi:10.7551/mitpress/1120.001.0001 · Cited by 8,890
Cressie, N. (1992) · Terra Nova · doi:10.1111/j.1365-3121.1992.tb00605.x · Cited by 8,823
Nelder, J. A., & Wedderburn, R. W. M. (1972) · Journal of the Royal Statistical Society. Series A (General) · doi:10.2307/2344614 · Cited by 7,194
Cressie, N. A. C. (1993) · Wiley Series in Probability and Statistics · doi:10.1002/9781119115151 · Cited by 6,507
PE, & Billingsley, P. (1996) · Journal of the American Statistical Association · doi:10.2307/2291440 · Cited by 6,481
Probabilistic Outputs for Support vector Machines and Comparisons to Regularized Likelihood Methods
Platt, J. C. (1999) · Cited by 6,385
Advances in kernel methods: support vector learning
Schölkopf, B., Burges, C. J. C., & Smola, A. J. (1999) · doi:10.5555/299094 · Cited by 5,789
Fast Training of Support Vector Machines Using Sequential Minimal Optimization
Platt, J. C. (1998) · Advances in Kernel Methods · doi:10.7551/mitpress/1130.003.0016 · Cited by 5,437
Aronszajn, N. (1950) · Transactions of the American Mathematical Society · doi:10.1090/s0002-9947-1950-0051437-7 · Cited by 5,380
Spline Models for Observational Data.
Muller, H.-G., & Wahba, G. (1991) · Journal of the American Statistical Association · doi:10.2307/2290434 · Cited by 4,928
Matheron, G. (1963) · Economic Geology · doi:10.2113/gsecongeo.58.8.1246 · Cited by 4,845
Bayesian Learning for Neural Networks
Neal, R. M. (1996) · Lecture Notes in Statistics · doi:10.1007/978-1-4612-0745-0 · Cited by 4,276
Spline Models for Observational Data
Wahba, G. (1990) · doi:10.1137/1.9781611970128 · Cited by 4,184
Networks for approximation and learning
Poggio, T., & Girosi, F. (1990) · Proceedings of the IEEE · doi:10.1109/5.58326 · Cited by 3,252
Advances in neural information processing systems 11
Peter Sollich, A. (1999) · Cited by 3,168
A Course in Probability Theory
(1974) · Elsevier eBooks · doi:10.1016/c2009-0-22097-6 · Cited by 2,668
Probability and Random Processes
(2004) · Elsevier eBooks · doi:10.1016/b978-0-12-172651-5.x5000-3 · Cited by 2,535
(1998) · Springer eBooks · doi:10.1007/978-94-011-5014-9 · Cited by 2,418
A statistical approach to some basic mine valuation problems on the Witwatersrand
Krige, D. G. (1951) · Journal of the Southern African Institute of Mining and Metallurgy · Cited by 2,409
Stein, M. L. (1999) · Springer Series in Statistics · doi:10.1007/978-1-4612-1494-6 · Cited by 2,348
McCullagh, P., & Nelder, J. A. (1983) · Springer eBooks · doi:10.1007/978-1-4899-3244-0 · Cited by 1,980
Nonparametric Regression and Generalized Linear Models
Green, P. J., & Silverman, B. W. (1994) · Springer eBooks · doi:10.1007/978-1-4899-4473-3 · Cited by 1,925
Interpolation of Spatial Data: Some Theory for Kriging
Rathbun, S. L., & Stein, M. L. (2000) · Journal of the American Statistical Association · doi:10.2307/2669494 · Cited by 1,916
Advances in Large-Margin Classifiers
(2000) · The MIT Press eBooks · doi:10.7551/mitpress/1113.001.0001 · Cited by 1,869
Probability and Random Processes.
Grimmett, G. R., & Stirzaker, D. R. (2001) · doi:10.1093/oso/9780198572237.001.0001 · Cited by 1,850
Foundations of the Theory of Probability
Brookes, B. C., & Kolmogorov, A. N. (1951) · The Mathematical Gazette · doi:10.2307/3611503 · Cited by 1,691
Hawkes, J., & Billingsley, P. (1987) · Journal of the Royal Statistical Society. Series A (General) · doi:10.2307/2982048 · Cited by 1,631
The intrinsic random functions and their applications
Matheron, G. (1973) · Advances in Applied Probability · doi:10.1017/s0001867800039379 · Cited by 1,498
Smith, J. Q., Bernardo, J. M., Berger, J. O., Dawid, A. P., & Smith, A. F. M. (1993) · Journal of the Royal Statistical Society. Series A (Statistics in Society) · doi:10.2307/2982740 · Cited by 1,097
Convolution kernels on discrete structures
Haussler, D. H. (1999) · Cited by 1,089
Kendall, W. S. (1982) · Bulletin of the London Mathematical Society · doi:10.1112/blms/14.1.59 · Cited by 1,040
Correlation Theory of Stationary and Related Random Functions
Yaglom, A. M. (1987) · Springer series in statistics · doi:10.1007/978-1-4612-4628-2 · Cited by 947
A Correspondence Between Bayesian Estimation on Stochastic Processes and Smoothing by Splines
Kimeldorf, G. S., & Wahba, G. (1970) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177697089 · Cited by 939
A family of algorithms for approximate bayesian inference
Minka, T. P., & Picard, R. W. (2001) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 882
Correlation theory of stationary and related random functions
Iaglom, A. M. (1987) · Springer eBooks · Cited by 769
Bayesian classification with Gaussian processes
Williams, C. K. I., & Barber, D. (1998) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.735807 · Cited by 766
Sparse On-Line Gaussian Processes
Csató, L., & Opper, M. (2002) · Neural Computation · doi:10.1162/089976602317250933 · Cited by 749
Diffusion Kernels on Graphs and Other Discrete Input Spaces
Kondor, R., & Lafferty, J. (2002) · Cited by 725
Curve Fitting and Optimal Design for Prediction
O'Hagan, A. (1978) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1978.tb01643.x · Cited by 661
Prediction with Gaussian Processes: From Linear Regression to Linear Prediction and Beyond
Williams, C. K. I. (1998) · Learning in Graphical Models · doi:10.1007/978-94-011-5014-9_23 · Cited by 659
The connection between regularization operators and support vector kernels
Smola, A. J., Schölkopf, B., & Müller, K.-R. (1998) · Neural Networks · doi:10.1016/s0893-6080(98)00032-x · Cited by 625
Bayesian methods for adaptive models
Mackay, D. (1992) · doi:10.7907/h3a1-wm07 · Cited by 610
Correctness of Belief Propagation in Gaussian Graphical Models of Arbitrary Topology
Weiss, Y., & Freeman, W. T. (2001) · Neural Computation · doi:10.1162/089976601750541769 · Cited by 560
Introduction to Hilbert Space: And the Theory of Spectral Multiplicity
Halmos, P. R. (1998) · Cited by 550
Fast Sparse Gaussian Process Methods: The Informative Vector Machine
Herbrich, R., Lawrence, N. D., & Seeger, M. (2002) · neural information processing systems · Cited by 488
Self-citation(2000) · CrossRef Listing of Deleted DOIs · doi:10.1162/153244303765208377 · Cited by 467
Foundations of the Theory of Probability.
David, F. N., Kolmogorov, A. N., & Khintchine, A. (1952) · Biometrika · doi:10.2307/2332488 · Cited by 440
SPLINE FUNCTIONS AND THE PROBLEM OF GRADUATION
Schoenberg, I. J. (1964) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.52.4.947 · Cited by 439
Girard, A., Rasmussen, C. E., Candela, J. Q., & Murray‐Smith, R. (2002) · Cited by 384
Monte Carlo Implementation of Gaussian Process Models for Bayesian Regression and Classification
Neal, R. M. (1997) · arXiv (Cornell University) · doi:10.48550/arxiv.physics/9701026 · Cited by 375
Information Theory for Continuous Systems
Ihara, S. (1993) · WORLD SCIENTIFIC eBooks · doi:10.1142/1676 · Cited by 358
Derivative Observations in Gaussian Process Models of Dynamic Systems
Solak, E., Murray‐Smith, R., Leith, D. J., et al. (2002) · Cited by 278
Maximum Entropy Discrimination
Jebara, T. (2004) · Springer eBooks · doi:10.1007/978-1-4419-9011-2_3 · Cited by 252
PAC-Bayesian Generalisation Error Bounds for Gaussian Process Classification
Seeger, M. (2003) · Journal of Machine Learning Research · doi:10.1162/153244303765208386 · Cited by 229
Self-citationGaussian Processes for Classification: Mean-Field Algorithms
Opper, M., & Winther, O. (2000) · Neural Computation · doi:10.1162/089976600300014881 · Cited by 209
Probabilistic Kernel Regression Models
Jaakkola, T., & Haussler, D. H. (1999) · Cited by 208
Bayesian Gaussian Process Models: PAC-Bayesian Generalisation Error Bounds and Sparse Approximations
Seeger, M. (2003) · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · Cited by 197
Self-citationComputation with Infinite Neural Networks
Williams, C. K. I. (1998) · Neural Computation · doi:10.1162/089976698300017412 · Cited by 159
Seeger, M. (1999) · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · Cited by 104
Self-citationGaussian regression and optimal finite dimensional linear models
Zhu, H., Williams, C. K. I., Rohwer, R., & Morciniec, M. (1997) · Aston Publications Explorer (Aston University) · Cited by 89
Ensemble Learning for Multi-Layer Networks
Barber, D., & Bishop, C. (1997) · Cited by 71
Covariance Kernels from Bayesian Generative Models
Seeger, M. (2001) · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · Cited by 68
Self-citationProbabilistic Methods for Support Vector Machines
Sollich, P. (1999) · Cited by 63
Modified Cholesky Factorizations in Interior-Point Algorithms for Linear Programming
Wright, S. J. (1999) · SIAM Journal on Optimization · doi:10.1137/s1052623496304712 · Cited by 52
A fast method for calculating the perceptron with maximal stability
Ruján, P. (1993) · Journal de Physique I · doi:10.1051/jp1:1993129 · Cited by 28
Bayesian methods for Support Vector machines and Gaussian processes
Seeger, M. (1999) · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · Cited by 16
Self-citation
Cited by
The WEKA data mining software: an update
Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., & Witten, I. H. (2009) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1656274.1656278 · Cited by 17,750
On hyperparameter optimization of machine learning algorithms: Theory and practice
Yang, L., & Shami, A. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.07.061 · Cited by 3,232
Aleatoric and epistemic uncertainty in machine learning: an introduction to concepts and methods
Hüllermeier, E., & Waegeman, W. (2021) · Machine Learning · doi:10.1007/s10994-021-05946-3 · Cited by 1,834
“Gaussian processes (Seeger, 2004) generalize the Bayesian approach from inference about multivariate (but finite-dimensional) random variables to inference about (infinite-dimensional) functions.”
Machine learning in chemoinformatics and drug discovery
Lo, Y.-C., Rensi, S. E., Torng, W., & Altman, R. B. (2018) · Drug Discovery Today · doi:10.1016/j.drudis.2018.05.010 · Cited by 960
(2024) · Multivariate Biomarker Discovery · doi:10.1017/9781009006767.012 · Cited by 809
Metadata
- Authors
- SEEGER, M.
- Venue
- International Journal of Neural Systems
- Publisher
- World Scientific Pub Co Pte Ltd
- Published
- 2004-04
- Volume
- 14
- Issue
- 02
- Pages
- 69-106
- ISSN
- 0129-0657 (ISSN-L)
- 1793-6462
- OpenAlex
- W1994005439
- Cited by
- 1,488
- References
- 74
- FWCI
- 16.95
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.