Probabilistic machine learning and artificial intelligence
The DOI matches its Crossref record; 102 of the 108 references it cites were found in a registry or an index.
Abstract
How can a machine learn from experience? Probabilistic modelling provides a framework for understanding what learning is, and has therefore emerged as one of the principal theoretical and practical approaches for designing machines that learn from data acquired through experience. The probabilistic framework, which describes how to represent and manipulate uncertainty about models and predictions, has a central role in scientific data analysis, machine learning, robotics, cognitive science and artificial intelligence. This Review provides an introduction to this framework, and discusses some of the state-of-the-art advances in the field, namely, probabilistic programming, Bayesian optimization, data compression and automatic model discovery.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Deriving from the Bayesian framework, GPR models have been widely applied to prognostic analysis as they are flexible, nonparametric and probabilistic [139].”
From Data-driven health estimation and lifetime prediction of lithium-ion batteries: A review (2019) · cites this paper as [139]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“To ameliorate this in GPR, one way is to minimize the negative log marginal likelihood [139].”
From Data-driven health estimation and lifetime prediction of lithium-ion batteries: A review (2019) · cites this paper as [139]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The performance of GPR is highly sensitive on the covariance function and the kernels should therefore be carefully selected to achieve high prediction accuracy [139].”
From Data-driven health estimation and lifetime prediction of lithium-ion batteries: A review (2019) · cites this paper as [139]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“These methods can be broadly classified into two categories: Bayesian approaches [40,90,106] and Frequentist, approaches [14,70,113]”
From Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty (2021) · cites this paper as [40]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 1×
“recent advances using deep neural networks to solve challenging pattern recognition problems such as speech recognition [5], image classification [6, 7], and prediction of words in text [8], do not overtly represent the uncertainty in the structure or parameters of those neural networks.”
BackgroundA Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb01338.x · Cited by 88,486
A Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb00917.x · Cited by 48,949
Gelman, A., Carlin, J. B., Stern, H. S., & Rubin, D. B. (2003) · doi:10.1201/9780429258480 · Cited by 27,276
A tutorial on hidden Markov models and selected applications in speech recognition
Rabiner, L. R. (1989) · Proceedings of the IEEE · doi:10.1109/5.18626 · Cited by 22,743 · cited 1×
“The HMM is a widely used probabilistic model for sequential and time-series data, which assumes the data were obtained by transitioning stochastically between a discrete number of hidden states 98 .”
MethodBook Review: Pattern Recognition and Machine Learning
Nasrabadi, N. M. (2007) · Journal of Electronic Imaging · doi:10.1117/1.2819119 · Cited by 22,066
Artificial intelligence: a modern approach
(1995) · Choice Reviews Online · doi:10.5860/choice.33-1577 · Cited by 21,974
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766 · cited 1×
“The model could have a large number of parameters compared with the data set (for example, the neural network recently used to achieve translations of English and French sentences that approached the accuracy of state-of-the-art methods is a probabilistic model with 384 million parameters 29 ).”
MethodGaussian Processes for Machine Learning
Rasmussen, C. E., & Williams, C. K. I. (2005) · doi:10.7551/mitpress/3206.001.0001 · Cited by 11,705
Artificial intelligence: A modern approach
Nilsson, N. J. (1996) · Artificial Intelligence · doi:10.1016/0004-3702(96)00007-0 · Cited by 10,583
Hinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A.-R., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., et al. (2012) · IEEE Signal Processing Magazine · doi:10.1109/msp.2012.2205597 · Cited by 10,260 · cited 1×
“recent advances using deep neural networks to solve challenging pattern recognition problems such as speech recognition [5], image classification [6, 7], and prediction of words in text [8], do not overtly represent the uncertainty in the structure or parameters of those neural networks.”
Background(2012) · Cited by 9,062
Efficient Global Optimization of Expensive Black-Box Functions
Jones, D. R., Schonlau, M., & Welch, W. J. (1998) · Journal of Global Optimization · doi:10.1023/a:1008306431147 · Cited by 8,070 · cited 1×
“…f on a domain X , the goal is to find a global maximizer x *: Bayesian optimization poses this as a problem in sequential decision theory: where should one evaluate next so as most quickly to maximize f , taking into account the gain in information about the unknown function f (refs 65,66)?”
BackgroundThrun, S. (2002) · Communications of the ACM · doi:10.1145/504729.504754 · Cited by 7,994
Koller, D., & Friedman, N. (2009) · The MIT Press eBooks · Cited by 7,555
Sequential Monte Carlo Methods in Practice
(2001) · Springer eBooks · doi:10.1007/978-1-4757-3437-9 · Cited by 7,366
Practical Bayesian Optimization of Machine Learning Algorithms
Snoek, J., Larochelle, H., & Adams, R. P. (2012) · arXiv · doi:10.48550/arxiv.1206.2944 · Cited by 6,691 · cited 1×
“A recent high-impact application has been the optimization of the training process for machine-learning models, including deep neural networks 70 .”
MethodInformation theory, inference, and learning algorithms
(2004) · Choice Reviews Online · doi:10.5860/choice.41-5949 · Cited by 6,362
(2009) · Cited by 6,153
WinBUGS - A Bayesian modelling framework: Concepts, structure, and extensibility
Lunn, D. J., Thomas, A., Best, N., & Spiegelhalter, D. (2000) · Statistics and Computing · doi:10.1023/a:1008929526011 · Cited by 5,368 · cited 1×
“BUGS 52 , Stan 53 , AutoBayes 54 and Infer.NET 55 allow only a restrictive class of models to be represented compared with systems based on Turing-complete languages.”
MethodInformation Theory, Inference, and Learning Algorithms
(2004) · Kybernetes · doi:10.1108/03684920410534506 · Cited by 4,937
A Bayesian Analysis of Some Nonparametric Problems
Ferguson, T. S. (1973) · The Annals of Statistics · doi:10.1214/aos/1176342360 · Cited by 4,697 · cited 1×
“Dirichlet processes are a non-parametric model with a long history in statistics 35 and are used for density estimation, clustering, time-series analysis and modelling the topics of documents 36 .”
MethodProbability theory the logic of science
Jaynes, E. T., & Bretthorst, G. L. (2003) · Cited by 4,654
Bayesian Learning for Neural Networks
Neal, R. M. (1996) · Lecture Notes in Statistics · doi:10.1007/978-1-4612-0745-0 · Cited by 4,276 · cited 2×
“For example, Bayesian neural networks represent the parameter uncertainty in neural networks 44 , and mixture models are a probabilistic analogue for clustering methods 78 .”
BackgroundMethodProbability Theory: The Logic of Science
Jaynes, E. T. (2003) · doi:10.1017/cbo9780511790423 · Cited by 4,045
An Introduction to Variational Methods for Graphical Models
Jordan, M. I., Ghahramani, Z., Jaakkola, T. S., & Saul, L. K. (1999) · Machine Learning · doi:10.1023/a:1007665907178 · Cited by 3,791 · cited 1×
“…not as stark as it appears: many gradient-based optimization meth-ods can be turned into integration methods through the use of Langevin and Hamiltonian Monte Carlo methods 27,28 , while integration problems can be turned into optimization problems through the use of variational approximations 24 .”
MethodSelf-citationNeural Information Processing Systems
(2018) · The Deep Learning Revolution · doi:10.7551/mitpress/11474.003.0014 · Cited by 3,730
An internal model for sensorimotor integration
Wolpert, D. M., Ghahramani, Z., & Jordan, M. I. (1995) · Science · doi:10.1126/science.7569931 · Cited by 3,553 · cited 1×
“Because of the force of these and many other arguments on the importance of a principled handling of uncertainty for intelligence, Bayesian probabilistic modelling has emerged not only as the theoretical foundation for rationality in AI systems but also as a model for normative behaviour in humans and animals [15, 16,…”
BackgroundSelf-citationEncyclopedia of Machine Learning
(2010) · Springer eBooks · doi:10.1007/978-0-387-30164-8 · Cited by 3,465
Handbook of Markov Chain Monte Carlo
Brooks, S., Gelman, A., Jones, G., & Meng, X.-L. (2011) · Chapman & Hall/CRC eBooks · doi:10.1201/b10905 · Cited by 2,993 · cited 1×
“However, this dichotomy is not as stark as it appears: many gradient-based optimisation methods can be turned into integration methods through the use of Langevin and Hamiltonian Monte Carlo methods [27, 28], while integration problems can be turned into optimisation problems through the use of variational approximati…”
MethodDistilling Free-Form Natural Laws from Experimental Data
Schmidt, M., & Lipson, H. (2009) · Science · doi:10.1126/science.1165893 · Cited by 2,890 · cited 1×
“83), the models attempt to capture general properties of functions (for example, smoothness, periodicity or trends) rather than a precise equation.”
BackgroundOverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks
Sermanet, P., Eigen, D., Zhang, X., Mathieu, M., Fergus, R., & LeCun, Y. (2013) · arXiv · doi:10.48550/arxiv.1312.6229 · Cited by 2,680 · cited 1×
“recent advances using deep neural networks to solve challenging pattern recognition problems such as speech recognition [5], image classification [6, 7], and prediction of words in text [8], do not overtly represent the uncertainty in the structure or parameters of those neural networks.”
BackgroundPattern recognition and machine learning
(2007) · Choice Reviews Online · doi:10.5860/choice.44-5091 · Cited by 2,650
Some aspects of the sequential design of experiments
Robbins, H. (1952) · Bulletin of the American Mathematical Society · doi:10.1090/s0002-9904-1952-09620-8 · Cited by 2,184 · cited 1×
“Such state-less sequential decision problems fall under the rubric of multi-arm bandits 72 , a subclass of reinforcement-learning problems.”
BackgroundHierarchical Dirichlet Processes
Teh, Y. W., Jordan, M. I., Beal, M. J., & Blei, D. M. (2006) · Journal of the American Statistical Association · doi:10.1198/016214506000000302 · Cited by 2,120 · cited 2×
“Dirichlet processes are a non-parametric model with a long history in statistics 35 and are used for density estimation, clustering, time-series analysis and modelling the topics of documents 36 .”
MethodBrochu, E., Cora, V. M., & de Freitas, N. (2010) · arXiv (Cornell University) · doi:10.48550/arxiv.1012.2599 · Cited by 2,011 · cited 1×
“The current best-performing global optimization methods maintain a Bayesian representation of the probability distribution over the uncertain function f being optimized, and use this uncertainty to decide where (in X ) to query next 67–69 .”
MethodPILCO: A Model-Based and Data-Efficient Approach to Policy Search
Deisenroth, M. P., & Rasmussen, C. E. (2011) · Cambridge University Engineering Department Publications Database · Cited by 1,945
How to grow a mind: Statistics, structure, and abstraction
Tenenbaum, J. B., Kemp, C., Griffiths, T. L., & Goodman, N. D. (2011) · Science · doi:10.1126/science.1192788 · Cited by 1,815 · cited 1×
“…the theoretical foundation for rationality in artificial intelligence systems, but also as a model for normative behaviour in humans and animals 15–18 (but see refs 19, 20 for a discussion), and much research is devoted to understanding how neural circuitry may be implementing Bayesian inference…”
BackgroundLa prévision : ses lois logiques, ses sources subjectives
de Finetti, B. (1937) · French digital mathematics library (Numdam) · Cited by 1,642
Riemann manifold langevin and hamiltonian monte carlo methods
Girolami, M., & Calderhead, B. (2011) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2010.00765.x · Cited by 1,473 · cited 1×
“…not as stark as it appears: many gradient-based optimization meth-ods can be turned into integration methods through the use of Langevin and Hamiltonian Monte Carlo methods 27,28 , while integration problems can be turned into optimization problems through the use of variational approximations 24 .”
MethodPerception as Bayesian Inference
(1996) · doi:10.1017/cbo9780511984037 · Cited by 1,425 · cited 1×
“…as the theoretical foundation for rationality in artificial intelligence systems, but also as a model for normative behaviour in humans and animals 15–18 (but see refs 19, 20 for a discussion), and much research is devoted to understanding how neural circuitry may be implementing Bayesian…”
Uncertainty in Artificial Intelligence
(1992) · Elsevier eBooks · doi:10.1016/c2013-0-12606-2 · Cited by 1,425
Expectation Propagation for approximate Bayesian inference
Minka, T. P. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.2294 · Cited by 1,392 · cited 1×
“Fortunately, a number of approximate integration algorithms have been developed, including Markov chain Monte Carlo (MCMC) meth-ods, variational approximations, expectation propagation and sequential Monte Carlo 23–26 .”
MethodAuto-WEKA: combined selection and hyperparameter optimization of classification algorithms
Thornton, C., Hutter, F., Hoos, H. H., & Leyton-Brown, K. (2013) · Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/2487575.2487629 · Cited by 1,359 · cited 2×
“AutoWeka is a recent project that automates learning classifiers, making heavy use of the Bayesian optimisation techniques described previously [71].”
MethodStochastic variational inference
Hoffman, M. D., Blei, D. M., Wang, C., & Paisley, J. W. (2013) · Journal of Machine Learning Research · doi:10.5555/2567709.2502622 · Cited by 1,333 · cited 1×
“Modern inference methods have made it possible to scale to millions of data points, making probabilistic methods computationally competitive with conventional methods 92–95 .”
MethodArtificial Intelligence and Statistics
Ziegel, E. (1989) · Technometrics · doi:10.1080/00401706.1989.10488504 · Cited by 1,217
Markov chain Monte Carlo without likelihoods
Marjoram, P., Molitor, J., Plagnol, V., & Tavaré, S. (2003) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0306899100 · Cited by 1,199 · cited 1×
“The fact that defining such universal inference algorithms for computer programs is even possible is somewhat surprising, but it is related to the generality of certain key ideas from sampling such as rejection sampling, sequential Monte Carlo methods 25 and ‘approximate Bayesian computation’ 49 .”
BackgroundA New Method of Locating the Maximum Point of an Arbitrary Multipeak Curve in the Presence of Noise
Kushner, H. J. (1964) · Journal of Basic Engineering · doi:10.1115/1.3653121 · Cited by 1,082 · cited 1×
“…f on a domain X , the goal is to find a global maximizer x *: Bayesian optimization poses this as a problem in sequential decision theory: where should one evaluate next so as most quickly to maximize f , taking into account the gain in information about the unknown function f (refs 65,66)?”
BackgroundBlumer, A., Ehrenfeucht, A., Haussler, D., & Warmuth, M. K. (1987) · Information Processing Letters · doi:10.1016/0020-0190(87)90114-1 · Cited by 1,054
Probabilistic Inference Using Markov Chain Monte Carlo Methods
Neal, R. M. (2011) · Cited by 1,047
Ockham's Razor and Bayesian Analysis
Jefferys, W. H., & Berger, J. O. (1992) · American Scientist · Cited by 706
Functional genomic hypothesis generation and experimentation by a robot scientist
King, R. D., Whelan, K. E., Jones, F. M., Reiser, P. G. K., Bryant, C. H., Muggleton, S. H., Kell, D. B., & Oliver, S. G. (2004) · Nature · doi:10.1038/nature02236 · Cited by 662 · cited 1×
“Important earlier work includes statistical expert systems 84,85 and the Robot Scientist, which integrates machine learning and scientific discovery in a closed loop with an experimental platform in microbiology to automate the design and execution of new experiments 86 .”
MethodDo the Right Thing: Studies in Limited Rationality
Russell, S., Wefald, E. H., Bobrow, D. G., Brady, M., & Davis, R. (1991) · The MIT Press eBooks · doi:10.7551/mitpress/2474.001.0001 · Cited by 655 · cited 1×
“Although probabilistic machine learning often defines how to solve a problem in principle, the central challenge in the field is finding how to do so in practice in a computationally efficient manner 90,91 .”
BackgroundOptimal Predictions in Everyday Cognition
Griffiths, T. L., & Tenenbaum, J. B. (2006) · Psychological Science · doi:10.1111/j.1467-9280.2006.01780.x · Cited by 639 · cited 1×
“Because of the force of these and many other arguments on the importance of a principled handling of uncertainty for intelligence, Bayesian probabilistic modelling has emerged not only as the theoretical foundation for rationality in AI systems but also as a model for normative behaviour in humans and animals [15, 16,…”
BackgroundChurch: a language for generative models
Goodman, N., Mansinghka, V., Roy, D. M., Bonawitz, K., & Tenenbaum, J. B. (2012) · arXiv · doi:10.48550/arxiv.1206.3255 · Cited by 577 · cited 2×
“In return for this restriction, inference in such languages can be much faster than for the more general languages, such as IBAL, BLOG, Figaro, Church, Venture, and Anglican [56, 57, 58, 59, 60, 61, 62].”
BackgroundLearning systems of concepts with an infinite relational model
Kemp, C., Tenenbaum, J. B., Griffiths, T. L., Yamada, T., & Ueda, N. (2006) · Cited by 550
Bayesian Brain: Probabilistic Approaches to Neural Coding
(2006) · The MIT Press eBooks · doi:10.7551/mitpress/9780262042383.001.0001 · Cited by 511 · cited 1×
“…foundation for rationality in artificial intelligence systems, but also as a model for normative behaviour in humans and animals 15–18 (but see refs 19, 20 for a discussion), and much research is devoted to understanding how neural circuitry may be implementing Bayesian inference 21,22 .”
BackgroundStatistical Decision Theory and Related Topics V
(1994) · Springer eBooks · doi:10.1007/978-1-4612-2618-5 · Cited by 507
The Algebra of Probable Inference
Barnard, G. A., & Cox, R. T. (1962) · Econometrica · doi:10.2307/1909904 · Cited by 494
Cox, R. (2001) · doi:10.56021/9780801869822 · Cited by 462
Entropy search for information-efficient global optimization
Hennig, P., & Schuler, C. J. (2012) · Journal of Machine Learning Research · doi:10.5555/2188385.2343701 · Cited by 443 · cited 1×
“The current best-performing global optimisation methods maintain a Bayesian representation of the probability distribution over the uncertain function f being optimised, and use this uncertainty to decide where (in X ) to query next [67, 68, 69].”
MethodNonparametric Latent Feature Models for Link Prediction
Miller, K. T., Jordan, M. I., & Griffiths, T. L. (2009) · neural information processing systems · Cited by 432
Oksanen, T., Tiusanen, J., & Kostamo, J. (2009) · Precision agriculture '09 · doi:10.3920/9789086866649_102 · Cited by 414
BLOG: Probabilistic Models with Unknown Objects
Milch, B., Marthi, B., Russell, S., Sontag, D., Ong, D. L., & Kolobov, A. (2007) · Introduction to Statistical Relational Learning · doi:10.7551/mitpress/7432.003.0015 · Cited by 390
(2010) · Cambridge University Press eBooks · doi:10.1017/cbo9780511802478 · Cited by 358
O'Hagan, A. (1991) · Journal of Statistical Planning and Inference · doi:10.1016/0378-3758(91)90002-v · Cited by 342 · cited 1×
“More generally, Bayesian optimization is a special case of Bayesian numerical computation 75,76 , which is re-emerging as a very active area of research (http://www.probabilistic-numerics.org), and includes topics such as solving ordinary differential equations and numerical integration.”
BackgroundThe Indian Buffet Process: An Introduction and Review
Griffiths, T. L., & Ghahramani, Z. (2011) · doi:10.5555/1953048.2021039 · Cited by 325 · cited 3×
“Elaborating the social network modelling example, an IBP-based model allows each person to belong to some subset of a large number of potential communities (for example, as defined by different families, workplaces, schools, hobbies, and so on) rather than a single community, and the probability of friendship between…”
MethodSelf-citationBayesian Spiking Neurons I: Inference
Deneve, S. (2008) · Neural Computation · doi:10.1162/neco.2008.20.1.91 · Cited by 313 · cited 1×
“…foundation for rationality in artificial intelligence systems, but also as a model for normative behaviour in humans and animals 15–18 (but see refs 19, 20 for a discussion), and much research is devoted to understanding how neural circuitry may be implementing Bayesian inference 21,22 .”
BackgroundSurpassing Human-Level Face Verification Performance on LFW with GaussianFace
Lu, C., & Tang, X. (2015) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v29i1.9797 · Cited by 303
Bayesian infinite mixture model based clustering of gene expression profiles
Medvedovic, M., & Sivaganesan, S. (2002) · Bioinformatics · doi:10.1093/bioinformatics/18.9.1194 · Cited by 281 · cited 1×
“Dirichlet processes have also been used for clustering gene-expression patterns 38,39 .”
MethodDiaconis, P. (1988) · Statistical Decision Theory and Related Topics IV · doi:10.1007/978-1-4613-8768-8_20 · Cited by 238 · cited 1×
“More generally, Bayesian optimisation is a special case of Bayesian numerical computation [75, 76] which is re-emerging as a very active area of research4, and also includes topics such as solving ordinary differential equations and numerical integration.”
BackgroundAusterity in MCMC Land: Cutting the Metropolis-Hastings Budget
Korattikara, A., Chen, Y., & Welling, M. (2013) · UvA-DARE (University of Amsterdam) · doi:10.48550/arxiv.1304.5299 · Cited by 225
Bishop, C. M. (2013) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2012.0222 · Cited by 211 · cited 1×
“Probabilistic programming languages create a very clear separation between the model and the inference procedures, encouraging model-based thinking 51 .”
BackgroundLightweight Implementations of Probabilistic Programming Languages Via Transformational Compilation
Wingate, D., Stuhlmüller, A., & Goodman, N. D. (2011) · International Conference on Artificial Intelligence and Statistics · Cited by 205
A new approach to probabilistic programming inference
Wood, F. (2014) · Open Collections · doi:10.14288/1.0044249 · Cited by 199
How Robust Are Probabilistic Models of Higher-Level Cognition?
Marcus, G. F., & Davis, E. (2013) · Psychological Science · doi:10.1177/0956797613495418 · Cited by 195 · cited 1×
“…foundation for rationality in artificial intelligence systems, but also as a model for normative behaviour in humans and animals 15–18 (but see refs 19, 20 for a discussion), and much research is devoted to understanding how neural circuitry may be implementing Bayesian inference 21,22 .”
BackgroundAutomatic Construction and Natural-Language Description of Nonparametric Regression Models
Lloyd, J., Duvenaud, D., Grosse, R., Tenenbaum, J., & Ghahramani, Z. (2014) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v28i1.8904 · Cited by 170
Declarative Probabilistic Programming with Datalog
Barany, V., ten Cate, B., Kimelfeld, B., Olteanu, D., & Vagena, Z. (2014) · arXiv (Cornell University) · doi:10.4230/lipics.icdt.2016.7 · Cited by 164
Predictive Entropy Search for Efficient Global Optimization of Black-box FunctionsPreprint
Hernández-Lobato, J. M., Hoffman, M. W., & Zoubin Ghahramani, A. (2014) · arXiv (Cornell University) · Cited by 162
Bayesian Nonparametrics: Subject index
Hjort, N. L., Holmes, C. C., Müller, P., & Walker, S. G. (2010) · Cited by 152
Bayesian Nonparametrics: List of contributors
Hjort, N. L., Holmes, C. C., Müller, P., & Walker, S. G. (2010) · Cited by 152
IBAL: a probabilistic rational programming language
Pfeffer, A. (2001) · Cited by 151
Neal, R. M. (1992) · Springer eBooks · doi:10.1007/978-94-017-2219-3_14 · Cited by 146 · cited 1×
“Many nonparametric models can be derived starting from a parametric model and considering that happens as the model grows to the limit of infinitely many parameters [30].”
BackgroundCollaborative Gaussian Processes for Preference Learning
Houlsby, N., Huszár, F., Ghahramani, Z., & Hernández-Lobato, J. M. (2012) · Cambridge University Engineering Department Publications Database · Cited by 141
Do the right thing: studies in limited rationality
(1992) · Choice Reviews Online · doi:10.5860/choice.29-5169 · Cited by 128
On statistics, computation and scalability
Jordan, M. I. (2013) · Bernoulli · doi:10.3150/12-bejsp17 · Cited by 127 · cited 1×
“Although probabilistic machine learning often defines how to solve a problem in principle, the central challenge in the field is finding how to do so in practice in a computationally efficient manner 90,91 .”
BackgroundPractical Probabilistic Programming
Pfeffer, A. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-21295-6_2 · Cited by 125 · cited 1×
“The basic idea in probabilistic programming it to use computer programs to represent probabilistic models2[45, 46, 47].”
MethodBayesian non-parametrics and the probabilistic approach to modelling
Ghahramani, Z. (2013) · Philosophical Transactions of the Royal Society A Mathematical Physical and Engineering Sciences · doi:10.1098/rsta.2011.0553 · Cited by 116 · cited 2×
“The probabilistic approach to modelling uses probability theory to express all forms of uncertainty 9 .”
BackgroundMethodSelf-citationAutoBayes: a system for generating data analysis programs from statistical models
Fischer, B., & Schumann, J. M. P. (2003) · Journal of Functional Programming · doi:10.1017/s0956796802004562 · Cited by 105
Effective Bayesian Inference for Stochastic Programs
Koller, D., McAllester, D., & Pfeffer, A. (1997) · Cited by 88
Constructing a logic of plausible inference: a guide to Cox’s theorem
Horn, K. S. V. (2003) · International Journal of Approximate Reasoning · doi:10.1016/s0888-613x(03)00051-3 · Cited by 86 · cited 1×
“The Cox axioms define some desiderata for representing beliefs; a consequence of these axioms is that ‘degrees of belief’, ranging from ‘impossible’ to ‘absolutely certain’, must follow all the rules of probability theory 10,12,13 .”
BackgroundAutomatic Construction and Natural-Language Description of Nonparametric Regression ModelsPreprint
Lloyd, J. R., Duvenaud, D., Grosse, R., et al. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1402.4304 · Cited by 80
Learning the Structure of Deep Sparse Graphical ModelsPreprint
Adams, R. P., Wallach, H., & Ghahramani, Z. (2009) · arXiv (Cornell University) · doi:10.48550/arxiv.1001.0160 · Cited by 54 · cited 1×
“The Indian buffet process (IBP) 40 is a non-parametric model that can be used for latent feature modelling, learning overlapping clusters, sparse matrix factorization, or to non-parametrically learn the structure of a deep network 41 .”
MethodSelf-citationWood, F., Gasthaus, J., Archambeau, C., James, L., & Teh, Y. W. (2011) · Communications of the ACM · doi:10.1145/1897816.1897842 · Cited by 51 · cited 2×
“It turns out that some of the world’s best compression algorithms (for example, Sequence Memoizer 79 and PPM with dynamic parameter updates 80 ) are equivalent to Bayesian non-parametric models of sequences, and improvements to compression are being made through a better understanding of how to…”
MethodPatterns in statistical strategy
Hand, D. J. (1986) · Addison-Wesley Longman Publishing Co., Inc. eBooks · Cited by 45
Modeling and Visualizing Uncertainty in Gene Expression Clusters Using Dirichlet Process Mixtures
Rasmussen, C. E., de la Cruz, B. J., Ghahramani, Z., & Wild, D. L. (2009) · IEEE Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbb.2007.70269 · Cited by 40 · cited 1×
“DPs have also been used for clustering gene expression patterns [38, 39].”
MethodSelf-citationRelevant and Robust: A Response to Marcus and Davis (2013)
Goodman, N. D., Frank, M. C., Griffiths, T. L., Tenenbaum, J. B., Battaglia, P. W., & Hamrick, J. B. (2015) · Psychological Science · doi:10.1177/0956797614559544 · Cited by 29
GLIMPSE: a knowledge-based front end for statistical analysis
Wolstenholme, D. E., O’Brien, C. M., & Nelder, J. A. (1988) · Knowledge-Based Systems · doi:10.1016/0950-7051(88)90075-5 · Cited by 22 · cited 1×
“Important earlier work includes statistical expert systems 84,85 and the Robot Scientist, which integrates machine learning and scientific discovery in a closed loop with an experimental platform in microbiology to automate the design and execution of new experiments 86 .”
MethodImproving PPM with Dynamic Parameter Updates
Steinruecken, C., Ghahramani, Z., & Mackay, D. (2015) · DCC · doi:10.1109/dcc.2015.77 · Cited by 17 · cited 1×
“It turns out that some of the world’s best compression algorithms (for example, Sequence Memoizer 79 and PPM with dynamic parameter updates 80 ) are equivalent to Bayesian non-parametric models of sequences, and improvements to compression are being made through a better understanding of how to…”
MethodSelf-citationFreer, C. E., Roy, D. M., & Tenenbaum, J. B. (2014) · Cambridge University Press eBooks · doi:10.1017/cbo9781107338579.007 · Cited by 13 · cited 1×
“In fact, for many of the recent probabilistic programming languages that are based on extending Turing-complete languages (a class that includes almost all commonly-used languages), it is possible to represent any computable probability distribution as a probabilistic program [48].”
BackgroundWoudhuysen, H. R. (1999) · TLS, the Times literary supplement/Times literary supplement on CD-ROM/TLS. Times literary supplement · Cited by 1
Research works printed but not linked: 14
- Bengio, Y., Ducharme, R., Vincent, P. & Janvin, C. A neural probabilistic language model. J. Mach. Learn. Res. 3, 1137–1155 (2003).
- Hinton, G. E., McClelland, J. L. & Rumelhart, D. E. in Parallel Distributed Processing: Explorations in the Microstructure of Cognition: Foundations 77–109 (MIT Press, 1986).
- Goodman, N. D. & Stuhlmüller, A. The Design and Implementation of Probabilistic Programming Languages. Available at http://dippl.org (2015).
- Mansinghka, V., Kulkarni, T. D., Perov, Y. N. & Tenenbaum, J. Approximate Bayesian image interpretation using generative probabilistic graphics programs. In Proc. Advances in Neural Information Processing Systems 26 1520–1528 (2013).
- Minka, T. P., Winn, J. M., Guiver, J. P. & Knowles, D. A. Infer.NET 2.4. http://research.microsoft.com/infernet (Microsoft Research, 2010).
- Pfeffer, A. Figaro: An Object-Oriented Probabilistic Programming Language. Tech. Rep. (Charles River Analytics, 2009).
- Mansinghka, V., Selsam, D. & Perov, Y. Venture: a higher-order probabilistic programming platform with programmable inference. Preprint at http://arxiv.org/abs/1404.0099 (2014). · arXiv
- Li, L., Wu, Y. & Russell, S. J. SWIFT: Compiled Inference for Probabilistic Programs. Report No. UCB/EECS-2015–12 (Univ. California, Berkeley, 2015).
- Bergstra, J. et al. Theano: a CPU and GPU math expression compiler. In Proc. 9th Python in Science Conference http://conference.scipy.org/proceedings/scipy2010/ (2010).
- Poupart, P. in Encyclopedia of Machine Learning 90–93 (Springer, 2010).
- Grosse, R. B., Salakhutdinov, R. & Tenenbaum, J. B. Exploiting compositionality to explore a large space of model structures. In Proc. Conference on Uncertainty in Artificial Intelligence 306–315 (2012).
- Welling, M. et al. Bayesian inference with big data: a snapshot from a workshop. ISBA Bulletin 21, https://bayesian.org/sites/default/files/fm/bulletins/1412.pdf (2014).
- Bakker, B. & Heskes, T. Task clustering and gating for Bayesian multitask learning. J. Mach. Learn. Res. 4, 83–99 (2003).
- Hensman, J., Fusi, N. & Lawrence, N. D. Gaussian processes for big data. In Proc. Conference on Uncertainty in Artificial Intelligence 244 (UAI, 2013).
Cited by
Dropout as a bayesian approximation: Representing model uncertainty in deep learning
Gal, Y., & Ghahramani, Z. (2015) · arXiv · doi:10.48550/arxiv.1506.02142 · Cited by 6,471
“For the 12 images, the model predicts classes [1 1 1 1 1 5 5 7 7 7 7 7].”
BackgroundSelf-citationDeep learning and process understanding for data-driven Earth system science
Reichstein, M., Camps-Valls, G., Stevens, B., Jung, M., Denzler, J., Carvalhais, N., & Prabhat (2019) · Nature · doi:10.1038/s41586-019-0912-1 · Cited by 5,673
How artificial intelligence will change the future of marketing
Davenport, T., Guha, A., Grewal, D., & Bressgott, T. (2020) · Journal of the Academy of Marketing Science · doi:10.1007/s11747-019-00696-0 · Cited by 2,875
“3 In contrast, context awareness continues to be developed, and researchers in computer science are working on moving AI capabilities forward, from task automation to context awareness (e.g., Ghahramani 2015; Mnih et al. 2015).”
BackgroundHuman-level concept learning through probabilistic program induction
Lake, B. M., Salakhutdinov, R., & Tenenbaum, J. B. (2015) · Science · doi:10.1126/science.aab3050 · Cited by 2,817
“Concepts are represented as simple probabilistic programs—that is, probabilistic generative models expressed as structured procedures in an abstract description language (17, 18).”
BackgroundBuilding machines that learn and think like people
Lake, B. M., Ullman, T. D., Tenenbaum, J. B., & Gershman, S. J. (2017) · Behavioral and Brain Sciences · doi:10.1017/s0140525x16001837 · Cited by 2,550
Metadata
- Authors
- Ghahramani, Z.
- Venue
- Nature
- Published
- 2015-05-27
- Volume
- 521
- Issue
- 7553
- Pages
- 452-459
- ISSN
- 0028-0836 (ISSN-L)
- 1476-4687
- OpenAlex
- W1494192115
- Cited by
- 1,995
- References
- 108
- Citations by year
- 2026: 166 · 2025: 231 · 2024: 228 · 2023: 206 · 2022: 217 · 2021: 265 · 2020: 235 · 2019: 210 · 2018: 102 · 2017: 84 · 2016: 39 · 2015: 11
- FWCI
- 2.97
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- www.springer.com/tdm (vor)
- Locations
- Nature (publishedVersion)
- PubMed (publishedVersion)
- Apollo (University of Cambridge) (acceptedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Cambridge University Engineering Department Publications Database (submittedVersion)
- Funders
- Engineering and Physical Sciences Research Council (EP/I036575/1)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Bayesian Modeling and Causal Inference
- Physical Sciences › Computer Science › Artificial Intelligence › Gaussian Processes and Bayesian Inference
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
- Keywords
- stochastic modeling, machine learning, artificial intelligence, uncertainty representation, probabilistic programming, Bayesian optimization, data compression, experiential learning
- MeSH terms
- Artificial Intelligence, Automation, Bayes Theorem, Models, Statistical, Statistics, Nonparametric, Uncertainty, Data Compression
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.