The DOI matches its Crossref record; 84 of the 84 references it cites were found in a registry or an index.
Spatial Point Processes
Abstract
Markov chain Monte Carlo (MCMC) originated with the classic paper of Metropolis et al. (1953), where it was used to simulate the distribution of states for a system of idealized molecules. Not long after, another approach tomolecular simulationwas introduced (Alder and Wainwright, 1959), in which the motion of the molecules was deterministic, following Newton’s laws ofmotion,which have an elegant formalization asHamiltonian dynamics. For finding the properties of bulk materials, these approaches are asymptotically equivalent, since even in a deterministic simulation, each local region of the material experiences effectively random influences from distant regions. Despite the large overlap in their application areas, the MCMC and molecular dynamics approaches have continued to coexist in the following decades (see Frenkel and Smit, 1996). In 1987, a landmark paper by Duane, Kennedy, Pendleton, and Roweth united the MCMC and molecular dynamics approaches. They called their method “hybrid Monte Carlo,” which abbreviates to “HMC,” but the phrase “Hamiltonian Monte Carlo,” retaining the abbreviation, is more specific and descriptive, and I will use it here. Duane et al. applied HMC not to molecular simulation, but to lattice field theory simulations of quantum chromodynamics. Statistical applications of HMC began with my use of it for neural network models (Neal, 1996a). I also provided a statistically-oriented tutorial on HMC in a review of MCMC methods (Neal, 1993, Chapter 5). There have been other applications of HMC to statistical problems (e.g. Ishwaran, 1999; Schmidt, 2009) and statisticallyoriented reviews (e.g. Liu, 2001, Chapter 9), but HMC still seems to be underappreciated by statisticians, and perhaps also by physicists outside the lattice field theory community. This review begins by describing Hamiltonian dynamics. Despite terminology that may be unfamiliar outside physics, the features of Hamiltonian dynamics that are needed for HMC are elementary. The differential equations of Hamiltonian dynamics must be discretized for computer implementation. The “leapfrog” scheme that is typically used is quite simple. Following this introduction to Hamiltonian dynamics, I describe how to use it to con- struct an MCMCmethod. The first step is to define a Hamiltonian function in terms of the probability distribution we wish to sample from. In addition to the variables we are interested in (the “position” variables), we must introduce auxiliary “momentum” variables, which typically have independent Gaussian distributions. The HMC method alternates simple updates for these momentum variables with Metropolis updates in which a new state is proposed by computing a trajectory according to Hamiltonian dynamics, implemented with the leapfrog method. A state proposed in this way can be distant from the the exploration of the state space that occurs whenMetropolis updates are done using a simple random-walkproposal distribution. (Analternativewayof avoiding randomwalks is touse short trajectories but only partially replace the momentum variables between trajectories, so that successive trajectories tend to move in the same direction.) After presenting the basic HMCmethod, I discuss practical issues of tuning the leapfrog stepsize and number of leapfrog steps, as well as theoretical results on the scaling of HMC with dimensionality. I then present a number of variations on HMC. The acceptance rate for HMC can be increased for many problems by looking at “windows” of states at the beginning and end of the trajectory. For many statistical problems, approximate computation of trajectories (e.g. using subsets of the data) may be beneficial. Tuning of HMC can be made easier using a “short-cut” in which trajectories computed with a bad choice of stepsize take little computation time. Finally, “tempering” methods may be useful when multiple isolated modes exist.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Equation of State Calculations by Fast Computing Machines
Metropolis, N., Rosenbluth, A. W., Rosenbluth, M. N., Teller, A. H., & Teller, E. (1953) · The Journal of Chemical Physics · doi:10.1063/1.1699114 · Cited by 37,192
Monte Carlo sampling methods using Markov chains and their applications.
Hastings, W. K. (1970) · Biometrika · doi:10.1093/biomet/57.1.97 · Cited by 15,195 · cited 1×
“not symmetrical, it must be accepted or rejected based on both the ratio of the probability densities of q∗ and q and on the ratio of the probability densities for proposing q from q∗ and vice versa (Hastings, 1970). To see the equivalence with HMC using one leapfrog step, we can write the Metropolis-Hastings acceptan…”
MethodGaussian Processes for Machine Learning
Rasmussen, C. E., & Williams, C. K. I. (2005) · doi:10.7551/mitpress/3206.001.0001 · Cited by 11,705
Reversible jump Markov chain Monte Carlo computation and Bayesian model determination
GREEN, P. J. (1995) · Biometrika · doi:10.1093/biomet/82.4.711 · Cited by 5,930
Bayesian Learning for Neural Networks
Neal, R. M. (1996) · Lecture Notes in Statistics · doi:10.1007/978-1-4612-0745-0 · Cited by 4,276
Duane, S., Kennedy, A. D., Pendleton, B. J., & Roweth, D. (1987) · Physics Letters B · doi:10.1016/0370-2693(87)91197-x · Cited by 3,872
A Measure of Asymptotic Efficiency for Tests of a Hypothesis Based on the sum of Observations
Chernoff, H. (1952) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177729330 · Cited by 3,550
Studies in Molecular Dynamics. I. General Method
Alder, B. J., & Wainwright, T. E. (1959) · The Journal of Chemical Physics · doi:10.1063/1.1730376 · Cited by 2,524
Ripley, B. D. (1977) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1977.tb01615.x · Cited by 2,511
Simulated Tempering: A New Monte Carlo Scheme
Marinari, E., & Parisi, G. (1992) · Europhysics Letters (EPL) · doi:10.1209/0295-5075/19/6/002 · Cited by 1,818
Cited by
brms : An R Package for Bayesian Multilevel Models Using Stan
Bürkner, P.-C. (2017) · Journal of Statistical Software · doi:10.18637/jss.v080.i01 · Cited by 9,476
“Currently, these are the static Hamiltonian Monte-Carlo (HMC) Sampler sometimes also referred to as Hybrid Monte-Carlo (Neal 2011, 2003; Duane et al. 1987) and its extension the No-U-Turn Sampler (NUTS) by Hoffman and Gelman (2014).”
BackgroundMethodInfluentialStan : A Probabilistic Programming Language
Carpenter, B., Gelman, A., Hoffman, M. D., Lee, D., Goodrich, B., Betancourt, M., Brubaker, M., Guo, J., Li, P., & Riddell, A. (2017) · Journal of Statistical Software · doi:10.18637/jss.v076.i01 · Cited by 7,754
“An additional benefit of RHMC are transitions that can cover much larger variations in density, making it uniquely suited to these models; see (Neal 2011). vector[K] beta; beta <- mu + sigma * beta_raw; } model { beta ~ normal(0,1); } This reparameterization removes the particular correlations…”
MethodAn Introduction to Variational Autoencoders
Kingma, D. P., & Welling, M. (2019) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000056 · Cited by 2,609
Riemann manifold langevin and hamiltonian monte carlo methods
Girolami, M., & Calderhead, B. (2011) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2010.00765.x · Cited by 1,473
A survey of uncertainty in deep neural networks
Gawlikowski, J., Tassi, C. R. N., Ali, M., Lee, J., Humt, M., Feng, J., Kruspe, A., Triebel, R., Jung, P., Roscher, R., et al. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10562-9 · Cited by 1,406
Variational Inference with Normalizing Flows
Rezende, D. J., & Mohamed, S. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.05770 · Cited by 1,253
Yang, L., Meng, X., & Karniadakis, G. E. (2021) · Journal of Computational Physics · doi:10.1016/j.jcp.2020.109913 · Cited by 1,169
Hands-On Bayesian Neural Networks—A Tutorial for Deep Learning Users
Jospin, L. V., Laga, H., Boussaid, F., Buntine, W., & Bennamoun, M. (2022) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2022.3155327 · Cited by 879
CHOMP: Covariant Hamiltonian optimization for motion planning
Zucker, M., Ratliff, N., Dragan, A. D., Pivtoraiko, M., Klingensmith, M., Dellin, C. M., Bagnell, J. A., & Srinivasa, S. S. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913488805 · Cited by 758
Yu, Z., Guindani, M., Grieco, S. F., Chen, L., Holmes, T. C., & Xu, X. (2022) · Neuron · doi:10.1016/j.neuron.2021.10.030 · Cited by 563
Stan: A Probabilistic Programming Language for Bayesian Inference and Optimization
Gelman, A., Lee, D., & Guo, J. (2015) · Journal of Educational and Behavioral Statistics · doi:10.3102/1076998615606113 · Cited by 540
A Conceptual Introduction to Hamiltonian Monte Carlo
Betancourt, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1701.02434 · Cited by 509
The geometric foundations of Hamiltonian Monte Carlo
Betancourt, M., Byrne, S., Livingstone, S., & Girolami, M. (2017) · Bernoulli · doi:10.3150/16-bej810 · Cited by 452
Stochastic Gradient Hamiltonian Monte Carlo
Chen, T., Fox, E. B., & Guestrin, C. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1402.4102 · Cited by 434
Markov Chain Monte Carlo and Variational Inference: Bridging the Gap
Salimans, T., Kingma, D. P., & Welling, M. (2014) · UvA-DARE (University of Amsterdam) · doi:10.48550/arxiv.1410.6460 · Cited by 343
Normalizing Flows for Probabilistic Modeling and Inference
Papamakarios, G., Nalisnick, E., Rezende, D. J., Mohamed, S., & Lakshminarayanan, B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.02762 · Cited by 318
Effat, M. B., & Ciucci, F. (2017) · Electrochimica Acta · doi:10.1016/j.electacta.2017.07.050 · Cited by 295
Entropy-SGD: biasing gradient descent into wide valleys*
Chaudhari, P., Choromanska, A., Soatto, S., LeCun, Y., Baldassi, C., Borgs, C., Chayes, J., Sagun, L., & Zecchina, R. (2019) · Journal of Statistical Mechanics: Theory and Experiment · doi:10.1088/1742-5468/ab39d9 · Cited by 291
Cygnus X-1 contains a 21–solar mass black hole—Implications for massive star winds
Miller-Jones, J. C. A., Bahramian, A., Orosz, J. A., Mandel, I., Gou, L., Maccarone, T. J., Neijssel, C. J., Zhao, X., Ziółkowski, J., Reid, M. J., et al. (2021) · Science · doi:10.1126/science.abb3363 · Cited by 284
A method for efficiently sampling from distributions with correlated dimensions.
Turner, B. M., Sederberg, P. B., Brown, S. D., & Steyvers, M. (2013) · Psychological Methods · doi:10.1037/a0032222 · Cited by 282
Bayesian inference with optimal maps
El Moselhy, T. A., & Marzouk, Y. M. (2012) · Journal of Computational Physics · doi:10.1016/j.jcp.2012.07.022 · Cited by 267
Bayesian Fusion of Multi-Band Images
Wei, Q., Dobigeon, N., & Tourneret, J.-Y. (2015) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2015.2407855 · Cited by 267
Evaluating Scalable Bayesian Deep Learning Methods for Robust Computer Vision
Gustafsson, F. K., Danelljan, M., & Schon, T. B. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00167 · Cited by 257
Consistency and fluctuations for stochastic gradient Langevin dynamics
(2016) · Warwick Research Archive Portal (University of Warwick) · doi:10.5555/2946645.2946652 · Cited by 104
Sensitivity to pain expectations: A Bayesian model of individual differences
Hoskin, R., Berzuini, C., Acosta-Kane, D., El-Deredy, W., Guo, H., & Talmi, D. (2019) · Cognition · doi:10.1016/j.cognition.2018.08.022 · Cited by 87
Mbalawata, I. S., Särkkä, S., & Haario, H. (2013) · Computational Statistics · doi:10.1007/s00180-012-0352-y · Cited by 77
Aitchison, L., & Lengyel, M. (2016) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1005186 · Cited by 67
Learning Weight Uncertainty with Stochastic Gradient MCMC for Shape Classification
Li, C., Stevens, A., Chen, C., Pu, Y., Gan, Z., & Carin, L. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.611 · Cited by 39
An efficient Bayesian inference framework for coalescent-based nonparametric phylodynamics
Lan, S., Palacios, J. A., Karcher, M., Minin, V. N., & Shahbaba, B. (2015) · Bioinformatics · doi:10.1093/bioinformatics/btv378 · Cited by 38
A generalized many-facet Rasch model and its Bayesian estimation using Hamiltonian Monte Carlo
Uto, M., & Ueno, M. (2020) · Behaviormetrika · doi:10.1007/s41237-020-00115-7 · Cited by 37
Improved Contrastive Divergence Training of Energy Based Models
Du, Y., Li, S., Tenenbaum, J., & Mordatch, I. (2020) · arXiv · doi:10.48550/arxiv.2012.01316 · Cited by 36
“Langevin sampling, our MCMC transition kernel, is prone to falling into local probability modes (Neal, 2011).”
BackgroundSpatial Prevalence and Determinants of Malaria among under-five Children in Ghana
Ejigu, B. A., & Wencheko, E. (2021) · doi:10.1101/2021.03.12.21253436 · Cited by 17
Diffusion Policy: Visuomotor Policy Learning via Action Diffusion
Chi, C., Xu, Z., Feng, S., Cousineau, E., Du, Y., Burchfiel, B., Tedrake, R., & Song, S. (2023) · arXiv · doi:10.48550/arxiv.2303.04137 · Cited by 12
“…the action score function (Song and Ermon, 2019) and performing Stochastic Langevin Dynamics sampling on this gradient fi eld, Diffusion policy can express arbitrary normalizable distributions (Neal, 2011), which includes multimodal action distributions, a well-known challenge for policy learning.”
Metadata
- Authors
- Huber, M.
- Venue
- Handbook of Markov Chain Monte Carlo
- Publisher
- Chapman and Hall/CRC
- Published
- 2011-05-10
- Pages
- 227-252
- OpenAlex
- W2478027467
- Cited by
- 1,176
- References
- 84
- FWCI
- 106.13
- Type
- book chapter
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.