The DOI matches its Crossref record; 47 of the 47 references it cites were found in a registry or an index.
Simulating normalizing constants: from importance sampling to bridge sampling to path sampling
Abstract
Computing (ratios of) normalizing constants of probability models is a fundamental computational problem for many statistical and scientific studies. Monte Carlo simulation is an effective technique, especially with complex and high-dimensional models. This paper aims to bring to the attention of general statistical audiences of some effective methods originating from theoretical physics and at the same time to explore these methods from a more statistical perspective, through establishing theoretical connections and illustrating their uses with statistical problems. We show that the acceptance ratio method and thermodynamic integration are natural generalizations of importance sampling, which is most familiar to statistical audiences. The former generalizes importance sampling through the use of a single "bridge" density and is thus a case of bridge sampling in the sense of Meng and Wong. Thermodynamic integration, which is also known in the numerical analysis literature as Ogata's method for high-dimensional integration, corresponds to the use of infinitely many and continuously connected bridges (and thus a "path"). Our path sampling formulation offers more flexibility and thus potential efficiency to thermodynamic integration, and the search of optimal paths turns out to have close connections with the Jeffreys prior density and the Rao and Hellinger distances between two densities. We provide an informative theoretical example as well as two empirical examples (involving 17- to 70-dimensional integrations) to illustrate the potential and implementation of path sampling. We also discuss some open problems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Equation of State Calculations by Fast Computing Machines
Metropolis, N., Rosenbluth, A. W., Rosenbluth, M. N., Teller, A. H., & Teller, E. (1953) · The Journal of Chemical Physics · doi:10.1063/1.1699114 · Cited by 37,192
Stochastic relaxation, gibbs distributions, and the bayesian restoration of images.
Geman, S., & Geman, D. (1984) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.1984.4767596 · Cited by 17,949
Inference from iterative simulation using multiple sequences
Gelman, A., & Rubin, D. B. (1992) · Statistical Science · doi:10.1214/ss/1177011136 · Cited by 16,627
Self-citationMonte Carlo sampling methods using Markov chains and their applications.
Hastings, W. K. (1970) · Biometrika · doi:10.1093/biomet/57.1.97 · Cited by 15,195
Sampling-based approaches to calculating marginal densities
Gelfand, A. E., & Smith, A. F. M. (1990) · Journal of the American Statistical Association · doi:10.1080/01621459.1990.10476213 · Cited by 6,606
Nonphysical sampling distributions in Monte Carlo free-energy estimation: Umbrella sampling
Torrie, G. M., & Valleau, J. P. (1977) · Journal of Computational Physics · doi:10.1016/0021-9991(77)90121-8 · Cited by 6,091
Spatial Interaction and the Statistical Analysis of Lattice Systems
Besag, J. (1974) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1974.tb00999.x · Cited by 6,067
Understanding Molecular Simulation
Frenkel, D., Smit, B., Tobochnik, J., McKay, S. R., & Christian, W. (1997) · Computers in Physics · doi:10.1063/1.4822570 · Cited by 5,395
Efficient estimation of free energy differences from Monte Carlo data
Bennett, C. H. (1976) · Journal of Computational Physics · doi:10.1016/0021-9991(76)90078-4 · Cited by 2,755
Path integrals in the theory of condensed helium
Ceperley, D. M. (1995) · Reviews of Modern Physics · doi:10.1103/revmodphys.67.279 · Cited by 2,470
Computing Bayes Factors by Combining Simulation and Asymptotic Approximations
Diciccio, T. J., Kass, R. E., Raftery, A., & Wasserman, L. (1997) · Journal of the American Statistical Association · doi:10.1080/01621459.1997.10474045 · Cited by 1,943
Marginal Likelihood from the Gibbs Output
Chib, S. (1995) · Journal of the American Statistical Association · doi:10.1080/01621459.1995.10476635 · Cited by 1,905
Simulated Tempering: A New Monte Carlo Scheme
Marinari, E., & Parisi, G. (1992) · Europhysics Letters (EPL) · doi:10.1209/0295-5075/19/6/002 · Cited by 1,818
Information and the Accuracy Attainable in the Estimation of Statistical Parameters
Rao, C. R. (1992) · Springer Series in Statistics · doi:10.1007/978-1-4612-0919-5_16 · Cited by 1,808
Sampling-Based Approaches to Calculating Marginal Densities
Gelfand, A. E., & Smith, A. F. M. (1990) · Journal of the American Statistical Association · doi:10.2307/2289776 · Cited by 1,504
Approximate Bayesian Inference with the Weighted Likelihood Bootstrap
Newton, M. A., & Raftery, A. E. (1994) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1994.tb01956.x · Cited by 1,487
Bayesian Model Choice: Asymptotics and Exact Calculations
Gelfand, A. E., & Dey, D. K. (1994) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1994.tb01996.x · Cited by 1,209
Multicanonical algorithms for first order phase transitions
Berg, B. A., & Neuhaus, T. (1991) · Physics Letters B · doi:10.1016/0370-2693(91)91256-u · Cited by 1,197
(1991) · University of Minnesota Digital Conservancy (University of Minnesota) · Cited by 1,077
Sequential Imputations and Bayesian Missing Data Problems
Kong, A., Liu, J. S., & Wong, W. H. (1994) · Journal of the American Statistical Association · doi:10.1080/01621459.1994.10476469 · Cited by 1,060
Monte Carlo Methods in Statistical Physics
(1986) · Topics in Current Physics · doi:10.1007/978-3-642-82803-4 · Cited by 1,058
Probabilistic Inference Using Markov Chain Monte Carlo Methods
Neal, R. M. (2011) · Cited by 1,047
Annealing Markov Chain Monte Carlo with Applications to Ancestral Inference
Geyer, C. J., & Thompson, E. A. (1995) · Journal of the American Statistical Association · doi:10.1080/01621459.1995.10476590 · Cited by 910
Constrained Monte Carlo Maximum Likelihood for Dependent Data
Geyer, C. J., & Thompson, E. A. (1992) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1992.tb01443.x · Cited by 874
SIMULATING RATIOS OF NORMALIZING CONSTANTS VIA A SIMPLE IDENTITY: A THEORETICAL EXPLORATION
Meng, X., & Wong, W. H. (1996) · Cited by 822
Self-citationEfficient Metropolis Jumping Rules
Gelman*, A., Roberts**, G. O., & Gilks***, W. R. (1996) · Bayesian Statistics 5 · doi:10.1093/oso/9780198523567.003.0038 · Cited by 796
Self-citationStatistical Inference for Spatial Processes
Ripley, B. D. (1988) · doi:10.1017/cbo9780511624131 · Cited by 546
New approach to spin-glass simulations
Berg, B. A., & Celik, T. (1992) · Physical Review Letters · doi:10.1103/physrevlett.69.2292 · Cited by 322
Hypothesis testing and model selection
(1995) · Markov Chain Monte Carlo in Practice · doi:10.1201/b14835-15 · Cited by 311
Evans, M., & Swartz, T. (1995) · Statistical Science · doi:10.1214/ss/1177009938 · Cited by 246
Estimating Bayes Factors via Posterior Simulation with the Laplace—Metropolis Estimator
Lewis, S. M., & Raftery, A. E. (1997) · Journal of the American Statistical Association · doi:10.1080/01621459.1997.10474016 · Cited by 227
Geometrical Foundations of Asymptotic Inference
Kass, R. E., & Vos, P. W. (1997) · Wiley Series in Probability and Statistics · doi:10.1002/9781118165980 · Cited by 226
Bayes Methods for Combining the Results of Cancer Studies in Humans and other Species
Dumouchel, W. H., & Harris, J. E. (1983) · Journal of the American Statistical Association · doi:10.1080/01621459.1983.10477968 · Cited by 193
Fitting Full-Information Item Factor Models and an Empirical Investigation of Bridge Sampling
Meng, X.-L., & Schilling, S. (1996) · Journal of the American Statistical Association · doi:10.1080/01621459.1996.10476995 · Cited by 161
Self-citationAtkinson, C. (1981) · Sankhya : The Indian journal of statistics/Sankhya · Cited by 160
A Monte Carlo method for high dimensional integration
Ogata, Y. (1989) · Numerische Mathematik · doi:10.1007/bf01406511 · Cited by 151
Likelihood Analysis of Spatial Point Patterns
Ogata, Y., & Tanemura, M. (1984) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.2517-6161.1984.tb01322.x · Cited by 148
Combining Historical and Randomized Controls for Assessing Trends in Proportions
Dempster, A. P., Selwyn, M. R., & Weeks, B. J. (1983) · Journal of the American Statistical Association · doi:10.1080/01621459.1983.10477953 · Cited by 126
Estimating the Probability of Events That Have Never Occurred: When Is Your Vote Decisive?
Gelman, A., King, G., & Boscardin, W. J. (1998) · Journal of the American Statistical Association · doi:10.2307/2669597 · Cited by 120
Self-citationOn Monte Carlo methods for estimating ratios of normalizing constants
Chen, M., & Shao, Q.-M. (1997) · The Annals of Statistics · doi:10.1214/aos/1031594732 · Cited by 108
Estimating Bayes Factors via Posterior Simulation With the Laplace-Metropolis Estimator
Lewis, S. M., & Raftery, A. E. (1997) · Journal of the American Statistical Association · doi:10.2307/2965712 · Cited by 103
Ott, J. (1979) · PubMed · Cited by 101
Introduction: Theory and “Technical” Aspects of Monte Carlo Simulations
Binder, K. (1986) · Topics in current physics · doi:10.1007/978-3-642-82803-4_1 · Cited by 96
Sobol, I. M. (1971) · Medical Entomology and Zoology · Cited by 84
Blocking Gibbs Sampling for Linkage Analysis in Large Pedigrees with Many Loops
Jensen, C. S., & Kong, A. (1999) · The American Journal of Human Genetics · doi:10.1086/302524 · Cited by 79
Free-energy computation and first-order phase transitions
Frenkel, D. (1986) · Utrecht University Repository (Utrecht University) · Cited by 75
Estimative and predictive distances
Mitchell, A. F. S. (1992) · Test · doi:10.1007/bf02562666 · Cited by 6
Cited by
Bayesian phylogenetic and phylodynamic data integration using BEAST 1.10
Suchard, M. A., Lemey, P., Baele, G., Ayres, D. L., Drummond, A. J., & Rambaut, A. (2018) · Virus Evolution · doi:10.1093/ve/vey016 · Cited by 3,792
(2003) · Cited by 1,680
Nested sampling for general Bayesian computation
Skilling, J. (2006) · Bayesian Analysis · doi:10.1214/06-ba127 · Cited by 1,670
A Bayesian Mixture Model for Across-Site Heterogeneities in the Amino-Acid Replacement Process
Lartillot, N., & Philippe, H. (2004) · Molecular Biology and Evolution · doi:10.1093/molbev/msh112 · Cited by 1,516
Riemann manifold langevin and hamiltonian monte carlo methods
Girolami, M., & Calderhead, B. (2011) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2010.00765.x · Cited by 1,473
Metadata
- Authors
- Gelman, A., & Meng, X.-L.
- Venue
- Statistical Science
- Publisher
- Institute of Mathematical Statistics
- Published
- 1998-05-01
- Volume
- 13
- Issue
- 2
- Pages
- 163-185
- ISSN
- 0883-4237 (ISSN-L)
- OpenAlex
- W2013164703
- Cited by
- 951
- References
- 47
- FWCI
- 11.83
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.