Sampling-based approaches to calculating marginal densities
Abstract
Stochastic substitution, the Gibbs sampler, and the sampling-importance-resampling algorithm can be viewed as three alternative sampling- (or Monte Carlo-) based approaches to the calculation of numerical estimates of marginal probability distributions. The three approaches will be reviewed, compared, and contrasted in relation to various joint probability structures frequently encountered in applications. In particular, the relevance of the approaches to calculating Bayesian posterior densities for a variety of structured models will be discussed and illustrated.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The latent variable of topics can be estimated through Gibbs sampling.
“since the words in a document are known, the latent variable of topics can be estimated through Gibbs sampling [74].”
From Personality, Gender, and Age in the Language of Social Media: The Open-Vocabulary Approach (2013) · cites this paper as [74]
Cites
Maximum Likelihood from Incomplete Data Via the <i>EM</i> Algorithm
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1977.tb01600.x · Cited by 50,444 · see citing works
Stochastic relaxation, gibbs distributions, and the bayesian restoration of images.
Geman, S., & Geman, D. (1984) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.1984.4767596 · Cited by 17,949 · see citing works
Monte Carlo sampling methods using Markov chains and their applications.
Hastings, W. K. (1970) · Biometrika · doi:10.1093/biomet/57.1.97 · Cited by 15,195 · see citing works
Spatial Interaction and the Statistical Analysis of Lattice Systems
Besag, J. (1974) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1974.tb00999.x · Cited by 6,067 · see citing works
Maximum likelihood estimation from incomplete data via the EM algorithm
Dempster, A. P. (1977) · Journal of the Royal Statistical Society Series A (Statistics in Society) · Cited by 4,506 · see citing works
Non-Uniform Random Variate Generation
Devroye, L. (1986) · Springer eBooks · doi:10.1007/978-1-4613-8643-8 · Cited by 4,110 · see citing works
The Calculation of Posterior Distributions by Data Augmentation
Tanner, M. A., & Wong, W. H. (1987) · Journal of the American Statistical Association · doi:10.1080/01621459.1987.10478458 · Cited by 3,929 · see citing works
Bayesian Inference in Statistical Analysis.
Kadane, J. B., Box, G. E. P., & Tiao, G. C. (1975) · Journal of the American Statistical Association · doi:10.2307/2285412 · Cited by 3,869 · see citing works
Ripley, B. D. (1987) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316726 · Cited by 2,241 · see citing works
Accurate Approximations for Posterior Moments and Marginal Densities
Tierney, L., & Kadane, J. B. (1986) · Journal of the American Statistical Association · doi:10.1080/01621459.1986.10478240 · Cited by 2,136 · see citing works
Bayesian Inference in Statistical Analysis
Hill, B. M., Box, G. E. P., & Tiao, G. C. (1974) · Technometrics · doi:10.2307/1267683 · Cited by 1,465 · see citing works
Parametric empirical Bayes inference: theory and applications
Morris, C. N. (1983) · Journal of the American Statistical Association · doi:10.1080/01621459.1983.10477920 · Cited by 1,448 · see citing works
Nonparametric Density Estimation: The L 1 View.
Taylor, C. C., Devroye, L., & Györfi, L. (1985) · Journal of the Royal Statistical Society Series A (General) · doi:10.2307/2981908 · Cited by 739 · see citing works
Maximum Likelihood Estimation from Incomplete Data
Hartley, H. O. (1958) · Biometrics · doi:10.2307/2527783 · Cited by 602 · see citing works
Computational Solution of Nonlinear Operator Equations
O., J. M., & Rall, L. B. (1970) · Mathematics of Computation · doi:10.2307/2004896 · Cited by 529 · see citing works
Applications of a Method for the Efficient Computation of Posterior Distributions
Naylor, J. C., & Smith, A. F. M. (1982) · Journal of the Royal Statistical Society Series C (Applied Statistics) · doi:10.2307/2347995 · Cited by 396 · see citing works
Self-citationNatural Exponential Families with Quadratic Variance Functions: Statistical Theory
Morris, C. N. (1983) · The Annals of Statistics · doi:10.1214/aos/1176346158 · Cited by 253 · see citing works
Antithetic acceleration of Monte Carlo integration in Bayesian inference
Geweke, J. F. (1988) · Journal of Econometrics · doi:10.1016/0304-4076(88)90027-9 · Cited by 204 · see citing works
Rubin, D. B. (1987) · Journal of the American Statistical Association · doi:10.1080/01621459.1987.10478461 · Cited by 113 · see citing works
The implementation of the bayesian paradigm
Smith, A. F. M., Skene, A. M., Shaw, J. E. H., Naylor, J. C., & Dransfield, M. T. (1985) · Communication in Statistics- Theory and Methods · doi:10.1080/03610928508828963 · Cited by 107 · see citing works
Self-citationA Quasirandom Approach to Integration in Bayesian Statistics
Shaw, J. E. H. (1988) · The Annals of Statistics · doi:10.1214/aos/1176350842 · Cited by 102 · see citing works
Statistical Decision Theory and Related Topics
(1977) · Elsevier eBooks · doi:10.1016/c2013-0-10779-9 · Cited by 94 · see citing works
Progress with Numerical and Graphical Methods for Practical Bayesian Statistics
Smith, A. F. M., Skene, A. M., Shaw, J. E. H., & Naylor, J. C. (1987) · Journal of the Royal Statistical Society Series D (The Statistician) · doi:10.2307/2348499 · Cited by 85 · see citing works
Self-citationRobust Empirical Bayes Analyses of Event Rates
Gaver, D. P., & O’Muircheartaigh, I. G. (1987) · Technometrics · doi:10.1080/00401706.1987.10488178 · Cited by 77 · see citing works
Econometric illustrations of novel numerical integration strategies for Bayesian inference
Naylor, J. C., & Smith, A. F. M. (1988) · Journal of Econometrics · doi:10.1016/0304-4076(88)90029-2 · Cited by 52 · see citing works
Self-citationSchwetlick, H. (1972) · ZAMM ‐ Journal of Applied Mathematics and Mechanics / Zeitschrift für Angewandte Mathematik und Mechanik · doi:10.1002/zamm.19720521021 · Cited by 36 · see citing works
On Parametric Density Estimation
El-Sayyad, G. M., SAMUDDIN, M., & Al-Harbey, A. A. (1989) · Biometrika · doi:10.1093/biomet/76.2.343 · Cited by 34 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Inference from iterative simulation using multiple sequences
Gelman, A., & Rubin, D. B. (1992) · Statistical Science · doi:10.1214/ss/1177011136 · Cited by 16,627 · see citing works
<b>brms</b> : An <i>R</i> Package for Bayesian Multilevel Models Using <i>Stan</i>
Bürkner, P.-C. (2017) · Journal of Statistical Software · doi:10.18637/jss.v080.i01 · Cited by 9,476 · see citing works
“…are primarily using combinations of Metropolis-Hastings updates (Metropolis, Rosenbluth, Rosenbluth, Teller, and Teller 1953; Hastings 1970) and Gibbs-sampling (Geman and Ge-man 1984; Gelfand and Smith 1990), sometimes also coupled with slice-sampling (Damien, WakeĄeld, and Walker 1999; Neal 2003).”
MethodBayesian Analysis of Radiocarbon Dates
Bronk Ramsey, C. (2009) · Radiocarbon · doi:10.1017/s0033822200033865 · Cited by 8,725 · see citing works
General methods for monitoring convergence of iterative simulations
Brooks, S. P., & Gelman, A. (1998) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.1998.10474787 · Cited by 6,045 · see citing works
Gelman, A. (2006) · Bayesian Analysis · doi:10.1214/06-ba117a · Cited by 4,050 · see citing works
“Typically, enough data will be available to estimate µ and σy that one can use any reasonable noninformative prior distribution—for example, p(µ, σy) ∝ 1 or p(µ, log σy) ∝ 1.”
Multiple imputation after 18+ years
Rubin, D. B. (1996) · Journal of the American Statistical Association · doi:10.1080/01621459.1996.10476908 · Cited by 4,072 · see citing works
Variable Selection via Gibbs Sampling
George, E. I., & McCulloch, R. E. (1993) · Journal of the American Statistical Association · doi:10.1080/01621459.1993.10476353 · Cited by 2,828 · see citing works
An introduction to MCMC for machine learning.
Andrieu, C., de Freitas, N., Doucet, A., & Jordan, M. I. (2003) · Machine Learning · doi:10.1023/a:1020281327116 · Cited by 2,451 · see citing works
Practical markov chain monte carlo
Geyer, C. J. (1992) · Statistical Science · doi:10.1214/ss/1177011137 · Cited by 2,045 · see citing works
Default Bayes factors for ANOVA designs
Rouder, J. N., Morey, R. D., Speckman, P. L., & Province, J. M. (2012) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2012.08.001 · Cited by 1,800 · see citing works
Bayesian computation via the Gibbs sampler and related Markov chain Monte Carlo methods
Smith, A. F. M., & Roberts, G. O. (1993) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1993.tb01466.x · Cited by 1,713 · see citing works
Self-citationPersonality, Gender, and Age in the Language of Social Media: The Open-Vocabulary Approach
Schwartz, H. A., Eichstaedt, J. C., Kern, M. L., Dziurzynski, L., Ramones, S. M., Agrawal, M., Shah, A., Kosinski, M., Stillwell, D., Seligman, M. E. P., et al. (2013) · PLoS ONE · doi:10.1371/journal.pone.0073791 · Cited by 1,700 · see citing works
Modeling the Drop-Out Mechanism in Repeated-Measures Studies
Little, R. J. A. (1995) · Journal of the American Statistical Association · doi:10.1080/01621459.1995.10476615 · Cited by 1,522 · see citing works
The Value of Purchase History Data in Target Marketing
Rossi, P. E., McCulloch, R. E., & Allenby, G. M. (1996) · Marketing Science · doi:10.1287/mksc.15.4.321 · Cited by 754 · see citing works
Bayesian approaches to random-effects meta-analysis: a comparative study
Smith, T. C., Spiegelhalter, D. J., & Thomas, A. (1995) · Statistics in Medicine · doi:10.1002/sim.4780142408 · Cited by 568 · see citing works
Parameter expansion to accelerate EM: the PX-EM algorithm
Liu, C. (1998) · Biometrika · doi:10.1093/biomet/85.4.755 · Cited by 433 · see citing works
The Effect of Improper Priors on Gibbs Sampling in Hierarchical Linear Mixed Models
Hobert, J. P., & Casella, G. (1996) · Journal of the American Statistical Association · doi:10.1080/01621459.1996.10476714 · Cited by 435 · see citing works
Clark, J. S., & Bjørnstad, O. N. (2004) · Ecology · doi:10.1890/03-0520 · Cited by 348 · see citing works
Air Pollution and Mortality: Estimating Regional and National Dose-Response Relationships
Dominici, F., Daniels, M., Zeger, S. L., & Samet, J. M. (2002) · Journal of the American Statistical Association · doi:10.1198/016214502753479266 · Cited by 303 · see citing works
A Disease-Mediated Trophic Cascade in the Serengeti and its Implications for Ecosystem C
Holdo, R. M., Sinclair, A. R. E., Dobson, A. P., Metzger, K. L., Bolker, B. M., Ritchie, M. E., & Holt, R. D. (2009) · PLoS Biology · doi:10.1371/journal.pbio.1000210 · Cited by 298 · see citing works
“We used WinBUGS 1.4 [49,62], which uses Gibbs sampling, a Markov Chain Monte Carlo (MCMC) technique [ 63 ], to generate these estimates.”
MethodInfluentialDominici, F., Samet, J. M., & Zeger, S. L. (2000) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/1467-985x.00170 · Cited by 286 · see citing works
Probabilistic projections of the total fertility rate for all countries
Alkema, L., Raftery, A. E., Gerland, P., Clark, S. J., Pelletier, F., Buettner, T., & Heilig, G. K. (2011) · Demography · doi:10.1007/s13524-011-0040-5 · Cited by 248 · see citing works
Calibration of radiocarbon results pertaining to related archaeological events
Buck, C. E., Litton, C. D., & Smith, A. F. M. (1992) · Journal of Archaeological Science · doi:10.1016/0305-4403(92)90025-x · Cited by 244 · see citing works
Self-citationReverse engineering and identification in systems biology: strategies, perspectives and challenges
Villaverde, A. F., & Banga, J. R. (2014) · Journal of The Royal Society Interface · doi:10.1098/rsif.2013.0505 · Cited by 229 · see citing works
Identifiability, improper priors and Gibbs sampling for generalized linear models
Gelfand, A. E., & Sahu, S. K. (1999) · Journal of the American Statistical Association · doi:10.1080/01621459.1999.10473840 · Cited by 242 · see citing works
Self-citationTao, F., Yokozawa, M., & Zhang, Z. (2009) · Agricultural and Forest Meteorology · doi:10.1016/j.agrformet.2008.11.004 · Cited by 231 · see citing works
Chao, F., You, D., Pedersen, J., Hug, L., & Alkema, L. (2018) · The Lancet Global Health · doi:10.1016/s2214-109x(18)30059-7 · Cited by 174 · see citing works
IIZUMI, T., YOKOZAWA, M., & NISHIMORI, M. (2009) · Agricultural and Forest Meteorology · doi:10.1016/j.agrformet.2008.08.015 · Cited by 191 · see citing works
Optimizing the Marketing Interventions Mix in Intermediate-Term CRM
Rust, R. T., & Verhoef, P. C. (2005) · Marketing Science · doi:10.1287/mksc.1040.0107 · Cited by 191 · see citing works
Meta‐analysis for 2 × 2 tables: A bayesian approach
Carlin, J. B. (1992) · Statistics in Medicine · doi:10.1002/sim.4780110202 · Cited by 164 · see citing works
Inference for Multivariate Normal Hierarchical Models
Everson, P. J., & Morris, C. N. (2000) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/1467-9868.00239 · Cited by 107 · see citing works
Trying to be precise about vagueness.
Senn, S. (2007) · Statistics in Medicine · doi:10.1002/sim.2639 · Cited by 105 · see citing works
Model and parameter uncertainty in IDF relationships under climate change
Chandra, R., Saha, U., & Mujumdar, P. P. (2015) · Advances in Water Resources · doi:10.1016/j.advwatres.2015.02.011 · Cited by 96 · see citing works
Current analysis platforms and methods for detecting copy number variation
Li, W., & Olivier, M. (2013) · Physiological Genomics · doi:10.1152/physiolgenomics.00082.2012 · Cited by 82 · see citing works
Simulation driven inferences for multiply imputed longitudinal datasets*
Demirtas, H. (2004) · Statistica Neerlandica · doi:10.1111/j.1467-9574.2004.00271.x · Cited by 74 · see citing works
Mapping malaria risk among children in Côte d’Ivoire using Bayesian geo-statistical models
Raso, G., Schur, N., Utzinger, J., Koudou, B. G., Tchicaya, E. S., Rohner, F., N’Goran, E. K., Silué, K. D., Matthys, B., Assi, S., et al. (2012) · Malaria Journal · doi:10.1186/1475-2875-11-160 · Cited by 67 · see citing works
Donkor, E., Kelly, M., Eliason, C., Amotoh, C., Gray, D. J., Clements, A. C. A., & Wangdi, K. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18116080 · Cited by 30 · see citing works
“Markov Chain Monte Carlo simulation with Gibbs sampling was used to estimate model parameters (18).”
MethodBreyer, L. A., & Roberts, G. O. (2001) · Methodology And Computing In Applied Probability · doi:10.1023/a:1012205210377 · Cited by 25 · see citing works
Wang, Y. W., Zhang, C., Ni, Y. Q., & Xu, X. Y. (2022) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2021.108147 · Cited by 24 · see citing works
Panunzi, S., Pompa, M., Borri, A., Piemonte, V., & De Gaetano, A. (2020) · PLoS ONE · doi:10.1371/journal.pone.0237215 · Cited by 21 · see citing works
Bayesian Genomic-Enabled Prediction as an Inverse Problem
Cuevas, J., Pérez-Elizalde, S., Soberanis, V., Pérez-Rodríguez, P., Gianola, D., & Crossa, J. (2014) · G3 Genes|Genomes|Genetics · doi:10.1534/g3.114.013094 · Cited by 10 · see citing works
- Authors
- Gelfand, A. E., & Smith, A. F. M.
- Affiliations
- University of Connecticut (US)
- University of London (GB)
- ORCID
- Gelfand, Alan E.
- Venue
- Journal of the American Statistical Association
- Publisher
- Informa UK Limited
- Published
- 1990-06
- Volume
- 85
- Issue
- 410
- Pages
- 398-409
- ISSN
- 0162-1459 (ISSN-L)
- 1537-274X
- OpenAlex
- W2083875149
- Cited by
- 6,606
- References
- 27
- Citations by year
- 2026: 77 · 2025: 116 · 2024: 119 · 2023: 119 · 2022: 123 · 2021: 222 · 2020: 194 · 2019: 269 · 2018: 190 · 2017: 191 · 2016: 205 · 2015: 216 · 2014: 282 · 2013: 269 · 2012: 294
- FWCI
- 109.42
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of the American Statistical Association (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Markov Chains and Monte Carlo Methods
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Physical Sciences › Computer Science › Artificial Intelligence › Bayesian Modeling and Causal Inference
- Keywords
- marginal densities, Gibbs sampling, sampling importance resampling, Monte Carlo estimation, structured models, marginal distributions
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
