The DOI matches its Crossref record; 45 of the 47 references it cites were found in a registry or an index, and 2 are not checked yet.
Rank-Normalization, Folding, and Localization: An Improved Rˆ for Assessing Convergence of MCMC (with Discussion)
Open accessAbstract
Markov chain Monte Carlo is a key computational tool in Bayesian statistics, but it can be challenging to monitor the convergence of an iterative stochastic algorithm. In this paper we show that the convergence diagnostic Rˆ of Gelman and Rubin (1992) has serious flaws. Traditional Rˆ will fail to correctly diagnose convergence failures when the chain has a heavy tail or when the variance varies across the chains. In this paper we propose an alternative rank-based diagnostic that fixes these problems. We also introduce a collection of quantile-based local efficiency measures, along with a practical approach for computing Monte Carlo error estimates for quantiles. We suggest that common trace plots should be replaced with rank plots from multiple chains. Finally, we give recommendations for how these methods should be used in practice.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“All of our model estimates met these criteria.”
From The Effects of Generative AI on Design Fixation and Divergent Thinking (2024) · cites this paper as [65]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Model inference was performed by Hamiltonian Monte Carlo with the NUTS sampler using PyStan 69 , the Python implementation of Stan.”
From Measuring the impact of COVID-19 vaccine misinformation on vaccination intent in the UK and USA (2021) · cites this paper as [71]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Gelman, A., Carlin, J. B., Stern, H. S., & Rubin, D. B. (2003) · doi:10.1201/9780429258480 · Cited by 27,276
Self-citationInference from iterative simulation using multiple sequences
Gelman, A., & Rubin, D. B. (1992) · Statistical Science · doi:10.1214/ss/1177011136 · Cited by 16,627
Self-citationMonte Carlo sampling methods using Markov chains and their applications.
Hastings, W. K. (1970) · Biometrika · doi:10.1093/biomet/57.1.97 · Cited by 15,195
Stan : A Probabilistic Programming Language
Carpenter, B., Gelman, A., Hoffman, M. D., Lee, D., Goodrich, B., Betancourt, M., Brubaker, M., Guo, J., Li, P., & Riddell, A. (2017) · Journal of Statistical Software · doi:10.18637/jss.v076.i01 · Cited by 7,754 · cited 2×
“Adapted from Gelman et al. (2013). packages for MCMC sampling such as Stan (Carpenter et al., 2017), JAGS (Plummer, 2003), WinBUGS (Lunn et al., 2000), OpenBUGS (Lunn et al., 2009), PyMC3 (Salvatier et al., 2016), and NIMBLE (de Valpine et al., 2017), which together are estimated to have hundreds…”
MethodSelf-citationThe use of ranks to avoid the assumption of normality implicit in the analysis of variance
Friedman, M. (1937) · Journal of the American Statistical Association · doi:10.1080/01621459.1937.10503522 · Cited by 7,005
Monte Carlo Statistical Methods
Robert, C., & Casella, G. (2004) · Springer Texts in Statistics · doi:10.1007/978-1-4757-4145-2 · Cited by 5,871
New extension of the Kalman filter to nonlinear systems
Julier, S., & Uhlmann, J. K. (1997) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.280797 · Cited by 5,619
WinBUGS - A Bayesian modelling framework: Concepts, structure, and extensibility
Lunn, D. J., Thomas, A., Best, N., & Spiegelhalter, D. J. (2000) · Statistics and Computing · doi:10.1023/a:1008929526011 · Cited by 5,397
JAGS: A program for analysis of Bayesian graphical models using Gibbs sampling
Plummer, M. (2003) · Cited by 4,720
CODA: convergence diagnosis and output analysis for MCMC
Plummer, M., Best, N., Cowles, K., & Vines, K. (2006) · Open Research Online (The Open University) · Cited by 3,317
The No-U-Turn Sampler: Adaptively Setting Path Lengths in Hamiltonian Monte CarloPreprint
Hoffman, M. D., & Gelman, A. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1111.4246 · Cited by 2,815
Self-citationProbabilistic programming in Python using PyMC3
Salvatier, J., Wiecki, T. V., & Fonnesbeck, C. (2016) · PeerJ Computer Science · doi:10.7717/peerj-cs.55 · Cited by 2,544
Markov Chain Monte Carlo Convergence Diagnostics: A Comparative Review
Cowles, M. K., & Carlin, B. P. (1996) · Journal of the American Statistical Association · doi:10.1080/01621459.1996.10476956 · Cited by 2,060
General Methods for Monitoring Convergence of Iterative Simulations
Brooks, S. P., & Gelman, A. (1998) · Journal of Computational and Graphical Statistics · doi:10.2307/1390675 · Cited by 2,017
Self-citationThe BUGS project: evolution, critique and future directions
Lunn, D., Spiegelhalter, D., Thomas, A., & Best, N. (2009) · Statistics in Medicine · doi:10.1002/sim.3680 · Cited by 1,876
Markov Chains and Stochastic Stability
Meyn, S., Tweedie, R. L., & Glynn, P. W. (2009) · Cambridge University Press eBooks · doi:10.1017/cbo9780511626630 · Cited by 1,688
(2000) · Time to knit · doi:10.1007/978-1-4899-4485-6 · Cited by 1,150
Programming With Models: Writing Statistical Algorithms for General Model Structures With NIMBLE
de Valpine, P., Turek, D., Paciorek, C. J., Anderson-Bergman, C., Lang, D. T., & Bodik, R. (2016) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.2016.1172487 · Cited by 1,069
Sequential Imputations and Bayesian Missing Data Problems
Kong, A., Liu, J. S., & Wong, W. H. (1994) · Journal of the American Statistical Association · doi:10.1080/01621459.1994.10476469 · Cited by 1,061
How Many Iterations in the Gibbs Sampler?
Raftery, A. E., & Lewis, S. (1991) · Defense Technical Information Center (DTIC) · doi:10.21236/ada640705 · Cited by 972
Bayesian WorkflowPreprint
Gelman, A., Vehtari, A., Simpson, D., Margossian, C. C., Carpenter, B., Yao, Y., Kennedy, L., Gabry, J., Bürkner, P., & Modrák, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.01808 · Cited by 553
Self-citationTesting Statistical Hypotheses of Equivalence and Noninferiority
Wellek, S. (2010) · doi:10.1201/ebk1439808184 · Cited by 548 · cited 1×
“In addition, small probability intervals can be used in practical equivalence testing (see, e.g., Wellek, 2010).”
MethodAsymptotic Normality and Efficiency of Certain Nonparametric Test Statistics
Chernoff, H., & Savage, I. R. (1958) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177706436 · Cited by 513
Bayesian inference in threshold models using Gibbs sampling
Sorensen, D., Andersen, S., Daniel Gianola, B., & Korsgaard, I. R. (1995) · Genetics Selection Evolution · doi:10.1186/1297-9686-27-3-229 · Cited by 370
Markov Chain Monte Carlo Convergence Diagnostics: A Comparative Review
Cowles, M. K., & Carlin, B. P. (1996) · Journal of the American Statistical Association · doi:10.2307/2291683 · Cited by 360
Introduction to Markov Chain Monte Carlo
Geyer, C. J. (2011) · Chapman & Hall/CRC eBooks · doi:10.1201/b10905-2 · Cited by 334
Convergence Diagnostics for Markov Chain Monte Carlo
Roy, V. (2019) · Annual Review of Statistics and Its Application · doi:10.1146/annurev-statistics-031219-041300 · Cited by 301
Markov Chain Monte Carlo: Can We Trust the Third Significant Figure?
Flegal, J. M., Haran, M., & Jones, G. L. (2008) · Statistical Science · doi:10.1214/08-sts257 · Cited by 287
Multivariate output analysis for Markov chain Monte Carlo
Vats, D., Flegal, J. M., & Jones, G. L. (2019) · Biometrika · doi:10.1093/biomet/asz002 · Cited by 278
Hamiltonian Monte Carlo for Hierarchical Models
Betancourt, M., & Girolami, M. (2015) · Chapman & Hall/CRC eBooks · doi:10.1201/b18502-5 · Cited by 257
Batch means and spectral variance estimators in Markov chain Monte Carlo
Flegal, J. M., & Jones, G. L. (2010) · The Annals of Statistics · doi:10.1214/09-aos735 · Cited by 240
Douc, R., Moulines, É., Priouret, P., et al. (2018) · Springer series in operations research/Springer series in operations research and financial engineering · doi:10.1007/978-3-319-97704-1 · Cited by 196
Revisiting the Gelman–Rubin Diagnostic
Vats, D., & Knudson, C. P. (2021) · Statistical Science · doi:10.1214/20-sts812 · Cited by 185
Renewal theory and computable convergence rates for geometrically ergodic Markov chains
Baxendale, P. H. (2005) · The Annals of Applied Probability · doi:10.1214/105051604000000710 · Cited by 181
Quantitative bounds on convergence of time-inhomogeneous Markov chains
Douc, R., Moulines, É., & Rosenthal, J. S. (2004) · The Annals of Applied Probability · doi:10.1214/105051604000000620 · Cited by 116
MCMC Convergence Diagnostics: A Reviewww
Mengersen, K. L., Robert, C. P., & Guihenneuc‐Jouyaux, C. (1999) · doi:10.1093/oso/9780198504856.003.0018 · Cited by 106
Estimating the asymptotic variance with batch means
Glynn, P. W., & Whitt, W. (1991) · Operations Research Letters · doi:10.1016/0167-6377(91)90019-l · Cited by 105
Sufficient burn-in for Gibbs samplers for a hierarchical random effects model
Jones, G. L., & Hobert, J. P. (2004) · The Annals of Statistics · doi:10.1214/009053604000000184 · Cited by 84
A Practical Sequential Stopping Rule for High-Dimensional Markov Chain Monte Carlo
Gong, L., & Flegal, J. M. (2015) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.2015.1044092 · Cited by 82
Unbiased Hamiltonian Monte Carlo with couplings
Heng, J., & Jacob, P. E. (2018) · Biometrika · doi:10.1093/biomet/asy074 · Cited by 46
[Practical Markov Chain Monte Carlo]: Rejoinder
Geyer, C. J. (1992) · Statistical Science · doi:10.1214/ss/1177011147 · Cited by 44
Markov chain Monte Carlo estimation of quantiles
Doss, C. R., Flegal, J. M., Jones, G. L., & Neath, R. C. (2014) · Electronic Journal of Statistics · doi:10.1214/14-ejs957 · Cited by 38
Introduction: Conceptual issues in quantum field theory
Cao, T. Y. (1999) · Cambridge University Press eBooks · doi:10.1017/cbo9780511470813.003 · Cited by 27
The Number of MCMC Draws Needed to Compute Bayesian Credible Bounds
Liu, J., Nordman, D. J., & Meeker, W. Q. (2016) · The American Statistician · doi:10.1080/00031305.2016.1158738 · Cited by 20
Corrigendum to “Bounds on regeneration times and convergence rates for Markov chains”
Roberts, G. O., & Tweedie, R. L. (2001) · Stochastic Processes and their Applications · doi:10.1016/s0304-4149(00)00074-0 · Cited by 13
Multiple Ergodicity For Reversible Markov Chains
Liu, Y. (2018) · Acta Mathematicae Applicatae Sinica English Series · doi:10.1007/s10255-018-0779-1
Flagged first, then by citations
Research works printed but not linked: 3
- Blom, G. (1958). Statistical Estimates and Transformed Beta-Variables. Wiley; New York.
- Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A., and Rubin, D. B. (2013). Bayesian Data Analysis, third edition. CRC Press. · DOI · Paper page
- Vehtari, A., Gelman, A., Simpson, D., Carpenter, B., and Bürkner, P.-C. (2020). “Rank-Normalization, Folding, and Localization: An Improved Rˆ for Assessing Convergence of MCMC. Supplementary Material.” Bayesian Analysis. · DOI
Cited by
Sorted by citations
dynesty: a dynamic nested sampling package for estimating Bayesian posteriors and evidences
Speagle, J. S. (2020) · Monthly Notices of the Royal Astronomical Society · doi:10.1093/mnras/staa278 · Cited by 2,697
Measuring the impact of COVID-19 vaccine misinformation on vaccination intent in the UK and USA
Loomba, S., de Figueiredo, A., Piatek, S. J., de Graaf, K., & Larson, H. J. (2021) · Nature Human Behaviour · doi:10.1038/s41562-021-01056-1 · Cited by 1,800
The NANOGrav 15 yr Data Set: Evidence for a Gravitational-wave Background
Agazie, G., Anumarlapudi, A., Archibald, A. M., Arzoumanian, Z., Baker, P. T., Bécsy, B., Blecha, L., Brazier, A., Brook, P. R., Burke-Spolaor, S., et al. (2023) · The Astrophysical Journal Letters · doi:10.3847/2041-8213/acdac6 · Cited by 1,489
Bayesian statistics and modelling
van de Schoot, R., Depaoli, S. A., King, R., Kramer, B. M. R., Märtens, K., Tadesse, M. G., Vannucci, M., Gelman, A., Veen, D., Willemsen, J. E., et al. (2021) · Nature Reviews Methods Primers · doi:10.1038/s43586-020-00001-2 · Cited by 1,338
Self-citationExploring the potential of using an AI language model for automated essay scoring
Mizumoto, A., & Eguchi, M. (2023) · Research Methods in Applied Linguistics · doi:10.1016/j.rmal.2023.100050 · Cited by 647
Metadata
- Authors
- Vehtari, A., Gelman, A., Simpson, D., Carpenter, B., & Bürkner, P.-C.
- Affiliations
- Aalto University (FI)
- Columbia University (US)
- University of Toronto (CA)
- Flatiron Institute (US)
- Venue
- Bayesian Analysis
- Publisher
- Institute of Mathematical Statistics
- Published
- 2021-06-01
- Volume
- 16
- Issue
- 2
- Pages
- 667-718
- ISSN
- 1931-6690 (ISSN-L)
- 1936-0975
- OpenAlex
- W2920804790
- Cited by
- 1,810
- References
- 32
- Citations by year
- 2026: 499 · 2025: 424 · 2024: 328 · 2023: 263 · 2022: 153 · 2021: 93
- FWCI
- 96.56
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by
- Locations
- Bayesian Analysis (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- HAL (submittedVersion, other-oa)
- Aaltodoc (Aalto University) (submittedVersion, cc-by)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Markov Chains and Monte Carlo Methods
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Physical Sciences › Mathematics › Statistics and Probability › Data Analysis with R
- Keywords
- convergence diagnostics, Markov chain Monte Carlo, heavy-tailed distributions, Monte Carlo error estimation, trace plots
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
