Computing Bayes factors using thermodynamic integration.
Open accessAbstract
In the Bayesian paradigm, a common method for comparing two models is to compute the Bayes factor, defined as the ratio of their respective marginal likelihoods. In recent phylogenetic works, the numerical evaluation of marginal likelihoods has often been performed using the harmonic mean estimation procedure. In the present article, we propose to employ another method, based on an analogy with statistical physics, called thermodynamic integration. We describe the method, propose an implementation, and show on two analytical examples that this numerical method yields reliable estimates. In contrast, the harmonic mean estimator leads to a strong overestimation of the marginal likelihood, which is all the more pronounced as the model is higher dimensional. As a result, the harmonic mean estimator systematically favors more parameter-rich models, an artefact that might explain some recent puzzling observations, based on harmonic mean estimates, suggesting that Bayes factors tend to overscore complex models. Finally, we apply our method to the comparison of several alternative models of amino-acid replacement. We confirm our previous observations, indicating that modeling pattern heterogeneity across sites tends to yield better models than standard empirical matrices.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Metropolis-coupled MCMC improves mixing between multiple modes of the posterior distribution and allows improved estimation of the marginal likelihood.
“We sample from the posterior distribution of model parameters using a Markov-chain Monte Carlo technique known as Metropolis-coupled MCMC, or [59], which improves mixing between multiple modes of the posterior distribution, and also allows improved estimation of the marginal likelihood [60].”
From Spatial Guilds in the Serengeti Food Web Revealed by a Bayesian Group Model (2011) · cites this paper as [60]
The Reliability of Marginal Likelihood Estimators section (page not provided) says thermodynamic integration gives more reliable marginal-likelihood estimates. The paper describes Metropolis-coupled heated chains as similar to its integration chains, but does not say that Metropolis-coupled MCMC itself improves marginal-likelihood estimation. The mixing claim is attached to a different citation in the citing sentence. (LLM, conf=0.85)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Thompson, J. D., Higgins, D. G., & Gibson, T. J. (1994) · Nucleic Acids Research · doi:10.1093/nar/22.22.4673 · Cited by 64,617 · see citing works
Estimating the Dimension of a Model
Schwarz, G. (1978) · The Annals of Statistics · doi:10.1214/aos/1176344136 · Cited by 47,033 · see citing works
Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A., & Rubin, D. B. (2013) · doi:10.1201/b16018 · Cited by 27,276 · see citing works
MRBAYES: Bayesian inference of phylogenetic trees.
Huelsenbeck, J. P., & Ronquist, F. (2001) · Bioinformatics · doi:10.1093/bioinformatics/17.8.754 · Cited by 21,937 · see citing works
Evolutionary trees from DNA sequences: A maximum likelihood approach.
Felsenstein, J. (1981) · Journal of Molecular Evolution · doi:10.1007/bf01734359 · Cited by 14,571 · see citing works
Kass, R. E., & Raftery, A. E. (1995) · Journal of the American Statistical Association · doi:10.1080/01621459.1995.10476572 · Cited by 12,558 · see citing works
Cross validatory choice and assessment of statistical predictions
Stone, M. (1974) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1974.tb00994.x · Cited by 10,881 · see citing works
Selection of Conserved Blocks from Multiple Alignments for Their Use in Phylogenetic Analysis
Castresana, J. (2000) · Molecular Biology and Evolution · doi:10.1093/oxfordjournals.molbev.a026334 · Cited by 10,447 · see citing works
The rapid generation of mutation data matrices from protein sequences
Jones, D. T., Taylor, W. R., & Thornton, J. M. (1992) · Computer applications in the biosciences · doi:10.1093/bioinformatics/8.3.275 · Cited by 6,916 · see citing works
Reversible Jump Markov Chain Monte Carlo Computation and Bayesian Model Determination
Green, P. J. (1995) · Biometrika · doi:10.1093/biomet/82.4.711 · Cited by 5,941 · see citing works
Probability theory the logic of science
Jaynes, E. T., & Bretthorst, G. L. (2003) · Cited by 4,600 · see citing works
Probability Theory: The Logic of Science
Jaynes, E. T. (2003) · doi:10.1017/cbo9780511790423 · Cited by 4,045 · see citing works
Phylogenetic relationships among ascomycetes: evidence from an RNA polymerse II subunit
Liu, Y. J., Whelen, S., & Hall, B. D. (1999) · Molecular Biology and Evolution · doi:10.1093/oxfordjournals.molbev.a026092 · Cited by 3,813 · see citing works
Yang, Z. (1994) · Journal of Molecular Evolution · doi:10.1007/bf00160154 · Cited by 2,960 · see citing works
Whelan, S., & Goldman, N. (2001) · Molecular Biology and Evolution · doi:10.1093/oxfordjournals.molbev.a003851 · Cited by 2,849 · see citing works
Markov Chain Sampling Methods for Dirichlet Process Mixture Models
Neal, R. M. (2000) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.2000.10474879 · Cited by 2,204 · see citing works
Practical markov chain monte carlo
Geyer, C. J. (1992) · Statistical Science · doi:10.1214/ss/1177011137 · Cited by 2,045 · see citing works
Bayesian Phylogenetic Analysis of Combined Data
Nylander, J. A. A., Ronquist, F., Huelsenbeck, J. P., & Nieves‐Aldrey, J. L. (2004) · Systematic Biology · doi:10.1080/10635150490264699 · Cited by 1,854 · see citing works
Marginal Likelihood from the Gibbs Output
Chib, S. (1995) · Journal of the American Statistical Association · doi:10.1080/01621459.1995.10476635 · Cited by 1,815 · see citing works
Markov Chasin Monte Carlo Algorithms for the Bayesian Analysis of Phylogenetic Trees
Larget, B. R., & Simon, D. L. (1999) · Molecular Biology and Evolution · doi:10.1093/oxfordjournals.molbev.a026160 · Cited by 1,624 · see citing works
A Bayesian Mixture Model for Across-Site Heterogeneities in the Amino-Acid Replacement Process
Lartillot, N., & Philippe, H. (2004) · Molecular Biology and Evolution · doi:10.1093/molbev/msh112 · Cited by 1,516 · see citing works
Self-citationBayesianly Justifiable and Relevant Frequency Calculations for the Applied Statistician
Rubin, D. (1984) · The Annals of Statistics · doi:10.1214/aos/1176346785 · Cited by 1,503 · see citing works
Approximate Bayesian Inference with the Weighted Likelihood Bootstrap
Newton, M. A., & Raftery, A. E. (1994) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.2517-6161.1994.tb01956.x · Cited by 1,491 · see citing works
Among-site rate variation and its impact on phylogenetic analyses
Yang, Z. (1996) · Trends in Ecology & Evolution · doi:10.1016/0169-5347(96)10041-0 · Cited by 1,202 · see citing works
Parallel Metropolis coupled Markov chain Monte Carlo for Bayesian phylogenetic inference
Altekar, G., Dwarkadas, S., Huelsenbeck, J. P., & Ronquist, F. (2004) · Bioinformatics · doi:10.1093/bioinformatics/btg427 · Cited by 1,138 · see citing works
Marginal Likelihood From the Metropolis–Hastings Output
Chib, S., & Jeliazkov, I. (2001) · Journal of the American Statistical Association · doi:10.1198/016214501750332848 · Cited by 1,093 · see citing works
Nedelman, J. R. (2005) · Computational Statistics · doi:10.1007/bf02741321 · Cited by 1,087 · see citing works
Yang, Z. (1993) · Molecular Biology and Evolution · doi:10.1093/oxfordjournals.molbev.a040082 · Cited by 1,040 · see citing works
The Intrinsic Bayes Factor for Model Selection and Prediction
Berger, J. O., & Pericchi, L. R. (1996) · Journal of the American Statistical Association · doi:10.1080/01621459.1996.10476668 · Cited by 1,008 · see citing works
Simulating normalizing constants: from importance sampling to bridge sampling to path sampling
Gelman, A., & Meng, X. (1998) · Statistical Science · doi:10.1214/ss/1028905934 · Cited by 986 · see citing works
Selecting the Best-Fit Model of Nucleotide Substitution
Posada, D., & Crandall, K. A. (2001) · Systematic Biology · doi:10.1080/10635150118469 · Cited by 937 · see citing works
Potential Applications and Pitfalls of Bayesian Inference of Phylogeny
Huelsenbeck, J. P., Larget, B. R., Miller, R. E., & Ronquist, F. (2002) · Systematic Biology · doi:10.1080/10635150290102366 · Cited by 811 · see citing works
Posterior Predictive $p$-Values
Meng, X. (1994) · The Annals of Statistics · doi:10.1214/aos/1176325622 · Cited by 792 · see citing works
Bayesian Selection of Continuous-Time Markov Chain Evolutionary Models
Suchard, M. A., Weiss, R. E., & Sinsheimer, J. S. (2001) · Molecular Biology and Evolution · doi:10.1093/oxfordjournals.molbev.a003872 · Cited by 791 · see citing works
Fractional Bayes Factors for Model Comparison
O’Hagan, A. (1995) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.2517-6161.1995.tb02017.x · Cited by 673 · see citing works
SIMULATING RATIOS OF NORMALIZING CONSTANTS VIA A SIMPLE IDENTITY: A THEORETICAL EXPLORATION
Meng, X., & Wong, W. H. (1996) · Cited by 640 · see citing works
Phylogeny estimation: traditional and Bayesian approaches
Holder, M. T., & Lewis, P. O. (2003) · Nature Reviews Genetics · doi:10.1038/nrg1044 · Cited by 631 · see citing works
MUST, a computer package of Management Utilities for Sequences and Trees
Philippe, H. (1993) · Nucleic Acids Research · doi:10.1093/nar/21.22.5264 · Cited by 600 · see citing works
Self-citationBayesian Phylogenetic Model Selection Using Reversible Jump Markov Chain Monte Carlo
Huelsenbeck, J. P. (2004) · Molecular Biology and Evolution · doi:10.1093/molbev/msh123 · Cited by 592 · see citing works
Pagel, M. D., & Meade, A. (2004) · Systematic Biology · doi:10.1080/10635150490468675 · Cited by 553 · see citing works
Philippe, H., Lartillot, N., & Brinkmann, H. (2005) · Molecular Biology and Evolution · doi:10.1093/molbev/msi111 · Cited by 551 · see citing works
Self-citationRaftery, A. E., & Lewis, S. M. (1992) · Statistical Science · doi:10.1214/ss/1177011143 · Cited by 541 · see citing works
Computing Bayes Factors Using a Generalization of the Savage-Dickey Density Ratio
Verdinelli, I., & Wasserman, L. (1995) · Journal of the American Statistical Association · doi:10.1080/01621459.1995.10476554 · Cited by 498 · see citing works
Some Tests of Significance, Treated by the Theory of Probability
Jeffreys, H. (1935) · Mathematical Proceedings of the Cambridge Philosophical Society · doi:10.1017/s030500410001330x · Cited by 442 · see citing works
Performance-Based Selection of Likelihood Models for Phylogeny Estimation
Minin, V. N., Abdo, Z., Joyce, P., & Sullivan, J. M. (2003) · Systematic Biology · doi:10.1080/10635150390235494 · Cited by 416 · see citing works
Mapping Mutations on Phylogenies
Nielsen, R. (2002) · Systematic Biology · doi:10.1080/10635150290102393 · Cited by 413 · see citing works
Aitkin, M. A. (1991) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.2517-6161.1991.tb01812.x · Cited by 362 · see citing works
Are Guinea Pigs Rodents? The Importance of Adequate Models in Molecular Phylogenetics
Sullivan, J. M., & Swofford, D. L. (1997) · Journal of Mammalian Evolution · doi:10.1023/a:1027314112438 · Cited by 283 · see citing works
An Empirical Assessment of Long-Branch Attraction Artefacts in Deep Eukaryotic Phylogenomics
Brinkmann, H., van der Giezen, M., Zhou, Y., de Raucourt, G. P., & Philippe, H. (2005) · Systematic Biology · doi:10.1080/10635150500234609 · Cited by 281 · see citing works
Self-citationSuccess of maximum likelihood phylogeny inference in the four-taxon case.
Gaut, B. S., & Lewis, P. O. (1995) · Molecular Biology and Evolution · doi:10.1093/oxfordjournals.molbev.a040183 · Cited by 245 · see citing works
Bayesian Model Adequacy and Choice in Phylogenetics
Bollback, J. P. (2002) · Molecular Biology and Evolution · doi:10.1093/oxfordjournals.molbev.a004175 · Cited by 215 · see citing works
Identifiability of Parameters in MCMC Bayesian Inference of Phylogeny
Rannala, B. (2002) · Systematic Biology · doi:10.1080/10635150290102429 · Cited by 190 · see citing works
A Monte Carlo method for high dimensional integration
Ogata, Y. (1989) · Numerische Mathematik · doi:10.1007/bf01406511 · Cited by 161 · see citing works
Hierarchical Phylogenetic Models for Analyzing Multipartite Sequence Data
Suchard, M. A., Ramirez, C. M., Sinsheimer, J. S., & Weiss, R. E. (2003) · Systematic Biology · doi:10.1080/10635150390238879 · Cited by 150 · see citing works
Long branch attraction, taxon sampling, and the earliest angiosperms: Amborella or monocots?
Stefanović, S., Rice, D. W., & Palmer, J. D. (2004) · BMC Evolutionary Biology · doi:10.1186/1471-2148-4-35 · Cited by 143 · see citing works
Irestedt, M., Fjeldså, J., Nylander, J. A. A., & Ericson, P. G. P. (2004) · BMC Evolutionary Biology · doi:10.1186/1471-2148-4-23 · Cited by 76 · see citing works
Waddell, P. J., Kishino, H., & Ota, R. (2002) · PubMed · Cited by 44 · see citing works
Shields, L. E. (2007) · Nursing Children and Young People · doi:10.7748/paed.19.5.37.s32 · Cited by 39 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- Han, C, and B. P. Carlin. 2000. MCMC methods for computing Bayes factors: A comparative review. Biometrika 82:711-732.
Cited by
Sorted by citations
Lartillot, N., Brinkmann, H., & Philippe, H. (2007) · BMC Evolutionary Biology · doi:10.1186/1471-2148-7-s1-s4 · Cited by 705 · see citing works
Self-citationA general comparison of relaxed molecular clock models
Lepage, T., Bryant, D., Philippe, H., & Lartillot, N. (2007) · Molecular Biology and Evolution · doi:10.1093/molbev/msm193 · Cited by 546 · see citing works
Self-citationEvent Horizon Telescope Collaboration, Akiyama, K., Alberdi, A., Alef, W., Algaba, J. C., Anantua, R., Asada, K., Azulay, R., Bach, U., Baczko, A.-K., et al. (2022) · The Astrophysical Journal Letters · doi:10.3847/2041-8213/ac6429 · Cited by 449 · see citing works
Slater, G. J., Harmon, L. J., & Alfaro, M. E. (2012) · Evolution · doi:10.1111/j.1558-5646.2012.01723.x · Cited by 334 · see citing works
Event Horizon Telescope Collaboration, Akiyama, K., Alberdi, A., Alef, W., Algaba, J. C., Anantua, R., Asada, K., Azulay, R., Bach, U., Baczko, A.-K., et al. (2022) · The Astrophysical Journal Letters · doi:10.3847/2041-8213/ac6736 · Cited by 369 · see citing works
“To compute the Bayesian evidence, THEMIS uses thermodynamic integration (e.g., Lartillot & Philippe 2006), which computes the log-evidence through ò b= á ñb ( ) ln d ln , 470 1 where á ñbln is the expectation of the log-likelihood taken over the distribution ⎜ ⎟ ⎛ ⎝ ⎞ ⎠p p=b b ( ) ( ) ( ) ( ) ( )(…”
MethodDeep metazoan phylogeny: When different genes tell different stories
Nosenko, T., Schreiber, F., Adamska, M., Adamski, M., Eitel, M., Hammel, J., Maldonado, M., Müller, W. E. G., Nickel, M., Schierwater, B., et al. (2013) · Molecular Phylogenetics and Evolution · doi:10.1016/j.ympev.2013.01.010 · Cited by 254 · see citing works
Bayesian ranking of biochemical system models
Vyshemirsky, V., & Girolami, M. A. (2008) · Bioinformatics · doi:10.1093/bioinformatics/btm607 · Cited by 181 · see citing works
Inferring speciation and extinction rates under different sampling schemes
Hohna, S., Stadler, T., Ronquist, F., & Britton, T. (2011) · Molecular Biology and Evolution · doi:10.1093/molbev/msr095 · Cited by 160 · see citing works
TAPAS: An Open-Source Software Package for Translational Neuromodeling and Computational Psychiatry
Frässle, S., Aponte, E. A., Bollmann, S., Brodersen, K. H., Do, C. T., Harrison, O. K., Harrison, S. J., Heinzle, J., Iglesias, S., Kasper, L., et al. (2021) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2021.680811 · Cited by 169 · see citing works
Clarke, J. A., & Middleton, K. M. (2008) · Systematic Biology · doi:10.1080/10635150802022231 · Cited by 137 · see citing works
Dating the arthropod tree based on large-scale transcriptome data
Rehm, P., Borner, J., Meusemann, K., von Reumont, B. M., Simon, S., Hadrys, H., Misof, B., & Burmester, T. (2011) · Molecular Phylogenetics and Evolution · doi:10.1016/j.ympev.2011.09.003 · Cited by 121 · see citing works
Spatial Guilds in the Serengeti Food Web Revealed by a Bayesian Group Model
Baskerville, E. B., Dobson, A. P., Bedford, T., Allesina, S., Anderson, T. M., & Pascual, M. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002321 · Cited by 107 · see citing works
“Markov-chain Monte Carlo Sampling We sample from the posterior distribution of model parameters using a Markov-chain Monte Carlo technique known as Metropolis-coupled MCMC, or (MC)(3) [59], which improves mixing between multiple modes of the posterior distribution, and also allows improved estimation of the marginal l…”
MethodAncestral state reconstruction of ontogeny supports a bilaterian affinity for <i>Dickinsonia</i>
Gold, D. A., Runnegar, B., Gehling, J. G., & Jacobs, D. K. (2015) · Evolution & Development · doi:10.1111/ede.12168 · Cited by 99 · see citing works
Properties of cell death models calibrated and compared using Bayesian approaches
Eydgahi, H., Chen, W. W., Muhlich, J. L., Vitkup, D., Tsitsiklis, J. N., & Sorger, P. K. (2013) · Molecular Systems Biology · doi:10.1038/msb.2012.69 · Cited by 84 · see citing works
THEMIS: A Parameter Estimation Framework for the Event Horizon Telescope
Broderick, A. E., Gold, R., Karami, M., Preciado-López, J. A., Tiede, P., Pu, H.-Y., Akiyama, K., Alberdi, A., Alef, W., Asada, K., et al. (2020) · The Astrophysical Journal · doi:10.3847/1538-4357/ab91a4 · Cited by 81 · see citing works
Ekman, S., & Blaalid, R. (2011) · Systematic Biology · doi:10.1093/sysbio/syr022 · Cited by 51 · see citing works
Gillespie, J. J., Phan, I. Q. H., Scheib, H., Subramanian, S., Edwards, T. E., Lehman, S. S., Piitulainen, H., Sayeedur Rahman, M., Rennoll-Bankert, K. E., Staker, B. L., et al. (2015) · mBio · doi:10.1128/mbio.01867-15 · Cited by 36 · see citing works
Johnston, A. R., & Anthony, N. M. (2012) · BMC Evolutionary Biology · doi:10.1186/1471-2148-12-120 · Cited by 36 · see citing works
- Authors
- Lartillot, N., & Philippe, H.
- Affiliations
- Centre National de la Recherche Scientifique (FR)
- Canadian Institute for Advanced Research (CA)
- Corresponding author
- Lartillot, Nicolas
- Venue
- Systematic Biology
- Publisher
- Oxford University Press (OUP)
- Published
- 2006-04-01
- Volume
- 55
- Issue
- 2
- Pages
- 195-207
- ISSN
- 1063-5157 (ISSN-L)
- 1076-836X
- OpenAlex
- W2098538101
- Cited by
- 668
- References
- 58
- Citations by year
- 2026: 21 · 2025: 26 · 2024: 26 · 2023: 44 · 2022: 33 · 2021: 49 · 2020: 48 · 2019: 61 · 2018: 40 · 2017: 36 · 2016: 25 · 2015: 36 · 2014: 53 · 2013: 34 · 2012: 31
- FWCI
- 18.34
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Systematic Biology (publishedVersion)
- PubMed (publishedVersion)
- HAL (submittedVersion, other-oa)
- Funders
- Centre National de la Recherche Scientifique
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Markov Chains and Monte Carlo Methods
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Phylogenetic Studies
- Keywords
- Bayes factor, thermodynamic integration, marginal likelihood, harmonic mean estimator, model comparison, phylogenetic model selection, site heterogeneity, statistical physics analogy, overfitting
- MeSH terms
- Amino Acid Sequence, Animals, Bayes Theorem, Models, Biological, Phylogeny, Proteins, Thermodynamics, Likelihood Functions, Evolution, Molecular
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
