Bayesian ranking of biochemical system models
Open accessAbstract
Motivation: There often are many alternative models of a biochemical system. Distinguishing models and finding the most suitable ones is an important challenge in Systems Biology, as such model ranking, by experimental evidence, will help to judge the support of the working hypotheses forming each model. Bayes factors are employed as a measure of evidential preference for one model over another. Marginal likelihood is a key component of Bayes factors, however computing the marginal likelihood is a difficult problem, as it involves integration of nonlinear functions in multidimensional space. There are a number of methods available to compute the marginal likelihood approximately. A detailed investigation of such methods is required to find ones that perform appropriately for biochemical modelling. Results: We assess four methods for estimation of the marginal likelihoods required for computing Bayes factors. The Prior Arithmetic Mean estimator, the Posterior Harmonic Mean estimator, the Annealed Importance Sampling and the Annealing-Melting Integration methods are investigated and compared on a typical case study in Systems Biology. This allows us to understand the stability of the analysis results and make reliable judgements in uncertain context. We investigate the variance of Bayes factor estimates, and highlight the stability of the Annealed Importance Sampling and the Annealing-Melting Integration methods for the purposes of comparing nonlinear models. Availability: Models used in this study are available in SBML format as the supplementary material to this article. Contact: vvv@dcs.gla.ac.uk Supplementary information: Supplementary data are available at Bioinformatics online.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Bayesian inference has gained attention as a flexible and formally coherent way to approach the problem of model calibration.
“As an alternative to heuristic optimization approaches, Bayesian inference has gained attention in recent years as a flexible and formally coherent way in which to approach the problem of model calibration1,6,7.”
From A framework for parameter estimation and model selection from experimental data in systems biology using approximate Bayesian computation (2014) · cites this paper as [7]
A Bayesian inference or MCMC approach may be expected to lead to the same result as evolutionary computing and thereby strengthens conclusions from such an approach.
“A conceptually quite different approach, which may of course be expected to lead to the same result (and which in consequence strengthens further any conclusions from e.g. an evolutionary computing approach), is an approach based on Bayesian inference”
From Towards a unifying, systems biology understanding of large-scale cellular death and destruction caused by poorly liganded iron: Parkinson’s, Huntington’s, Alzheimer’s, prions, bactericides, chemical toxicology and others as examples (2010) · cites this paper as Vyshemirsky and Girolami 2008
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Optimization by Simulated Annealing
Kirkpatrick, S., Gelatt, C. D., & Vecchi, M. P. (1983) · Science · doi:10.1126/science.220.4598.671 · Cited by 45,134 · see citing works
Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A., & Rubin, D. B. (2013) · doi:10.1201/b16018 · Cited by 27,276 · see citing works
Monte Carlo sampling methods using Markov chains and their applications.
Hastings, W. K. (1970) · Biometrika · doi:10.1093/biomet/57.1.97 · Cited by 15,195 · see citing works
Gaussian Processes for Machine Learning
Rasmussen, C. E., & Williams, C. K. I. (2005) · doi:10.7551/mitpress/3206.001.0001 · Cited by 11,705 · see citing works
Jeffreys, H., Moran, P. A. P., & Chatfield, C. (1922) · Nature · doi:10.1038/109132a0 · Cited by 7,837 · see citing works
Information theory, inference, and learning algorithms
(2004) · Choice Reviews Online · doi:10.5860/choice.41-5949 · Cited by 6,535 · see citing works
Information Theory, Inference, and Learning Algorithms
(2004) · Kybernetes · doi:10.1108/03684920410534506 · Cited by 4,937 · see citing works
A synthetic oscillatory network of transcriptional regulators
Elowitz, M., & Leibler, S. (2000) · Nature · doi:10.1038/35002125 · Cited by 4,812 · see citing works
Probability theory the logic of science
Jaynes, E. T., & Bretthorst, G. L. (2003) · Cited by 4,600 · see citing works
Probability Theory: The Logic of Science
Jaynes, E. T. (2003) · doi:10.1017/cbo9780511790423 · Cited by 4,045 · see citing works
Modeling and Simulation of Genetic Regulatory Systems: A Literature Review
de Jong, H. (2002) · Journal of Computational Biology · doi:10.1089/10665270252833208 · Cited by 2,660 · see citing works
COPASI—a COmplex PAthway SImulator
Hoops, S., Sahle, S., Gauges, R., Lee, C., Pahle, J., Simus, N., Singhal, M., Xu, L., Mendes, P., & Kummer, U. (2006) · Bioinformatics · doi:10.1093/bioinformatics/btl485 · Cited by 2,618 · see citing works
Roux, P. P., & Blenis, J. (2004) · Microbiology and Molecular Biology Reviews · doi:10.1128/mmbr.68.2.320-344.2004 · Cited by 2,494 · see citing works
Bernardo, J. M., & Smith, A. F. M. (1994) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316870 · Cited by 2,178 · see citing works
Approximate Bayesian Inference with the Weighted Likelihood Bootstrap
Newton, M. A., & Raftery, A. E. (1994) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.2517-6161.1994.tb01956.x · Cited by 1,491 · see citing works
Neal, R. M. (2001) · Statistics and Computing · doi:10.1023/a:1008923215028 · Cited by 1,208 · see citing works
Toward Evidence-Based Medical Statistics. 1: The P Value Fallacy
Goodman, S. N. (1999) · Annals of Internal Medicine · doi:10.7326/0003-4819-130-12-199906150-00008 · Cited by 1,188 · see citing works
Probabilistic Inference Using Markov Chain Monte Carlo Methods
Neal, R. M. (2011) · Cited by 1,035 · see citing works
Simulating normalizing constants: from importance sampling to bridge sampling to path sampling
Gelman, A., & Meng, X. (1998) · Statistical Science · doi:10.1214/ss/1028905934 · Cited by 986 · see citing works
Schoeberl, B., Eichler-Jonsson, C., Gilles, E. D., & Müller, G. I. (2002) · Nature Biotechnology · doi:10.1038/nbt0402-370 · Cited by 952 · see citing works
Computing Bayes factors using thermodynamic integration.
Lartillot, N., & Philippe, H. (2006) · Systematic Biology · doi:10.1080/10635150500433722 · Cited by 668 · see citing works
Husmeier, D. (2003) · Bioinformatics · doi:10.1093/bioinformatics/btg313 · Cited by 535 · see citing works
Bayesian Methods for Nonlinear Classification and Regression
Denison, D. G. T. (2002) · Medical Entomology and Zoology · Cited by 498 · see citing works
Choosing Models for Cross-Classifications
Raftery, A. E. (1986) · American Sociological Review · doi:10.2307/2095483 · Cited by 430 · see citing works
Kao, S.-C., Jaiswal, R. K., Kölch, W., & Landreth, G. E. (2001) · Journal of Biological Chemistry · doi:10.1074/jbc.m008870200 · Cited by 381 · see citing works
Marginal Likelihood Estimation via Power Posteriors
Friel, N., & Pettitt, A. N. (2008) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.1467-9868.2007.00650.x · Cited by 359 · see citing works
Werhli, A. V., Grzegorczyk, M. A., & Husmeier, D. (2006) · Bioinformatics · doi:10.1093/bioinformatics/btl391 · Cited by 338 · see citing works
Vehtari, A. (2002) · Journal of the Royal Statistical Society Series B (Methodological) · Cited by 333 · see citing works
Estimating the Integrated Likelihood via Posterior Simulation Using the Harmonic Mean Identity
Raftery, A. E., Newton, M. A., Satagopan, J. M., & Krivitsky, P. N. (2007) · doi:10.1093/oso/9780199214655.003.0015 · Cited by 287 · see citing works
The statistical mechanics of complex signaling networks: Nerve growth factor signaling.
Brown, K. S., Hill, C. C., Calero, G. A., Myers, C. R., Lee, K. H., Sethna, J. P., & Cerione, R. A. (2004) · Physical Biology · doi:10.1088/1478-3967/1/3/006 · Cited by 265 · see citing works
Estimating Bayes Factors via Posterior Simulation with the Laplace—Metropolis Estimator
Lewis, S. M., & Raftery, A. E. (1997) · Journal of the American Statistical Association · doi:10.1080/01621459.1997.10474016 · Cited by 231 · see citing works
Bayesian methods in bioinformatics and computational systems biology
Wilkinson, D. J. (2006) · Briefings in Bioinformatics · doi:10.1093/bib/bbm007 · Cited by 230 · see citing works
Non-parametric quantification of protein lysate arrays
Hu, J., He, X., Baggerly, K., Coombes, K. R., Hennessy, B. T., & Mills, G. B. (2007) · Bioinformatics · doi:10.1093/bioinformatics/btm283 · Cited by 224 · see citing works
Artificial gene networks for objective comparison of analysis algorithms
Mendes, P., Sha, W., & Ye, K. (2003) · Bioinformatics · doi:10.1093/bioinformatics/btg1069 · Cited by 208 · see citing works
A Monte Carlo method for high dimensional integration
Ogata, Y. (1989) · Numerische Mathematik · doi:10.1007/bf01406511 · Cited by 161 · see citing works
Bayesian Measures of Model Complexity and Fit
Spiegelhalter, D. J., Best, N. G., Carlin, B. P., & Van Der Linde, A. (2002) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/1467-9868.00353 · Cited by 151 · see citing works
Mathematical Modeling of the Influence of RKIP on the ERK Signaling Pathway
Kwang-Hyun, C., Shin, S., Kim, H., Olaf Wolkenhauer, A., McFerran, B. W., & Kölch, W. (2003) · Lecture notes in computer science · doi:10.1007/3-540-36481-1_11 · Cited by 101 · see citing works
Coherent Analysis of Forensic Identification Evidence
Dawid, A. P., & Mortera, J. (1996) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.2517-6161.1996.tb02091.x · Cited by 77 · see citing works
Bayesian model-based inference of transcription factor activity
Rogers, S., Khanin, R., & Girolami, M. (2007) · BMC Bioinformatics · doi:10.1186/1471-2105-8-s2-s2 · Cited by 57 · see citing works
Self-citationBayesian inference for dynamic transcriptional regulation; the Hes1 system as a case study
Heron, E. A., Finkenstadt, B. F., & Rand, D. A. J. (2007) · Bioinformatics · doi:10.1093/bioinformatics/btm367 · Cited by 54 · see citing works
Bayes Factors for Nonlinear Hypotheses and Likelihood Distributions
McCulloch, R. E., & Rossi, P. E. (1992) · Biometrika · doi:10.1093/biomet/79.4.663 · Cited by 40 · see citing works
Signal transduction network motifs and biological memory
Han, Z., Vondriska, T. M., Yang, L., MacLellan, W. R., Weiss, J. N., & Qu, Z. (2007) · Journal of Theoretical Biology · doi:10.1016/j.jtbi.2007.01.022 · Cited by 29 · see citing works
Bayesian Inference from Observations of Solar‐like Oscillations
Brewer, B. J., Bedding, T. R., Kjeldsen, H., et al. (2006) · The Astrophysical Journal · doi:10.1086/508766 · Cited by 23 · see citing works
Sample Selection in Radiocarbon Dating
Christen, J. A., & Buck, C. E. (1998) · Journal of the Royal Statistical Society Series C (Applied Statistics) · doi:10.1111/1467-9876.00127 · Cited by 21 · see citing works
An assessment of the role of computing in systems biology
Burbeck, S., & Jordan, K. E. (2006) · IBM Journal of Research and Development · doi:10.1147/rd.506.0529 · Cited by 8 · see citing works
A continuum mathematical model of endothelial layer maintenance and senescence
Wang, Y., Aguda, B. D., & Friedman, A. (2007) · Theoretical Biology and Medical Modelling · doi:10.1186/1742-4682-4-30 · Cited by 7 · see citing works
Bayes Factors for Nonlinear Hypotheses and Likelihood Distributions
McCulloch, R. E., & Rossi, P. E. (1992) · Biometrika · doi:10.2307/2337222 · Cited by 3 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- Jeffreys,H. (1961) Theory of Probability. 3rd edn. Clarendon Press, Oxford.
- Voit,E.O. (2000) Computational Analysis of Biochemical Systems. Cambridge University Press, Cambridge.
Cited by
Sorted by citations
Toni, T., Welch, D., Strelkowa, N., Ipsen, A., & Stumpf, M. P. H. (2009) · Journal of The Royal Society Interface · doi:10.1098/rsif.2008.0172 · Cited by 1,498 · see citing works
Riemann manifold langevin and hamiltonian monte carlo methods
Girolami, M., & Calderhead, B. (2011) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2010.00765.x · Cited by 1,473 · see citing works
Self-citationComparing families of dynamic causal models
Penny, W. D., Stephan, K. E., Daunizeau, J., Rosa, M. J., Friston, K. J., Schofield, T. M., & Leff, A. P. (2010) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000709 · Cited by 688 · see citing works
Chen, W. W., Schoeberl, B., Jasper, P. J., Niepel, M., Nielsen, U. B., Lauffenburger, D. A., & Sorger, P. K. (2009) · Molecular Systems Biology · doi:10.1038/msb.2008.74 · Cited by 391 · see citing works
Kell, D. B. (2010) · Archives of Toxicology · doi:10.1007/s00204-010-0577-x · Cited by 361 · see citing works
Simulation-based model selection for dynamical systems in systems and population biology
Toni, T., & Stumpf, M. P. H. (2010) · Bioinformatics · doi:10.1093/bioinformatics/btp619 · Cited by 260 · see citing works
Reverse engineering and identification in systems biology: strategies, perspectives and challenges
Villaverde, A. F., & Banga, J. R. (2014) · Journal of The Royal Society Interface · doi:10.1098/rsif.2013.0505 · Cited by 229 · see citing works
Liepe, J., Kirk, P., Filippi, S., Toni, T., Barnes, C. P., & Stumpf, M. P. H. (2014) · Nature Protocols · doi:10.1038/nprot.2014.025 · Cited by 217 · see citing works
Maximizing the information content of experiments in systems biology
Liepe, J., Filippi, S., Komorowski, M., & Stumpf, M. P. H. (2013) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002888 · Cited by 170 · see citing works
Model selection in systems and synthetic biology
Kirk, P., Thorne, T., & Stumpf, M. P. (2013) · Current Opinion in Biotechnology · doi:10.1016/j.copbio.2013.03.012 · Cited by 118 · see citing works
Parameter uncertainty in biochemical models described by ordinary differential equations
Vanlier, J., Tiemann, C. A., Hilbers, P. A. J., & van Riel, N. A. W. (2013) · Mathematical Biosciences · doi:10.1016/j.mbs.2013.03.006 · Cited by 116 · see citing works
Xu, T.-R., Vyshemirsky, V., Gormand, A., von Kriegsheim, A., Girolami, M., Baillie, G. S., Ketley, D., Dunlop, A. J., Milligan, G., Houslay, M. D., et al. (2010) · Science Signaling · doi:10.1126/scisignal.2000517 · Cited by 119 · see citing works
Self-citationSilk, D., Kirk, P. D. W., Barnes, C. P., Toni, T., Rose, A., Moon, S., Dallman, M. J., & Stumpf, M. P. H. (2011) · Nature Communications · doi:10.1038/ncomms1496 · Cited by 76 · see citing works
BioBayes: A software package for Bayesian inference in systems biology
Vyshemirsky, V., & Girolami, M. (2008) · Bioinformatics · doi:10.1093/bioinformatics/btn338 · Cited by 60 · see citing works
Self-citationLiepe, J., Taylor, H., Barnes, C. P., Huvet, M., Bugeon, L., Thorne, T., Lamb, J. R., Dallman, M. J., & Stumpf, M. P. H. (2012) · Integrative Biology · doi:10.1039/c2ib00175f · Cited by 33 · see citing works
Toni, T., Ozaki, Y.-I., Kirk, P., Kuroda, S., & Stumpf, M. P. H. (2012) · Molecular BioSystems · doi:10.1039/c2mb05493k · Cited by 28 · see citing works
Bayesian inference of the sites of perturbations in metabolic pathways via Markov chain Monte Carlo
Jayawardhana, B., Kell, D. B., & Rattray, M. (2008) · Bioinformatics · doi:10.1093/bioinformatics/btn103 · Cited by 26 · see citing works
- Authors
- Vyshemirsky, V., & Girolami, M. A.
- Affiliations
- University of Glasgow (GB)
- Corresponding author
- Vyshemirsky, Vladislav
- ORCID
- Girolami, Mark A.
- Venue
- Bioinformatics
- Publisher
- Oxford University Press (OUP)
- Published
- 2007-12-05
- Volume
- 24
- Issue
- 6
- Pages
- 833-839
- ISSN
- 1367-4803 (ISSN-L)
- 1367-4811
- OpenAlex
- W2169264416
- Cited by
- 181
- References
- 47
- Citations by year
- 2026: 4 · 2025: 2 · 2024: 3 · 2023: 1 · 2021: 7 · 2020: 11 · 2019: 9 · 2018: 10 · 2017: 9 · 2016: 16 · 2015: 14 · 2014: 26 · 2013: 15 · 2012: 13
- FWCI
- 4.10
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Bioinformatics (publishedVersion)
- PubMed (publishedVersion)
- ENLIGHTEN (Jurnal Bimbingan dan Konseling Islam) (submittedVersion)
- UCL Discovery (University College London) (submittedVersion)
- Cambridge University Engineering Department Publications Database (submittedVersion)
- Funders
- Microsoft Research
- Engineering and Physical Sciences Research Council (EP/E052029/1)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene Regulatory Network Analysis
- Physical Sciences › Mathematics › Statistics and Probability › Markov Chains and Monte Carlo Methods
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Keywords
- model ranking, Bayesian model selection, Bayes factor, marginal likelihood estimation, systems biology, annealed importance sampling
- MeSH terms
- Algorithms, Artificial Intelligence, Bayes Theorem, Biochemistry, Computer Simulation, Models, Biological, Pattern Recognition, Automated, Signal Transduction, Proteome
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
