Simulation-based model selection for dynamical systems in systems and population biology
Open accessThe DOI matches its Crossref record; 43 of the 43 references it cites were found in a registry or an index; 3 of the 9 sources its claims cite were checked for support.
Abstract
Motivation: Computer simulations have become an important tool across the biomedical sciences and beyond. For many important problems several different models or hypotheses exist and choosing which one best describes reality or observed data is not straightforward. We therefore require suitable statistical tools that allow us to choose rationally between different mechanistic models of, e.g. signal transduction or gene regulation networks. This is particularly challenging in systems biology where only a small number of molecular species can be assayed at any given time and all measurements are subject to measurement uncertainty. Results: Here, we develop such a model selection framework based on approximate Bayesian computation and employing sequential Monte Carlo sampling. We show that our approach can be applied across a wide range of biological scenarios, and we illustrate its use on real data describing influenza dynamics and the JAK-STAT signalling pathway. Bayesian model selection strikes a balance between the complexity of the simulation models and their ability to describe observed data. The present approach enables us to employ the whole formal apparatus to any system that can be (efficiently) simulated, even when exact likelihoods are computationally intractable. Contact: ttoni@imperial.ac.uk; m.stumpf@imperial.ac.uk Supplementary information: Supplementary data are available at Bioinformatics online.
Overview
CoChat Summary
TL;DR
The paper develops an approximate Bayesian computation (ABC) model selection framework using sequential Monte Carlo (SMC) for dynamical systems where exact likelihoods are intractable. It demonstrates the method on stochastic reaction kinetics, Gibbs random fields, influenza outbreak data, and JAK-STAT signalling models.
Key findings
- ABC SMC model selection identified the correct model with high confidence in a synthetic stochastic reaction kinetics example.
- For Gibbs random fields, ABC SMC estimated posterior model distributions correctly and yielded approximately 50-fold speed-up on average compared with ABC rejection.
- For influenza outbreaks of the same strain, the posterior probability of the two-parameter shared-parameter model was 0.98 (median over 10 runs), supporting shared epidemiological characteristics.
- For JAK-STAT signalling, Bayes factors indicated strong evidence in favour of model 3 over model 1, positive evidence for model 3 over model 2, and positive evidence for model 2 over model 1, favouring the cycling model without a time delay.
Methods
- The authors introduce ABC SMC model selection on the joint space of model indicators and parameters, propagating particles through intermediate distributions with decreasing tolerances.
- The algorithm requires user-defined priors, distance functions, tolerance schedules, and perturbation kernels; in the examples, uniform priors were used for parameters and models.
- Applications included stochastic chemical reaction models, Gibbs random fields with known posteriors, influenza A/B outbreak datasets, and non-nested ordinary and delay differential equation models of STAT5 signalling.
Limitations
- The paper states that routine application to complex models with hundreds or thousands of parameters will require further numerical developments due to the high computational cost of repeated simulations.
- The paper states that SMC-based ABC methods are highly parallelizable and suggests future work exploit this, along with regression adjustment, automatic tolerance schedules, and more sophisticated perturbation kernels.
- Evident from the text: performance depends on choices of distance function, tolerance schedule, and perturbation kernels, and the examples use uniform priors and relatively small particle numbers (e.g. N=500 or N=1000).
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures




Verification
Claims the paper makes
3 Valid · 0 Not valid · 3 Not checked · 28 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“If ϵ is sufficiently small then this distribution will be a good approximation for the ‘true’ posterior distribution, P(θ|D0).”
§ Simulation-based model selection for dynamical systems in systems and population biology > 1 INTRODUCTION
What the sources say
- Not checked yetcites Beaumont et al., 2002 matched to “Approximate Bayesian computation in population genetics” (2002) source
- Partly supportedcites Marjoram et al., 2003 matched to “Markov chain Monte Carlo without likelihoods” (2003) sourceExperimental — accuracy not yet measured
The Abstract paragraph following algorithms C1–C4 supports the ε-limit half of the claim: as ε → ∞ the accepted draws come from the prior, if ε = 0 they come from the posterior density f(θ|𝒟), and 'the choice of ε therefore reflects a tension between computability and accuracy'. However, the paper reserves the label 'approximate Bayesian computation' for the summary-statistic scheme D1–D4, and there it conditions agreement with the true posterior on S being sufficient, noting that 'in practice it will be hard, if not impossible, to identify a suitable set of sufficient statistics'. Its Results make the omission concrete: with S = V the posterior mean of T was ≈1.75 against the 'true' 0.68, and even at ε = 0 with S = (V, H) (Table 4) the mean was 0.59, with the authors sounding 'a note of caution' that summarizing the data can have unexpected effects on the posterior. So the unqualified assertion that a small ε alone makes the ABC output a good approximation to P(θ|D0) is only partly what this source states. (LLM, conf=0.70)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Both approaches have been applied under the ABC rejection scheme, which is computationally prohibitive for models with even an only moderate number of parameters (Grelaud et al., 2009; Wilkinson, 2007).”
§ Simulation-based model selection for dynamical systems in systems and population biology > 2 ABC FOR MODEL SELECTION
What the sources say
- Partly supportedcites Grelaud et al., 2009 matched to “ABC likelihood-free methods for model choice in Gibbs random fields” (2009) sourceExperimental — accuracy not yet measured
Our copy supports the computational burden of the ABC rejection scheme but not the stated cause: in the 'Exact rejection algorithm' section the authors say the exact (ε=0) version has an 'acceptance probability [that] may be too low to be practical', and in §3.1 (Toy example) 4×10^6 loops yielded as few as 12 acceptances in the worst case; §1.2 adds that the normalising-constant sum 'involves too many terms to be manageable'. However, the paper attributes the difficulty to the dimension of the data/summary statistics and the intractable normalising constant, never to 'even a moderate number of parameters'; it also hedges where the claim states prohibitive as fact, and its overall thrust (§2.1, §4 Discussion) is that ABC is 'naturally tuned' for Gibbs-random-field model choice and implementable on realistic datasets. (LLM, conf=0.70)
- Not checked yetcites Wilkinson, 2007
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“The algorithm presented in Toni et al. (2009) is a special case of the above algorithm for discrete uniform KM kernel and uniform prior distribution of the model P(m).”
§ Simulation-based model selection for dynamical systems in systems and population biology > 2 ABC FOR MODEL SELECTION > 2.1 ABC SMC model selection algorithm on the joint space
What the sources say
- Partly supportedcites Toni et al. (2009) matched to “Approximate Bayesian computation scheme for parameter inference and model selection in dynamical systems” (2009) sourceExperimental — accuracy not yet measured
Our copy confirms the structural half of the characterisation: the model-selection scheme treats m ∈ {1,…,M} as an extra discrete parameter and draws m* afresh from the prior π(m) at every population (2.2 Model selection, p. 5, step MS2.1), with uniform perturbation kernels and a discrete uniform model prior in the worked examples (3.1.1, p. 7; Fig. 6 caption 'Population 0 (discrete uniform prior distribution)', 3.3, p. 15). What the source does not support is the subsumption relation itself: Toni et al. nowhere state that their algorithm is a special case of any more general ABC SMC formulation — the only special case they assert is T = 1 reducing ABC SMC to the ABC rejection sampler (2.1, p. 4), and their Appendix B contrasts (rather than nests) their SIS-based scheme with Sisson et al.'s ABC PRC (pp. 21–22). The 'special case of the above algorithm' is therefore the citing authors' own derivation relative to their algorithm, and Toni et al. also present π(m) and K_t generally (uniform or Gaussian), not restricted to the discrete-uniform/uniform-prior setting the claim names. (LLM, conf=0.72)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This was confirmed by applying our model selection algorithm; the inferred posterior marginal model probability of a two-parameter model was negligible (data not shown).”
§ Simulation-based model selection for dynamical systems in systems and population biology > 3 RESULTS > 3.3 Infuenza infection outbreaks
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Figure 3b shows the marginal posterior distribution of the model; the posterior probability of model 1 is 0.98 (median over 10 runs), which gives unambiguous support to model 1, meaning that outbreaks of the same strain share the same dynamics.”
§ Simulation-based model selection for dynamical systems in systems and population biology > 3 RESULTS > 3.3 Infuenza infection outbreaks
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
17 more claims
Create a free account to see all 34 claims and to verify the 6 references still open.
- Probably not a substantial claim. 7
- It may state two separate claims as one. 7
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“In addition to parameter estimation, ABC approaches can also be used for model ranking and selection34.”
From A framework for parameter estimation and model selection from experimental data in systems biology using approximate Bayesian computation (2014) · cites this paper as [34]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Simulation-based model selection for dynamical systems in systems and population biology > 3 RESULTS > 3.3 Infuenza infection outbreaks
“find some evidence that model (1) explains the data better than model (1)”
§ Simulation-based model selection for dynamical systems in systems and population biology > 3 RESULTS > 3.3 Infuenza infection outbreaks
“Such a model can be written as (O'Neill et al., 2000) (2)”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Simulation-based model selection for dynamical systems in systems and population biology > 3 RESULTS > 3.3 Infuenza infection outbreaks
“We thus explore a further model with three parameters, qc1, qc2, qh (model 1), where the two outbreaks share the same within-household characteristics (qh), and compare it against and the four-parameter model (model 2).”
§ Simulation-based model selection for dynamical systems in systems and population biology > 3 RESULTS > 3.3 Infuenza infection outbreaks
“(d) Estimation of a posterior marginal distribution. Model 1 is a two-parameter and model 2 a three-parameter model (1).”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Simulation-based model selection for dynamical systems in systems and population biology > 3 RESULTS > 3.2 Gibbs random fields
“P(xi+1=xi|xi)=θ1/(1+exp(θ1)) for i=2,…, xn.”
§ Simulation-based model selection for dynamical systems in systems and population biology > 3 RESULTS > 3.2 Gibbs random fields
“Both models, m0 and m1, are defined on a sequence of n binary random variables, x=(x1,…, xn), xi∈{0, 1}”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Simulation-based model selection for dynamical systems in systems and population biology > 3 RESULTS > 3.2 Gibbs random fields
“m0 is a collection of n i.i.d. Bernoulli random variables with probability θ0/(1+exp(θ0));”
§ Simulation-based model selection for dynamical systems in systems and population biology > 3 RESULTS > 3.2 Gibbs random fields
“θ0 ∼ U(−5, 5)”
Citations
Cites
Model selection and multimodel inference: a practical information-theoretic approach.
Guthery, F. S., Burnham, K. P., & Anderson, D. R. (2003) · The Journal of Wildlife Management · doi:10.2307/3802723 · Cited by 41,495 · cited 2×
“The present approach enables us to employ the whole formal apparatus to any system that can be (efficiently) simulated, even when exact likelihoods are computationally intractable.”
BackgroundMethodGelman, A., Carlin, J. B., Stern, H. S., & Rubin, D. B. (2003) · doi:10.1201/9780429258480 · Cited by 27,276
Kass, R. E., & Raftery, A. E. (1995) · Journal of the American Statistical Association · doi:10.1080/01621459.1995.10476572 · Cited by 12,558
Exact stochastic simulation of coupled chemical reactions
Gillespie, D. T. (1977) · The Journal of Physical Chemistry · doi:10.1021/j100540a008 · Cited by 10,136 · cited 2×
“We simulate synthetic datasets of Y measured at 20 time points using Gillespie algorithm (Gillespie, 1977) from model 2 with parameter k2 =30 and initial conditions X0 =40, Y0”
MethodSequential Monte Carlo Methods in Practice
(2001) · Springer eBooks · doi:10.1007/978-1-4757-3437-9 · Cited by 7,366
Novel Proteinaceous Infectious Particles Cause Scrapie
Prusiner, S. B. (1982) · Science · doi:10.1126/science.6801762 · Cited by 5,118 · cited 3×
“We use data from influenza A (H3N2) outbreaks that occurred in 1977–1978 and 1980–1981 in Tecomseh, Michigan (Addy et al., 1991, Supplementary Table 2), and a second dataset of an influenza B infection outbreak in 1975–1976 and influenza A (H1N1) infection outbreak in 1978–1979 in Seattle, Washington (Longini and Koop…”
BackgroundDarnell, J. E. (1997) · Science · doi:10.1126/science.277.5332.1630 · Cited by 3,979
Approximate Bayesian computation in population genetics
Beaumont, M. A., Zhang, W., & Balding, D. J. (2002) · Genetics · doi:10.1093/genetics/162.4.2025 · Cited by 2,932
Computing Bayes Factors by Combining Simulation and Asymptotic Approximations
Diciccio, T. J., Kass, R. E., Raftery, A., & Wasserman, L. (1997) · Journal of the American Statistical Association · doi:10.1080/01621459.1997.10474045 · Cited by 1,943
Bayesian Data Analysis, 2nd edn
Brix, A. (2005) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2004.00347_4.x · Cited by 1,767
Research works printed but not linked: 2
- Jeffreys, 1939, Theory of Probability
- Wilkinson, 2007, Bayesian inference of primate divergence times
Cited by
Items 101–200 of 261
Erguler, K., Mendel, J., Petrić, D. V., Petrić, M., Kavran, M., Demirok, M. C., Günay, F., Georgiades, P., Alten, B., & Lelieveld, J. (2022) · Scientific Reports · doi:10.1038/s41598-022-15806-2 · Cited by 19
Fay, D., Moore, A. W., Brown, K. N., Filosi, M., & Jurman, G. (2014) · Journal of Complex Networks · doi:10.1093/comnet/cnu009 · Cited by 19
Individual Trait Variation and Diversity in Food Webs
Melián, C. J., Baldó, F., Matthews, B., Vílas, C., González‐Ortegón, E., Drake, P., & Williams, R. J. (2014) · Advances in ecological research/Advances in Ecological Research · doi:10.1016/b978-0-12-801374-8.00006-2 · Cited by 22
“The tolerance є 0 is the desired level of agreement between the simulations () and the empirical data () (Toni and Stumpf, 2010).”
BackgroundMethodInfluentialReverse engineering gene regulatory networks using approximate Bayesian computation
Rau, A., Jaffrézic, F., Foulley, J. L., & Doerge, R. W. (2011) · Statistics and Computing · doi:10.1007/s11222-011-9309-1 · Cited by 18
In silico model‐based inference: A contemporary approach for hypothesis testing in network biology
Klinke, D. J. (2014) · Biotechnology Progress · doi:10.1002/btpr.1982 · Cited by 16
Inferring Learning Strategies from Cultural Frequency Data
Kandler, A., & Powell, A. (2015) · Springer eBooks · doi:10.1007/978-4-431-55363-2_7 · Cited by 20
Universal dynamical properties preclude standard clustering in a large class of biochemical data
Gomez, F., Stoop, R., & Stoop, R. (2014) · Bioinformatics · doi:10.1093/bioinformatics/btu332 · Cited by 15
“In particular, approximate Bayesian computation (ABC) methods (Toni and Stumpf, 2010) that are normally used for Bayesian parameter inference where the stochasticity of the model makes the calculation of the likelihood density prohibitively expensive, might be expected to fail, as the shrimp…”
MethodInfluentialAn Information Theoretic Criterion for Empirical Validation of Time Series ModelsPreprint
Lamperti, F. (2015) · SSRN Electronic Journal · doi:10.2139/ssrn.2570828 · Cited by 16
Marin, J., Pudlo, P., Estoup, A., & Robert, C. P. (2018) · Chapman & Hall/CRC eBooks · doi:10.1201/9781315117195-6 · Cited by 26
Lee, X. J., Drovandi, C., & Pettitt, A. N. (2014) · Biometrics · doi:10.1111/biom.12249 · Cited by 15
“Toni et al. (2009) and Toni and Stumpf (2010) investigated the use of their sequential Monte Carlo ABC (SMC ABC) algorithm for both parameter estimation and model choice purposes.”
BackgroundMethodResultVakilzadeh, M. K., Beck, J. L., & Abrahamsson, T. J. S. (2018) · SIAM Journal on Scientific Computing · doi:10.1137/16m1090466 · Cited by 18
“Toni et al. [51] and Toni and Stumpf [50] developed an ABC algorithm based on Sequential Monte Carlo (ABC-SMC) and modified it such that a model index is incorporated within the model parameters.”
MethodPastor-Poquet, V., Papirio, S., Harmand, J., Steyer, J., Trably, É., Escudié, R., & Esposito, G. (2019) · Water Research · doi:10.1016/j.watres.2019.114932 · Cited by 17
Loiola, B. R., Orlande, H. R. B., & Dulikravich, G. S. (2020) · International Journal of Thermal Sciences · doi:10.1016/j.ijthermalsci.2019.106243 · Cited by 15
Model Extension and Model Selection
Sunnåker, M., & Stelling, J. (2015) · Studies in mechanobiology, tissue engineering and biomaterials · doi:10.1007/978-3-319-21296-8_9 · Cited by 16
“These methods simultaneously evaluate evidence both against and in favor of models [55].”
MethodEleutheroside B1 mediates its anti-influenza activity through POLR2A and N-glycosylation
Yan, W., Zheng, C., He, J., Zhang, W., Huang, X., Li, X., Wang, Y., & Wang, X. (2018) · International Journal of Molecular Medicine · doi:10.3892/ijmm.2018.3863 · Cited by 17
“EPOR is crucial for the phosphorylation of signal transducer and activator of transcription (STAT) 5, which acts as an important molecule in the Janus kinase (JAK)-STAT signaling pathway (22).”
Alamoudi, E., Schälte, Y., Müller, R., Starruß, J., Bundgaard, N., Graw, F., Brusch, L., & Hasenauer, J. (2023) · Bioinformatics · doi:10.1093/bioinformatics/btad674 · Cited by 13
Automated Bayesian model development for frequency detection in biological time series
Granqvist, E., Oldroyd, G. E. D., & Morris, R. J. (2011) · BMC Systems Biology · doi:10.1186/1752-0509-5-97 · Cited by 15
“Thus, Bayesian model comparison naturally follows the principle of Occam’s razor [33,34].”
BackgroundMulti-model and network inference based on ensemble estimates: avoiding the madness of crowds
Stumpf, M. P. H. (2020) · Journal of The Royal Society Interface · doi:10.1098/rsif.2020.0419 · Cited by 17
Self-citationApproximate Bayesian inference for complex ecosystems
Stumpf, M. P. H. (2016) · Cited by 13
Self-citationEstimation of Flow-Accelerated Corrosion Rate in Nuclear Piping System
Hazra, I., Pandey, M. D., & Jyrkama, M. I. (2019) · Journal of Nuclear Engineering and Radiation Science · doi:10.1115/1.4044407 · Cited by 14
Topological augmentation to infer hidden processes in biological systems
Sunnåker, M., Zamora-Sillero, E., de Lomana, A. L. G., Rudroff, F., Sauer, U., Stelling, J., & Wagner, A. (2013) · Bioinformatics · doi:10.1093/bioinformatics/btt638 · Cited by 12
“Results: Topological augmentation starts from a simple model that is unable to explain the experimental data and augments its topology by adding new terms that capture the experimental behavior.”
BackgroundTowards a simplification of models using regression trees
Eynaud, Y., Nérini, D., Baklouti, M., & Poggiale, J. (2012) · Journal of The Royal Society Interface · doi:10.1098/rsif.2012.0613 · Cited by 12
“For instance, if the method presented in [20,21] does not allow one to build simplified models, it is likely to be of great help when searching MS among several possibilities.”
Approximate Bayesian Computations to fit and compare insurance loss models
Goffard, P.-O., & Laub, P. J. (2021) · Insurance Mathematics and Economics · doi:10.1016/j.insmatheco.2021.06.002 · Cited by 16
“The sequential Monte Carlo idea used in Algorithm 5 have been adapted in the works of Toni and Stumpf [31] and Prangle et al. [23] to improve the sampling efficiency.”
MethodAverting A Crisis In Simulation-Based InferencePreprint
Hermans, J., Delaunoy, A., Rozet, F., Wehenkel, A., & Louppe, G. (2021) · arXiv (Cornell University) · Cited by 24
FAMoS: A Flexible and dynamic Algorithm for Model Selection to analyse complex systems dynamics
Gabel, M., Hohl, T., Imle, A., Fackler, O., & Graw, F. (2019) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1007230 · Cited by 11
“Several existing model selection algorithms already provide advanced fitting methods for the challenging analyses of dynamical systems while relying on model comparisons across the whole model space [2, 14, 17, 18, 46].”
West, T. O., Berthouze, L., Farmer, S. F., Cagnan, H., & Litvak, V. (2021) · NeuroImage · doi:10.1016/j.neuroimage.2021.118020 · Cited by 12
Cellular connectomes as arbiters of local circuit models in the cerebral cortex
Klinger, E., Motta, A., Marr, C., Theis, F. J., & Helmstaedter, M. N. (2021) · Nature Communications · doi:10.1038/s41467-021-22856-z · Cited by 14
Thorn, G. J., King, J. R., & Jabbari, S. (2012) · Mathematical Biosciences · doi:10.1016/j.mbs.2012.11.004 · Cited by 17
Jiao, Y., O'Reilly, R., Smith, E. P., Orth, D. J., & Jardim, H. E. E. (2016) · ICES Journal of Marine Science · doi:10.1093/icesjms/fsw036 · Cited by 13
Parameter estimation and model selection for water sorption in a wood fibre material
Berger, J., Colinart, T., Loiola, B. R., & Orlande, H. R. B. (2020) · Wood Science and Technology · doi:10.1007/s00226-020-01206-0 · Cited by 11
Reply to Robert et al.: Model criticism informs model choice and model comparison
Ratmann, O., Andrieu, C., Wiuf, C., & Richardson, S. T. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0912887107 · Cited by 11
“do not address the problem of choosing a model from a suite of candidates. Model comparison when the likelihood cannot be readily evaluated is not the topic of [4], and has been introduced elsewhere [18, 19, 20, 21, 22]. Example 6 Let us re-visit RMC’s Poisson example [1, 2] in order to (a) illustrate model criticism…”
MethodJiang, J., Zhou, Y., Jafarsalehi, G., Wang, X., Ahn, S., & Lee, J. D. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2025.3526318 · Cited by 19
“A ‘distance’ function is then used to evaluate closeness between the observed and simulated data and accept those that are close to approximate the posterior joint distributions [34].”
Bayesian and Algebraic Strategies to Design in Synthetic Biology
Araujo, R. P., Vittadello, S. T., & Stumpf, M. P. H. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2021.3129527 · Cited by 10
“Basing design on Bayesian model selection [4], [70], [71] has two intrinsic advantages: (i) we are able to balance the performance and the robustness of different designs; and (ii) for ABC methods we can, at least in principle, apply Bayesian design to arbitrary models as long as we are able to…”
Self-citationHubble constant by natural selection: Evolution chips in the Hubble tension
Bernardo, R. C., & Lee, Y. (2023) · Astronomy and Computing · doi:10.1016/j.ascom.2023.100740 · Cited by 12
Woods, M. L., & Barnes, C. P. (2016) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1005131 · Cited by 11
“To check that our model parameters were robust to adding additional data, we performed ABC SMC again on nine data sets.”
MethodInfluentialKlinger, E., & Hasenauer, J. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-67471-1_8 · Cited by 10
Inference of Model Parameters Using Particle Filter Algorithm and Copula Distributions
Deng, Z., Zhang, X., & Tian, T. (2018) · IEEE Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbb.2018.2880974 · Cited by 9
“Thus one of the challenges in mathematical modelling is to infer the parameter values in the model based on the noisy data [4], [5], [6], [7], [8].”
BackgroundBeguerisse-Díaz, M., Wang, B., Desikan, R., & Barahona, M. (2012) · Journal of The Royal Society Interface · doi:10.1098/rsif.2011.0767 · Cited by 8
“Ideally, this process could help close the modelling-experiment loop by: suggesting specific experimental measurements; identifying relevant parameters to be measured; or discriminating between alternative models (Gutenkunst et al., 2007; Toni and Stumpf, 2009; Yates et al., 2001).”
BackgroundMethodBayesian model selection for the yeast GATA-factor network: A comparison of computational approaches
Milias‐Argeitis, A., Porreca, R., Summers, S., & Lygeros, J. (2010) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc.2010.5717307 · Cited by 8
“Several methods have been proposed to tackle this problem and in this paper we consider three of them: Annealed Importance Sampling [2], a method based on Markov Chain Monte Carlo (MCMC) which we will refer to as the Chib method [3], and Approximate Bayesian Computation [4].”
MethodInfluentialMason, P. (2015) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2015.10.012 · Cited by 9
Mike, W. (2013) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780199695607.003.0008 · Cited by 9
Janzen, T., & Etienne, R. S. (2016) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/085431 · Cited by 10
Model-based selection of the robust JAK-STAT activation mechanism
Rybiński, M., & Gambin, A. (2012) · Journal of Theoretical Biology · doi:10.1016/j.jtbi.2012.04.031 · Cited by 10
“…all sizes and is sensitive not only to the size complexity but also to the functional complexity, i.e. it is sensitive to how complicated is the model output expression when related to model parameters (for biological case-studies see, e.g., Vyshemirsky and Girolami, 2008b; Toni and Stumpf, 2010).”
BackgroundEllis, J., Brown, E., Colenutt, C., Schley, D., & Gubbins, S. (2024) · Epidemics · doi:10.1016/j.epidem.2024.100740 · Cited by 11
Schälte, Y., & Hasenauer, J. (2023) · PLoS ONE · doi:10.1371/journal.pone.0285836 · Cited by 8
Bayesian Parameter Inference and Model Selection by Population Annealing in Systems Biology
Murakami, Y. (2014) · PLoS ONE · doi:10.1371/journal.pone.0104057 · Cited by 9
“the problem, ABC-SMC algorithm was developed [15,18,23].”
InfluentialInferring Mechanistic Parameters from Amyloid Formation Kinetics by Approximate Bayesian Computation
Nakatani-Webster, E., & Nath, A. (2017) · Biophysical Journal · doi:10.1016/j.bpj.2017.01.011 · Cited by 10
Gower, G., Picazo, P. I., Lindgren, F., & Racimo, F. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.04.27.538386 · Cited by 7
“Another approach would be to include a categorical variable as an inferrable parameter that chooses between a set of competing models (Toni et al. , 2009, Toni & Stumpf, 2010).”
BackgroundLoisel, L., Raquin, V., Ratinier, M., Ezanno, P., & Beaunée, G. (2025) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1013393 · Cited by 11
“We added a step to the inference process to select the best model [76] among four options: IbetaDbeta, IbetaDexpo, IexpoDbeta, and IexpoDexpo.”
Numerical simulation of time partial fractional diffusion model by Laplace transform
Ali, A., Suwan, I., Abdeljawad, T., & Abdullah (2021) · AIMS Mathematics · doi:10.3934/math.2022159 · Cited by 7
Ferreira, J. R., Sena, A. P., Coutinho, J. P. D. S., Estumano, D. C., & Macêdo, E. N. (2023) · Numerical Heat Transfer Part A Applications · doi:10.1080/10407782.2023.2226820 · Cited by 8
“New populations of size N are then generated by sampling the model from its prior and sampling the model parameters from their values at the previous populations [9, 13, 15, 16, 31 – 33].”
Thijssen, B., Dijkstra, T. M. H., Heskes, T., & Wessels, L. F. A. (2017) · Bioinformatics · doi:10.1093/bioinformatics/btx666 · Cited by 8
ABC(SMC)$$^2$$: Simultaneous Inference and Model Checking of Chemical Reaction Networks
Molyneux, G. W., & Abate, A. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-60327-4_14 · Cited by 7
“ABCSMC algorithms [46, 47] (cf. Appendix A) are designed to overcome this burden by constructing a sequence of slowly-changing intermediate distributions, f m ( θ ), m = 0 , . . . , M , where f 0 ( θ ) = π ( θ ) is the initial sampling distribution and f M ( θ ) = f ( θ ) is the target distribution…”
Hendricks, J., Neumann, C., & Saunders, C. P. (2021) · Electronic Journal of Statistics · doi:10.1214/21-ejs1813 · Cited by 13
“The original ABC approach for assigning a posterior probability to a model, given the observed data, is a function of the ratio of the number of times that model index has been retained over the total number of times any model was retained (Grelaud et al., 2009; Toni and Stumpf, 2010; Didelot et al., 2011; Robert et a…”
BackgroundMethodInfluentialSmith, R. L., & Grohn, Y. T. (2015) · PLoS ONE · doi:10.1371/journal.pone.0129535 · Cited by 11
How informative is your kinetic model?: using resampling methods for model invalidation
Hasdemir, D., Hoefsloot, H. C. J., Westerhuis, J. A., & Smilde, A. K. (2014) · BMC Systems Biology · doi:10.1186/1752-0509-8-61 · Cited by 8
“This family of Bayesian methods unfortunately still remain unemployed in the field due to the need for smart assumptions on parameters’ prior distributions and their costliness in calculation of bulky integrals despite promising effort regarding the second obstacle [15,16].”
Estimation of autocorrelation timescales with Approximate Bayesian ComputationsPreprint
Zeraati, R., Engel, T. A., & Levina, A. (2020) · Cold Spring Harbor Laboratory Institutional Repository (Cold Spring Harbor Laboratory) · doi:10.1101/2020.08.11.245944 · Cited by 9
Flevaris, K., Kotidis, P., & Kontoravdi, C. (2024) · Analytical and Bioanalytical Chemistry · doi:10.1007/s00216-024-05522-3 · Cited by 10
Identifying latent dynamic components in biological systems
Kondofersky, I., Fuchs, C., & Theis, F. J. (2015) · IET Systems Biology · doi:10.1049/iet-syb.2014.0013 · Cited by 7
TopoFilter: a MATLAB package for mechanistic model identification in systems biology
Rybiński, M., Möller, S., Sunnåker, M., Lormeau, C., & Stelling, J. (2020) · BMC Bioinformatics · doi:10.1186/s12859-020-3343-y · Cited by 7
A Probabilistic Approach to Explore Signal Execution Mechanisms With Limited Experimental Data
Kochen, M. A., & Lopez, C. F. (2020) · Frontiers in Genetics · doi:10.3389/fgene.2020.00686 · Cited by 5
“Approximate Bayesian Computation, for example, can estimate parameter distributions when a given model is too complex to be analyzed analytically, as is typical for complex biological systems (Toni et al., 2009; Toni and Stumpf, 2010).”
BackgroundUsing constraints and their value for optimization of large ODE systems
Domijan, M., & Rand, D. A. J. (2015) · Journal of The Royal Society Interface · doi:10.1098/rsif.2014.1303 · Cited by 7
Tutorial on ABC rejection and ABC SMC for parameter estimation and model selectionPreprint
Toni, T., & Stumpf, M. P. H. (2009) · arXiv (Cornell University) · doi:10.48550/arxiv.0910.4472 · Cited by 15
“C rejection for parameter estimation [1, 2], 2.ABC SMC for parameter estimation [3, 4, 5], 3.ABC rejection for model selection on the joint space [6], 4.ABC SMC for model selection on the joint space [7]. We suggest to read this tutorial from the beginning. We start with a detailed explanation of the ABC rejection alg…”
MethodSelf-citationNetworks in nature: dynamics, evolution, and modularity
Agarwal, S. (2012) · Oxford University Research Archive (ORA) (University of Oxford) · doi:10.5287/ora-xr1x2v5o5 · Cited by 6
Muthiah, A., Angulo, M. S., Walker, N. N., Keller, S. R., & Lee, J. K. (2018) · PLoS ONE · doi:10.1371/journal.pone.0207325 · Cited by 5
Feng, J., Zhang, X., & Tian, T. (2024) · International Journal of Molecular Sciences · doi:10.3390/ijms251810204 · Cited by 5
“Parameter inference is essential not only for model selection us-ing statistical criteria but also for reverse-engineering regulatory networks with omics datasets [239–241].”
BackgroundAlwan, A., & Aluru, N. R. (2015) · International Journal for Uncertainty Quantification · doi:10.1615/int.j.uncertaintyquantification.2015011166 · Cited by 5
Murakami, Y., & Takada, S. (2013) · PLoS ONE · doi:10.1371/journal.pone.0074178 · Cited by 5
Muthiah, A., Angulo, M. S., Walker, N. N., Keller, S. R., & Lee, J. K. (2018) · PLoS ONE · doi:10.1371/journal.pone.0204100 · Cited by 6
Computational Approaches to Study Gene Regulatory Networks
Omranian, N., & Nikoloski, Z. (2017) · Methods in molecular biology · doi:10.1007/978-1-4939-7125-1_18 · Cited by 6
Supermodeling: The Next Level of Abstraction in the Use of Data Assimilation
Sendera, M., Duane, G. S., & Dzwinel, W. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-50433-5_11 · Cited by 5
West, B., Wood, A. J., & Ungár, D. (2021) · Methods in molecular biology · doi:10.1007/978-1-0716-1685-7_10 · Cited by 4
Models and analyses to understand threats to polio eradication
Koopman, J. S. (2017) · BMC Medicine · doi:10.1186/s12916-017-0991-5 · Cited by 4
“An early development in these models involves approximate Bayesian computation using sequential Monte Carlo fitting [12, 13].”
MethodGrowth Dynamics of Ductal Carcinoma in Situ Recapitulate Normal Breast DevelopmentPreprint
Ryser, M. D., Greenwald, M. A., Sorribes, I. C., King, L. M., Hall, A. H. S., Geradts, J., Weaver, D. L., Mallo, D., Holloway, S. T., Monyak, D. L., et al. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.10.01.560370 · Cited by 4
Automated analysis of biological oscillator models using mode decomposition
Konopka, T. (2011) · Bioinformatics · doi:10.1093/bioinformatics/btr069 · Cited by 3
Thorn, G. J., & King, J. R. (2015) · Mathematical Biosciences · doi:10.1016/j.mbs.2015.10.016 · Cited by 3
Shin, S., & Nguyen, L. K. (2023) · Methods in molecular biology · doi:10.1007/978-1-0716-3008-2_17 · Cited by 3
Xiang, Z., Liu, Z., & Dinh, K. N. (2024) · Scientific Reports · doi:10.1038/s41598-024-67842-9 · Cited by 4
“Algorithms such as sequential Monte Carlo (SMC) 49 or Markov Chain Monte Carlo (MCMC) 33 , which use a metric to compare statistics between the model and data, would likely suffer more from the increased noise level of these measures.”
Learning governing equations of unobserved states in dynamical systems
Grigorian, G. A., George, S. V., & Arridge, S. (2024) · Physica D Nonlinear Phenomena · doi:10.1016/j.physd.2024.134499 · Cited by 11
da Costa, J. M. J., Orlande, H. R. B., Lione, V. D. O. F., Lima, A. G. F., Cardoso, T. D. C. S., & Varón, L. A. B. (2018) · Journal of Computational Biology · doi:10.1089/cmb.2017.0130 · Cited by 3
“…) (see steps 1 and 5 in Table 2) was started with a large value to avoid too many rejections of particles in the first populations (Toni et al., 2009; Toni and Stumpf, 2010b; Liepe et al., 2014; da Costa et al., 2017), specially because priors with very large variances were used for the parameters.”
Model selection in stochastic chemical reaction networks using flow cytometry data
Lillacci, G., & Khammash, M. (2011) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc.2011.6161417 · Cited by 3
Robert, C. P. (2011) · Winter Simulation Conference · doi:10.1109/wsc.2011.6147744 · Cited by 5
MacKay, N., Price, C. R., & Wood, A. J. (2016) · Historical Methods A Journal of Quantitative and Interdisciplinary History · doi:10.1080/01615440.2015.1072071 · Cited by 5
“Because the space of possible results is so wide (in common with other without-likelihood studies implemented on IBMs (Csillery et al. 2010; Kennedy, and O ’ Hagan 2001; Toni and Stumpf 2010), we need a strategy to re fi ne our parameters.”
BackgroundThe impact of experimental design choices on parameter inference for models of growing cell colonies
Parker, A. J., Simpson, M. J., & Baker, R. E. (2018) · Royal Society Open Science · doi:10.1098/rsos.180384 · Cited by 4
Zhang, H., Chen, J., & Tian, T. (2020) · IEEE Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbb.2020.3039490 · Cited by 3
“A number of methods have been developed including ABC-MCMC [17], [18], and sequential Monte-Carlo sampling algo-rithm [19], [20], [21].”
MethodSemi-automatic selection of summary statistics for ABC model choicePreprint
Prangle, D., Fearnhead, P., Cox, M. P., Biggs, P. J., & French, N. P. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1302.5624 · Cited by 3
“More efficient ABC model choice algorithms have been proposed, mainly based on sequential Monte Carlo (SMC) (e.g. Toni and Stumpf, 2010; Del Moral et al., 2012).”
MethodAbdessalem, A. B. (2024) · Quality and Reliability Engineering International · doi:10.1002/qre.3497 · Cited by 2
“The concepts outlined in the above algorithm can be extended to incorporate model selection in addition to parameter estimation.”
BackgroundUnderstanding climate-sensitive tick development and diapause with a structured population model
Erguler, K., Saratsis, A., Dobler, G., & Chitimia‐Dobler, L. (2025) · Frontiers in Veterinary Science · doi:10.3389/fvets.2025.1553557 · Cited by 2
Quantitative calibration of a spatial QSP model identifies fibroblast impact on HCC immunotherapy
Zhang, S., Wang, H., Cho, Y., da Rocha, H. L., Wong, W., Yarchoan, M. M., Jaffee, E. M., Ho, W. J., Kagohara, L. T., Fertig, E. J., et al. (2026) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2525799123 · Cited by 2
Evolutionary and physics-guided modulation of Ca$$^{2+}$$ sensitivity in Cardiac troponin C
Basit, A. (2026) · Journal of Molecular Modeling · doi:10.1007/s00894-026-06846-w · Cited by 2
Hasegawa, T., Niida, A., Mori, T., Shimamura, T., Yamaguchi, R., Miyano, S., Akutsu, T., & Imoto, S. (2015) · Computational Statistics & Data Analysis · doi:10.1016/j.csda.2015.08.003 · Cited by 2
Wang, S., Shen, Y., Shi, C., Wang, T., Wei, Z., & Li, H. (2014) · The Scientific World JOURNAL · doi:10.1155/2014/625754 · Cited by 2
Time-Series Filtering for Replicated Observations via a Kernel Approximate Bayesian Computation
Hasegawa, T., Kojima, K., Kawai, Y., & Nagasaki, M. (2018) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2018.2872864 · Cited by 2
“Thus, a previous study [13] proposed to use adaptive SMC-ABC [12], [31], [32] to select the appropriate at each time point in order to reduce the estimation bias while retaining the number of unique particles.”
MethodAlvarez-Buylla, E. R., Martínez‐García, J. C., Dávila-Velderrain, J., Domínguez‐Hüttinger, E., & Martínez-Sánchez, M. E. (2018) · Advances in experimental medicine and biology · doi:10.1007/978-3-319-89354-9_2 · Cited by 2
Optimality-Preserving Reduction of Chemical Reaction Networks
Larsen, K. G., Toller, D., Tribastone, M., Tschaikowski, M., & Vandin, A. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-75107-3_2 · Cited by 2
Simulation-based Bayesian Analysis of Complex Data.
Marjoram, P., Hamblin, S., & Foley, B. R. (2015) · PubMed · Cited by 3
Repich, M. (2010) · Unitn-eprints PhD (University of Trento) · doi:10.15168/11572_369232 · Cited by 2
Data adaptation in HANDY economy-ideology modelPreprint
Sendera, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.04309 · Cited by 6
“In this case, under consideration are marginal probablilities of the models [66, 64]:”
BackgroundMethodInfluentialRohmer, J., Filippini, A. G., & Pedreros, R. (2023) · Ocean Modelling · doi:10.1016/j.ocemod.2023.102275 · Cited by 2
An Approximate Bayesian Computation Approach for Embryonic Astrocyte Migration Model Reduction
Stepien, T. L. (2024) · Bulletin of Mathematical Biology · doi:10.1007/s11538-024-01354-5 · Cited by 2
“…{ m 1 , m 2 , . . . , m k } ,(2)sample parameters for that model from the prior distributions, (3) run a simulation for the parameter set and calculate the error, (4) accept the model and parameter set if the error is less than τ (Grelaud et al. 2009; Toni and Stumpf 2009).”
Metadata
- Authors
- Toni, T., & Stumpf, M. P. H.
- Affiliations
- Imperial College London (GB)
- Corresponding author
- Toni, Tina
- Stumpf, Michael P. H.
- Venue
- Bioinformatics
- Publisher
- Oxford University Press (OUP)
- Published
- 2009-10-29
- Volume
- 26
- Issue
- 1
- Pages
- 104-110
- ISSN
- 1367-4803 (ISSN-L)
- 1367-4811
- PMC
- PMC2796821
- OpenAlex
- W2126351705
- Cited by
- 260
- References
- 43
- Citations by year
- 2026: 7 · 2025: 12 · 2024: 11 · 2023: 13 · 2022: 6 · 2021: 19 · 2020: 16 · 2019: 15 · 2018: 19 · 2017: 16 · 2016: 15 · 2015: 20 · 2014: 22 · 2013: 19 · 2012: 18
- FWCI
- 6.70
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Bioinformatics (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Minerva Access (University of Melbourne) (submittedVersion, cc-by-nc)
- arXiv (Cornell University)
- Funders
- Directorate for Biological Sciences
- Medical Research Council
- Biotechnology and Biological Sciences Research Council (BB/F005210/2, BB/F005210/1, BB/C519670/1, BB/G009374/1)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene Regulatory Network Analysis
- Physical Sciences › Mathematics › Statistics and Probability › Markov Chains and Monte Carlo Methods
- Social Sciences › Arts and Humanities › History and Philosophy of Science › Philosophy of Biology and Scientific Explanation
- Keywords
- model selection, dynamical systems, approximate Bayesian computation, sequential Monte Carlo, signal transduction, gene regulatory networks, JAK/STAT signaling, population biology, measurement uncertainty, mechanistic model
- MeSH terms
- Algorithms, Animals, Bayes Theorem, Computer Simulation, Data Interpretation, Statistical, Humans, Models, Biological, Population Dynamics, Models, Statistical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-04, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-04 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.