CoChat

Simulation-based model selection for dynamical systems in systems and population biology

Open access
Toni, Tina, Stumpf, Michael P. H. · 2010 · Bioinformatics

The DOI matches its Crossref record; 43 of the 43 references it cites were found in a registry or an index; 3 of the 9 sources its claims cite were checked for support.

Abstract

Motivation: Computer simulations have become an important tool across the biomedical sciences and beyond. For many important problems several different models or hypotheses exist and choosing which one best describes reality or observed data is not straightforward. We therefore require suitable statistical tools that allow us to choose rationally between different mechanistic models of, e.g. signal transduction or gene regulation networks. This is particularly challenging in systems biology where only a small number of molecular species can be assayed at any given time and all measurements are subject to measurement uncertainty. Results: Here, we develop such a model selection framework based on approximate Bayesian computation and employing sequential Monte Carlo sampling. We show that our approach can be applied across a wide range of biological scenarios, and we illustrate its use on real data describing influenza dynamics and the JAK-STAT signalling pathway. Bayesian model selection strikes a balance between the complexity of the simulation models and their ability to describe observed data. The present approach enables us to employ the whole formal apparatus to any system that can be (efficiently) simulated, even when exact likelihoods are computationally intractable. Contact: ttoni@imperial.ac.uk; m.stumpf@imperial.ac.uk Supplementary information: Supplementary data are available at Bioinformatics online.

Overview

CoChat Summary

TL;DR

The paper develops an approximate Bayesian computation (ABC) model selection framework using sequential Monte Carlo (SMC) for dynamical systems where exact likelihoods are intractable. It demonstrates the method on stochastic reaction kinetics, Gibbs random fields, influenza outbreak data, and JAK-STAT signalling models.

Key findings

  • ABC SMC model selection identified the correct model with high confidence in a synthetic stochastic reaction kinetics example.
  • For Gibbs random fields, ABC SMC estimated posterior model distributions correctly and yielded approximately 50-fold speed-up on average compared with ABC rejection.
  • For influenza outbreaks of the same strain, the posterior probability of the two-parameter shared-parameter model was 0.98 (median over 10 runs), supporting shared epidemiological characteristics.
  • For JAK-STAT signalling, Bayes factors indicated strong evidence in favour of model 3 over model 1, positive evidence for model 3 over model 2, and positive evidence for model 2 over model 1, favouring the cycling model without a time delay.

Methods

  • The authors introduce ABC SMC model selection on the joint space of model indicators and parameters, propagating particles through intermediate distributions with decreasing tolerances.
  • The algorithm requires user-defined priors, distance functions, tolerance schedules, and perturbation kernels; in the examples, uniform priors were used for parameters and models.
  • Applications included stochastic chemical reaction models, Gibbs random fields with known posteriors, influenza A/B outbreak datasets, and non-nested ordinary and delay differential equation models of STAT5 signalling.

Limitations

  • The paper states that routine application to complex models with hundreds or thousands of parameters will require further numerical developments due to the high computational cost of repeated simulations.
  • The paper states that SMC-based ABC methods are highly parallelizable and suggests future work exploit this, along with regression adjustment, automatic tolerance schedules, and more sophisticated perturbation kernels.
  • Evident from the text: performance depends on choices of distance function, tolerance schedule, and perturbation kernels, and the examples use uniform priors and relatively small particle numbers (e.g. N=500 or N=1000).

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1093/bioinformatics/btp619

Cites

43 of 43 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Fig. 1. (a) Stochastic trajectories of species X (red) and Y (blue). Model 1 is simulated for k1=2.1 (dashed line), model 2 for k2=30 (full line). Data points are represented by circles. (b) We have repeated the model selection run 20 times; the red sections present 25% and 75% quantiles around the median. Prior distribution P(m) is chosen uniform and k1, k2 ∼ U(0, 100). Perturbation kernels are chosen as follows: KPt(k|k*)=U(−σ, σ), σ=2 (max{k}t−1−min{k}t−1) and KMt(m|m*)=0.7 if m=m* and 0.3 otherwise. Number of particles N=1000. Bt=1. Distance function is mean squared error and tolerance schedule ϵ={3000, 1400, 600, 140, 40}.
Fig. 1. (a) Stochastic trajectories of species X (red) and Y (blue). Model 1 is simulated for k1=2.1 (dashed line), model 2 for k2=30 (full line). Data points are represented by circles. (b) We have repeated the model selection run 20 times; the red sections present 25% and 75% quantiles around the median. Prior distribution P(m) is chosen uniform and k1, k2 ∼ U(0, 100). Perturbation kernels are chosen as follows: KPt(k|k*)=U(−σ, σ), σ=2 (max{k}t−1−min{k}t−1) and KMt(m|m*)=0.7 if m=m* and 0.3 otherwise. Number of particles N=1000. Bt=1. Distance function is mean squared error and tolerance schedule ϵ={3000, 1400, 600, 140, 40}.CC BY-NC · PubMed Central
Fig. 2. (a) True versus inferred posterior distribution of model m0. In ABC SMC, we use the Euclidian distance . N=500. Bt=1. Tolerance schedule: ϵ={9, 4, 3, 2, 1, 0}. Perturbation kernels: KMt(m|m*)=0.75 if m=m* and 0.25 otherwise; KPt(θ|θ*)=U(−σ, σ), σ=0.5 (max{θ}t−1−min{θ}t−1). We have excluded those datasets for which all states are in 0 or 1 (for which P(m=0)≈0.3094 is also correctly inferred) from the analysis. (b) Comparison of the number of simulation steps needed by ABC rejection (nRej) and ABC SMC (nSMC); ABC SMC yields ∼50-fold speed-up on average.
Fig. 2. (a) True versus inferred posterior distribution of model m0. In ABC SMC, we use the Euclidian distance . N=500. Bt=1. Tolerance schedule: ϵ={9, 4, 3, 2, 1, 0}. Perturbation kernels: KMt(m|m*)=0.75 if m=m* and 0.25 otherwise; KPt(θ|θ*)=U(−σ, σ), σ=0.5 (max{θ}t−1−min{θ}t−1). We have excluded those datasets for which all states are in 0 or 1 (for which P(m=0)≈0.3094 is also correctly inferred) from the analysis. (b) Comparison of the number of simulation steps needed by ABC rejection (nRej) and ABC SMC (nSMC); ABC SMC yields ∼50-fold speed-up on average.CC BY-NC · PubMed Central
Fig. 3. (a) ABC SMC posterior distributions for parameters inferred for a four-parameter model from the data in Supplementary Table 2. Marginal posterior distributions of parameters qc1, qh1 (red) and qc2, qh2 (blue). (b) Approximation of a posterior marginal distribution P(m|D0). Model 1 is a two-parameter and model 2 a four-parameter model (1). All intermediate populations are shown in Supplementary Figure 1a. (c) The same as (a) but here the data used is from Supplementary Table 3. (d) Estimation of a posterior marginal distribution. Model 1 is a two-parameter and model 2 a three-parameter model (1). All intermediate populations are shown in Supplementary Figure 1b.
Fig. 3. (a) ABC SMC posterior distributions for parameters inferred for a four-parameter model from the data in Supplementary Table 2. Marginal posterior distributions of parameters qc1, qh1 (red) and qc2, qh2 (blue). (b) Approximation of a posterior marginal distribution P(m|D0). Model 1 is a two-parameter and model 2 a four-parameter model (1). All intermediate populations are shown in Supplementary Figure 1a. (c) The same as (a) but here the data used is from Supplementary Table 3. (d) Estimation of a posterior marginal distribution. Model 1 is a two-parameter and model 2 a three-parameter model (1). All intermediate populations are shown in Supplementary Figure 1b.CC BY-NC · PubMed Central
Fig. 4. (a) STAT5 signalling pathway. Adapted from (Arbouzova and Zeidler, 2006). (b) Histograms show populations of the model parameter m. Population 20 represents the approximation of the marginal posterior distribution of m. Tolerance schedule: ϵ={200, 100, 50, 35, 30, 25, 22, 20, 19, 18, 17, 16, 15, 14, 13, 12, 11, 10, 9, 8}. Perturbation kernels: KMt(m|m*)=0.6 if m=m* and 0.2 otherwise; KPt(θ|θ*)=U(−σ, σ), σ=0.5 (max{θ}t−1−min{θ}t−1). N=500. Distance function: , with D0={y0(1), y0(2)}, D*={y*(1), y*(2)} and y(1) the total amount of phosphoryalated STAT5 in the cytoplasm and y(2) the total amount of STAT5 in the cytoplasm. σD0(1) and σD0(2) are the associated confidence intervals; reassuringly, other distance functions, e.g. the square root of the sum of squared errors yield identical model selection results (data not shown).
Fig. 4. (a) STAT5 signalling pathway. Adapted from (Arbouzova and Zeidler, 2006). (b) Histograms show populations of the model parameter m. Population 20 represents the approximation of the marginal posterior distribution of m. Tolerance schedule: ϵ={200, 100, 50, 35, 30, 25, 22, 20, 19, 18, 17, 16, 15, 14, 13, 12, 11, 10, 9, 8}. Perturbation kernels: KMt(m|m*)=0.6 if m=m* and 0.2 otherwise; KPt(θ|θ*)=U(−σ, σ), σ=0.5 (max{θ}t−1−min{θ}t−1). N=500. Distance function: , with D0={y0(1), y0(2)}, D*={y*(1), y*(2)} and y(1) the total amount of phosphoryalated STAT5 in the cytoplasm and y(2) the total amount of STAT5 in the cytoplasm. σD0(1) and σD0(2) are the associated confidence intervals; reassuringly, other distance functions, e.g. the square root of the sum of squared errors yield identical model selection results (data not shown).CC BY-NC · PubMed Central

Verification

Claims the paper makes

3 Valid · 0 Not valid · 3 Not checked · 28 Own results and interpretations

17 more claims

Create a free account to see all 34 claims and to verify the 6 references still open.

See all claims — free account
  • Probably not a substantial claim. 7
  • It may state two separate claims as one. 7

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

4 of 4

Citations

Cites

Research works printed but not linked: 2

  1. Jeffreys, 1939, Theory of Probability
  2. Wilkinson, 2007, Bayesian inference of primate divergence times

Cited by

See all 261

Metadata

Authors
Toni, T., & Stumpf, M. P. H.
Affiliations
Imperial College London (GB)
Corresponding author
  • Toni, Tina
  • Stumpf, Michael P. H.
Venue
Bioinformatics
Published
2009-10-29
Volume
26
Issue
1
Pages
104-110
ISSN
  • 1367-4803 (ISSN-L)
  • 1367-4811
OpenAlex
W2126351705
Cited by
260
References
43
Citations by year
2026: 7 · 2025: 12 · 2024: 11 · 2023: 13 · 2022: 6 · 2021: 19 · 2020: 16 · 2019: 15 · 2018: 19 · 2017: 16 · 2016: 15 · 2015: 20 · 2014: 22 · 2013: 19 · 2012: 18
FWCI
6.70
Type
journal article
Language
en
Open access
Locations
  • Bioinformatics (publishedVersion, cc-by-nc)
  • PubMed Central (publishedVersion, cc-by-nc)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • Minerva Access (University of Melbourne) (submittedVersion, cc-by-nc)
  • arXiv (Cornell University)
Funders
  • Directorate for Biological Sciences
  • Medical Research Council
  • Biotechnology and Biological Sciences Research Council (BB/F005210/2, BB/F005210/1, BB/C519670/1, BB/G009374/1)
Topics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene Regulatory Network Analysis
  • Physical Sciences › Mathematics › Statistics and Probability › Markov Chains and Monte Carlo Methods
  • Social Sciences › Arts and Humanities › History and Philosophy of Science › Philosophy of Biology and Scientific Explanation
Keywords
model selection, dynamical systems, approximate Bayesian computation, sequential Monte Carlo, signal transduction, gene regulatory networks, JAK/STAT signaling, population biology, measurement uncertainty, mechanistic model
MeSH terms
Algorithms, Animals, Bayes Theorem, Computer Simulation, Data Interpretation, Statistical, Humans, Models, Biological, Population Dynamics, Models, Statistical

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-04, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-04 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (qwen3.8-omni-flash).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.