CoChat

A Bayesian approach to targeted experiment design

Open access
Vanlier, J., Tiemann, C. A., Hilbers, P. A. J., van Riel, N. A. W. · 2012 · Bioinformatics

The DOI matches its Crossref record; 27 of the 27 references it cites were found in a registry or an index; 1 of the 13 sources its claims cite were checked for support.

Abstract

Motivation: Systems biology employs mathematical modelling to further our understanding of biochemical pathways. Since the amount of experimental data on which the models are parameterized is often limited, these models exhibit large uncertainty in both parameters and predictions. Statistical methods can be used to select experiments that will reduce such uncertainty in an optimal manner. However, existing methods for optimal experiment design (OED) rely on assumptions that are inappropriate when data are scarce considering model complexity. Results: We have developed a novel method to perform OED for models that cope with large parameter uncertainty. We employ a Bayesian approach involving importance sampling of the posterior predictive distribution to predict the efficacy of a new measurement at reducing the uncertainty of a selected prediction. We demonstrate the method by applying it to a case where we show that specific combinations of experiments result in more precise predictions. Availability and implementation: Source code is available at: http://bmi.bmt.tue.nl/sysbio/software/pua.html Contact: j.vanlier@tue.nl; N.A.W.v.Riel@tue.nl Supplementary information: Supplementary data are available at Bioinformatics online.

Overview

CoChat Summary

TL;DR

The paper proposes a Bayesian, sampling-based method for optimal experiment design that targets reduction of uncertainty in a prediction of interest under large parameter uncertainty. It uses posterior predictive distributions and importance sampling to estimate expected variance reduction, demonstrated on a JAK-STAT signalling model.

Key findings

  • The method predicts the efficacy of new measurements by computing expected variance reduction (EVR) from the posterior predictive distribution, without requiring linearization around a best-fit parameter set.
  • In the JAK-STAT case study, the largest reduction in variance for the time-to-peak of dimerized nuclear STAT (state 4) was obtained by measuring state 1 at an early and a late time point; measuring state 3 and state 1 gave smaller reductions but required less stringent timing accuracy.
  • Additional measurements on the two observables already used to parameterize the original posterior (outputs 5 and 6) provided very little additional variance reduction.
  • The linear variance reduction (LVR) approximation agreed qualitatively with the importance-sampling SVR and was about 100-fold faster for T=1000 samples.

Methods

  • Bayesian workflow: compute posterior parameter distribution via Metropolis–Hastings MCMC, simulate posterior predictive distributions for candidate experiments, and estimate expected variance reduction by self-normalized importance sampling; a Gaussian LVR approximation is also derived.
  • Applied to a JAK-STAT pathway model using data from Swameye et al. (dataset 1), with measured total STAT and cytoplasmic phosphorylated STAT, uniform log-space priors for kinetic parameters, a Gaussian initial condition (μ=200 nM, σ=20 nM), and a bounded non-identifiable parameter two.
  • Two MCMC chains were run up to one million parameter sets and thinned to 10,000 samples; experiment space was sampled by Monte Carlo, and SVR computation was parallelized on a GPU using OpenCL.

Limitations

  • The paper states that measurement timing is assumed instantaneous/infinitely accurate, and incorporating temporal inaccuracies remains an open challenge; it also notes that design is based on the expected value of a distribution of variance reductions, leaving the trade-off between EVR and its inaccuracy open.
  • The paper states that parameters must be identifiable or restricted by finite priors, MCMC convergence is hard to assess, prior dependence should be verified a posteriori, strongly tailed distributions can make variance estimates unreliable, and low effective sample size can make importance-sampling estimates poor.
  • The paper states that obtaining PPDs and performing experiment design is computationally expensive, and densely sampling experiment space can become prohibitively time consuming for many experiments; the text does not report prospective wet-lab validation of the proposed designs (evident from the text).

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1093/bioinformatics/bts092

Cites

27 of 27 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Fig. 1. Illustration of the effect of adding a new data point on the PPD. Shown on the top right is the PPD at one specific time point for two predictions with a subset of the samples of the chain indicated with white points. The square denotes the location of the ‘new measurement’. Prediction A refers to a prediction of which a new measurement can be performed (observable), whereas B denotes the prediction of interest. Here the grey distribution corresponds to the PPD before the new measurement, whereas the white Gaussian corresponds to the error model of the new measurement. Due to additional constraints imposed by this new measurement in combination with the old data and the model, the distribution on the hypothesis side is also updated in light of the new data point and shown in white.
Fig. 1. Illustration of the effect of adding a new data point on the PPD. Shown on the top right is the PPD at one specific time point for two predictions with a subset of the samples of the chain indicated with white points. The square denotes the location of the ‘new measurement’. Prediction A refers to a prediction of which a new measurement can be performed (observable), whereas B denotes the prediction of interest. Here the grey distribution corresponds to the PPD before the new measurement, whereas the white Gaussian corresponds to the error model of the new measurement. Due to additional constraints imposed by this new measurement in combination with the old data and the model, the distribution on the hypothesis side is also updated in light of the new data point and shown in white.CC BY-NC · PubMed Central
Fig. 2. Model of the JAK-STAT pathway. In this model u1 serves as driving input, while the total concentration of STAT (x1+x2+2x3) and the total concentration of phosphorylated STAT in the cytoplasm (x2+2x3) were measured. Note that the step from x4 back to x1 is associated with a delay.
Fig. 2. Model of the JAK-STAT pathway. In this model u1 serves as driving input, while the total concentration of STAT (x1+x2+2x3) and the total concentration of phosphorylated STAT in the cytoplasm (x2+2x3) were measured. Note that the step from x4 back to x1 is associated with a delay.CC BY-NC · PubMed Central
Fig. 3. Top left: one simulated time course of state 3 superimposed on the PPD. Two time points are indicated with circles. Bottom left: correlation coefficient between states 3 and 4 and SVR of state 4 based on a measurement of state 3 (SVR). The relation between the two states at the indicated time points is shown in both scatter plot and 2D histogram form. The former shows the actual samples from the PPD for one point in time. Here the dots represent simulated values belonging to different parameter sets from the MCMC chain. In the histogram the colour indicates the number of samples in a particular region which is proportional to the probability density.
Fig. 3. Top left: one simulated time course of state 3 superimposed on the PPD. Two time points are indicated with circles. Bottom left: correlation coefficient between states 3 and 4 and SVR of state 4 based on a measurement of state 3 (SVR). The relation between the two states at the indicated time points is shown in both scatter plot and 2D histogram form. The former shows the actual samples from the PPD for one point in time. Here the dots represent simulated values belonging to different parameter sets from the MCMC chain. In the histogram the colour indicates the number of samples in a particular region which is proportional to the probability density.CC BY-NC · PubMed Central
Fig. 4. Variance reduction of the peak time of dimerized STAT (x4) with respect to two new measurements. (A) Each axis represents an experiment, where the different model outputs are numbered. Numbers 1 to 3 correspond to the first three states whereas 4 and 5 correspond to the sums of states on which the original PPD was parametrized. Note that each block on each axis corresponds to an entire time series. The block corresponding to experiments involving state 1 is shown enlarged in (B). Variance reduction is computed using the importance sampling method.
Fig. 4. Variance reduction of the peak time of dimerized STAT (x4) with respect to two new measurements. (A) Each axis represents an experiment, where the different model outputs are numbered. Numbers 1 to 3 correspond to the first three states whereas 4 and 5 correspond to the sums of states on which the original PPD was parametrized. Note that each block on each axis corresponds to an entire time series. The block corresponding to experiments involving state 1 is shown enlarged in (B). Variance reduction is computed using the importance sampling method.CC BY-NC · PubMed Central
Fig. 5. Comparison of two methods for calculating the variance reduction. Variance reduction of the peak time of dimerized STAT (x4) with respect to two new measurements. (A) LVR. (B) Difference between the variance reduction computed by means of LVR and importance sampling (shown in Fig. 4).
Fig. 5. Comparison of two methods for calculating the variance reduction. Variance reduction of the peak time of dimerized STAT (x4) with respect to two new measurements. (A) LVR. (B) Difference between the variance reduction computed by means of LVR and importance sampling (shown in Fig. 4).CC BY-NC · PubMed Central

Verification

Claims the paper makes

1 Valid · 0 Not valid · 5 Not checked · 40 Own results and interpretations

25 more claims

Create a free account to see all 46 claims and to verify the 12 references still open.

See all claims — free account
  • It may state two separate claims as one. 8
  • Probably not a substantial claim. 10

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

Show all 27 references

Cited by

See all 98

Metadata

Authors
Vanlier, J., Tiemann, C. A., Hilbers, P. A. J., & van Riel, N. A. W.
Affiliations
Eindhoven University of Technology (NL)
Corresponding author
Vanlier, J.
Venue
Bioinformatics
Published
2012-02-24
Volume
28
Issue
8
Pages
1136-1142
ISSN
  • 1367-4803 (ISSN-L)
  • 1367-4811
OpenAlex
W2110690917
Cited by
98
References
27
Citations by year
2026: 4 · 2025: 6 · 2024: 5 · 2023: 3 · 2022: 4 · 2021: 6 · 2020: 10 · 2019: 7 · 2018: 9 · 2017: 10 · 2016: 9 · 2015: 5 · 2014: 12 · 2013: 5 · 2012: 3
FWCI
10.58
Type
journal article
Language
en
Open access
Locations
  • Bioinformatics (publishedVersion, cc-by-nc)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by-nc)
  • PubMed Central (publishedVersion, cc-by-nc)
  • PubMed (publishedVersion)
  • TU/e Research Portal (publishedVersion)
  • TU/e Research Portal (submittedVersion)
Topics
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Optimal Experimental Design Methods
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene Regulatory Network Analysis
  • Physical Sciences › Engineering › Control and Systems Engineering › Control Systems and Identification
Keywords
optimal experiment design, Bayesian optimization, parameter uncertainty, posterior predictive distribution, importance sampling, systems biology modeling, data scarcity, prediction precision
MeSH terms
Algorithms, Bayes Theorem, Monte Carlo Method, Programming Languages, Research Design, Signal Transduction, Uncertainty, Systems Biology, STAT Transcription Factors, Janus Kinases

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (qwen3.8-omni-flash).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: A Bayesian approach to targeted experiment design. https://app.cochat.ai/papers/10.1093/bioinformatics/bts092
The paper
Vanlier, J., Tiemann, C. A., Hilbers, P. A. J., & van Riel, N. A. W. (2012). A Bayesian approach to targeted experiment design. Bioinformatics. https://doi.org/10.1093/bioinformatics/bts092

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.