CoChat

Likelihood based observability analysis and confidence intervals for predictions of dynamic models

Open access
Kreutz, Clemens, Raue, Andreas, Timmer, Jens · 2012 · BMC Systems Biology

The DOI matches its Crossref record; 22 of the 22 references it cites were found in a registry or an index; 5 of the 21 sources its claims cite were checked for support.

Abstract

BACKGROUND: Predicting a system's behavior based on a mathematical model is a primary task in Systems Biology. If the model parameters are estimated from experimental data, the parameter uncertainty has to be translated into confidence intervals for model predictions. For dynamic models of biochemical networks, the nonlinearity in combination with the large number of parameters hampers the calculation of prediction confidence intervals and renders classical approaches as hardly feasible. RESULTS: In this article reliable confidence intervals are calculated based on the prediction profile likelihood. Such prediction confidence intervals of the dynamic states can be utilized for a data-based observability analysis. The method is also applicable if there are non-identifiable parameters yielding to some insufficiently specified model predictions that can be interpreted as non-observability. Moreover, a validation profile likelihood is introduced that should be applied when noisy validation experiments are to be interpreted. CONCLUSIONS: The presented methodology allows the propagation of uncertainty from experimental to model predictions. Although presented in the context of ordinary differential equations, the concept is general and also applicable to other types of models. Matlab code which can be used as a template to implement the method is provided at http://www.fdmold.uni-freiburg.de/∼ckreutz/PPL.

Overview

CoChat Summary

TL;DR

The paper presents a likelihood-based prediction profile likelihood method for computing confidence intervals for predictions of nonlinear dynamic models without sampling high-dimensional parameter spaces. It applies the method to a small reaction model and a MAP kinase ODE model, and introduces validation profile likelihood and data-based observability analysis.

Key findings

  • Prediction profile likelihood (PPL) yields prediction confidence intervals by constrained maximum likelihood, avoiding explicit high-dimensional parameter sampling; thresholding the PPL gives statistically accurate intervals.
  • A validation profile likelihood (VPL) extends the approach to noisy validation experiments; validation confidence intervals are larger than prediction confidence intervals because they account for measurement error.
  • Flat PPLs produce unbounded prediction confidence intervals and indicate non-observability. In the small model, measuring C only at t = 0,2,…,20 gave unbounded PCIs for A(t) and B(t) for t > 0, while B(0) was known to be zero.
  • In the MAP kinase model, 14 parameters were estimated from three times eleven data points; the largest absolute PCI was 176 nM for Erk at t = 181 seconds, Raf/Raf* PCIs were below 40 nM, and Mek* intervals nearly spanned 0–100 nM.

Methods

  • For ODE state-space models with additive Gaussian noise, the PPL is defined as the maximum likelihood over parameters satisfying the constraint that the model response equals a fixed prediction value z; confidence intervals use a chi-square threshold. The VPL relaxes this constraint and penalizes deviation from a validation data point by its standard deviation.
  • Two examples are used: a small consecutive reaction A→B→C with rates θ1 = 0.05, θ2 = 0.1, initial A(0) = θ3 = 1, B(0) = 0, C(0) = 0, C measured at t = 0,10,…,100 with σ = 0.1; and a MAP kinase ODE model with eight dynamic states, 14 estimated parameters, measurements at 0,100,…,1000 seconds, and σ = 10 nM.
  • Analysis uses constrained optimization/profile likelihood calculations, thresholding, and Monte-Carlo coverage checks in Additional file 1; Matlab template code is provided.

Limitations

  • Stated: prediction confidence regions are point-wise; multiple predictions from one data set are statistically dependent, and a common region for two dependent predictions would require a two-dimensional prediction profile likelihood, which is beyond the article’s scope.
  • Stated: Bayesian MCMC could perform better if a comprehensive prior were available, but in cell biology prior knowledge is very restricted; prior information can be incorporated via maximum a-posteriori estimation.
  • Evident from the text: the method relies on reliable numerical parameter optimization and, in the main derivation, assumes additive Gaussian noise with known variance; if assumptions are violated, the threshold must be generated empirically by Monte-Carlo simulations.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1186/1752-0509-6-120

Cites

22 of 22 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Figures

Figure 1 Illustration model. The three figures in panel (a) show the dynamics and measurement realization for the small model used for illustration purpose. C(t) is measured and the dynamics of all states, i.e. A(t), B(t), and C(t), is intended to be predicted. Panel (b) shows as an example the prediction profile likelihood (gray dashed curve) and validation profile likelihood (black dashed curve) of A(t = 10). Thresholding yields confidence intervals for prediction (gray vertical lines) and validation (black vertical lines). The threshold and the respective projections correspond to the α = 90% confidence interval. The VCIs are larger than the PCIs, because they account for the measurement error of a validation data point. Panels (c)-(e) show prediction confidence intervals (gray) for the unobserved states A(t), B(t), as well as for the measured state C(t). The prediction profile likelihood functions are plotted as black curves in vertical direction. Non-observability is illustrated in panels (f)-(h). Panel (f) shows a realization of the measurements for a design which does not provide sufficient information about the steady state of C. This leads to a flat prediction profile likelihood for large values for A(t) as shown in panel (g), as well as for B(t) for t > 0 as plotted in panel (h). A flat prediction profile likelihood in turn yields unbounded prediction and validation confidence intervals and non-observability of A(t) and B(t) as indicated by the gray shaded regions.
Figure 1 Illustration model. The three figures in panel (a) show the dynamics and measurement realization for the small model used for illustration purpose. C(t) is measured and the dynamics of all states, i.e. A(t), B(t), and C(t), is intended to be predicted. Panel (b) shows as an example the prediction profile likelihood (gray dashed curve) and validation profile likelihood (black dashed curve) of A(t = 10). Thresholding yields confidence intervals for prediction (gray vertical lines) and validation (black vertical lines). The threshold and the respective projections correspond to the α = 90% confidence interval. The VCIs are larger than the PCIs, because they account for the measurement error of a validation data point. Panels (c)-(e) show prediction confidence intervals (gray) for the unobserved states A(t), B(t), as well as for the measured state C(t). The prediction profile likelihood functions are plotted as black curves in vertical direction. Non-observability is illustrated in panels (f)-(h). Panel (f) shows a realization of the measurements for a design which does not provide sufficient information about the steady state of C. This leads to a flat prediction profile likelihood for large values for A(t) as shown in panel (g), as well as for B(t) for t > 0 as plotted in panel (h). A flat prediction profile likelihood in turn yields unbounded prediction and validation confidence intervals and non-observability of A(t) and B(t) as indicated by the gray shaded regions.CC BY · PubMed Central
Figure 2 MAP kinase model. Panel (a) shows the MAP kinase model according to [25]. It is assumed that the phosphorylated compounds are measured. The dynamics of all compounds is intended to be predicted to illustrate the prediction profile likelihood approach. In panel (b) the dynamics of the MAP kinase model as well as simulated data set are plotted. The 90% confidence intervals of the dynamic variables for predictions (dark gray) and for validation experiments (light gray) for this noise realization are plotted in panel (c). The size of the prediction confidence interval (PCI) is plotted as a dashed-dotted line. In absolute concentrations, the dynamics of Erk∗∗has the largest PCI at t = 181 seconds, i.e. when the negative feedback is activated. Also, the dynamics of Mek∗is only badly observable in our example. Measurements of both would be very informative for better calibrating the model.
Figure 2 MAP kinase model. Panel (a) shows the MAP kinase model according to [25]. It is assumed that the phosphorylated compounds are measured. The dynamics of all compounds is intended to be predicted to illustrate the prediction profile likelihood approach. In panel (b) the dynamics of the MAP kinase model as well as simulated data set are plotted. The 90% confidence intervals of the dynamic variables for predictions (dark gray) and for validation experiments (light gray) for this noise realization are plotted in panel (c). The size of the prediction confidence interval (PCI) is plotted as a dashed-dotted line. In absolute concentrations, the dynamics of Erk∗∗has the largest PCI at t = 181 seconds, i.e. when the negative feedback is activated. Also, the dynamics of Mek∗is only badly observable in our example. Measurements of both would be very informative for better calibrating the model.CC BY · PubMed Central

Verification

Claims the paper makes

1 Valid · 5 Not valid · 13 Not checked · 62 Own results and interpretations

42 more claims

Create a free account to see all 81 claims and to verify the 16 references still open.

See all claims — free account
  • It may state two separate claims as one. 15
  • Probably not a substantial claim. 20

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 22 references

Cited by

See all 134

Metadata

Authors
Kreutz, C., Raue, A., & Timmer, J.
Affiliations
University of Freiburg (DE)
Corresponding author
Kreutz, Clemens
Venue
BMC Systems Biology
Published
2012
Volume
6
Issue
1
Pages
120
ISSN
1752-0509 (ISSN-L)
OpenAlex
W2131230954
Cited by
134
References
21
Citations by year
2026: 4 · 2025: 19 · 2024: 8 · 2023: 8 · 2022: 12 · 2021: 7 · 2020: 10 · 2019: 13 · 2018: 6 · 2017: 9 · 2016: 8 · 2015: 8 · 2014: 9 · 2013: 9 · 2012: 4
FWCI
3.85
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • BMC Systems Biology (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • DiVA (Linkoping University) (submittedVersion, other-oa)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
  • FreiDok plus (Universitätsbibliothek Freiburg) (submittedVersion)
  • Site cant be reached (submittedVersion, cc-by)
  • OPUS (Augsburg University) (submittedVersion, cc-by)
  • arXiv (Cornell University)
Funders
  • Deutsches Krebsforschungszentrum
  • Deutsche Forschungsgemeinschaft
Topics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene Regulatory Network Analysis
  • Physical Sciences › Engineering › Control and Systems Engineering › Control Systems and Identification
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Microbial Metabolic Engineering and Bioproduction
Keywords
confidence intervals, observability analysis, parameter uncertainty, dynamic model, biochemical reaction networks, non-identifiable parameters, error propagation, ordinary differential equations, model validation
MeSH terms
Models, Biological, Likelihood Functions, Mitogen-Activated Protein Kinases, MAP Kinase Signaling System, Systems Biology

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (qwen3.8-omni-flash).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Likelihood based observability analysis and confidence intervals for predictions of dynamic models. https://app.cochat.ai/papers/10.1186/1752-0509-6-120
The paper
Kreutz, C., Raue, A., & Timmer, J. (2012). Likelihood based observability analysis and confidence intervals for predictions of dynamic models. BMC Systems Biology. https://doi.org/10.1186/1752-0509-6-120

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.