CoChat
Sign Up
Cited by 1,442Gene Regulatory Network AnalysisControl Systems and IdentificationMicrobial Metabolic Engineering and Bioproduction

Structural and practical identifiability analysis of partially observed dynamical models by exploiting the profile likelihood

Open access
Raue, A., Kreutz, C., Maiwald, T., Bachmann, J., Schilling, M., Klingmüller, U., Timmer, J.·2009·Bioinformatics
DOI 10.1093/bioinformatics/btp358
Open this paper in CoChat — free accountOpen PDF

Abstract

MOTIVATION: Mathematical description of biological reaction networks by differential equations leads to large models whose parameters are calibrated in order to optimally explain experimental data. Often only parts of the model can be observed directly. Given a model that sufficiently describes the measured data, it is important to infer how well model parameters are determined by the amount and quality of experimental data. This knowledge is essential for further investigation of model predictions. For this reason a major topic in modeling is identifiability analysis. RESULTS: We suggest an approach that exploits the profile likelihood. It enables to detect structural non-identifiabilities, which manifest in functionally related model parameters. Furthermore, practical non-identifiabilities are detected, that might arise due to limited amount and quality of experimental data. Last but not least confidence intervals can be derived. The results are easy to interpret and can be used for experimental planning and for model reduction. AVAILABILITY: An implementation is freely available for MATLAB and the PottersWheel modeling toolbox at http://web.me.com/andreas.raue/profile/software.html. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

AI snapshot

Written from the paper’s full text.

TL;DR

The paper presents a profile-likelihood method for assessing structural and practical identifiability in partially observed dynamical models and for deriving likelihood-based confidence intervals. Applied to a JAK-STAT signaling model, it identifies four structurally non-identifiable parameters and one practically non-identifiable parameter, and shows how additional measurements can resolve these ambiguities.

Key findings

  • Profile likelihood distinguishes structural non-identifiability, seen as a flat profile, from practical non-identifiability, seen as a profile with a minimum that fails to cross the confidence threshold in one or both directions; identifiable parameters cross the threshold on both sides.
  • In the JAK-STAT application, parameters p2, x1(0), s1, and s2 were structurally non-identifiable, with functional relations p2·x1(0)=const, s1·x1(0)=const, and s2·x1(0)=const; p3 was practically non-identifiable for increasing values.
  • Table 1 reports infinite likelihood-based intervals for the structurally non-identifiable parameters, whereas Hessian-based asymptotic intervals were finite but misleadingly narrow, e.g. p2: −33 to +31, x1(0): −31 to +32, s1: −32 to +31, s2: −32 to +31; for p3, the profile-likelihood interval was −1.14 to +∞ versus −1.14 to +0.15 from the Hessian.
  • Adding hypothetical measurements of x1(0)=200±20 nM and x3/(x2+x3)=0.90±0.05 at t=20 min made p2, x1(0), s1, and s2 structurally identifiable and p3 practically identifiable, while p2 remained practically non-identifiable. Profile-likelihood computation took 54±18 s per parameter on a 1 GHz CPU with 2 GB RAM.

Methods

  • The approach uses an ODE model with weighted least-squares objective χ², computes the profile likelihood for each parameter by re-optimizing all other parameters at fixed values of the target parameter, and derives likelihood-based confidence intervals from χ² thresholds.
  • The application is a JAK-STAT signaling model inspired by Swameye et al., calibrated to experimental data; observables are phosphorylated STAT in cytoplasm (y1) and total STAT in cytoplasm (y2) measured by quantitative western blotting, with cytoplasm and nucleus volumes of 1400 µm³ and 450 µm³.
  • Parameters calibrated include p1–p4, s1, s2, and x1(0), with x2(0)–x4(0) set to zero; the model was implemented and fitted with PottersWheel, and experimental planning and model reduction were explored via profile-likelihood trajectories, Monte Carlo simulations, and likelihood-ratio thresholds.

Limitations

  • The authors state that structural non-identifiability cannot be resolved by refining existing measurements or reducing noise; it requires a qualitatively new measurement that changes the observation mapping, such as observing additional species. Practical non-identifiability can be resolved by increasing the amount or quality of data and/or selecting better measurement time points.
  • The authors note that recovering functional relations unambiguously from parameter changes along the profile likelihood is only possible when the corresponding manifold is one-dimensional; higher-dimensional manifolds require further analysis, such as the approach of Hengl et al.
  • Evident from the text: the method is demonstrated on a single JAK-STAT model and dataset, with no broad empirical comparison across many model classes; the approach also assumes a sufficiently adequate model and normally distributed measurement noise.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1093/bioinformatics/btp358

Cites

28 of 28 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

1 of 12 cited sources checked · 0 supported · 0 flagged · accuracy not yet measured

  • Findingown data

    The largest discrepancy between asymptotic and likelihood-based confidence intervals occurs for practically non-identifiable parameters, where asymptotic intervals are significantly smaller.

    “The largest discrepancy occurs for practically non-identifiable parameters, where asymptotic confidence intervals are significantly smaller than likelihood-based confidence intervals.”
    Quote in the paperSubstantial claim
  • Findingown data

    The variability of predicted model observables y along the profile likelihood of p3 remains consistent with the measurement errors.

    “The variability of the predicted model observables y along the profile likelihood of p 3 remains consistent with the measurement errors, as shown in Figure 6.”
    Quote in the paperSubstantial claim
  • Findingown data

    The profile likelihood plots reveal four structurally non-identifiable parameters p2, x1(0), s1, s2 in the JAK-STAT model.

    “The resulting plots of profile likelihood versus parameter reveal four structurally non-identifiable parameters p 2 , x 1 (0), s 1 , s 2 by their flat profile likelihood, see Figure 3, upper panel.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: An appropriate model M that sufficiently describes the available experimental data is assumed.
  • Findingown data

    For identifiable parameters, discrepancies between finite sample confidence intervals from the profile likelihood and asymptotic confidence intervals from the Hessian are small.

    “For identifiable parameters the discrepancies are small.”
    Quote in the paperNot a substantial claim
  • Findingown data

    The functional relations connecting structurally non-identifiable parameters p2, x1(0), s1, s2 can be recovered from changes of remaining parameters along the profile likelihood.

    “The functional relations h connecting these structurally non-identifiable parameters can be recovered from the changes of the remaining parameters, while calculating the profile likelihood of the structurally non-identifiable parameters, as shown in Figure 4.”
    Quote in the paperSubstantial claim
  • May combine two claims. 14
  • Classified as probably not a substantial claim. 13

  • May combine two claims.

    Premisecitation

    The Hengl et al. (2007) approach detects structural non-identifiability via functional relations for arbitrary models but is not intended to detect practical non-identifiability.

    “An approach to detect structural non-identifiability by the corresponding functional relations was introduced by Hengl et al. (2007).It is able to detect flatness of likelihood for arbitrary models, but it is not intended to detect practical non-identifiability.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Hengl et al., 2007 source

      The Approach and Conclusion describe detecting structural non-identifiability through functionally related parameters in nonlinear models. But the Introduction explicitly lists practical identifiability analysis, accounting for data quality, as an aim of the method, contrary to the claim that it is not intended for that purpose. The passages do not specifically describe the method as detecting likelihood flatness. (LLM, conf=0.90)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Findingown data

    Spots of largest variability suggest where and when a measurement of a species most efficiently determines these parameters: x1 at t=0 or t>50; x2 to x4 at 5<t<30.

    “Spots of largest variability suggest where and when a measurement of a species most efficiently determines theses parameters: x 1 at times t = 0 or t > 50; x 2 to x 4 at times 5 < t < 30.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Calculating the profile likelihood takes 54 ± 18 s per parameter in the JAK-STAT application.

    “Calculating the profile likelihood takes 54 ± 18 s per parameter, using an implementation of the approach embedded in the Potters-Wheel fitting toolbox (1 GHz CPU, 2 GB RAM), which is described in the Supplementary Material.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    After additional measurements, parameter p3 becomes practically identifiable, while previously structurally non-identifiable parameter p2 remains practically non-identifiable.

    “Parameter p 3 becomes practically identifiable, only previously structurally non-identifiable parameter p 2 remains practically non-identifiable.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The trajectories of internal states along the profile likelihood of p3 show large antipodal variability of species x2 and x3, revealing that the experimental setup is inappropriate to estimate p3 with confidence.

    “The trajectories of the internal model states x along the profile likelihood of the practically non-identifiable parameter p 3 shown in Figure 6 comprises large antipodal variability of species x 2 and x 3 , revealing that the experimental setup is inappropriate to estimate this parameter with confidence.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The manifold for the JAK-STAT structural non-identifiability can be characterized as h1=p2·x1(0)-const=0, h2=s1·x1(0)-const=0, and h3=s2·x1(0)-const=0.

    “h 1 (θ sub ) = p 2 •x 1 (0)-const = 0 h 2 (θ sub ) = s 1 •x 1 (0)-const = 0 ( 1 1 ) h 3 (θ sub ) = s 2 •x 1 (0)-const = 0”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The approach was applied to a JAK-STAT model; non-identifiable parameters were detected, revealing limitations in the experimental setup, and additional measurements that efficiently improve parameter identification were suggested and validated.

    “The approach was applied to a model of the JAK-STAT signaling pathway.Non-identifiable parameters were detected, revealing limitations in the experimental setup.Additional measurements that efficiently improve parameter identification were suggested and validated.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given

    Asymptotic confidence intervals are exact if the observables depend linearly on θ.

    “They are exact if the observables y depend linearly on θ.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Existing a priori methods for structural non-identifiability become rapidly infeasible with increasing model size and cannot detect practical non-identifiability.

    “Unfortunately, these methods become rapidly infeasible with increasing model size (Margaria et al., 2001;White et al., 2001).Practical non-identifiability cannot be detected, since experimental data are disregarded.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites Margaria et al., 2001 source
    • cites White et al., 2001 source

  • Classified as probably not a substantial claim.

    Premisecitation

    Even identifiable parameters can only be determined within confidence intervals containing the true value with desired probability.

    “Even identifiable parameters can only be determined within confidence intervals, which contain the true value of the parameter with a desired probability (Lehmann and Leo, 1983).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Lehmann and Leo, 1983

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Premisereasoning

    Curvature-based methods are appropriate if functional relations h are linear, but this is often not the case for ODE reaction networks, and they cannot detect practical non-identifiability.

    “These methods are appropriate if functional relations h between the parameters emerging from structural non-identifiability are linear.This is often not the case for reaction networks modeled by ODE, because observables depend non-linearly on the parameters.Practical non-identifiability cannot be detected, because a quadratic approximation is not able to explain increasing but limited behavior of”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites Jacquez and Greif, 1985 source
    • cites Vajda et al., 1989a

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Yao et al., 2003 source

  • May combine two claims.

    Interpretationcitation

    For reaction networks, observables depend nonlinearly on θ and the amount and quality of experimental data is often insufficient, so asymptotic confidence intervals might not be appropriate.

    “However, even for the simplest reaction network the observables y depend nonlinearly on θ and the amount and quality of experimental data is often insufficient.Therefore, asymptotic confidence intervals might not be appropriate (Joshi et al., 2006).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Joshi et al., 2006 source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Likelihood-based confidence intervals are superior to asymptotic confidence intervals for finite samples.

    “Likelihood-based confidence intervals are considered superior to asymptotic confidence intervals for finite samples (Neale and Miller, 1997).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Neale and Miller, 1997 source

  • May combine two claims.

    Interpretationnone given

    Along a practical non-identifiability, observables change only negligibly within measurement accuracy, but internal states may vary strongly.

    “Along a practical non-identifiability, the observable y change only negligibly remaining compliant with the given measurement accuracy.Nevertheless, model behavior in terms of internal states x might vary strongly.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationnone given

    Practical non-identifiability arises frequently if amount and quality of experimental data is insufficient and manifests in an infinite confidence interval.

    “This arises frequently if amount and quality of experimental data is insufficient and manifests in a confidence interval that is infinite.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    The points where the profile likelihood passes threshold α represent likelihood-based confidence intervals.

    “The points of passover represent likelihoodbased confidence intervals as defined in Equation ( 8) (Royston, 2007).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Royston, 2007 source

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationreasoning

    The results of the profile likelihood approach are easy to interpret.

    “The results are easy to interpret”
    • Also stated in: § 6 DISCUSSION · plain
      “The approach results in easily interpretable plots of profile likelihood versus parameter.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationnone given

    The profile likelihood of a practically non-identifiable parameter has a minimum but does not exceed threshold α for increasing and/or decreasing θi.

    “The profile likelihood of a practically non-identifiable parameter has a minimum, but is not excessing a threshold α for increasing and/or decreasing values of θ i (see Definition 1).”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    The approach gives insight into a complex and high-dimensional parameter space.

    “This gives insight into a complex and high-dimensional parameter space.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning

    The approach can be automated, but its output might be evaluated visually.

    “It can be automated, but an explicit advantage is that the output might be evaluated visually.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    The approach can be applied to any parameter estimation problem where a likelihood or similar objective criterion is available, e.g. PDE or SDE.

    “The approach can be applied to any parameter estimation problem, where a likelihood or a similar objective criterion is available, e.g.partial differential equations (PDE) or stochastic differential equations (SDE).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    The profile likelihood can be calculated for each parameter separately, allowing restriction to parameters relevant for the biological question and parallelization.

    “The profile likelihood can be calculated for each parameter separately.Thereby it is possible to restrict the analysis to the parameters relevant for the biological question.Moreover, this allows to perfectly parallelize the approach, which is a major benefit for its scalability.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    Working in logarithmic parameter space linearizes the functional relations given in Equation (11).

    “Working in logarithmic parameter space linearizes the functional relations given in Equation (11).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationnone given

    The impact of new measurements can be evaluated by Monte Carlo simulations.

    “The impact of new measurements can be evaluated by Monte Carlo simulations.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    Identifiability analysis ensures that model complexity is tailored to the information content given by the experimental data.

    “Thus, identifiability analysis ensures that the model complexity is tailored to the information content given by the experimental data.”
    Quote in the paperSubstantial claim
25 of 25

  • The authors state this as an assumption.

    The measurement noise is normally distributed.

    “The measurement noise (t) is assumed to be normally distributed.”

  • The authors state this as an assumption.

    An appropriate model M that sufficiently describes the available experimental data is assumed.

    “Furthermore, an appropriate model M that sufficiently describes the available experimental data is assumed.”

  • The paper relies on this without support.

    No approach for explicit testing of practical non-identifiability is available.

    “Similar to a clear formal definition, an approach for explicit testing of practical non-identifiability is not available to our knowledge.”

  • The authors state this as an assumption.

    The initial values of species x2 to x4 are equal to zero.

    “The initial values of species x 2 to x 4 are assumed to be equal to zero.”

  • The authors state this as an assumption.

    For the model-reduction argument, a parameter θi is practically non-identifiable for decreasing parameter value.

    “Assume that a parameter θ i is practically non-indentifiable for decreasing parameter value.”
5 of 5

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Parameter estimations usually yield statistical information related to parameter precision, not accuracy, such as standard errors or correlation matrices, and sometimes information from more sophisticated approaches such as profile likelihood.

    “Such parameter estimations usually yield statistical information related to parameter precision (importantly, not accuracy) such as standard errors or correlation matrices, and sometimes information derived from more sophisticated approaches such as profile likelihood.39”

    From Best Practices to Maximize the Use and Reuse of Quantitative and Systems Pharmacology Models: Recommendations From the United Kingdom Quantitative and Systems Pharmacology Network (2019) · cites this paper as [39]

    Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

    Experimental — accuracy not yet measured

  • Raue et al. (2009) referred to likelihood ratio C.I.s as finite sample C.I.s, to underscore the superiority of likelihood ratio C.I.s over Wald-type C.I.s.

    “Raue et al. (2009) referred to likelihood ratio C.I.s as finite sample C.I.s, to underscore the superiority of likelihood ratio C.I.s over Wald-type C.I.s.”

    From rSalvador: An R Package for the Fluctuation Experiment (2017) · cites this paper as Raue et al. 2009

  • When two models are nested, their likelihood ratio is approximately distributed according to a chi-square distribution with p degrees of freedom, where p is the difference in the number of parameters.

    “When two models are nested (Mð hPL Þ and Mð hopt Þ) their likelihood ratio is approximately distributed according to a v 2 p distribution.Here p refers to the number of degrees of freedom which is defined as the difference in the number of parameters (which in this case is 1).”

    From Parameter uncertainty in biochemical models described by ordinary differential equations (2013) · cites this paper as [36]

  • In cases where asymptotic intervals are inappropriate, more reliable confidence intervals can be obtained using the Profile Likelihood method.

    “In such cases, more reliable confidence intervals can be obtained using the Profile Likelihood method [36].”

    From Parameter uncertainty in biochemical models described by ordinary differential equations (2013) · cites this paper as [36]

  • Techniques for determining which states to measure exist in both frequentist and Bayesian settings.

    “Techniques for this exist in both a frequentist [13,36,91] as well as a Bayesian setting [87,90].”

    From Parameter uncertainty in biochemical models described by ordinary differential equations (2013) · cites this paper as [36]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Concentration unit conflicts with figure labels

    Minor

    The text says concentrations are modeled in nanometers, a unit of length, while the figure labels use nM, a unit of concentration.

1 of 1

Cites

Flagged first, then by citations

1 of the 29 references couldn’t be matched to a known paper.

Cited by

Sorted by citations

Related

Authors
Raue, A., Kreutz, C., Maiwald, T., Bachmann, J., Schilling, M., Klingmüller, U., & Timmer, J.
Affiliations
University of Freiburg (DE)
Corresponding author
Raue, A.
ORCID
Venue
Bioinformatics
Publisher
Oxford University Press (OUP)
Published
2009-06-08
Volume
25
Issue
15
Pages
1923-1929
ISSN
  • 1367-4803 (ISSN-L)
  • 1367-4811
DOI
10.1093/bioinformatics/btp358
OpenAlex
W2168648059
Cited by
1,442
References
29
Citations by year
2026: 209 · 2025: 107 · 2024: 103 · 2023: 82 · 2022: 98 · 2021: 118 · 2020: 97 · 2019: 116 · 2018: 94 · 2017: 78 · 2016: 87 · 2015: 64 · 2014: 52 · 2013: 54 · 2012: 39
FWCI
10.99
Type
journal article
Language
en
Open access
Locations
  • Bioinformatics (publishedVersion)
  • PubMed (publishedVersion)
  • OPUS (Augsburg University) (submittedVersion)
Funders
  • Deutsches Krebsforschungszentrum
  • European Commission (223188, EU-FP7, FP7 HEALTH)
  • Bundesministerium für Bildung und Forschung (FRISYS 0313921, 0313074D, 0313921)
  • FP7 Health
Topics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene Regulatory Network Analysis
  • Physical Sciences › Engineering › Control and Systems Engineering › Control Systems and Identification
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Microbial Metabolic Engineering and Bioproduction
Keywords
identifiability analysis, profile likelihood, parameter estimation, confidence intervals, experimental design, model reduction
MeSH terms
Algorithms, Models, Biological, Probability, Computational Biology

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-11.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-11.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review