Optimal Experimental Design for Parameter Estimation of a Cell Signaling Model
Open accessThe DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index; 10 of the 22 sources its claims cite were checked for support.
Abstract
Differential equation models that describe the dynamic changes of biochemical signaling states are important tools to understand cellular behavior. An essential task in building such representations is to infer the affinities, rate constants, and other parameters of a model from actual measurement data. However, intuitive measurement protocols often fail to generate data that restrict the range of possible parameter values. Here we utilized a numerical method to iteratively design optimal live-cell fluorescence microscopy experiments in order to reveal pharmacological and kinetic parameters of a phosphatidylinositol 3,4,5-trisphosphate (PIP(3)) second messenger signaling process that is deregulated in many tumors. The experimental approach included the activation of endogenous phosphoinositide 3-kinase (PI3K) by chemically induced recruitment of a regulatory peptide, reversible inhibition of PI3K using a kinase inhibitor, and monitoring of the PI3K-mediated production of PIP(3) lipids using the pleckstrin homology (PH) domain of Akt. We found that an intuitively planned and established experimental protocol did not yield data from which relevant parameters could be inferred. Starting from a set of poorly defined model parameters derived from the intuitively planned experiment, we calculated concentration-time profiles for both the inducing and the inhibitory compound that would minimize the predicted uncertainty of parameter estimates. Two cycles of optimization and experimentation were sufficient to narrowly confine the model parameters, with the mean variance of estimates dropping more than sixty-fold. Thus, optimal experimental design proved to be a powerful strategy to minimize the number of experiments needed to infer biological parameters from a cell signaling assay.
Overview
CoChat Summary
TL;DR
The paper applies optimal experimental design to a live-cell PI3K/PIP3 signaling assay and shows that an intuitively planned protocol yields poorly defined parameters, whereas numerically optimized drug concentration-time profiles sharply reduce parameter uncertainty. Two cycles of optimization and experimentation reduced the mean variance of estimates from 193.5 to 2.9, more than sixty-fold.
Key findings
- The established protocol using iRap and LY294002 produced high parameter uncertainty and strong covariance; the initial fit had a mean variance of 193.5 and relative uncertainties indicating the intuitive protocol was not suitable for defining parameters of interest.
- A first numerically optimized experiment was predicted to reduce the mean of variances more than 50-fold compared with repeating the established protocol; after two optimization/experimentation cycles, the mean variance dropped from 193.5 to 2.9.
- The optimized protocols were non-intuitive but informative: the first used low-dose iRap, then higher iRap with LY29, then LY29 washout with iRap; the second used constrained drug additions and optimized time points, with the determinant criterion focusing on the LY29 inhibitor constant.
- Data from a single optimized experiment largely eliminated correlation between estimates, and the eigenvalue spectrum of the covariance matrix was consistently reduced, though some parameter estimates still varied strongly across experiments.
Methods
- NIH 3T3 fibroblasts were transfected with Lyn-FRB, CF-p85, and Y-PH constructs, serum-starved, and imaged by live-cell confocal fluorescence microscopy; cytosolic fluorescence of CF-p85 and Y-PH was quantified from single cells, with the traditional protocol using 25 frames captured every minute.
- A system of ordinary differential equations modeled PIP3, CF-p85/iRap/Lyn-FRB complex formation, and Y-PH membrane binding; parameters were estimated by weighted least squares using a multiple shooting Gauss-Newton algorithm, with measurement errors estimated from individual frames.
- Optimal experimental designs were computed by minimizing functions of the parameter covariance matrix, using sequential quadratic programming in VPLAN/PARFIT; the trace criterion was used for the first optimized design, and the determinant criterion under dynamic constraints for the second.
Limitations
- The paper states that parameter estimates varied strongly from experiment to experiment, with most relative uncertainties remaining above 100%, and that the exogenous parameter describing CF-p85 association to Lyn-FRB increased in uncertainty because Lyn-FRB is not fluorescently tagged and is virtually unidentifiable.
- The paper notes that minimizing the determinant can focus experimental effort on a single parameter and be sensitive to overall uncertainty, and that experimental imperfection such as residual LY29 after washout introduced uncertainty, though it did not significantly affect the realized design performance.
- The paper states that determined parameter values are model-specific and that future optimization methods need to address robustness to parameter uncertainty, model structure, and experimental uncertainty; evident from the text, the model also relies on simplifying assumptions such as irreversible binding steps and Y-PH not shielding PIP3 from degradation.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures




Verification
Claims the paper makes
6 Valid · 2 Not valid · 11 Not checked · 74 Own results and interpretations
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“In their study, Gutenkunst et al. used published intuitively designed experimental protocols to characterize parameter identifiability in different models [6].”
§ Optimal Experimental Design for Parameter Estimation of a Cell Signaling Model > Results > Posterior assessment
What the sources say
- Disputedcites [6] RN Gutenkunst, 2007, Universally sloppy parameter sensitivities in systems biology models., PLoS Comput Biol sourceExperimental — accuracy not yet measured
The source does examine parameter identifiability/constraining across many published models (Introduction and Discussion: 'By examining 17 models from the systems biology literature... we showed that their parameter sensitivities all share striking common features deemed sloppy'), but it did not use published experimental protocols to do so. In Results > Systems Biology Models Have Sloppy Sensitivity Spectra the models were taken from the BioModels database and behaviour change is defined as 'a kind of continuous least-squares fit of parameters θ to data simulated from published parameters θ*', with the caveat that 'In most cases, these parameters were not systematically fit or measured in the original publication.' Results > Parameter Values from Collective Fits calls this 'an idealized fitting procedure' / 'this idealized situation' (synthetic perfect-fit time-series, not experiments), and Results > Predictions from Direct Parameter Measurements relies on 'hypothetical direct parameter measurements... centered about the original best-fit values.' The Discussion instead recommends optimal experimental design for future experiments, contrary to the claim that existing protocols were used. (LLM, conf=0.72)
Checked by a reasoning model — may be wrong.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
Every source the paper cites for this claim was checked and supports it.
In the paper
“Although the computed experimental protocols and sampling schemes are unavoidably based only on approximate models and parameter values, experience shows that data evaluated from optimized experiments are typically more informative than data from intuitively planned experiments[8].”
§ Optimal Experimental Design for Parameter Estimation of a Cell Signaling Model > Introduction
What the sources say
- Supportedcites [8] I Bauer, 2000, Numerical methods for optimum experimental design in DAE systems., J Comput Appl Math source
The comparison in §7.6 reports that two optimized experiments produced more precise parameter estimates than 15 intuitively designed experiments; Table 2 also shows a substantially lower A-criterion value for the optimized design. The paper concludes that this comparison demonstrates the potential of numerical design to save experimental effort, supporting the citing sentence’s qualitative comparison. (LLM, conf=0.88)
hedgedstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Strong covariance between parameters of reaction cascades, fast reversible reactions, and binding events makes it difficult to constrain single parameter values [3],[6],[10],[23].”
§ Optimal Experimental Design for Parameter Estimation of a Cell Signaling Model > Discussion > Enhancing the identifiability of endogenous parameters by chemical biology
What the sources say
- Supportedcites [3] KS Brown, 2004, The statistical mechanics of complex signaling networks: nerve growth factor signaling., Phys Biol source
Section 3.3 gives the binding-event case explicitly: for a first-order protein binding reaction with k_on and k_off, only the equilibrium constant K_d = k_on/k_off matters, so the two individual rates can be scaled together freely — the very covariance that leaves single rate constants unconstrained. Section 3.5 states the 'enormous error bars on the rate constants' and that 'other parameters may shift to compensate for part of the change in a single parameter leads to larger error ranges', concluding 'we cannot predict reaction rate constants with our time series data'; the Conclusions attribute this to 'covariance of other rate constants that can compensate' in the PC12 NGF/Erk phosphorylation cascade. The only element not spelled out is the word 'fast' for the reversible steps; the reversible on/off binding structure itself is the paper's example. (LLM, conf=0.82)
- Partly supportedcites [6] RN Gutenkunst, 2007, Universally sloppy parameter sensitivities in systems biology models., PLoS Comput Biol sourceExperimental — accuracy not yet measured
Gutenkunst et al. (2007) supports the core of the claim for cascades and binding: Results 'Parameter Values from Collective Fits' states that 'nonlinear multiparameter models often allow a wide range of correlated parameters to fit data,' with the 1987 Brodersen ligand-binding (hemoglobin/albumin) example showing 'individual parameter values spanning huge ranges' and the same sloppy eigenvalue spectrum, and the Introduction/Results on the PC12 growth-factor (kinase-cascade) model note that collective fitting 'did not tightly constrain any single parameter value' ('Not a single parameter was constrained to better than a factor of 50'). However, our copy does not attribute such covariance to 'fast reversible reactions': the Discussion raises time/concentration-scale separation only as Brown and Sethna's proposed cause [20], doubts it explains the sloppiness, and reports that rescaling all concentrations and rate constants to 1 leaves the model sloppy (Text S4). So the reaction-cascade and binding-event parts are supported while the fast-reversible-reaction part is not. (LLM, conf=0.78)
- Not checked yetcites [10] D Faller, 2003, Simulation methods for optimal experimental design in systems biology., Simulation source
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Finally, reflects the activity of phosphatases like phosphatase and tensin homolog (PTEN) which is lost in many cancers [26].”
§ Optimal Experimental Design for Parameter Estimation of a Cell Signaling Model > Discussion > Pharmacological, biochemical, and clinical cell signaling assays
What the sources say
- Partly supportedcites [26] K Stemke-Hale, 2008, An integrative genomic and proteomic analysis of pik3ca, pten, and akt mutations in breast cancer., Cancer Res sourceExperimental — accuracy not yet measured
The Introduction identifies PTEN as a phosphatase that reverses PI3K activity and reports PTEN protein loss in about 30% of breast cancers. The Results link low PTEN levels to increased PI3K-pathway signaling. The paper does not identify the unnamed parameter or establish that it reflects PTEN activity, and its quantified PTEN-loss finding is limited to breast cancer. (LLM, conf=0.83)
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Independently of the size of these models, they found that a substantial fraction of eigenvectors of the uncertainty region ran along multiple axes in parameter space.”
§ Optimal Experimental Design for Parameter Estimation of a Cell Signaling Model > Results > Posterior assessment
What the sources say
- Supportedcites [6] RN Gutenkunst, 2007, Universally sloppy parameter sensitivities in systems biology models., PLoS Comput Biol source
The Results section 'Systems Biology Models Have Sloppy Sensitivity Spectra' states that 'the eigenvectors often tend to involve significant components of many different parameters' and, via the I/P analysis of the uncertainty ellipsoids' principal axes, that 'very few axes have I / P ≈ 1; the ellipses are skewed from single parameter directions' — supporting the claim's core point that a substantial fraction of the uncertainty region's eigenvectors run along multiple parameter axes. The 'independently of model size' qualifier is carried implicitly rather than as an explicit size comparison: the paper reports the property for all 17 models in its collection (Figure 1 caption lists them 'in order of increasing number of free parameters'; Discussion: 'they all span many decades roughly evenly and tend not be aligned with single parameters'), i.e. universality across models of differing parameter counts. (LLM, conf=0.85)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
43 more claims
Create a free account to see all 93 claims and to verify the 12 references still open.
- Probably not a substantial claim. 24
- It may state two separate claims as one. 22
- The quote only partly states this claim. 2
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“The topic of optimal design of dynamic experiments in biological systems is receiving increased attention [144–152].”
From Reverse engineering and identification in systems biology: strategies, perspectives and challenges (2014) · cites this paper as [144]
In the citing paper
“Previous approaches had taken a more local approach [25], [29], [30], [47] that relied on initial parameter guesses and often data”
From Maximizing the information content of experiments in systems biology (2013) · cites this paper as [30]
In the citing paper
“For example, Bandara et al. [144] showed how two cycles of optimization and experimentation were enough to increase parameter identifiability very significantly.”
From Reverse engineering and identification in systems biology: strategies, perspectives and challenges (2014) · cites this paper as [144]
Citations
Cites
SNOPT: An SQP Algorithm for Large-Scale Constrained Optimization
Gill, P. E., Murray, W., & Saunders, M. A. (2005) · SIAM Review · doi:10.1137/s0036144504446096 · Cited by 2,834
Computational systems biology.
Kitano, H. (2002) · Nature · doi:10.1038/nature01254 · Cited by 2,209
Universally Sloppy Parameter Sensitivities in Systems Biology Models
Gutenkunst, R. N., Waterfall, J. J., Casey, F. P., Brown, K. S., Myers, C. R., & Sethna, J. P. (2007) · PLoS Computational Biology · doi:10.1371/journal.pcbi.0030189 · Cited by 1,261 · cited 5×
“In their study, Gutenkunst et al. used published intuitively designed experimental protocols to characterize parameter identifiability in different models [6].”
BackgroundMethodInfluentialOscillations in NF-κB Signaling Control the Dynamics of Gene Expression
Nelson, D. E., Ihekwaba, A. E. C., Elliott, M., Johnson, J. R., Gibney, C. A., Foreman, B. E., Nelson, G., See, V., Horton, C. A., Spiller, D. G., et al. (2004) · Science · doi:10.1126/science.1099962 · Cited by 1,186 · cited 1×
“Since strength, timing, and combination of inputs typically determine the output behavior in a non-linear manner, dynamic models of such signaling networks in the form of differential equations are becoming widely used to complement experimental studies [1,2].”
An integrative genomic and proteomic analysis of pik3ca, pten, and akt mutations in breast cancer.
Stemke-Hale, K., Gonzalez-Angulo, A. M., Lluch, A., Neve, R. M., Kuo, W.-L., Davies, M., Carey, M., Hu, Z., Guan, Y., Sahin, A., et al. (2008) · Cancer Research · doi:10.1158/0008-5472.can-07-6854 · Cited by 1,054 · cited 1×
“Finally, w reflects the activity of phosphatases like phosphatase and tensin homolog (PTEN) which is lost in many cancers [26].”
BackgroundScientific Computing: An Introductory Survey, Revised Second Edition
Heath, M. T. (2018) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9781611975581 · Cited by 782
Characterization of the FKBP·Rapamycin·FRB Ternary Complex
Banaszynski, L. A., Liu, C. W., & Wandless, T. J. (2005) · Journal of the American Chemical Society · doi:10.1021/ja043277y · Cited by 609 · cited 1×
“This simplified model assumes that the formation of the complex of CF-p85, iRap, and Lyn-FRB begins with the binding of iRap to CF-p85 [19], that both this reaction and the subsequent binding of Lyn-FRB are irreversible, and that Y-PH does not shield PIP 3 from degradation.”
BackgroundRapid Chemically Induced Changes of PtdIns(4,5)P 2 Gate KCNQ Ion Channels
Suh, B.-C., Inoue, T., Meyer, T., & Hille, B. (2006) · Science · doi:10.1126/science.1131163 · Cited by 505 · cited 2×
“CF-p85 is described in [17], and Lyn-FRB (as Lyn 11 { linker { FRB ) in [27].”
BackgroundSelf-citationInterlinked Fast and Slow Positive Feedback Loops Drive Reliable Cell Decisions
Brandman, O., Ferrell, J. E., Li, R., & Meyer, T. (2005) · Science · doi:10.1126/science.1113834 · Cited by 466 · cited 1×
“For example, signaling pathways often translate graded stimuli into all-or-none responses [22], such that downstream observations become insensitive to upstream kinetic parameters.”
BackgroundSelf-citationADIFOR–Generating Derivative Codes from Fortran Programs
Bischof, C., Carle, A., Corliss, G., Griewank, A., & Hovland, P. (1992) · Scientific Programming · doi:10.1155/1992/717832 · Cited by 436
Cited by
Structural identifiability of systems biology models: a critical comparison of methods
Chis, O.-T., Banga, J. R., & Balsa-Canto, E. (2011) · PLoS ONE · doi:10.1371/journal.pone.0027755 · Cited by 448
“In this regard, recent works suggest the use of model based (optimal) experimental design to iteratively improve the quality of parameter estimates [10–13].”
MethodClassic and contemporary approaches to modeling biochemical reactions
Chen, W. W., Niepel, M., & Sorger, P. K. (2010) · Genes & Development · doi:10.1101/gad.1945410 · Cited by 286
“Nonidentifiability arising from experimental errors and model structures interact in real experiments to determine the overall precision of estimation: The lower the experimental error, the greater our ability to distinguish small differences in the value of the objective function (Bandara et al. 2009).”
BackgroundStructural Identifiability of Dynamic Systems Biology Models
Villaverde, A. F., Barreiro, A., & Papachristodoulou, A. (2016) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1005153 · Cited by 272
Reverse engineering and identification in systems biology: strategies, perspectives and challenges
Villaverde, A. F., & Banga, J. R. (2014) · Journal of The Royal Society Interface · doi:10.1098/rsif.2013.0505 · Cited by 229
“[144] showed how two cycles of optimization and experimentation were enough to increase parameter identifiability very significantly.”
BackgroundRobust and efficient parameter estimation in dynamic models of biological systems
Gábor, A., & Banga, J. R. (2015) · BMC Systems Biology · doi:10.1186/s12918-015-0219-2 · Cited by 175
“Further, dynamic model-based approaches have also been used to identify possible ways of intervention or (re-)design, such as in optimal experimental design [17–21], metabolic engineering [22] and synthetic biology [23, 24].”
Method
Metadata
- Authors
- Bandara, S., Schlöder, J. P., Eils, R., Bock, H. G., & Meyer, T.
- Affiliations
- Stanford University (US)
- Heidelberg University (DE)
- German Cancer Research Center (DE)
- Corresponding author
- Bandara, Samuel
- Meyer, Tobias
- Venue
- PLoS Computational Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2009-11-06
- Volume
- 5
- Issue
- 11
- Pages
- e1000558
- ISSN
- 1553-734X (ISSN-L)
- 1553-7358
- PMC
- PMC2775273
- OpenAlex
- W1979892077
- Semantic Scholar
- e8a71562fd515b200d22ce0ceab058be500c70da
- Cited by
- 135
- References
- 34
- Citations by year
- 2026: 1 · 2025: 5 · 2024: 5 · 2023: 6 · 2022: 5 · 2021: 5 · 2020: 5 · 2019: 7 · 2018: 12 · 2017: 9 · 2016: 8 · 2015: 9 · 2014: 10 · 2013: 14 · 2012: 14
- FWCI
- 4.57
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS Computational Biology (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Funders
- Microsoft Research
- National Institutes of Health (MH064801)
- National Institute of Mental Health (R01MH064801, MH 064801)
- National Institute of General Medical Sciences (R01GM048113, R01GM063702, R01GM030179, GM063702)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene Regulatory Network Analysis
- Social Sciences › Decision Sciences › Management Science and Operations Research › Optimal Experimental Design Methods
- Physical Sciences › Engineering › Control and Systems Engineering › Control Systems and Identification
- Keywords
- optimal experimental design, parameter estimation, signal transduction, PIP3 signaling, phosphatidylinositol 3-kinase, Akt, pleckstrin homology domain, live-cell fluorescence microscopy, differential equation model, kinase inhibitors, parameter uncertainty
- MeSH terms
- Algorithms, Animals, Cell Communication, Kinetics, Research Design, Protein Structure, Tertiary, Microscopy, Confocal, Phosphatidylinositol 3-Kinases, NIH 3T3 Cells, Systems Biology, Mice
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Optimal Experimental Design for Parameter Estimation of a Cell Signaling Model. https://app.cochat.ai/papers/10.1371/journal.pcbi.1000558
- The paper
- Bandara, S., Schlöder, J. P., Eils, R., Bock, H. G., & Meyer, T. (2009). Optimal Experimental Design for Parameter Estimation of a Cell Signaling Model. PLoS Computational Biology. https://doi.org/10.1371/journal.pcbi.1000558
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.