Universally Sloppy Parameter Sensitivities in Systems Biology Models
Open accessThe DOI matches its Crossref record; 44 of the 44 references it cites were found in a registry or an index; 24 of the 45 sources its claims cite were checked for support.
Abstract
Quantitative computational models play an increasingly important role in modern biology. Such models typically involve many free parameters, and assigning their values is often a substantial obstacle to model development. Directly measuring in vivo biochemical parameters is difficult, and collectively fitting them to other experimental data often yields large parameter uncertainties. Nevertheless, in earlier work we showed in a growth-factor-signaling model that collective fitting could yield well-constrained predictions, even when it left individual parameters very poorly constrained. We also showed that the model had a "sloppy" spectrum of parameter sensitivities, with eigenvalues roughly evenly distributed over many decades. Here we use a collection of models from the literature to test whether such sloppy spectra are common in systems biology. Strikingly, we find that every model we examine has a sloppy spectrum of sensitivities. We also test several consequences of this sloppiness for building predictive models. In particular, sloppiness suggests that collective fits to even large amounts of ideal time-series data will often leave many parameters poorly constrained. Tests over our model collection are consistent with this suggestion. This difficulty with collective fits may seem to argue for direct parameter measurements, but sloppiness also implies that such measurements must be formidably precise and complete to usefully constrain many model predictions. We confirm this implication in our growth-factor-signaling model. Our results suggest that sloppy sensitivity spectra are universal in systems biology models. The prevalence of sloppiness highlights the power of collective fits and suggests that modelers should focus on predictions rather than on parameters.
Overview
CoChat Summary
TL;DR
Across 17 systems biology models, every model examined had a “sloppy” parameter sensitivity spectrum, with eigenvalues spanning many decades roughly evenly. Sloppiness helps explain why collective fits leave individual parameters poorly constrained yet can yield tight predictions, while direct parameter measurements must be both precise and complete to constrain predictions.
Key findings
- All 17 models exhibited sloppy sensitivity spectra; all but one spanned more than 10^6 in eigenvalues, and few principal axes aligned with bare parameter axes (I/P ≈ 1 was rare).
- In idealized collective fits using 100 times as many time-series data points as parameters, each with 10% uncertainty, most models left many parameters with greater than 100% uncertainty (Σ > 1).
- In the 48-parameter Brown et al. growth-factor-signaling model, collective fitting to 14 experiments and 68 data points left all 48 parameters with 95% confidence intervals spanning more than a factor of 50, yet an ERK prediction under EGF stimulation with PI3K inhibition was tight.
- To match that prediction with direct measurements, all 24 involved parameters needed measurement within ±25%; leaving one of about 18 of the 24 involved parameters unmeasured, even bounded to a 1,000-fold range, produced a prediction with very large uncertainty.
Methods
- Collected 17 ordinary-differential-equation systems biology models from the literature and BioModels, ranging from about ten to more than 200 parameters, and quantified behavior change by average squared change in molecular species time courses.
- Analyzed the Hessian matrix Hχ2 with derivatives taken with respect to log parameters, using sensitivity equations rather than finite differences; computations used SloppyCell version 0.81.
- For parameter uncertainties, rescaled Hχ2 for Nd data points with fractional uncertainty f and computed relative 95% confidence interval size Σ; for prediction uncertainties in the Brown et al. model, generated 1,000 parameter sets consistent with stated uncertainties and calculated ERK time courses.
Limitations
- Stated: the paper focuses on parameter uncertainties, not model-structure uncertainty; sloppiness depends on the model and the data fit, and exhaustive experiments designed to decouple parameters would not yield sloppy sensitivities.
- Stated: successful fits and predictions may arise from models that are incorrect in significant ways, and potential model-refinement methods demand a complete understanding unavailable for many biological systems.
- Evident: most collected models were not systematically fit or measured in their original publications, and the sensitivity spectra rely on a local quadratic Hessian approximation, though Monte Carlo analysis in the Brown et al. model supports the nonlinear χ2 picture.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures
![Figure 1 Parameter Sensitivity Spectra The quantities we calculate from H χ2 are illustrated in (A), while (B) and (C) show that all the models we examined have sloppy sensitivity spectra. (A) Analyzing H χ2 corresponds to approximating the surfaces of constant model behavior change (constant χ 2) as ellipsoids. The width of each principal axis is proportional to one over the square root of the corresponding eigenvalue. The inner ellipsoid's projection onto and intersection with the θ 1 axis are denoted by P 1 and I1, respectively. (B) Plotted are the eigenvalue spectra of H χ2 for our collection of systems biology models. The many decades generally spanned indicate that the ellipses have a very large aspect ratio. (The spectra have each been normalized by their largest eigenvalue. Not all values are visible for all models.) (C) Plotted is the spectrum of I / P for each parameter in each model in our collection. Generally very few parameters have I / P ≈ 1, suggesting that the ellipses are skewed from the bare parameter axes. (Not all values are visible for all models.) The models are plotted in order of increasing number of free parameters and are: (a) eukaryotic cell cycle [28], (b) Xenopus egg cell cycle [29], (c) eukaryotic mitosis [30], (d) generic circadian rhythm [31], (e) nicotinic acetylcholine intra-receptor dynamics [32], (f) generic kinase cascade [33], (g) Xenopus Wnt signaling [34], (h) Drosophila circadian rhythm [35], (i) rat growth-factor signaling [21], (j) Drosophila segment polarity [36], (k) Drosophila circadian rhythm [37], (l) Arabidopsis circadian rhythm [2], (m) in silico regulatory network [25], (n) human purine metabolism [38], (o) Escherichia coli carbon metabolism [39], (p) budding yeast cell cycle [40], (q) rat growth-factor signaling [41].](https://figures.cochat.ai/paper-figures/e59ef10af9fc4bcdbd3fb192663d91bb/universally-sloppy-parameter-sensitivities-in-systems-biology-models-figure-1.jpg)


![Figure 4 Parameter and Prediction Uncertainties (A) Our example prediction is for ERK activity upon EGF stimulation given PI3K inhibition in this 48-parameter model of growth-factor signaling in PC12 cells [21]. (B) Shaded regions are 95% confidence intervals calculated via exhaustive Monte Carlo for our example prediction given various scenarios for constraining parameter values. (C) Plotted is the relative size Σ of the 95% confidence interval for each parameter. The scenarios represented are: (red, squares) all model parameters individually measured to high precision, (blue, triangles) all parameters precisely measured, except one estimated to low precision, (yellow, circles) all parameters collectively fit to 14 real cell-biology experiments. Precisely measured individual parameter values enable a tight prediction, (B) middle red band; but even one poorly known parameter can destroy predictive power, (B) wide blue band. In contrast, the collective fit yields a tight prediction, (B) tightest yellow band; but only very loose parameter constraints, (C) circles. The large parameter uncertainties from the collective fit, (C) circles, calculated here by Monte Carlo are qualitatively similar to those seen in the linearized fit to idealized data (Figure 3, model (i)). (For clarity, the dashed red lines trace the boundary of the red confidence interval.)](https://figures.cochat.ai/paper-figures/a84b91b2f345405fa25712988f5e8df2/universally-sloppy-parameter-sensitivities-in-systems-biology-models-figure-4.jpg)
Verification
Claims the paper makes
20 Valid · 3 Not valid · 7 Not checked · 41 Own results and interpretations
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“but even one poorly known parameter can destroy predictive power, (B) wide blue band.”
- Also stated in: § Predictions from Direct Parameter Measurements · hedged
“The resulting prediction (blue in Figure 4B) has very large uncertainty and would likely be useless.”
§ Universally Sloppy Parameter Sensitivities in Systems Biology Models > Results > Predictions from Direct Parameter Measurements
What the sources say
- Disputedcites [21] 2004, The statistical mechanics of complex signaling networks: Nerve growth factor signaling., Phys Biol sourceExperimental — accuracy not yet measured
The paper makes the contrary point for its PC12 model: Section 3.6 (page unavailable) says it extracts precise predictions even though each rate constant varies over many orders of magnitude. Section 3.5 explains that uncertainty in individual parameters need not imply uncertainty in the combinations that determine predictions. (LLM, conf=0.85)
Checked by a reasoning model — may be wrong.
plainstates a causeSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Predictions from Direct Parameter Measurements · hedged
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“Precisely measured individual parameter values enable a tight prediction, (B) middle red band”
§ Universally Sloppy Parameter Sensitivities in Systems Biology Models > Results > Predictions from Direct Parameter Measurements
What the sources say
- Disputedcites [21] 2004, The statistical mechanics of complex signaling networks: Nerve growth factor signaling., Phys Biol sourceExperimental — accuracy not yet measured
Split into 2 atoms — 0 confirmed, 1 partially supported, 1 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
Checked by a reasoning model — may be wrong.
plainstates a causeSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
Every source the paper cites for this claim was checked and supports it.
In the paper
“(1) Fundamentally, sloppiness involves an extraordinarily singular coordinate transformation in parameter space between the bare parameters natural in biology (e.g., binding affinities and rate constants) and the eigenparameters controlling system behavior, as discussed in [23].”
§ Universally Sloppy Parameter Sensitivities in Systems Biology Models > Discussion
What the sources say
- Supportedcites [23] 2006, Sloppy-model universality class and the Vandermonde matrix., Phys Rev Lett source
The conclusion (p. 4) says that sloppiness arises because parameters natural for experimental manipulation or description are often a severe distortion of the basis natural for describing system behavior. The discussion on p. 2 identifies biochemical rate constants as biologically natural parameters and describes the transformation to unsloppy parameters as nonlinear and often not single-valued. (LLM, conf=0.92)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“They collectively fit their model's 48 parameters to 68 data points from 14 cell-biology experiments (mostly Western blots).”
§ Universally Sloppy Parameter Sensitivities in Systems Biology Models > Introduction
What the sources say
- Partly supportedcites [21] 2004, The statistical mechanics of complex signaling networks: Nerve growth factor signaling., Phys Biol sourceExperimental — accuracy not yet measured
Section 3.3 reports fitting 59 parameters to 68 data points; the model has 48 rate-constant parameters, but the fit also included additional parameters. Section 3.2 says the data comprised fourteen time series from seven experiments, not fourteen experiments. The cited passages do not establish that the fitting data were mostly Western blots. (LLM, conf=0.91)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“To usefully constrain Brown et al.'s model, direct parameter measurements would need to be both precise and complete.”
§ Universally Sloppy Parameter Sensitivities in Systems Biology Models > Results > Predictions from Direct Parameter Measurements
What the sources say
- Partly supportedcites [21] 2004, The statistical mechanics of complex signaling networks: Nerve growth factor signaling., Phys Biol sourceExperimental — accuracy not yet measured
The Introduction (page unavailable) says many model parameters have not been measured experimentally and that in-vitro measurements may differ substantially from values in cells. Section 3.2 (page unavailable) says parameter measurements can be incorporated into the model, but it does not say they must be both precise and complete to constrain it usefully; the paper also describes making predictions from time-series data despite uncertain individual parameters. (LLM, conf=0.76)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
37 more claims · 12 with their cited sources checked
Create a free account to see all 71 claims and every source verdict.
- It may state two separate claims as one. 18
- Probably not a substantial claim. 11
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“In their study, Gutenkunst et al. used published intuitively designed experimental protocols to characterize parameter identifiability in different models [6].”
From Optimal Experimental Design for Parameter Estimation of a Cell Signaling Model (2009) · cites this paper as [6]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This uncertainty, it has turned out, can play a pivotal role in the analysis of a system’s dynamics16–18.”
From A framework for parameter estimation and model selection from experimental data in systems biology using approximate Bayesian computation (2014) · cites this paper as [16]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Recently we have shown that the collective behavior of multiparameter models is most often sloppy : insensitive to changes except along a few 'stiff' combinations of parameters, with an enormous sloppy neutral subspace.”
From Sloppiness, robustness, and evolvability in systems biology (2008) · cites this paper as [32]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Strong covariance between parameters of reaction cascades, fast reversible reactions, and binding events makes it difficult to constrain single parameter values [3],[6],[10],[23].”
From Optimal Experimental Design for Parameter Estimation of a Cell Signaling Model (2009) · cites this paper as [6]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Moreover, the structure of such models was suggested to reflect natural robustness to changes in many parameter values [3],[6].”
From Optimal Experimental Design for Parameter Estimation of a Cell Signaling Model (2009) · cites this paper as [6]
Experimental — accuracy not yet measured
28 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Hucka, M., Finney, A., Sauro, H. M., Bolouri, H., Doyle, J. C., Kitano, H., Arkin, A. P., Bornstein, B. J., Bray, D., Cornish-Bowden, A., et al. (2003) · Bioinformatics · doi:10.1093/bioinformatics/btg015 · Cited by 3,060 · cited 7×
“SBML files and SloppyCell scripts to reproduce all presented calculations are in Dataset S1.”
BackgroundMethodInfluentialComputational systems biology.
Kitano, H. (2002) · Nature · doi:10.1038/nature01254 · Cited by 2,209
Causal Protein-Signaling Networks Derived from Multiparameter Single-Cell Data
Sachs, K., Perez, O., Pe'er, D., Lauffenburger, D. A., & Nolan, G. P. (2005) · Science · doi:10.1126/science.1105809 · Cited by 1,708 · cited 2×
“[7] Sachs K, Perez O, Pe’er D, Lauffenburger DA, Nolan GP”
BackgroundMinton, A. P. (2001) · Journal of Biological Chemistry · doi:10.1074/jbc.r100005200 · Cited by 1,474 · cited 2×
“[9] Minton AP (2001) The influence of macromolecular”
BackgroundMethodUnderstanding the control of metabolism
(1997) · Choice Reviews Online · doi:10.5860/choice.34-6252 · Cited by 1,231
The segment polarity network is a robust developmental module
von Dassow, G., Meir, E., Munro, E. M., & Odell, G. M. (2000) · Nature · doi:10.1038/35018085 · Cited by 1,144
Le Novere, N. (2006) · Nucleic Acids Research · doi:10.1093/nar/gkj092 · Cited by 762 · cited 3×
“Models discussed that are in the BioModels database [42] are: (a) BIOMD0000000005, (c) BIOMD0000000003, (d) BIOMD0000000035, (e) BIOMD0000000002, (f) BIOMD0000000010, (h) BIOMD0000000021, (i) BIOMD0000000033, (k) BIOMD0000000022, (l) BIOMD0000000055, (n) BIOMD0000000015, (o) BIOMD0000000051, (p) BIOMD0000000056, (q) B…”
BackgroundMethodThe Roles of APC and Axin Derived from Experimental and Theoretical Analysis of the Wnt Pathway
Lee, E., Salic, A., Krüger, R., Heinrich, R., & Kirschner, M. W. (2003) · PLoS Biology · doi:10.1371/journal.pbio.0000010 · Cited by 708
Teusink, B., Passarge, J., Reijenga, C. A., Esgalhado, E., van der Weijden, C. C., Schepper, M., Walsh, M. C., Bakker, B. M., van Dam, K., Westerhoff, H. V., et al. (2000) · European Journal of Biochemistry · doi:10.1046/j.1432-1327.2000.01527.x · Cited by 694 · cited 2×
“[10] Teusink B, Passarge J, Reijenga CA, Esgalhado E, van der Weijden CC, et al.”
BackgroundMethodMendes, P., & Kell, D. (1998) · Bioinformatics · doi:10.1093/bioinformatics/14.10.869 · Cited by 676 · cited 2×
“On the other hand, collectively fitting parameters [11, 12] by optimizing the agreement between the model and available data often yields large parameter uncertainties [13, 14, 15].”
Background
Research works printed but not linked: 1
- MehtaML 2004 Random Matrices. 3rd edition Amsterdam Academic Press 706
Cited by
Guidelines for the use and interpretation of assays for monitoring autophagy (4th edition) 1
Klionsky, D. J., Abdel-Aziz, A. K., Abdelfatah, S., Abdellatif, M., Abdoli, A., Abel, S., Abeliovich, H., Abildgaard, M. H., Abudu, Y. P., Acevedo-Arozena, A., et al. (2021) · Autophagy · doi:10.1080/15548627.2020.1797280 · Cited by 2,897
Absolute metabolite concentrations and implied enzyme active site occupancy in Escherichia coli
Bennett, B. D., Kimball, E. H., Gao, M., Osterhout, R., Van Dien, S. J., & Rabinowitz, J. D. (2009) · Nature Chemical Biology · doi:10.1038/nchembio.186 · Cited by 1,830
“This facilitates otherwise intractable parameter identification challenges in building dynamic Michaelis-Menten models of metabolis”
BackgroundToni, T., Welch, D., Strelkowa, N., Ipsen, A., & Stumpf, M. P. H. (2009) · Journal of The Royal Society Interface · doi:10.1098/rsif.2008.0172 · Cited by 1,498
“In other words, the smaller PCs correspond to stiff parameter combinations, while the larger PCs may correspond to sloppy parameter combinations (Gutenkunst et al. , 2007).”
BackgroundRiemann manifold langevin and hamiltonian monte carlo methods
Girolami, M., & Calderhead, B. (2011) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2010.00765.x · Cited by 1,473
Ulissi, Z. W., Medford, A. J., Bligaard, T., & Nørskov, J. K. (2017) · Nature Communications · doi:10.1038/ncomms14621 · Cited by 614
Zhang, X., Trame, M., Lesko, L., & Schmidt, S. (2015) · CPT: Pharmacometrics & Systems Pharmacology · doi:10.1002/psp4.6 · Cited by 467
Structural identifiability of systems biology models: a critical comparison of methods
Chis, O.-T., Banga, J. R., & Balsa-Canto, E. (2011) · PLoS ONE · doi:10.1371/journal.pone.0027755 · Cited by 448
“This has been in fact the case in many examples found in the literature [4–8].”
ResultMechanistic links between cellular trade-offs, gene expression, and growth
Weiße, A. Y., Oyarzún, D. A., Danos, V., & Swain, P. S. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1416533112 · Cited by 403
Chen, W. W., Schoeberl, B., Jasper, P. J., Niepel, M., Nielsen, U. B., Lauffenburger, D. A., & Sorger, P. K. (2009) · Molecular Systems Biology · doi:10.1038/msb.2008.74 · Cited by 391
“Why then, do we not simply double or triple the amount of data used for model training? Unfortunately, it has been shown that even a complete set of time-resolved measurements of all species in a complex reaction model is insufficient to fully constrain all parameters due to the ‘sloppy’ nature of the fitting procedure”
BackgroundChou, I.-C., & Voit, E. O. (2009) · Mathematical Biosciences · doi:10.1016/j.mbs.2009.03.002 · Cited by 390
“some articles [130,132,133], but will require more work.”
BackgroundSensitivity analysis approaches applied to systems biology models
Zi, Z. (2011) · IET Systems Biology · doi:10.1049/iet-syb.2011.0015 · Cited by 384
“They found that systems biology models exhibit sloppy sensitivity spectra because the behaviour of the models is very insensitive to many parameter changes [72].”
BackgroundSystems biology informed deep learning for inferring parameters and hidden dynamics
Yazdani, A., Lu, L., Raissi, M., & Karniadakis, G. E. (2020) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1007575 · Cited by 329
Pervasive robustness in biological systems
Félix, M.-A., & Barkoulas, M. (2015) · Nature Reviews Genetics · doi:10.1038/nrg3949 · Cited by 325
Predicting perturbation patterns from the topology of biological networks
Santolini, M., & Barabási, A.-L. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1720589115 · Cited by 323
“Indeed, only a small subset of these model parameters affect the overall dynamics, a property known as “sloppiness” (25).”
BackgroundParameter estimation and model selection in computational biology
Lillacci, G., & Khammash, M. (2010) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000696 · Cited by 325
“Such ‘‘sloppiness’’ has been suggested as a factor that makes parameter estimation difficult [28].”
BackgroundSystems biology: parameter estimation for biochemical models
Ashyraliyev, M., Fomekong‐Nanfack, Y., Kaandorp, J. A., & Blom, J. G. (2009) · The FEBS Journal · doi:10.1111/j.1742-4658.2008.06844.x · Cited by 323
Perspective: Sloppiness and emergent theories in physics, biology, and beyond
Transtrum, M. K., Machta, B. B., Brown, K. S., Daniels, B. C., Myers, C. R., & Sethna, J. P. (2015) · The Journal of Chemical Physics · doi:10.1063/1.4923066 · Cited by 311
Self-citationSynthetic in vitro transcriptional oscillators
Kim, J., & Winfree, E. (2011) · Molecular Systems Biology · doi:10.1038/msb.2010.119 · Cited by 313
“Synthetic in vitro transcriptional oscillators”
Kinetic models in industrial biotechnology – Improving cell factory performance
Almquist, J., Cvijovic, M., Hatzimanikatis, V., Nielsen, J., & Jirstrand, M. (2014) · Metabolic Engineering · doi:10.1016/j.ymben.2014.03.007 · Cited by 293
“…is the level of confidence in the obtained estimates and whether it is possible at all to uniquely assign values to the parameters (Cedersund, 2006; Gutenkunst et al., 2007b; Ashyraliyev et al., 2009; Roper et al., 2010; Raue et al., 2011; Erguler and Stumpf, 2011; Meshkat et al., 2011; Hattersley…”
BackgroundParameter Space Compression Underlies Emergent Theories and Predictive Models
Machta, B. B., Chachra, R., Transtrum, M. K., & Sethna, J. P. (2013) · Science · doi:10.1126/science.1238723 · Cited by 285
Self-citationClassic and contemporary approaches to modeling biochemical reactions
Chen, W. W., Niepel, M., & Sorger, P. K. (2010) · Genes & Development · doi:10.1101/gad.1945410 · Cited by 286
“Sethna and colleagues (Gutenkunst et al. 2007) describe such models as ‘‘sloppy,’’ insofar as most parameter values are very poorly determined.”
BackgroundAguilar, J., Zhang, T., Qian, F., Kingsbury, M., McInroe, B., Mazouchova, N., Li, C., Maladen, R., Gong, C., Travers, M., et al. (2016) · Reports on Progress in Physics · doi:10.1088/0034-4885/79/11/110001 · Cited by 273
Simulation-based model selection for dynamical systems in systems and population biology
Toni, T., & Stumpf, M. P. H. (2010) · Bioinformatics · doi:10.1093/bioinformatics/btp619 · Cited by 260
Bacterial adaptation through distributed sensing of metabolic fluxes
Kotte, O., Zaugg, J. B., & Heinemann, M. (2010) · Molecular Systems Biology · doi:10.1038/msb.2010.10 · Cited by 260
“The problem of many uncertain parameter values of kinetic models is mitigated by the fact that the structure of mechanistic systems biology models already tightly constrains the possible system behaviors, leaving most parameter values unimportant (Brown et al, 2004; Gutenkunst et al, 2007).”
BackgroundWeitz, J. S., Stock, C. A., Wilhelm, S. W., Bourouiba, L., Coleman, M. L., Buchan, A., Follows, M. J., Fuhrman, J. A., Jover, L. F., Lennon, J. T., et al. (2015) · The ISME Journal · doi:10.1038/ismej.2014.220 · Cited by 256
“Moving from testing predictions to testing parameters (and functional interactions) is an important challenge in reconciling complex models with data (Gutenkunst et al., 2007).”
BackgroundResultTraining deep neural density estimators to identify mechanistic models of neural dynamics
Gonçalves, P. J., Lueckmann, J.-M., Deistler, M., Nonnenmacher, M., Öcal, K., Bassetto, G., Chintaluri, C., Podlaski, W. F., Haddad, S. A., Vogels, T. P., et al. (2020) · eLife · doi:10.7554/elife.56261 · Cited by 239
Models in biology: ‘accurate descriptions of our pathetic thinking’
Gunawardena, J. (2014) · BMC Biology · doi:10.1186/1741-7007-12-29 · Cited by 243
“Fitting only constrains some parameters; a good rule of thumb is that 20% of the parameters are well constrained by fitting, while 80% are not [38].”
Reverse engineering and identification in systems biology: strategies, perspectives and challenges
Villaverde, A. F., & Banga, J. R. (2014) · Journal of The Royal Society Interface · doi:10.1098/rsif.2013.0505 · Cited by 229
“[207] is a valuable contribution which nicely illustrates the difficulties that plague parameter estimation problems in dynamic models, we believe that:”
BackgroundInfluentialKohl, P., Crampin, E. J., Quinn, T. A., & Noble, D. (2010) · Clinical Pharmacology & Therapeutics · doi:10.1038/clpt.2010.92 · Cited by 231
Diversity in the dynamical behaviour of a compartmentalized programmable biochemical oscillator
Weitz, M., Kim, J., Kapsner, K., Winfree, E., Franco, E., & Simmel, F. C. (2014) · Nature Chemistry · doi:10.1038/nchem.1869 · Cited by 229
Systems biology primer: the basic methods and approaches
Tavassoly, I., Goldfarb, J., & Iyengar, R. (2018) · Essays in Biochemistry · doi:10.1042/ebc20180003 · Cited by 218
Liepe, J., Kirk, P., Filippi, S., Toni, T., Barnes, C. P., & Stumpf, M. P. H. (2014) · Nature Protocols · doi:10.1038/nprot.2014.025 · Cited by 217
“In most biological systems, however, only a few reaction rates can be inferred given an observed data set, and different parameter vectors can explain the observed data (almost) equally well 16,18,63 .”
BackgroundFlexible multitask computation in recurrent networks utilizes shared dynamical motifs
Driscoll, L., Shenoy, K., & Sussillo, D. (2022) · doi:10.1101/2022.08.15.503870 · Cited by 217
Understanding Polymorphism in Organic Semiconductor Thin Films through Nanoconfinement
Diao, Y., Lenn, K. M., Lee, W.-Y., Blood-Forsythe, M. A., Xu, J., Mao, Y., Kim, Y., Reinspach, J. A., Park, S., Aspuru-Guzik, A., et al. (2014) · Journal of the American Chemical Society · doi:10.1021/ja507179d · Cited by 210
Geometry of nonlinear least squares with applications to sloppy models and optimization
Transtrum, M. K., Machta, B. B., & Sethna, J. P. (2011) · Physical Review E · doi:10.1103/physreve.83.036701 · Cited by 201
Self-citationSloppiness, robustness, and evolvability in systems biology
Daniels, B. C., Chen, Y.-J., Sethna, J. P., Gutenkunst, R. N., & Myers, C. R. (2008) · Current Opinion in Biotechnology · doi:10.1016/j.copbio.2008.06.008 · Cited by 192
“[31] and [32] generate ensembles of parameters (chemotypes) consistent with a given dynatype for an EGF/NGF signaling pathway in PC12 cells, where the dynatype is constrained to fit available experimental data.”
BackgroundMethodInfluentialSelf-citationAngelikopoulos, P., Papadimitriou, C., & Koumoutsakos, P. (2012) · The Journal of Chemical Physics · doi:10.1063/1.4757266 · Cited by 198
Automated adaptive inference of phenomenological dynamical models
Daniels, B. C., & Nemenman, I. (2015) · Nature Communications · doi:10.1038/ncomms9133 · Cited by 194
“Further, specification of myriads of microscopic mechanistic parameters in such models demands vast data sets and computational resources, and is hard even for very large data sets due to widely varying sensitivities of predictions to the parameters 3 .”
Statistical inference of the time-varying structure of gene-regulation networks
Lèbre, S., Becq, J., Devaux, F., Stumpf, M. P., & Lelandais, G. (2010) · BMC Systems Biology · doi:10.1186/1752-0509-4-130 · Cited by 187
Interrogating the topological robustness of gene regulatory circuits by randomization
Huang, B., Lu, M., Jia, D., Ben-Jacob, E., Levine, H., & Onuchic, J. N. (2017) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1005456 · Cited by 186
150 Years of the Mass Action Law
Voit, E. O., Martens, H. A., & Omholt, S. W. (2015) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1004012 · Cited by 180
“This type of metamodeling also assists with the assessment of ‘‘neutral’’ or ‘‘sloppy’’ parameter sets, which all offer similarly good data fits [60,61].”
Allen, R., Rieger, T., & Musante, C. (2016) · CPT: Pharmacometrics & Systems Pharmacology · doi:10.1002/psp4.12063 · Cited by 179
Why are Nonlinear Fits to Data so Challenging?
Transtrum, M. K., Machta, B. B., & Sethna, J. P. (2010) · Physical Review Letters · doi:10.1103/physrevlett.104.060201 · Cited by 181
Self-citationMuszkiewicz, A., Britton, O. J., Gemmell, P., Passini, E., Sánchez, C., Zhou, X., Carusi, A., Quinn, T. A., Burrage, K., Bueno-Orovio, A., et al. (2016) · Progress in Biophysics and Molecular Biology · doi:10.1016/j.pbiomolbio.2015.12.002 · Cited by 178
“, 2012) showed the correspondence between gene expression data and a population of computer models of the mouse ventricular action potential, supporting the hypothesis that biological systems are generally ‘sloppy’ (Gutenkunst et al., 2007), i.”
BackgroundRobust and efficient parameter estimation in dynamic models of biological systems
Gábor, A., & Banga, J. R. (2015) · BMC Systems Biology · doi:10.1186/s12918-015-0219-2 · Cited by 175
“This eigenvalue decomposition has been widely used in the estimation literature [72, 98–102].”
MethodMetabolomics-driven quantitative analysis of ammonia assimilation in E. coli
Yuan, J., Doucette, C. D., Fowler, W. U., Feng, X., Piazza, M., Rabitz, H. A., Wingreen, N. S., & Rabinowitz, J. D. (2009) · Molecular Systems Biology · doi:10.1038/msb.2009.60 · Cited by 178
“The dichotomy between diverse parameter values and tight simulation results in metabolic models has been described earlier (Gutenkunst et al, 2007; Piazza et al, 2008).”
Biochemical Systems Theory: A Review
Voit, E. O. (2013) · ISRN Biomathematics · doi:10.1155/2013/897658 · Cited by 173
“While essentially all estimation methods described above had as the ultimate target “the one optimal solution,” several groups in recent years emphasized that biological systems models are not necessarily completely identi�able from the available data [342], but “sloppy” [343], and that one might be better advised to…”
BackgroundAdvances and selected recent developments in state and parameter estimation
Kravaris, C., Hahn, J., & Chu, Y. (2013) · Computers & Chemical Engineering · doi:10.1016/j.compchemeng.2012.06.001 · Cited by 177
“These limitations regarding the availability and quality of the data pose further challenges to the estimation problem since the optimal solution of the parameter values can be sensitive to variations in the data (Gutenkunst et al., 2007).”
Heathcote, A., Lin, Y.-S., Reynolds, A., Strickland, L., Gretton, M., & Matzke, D. (2019) · Behavior Research Methods · doi:10.3758/s13428-018-1067-y · Cited by 168
“As a result, such models are B sloppy ^ — a pervasive phenomenon in models of biological systems (Gutenkunst et al., 2007) — meaning that their parameters can be highly correlated.”
BackgroundComputational neuroimaging strategies for single patient predictions
Stephan, K. E., Schlagenhauf, F., Huys, Q. J. M., Raman, S., Aponte, E. A., Brodersen, K. H., Rigoux, L., Moran, R. J., Daunizeau, J., Dolan, R. J., et al. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2016.06.038 · Cited by 166
“systems, parameters often show strong interdependencies (Gutenkunst et al., 2007), leading to potential identifiability problems”
InfluentialMaximizing the information content of experiments in systems biology
Liepe, J., Filippi, S., Komorowski, M., & Stumpf, M. P. H. (2013) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002888 · Cited by 170
“Unfortunately obtaining reliable parameter estimates for dynamical systems is plagued with difficulties [16,17].”
BackgroundMethodSensitivity analysis methods in the biomedical sciences
Qian, G., & Mahdi, A. (2020) · Mathematical Biosciences · doi:10.1016/j.mbs.2020.108306 · Cited by 169
“It has been observed that although high-throughput methods are well-suited for discovering interact ions, they remain of limited use for the measurement of biological and biochemical parameters [103, 49].”
BackgroundSaa, P. A., & Nielsen, L. K. (2017) · Biotechnology Advances · doi:10.1016/j.biotechadv.2017.09.005 · Cited by 166
“Indeed, scrutiny of most kinetic models has revealed that good predictions do not necessarily require precise parameters (Gutenkunst et al., 2007).”
BackgroundMethodInfluentialCardiac cell modelling: Observations from the heart of the cardiac physiome project
Fink, M., Niederer, S. A., Cherry, E. M., Fenton, F. H., Koivumäki, J. T., Seemann, G., Thul, R., Zhang, H., Sachse, F. B., Beard, D., et al. (2011) · Progress in Biophysics and Molecular Biology · doi:10.1016/j.pbiomolbio.2010.03.002 · Cited by 164
“We note, however, that in a recent analysis of parameter sensitivity in models of a range of biological networks Gutenkunst et al. (2007) identified parameter 'sloppiness'—implying that despite being ill-constrained, the precise determination of at least some parameters may not be essential, while…”
ResultFlexible multitask computation in recurrent networks utilizes shared dynamical motifs
Driscoll, L. N., Shenoy, K., & Sussillo, D. (2024) · Nature Neuroscience · doi:10.1038/s41593-024-01668-6 · Cited by 214
Pokhilko, A., Hodge, S. K., Stratford, K., Knox, K., Edwards, K. D., Thomson, A. W., Mizuno, T., & Millar, A. J. (2010) · Molecular Systems Biology · doi:10.1038/msb.2010.69 · Cited by 159
“The final model's behaviour may be robust (insensitive) to variation in many parameter values, but this does not justify a post hoc deprecation of laborious parameter measurements”
Sloppy models, parameter uncertainty, and the role of experimental design
Apgar, J. F., Witmer, D. K., White, F. M., & Tidor, B. (2010) · Molecular BioSystems · doi:10.1039/b918098b · Cited by 146
Jagged–Delta asymmetry in Notch signaling can give rise to a Sender/Receiver hybrid phenotype
Boareto, M., Jolly, M. K., Lu, M., Onuchic, J. N., Clementi, C., & Ben‐Jacob, E. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1416287112 · Cited by 165
Multiscale models of angiogenesis
Qutub, A. A., Gabhann, F. M., Karagiannis, E. D., Vempati, P., & Popel, A. S. (2009) · IEEE Engineering in Medicine and Biology Magazine · doi:10.1109/memb.2009.931791 · Cited by 165
Addressing parameter identifiability by model-based experimentation
Raue, A., Kreutz, C., Maiwald, T., Klingmüller, U., & Timmer, J. (2011) · IET Systems Biology · doi:10.1049/iet-syb.2010.0061 · Cited by 145
“[43] RN Gutenkunst, JJ Waterfall, FP Casey, KS Brown, CR Myers, and JP Sethna.”
BackgroundMethodNutritional Systems Biology Modeling: From Molecular Mechanisms to Physiology
de Graaf, A. A., Freidig, A. P., de Roos, B., Jamshidi, N., Heinemann, M., Rullmann, J. A. C., Hall, K. D., Adiels, M., & van Ommen, B. (2009) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000554 · Cited by 158
“In fact, it was recently shown that complex system behavior is often largely defined by the model structure (i.e., the interaction topology) [32,33].”
Regression Analysis for Constraining Free Parameters in Electrophysiological Models of Cardiac Cells
Sarkar, A. X., & Sobie, E. A. (2010) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000914 · Cited by 146
“The premise underlying this strategy is therefore similar to ideas advanced by Sethna and colleagues in discussions of model ‘‘sloppiness’’ [25,26].”
ResultFröhlich, F., Kessler, T., Weindl, D., Shadrin, A., Schmiester, L., Hache, H., Muradyan, A., Schütte, M., Lim, J.-H., Heinig, M., et al. (2018) · Cell Systems · doi:10.1016/j.cels.2018.10.013 · Cited by 143
“The broad distribution of eigenvalues implies pronounced parameter sloppiness (Gutenkunst et al., 2007).”
BackgroundMathematical modeling of gene expression: a guide for the perplexed biologist
Ay, A. A., & Arnosti, D. N. (2011) · Critical Reviews in Biochemistry and Molecular Biology · doi:10.3109/10409238.2011.556597 · Cited by 159
“Such analysis has been applied to diverse models, including differential equation and thermodynamic models (Gutenkunst et al., 2007; Ingalls, 2008; Dresch et al., 2010).”
MethodA Noisy Paracrine Signal Determines the Cellular NF-κB Response to Lipopolysaccharide
Lee, T. K., Denny, E. M., Sanghvi, J. C., Gaston, J. E., Maynard, N. D., Hughey, J., & Covert, M. W. (2009) · Science Signaling · doi:10.1126/scisignal.2000599 · Cited by 141
“It should be noted that these parameter distributions do not necessarily represent biologically relevant kinetic rates, given the stochastic nature of reactions at the single-cell level (40) and loose parameter sensitivities in large models of biological systems (50).”
Complex systems dynamics in aging: new evidence, continuing questions
Cohen, A. A. (2015) · Biogerontology · doi:10.1007/s10522-015-9584-x · Cited by 148
“Precisely because the networks are complex dynamic systems, the consequences of missing or slightly erroneous information in network construction are difficult to infer and could be large (Gutenkunst et al. 2007).”
BackgroundRethinking Transcriptional Activation in the Arabidopsis Circadian Clock
Fogelmark, K., & Troein, C. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003705 · Cited by 136
Mechanistic simulations of inflammation: Current state and future prospects
Yoram Vodovotz, B., Constantine, G. M., Rubin, J. E., CSETE, M. E., Voit, E. O., & An, G. C. (2008) · Mathematical Biosciences · doi:10.1016/j.mbs.2008.07.013 · Cited by 142
“Fortuitously, there is a strong suggestion that this type of qualitative approximation is useful in capturing the essential dynamics of signal transduction cascades [55–57].”
Optimal Experimental Design for Parameter Estimation of a Cell Signaling Model
Bandara, S., Schlöder, J. P., Eils, R., Bock, H. G., & Meyer, T. (2009) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000558 · Cited by 135
“In their study, Gutenkunst et al. used published intuitively designed experimental protocols to characterize parameter identifiability in different models [6].”
BackgroundMethodInfluentialObservability and Structural Identifiability of Nonlinear Biological Systems
Villaverde, A. F. (2019) · Complexity · doi:10.1155/2019/8497093 · Cited by 176
“These reasons help explain why some modellers are reluctant to analyse these properties of their models [19], which might be understandable taking into account that even the need of determining parameter values has been questioned in the context of biological modelling [24].”
BackgroundModel Reduction by Manifold Boundaries
Transtrum, M. K., & Qiu, P. (2014) · Physical Review Letters · doi:10.1103/physrevlett.113.098701 · Cited by 166
Systems biology of microbial metabolism
Heinemann, M., & Sauer, U. (2010) · Current Opinion in Microbiology · doi:10.1016/j.mib.2010.02.005 · Cited by 152
Ensemble Control of Finite-Dimensional Time-Varying Linear Systems
Li, J.-S. (2010) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2010.2060259 · Cited by 124
“parameters, such as kinetic constants, which are unknown or only weakly constrained by existing experimental knowledge [ 29 ].”
Statistical Design of Experiments for Synthetic Biology
Gilman, J., Walls, L. E., Bandiera, L., & Menolascina, F. (2021) · ACS Synthetic Biology · doi:10.1021/acssynbio.0c00385 · Cited by 129
Cell–cell interaction networks regulate blood stem and progenitor cell fate
Kirouac, D. C., Madlambayan, G. J., Yu, M., Sykes, E. A., Ito, C. Y., & Zandstra, P. W. (2009) · Molecular Systems Biology · doi:10.1038/msb.2009.49 · Cited by 134
“4 Molecular Systems Biology 2009 & 2009 EMBO and Macmillan Publishers Limited consistent feature of systems biology models, and may reflect underlying robustness of biological networks (Gutenkunst et al, 2007).”
BackgroundParameter uncertainty in biochemical models described by ordinary differential equations
Vanlier, J., Tiemann, C. A., Hilbers, P. A. J., & van Riel, N. A. W. (2013) · Mathematical Biosciences · doi:10.1016/j.mbs.2013.03.006 · Cited by 116
“Depending on the model, these first order sensitivities can either be computed by solving the sensitivity equations (preferred) [12,42], or by means of finite differences (for which strict solver tolerances are required to ensure reliable derivatives).”
BackgroundMethodInfluentialInverse problems in systems biology
Engl, H. W., Flamm, C., Kügler, P., Lu, J., Müller, S., & Schuster, P. (2009) · Inverse Problems · doi:10.1088/0266-5611/25/12/123014 · Cited by 122
“[61] R.”
BackgroundMetabolomics in systems microbiology
Reaves, M. L., & Rabinowitz, J. D. (2010) · Current Opinion in Biotechnology · doi:10.1016/j.copbio.2010.10.001 · Cited by 143
“A difficulty of such models is that they typically involve a large number of unmeasured parameters [44].”
A short history of causal modeling of fMRI data
Stephan, K. E., & Roebroeck, A. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.01.034 · Cited by 112
“knowledge, it is possible that it allows for inference on certain (neuronal) mechanisms, but not on others, due to conditional dependencies among parameters that are typical for biological systems (Gutenkunst et al., 2007; Stephan et al., 2007b).”
Tennøe, S., Halnes, G., & Einevoll, G. T. (2018) · Frontiers in Neuroinformatics · doi:10.3389/fninf.2018.00049 · Cited by 135
“Other methods that have been applied are local methods, which are multi-dimensional, but confined to exploring small perturbations surrounding a single point in the parameter space (see e.g., Gutenkunst et al., 2007; Blomquist et al., 2009; O’Donnell et al., 2017).”
MethodProduction of biofuels and biochemicals: in need of an ORACLE
Mišković, L., & Vassily Hatzimanikatis, A. (2010) · Trends in biotechnology · doi:10.1016/j.tibtech.2010.05.003 · Cited by 110
Systematic Construction of Kinetic Models from Genome-Scale Metabolic Networks
Stanford, N., Lubitz, T., Smallbone, K., Klipp, E., Mendes, P., & Liebermeister, W. (2013) · PLoS ONE · doi:10.1371/journal.pone.0079195 · Cited by 135
“To quote Gutenkunst et al (2007) [26]: ‘‘For computational modeling to be useful in incompletely understood systems, we must focus not on building the final, perfect, model with all parameters precisely determined, but on building incomplete, tentative, and falsifiable models in the most expressive…”
BackgroundA practical approach to parameter estimation applied to model predicting heart rate regulation
Olufsen, M. S., & Ottesen, J. T. (2013) · Journal of Mathematical Biology · doi:10.1007/s00285-012-0535-8 · Cited by 106
“A large number of techniques exist including Monte Carlo methods [28,29,68], various forms of principal component methods [55], local [33,35,53,56,63] and global [33,44,59,61] methods, as well as Kalman filtering methods [4,8,31].”
Dynamics of the Dorsal morphogen gradient
Kanodia, J., Rikhy, R., Kim, Y., Lund, V. K., DeLotto, R., Lippincott‐Schwartz, J., & Shvartsman, S. Y. (2009) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0912395106 · Cited by 107
“A similar approach has been successfully used to make model-based statistical predictions about the dynamics of other cell signaling systems (21).”
Identifiability analysis for stochastic differential equation models in systems biology
Browning, A. P., Warne, D. J., Burrage, K., Baker, R. E., & Simpson, M. J. (2020) · Journal of The Royal Society Interface · doi:10.1098/rsif.2020.0652 · Cited by 143
“Establishing the identifiability of model parameters is critical as predictions, and parameter estimates, from an non-identifiable model cannot be trusted [27–30].”
BackgroundAdvancing metabolic models with kinetic information
Link, H., Christodoulou, D., & Sauer, U. (2014) · Current Opinion in Biotechnology · doi:10.1016/j.copbio.2014.01.015 · Cited by 117
“The success of these random sampling methods shows that good predictions do not necessarily require precise parameters, as already realized a couple of years ago [26].”
BackgroundThe Limitations of Model-Based Experimental Design and Parameter Estimation in Sloppy Systems
White, A., Tolman, M., Thames, H. D., Withers, H. R., Mason, K. A., & Transtrum, M. K. (2016) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1005227 · Cited by 124
“Examples of sloppy, unidentifiable models (first column) abound in the systems biology literature (see references [8,9,12] for several examples).”
BackgroundMethodInfluentialCanelas, A. B., Ras, C., ten Pierick, A., van Gulik, W. M., & Heijnen, J. J. (2011) · Metabolic Engineering · doi:10.1016/j.ymben.2011.02.005 · Cited by 114
“Additionally, the very characteristics of metabolic systems can aggravate non-identifiability problems due to low parameter sensitivities (Waterfall et al., 2006; Gutenkunst et al., 2007), correlations between metabolite profiles and pseudo-steady-state coupling of high turnover pools (Nikerel et al.”
BackgroundA bayesian approach to targeted experiment design
Vanlier, J., Tiemann, C. A., Hilbers, P. A. J., & van Riel, N. A. W. (2012) · Bioinformatics · doi:10.1093/bioinformatics/bts092 · Cited by 98
“We employ a Bayesian approach involving importance sampling of the posterior predictive distribution to predict the efficacy of a new measurement at reducing the uncertainty of a selected prediction.”
MethodCalibration of ionic and cellular cardiac electrophysiology models
Whittaker, D. J., Clerx, M., Lei, C. L., Christini, D. J., & Mirams, G. R. (2020) · WIREs Systems Biology and Medicine · doi:10.1002/wsbm.1482 · Cited by 103
“Such behavior has been termed sloppiness and it has been suggested that it is a “ universal ” property of systems biology models (Brown & Sethna, 2003; Gutenkunst et al., 2007; Waterfall et al., 2006).”
BackgroundGene Circuit Analysis of the Terminal Gap Gene huckebein
Ashyraliyev, M., Siggens, K., Janssens, H., Blom, J. G., Akam, M. E., & Jaeger, J. (2009) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000548 · Cited by 100
“It has been shown that biological network models always contain at least a few parameters which cannot be determined, and that this is usually due to parameter correlations [34].”
BackgroundInfluentialIdentification of models of heterogeneous cell populations from population snapshot data
Hasenauer, J., Waldherr, S., Doszczak, M., Radde, N. E., Scheurich, P., & Allgöwer, F. (2011) · BMC Bioinformatics · doi:10.1186/1471-2105-12-125 · Cited by 102
Genetic Diversity in the Interference Selection Limit
Good, B. H., Walczak, A. M., Neher, R. A., & Desai, M. M. (2014) · PLoS Genetics · doi:10.1371/journal.pgen.1004222 · Cited by 116
“This suggests a degree of ‘‘sloppiness’’ [81] in the underlying model, which can lead to large intrinsic uncertainties in the parameter estimates and a strong sensitivity to measurement noise.”
BackgroundExperiment and mathematical modeling of gene expression dynamics in a cell-free system
Stögbauer, T., Windhager, L., Zimmer, R., & Rädler, J. O. (2012) · Integrative Biology · doi:10.1039/c2ib00102k · Cited by 96
Topological sensitivity analysis for systems biology
Babtie, A. C., Kirk, P., & Stumpf, M. P. H. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1414026112 · Cited by 98
Daniels, B. C., & Nemenman, I. M. (2015) · PLoS ONE · doi:10.1371/journal.pone.0119821 · Cited by 95
“This fails spectacularly in complex nonlinear ODE models, with typically widely varying sensitivities in different parameter space directions [7].”
BackgroundInfluentialGenetic variants in Alzheimer disease — molecular and brain network approaches
Gaiteri, C., Mostafavi, S., Honey, C. J., Jager, P. L. D., & Bennett, D. A. (2016) · Nature Reviews Neurology · doi:10.1038/nrneurol.2016.84 · Cited by 103
On the Role of Theory and Modeling in Neuroscience
Levenstein, D., Alvarez, V. A., Amarasingham, A., Azab, H., Chen, Z. S., Gerkin, R. C., Hasenstaub, A. R., Iyer, R., Jolivet, R., Marzen, S., et al. (2023) · Journal of Neuroscience · doi:10.1523/jneurosci.1179-22.2022 · Cited by 119
Enabling battery digital twins at the industrial scale
Dubarry, M., Howey, D. A., & Wu, B. (2023) · Joule · doi:10.1016/j.joule.2023.05.005 · Cited by 120
A Six‐Stage Workflow for Robust Application of Systems Pharmacology
Gadkar, K., Kirouac, D., Mager, D., van der Graaf, P., & Ramanujan, S. (2016) · CPT: Pharmacometrics & Systems Pharmacology · doi:10.1002/psp4.12071 · Cited by 93
Metadata
- Authors
- Gutenkunst, R. N., Waterfall, J. J., Casey, F. P., Brown, K. S., Myers, C. R., & Sethna, J. P.
- Affiliations
- Cornell University (US)
- University College Dublin (IE)
- Harvard University (US)
- Corresponding author
- Gutenkunst, Ryan N
- ORCID
- Venue
- PLoS Computational Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2007-10-05
- Volume
- 3
- Issue
- 10
- Pages
- e189
- ISSN
- 1553-734X (ISSN-L)
- 1553-7358
- PMC
- PMC2000971
- OpenAlex
- W2161304688
- Cited by
- 1,261
- References
- 44
- Citations by year
- 2026: 99 · 2025: 72 · 2024: 58 · 2023: 61 · 2022: 64 · 2021: 66 · 2020: 63 · 2019: 64 · 2018: 64 · 2017: 68 · 2016: 69 · 2015: 89 · 2014: 72 · 2013: 71 · 2012: 71
- FWCI
- 18.13
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS Computational Biology (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Figshare (submittedVersion, cc-by)
- arXiv (Cornell University)
- Funders
- National Science Foundation
- U.S. Department of Energy
- U.S. Department of Agriculture
- National Institutes of Health
- Agricultural Research Service
- National Institute of General Medical Sciences (T32GM008267)
- Division of Materials Research
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene Regulatory Network Analysis
- Physical Sciences › Engineering › Control and Systems Engineering › Control Systems and Identification
- Physical Sciences › Mathematics › Modeling and Simulation › Mathematical Biology Tumor Growth
- Keywords
- parameter sensitivity, parameter identifiability, growth factor signaling, systems biology models, predictive modeling, time series
- MeSH terms
- Algorithms, Computer Simulation, Half-Life, Models, Biological, Monte Carlo Method, Probability, Sensitivity and Specificity, Meta-Analysis as Topic, Models, Statistical, Nonlinear Dynamics, Systems Biology, Metabolic Networks and Pathways
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.