CoChat

Universally Sloppy Parameter Sensitivities in Systems Biology Models

Open access

The DOI matches its Crossref record; 44 of the 44 references it cites were found in a registry or an index; 24 of the 45 sources its claims cite were checked for support.

Abstract

Quantitative computational models play an increasingly important role in modern biology. Such models typically involve many free parameters, and assigning their values is often a substantial obstacle to model development. Directly measuring in vivo biochemical parameters is difficult, and collectively fitting them to other experimental data often yields large parameter uncertainties. Nevertheless, in earlier work we showed in a growth-factor-signaling model that collective fitting could yield well-constrained predictions, even when it left individual parameters very poorly constrained. We also showed that the model had a "sloppy" spectrum of parameter sensitivities, with eigenvalues roughly evenly distributed over many decades. Here we use a collection of models from the literature to test whether such sloppy spectra are common in systems biology. Strikingly, we find that every model we examine has a sloppy spectrum of sensitivities. We also test several consequences of this sloppiness for building predictive models. In particular, sloppiness suggests that collective fits to even large amounts of ideal time-series data will often leave many parameters poorly constrained. Tests over our model collection are consistent with this suggestion. This difficulty with collective fits may seem to argue for direct parameter measurements, but sloppiness also implies that such measurements must be formidably precise and complete to usefully constrain many model predictions. We confirm this implication in our growth-factor-signaling model. Our results suggest that sloppy sensitivity spectra are universal in systems biology models. The prevalence of sloppiness highlights the power of collective fits and suggests that modelers should focus on predictions rather than on parameters.

Overview

CoChat Summary

TL;DR

Across 17 systems biology models, every model examined had a “sloppy” parameter sensitivity spectrum, with eigenvalues spanning many decades roughly evenly. Sloppiness helps explain why collective fits leave individual parameters poorly constrained yet can yield tight predictions, while direct parameter measurements must be both precise and complete to constrain predictions.

Key findings

  • All 17 models exhibited sloppy sensitivity spectra; all but one spanned more than 10^6 in eigenvalues, and few principal axes aligned with bare parameter axes (I/P ≈ 1 was rare).
  • In idealized collective fits using 100 times as many time-series data points as parameters, each with 10% uncertainty, most models left many parameters with greater than 100% uncertainty (Σ > 1).
  • In the 48-parameter Brown et al. growth-factor-signaling model, collective fitting to 14 experiments and 68 data points left all 48 parameters with 95% confidence intervals spanning more than a factor of 50, yet an ERK prediction under EGF stimulation with PI3K inhibition was tight.
  • To match that prediction with direct measurements, all 24 involved parameters needed measurement within ±25%; leaving one of about 18 of the 24 involved parameters unmeasured, even bounded to a 1,000-fold range, produced a prediction with very large uncertainty.

Methods

  • Collected 17 ordinary-differential-equation systems biology models from the literature and BioModels, ranging from about ten to more than 200 parameters, and quantified behavior change by average squared change in molecular species time courses.
  • Analyzed the Hessian matrix Hχ2 with derivatives taken with respect to log parameters, using sensitivity equations rather than finite differences; computations used SloppyCell version 0.81.
  • For parameter uncertainties, rescaled Hχ2 for Nd data points with fractional uncertainty f and computed relative 95% confidence interval size Σ; for prediction uncertainties in the Brown et al. model, generated 1,000 parameter sets consistent with stated uncertainties and calculated ERK time courses.

Limitations

  • Stated: the paper focuses on parameter uncertainties, not model-structure uncertainty; sloppiness depends on the model and the data fit, and exhaustive experiments designed to decouple parameters would not yield sloppy sensitivities.
  • Stated: successful fits and predictions may arise from models that are incorrect in significant ways, and potential model-refinement methods demand a complete understanding unavailable for many biological systems.
  • Evident: most collected models were not systematically fit or measured in their original publications, and the sensitivity spectra rely on a local quadratic Hessian approximation, though Monte Carlo analysis in the Brown et al. model supports the nonlinear χ2 picture.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1371/journal.pcbi.0030189

Cites

44 of 44 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Figure 1 Parameter Sensitivity Spectra The quantities we calculate from H χ2 are illustrated in (A), while (B) and (C) show that all the models we examined have sloppy sensitivity spectra. (A) Analyzing H χ2 corresponds to approximating the surfaces of constant model behavior change (constant χ 2) as ellipsoids. The width of each principal axis is proportional to one over the square root of the corresponding eigenvalue. The inner ellipsoid's projection onto and intersection with the θ 1 axis are denoted by P 1 and I1, respectively. (B) Plotted are the eigenvalue spectra of H χ2 for our collection of systems biology models. The many decades generally spanned indicate that the ellipses have a very large aspect ratio. (The spectra have each been normalized by their largest eigenvalue. Not all values are visible for all models.) (C) Plotted is the spectrum of I / P for each parameter in each model in our collection. Generally very few parameters have I / P ≈ 1, suggesting that the ellipses are skewed from the bare parameter axes. (Not all values are visible for all models.) The models are plotted in order of increasing number of free parameters and are: (a) eukaryotic cell cycle [28], (b) Xenopus egg cell cycle [29], (c) eukaryotic mitosis [30], (d) generic circadian rhythm [31], (e) nicotinic acetylcholine intra-receptor dynamics [32], (f) generic kinase cascade [33], (g) Xenopus Wnt signaling [34], (h) Drosophila circadian rhythm [35], (i) rat growth-factor signaling [21], (j) Drosophila segment polarity [36], (k) Drosophila circadian rhythm [37], (l) Arabidopsis circadian rhythm [2], (m) in silico regulatory network [25], (n) human purine metabolism [38], (o) Escherichia coli carbon metabolism [39], (p) budding yeast cell cycle [40], (q) rat growth-factor signaling [41].
Figure 1 Parameter Sensitivity Spectra The quantities we calculate from H χ2 are illustrated in (A), while (B) and (C) show that all the models we examined have sloppy sensitivity spectra. (A) Analyzing H χ2 corresponds to approximating the surfaces of constant model behavior change (constant χ 2) as ellipsoids. The width of each principal axis is proportional to one over the square root of the corresponding eigenvalue. The inner ellipsoid's projection onto and intersection with the θ 1 axis are denoted by P 1 and I1, respectively. (B) Plotted are the eigenvalue spectra of H χ2 for our collection of systems biology models. The many decades generally spanned indicate that the ellipses have a very large aspect ratio. (The spectra have each been normalized by their largest eigenvalue. Not all values are visible for all models.) (C) Plotted is the spectrum of I / P for each parameter in each model in our collection. Generally very few parameters have I / P ≈ 1, suggesting that the ellipses are skewed from the bare parameter axes. (Not all values are visible for all models.) The models are plotted in order of increasing number of free parameters and are: (a) eukaryotic cell cycle [28], (b) Xenopus egg cell cycle [29], (c) eukaryotic mitosis [30], (d) generic circadian rhythm [31], (e) nicotinic acetylcholine intra-receptor dynamics [32], (f) generic kinase cascade [33], (g) Xenopus Wnt signaling [34], (h) Drosophila circadian rhythm [35], (i) rat growth-factor signaling [21], (j) Drosophila segment polarity [36], (k) Drosophila circadian rhythm [37], (l) Arabidopsis circadian rhythm [2], (m) in silico regulatory network [25], (n) human purine metabolism [38], (o) Escherichia coli carbon metabolism [39], (p) budding yeast cell cycle [40], (q) rat growth-factor signaling [41].CC BY · PubMed Central
Figure 2 Sloppiness and Uncertainties As in Figure 1A, the contours represent surfaces of constant model behavior deviation. The clouds of points represent parameter set ensembles. (A) Collective fitting of model parameters naturally constrains the parameter set ensemble along stiff directions and allows it to expand along sloppy directions. The resulting ensemble may be very large, yet encompass little variation in model behavior, yielding small prediction uncertainties despite large parameter uncertainties. (Σ1 denotes the 95% confidence for the value of θ 1.) (B) If all parameters are directly measured to the same precision, the parameter set ensemble is spherical. The measurement precision required for well-constrained predictions is set by the stiffest direction. (C) If one parameter (here θ 2) is known less precisely than the rest, the cloud is ellipsoidal. If not aligned with a sloppy direction, the cloud will admit many model behaviors and yield large prediction uncertainties. (Note that the aspect ratio of the real contours can be greater than 1,000.)
Figure 2 Sloppiness and Uncertainties As in Figure 1A, the contours represent surfaces of constant model behavior deviation. The clouds of points represent parameter set ensembles. (A) Collective fitting of model parameters naturally constrains the parameter set ensemble along stiff directions and allows it to expand along sloppy directions. The resulting ensemble may be very large, yet encompass little variation in model behavior, yielding small prediction uncertainties despite large parameter uncertainties. (Σ1 denotes the 95% confidence for the value of θ 1.) (B) If all parameters are directly measured to the same precision, the parameter set ensemble is spherical. The measurement precision required for well-constrained predictions is set by the stiffest direction. (C) If one parameter (here θ 2) is known less precisely than the rest, the cloud is ellipsoidal. If not aligned with a sloppy direction, the cloud will admit many model behaviors and yield large prediction uncertainties. (Note that the aspect ratio of the real contours can be greater than 1,000.)CC BY · PubMed Central
Figure 3 Fitting Parameters to Idealized Data Shown are histograms of the relative confidence interval size Σ for each parameter in each model of our collection, after fitting 100 times as many time-series data points (each with 10% uncertainty) as parameters. In most cases, a large number of parameters are left with greater than 100% uncertainty. (A parameter constrained with 95% probability to lie between 1 and 100 would have Σ ≈ 100.) Labels are as in Figure 1.
Figure 3 Fitting Parameters to Idealized Data Shown are histograms of the relative confidence interval size Σ for each parameter in each model of our collection, after fitting 100 times as many time-series data points (each with 10% uncertainty) as parameters. In most cases, a large number of parameters are left with greater than 100% uncertainty. (A parameter constrained with 95% probability to lie between 1 and 100 would have Σ ≈ 100.) Labels are as in Figure 1.CC BY · PubMed Central
Figure 4 Parameter and Prediction Uncertainties (A) Our example prediction is for ERK activity upon EGF stimulation given PI3K inhibition in this 48-parameter model of growth-factor signaling in PC12 cells [21]. (B) Shaded regions are 95% confidence intervals calculated via exhaustive Monte Carlo for our example prediction given various scenarios for constraining parameter values. (C) Plotted is the relative size Σ of the 95% confidence interval for each parameter. The scenarios represented are: (red, squares) all model parameters individually measured to high precision, (blue, triangles) all parameters precisely measured, except one estimated to low precision, (yellow, circles) all parameters collectively fit to 14 real cell-biology experiments. Precisely measured individual parameter values enable a tight prediction, (B) middle red band; but even one poorly known parameter can destroy predictive power, (B) wide blue band. In contrast, the collective fit yields a tight prediction, (B) tightest yellow band; but only very loose parameter constraints, (C) circles. The large parameter uncertainties from the collective fit, (C) circles, calculated here by Monte Carlo are qualitatively similar to those seen in the linearized fit to idealized data (Figure 3, model (i)). (For clarity, the dashed red lines trace the boundary of the red confidence interval.)
Figure 4 Parameter and Prediction Uncertainties (A) Our example prediction is for ERK activity upon EGF stimulation given PI3K inhibition in this 48-parameter model of growth-factor signaling in PC12 cells [21]. (B) Shaded regions are 95% confidence intervals calculated via exhaustive Monte Carlo for our example prediction given various scenarios for constraining parameter values. (C) Plotted is the relative size Σ of the 95% confidence interval for each parameter. The scenarios represented are: (red, squares) all model parameters individually measured to high precision, (blue, triangles) all parameters precisely measured, except one estimated to low precision, (yellow, circles) all parameters collectively fit to 14 real cell-biology experiments. Precisely measured individual parameter values enable a tight prediction, (B) middle red band; but even one poorly known parameter can destroy predictive power, (B) wide blue band. In contrast, the collective fit yields a tight prediction, (B) tightest yellow band; but only very loose parameter constraints, (C) circles. The large parameter uncertainties from the collective fit, (C) circles, calculated here by Monte Carlo are qualitatively similar to those seen in the linearized fit to idealized data (Figure 3, model (i)). (For clarity, the dashed red lines trace the boundary of the red confidence interval.)CC BY · PubMed Central

Verification

Claims the paper makes

20 Valid · 3 Not valid · 7 Not checked · 41 Own results and interpretations

37 more claims · 12 with their cited sources checked

Create a free account to see all 71 claims and every source verdict.

See all claims — free account
  • It may state two separate claims as one. 18
  • Probably not a substantial claim. 11

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

28 more claims about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Citations

Cites

Research works printed but not linked: 1

  1. MehtaML 2004 Random Matrices. 3rd edition Amsterdam Academic Press 706

Cited by

See all 1,273

Metadata

Authors
Gutenkunst, R. N., Waterfall, J. J., Casey, F. P., Brown, K. S., Myers, C. R., & Sethna, J. P.
Affiliations
  • Cornell University (US)
  • University College Dublin (IE)
  • Harvard University (US)
Corresponding author
Gutenkunst, Ryan N
Venue
PLoS Computational Biology
Published
2007-10-05
Volume
3
Issue
10
Pages
e189
ISSN
  • 1553-734X (ISSN-L)
  • 1553-7358
OpenAlex
W2161304688
Cited by
1,261
References
44
Citations by year
2026: 99 · 2025: 72 · 2024: 58 · 2023: 61 · 2022: 64 · 2021: 66 · 2020: 63 · 2019: 64 · 2018: 64 · 2017: 68 · 2016: 69 · 2015: 89 · 2014: 72 · 2013: 71 · 2012: 71
FWCI
18.13
Type
journal article
Language
en
Open access
Locations
  • PLoS Computational Biology (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
  • Figshare (submittedVersion, cc-by)
  • arXiv (Cornell University)
Funders
  • National Science Foundation
  • U.S. Department of Energy
  • U.S. Department of Agriculture
  • National Institutes of Health
  • Agricultural Research Service
  • National Institute of General Medical Sciences (T32GM008267)
  • Division of Materials Research
Topics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene Regulatory Network Analysis
  • Physical Sciences › Engineering › Control and Systems Engineering › Control Systems and Identification
  • Physical Sciences › Mathematics › Modeling and Simulation › Mathematical Biology Tumor Growth
Keywords
parameter sensitivity, parameter identifiability, growth factor signaling, systems biology models, predictive modeling, time series
MeSH terms
Algorithms, Computer Simulation, Half-Life, Models, Biological, Monte Carlo Method, Probability, Sensitivity and Specificity, Meta-Analysis as Topic, Models, Statistical, Nonlinear Dynamics, Systems Biology, Metabolic Networks and Pathways

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.