Practical limits for reverse engineering of dynamical systems: a statistical analysis of sensitivity and parameter inferability in systems biology models
The DOI matches its Crossref record; 38 of the 39 references it cites were found in a registry or an index, and 1 are not checked yet.
Abstract
The size and complexity of cellular systems make building predictive models an extremely difficult task. In principle dynamical time-course data can be used to elucidate the structure of the underlying molecular mechanisms, but a central and recurring problem is that many and very different models can be fitted to experimental data, especially when the latter are limited and subject to noise. Even given a model, estimating its parameters remains challenging in real-world systems. Here we present a comprehensive analysis of 180 systems biology models, which allows us to classify the parameters with respect to their contribution to the overall dynamical behaviour of the different systems. Our results reveal candidate elements of control in biochemical pathways that differentially contribute to dynamics. We introduce sensitivity profiles that concisely characterize parameter sensitivity and demonstrate how this can be connected to variability in data. Systematically linking data and model sloppiness allows us to extract features of dynamical systems that determine how well parameters can be estimated from time-course measurements, and associates the extent of data required for parameter inference with the model structure, and also with the global dynamical state of the system. The comprehensive analysis of so many systems biology models reaffirms the inability to estimate precisely most model or kinetic parameters as a generic feature of dynamical systems, and provides safe guidelines for performing better inferences and model predictions in the context of reverse engineering of mathematical models for biological systems.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“This uncertainty, it has turned out, can play a pivotal role in the analysis of a system’s dynamics16–18.”
From A framework for parameter estimation and model selection from experimental data in systems biology using approximate Bayesian computation (2014) · cites this paper as [18]
In the citing paper
“In most biological systems, however, only a few reaction rates can be inferred given an observed dataset and different parameter vectors can explain the observed data (almost) equally well16,18,67.”
From A framework for parameter estimation and model selection from experimental data in systems biology using approximate Bayesian computation (2014) · cites this paper as [18]
In the citing paper
“With an optimal design we can overcome the problems of sloppy parameters [16] (which are, of course, dependent on the experimental intervention chosen [17]) and can narrow down the posterior probability intervals of parameters.”
From Maximizing the information content of experiments in systems biology (2013) · cites this paper as [17]
In the citing paper
“The Bayesian view will also help in establishing the practical limits for reverse engineering of kinetic models [236].”
From Reverse engineering and identification in systems biology: strategies, perspectives and challenges (2014) · cites this paper as [236]
Citations
Cites
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482
Gelman, A., Carlin, J. B., Stern, H. S., & Rubin, D. B. (2003) · doi:10.1201/9780429258480 · Cited by 27,276
Hucka, M., Finney, A., Sauro, H. M., Bolouri, H., Doyle, J. C., Kitano, H., Arkin, A. P., Bornstein, B. J., Bray, D., Cornish-Bowden, A., et al. (2003) · Bioinformatics · doi:10.1093/bioinformatics/btg015 · Cited by 3,060
Faith, J. J., Hayete, B., Thaden, J. T., Mogno, I., Wierzbowski, J., Cottarel, G., Kasif, S., Collins, J. J., & Gardner, T. S. (2007) · PLoS Biology · doi:10.1371/journal.pbio.0050008 · Cited by 1,563
Toni, T., Welch, D., Strelkowa, N., Ipsen, A., & Stumpf, M. P. H. (2009) · Journal of The Royal Society Interface · doi:10.1098/rsif.2008.0172 · Cited by 1,498
Self-citationReverse engineering of regulatory networks in human B cells
Basso, K., Margolin, A. A., Stolovitzky, G., Klein, U., Dalla-Favera, R., & Califano, A. (2005) · Nature Genetics · doi:10.1038/ng1532 · Cited by 1,404
Universally Sloppy Parameter Sensitivities in Systems Biology Models
Gutenkunst, R. N., Waterfall, J. J., Casey, F. P., Brown, K. S., Myers, C. R., & Sethna, J. P. (2007) · PLoS Computational Biology · doi:10.1371/journal.pcbi.0030189 · Cited by 1,261
Inferring Genetic Networks and Identifying Compound Mode of Action via Expression Profiling
Gardner, T. S., di Bernardo, D., Lorenz, D., & Collins, J. J. (2003) · Science · doi:10.1126/science.1081900 · Cited by 1,117
Probabilistic Sensitivity Analysis of Complex Models: A Bayesian Approach
Oakley, J. E., & O'Hagan, A. (2004) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2004.05304.x · Cited by 1,085
Metabolic control analysis: a survey of its theoretical and experimental development
Fell, D. A. (1992) · Biochemical Journal · doi:10.1042/bj2860313 · Cited by 782
Teusink, B., Passarge, J., Reijenga, C. A., Esgalhado, E., van der Weijden, C. C., Schepper, M., Walsh, M. C., Bakker, B. M., van Dam, K., Westerhoff, H. V., et al. (2000) · European Journal of Biochemistry · doi:10.1046/j.1432-1327.2000.01527.x · Cited by 694
Neuropsychological mechanisms of interval timing behavior
Matell, M. S., & Meck, W. H. (2000) · BioEssays · doi:10.1002/(sici)1521-1878(200001)22:1<94::aid-bies14>3.0.co;2-e · Cited by 519
Diversity-based, model-guided construction of synthetic gene networks with predicted functions
Ellis, T., Wang, X., & Collins, J. J. (2009) · Nature Biotechnology · doi:10.1038/nbt.1536 · Cited by 449
Statistical mechanical approaches to models with many poorly known parameters.
Brown, K. S., & Sethna, J. P. (2003) · Physical Review E · doi:10.1103/physreve.68.021904 · Cited by 398
Chemogenomic profiling on a genome-wide scale using reverse-engineered gene networks
di Bernardo, D., Thompson, M. J., Gardner, T. S., Chobot, S. E., Eastwood, E. L., Wojtovich, A. P., Elliott, S. J., Schaus, S. E., & Collins, J. J. (2005) · Nature Biotechnology · doi:10.1038/nbt1075 · Cited by 344
Parameter estimation and model selection in computational biology
Lillacci, G., & Khammash, M. (2010) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000696 · Cited by 325
Models from experiments: combinatorial drug perturbations of cancer cells
Nelander, S., Wang, W., Nilsson, B., She, Q., Pratilas, C., Rosen, N., Gennemark, P., & Sander, C. (2008) · Molecular Systems Biology · doi:10.1038/msb.2008.53 · Cited by 187
Leloup, J.-C., & Goldbeter, A. (1999) · Journal of Theoretical Biology · doi:10.1006/jtbi.1999.0924 · Cited by 159
Sloppy models, parameter uncertainty, and the role of experimental design
Apgar, J. F., Witmer, D. K., White, F. M., & Tidor, B. (2010) · Molecular BioSystems · doi:10.1039/b918098b · Cited by 146
Optimal Experimental Design for Parameter Estimation of a Cell Signaling Model
Bandara, S., Schlöder, J. P., Eils, R., Bock, H. G., & Meyer, T. (2009) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000558 · Cited by 135
Schemes of flux control in a model of Saccharomyces cerevisiae glycolysis
Pritchard, L., & Kell, D. B. (2002) · European Journal of Biochemistry · doi:10.1046/j.1432-1033.2002.03055.x · Cited by 102
Hopf bifurcation in an Internet congestion control model
Li, C., Chen, G., Liao, X., & Yu, J. (2003) · Chaos Solitons & Fractals · doi:10.1016/s0960-0779(03)00269-8 · Cited by 100
Modeling the molecular regulatory mechanism of circadian rhythms in Drosophila
Leloup, J.-C., & Goldbeter, A. (2000) · BioEssays · doi:10.1002/(sici)1521-1878(200001)22:1<84::aid-bies13>3.0.co;2-i · Cited by 98
Parameter estimation and determinability analysis applied to Drosophila gap gene circuits
Ashyraliyev, M., Jaeger, J., & Blom, J. G. (2008) · BMC Systems Biology · doi:10.1186/1752-0509-2-83 · Cited by 89
Glyceraldehyde-3-phosphate Dehydrogenase Is Regulated on a Daily Basis by the Circadian Clock
Shinohara, M. L., Loros, J., & Dunlap, J. (1998) · Journal of Biological Chemistry · doi:10.1074/jbc.273.1.446 · Cited by 88
Stimulus Design for Model Selection and Validation in Cell Signaling
Apgar, J. F., Toettcher, J. E., Endy, D., White, F. M., & Tidor, B. (2008) · PLoS Computational Biology · doi:10.1371/journal.pcbi.0040030 · Cited by 79
Smallest chemical reaction system with Hopf bifurcation
Wilhelm, T. E., & Heinrich, R. (1995) · Journal of Mathematical Chemistry · doi:10.1007/bf01165134 · Cited by 67
‘Glocal’ Robustness Analysis and Model Discrimination for Circadian Oscillators
Hafner, M., Koeppl, H., Hasler, M., & Wagner, A. (2009) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000534 · Cited by 63
Mélykúti, B., August, E., Papachristodoulou, A., & El-Samad, H. (2010) · BMC Systems Biology · doi:10.1186/1752-0509-4-38 · Cited by 58
Alternative attractors may boost uncertainty and sensitivity in ecological models
van Nes, E. H., & Scheffer, M. (2002) · Ecological Modelling · doi:10.1016/s0304-3800(02)00278-8 · Cited by 48
The ABC of reverse engineering biological signalling systems
Secrier, M., Toni, T., & Stumpf, M. P. H. (2009) · Molecular BioSystems · doi:10.1039/b908951a · Cited by 47
Self-citationTasseff, R. A., Nayak, S., Salim, S., Kaushik, P., Rizvi, N. F., & Varner, J. D. (2010) · PLoS ONE · doi:10.1371/journal.pone.0008864 · Cited by 41
Parameter Inference for Biochemical Systems that Undergo a Hopf Bifurcation
Kirk, P., Toni, T., & Stumpf, M. P. H. (2008) · Biophysical Journal · doi:10.1529/biophysj.107.126086 · Cited by 31
Self-citationInferring Drosophila gap gene regulatory network: a parameter sensitivity and perturbation analysis
Fomekong‐Nanfack, Y., Postma, M., & Kaandorp, J. A. (2009) · BMC Systems Biology · doi:10.1186/1752-0509-3-94 · Cited by 31
Chaves, M., Sengupta, A. M., & Sontag, E. D. (2008) · Journal of Mathematical Biology · doi:10.1007/s00285-008-0230-y · Cited by 27
Garny, A., Cooper, J., & Hunter, P. J. (2009) · WIREs Systems Biology and Medicine · doi:10.1002/wsbm.63 · Cited by 26
Nikolov, S. G., Lai, X., Liebal, U. W., Wolkenhauer, O., & Vera, J. (2010) · International Journal of Systems Science · doi:10.1080/00207720903147746 · Cited by 21
Dynamic sensitivity and control analyses of metabolic insulin signalling pathways
Liu, E., & Yuan, J. (2009) · IET Systems Biology · doi:10.1049/iet-syb.2008.0178 · Cited by 9
Dynamic Bayesian sensitivity analysis of a myocardial metabolic model
Daniela Calvetti, A., Hageman, R. K., Occhipinti, R., & Somersalo, E. (2007) · Mathematical Biosciences · doi:10.1016/j.mbs.2007.10.006 · Cited by 8
Research works printed but not linked: 7
- Garny, 2009, Wiley Interdiscip. Rev.: Syst. Biol.Med., 2, 134
- Coelho, 2009, PLoS Comput. Biol., 1
- Dayarian, 2009, PLoS Comput. Biol., 1
- Mackay, 2003, Information theory, inference and learning algorithms
- Teusink, 2000, FEBS J., 267, 5313
- Pritchard, 2002, FEBS J., 269, 3894
- Lillacci, 2010, PLoS Comput. Biol., 6, e1000696, 10.1371/journal.pcbi.1000696 · DOI · Paper page
Cited by
Perspective: Sloppiness and emergent theories in physics, biology, and beyond
Transtrum, M. K., Machta, B. B., Brown, K. S., Daniels, B. C., Myers, C. R., & Sethna, J. P. (2015) · The Journal of Chemical Physics · doi:10.1063/1.4923066 · Cited by 311
Kinetic models in industrial biotechnology – Improving cell factory performance
Almquist, J., Cvijovic, M., Hatzimanikatis, V., Nielsen, J., & Jirstrand, M. (2014) · Metabolic Engineering · doi:10.1016/j.ymben.2014.03.007 · Cited by 293
Reverse engineering and identification in systems biology: strategies, perspectives and challenges
Villaverde, A. F., & Banga, J. R. (2014) · Journal of The Royal Society Interface · doi:10.1098/rsif.2013.0505 · Cited by 229
“The Bayesian view will also help in establishing the practical limits for reverse engineering of kinetic models [236].”
BackgroundLiepe, J., Kirk, P., Filippi, S., Toni, T., Barnes, C. P., & Stumpf, M. P. H. (2014) · Nature Protocols · doi:10.1038/nprot.2014.025 · Cited by 217
“In most biological systems, however, only a few reaction rates can be inferred given an observed data set, and different parameter vectors can explain the observed data (almost) equally well 16,18,63 .”
BackgroundSelf-citationSensitivity, robustness, and identifiability in stochastic chemical kinetics models
Komorowski, M., Costa, M. J., Rand, D. A., & Stumpf, M. P. H. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1015814108 · Cited by 194
Self-citation
Metadata
- Authors
- Erguler, K., & Stumpf, M. P. H.
- Affiliations
- Imperial College London (GB)
- Corresponding author
- Stumpf, Michael P H
- Venue
- Molecular BioSystems
- Publisher
- Oxford University Press (OUP)
- Published
- 2011-03-04
- Volume
- 7
- Issue
- 5
- Pages
- 1593-1602
- ISSN
- 1742-2051 (ISSN-L)
- 1742-206X
- OpenAlex
- W2077259222
- Semantic Scholar
- 330e8c6df2bf2adb506714cd065e9d1b21b1dc82
- Cited by
- 91
- References
- 37
- Citations by year
- 2026: 1 · 2025: 1 · 2024: 3 · 2023: 3 · 2022: 3 · 2021: 3 · 2020: 7 · 2019: 8 · 2018: 7 · 2017: 6 · 2016: 6 · 2015: 11 · 2014: 12 · 2013: 10 · 2012: 7
- FWCI
- 4.14
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Molecular BioSystems (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Wellcome Trust
- Royal Society
- Biotechnology and Biological Sciences Research Council (BB/F005210/2, BB/F005210/1, BB/G007934/1)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene Regulatory Network Analysis
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Microbial Metabolic Engineering and Bioproduction
- Physical Sciences › Engineering › Control and Systems Engineering › Control Systems and Identification
- Keywords
- reverse engineering, dynamical systems, parameter identifiability, parameter sensitivity, sensitivity profiles, parameter inference, time course data, systems biology models, data variability, model structure
- MeSH terms
- Algorithms, Animals, Bayes Theorem, Computer Simulation, Humans, Models, Biological, Reproducibility of Results, Signal Transduction, Feedback, Physiological, Systems Biology, Circadian Clocks
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.