A protocol for dynamic model calibration
Open accessThe DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Abstract
Ordinary differential equation models are nowadays widely used for the mechanistic description of biological processes and their temporal evolution. These models typically have many unknown and nonmeasurable parameters, which have to be determined by fitting the model to experimental data. In order to perform this task, known as parameter estimation or model calibration, the modeller faces challenges such as poor parameter identifiability, lack of sufficiently informative experimental data and the existence of local minima in the objective function landscape. These issues tend to worsen with larger model sizes, increasing the computational complexity and the number of unknown parameters. An incorrectly calibrated model is problematic because it may result in inaccurate predictions and misleading conclusions. For nonexpert users, there are a large number of potential pitfalls. Here, we provide a protocol that guides the user through all the steps involved in the calibration of dynamic models. We illustrate the methodology with two models and provide all the code required to reproduce the results and perform the same analysis on new models. Our protocol provides practitioners and researchers in biological modelling with a one-stop guide that is at the same time compact and sufficiently comprehensive to cover all aspects of the problem.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
SciPy 1.0: fundamental algorithms for scientific computing in Python
Virtanen, P., Gommers, R., Oliphant, T. E., Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, P., Weckesser, W., Bright, J., et al. (2020) · Nature Methods · doi:10.1038/s41592-019-0686-2 · Cited by 38,430
SciPy 1.0: fundamental algorithms for scientific computing in Python
Virtanen, P., Gommers, R., Oliphant, T. E., Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, P., Weckesser, W., Bright, J., et al. (2019) · Apollo - University of Cambridge Repository · doi:10.17863/cam.64134 · Cited by 10,460
Efron, B., & Tibshirani, R. (1986) · Statistical Science · doi:10.1214/ss/1177013815 · Cited by 6,033
Bozdogan, H. (1987) · Psychometrika · doi:10.1007/bf02294361 · Cited by 4,468 · cited 1×
“Measures such as the Akaike Information Criterion (AIC) [8] and the Bayesian Information Criterion (BIC) [97] take into account the goodness of fit and a penalty based on the number of estimated parameters, and can thus be used to quantify this trade-off.”
Hucka, M., Finney, A., Sauro, H. M., Bolouri, H., Doyle, J. C., Kitano, H., Arkin, A. P., Bornstein, B. J., Bray, D., Cornish-Bowden, A., et al. (2003) · Bioinformatics · doi:10.1093/bioinformatics/btg015 · Cited by 3,060 · cited 1×
“The model can be specified in a standard format such as SBML, to take advantage of the ecosystem of tools that already support standard format [40].”
Toni, T., Welch, D., Strelkowa, N., Ipsen, A., & Stumpf, M. P. H. (2009) · Journal of The Royal Society Interface · doi:10.1098/rsif.2008.0172 · Cited by 1,498
Raue, A., Kreutz, C., Maiwald, T., Bachmann, J., Schilling, M., Klingmüller, U., & Timmer, J. (2009) · Bioinformatics · doi:10.1093/bioinformatics/btp358 · Cited by 1,442
Kernel methods in system identification, machine learning and function estimation: A survey
Pillonetto, G., Dinuzzo, F., Chen, T., De Nicolao, G., & Ljung, L. (2014) · Automatica · doi:10.1016/j.automatica.2014.01.001 · Cited by 741
On Identifiability of Nonlinear ODE Models and Applications in Viral Dynamics
Miao, H., Xia, X., Perelson, A. S., & Wu, H. (2011) · SIAM Review · doi:10.1137/090757009 · Cited by 610
Structural identifiability of systems biology models: a critical comparison of methods
Chis, O.-T., Banga, J. R., & Balsa-Canto, E. (2011) · PLoS ONE · doi:10.1371/journal.pone.0027755 · Cited by 448 · cited 1×
“Structural identifiability (SI) can be analysed using a broad spectrum of methods exploiting, e.g., Taylor and generating series, differential algebra, differential geometry, and probabilistic numerics [15,59,89].”
MethodSelf-citationWeb-based kinetic modelling using JWS Online
Olivier, B. G., & Snoep, J. L. (2004) · Bioinformatics · doi:10.1093/bioinformatics/bth200 · Cited by 326 · cited 1×
“Furthermore, already available models can be used, e.g. from JWS Online [61] or Biomodels [51], and information about kinetic parameters might be extracted, e.g. from BRENDA [13] or Sabio-RK [102].”
Systems biology: parameter estimation for biochemical models
Ashyraliyev, M., Fomekong‐Nanfack, Y., Kaandorp, J. A., & Blom, J. G. (2009) · The FEBS Journal · doi:10.1111/j.1742-4658.2008.06844.x · Cited by 323 · cited 1×
“In model calibration, the mismatch between simulated model output and experimental data is minimised to find the best parameter values [2,28,29,42,64].”
The statistical mechanics of complex signaling networks: Nerve growth factor signaling.
Brown, K. S., Hill, C. C., Calero, G. A., Myers, C. R., Lee, K. H., Sethna, J. P., & Cerione, R. A. (2004) · Physical Biology · doi:10.1088/1478-3967/1/3/006 · Cited by 265 · cited 1×
“Brown et al. used statistical mechanics considerations [10] to build the ensemble.”
Linking data to models: data regression
Jaqaman, K., & Danuser, G. (2006) · Nature Reviews Molecular Cell Biology · doi:10.1038/nrm2030 · Cited by 243
Reverse engineering and identification in systems biology: strategies, perspectives and challenges
Villaverde, A. F., & Banga, J. R. (2014) · Journal of The Royal Society Interface · doi:10.1098/rsif.2013.0505 · Cited by 229 · cited 1×
“It may be seen as part of a more general problem sometimes called reverse engineering [90] or (nonlinear) systems identification [70].”
Self-citationLiepe, J., Kirk, P., Filippi, S., Toni, T., Barnes, C. P., & Stumpf, M. P. H. (2014) · Nature Protocols · doi:10.1038/nprot.2014.025 · Cited by 217 · cited 1×
“Experimental data is used to compute a posterior distribution that describes the uncertainty of the problem [41,53,83].”
An iterative identification procedure for dynamic modeling of biochemical networks
Balsa-Canto, E., Alonso, A. A., & Banga, J. R. (2010) · BMC Systems Biology · doi:10.1186/1752-0509-4-11 · Cited by 202
Self-citationExploiting the bootstrap method for quantifying parameter confidence intervals in dynamical systems
Joshi, M., Seidel-Morgenstern, A., & Kremling, A. (2006) · Metabolic Engineering · doi:10.1016/j.ymben.2006.04.003 · Cited by 190
Bayesian ranking of biochemical system models
Vyshemirsky, V., & Girolami, M. A. (2008) · Bioinformatics · doi:10.1093/bioinformatics/btm607 · Cited by 181
Fröhlich, F., Kessler, T., Weindl, D., Shadrin, A., Schmiester, L., Hache, H., Muradyan, A., Schütte, M., Lim, J.-H., Heinig, M., et al. (2018) · Cell Systems · doi:10.1016/j.cels.2018.10.013 · Cited by 143 · cited 2×
“Large models can even possess thousands of state variables and parameters [25].”
BackgroundSelf-citationParameter estimation and optimal experimental design
Banga, J. R., & Balsa-Canto, E. (2008) · Essays in Biochemistry · doi:10.1042/bse0450195 · Cited by 141 · cited 3×
“In [6] it is suggested to obtain different parameters by solving the same parameter estimation problem (i.e. with the same dataset) starting from different initial points; this is similar to the “multi-start” procedure in [26].”
Self-citationLikelihood based observability analysis and confidence intervals for predictions of dynamic models
Kreutz, C., Raue, A., & Timmer, J. (2012) · BMC Systems Biology · doi:10.1186/1752-0509-6-120 · Cited by 134 · cited 1×
“The prediction profile likelihood method (PPL) calculates confidence intervals for the states by performing constrained optimisation of the likelihood using a fixed prediction value as a nonlinear constraint [49].”
MethodComputational procedures for optimal experimental design in biological systems
Balsa-Canto, E., Alonso, A. A., & Banga, J. R. (2008) · IET Systems Biology · doi:10.1049/iet-syb:20070069 · Cited by 124 · cited 1×
“The optimisation problem can be solved by control vector parameterization [4].”
Self-citationPEtab—Interoperable specification of parameter estimation problems in systems biology
Schmiester, L., Schälte, Y., Bergmann, F. T., Camba, T., Dudkin, E., Egert, J., Fröhlich, F., Fuhrmann, L., Hauber, A. L., Kemmer, S., et al. (2021) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1008646 · Cited by 96
Self-citationA Six‐Stage Workflow for Robust Application of Systems Pharmacology
Gadkar, K., Kirouac, D., Mager, D., van der Graaf, P., & Ramanujan, S. (2016) · CPT: Pharmacometrics & Systems Pharmacology · doi:10.1002/psp4.12071 · Cited by 93 · cited 1×
“In model calibration, the mismatch between simulated model output and experimental data is minimised to find the best parameter values [2,28,29,42,64].”
Casey, F. P., Baird, D., Feng, Q., Gutenkunst, R. N., Waterfall, J. J., Myers, C. R., Brown, K. S., Cerione, R. A., & Sethna, J. P. (2007) · IET Systems Biology · doi:10.1049/iet-syb:20060065 · Cited by 77
Data-driven reverse engineering of signaling pathways using ensembles of dynamic models
Henriques, D., Villaverde, A. F., Rocha, M., Saez-Rodriguez, J., & Banga, J. R. (2017) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1005379 · Cited by 44
Self-citationConvergence in parameters and predictions using computational experimental design
Hagen, D. R., White, J. K., & Tidor, B. (2013) · Interface Focus · doi:10.1098/rsfs.2013.0008 · Cited by 41 · cited 1×
“OED can be performed with several purposes: decreasing the uncertainty in parameter estimates [6,7,22,63,78], decreasing the uncertainty in model predictions [32,48], improving controller performance [30], or discriminating between model alternatives [12,98].”
Cited by
Roux, S., Camargo, A. P., Coutinho, F. H., Dabdoub, S. M., Dutilh, B. E., Nayfach, S., & Tritt, A. (2023) · PLOS Biology · doi:10.1371/journal.pbio.3002083 · Cited by 466
Assessing the Value of ChatGPT for Clinical Decision Support Optimization
Liu, S., Wright, A. P., Patterson, B. L., Wanderer, J. P., Turer, R. W., Nelson, S. D., McCoy, A. B., Sittig, D. F., & Wright, A. (2023) · doi:10.1101/2023.02.21.23286254 · Cited by 90
Kagaya, Y., Zhang, Z., Ibtehaz, N., Wang, X., Nakamura, T., Punuru, P. D., & Kihara, D. (2025) · Nature Communications · doi:10.1038/s41467-025-56261-7 · Cited by 70
Tran, N. D., Subramani, K., & Su, C. T.-T. (2025) · mAbs · doi:10.1080/19420862.2025.2531227
Metadata
- Authors
- Villaverde, A. F., Pathirana, D., Fröhlich, F., Hasenauer, J., & Banga, J. R.
- Venue
- Briefings in Bioinformatics
- Publisher
- Oxford University Press (OUP)
- Published
- 2021-10-09
- Volume
- 23
- Issue
- 1
- ISSN
- 1467-5463 (ISSN-L)
- 1477-4054
- OpenAlex
- W3165512555
- Cited by
- 137
- References
- 28
- FWCI
- 4.70
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.