An optimal experimental design approach to model discrimination in dynamic biochemical systems
The DOI matches its Crossref record; 33 of the 35 references it cites were found in a registry or an index, and 1 are not checked yet.
Abstract
Motivation: Finding suitable models of dynamic biochemical systems is an important task in systems biology approaches to the biosciences. On the one hand, a correct model helps to understand the underlying mechanisms and on the other hand, one can use the model to predict the behavior of a biological system under various circumstances. Typically, before the correct model of a biochemical system is found, different hypothetical models might be reasonable and consistent with previous knowledge and available data. The main goal now is to find the best suited model out of different hypotheses. The process of falsifying inappropriate candidate models is called model discrimination. Results: We have developed a new computational tool to compute optimal experiments for biochemical kinetic systems with underlying ordinary differential equation (ODE) models for the purpose of model discrimination. We were inspired by the demands of biological experimentalists which perform one run measurement where perturbations to the system are possible. We provide a criterion which calculates the number and location of time points of optimal measurements as well as optimal initial conditions and optimal perturbations to the system. Availability: The model discrimination algorithm described here is implemented in C++ in the package ModelDiscriminationToolkit. The source code can be downloaded from http://omnibus.unifreiburg.de/~ds500/_software.html Contact: dirk.lebiedz@biologie.uni-freiburg.de Supplementary information: Supplementary data are available at Bioinformatics online.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The topic of optimal design of dynamic experiments in biological systems is receiving increased attention [144–152].”
From Reverse engineering and identification in systems biology: strategies, perspectives and challenges (2014) · cites this paper as [150]
Citations
Cites
Model selection and multimodel inference: a practical information-theoretic approach.
Guthery, F. S., Burnham, K. P., & Anderson, D. R. (2003) · The Journal of Wildlife Management · doi:10.2307/3802723 · Cited by 41,495
Wächter, A., & Biegler, L. T. (2006) · Mathematical Programming · doi:10.1007/s10107-004-0559-y · Cited by 9,483
Introduction to Numerical Analysis.
Froberg, C.-E., Stoer, J., & Bulirsch, R. (1981) · Mathematics of Computation · doi:10.2307/2007454 · Cited by 5,463
Introduction to Numerical Analysis
Stoer, J., & Bulirsch, R. (1980) · Springer eBooks · doi:10.1007/978-1-4757-5592-3 · Cited by 2,154
A Multiple Shooting Algorithm for Direct Solution of Optimal Control Problems *
Bock, H. G., & Plitt, K. J. (1984) · IFAC Proceedings Volumes · doi:10.1016/s1474-6670(17)61205-9 · Cited by 1,306 · cited 1×
“Horn (1987), Stricker (1994) or Timmer et al.”
BackgroundBiochemical Oscillations and Cellular Rhythms: The Molecular Bases of Periodic and Chaotic Behaviour
Goldbeter, A., & Keizer, J. (1998) · Physics Today · doi:10.1063/1.882194 · Cited by 610
Biochemical Oscillations and Cellular Rhythms: The Molecular Bases of Periodic and Chaotic Behaviour
Goldbeter, A., & Berridge, M. J. (1996) · Cambridge University Press eBooks · doi:10.1017/cbo9780511608193 · Cited by 608
The design of experiments for discriminating between two rival models
ATKINSON, A. C., & FEDOROV, V. V. (1975) · Biometrika · doi:10.1093/biomet/62.1.57 · Cited by 301 · cited 2×
“(2007) demonstrate that KL-optimality is consistent with T-optimality (Atkinson and Fedorov (1975)) and generalized T-optimality (Uciński and Bogacka (2004)) which are well known model discrimination criteria.”
BackgroundParametric Statistical Inference
(1981) · Elsevier eBooks · doi:10.1016/c2013-0-03459-7 · Cited by 269
Systems biology: experimental design
Kreutz, C., & Timmer, J. (2009) · The FEBS Journal · doi:10.1111/j.1742-4658.2008.06843.x · Cited by 257
Research works printed but not linked: 5
- Bock, 1987, Randwertproblemmethoden zur Parameteridentifizierung in Systemen nichtlinearer Differentialgleichungen
- HSL, 2007, A collection of fortran codes for large-scale scientific computation.
- Körkel, 1999, A sequential approach for nonlinear optimum experimental design in DAE systems, 338
- Uciński, 2004, Heteroscedastic T-optimum designs for multiresponse dynamic models, 191
- Wächter, 2002, An interior point algorithm for large-scale nonlinear optimization with applications in process engineering
Cited by
Reverse engineering and identification in systems biology: strategies, perspectives and challenges
Villaverde, A. F., & Banga, J. R. (2014) · Journal of The Royal Society Interface · doi:10.1098/rsif.2013.0505 · Cited by 229
“Other methods have exploited sigma-point approaches [151] or Kullback–Leibler optimality [150,152].”
MethodEgea, J. A., Henriques, D., Cokelaer, T., Villaverde, A. F., MacNamara, A., Danciu, D.-P., Banga, J. R., & Saez-Rodriguez, J. (2014) · BMC Bioinformatics · doi:10.1186/1471-2105-15-136 · Cited by 176
“Some recent successful applications of eSS in the field of systems biology can be found in [15-26].”
BackgroundAdvancing metabolic models with kinetic information
Link, H., Christodoulou, D., & Sauer, U. (2014) · Current Opinion in Biotechnology · doi:10.1016/j.copbio.2014.01.015 · Cited by 117
“Other computational methods aim to design informative experiments to decide among structurally different models [52,53].”
MethodOnline optimal experimental re‐design in robotic parallel fed‐batch cultivation facilities
Bournazou, M. N. C., Barz, T., Nickel, D. B., C., D. C. L., Glauche, F., Knepper, A., & Neubauer, P. B. (2016) · Biotechnology and Bioengineering · doi:10.1002/bit.26192 · Cited by 100
“Despite various publications using OED in biotechnology (Franceschini and Macchietto, 2007b; Galvanin et al., 2011; Gernaey et al., 2002; Kreutz and Timmer, 2009; L opez C ardenas et al., 2013; Skanda and Lebiedz, 2010) and design of experiments applied in HTS, to our knowledge, there are no publications dealing with…”
MethodA bayesian approach to targeted experiment design
Vanlier, J., Tiemann, C. A., Hilbers, P. A. J., & van Riel, N. A. W. (2012) · Bioinformatics · doi:10.1093/bioinformatics/bts092 · Cited by 98
“As a last remark we would like to point out that if the goal of the experiment is to discriminate between models, alternative approaches (Skanda and Lebiedz, 2010) could be relevant to explore.”
BackgroundEisenberg, M. C., & Jain, H. V. (2017) · Journal of Theoretical Biology · doi:10.1016/j.jtbi.2017.07.018 · Cited by 79
“…of the ways that identifiability can inform experiment design and data collection strategies, the reader is referred to Kreutz et al. (2012), Skanda and Lebiedz (2010), Anguelova et al. (2012), Cheung et al. (2013), Chu and Hahn (2008), Apgar et al. (2010), Banga and Balsa-Canto (2008),…”
BackgroundMethodNear-optimal experimental design for model selection in systems biology
Busetto, A. G., Hauser, A., Krummenacher, G., Sunnåker, M., Dimopoulos, S., Ong, C. S., Stelling, J., & Buhmann, J. M. (2013) · Bioinformatics · doi:10.1093/bioinformatics/btt436 · Cited by 60
“In classical statistics, ensemble noncentrality constitutes the reference technique to design experiments for model selection [35, 1, 14].”
MethodOptimal experiment design for model selection in biochemical networks
Vanlier, J., Tiemann, C. A., Hilbers, P. A. J., & van Riel, N. A. W. (2014) · BMC Systems Biology · doi:10.1186/1752-0509-8-20 · Cited by 55
“Existing methods of OED for model selection are usually based on assuming an uncertainty distribution around best parameter estimates [24,25] or model linearization [26].”
Simultaneous model discrimination and parameter estimation in dynamic models of cellular systems
Rodriguez-Fernández, M., Rehberg, M., Kremling, A., & Banga, J. R. (2013) · BMC Systems Biology · doi:10.1186/1752-0509-7-76 · Cited by 48
“The related topic of optimal experimental design for parameter estimation [3,13] and for model discrimination [14-16] is receiving increased attention in recent years.”
BackgroundKrausch, N., Barz, T., Sawatzki, A., Gruber, M., Kamel, S., Neubauer, P. B., & Bournazou, M. N. C. (2019) · Frontiers in Bioengineering and Biotechnology · doi:10.3389/fbioe.2019.00122 · Cited by 45
“, the time points when measurements are taken (Skanda and Lebiedz, 2010).”
BackgroundA Bayesian active learning strategy for sequential experimental design in systems biology
Pauwels, E., Lajaunie, C., & Vert, J. (2014) · BMC Systems Biology · doi:10.1186/s12918-014-0102-6 · Cited by 35
“When the goal of the modeling approach is not to estimate the parameters per se, but to understand and simulate the system, other authors have also considered the problem of experimental design to improve the predictions made by the model [39-41], or to discriminate between different candidate models [42-45].”
BackgroundSimultaneous Segmentation and Superquadrics Fitting in Laser-Range Data
Pascoal, R., Santos, V. M. F., Premebida, C., & Nunes, U. (2014) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2014.2321899 · Cited by 32
“The hyperbolic tangent is a C ∞ function and has also been suggested for other optimization problems [25]; thus, the self-occlusion objective function is expressed as where G ∇ takes values in the interval ]0, 1[, with zero being the limit value when all points lie on the visible side of the…”
BackgroundOptimal design of stimulus experiments for robust discrimination of biochemical reaction networks
Flassig, R. J., & Sundmacher, K. (2012) · Bioinformatics · doi:10.1093/bioinformatics/bts585 · Cited by 31
“An OESD aiming at model discrimination for biological systems has been addressed by few authors (Apgar et al ., 2008; Melykuti et al ., 2010; Skanda and Lebiedz, 2010).”
BackgroundMethodNew opportunities for optimal design of dynamic experiments in systems and synthetic biology
Braniff, N., & Ingalls, B. (2018) · Current Opinion in Systems Biology · doi:10.1016/j.coisb.2018.02.005 · Cited by 30
GPdoemd: A Python package for design of experiments for model discrimination
Olofsson, S., Hebing, L., Niedenführ, S., Deisenroth, M. P., & Misener, R. (2019) · Computers & Chemical Engineering · doi:10.1016/j.compchemeng.2019.03.010 · Cited by 29
“Neither do Skanda and Lebiedz (2010, 2013).”
Prediction uncertainty and optimal experimental design for learning dynamical systems
Letham, B., Letham, P. A., Rudin, C. D., & Browne, E. P. (2016) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/1.4953795 · Cited by 28
“Optimal experimental design has typically been studied in the context of parameter estimation 14–16 or, more recently, model selection and discrimination 17–21 .”
BackgroundOptimization of Time-Course Experiments for Kinetic Model Discrimination
Lages, N., CORDEIRO, C. A., Silva, M. S., Freire, A. P., & Ferreira, A. E. N. (2012) · PLoS ONE · doi:10.1371/journal.pone.0032749 · Cited by 21
“This idea has been explored before for two candidatemodel problems [2,3,4,5].”
BackgroundInput design for discrimination between classes of LTI models
Cheong, S., & Manchester, I. R. (2015) · Automatica · doi:10.1016/j.automatica.2014.12.005 · Cited by 20
“In contrast, Skanda and Lebiedz (2010) proposes maximizing the Kullback–Leibler (K–”
BackgroundOptimal experimental design for parameter identification and model selection
Schenkendorf, R. (2014) · MPG.PuRe (Max Planck Society) · doi:10.25673/4085 · Cited by 18
Online model selection approach based on Unscented Kalman Filtering
Schenkendorf, R., & Mangold, M. (2012) · Journal of Process Control · doi:10.1016/j.jprocont.2012.10.009 · Cited by 17
“Here, Optimal Experimental Design (OED) comes into play [6, 7, 8].”
Robust optimal design of experiments for model discrimination using an interactive software tool
Stegmaier, J., Skanda, D., & Lebiedz, D. (2013) · PLoS ONE · doi:10.1371/journal.pone.0055723 · Cited by 15
“A detailed derivation of the adjusted optimization criterion as well as the statistical background can be found in [10,13].”
BackgroundMethodInfluentialSelf-citationA robust optimization approach to experimental design for model discrimination of dynamical systems
Skanda, D., & Lebiedz, D. (2012) · Mathematical Programming · doi:10.1007/s10107-012-0532-0 · Cited by 14
“Building on our previous work [43] we present an algorithm to compute robust optimal experimental designs.”
MethodSelf-citationIdentifying Optimal Models to Represent Biochemical Systems
Apri, M., de Gee, M., van Mourik, S., & Molenaar, J. (2014) · PLoS ONE · doi:10.1371/journal.pone.0083664 · Cited by 11
“To check the predictive power of both models, an optimal experimental design approach to discrimate the models is utilized [19–22].”
MethodMartin-Casas, M., & Mesbah, A. (2016) · IEEE Life Sciences Letters · doi:10.1109/lls.2016.2644645 · Cited by 10
“idation of model structures that cannot adequately describe experimental data (see [2] and [3]).”
BackgroundOptimum Experimental Design by Shape Optimization of Specimens in Linear Elasticity
Etling, T., & Herzog, R. (2018) · SIAM Journal on Applied Mathematics · doi:10.1137/17m1147743 · Cited by 10
Keesman, K. J., & Walter, E. (2014) · Automatica · doi:10.1016/j.automatica.2014.03.022 · Cited by 9
“Applications of experiment design for discrimination can be found in domains as diverse as chemistry Schwaab et al. (2006), machine learning Rajamoney (1993), system biology Kreutz and Timmer (2009), Skanda and Lebiedz (2010) and psychology Myung and Pitt (2009).”
BackgroundOptimal Test Selection for Prediction Uncertainty Reduction
Mullins, J., Mahadevan, S., & Urbina, A. (2016) · Journal of Verification Validation and Uncertainty Quantification · doi:10.1115/1.4035204 · Cited by 8
Input design for model discrimination and fault detection via convex relaxation
Cheong, S., & Manchester, I. R. (2014) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.1109/acc.2014.6859400 · Cited by 7
“In contrast, [11] proposes maximizing the Kullback-Leibler (K-L) divergence [12] of the probability density functions (PDFs) of the output signals of models from each other, assuming that initial conditions of the system can be chosen and a measurement noise signal is an iid random process with a normal distribution.”
Robust optimal experimental design for model discrimination of kinetic ODE systems
Skanda, D. (2012) · FreiDok plus (Universitätsbibliothek Freiburg) · Cited by 7
Self-citationModel Discrimination in Dynamic Molecular Systems: Application to Parotid De-differentiation Network
Kim, J., Li, J., Venkatesh, S. G., Darling, D. S., & Rempała, G. A. (2013) · Journal of Computational Biology · doi:10.1089/cmb.2011.0222 · Cited by 6
“…(BF) criterion (see, e.g., Hastie et al., 2001, Chapter 7; for a possible alternative based on Bayesian information (see, e.g., Jones, 2011, and Skanda and Lebiedz, 2010) where both the numerator and the denominator integrals are now approximated by the numerically averaging p ( y j h i ‚ M i…”
Robust Model Selection: Flatness-Based Optimal Experimental Design for a Biocatalytic Reaction
Schulze, M., & Schenkendorf, R. (2020) · Processes · doi:10.3390/pr8020190 · Cited by 7
Design of Dynamic Experiments for Black-Box Model Discrimination.Preprint
Olofsson, S., Schultz, E. S., Mhamdi, A., Mitsos, A., Deisenroth, M. P., & Misener, R. (2021) · arXiv (Cornell University) · Cited by 6
Robust Optimal Design of Synthetic Biological Networks
Lebiedz, D., Rehberg, M., & Skanda, D. (2011) · Methods in molecular biology · doi:10.1007/978-1-61779-412-4_3 · Cited by 5
Self-citationNumerical aspects of uncertainty in the design of optimal experiments for model discrimination
Hoffmann, C. (2017) · heiDOK (Heidelberg University) · doi:10.11588/heidok.00022612 · Cited by 4
Set-based Experiment Design for Model Discrimination Using Bilevel Optimization
Rudolph, N., Streif, S., & Findeisen, R. (2016) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2016.12.142 · Cited by 3
“Other noteworthy examples regarding experiment design for model discrimination are, e. g., Skanda and Lebiedz (2010) and Streif et al. (2014).”
BackgroundOptimal input design for model discrimination
Keesman, K. J., & Walter, E. (2012) · IFAC Proceedings Volumes · doi:10.3182/20120711-3-be-2027.00126 · Cited by 3
Simplified Reaction Models for Combustion in Gas Turbine Combustion Chambers
Lebiedz, D., & Siehr, J. (2012) · Fluid mechanics and its applications · doi:10.1007/978-94-007-5320-4_5 · Cited by 3
Self-citationOlofsson, S., Schultz, E. S., Mhamdi, A., Mitsos, A., Deisenroth, M. P., & Misener, R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.03782 · Cited by 3
“None of Espie and Macchietto (1989), Chen and Asprey (2003), or Skanda and Lebiedz (2010, 2013) consider process noise or uncertainty in the initial state x (0) = (cid:126)x 0 or control signal u ( t ).”
BackgroundInfluentialWang, X., & Zhang, Y. (2016) · Communication in Statistics- Theory and Methods · doi:10.1080/03610926.2014.957860 · Cited by 2
“In the last decades, the use of optimal experimental design has gained world-wide acceptance as an essential tool for improving the quality and efficiency of model identification and parameter estimation (Whitile, 1972; Versyck et al., 1998; Jin et al., 2005; Skanda and Lebiedz, 2010).”
MethodStatistical model identification - dynamical processes and large-scale networks in systems biology
Flassig, R. (2014) · Digitalen Hochschulbibliothek Sachsen-Anhalt (Universitäts- und Landesbibliothek Sachsen-Anhalt) · doi:10.25673/4147 · Cited by 2
Parameter and topology uncertainty for optimal experimental design
Hagen, D. (2014) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 1
Optimal Experimental Design Applied to Models of Microbial Gene Regulation
Braniff, N. (2020) · UWSpace (University of Waterloo) · Cited by 1
Boerries, M., Eils, R., & Busch, H. S. (2011) · Wiley-VCH eBooks · doi:10.1002/3527600906.mcb.200500051.pub2 · Cited by 1
Health monitoring of Gas turbine engines: Framework design and strategies
Relan, R. (2015) · White Rose eTheses Online (University of Leeds, The University of Sheffield, University of York)
Model Discrimination Using Bounded Error Estimation Approach
Chattopadhyay, S. (2015) · NCSU Libraries Repository (North Carolina State University Libraries)
Tackling complex models in systems biology
Apri, M. (2013) · doi:10.18174/247957
Huschto, T. (2014) · heiDOK (Heidelberg University) · doi:10.11588/heidok.00017779
Bayesian experiment design and estimation for probabilistic modeling of biological systems
Martin-Casas, M. (2019) · eScholarship (California Digital Library)
Metadata
- Authors
- Skanda, D., & Lebiedz, D.
- Affiliations
- University of Freiburg (DE)
- ORCID
- Lebiedz, Dirk
- Venue
- Bioinformatics
- Publisher
- Oxford University Press (OUP)
- Published
- 2010-02-21
- Volume
- 26
- Issue
- 7
- Pages
- 939-945
- ISSN
- 1367-4803 (ISSN-L)
- 1367-4811
- OpenAlex
- W2144197118
- Cited by
- 42
- References
- 35
- Citations by year
- 2021: 2 · 2020: 2 · 2019: 2 · 2018: 2 · 2017: 1 · 2016: 6 · 2015: 1 · 2014: 11 · 2013: 6 · 2012: 7
- FWCI
- 1.95
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Bioinformatics (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene Regulatory Network Analysis
- Social Sciences › Decision Sciences › Management Science and Operations Research › Optimal Experimental Design Methods
- Physical Sciences › Engineering › Control and Systems Engineering › Control Systems and Identification
- Keywords
- model discrimination, optimal experimental design, ordinary differential equations, time point selection, optimal perturbations, optimal initial conditions, systems biology
- MeSH terms
- Algorithms, Computational Biology, Systems Biology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.