A Global Parallel Model Based Design of Experiments Method to Minimize Model Output Uncertainty
The DOI matches its Crossref record; 53 of the 53 references it cites were found in a registry or an index.
Abstract
Model-based experiment design specifies the data to be collected that will most effectively characterize the biological system under study. Existing model-based design of experiment algorithms have primarily relied on Fisher Information Matrix-based methods to choose the best experiment in a sequential manner. However, these are largely local methods that require an initial estimate of the parameter values, which are often highly uncertain, particularly when data is limited. In this paper, we provide an approach to specify an informative sequence of multiple design points (parallel design) that will constrain the dynamical uncertainty of the biological system responses to within experimentally detectable limits as specified by the estimated experimental noise. The method is based upon computationally efficient sparse grids and requires only a bounded uncertain parameter space; it does not rely upon initial parameter estimates. The design sequence emerges through the use of scenario trees with experimental design points chosen to minimize the uncertainty in the predicted dynamics of the measurable responses of the system. The algorithm was illustrated herein using a T cell activation model for three problems that ranged in dimension from 2D to 19D. The results demonstrate that it is possible to extract useful information from a mathematical model where traditional model-based design of experiments approaches most certainly fail. The experiments designed via this method fully constrain the model output dynamics to within experimentally resolvable limits. The method is effective for highly uncertain biological systems characterized by deterministic mathematical models with limited data sets. Also, it is highly modular and can be modified to include a variety of methodologies such as input design and model discrimination.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Experimental design with a continuous set of experiments requires a different approach, as for example shown in [50].”
From Maximizing the information content of experiments in systems biology (2013) · cites this paper as [50]
Citations
Cites
Universally Sloppy Parameter Sensitivities in Systems Biology Models
Gutenkunst, R. N., Waterfall, J. J., Casey, F. P., Brown, K. S., Myers, C. R., & Sethna, J. P. (2007) · PLoS Computational Biology · doi:10.1371/journal.pcbi.0030189 · Cited by 1,261
Model-based design of experiments for parameter precision: State of the art
Franceschini, G., & Macchietto, S. (2008) · Chemical Engineering Science · doi:10.1016/j.ces.2007.11.034 · Cited by 818
Generating Scenario Trees for Multistage Decision Problems
Høyland, K., & Wallace, S. W. (2001) · Management Science · doi:10.1287/mnsc.47.2.295.9834 · Cited by 673
Dimension?Adaptive Tensor?Product Quadrature
Gerstner, T., & Griebel, M. (2003) · Computing · doi:10.1007/s00607-003-0015-5 · Cited by 544
Discrimination Among Mechanistic Models
Box, G. E. P., & Hill, W. J. (1967) · Technometrics · doi:10.1080/00401706.1967.10490441 · Cited by 426
Parameter estimation and model selection in computational biology
Lillacci, G., & Khammash, M. (2010) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000696 · Cited by 325 · cited 1×
“…Cho et al. (2003), Kutalik et al. (2004), Joshi et al. (2006), Rodriguez-Fernandez et al. (2006), Gutenkunst et al. (2007), Chu and Hahn (2008), Lillacci and Khammash (2010), Van Derlinden et al. (2010) and/or discriminate between rival models as focused on in Box and Hill (1967), Pritchard…”
BackgroundA hybrid approach for efficient and robust parameter estimation in biochemical pathways
Rodriguez-Fernandez, M., Mendes, P., & Banga, J. R. (2006) · Biosystems · doi:10.1016/j.biosystems.2005.06.016 · Cited by 304 · cited 1×
“…(2008a), Pronzato and Walter (1994), Beck and Woodbury (1998), Emery et al. (2000), Cho et al. (2003), Kutalik et al. (2004), Joshi et al. (2006), Rodriguez-Fernandez et al. (2006), Gutenkunst et al. (2007), Chu and Hahn (2008), Lillacci and Khammash (2010), Van Derlinden et al. (2010) and/or…”
BackgroundSystems biology: experimental design
Kreutz, C., & Timmer, J. (2009) · The FEBS Journal · doi:10.1111/j.1742-4658.2008.06843.x · Cited by 257 · cited 2×
“This approach is generally called model-based design of optimal experiments (MBDOE) or quantitative experiment design (Franceschini and Macchietto 2008a; Galvanin et al. 2007; Kreutz and Timmer 2009; Pronzato 2008).”
BackgroundMethodOptimal experimental design and some related control problems
Pronzato, L. (2008) · Automatica · doi:10.1016/j.automatica.2007.05.016 · Cited by 243
An iterative identification procedure for dynamic modeling of biochemical networks
Balsa-Canto, E., Alonso, A. A., & Banga, J. R. (2010) · BMC Systems Biology · doi:10.1186/1752-0509-4-11 · Cited by 202 · cited 1×
“Global methods are emerging to circumvent this limitation and are under active investigation (Cho et al. 2003; Joshi et al. 2006; Donahue et al. 2010; Balsa-Canto et al. 2010; Chu and Hahn 2010).”
MethodDesigning robust optimal dynamic experiments
Asprey, S. P., & Macchietto, S. (2002) · Journal of Process Control · doi:10.1016/s0959-1524(01)00020-8 · Cited by 199 · cited 1×
“The sequential approach is far more suited to systems in which data is fairly easy and inexpensive to obtain such as with continuous feedback and control with computer interface (as in the case of many bioreactors) (Asprey and Macchietto 2002).”
BackgroundAlgorithm 847: Spinterp: piecewise multilinear hierarchical sparse grid interpolation in MATLAB
Klimke, A., & Wohlmuth, B. (2005) · ACM Transactions on Mathematical Software · doi:10.1145/1114268.1114275 · Cited by 197
Exploiting the bootstrap method for quantifying parameter confidence intervals in dynamical systems
Joshi, M., Seidel-Morgenstern, A., & Kremling, A. (2006) · Metabolic Engineering · doi:10.1016/j.ymben.2006.04.003 · Cited by 190
Cho, K.-H., Shin, S.-Y., Kolch, W., & Wolkenhauer, O. (2003) · SIMULATION · doi:10.1177/0037549703040943 · Cited by 172 · cited 2×
“…model parameters as implemented in Franceschini and Macchietto (2008a), Pronzato and Walter (1994), Beck and Woodbury (1998), Emery et al. (2000), Cho et al. (2003), Kutalik et al. (2004), Joshi et al. (2006), Rodriguez-Fernandez et al. (2006), Gutenkunst et al. (2007), Chu and Hahn (2008),…”
MethodIterative approach to model identification of biological networks
Gadkar, K. G., Gunawan, R., & Doyle, F. J. (2005) · BMC Bioinformatics · doi:10.1186/1471-2105-6-155 · Cited by 156
Simulation and optimization approaches to scenario tree generation
Gülpınar, N., Rüstem, B., & Settergren, R. (2003) · Journal of Economic Dynamics and Control · doi:10.1016/s0165-1889(03)00113-1 · Cited by 154
Inverse problems and parameter estimation: integration of measurements and analysis
Beck, J. V., & Woodbury, K. A. (1998) · Measurement Science and Technology · doi:10.1088/0957-0233/9/6/001 · Cited by 151
R.A. Fisher and the Design of Experiments, 1922–1926
Box, J. F. (1980) · The American Statistician · doi:10.1080/00031305.1980.10482701 · Cited by 148
European symposium on computer aided process engineering
(2004) · Chemical Engineering Journal · doi:10.1016/j.cej.2004.06.001 · Cited by 141
Optimal Experimental Design for Parameter Estimation of a Cell Signaling Model
Bandara, S., Schlöder, J. P., Eils, R., Bock, H. G., & Meyer, T. (2009) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000558 · Cited by 135 · cited 1×
“Existing MBDOE algorithms have primarily relied on the Fisher Information Matrix (FIM) to design the best experiment (Kutalik et al. 2004; Gutenkunst et al. 2007; Franceschini and Macchietto 2005, 2007; Balsa-Canto et al. 2008; Bandara et al. 2009; Casey et al. 2006).”
MethodKremling, A., Fischer, S., Gadkar, K., Doyle, F. J., Sauter, T., Bullinger, E., Allgöwer, F., & Gilles, E. D. (2004) · Genome Research · doi:10.1101/gr.1226004 · Cited by 127 · cited 1×
“…Hahn (2008), Lillacci and Khammash (2010), Van Derlinden et al. (2010) and/or discriminate between rival models as focused on in Box and Hill (1967), Pritchard and Bacon (1974), Ferraris et al. (1984), Chen and Asprey (2003), Kremling et al. (2004), Vatcheva et al. (2006), Donckels et al. (2009).”
BackgroundComputational procedures for optimal experimental design in biological systems
Balsa-Canto, E., Alonso, A. A., & Banga, J. R. (2008) · IET Systems Biology · doi:10.1049/iet-syb:20070069 · Cited by 124 · cited 1×
“Existing MBDOE algorithms have primarily relied on the Fisher Information Matrix (FIM) to design the best experiment (Kutalik et al. 2004; Gutenkunst et al. 2007; Franceschini and Macchietto 2005, 2007; Balsa-Canto et al. 2008; Bandara et al. 2009; Casey et al. 2006).”
MethodModel-Based Design of Parallel Experiments
Galvanin, F., Macchietto, S., & Bezzo, F. (2007) · Industrial & Engineering Chemistry Research · doi:10.1021/ie0611406 · Cited by 107
The optimal design of dynamic experiments
Espie, D., & Macchietto, S. (1989) · AIChE Journal · doi:10.1002/aic.690350206 · Cited by 107
An error model for protein quantification
Kreutz, C., Rodriguez, M. M. B., Maiwald, T., Seidl, M., Blum, H. E., Mohr, L., & Timmer, J. (2007) · Bioinformatics · doi:10.1093/bioinformatics/btm397 · Cited by 107
Stochastic effects and bistability in T cell receptor signaling
Tomasz Lipniacki, B., Hat, B., Faeder, J. R., & Hlavacek, W. S. (2008) · Journal of Theoretical Biology · doi:10.1016/j.jtbi.2008.05.001 · Cited by 96
One hundred years of the design of experiments on and off the pages of Biometrika
Atkinson, A. C. (2001) · Biometrika · doi:10.1093/biomet/88.1.53 · Cited by 86
Prospects for reducing correlations among parameter estimates in kinetic models
Pritchard, D. J., & Bacon, D. W. (1978) · Chemical Engineering Science · doi:10.1016/0009-2509(78)85205-1 · Cited by 84
Discrimination among Mechanistic Models
Box, G. E. P., & Hill, W. J. (1967) · Technometrics · doi:10.2307/1266318 · Cited by 83
Experiment selection for the discrimination of semi-quantitative models of dynamical systems
Vatcheva, I., de Jong, H., Bernard, O., & Mars, N. J. I. (2006) · Artificial Intelligence · doi:10.1016/j.artint.2005.11.001 · Cited by 82
Sequential experimental design for model discrimination in the case of multiple responses
Ferraris, G. B., Forzatti, P., Emig, G., & Hofmann, H. (1984) · Chemical Engineering Science · doi:10.1016/0009-2509(84)80132-3 · Cited by 81
Optimal sampling time selection for parameter estimation in dynamic pathway modeling
Kutalik, Z., Cho, K.-H., & Wolkenhauer, O. (2004) · Biosystems · doi:10.1016/j.biosystems.2004.03.007 · Cited by 79
Casey, F. P., Baird, D., Feng, Q., Gutenkunst, R. N., Waterfall, J. J., Myers, C. R., Brown, K. S., Cerione, R. A., & Sethna, J. P. (2007) · IET Systems Biology · doi:10.1049/iet-syb:20060065 · Cited by 77
Chen, B. H., & Asprey, S. P. (2003) · Industrial & Engineering Chemistry Research · doi:10.1021/ie0203025 · Cited by 72
Novel anticorrelation criteria for model‐based experiment design: Theory and formulations
Franceschini, G., & Macchietto, S. (2008) · AIChE Journal · doi:10.1002/aic.11429 · Cited by 67
Uncertainties in parameter estimation: the optimal experiment design
Emery, A. F., Nenarokomov, A. V., & Fadale, T. D. (2000) · International Journal of Heat and Mass Transfer · doi:10.1016/s0017-9310(99)00378-6 · Cited by 64
Validation of a Model for Biodiesel Production through Model-Based Experiment Design
Franceschini, G., & Macchietto, S. (2006) · Industrial & Engineering Chemistry Research · doi:10.1021/ie060758c · Cited by 61
Variance-Based Global Sensitivity Analysis via Sparse-Grid Interpolation and Cubature
Buzzard, G. T., & Xiu, D. (2010) · Communications in Computational Physics · doi:10.4208/cicp.230909.160310s · Cited by 59
Self-citationR. A. Fisher and Experimental Design: A Review
Preece, D. A. (1990) · Biometrics · doi:10.2307/2532438 · Cited by 57
An anticipatory approach to optimal experimental design for model discrimination
Donckels, B. M. R., Pauw, D. J. W. D., Baets, B. D., Maertens, J., & Vanrolleghem, P. A. (2008) · Chemometrics and Intelligent Laboratory Systems · doi:10.1016/j.chemolab.2008.08.002 · Cited by 41
Chu, Y., & Hahn, J. (2008) · AIChE Journal · doi:10.1002/aic.11562 · Cited by 41
R. A. Fisher and the Design of Experiments, 1922-1926
Box, J. F. (1980) · The American Statistician · doi:10.2307/2682986 · Cited by 38
Derlinden, E. V., Bernaerts, K., & van Impe, J. F. M. (2010) · Journal of Theoretical Biology · doi:10.1016/j.jtbi.2010.01.003 · Cited by 35
Franceschini, G., & Macchietto, S. (2008) · Industrial & Engineering Chemistry Research · doi:10.1021/ie071053t · Cited by 31
Minimum-volume ellipsoids containing compact sets
Pronzato, L., & Walter, E. (1994) · Automatica · doi:10.1016/0005-1098(94)90075-2 · Cited by 27
He, F., Brown, M., & Yue, H. (2010) · International Journal of Robust and Nonlinear Control · doi:10.1002/rnc.1558 · Cited by 24
Recent advances in multiple‐objective design strategies
Wong, W. K. (1999) · Statistica Neerlandica · doi:10.1111/1467-9574.00111 · Cited by 23
Donahue, M. M., Buzzard, G. T., & Rundell, A. E. (2010) · IET Systems Biology · doi:10.1049/iet-syb.2009.0031 · Cited by 20
Self-citationQuantitative Optimal Experimental Design Using Global Sensitivity Analysis via Quasi-Linearization
Chu, Y., & Hahn, J. (2010) · Industrial & Engineering Chemistry Research · doi:10.1021/ie9009827 · Cited by 18
Potential pitfalls in model discrimination
Pritchard, D. J., & Bacon, D. W. (1974) · The Canadian Journal of Chemical Engineering · doi:10.1002/cjce.5450520116 · Cited by 13
Optimal experimental design in an EGFR signaling and down-regulation modelPreprint
Casey, F., Baird, D., Feng, Q., et al. (2006) · arXiv (Cornell University) · doi:10.48550/arxiv.q-bio/0610024 · Cited by 9
A numerical experiment design study on a biodiesel production process
Franceschini, G., & Macchietto, S. (2005) · Computer-aided chemical engineering/Computer aided chemical engineering · doi:10.1016/s1570-7946(05)80180-4 · Cited by 4
An integrated bioenergetics modeling approach to mitochondrial permeability transition
Bazil, J. N. (2010) · Purdue e-Pubs (Purdue University System) · Cited by 1
Self-citation
Cited by
Maximizing the information content of experiments in systems biology
Liepe, J., Filippi, S., Komorowski, M., & Stumpf, M. P. H. (2013) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002888 · Cited by 170
“approach, as for example shown in [50].”
The Inferred Cardiogenic Gene Regulatory Network in the Mammalian Heart
Bazil, J. N., Stamm, K. D., Li, X., Thiagarajan, R., Nelson, T. J., Tomita-Mitchell, A., & Beard, D. A. (2014) · PLoS ONE · doi:10.1371/journal.pone.0100842 · Cited by 8
Self-citation
Metadata
- Authors
- Bazil, J. N., Buzzard, G. T., & Rundell, A. E.
- Affiliations
- Purdue University West Lafayette (US)
- Corresponding author
- Rundell, Ann E.
- ORCID
- Bazil, Jason N.
- Venue
- Bulletin of Mathematical Biology
- Published
- 2011-10-12
- Volume
- 74
- Issue
- 3
- Pages
- 688-716
- ISSN
- 0092-8240 (ISSN-L)
- 1522-9602
- OpenAlex
- W2039284810
- Cited by
- 28
- References
- 53
- Citations by year
- 2024: 2 · 2022: 2 · 2020: 1 · 2019: 2 · 2018: 4 · 2017: 2 · 2016: 1 · 2015: 1 · 2014: 6 · 2013: 6
- FWCI
- 5.63
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Bulletin of Mathematical Biology (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Science Foundation
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Optimal Experimental Design Methods
- Physical Sciences › Engineering › Control and Systems Engineering › Control Systems and Identification
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene Regulatory Network Analysis
- Keywords
- model-based design of experiments, sparse grids, scenario tree, model output uncertainty, parameter uncertainty, biological systems, Fisher information matrix, model discrimination, input design, high-dimensional parameter spaces
- MeSH terms
- Algorithms, Lymphocyte Activation, Models, Biological, Receptors, Antigen, T-Cell, Research Design, T-Lymphocytes, Systems Biology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.