Trajectory-oriented Bayesian experiment design versus Fisher A-optimal design: an in depth comparison study
Open accessThe DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
Abstract
Motivation: Experiment design strategies for biomedical models with the purpose of parameter estimation or model discrimination are in the focus of intense research. Experimental limitations such as sparse and noisy data result in unidentifiable parameters and render-related design tasks challenging problems. Often, the temporal resolution of data is a limiting factor and the amount of possible experimental interventions is finite. To address this issue, we propose a Bayesian experiment design algorithm to minimize the prediction uncertainty for a given set of experiments and compare it to traditional A-optimal design. Results: In an in depth numerical study involving an ordinary differential equation model of the trans-Golgi network with 12 partly non-identifiable parameters, we minimized the prediction uncertainty efficiently for predefined scenarios. The introduced method results in twice the prediction precision as the same amount of A-optimal designed experiments while introducing a useful stopping criterion. The simulation intensity of the algorithm's major design step is thereby reasonably affordable. Besides smaller variances in the predicted trajectories compared with Fisher design, we could also achieve smaller parameter posterior distribution entropies, rendering this method superior to A-optimal Fisher design also in the parameter space. Availability: Necessary software/toolbox information are available in the supplementary material. The project script including example data can be downloaded from http://www.ist.uni-stuttgart.de/%7eweber/BayesFisher2012. Contact: patrick.weber@ist.uni-stuttgart.de Supplementary Information: Supplementary data are available at Bioinformatics online.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures






Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Techniques for this exist in both a frequentist [13,36,91] as well as a Bayesian setting [87,90].”
From Parameter uncertainty in biochemical models described by ordinary differential equations (2013) · cites this paper as [90]
In the citing paper
“This helps uncovering how different predictions in the model relate to each other dynamically and therefore gives insights in how the system behaves [89,90].”
From Parameter uncertainty in biochemical models described by ordinary differential equations (2013) · cites this paper as [90]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Bayesian Experimental Design: A Review
Chaloner, K., & Verdinelli, I. (1995) · Statistical Science · doi:10.1214/ss/1177009939 · Cited by 1,839 · cited 2×
“, 2009), Bayesian methods (Chaloner and Verdinelli, 1995; Kramer and Radde, 2010; Wilkinson, 2007) and optimization-based methods (Kreutz et al.”
MethodHaario, H., Laine, M., Mira, A., & Saksman, E. (2006) · Statistics and Computing · doi:10.1007/s11222-006-9438-0 · Cited by 1,498
Wheeler, R. E., Atkinson, A. C., & Donev, A. N. (1993) · Technometrics · doi:10.2307/1270278 · Cited by 1,447
Inhibition of Translational Initiation by Let-7 MicroRNA in Human Cells
Pillai, R. S., Bhattacharyya, S. N., Artus, C. G., Zoller, T., Cougot, N., Basyuk, E., Bertrand, E., & Filipowicz, W. (2005) · Science · doi:10.1126/science.1115079 · Cited by 1,432
Universally Sloppy Parameter Sensitivities in Systems Biology Models
Gutenkunst, R. N., Waterfall, J. J., Casey, F. P., Brown, K. S., Myers, C. R., & Sethna, J. P. (2007) · PLoS Computational Biology · doi:10.1371/journal.pcbi.0030189 · Cited by 1,261 · cited 1×
“Results: In an in depth numerical study involving an ordinary differential equation model of the trans-Golgi network with 12 partly non-identifiable parameters, we minimized the prediction uncertainty efficiently for predefined scenarios.”
BackgroundAtkinson, A. C., & Donev, A. N. (1992) · doi:10.1093/oso/9780198522546.001.0001 · Cited by 752
Li, C., Donizelli, M., Rodriguez, N., Dharuri, H., Endler, L., Chelliah, V., Li, L., He, E., Henry, A., Stefan, M. I., et al. (2010) · BMC Systems Biology · doi:10.1186/1752-0509-4-92 · Cited by 550 · cited 1×
“Experimental limitations such as sparse and noisy data result in unidentifiable parameters and renderrelated design tasks challenging problems.”
BackgroundSystems biology toolbox for matlab: a computational platform for research in systems biology
Schmidt, H., & Jirstrand, M. (2006) · Bioinformatics · doi:10.1093/bioinformatics/bti799 · Cited by 473 · cited 2×
“, 2006) have been combined with the fast ODE integrators of the SBtoolbox2 (Schmidt and Jirstrand, 2006) and the parallel computing capabilities of Matlab.”
Quantifying Western blots: Pitfalls of densitometry
Gassmann, M., Grenacher, B., Rohde, B., & Vogel, J. (2009) · ELECTROPHORESIS · doi:10.1002/elps.200800720 · Cited by 365 · cited 1×
“3.1 A model for secretion control at the trans-Golgi network Protein secretion in mammalian cells is a highly regulated process, in which the regulation of the formation of transport vesicles at the trans-Golgi network plays a crucial role.”
BackgroundInfluentialHausser, A., Storz, P., Märtens, S., Link, G., Toker, A., & Pfizenmaier, K. (2005) · Nature Cell Biology · doi:10.1038/ncb1289 · Cited by 333 · cited 1×
“Both PKD and DAG have also been shown to play a direct role in vesicle formation (Bard and Malhotra, 2006; Hausser et al., 2005).”
BackgroundProteome Half-Life Dynamics in Living Human Cells
Eden, E., Geva-Zatorsky, N., Issaeva, I., Cohen, A., Dekel, E., Danon, T., Cohen, L., Mayo, A., & Alon, U. (2011) · Science · doi:10.1126/science.1199784 · Cited by 326
The statistical mechanics of complex signaling networks: Nerve growth factor signaling.
Brown, K. S., Hill, C. C., Calero, G. A., Myers, C. R., Lee, K. H., Sethna, J. P., & Cerione, R. A. (2004) · Physical Biology · doi:10.1088/1478-3967/1/3/006 · Cited by 265 · cited 1×
“Besides smaller variances in the predicted trajectories compared with Fisher design, we could also achieve smaller parameter posterior distribution entropies, rendering this method superior to A-optimal Fisher design also in the parameter space.”
MethodBayesian methods in bioinformatics and computational systems biology
Wilkinson, D. J. (2006) · Briefings in Bioinformatics · doi:10.1093/bib/bbm007 · Cited by 230 · cited 2×
“…several experiment design methods have been successfully developed including classical Fisher information matrix (FIM) design (Bandara et al., 2009), Bayesian methods (Chaloner and Verdinelli, 1995; Kramer and Radde, 2010; Wilkinson, 2007) and optimization-based methods (Kreutz et al., 2011).”
MethodThe Formation of TGN-to-Plasma-Membrane Transport Carriers
Bard, F., & Malhotra, V. (2006) · Annual Review of Cell and Developmental Biology · doi:10.1146/annurev.cellbio.21.012704.133126 · Cited by 209 · cited 2×
“DAG is located in the Golgi membrane and recruits and thereby activates PKD (Bard and Malhotra, 2006).”
BackgroundFugmann, T., Hausser, A., Schöffler, P., Schmid, S., Pfizenmaier, K., & Olayioye, M. A. (2007) · The Journal of Cell Biology · doi:10.1083/jcb.200612017 · Cited by 200
Optimal Experimental Design for Parameter Estimation of a Cell Signaling Model
Bandara, S., Schlöder, J. P., Eils, R., Bock, H. G., & Meyer, T. (2009) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000558 · Cited by 135 · cited 1×
“The introduced method results in twice the prediction precision as the same amount of A-optimal designed experiments while introducing a useful stopping criterion.”
MethodLikelihood based observability analysis and confidence intervals for predictions of dynamic models
Kreutz, C., Raue, A., & Timmer, J. (2012) · BMC Systems Biology · doi:10.1186/1752-0509-6-120 · Cited by 134 · cited 1×
“The simulation intensity of the algorithm’s major design step is thereby reasonably affordable.”
BackgroundAn error model for protein quantification
Kreutz, C., Rodriguez, M. M. B., Maiwald, T., Seidl, M., Blum, H. E., Mohr, L., & Timmer, J. (2007) · Bioinformatics · doi:10.1093/bioinformatics/btm397 · Cited by 107
A bayesian approach to targeted experiment design
Vanlier, J., Tiemann, C. A., Hilbers, P. A. J., & van Riel, N. A. W. (2012) · Bioinformatics · doi:10.1093/bioinformatics/bts092 · Cited by 98
BECKER, E., FLORIN, L., PFIZENMAIER, K., & KAUFMANN, H. (2008) · Journal of Biotechnology · doi:10.1016/j.jbiotec.2008.03.008 · Cited by 86
Casey, F. P., Baird, D., Feng, Q., Gutenkunst, R. N., Waterfall, J. J., Myers, C. R., Brown, K. S., Cerione, R. A., & Sethna, J. P. (2007) · IET Systems Biology · doi:10.1049/iet-syb:20060065 · Cited by 77 · cited 1×
“The project script including example data can be downloaded from http://www.ist.unistuttgart.de/%7eweber/BayesFisher2012.”
MethodAn empirical Bayesian approach for model-based inference of cellular signaling networks
Klinke, D. J. (2009) · BMC Bioinformatics · doi:10.1186/1471-2105-10-371 · Cited by 68
Donckels, B. M. R., Pauw, D. J. W. D., Vanrolleghem, P. A., & Baets, B. D. (2011) · Chemometrics and Intelligent Laboratory Systems · doi:10.1016/j.chemolab.2011.06.008 · Cited by 18
Donckels, B. M. R., Pauw, D. J. W. D., Vanrolleghem, P. A., & Bernard De Baets, C. (2009) · Journal of Computational Chemistry · doi:10.1002/jcc.21171 · Cited by 13
Kramer, A., & Radde, N. E. (2010) · Procedia Computer Science · doi:10.1016/j.procs.2010.04.184 · Cited by 10 · cited 2×
“, 2009), Bayesian methods (Chaloner and Verdinelli, 1995; Kramer and Radde, 2010; Wilkinson, 2007) and optimization-based methods (Kreutz et al.”
MethodSelf-citation
Cited by
Parameter uncertainty in biochemical models described by ordinary differential equations
Vanlier, J., Tiemann, C. A., Hilbers, P. A. J., & van Riel, N. A. W. (2013) · Mathematical Biosciences · doi:10.1016/j.mbs.2013.03.006 · Cited by 116
The Limitations of Model-Based Experimental Design and Parameter Estimation in Sloppy Systems
White, A., Tolman, M., Thames, H. D., Withers, H. R., Mason, K. A., & Transtrum, M. K. (2016) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1005227 · Cited by 124
Eriksson, O., Jauhiainen, A., Sasane, S. M., Kramer, A., Nair, A. G., Sartorius, C., & Kotaleski, J. H. (2018) · Bioinformatics · doi:10.1093/bioinformatics/bty607 · Cited by 33
Self-citationA Bayesian active learning strategy for sequential experimental design in systems biology
Pauwels, E., Lajaunie, C., & Vert, J. (2014) · BMC Systems Biology · doi:10.1186/s12918-014-0102-6 · Cited by 35
New opportunities for optimal design of dynamic experiments in systems and synthetic biology
Braniff, N., & Ingalls, B. (2018) · Current Opinion in Systems Biology · doi:10.1016/j.coisb.2018.02.005 · Cited by 30
Metadata
- Authors
- Weber, P., Kramer, A., Dingler, C., & Radde, N.
- Affiliations
- University of Stuttgart (DE)
- Corresponding author
- Weber, Patrick
- Venue
- Bioinformatics
- Publisher
- Oxford University Press (OUP)
- Published
- 2012-09-03
- Volume
- 28
- Issue
- 18
- Pages
- i535-i541
- ISSN
- 1367-4803 (ISSN-L)
- 1367-4811
- PMC
- PMC3436804
- OpenAlex
- W2146888637
- Semantic Scholar
- 7f9353e79e4987a8419f0499afa640517e46fb4e
- Cited by
- 14
- References
- 25
- Citations by year
- 2019: 2 · 2018: 4 · 2017: 2 · 2016: 1 · 2015: 2 · 2014: 2 · 2013: 1
- FWCI
- 2.12
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Bioinformatics (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Funders
- Deutsche Forschungsgemeinschaft (EXC 3101, EXC 310)
- Universität Stuttgart (EXC310/1)
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Optimal Experimental Design Methods
- Physical Sciences › Engineering › Control and Systems Engineering › Control Systems and Identification
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene Regulatory Network Analysis
- Keywords
- A-optimal design, Fisher information, forecast uncertainty, parameter identifiability, ordinary differential equation model, stopping rules, model discrimination, biomedical model
- MeSH terms
- Algorithms, Bayes Theorem, Models, Biological, Research Design, trans-Golgi Network, Uncertainty, Secretory Pathway
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.