Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.

Robust Optimal Design of Experiments for Model Discrimination Using an Interactive Software Tool

Open access

Abstract

Mathematical modeling of biochemical processes significantly contributes to a better understanding of biological functionality and underlying dynamic mechanisms. To support time consuming and costly lab experiments, kinetic reaction equations can be formulated as a set of ordinary differential equations, which in turn allows to simulate and compare hypothetical models in silico. To identify new experimental designs that are able to discriminate between investigated models, the approach used in this work solves a semi-infinite constrained nonlinear optimization problem using derivative based numerical algorithms. The method takes into account parameter variabilities such that new experimental designs are robust against parameter changes while maintaining the optimal potential to discriminate between hypothetical models. In this contribution we present a newly developed software tool that offers a convenient graphical user interface for model discrimination. We demonstrate the beneficial operation of the discrimination approach and the usefulness of the software tool by analyzing a realistic benchmark experiment from literature. New robust optimal designs that allow to discriminate between the investigated model hypotheses of the benchmark experiment are successfully calculated and yield promising results. The involved robustification approach provides maximally discriminating experiments for the worst parameter configurations, which can be used to estimate the meaningfulness of upcoming experiments. A major benefit of the graphical user interface is the ability to interactively investigate the model behavior and the clear arrangement of numerous variables. In addition to a brief theoretical overview of the discrimination method and the functionality of the software tool, the importance of robustness of experimental designs against parameter variability is demonstrated on a biochemical benchmark problem. The software is licensed under the GNU General Public License and freely available at http://sourceforge.net/projects/mdtgui/.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1371/journal.pone.0055723

Cites

29 of 29 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 Screenshot of the graphical user interface. The main working areas of the GUI are labeled. The application comprises a plot window (1), a console output window (2) and an extensive settings dialog (3).
Figure 1 Screenshot of the graphical user interface. The main working areas of the GUI are labeled. The application comprises a plot window (1), a console output window (2) and an extensive settings dialog (3).CC BY · PubMed Central
Figure 2 Substrate and product time courses of the simple model and the hysteretic model with estimated parameters. The values of one out of five measurements are shown as dots. A lack-of-fit test, which is performed on both models with the generated experimental data, ensures that none of these models can be discarded in advance. The depicted experiment is used as initial design of the discrimination approach.
Figure 2 Substrate and product time courses of the simple model and the hysteretic model with estimated parameters. The values of one out of five measurements are shown as dots. A lack-of-fit test, which is performed on both models with the generated experimental data, ensures that none of these models can be discarded in advance. The depicted experiment is used as initial design of the discrimination approach.CC BY · PubMed Central
Figure 3 The residual vs. run time plots of the simple and the hysteretic model for the initial design. The images in the upper and lower part show the residuals of the substrate and the product curves, respectively. None of the plots shows a noticeable trend, which indicates a good fit of both considered models.
Figure 3 The residual vs. run time plots of the simple and the hysteretic model for the initial design. The images in the upper and lower part show the residuals of the substrate and the product curves, respectively. None of the plots shows a noticeable trend, which indicates a good fit of both considered models.CC BY · PubMed Central
Figure 4 Robustification gaps and objective values. The robustification gap is displayed in the left part of the figure. The right part shows the corresponding objective values of the respective new designs (solid line) and the objective values of this design with re-estimated worst case parameters (dashed line). All values are plotted versus the corresponding iteration number.
Figure 4 Robustification gaps and objective values. The robustification gap is displayed in the left part of the figure. The right part shows the corresponding objective values of the respective new designs (solid line) and the objective values of this design with re-estimated worst case parameters (dashed line). All values are plotted versus the corresponding iteration number.CC BY · PubMed Central
Figure 5 Time courses after discrimination. The time courses for substrate and product of the two models after discrimination compared to the experimental data. The worst case discrimination is shown. Other parameters in the feasible range lead to larger model distances.
Figure 5 Time courses after discrimination. The time courses for substrate and product of the two models after discrimination compared to the experimental data. The worst case discrimination is shown. Other parameters in the feasible range lead to larger model distances.CC BY · PubMed Central
Figure 6 Time courses after discrimination with re-estimated parameters. The time courses for substrate and product of the two models after discrimination compared to the simulated experimental data. The parameters have been re-estimated using measurement data of the new design and the initial design. For a correct model it should be possible to determine parameter values that sufficiently fit all available measurements.
Figure 6 Time courses after discrimination with re-estimated parameters. The time courses for substrate and product of the two models after discrimination compared to the simulated experimental data. The parameters have been re-estimated using measurement data of the new design and the initial design. For a correct model it should be possible to determine parameter values that sufficiently fit all available measurements.CC BY · PubMed Central
Figure 7 Residual plots after discrimination with re-estimated parameters for both models. The residuals of the simple model show a clear trend, which indicates the bad fit of the model even with re-estimated parameters (B, D). Contrary, the good fit of the hysteretic model is proofed by residuals being normally distributed around zero (A, C).
Figure 7 Residual plots after discrimination with re-estimated parameters for both models. The residuals of the simple model show a clear trend, which indicates the bad fit of the model even with re-estimated parameters (B, D). Contrary, the good fit of the hysteretic model is proofed by residuals being normally distributed around zero (A, C).CC BY · PubMed Central
Figure 8 Histogram plot the KL-divergence distribution for 1000 random parameters of the simple model. Only values larger than the red bar are observed. This indicates that no lower distances than the proposed minimal distance of the robust optimal design have to be expected.
Figure 8 Histogram plot the KL-divergence distribution for 1000 random parameters of the simple model. Only values larger than the red bar are observed. This indicates that no lower distances than the proposed minimal distance of the robust optimal design have to be expected.CC BY · PubMed Central
Figure 9 Histogram plot of the KL-divergence distribution of model distances for 1000 random parameters of the simple model. The experimental designs are optimized but lack of robustification of any parameters (A) and additionally lack of perturbations (B). KL-divergence values below the value of the red bar indicate that there are parameter combinations for which the discrimination is worse than the one of the proposed optimal design and decisions might become ambiguous.
Figure 9 Histogram plot of the KL-divergence distribution of model distances for 1000 random parameters of the simple model. The experimental designs are optimized but lack of robustification of any parameters (A) and additionally lack of perturbations (B). KL-divergence values below the value of the red bar indicate that there are parameter combinations for which the discrimination is worse than the one of the proposed optimal design and decisions might become ambiguous.CC BY · PubMed Central
Figure 10 Initial time courses for biomass, substrate, , and the enzyme. The initial time series for biomass and substrate concentration (A), species (B), species (C) and the enzyme (D) are shown. In image (A) both models lead to the same response. The remaining time courses show only small differences. None of the models clearly fits the data worse. The vertical bars indicate perturbation time points.
Figure 10 Initial time courses for biomass, substrate, , and the enzyme. The initial time series for biomass and substrate concentration (A), species (B), species (C) and the enzyme (D) are shown. In image (A) both models lead to the same response. The remaining time courses show only small differences. None of the models clearly fits the data worse. The vertical bars indicate perturbation time points.CC BY · PubMed Central
Figure 11 Robustification gaps and objective values for the second experiment. The robustification gap is displayed in (A), whereas (B) shows the corresponding objective values of the respective new designs (solid line) and the objective values of this design with re-estimated worst case parameters (dashed line). All values are plotted versus the corresponding iteration number.
Figure 11 Robustification gaps and objective values for the second experiment. The robustification gap is displayed in (A), whereas (B) shows the corresponding objective values of the respective new designs (solid line) and the objective values of this design with re-estimated worst case parameters (dashed line). All values are plotted versus the corresponding iteration number.CC BY · PubMed Central
Figure 12 Time courses of the enzyme for the initial and the new design for the second experiment. Trajectories of the enzyme for the benchmark experiment (left) and the new experiment (right) with re-estimated parameters are shown. Model A clearly fits the data worse in both cases. The vertical bars indicate perturbation time points.
Figure 12 Time courses of the enzyme for the initial and the new design for the second experiment. Trajectories of the enzyme for the benchmark experiment (left) and the new experiment (right) with re-estimated parameters are shown. Model A clearly fits the data worse in both cases. The vertical bars indicate perturbation time points.CC BY · PubMed Central
Figure 13 Residual plots after discrimination with re-estimated parameters for the second experiment. The results for model A (A) and for model B (B) are shown. The residuals of model A show a clear trend, which indicates a bad fit of the model.
Figure 13 Residual plots after discrimination with re-estimated parameters for the second experiment. The results for model A (A) and for model B (B) are shown. The residuals of model A show a clear trend, which indicates a bad fit of the model.CC BY · PubMed Central
Figure 14 Histogram plot of the KL-divergence distribution for the second experiment. Distribution of the KL-divergence for different values of , randomly distributed in the interval [0.005; 10.0]. The red line represents the KL-divergence achieved by the worst-case parameter for this design.
Figure 14 Histogram plot of the KL-divergence distribution for the second experiment. Distribution of the KL-divergence for different values of , randomly distributed in the interval [0.005; 10.0]. The red line represents the KL-divergence achieved by the worst-case parameter for this design.CC BY · PubMed Central
Figure 15 Time courses of the optimal design proposed in Large Steps on the Inputs by Kremling et al. after re-estimation of parameter . The (A) and enzyme concentration (B) time courses for the optimal design proposed by Kremling et al. are shown. Upon re-estimation of the parameter , the model responses become quite similar, which shows the necessity of the robustification approach. The vertical bars indicate perturbation time points.
Figure 15 Time courses of the optimal design proposed in Large Steps on the Inputs by Kremling et al. after re-estimation of parameter . The (A) and enzyme concentration (B) time courses for the optimal design proposed by Kremling et al. are shown. Upon re-estimation of the parameter , the model responses become quite similar, which shows the necessity of the robustification approach. The vertical bars indicate perturbation time points.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 29 references

Research works printed but not linked: 2

  1. Wächter A (2002) An Interior Point Algorithm for Large-Scale Nonlinear Optimization with Applications in Process Engineering. PhD thesis, Carnegie Mellon University.
  2. Skanda D (2012) Robust Optimal Experimental Design for Model Discrimination of Kinetic ODE Systems. PhD thesis, Albert-Ludwigs University Freiburg: urn:nbn:de:bsz:25-opus-87870.

Cited by

See all 15

Metadata

Authors
Stegmaier, J., Skanda, D., & Lebiedz, D.
Affiliations
  • Karlsruhe Institute of Technology (DE)
  • University of Freiburg (DE)
  • Universität Ulm (DE)
Corresponding author
Lebiedz, Dirk
Venue
PLoS ONE
Published
2013-02-04
Volume
8
Issue
2
Pages
e55723
ISSN
1932-6203 (ISSN-L)
OpenAlex
W2088262268
Cited by
15
References
29
Citations by year
2024: 1 · 2020: 1 · 2018: 2 · 2017: 3 · 2015: 3 · 2014: 4 · 2013: 1
FWCI
3.65
Type
journal article
Language
en
Open access
Locations
  • PLoS ONE (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • FreiDok plus (Universitätsbibliothek Freiburg) (submittedVersion)
Funders
  • Deutsche Forschungsgemeinschaft
  • Bundesministerium für Bildung und Forschung (0315896A)
Topics
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Optimal Experimental Design Methods
  • Physical Sciences › Engineering › Control and Systems Engineering › Control Systems and Identification
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene Regulatory Network Analysis
Keywords
optimal experimental design, model discrimination, robust design, parameter variation, gradient-based optimization, biochemical modeling, ordinary differential equations, graphical user interface, benchmark tests
MeSH terms
Algorithms, Animals, Computer Simulation, Humans, Kinetics, Models, Biological, Research Design, Software, Systems Biology, Metabolic Networks and Pathways, Enzyme Assays

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Robust Optimal Design of Experiments for Model Discrimination Using an Interactive Software Tool. https://app.cochat.ai/papers/10.1371/journal.pone.0055723
The paper
Stegmaier, J., Skanda, D., & Lebiedz, D. (2013). Robust Optimal Design of Experiments for Model Discrimination Using an Interactive Software Tool. PLoS ONE. https://doi.org/10.1371/journal.pone.0055723

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.