CoChat

DREAM4: Combining Genetic and Dynamic Information to Identify Biological Networks and Dynamical Models

Open access
Greenfield, Alex, Madar, Aviv, Ostrer, Harry, Bonneau, Richard · 2010 · PLoS ONE

The DOI matches its Crossref record; 51 of the 51 references it cites were found in a registry or an index.

Abstract

BACKGROUND: Current technologies have lead to the availability of multiple genomic data types in sufficient quantity and quality to serve as a basis for automatic global network inference. Accordingly, there are currently a large variety of network inference methods that learn regulatory networks to varying degrees of detail. These methods have different strengths and weaknesses and thus can be complementary. However, combining different methods in a mutually reinforcing manner remains a challenge. METHODOLOGY: We investigate how three scalable methods can be combined into a useful network inference pipeline. The first is a novel t-test-based method that relies on a comprehensive steady-state knock-out dataset to rank regulatory interactions. The remaining two are previously published mutual information and ordinary differential equation based methods (tlCLR and Inferelator 1.0, respectively) that use both time-series and steady-state data to rank regulatory interactions; the latter has the added advantage of also inferring dynamic models of gene regulation which can be used to predict the system's response to new perturbations. CONCLUSION/SIGNIFICANCE: Our t-test based method proved powerful at ranking regulatory interactions, tying for first out of methods in the DREAM4 100-gene in-silico network inference challenge. We demonstrate complementarity between this method and the two methods that take advantage of time-series data by combining the three into a pipeline whose ability to rank regulatory interactions is markedly improved compared to either method alone. Moreover, the pipeline is able to accurately predict the response of the system to new conditions (in this case new double knock-out genetic perturbations). Our evaluation of the performance of multiple methods for network inference suggests avenues for future methods development and provides simple considerations for genomic experimental design. Our code is publicly available at http://err.bio.nyu.edu/inferelator/.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1371/journal.pone.0013397

Cites

51 of 51 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 Network inference pipelines tested. We developed and tested four network inference pipelines composed of the methods described in Table 1. Pipeline 1, MCZ, uses median corrected z-scores on the steady state genetic knock-out data (eq. 2). We submitted topology predictions from this method, tying for place. Pipeline 2, tlCLR-Inferelator, (eq. 19) uses tlCLR as a feature selection procedure, followed by further model selection and parameterization by the Inferelator 1.0. This pipeline was previously published and placed for the DREAM3 network inference competition [31]. Pipeline 3, tlCLR-Inferelator+MCZ, (eq. 20, developed to test the complementarity of the topology predictions made by pipelines 1 and 2). Pipeline 3 combines the results form pipelines 1 and 2. Double knock-out predictions from this pipeline were submitted, placing (topology predictions from this pipeline were submitted, placing ). Pipeline 4, Resampling+MCZ, (eq. 22) presents an alternate way to combine predictions made from complementary methods. In pipeline 4 a resampling approach was applied to pipeline 2 and the results were combined with pipeline 1 as described in the methods sections. Topology predictions from pipeline 4 outperformed those of pipeline 3, and double knock-out predictions were on par with those of pipeline 3.
Figure 1 Network inference pipelines tested. We developed and tested four network inference pipelines composed of the methods described in Table 1. Pipeline 1, MCZ, uses median corrected z-scores on the steady state genetic knock-out data (eq. 2). We submitted topology predictions from this method, tying for place. Pipeline 2, tlCLR-Inferelator, (eq. 19) uses tlCLR as a feature selection procedure, followed by further model selection and parameterization by the Inferelator 1.0. This pipeline was previously published and placed for the DREAM3 network inference competition [31]. Pipeline 3, tlCLR-Inferelator+MCZ, (eq. 20, developed to test the complementarity of the topology predictions made by pipelines 1 and 2). Pipeline 3 combines the results form pipelines 1 and 2. Double knock-out predictions from this pipeline were submitted, placing (topology predictions from this pipeline were submitted, placing ). Pipeline 4, Resampling+MCZ, (eq. 22) presents an alternate way to combine predictions made from complementary methods. In pipeline 4 a resampling approach was applied to pipeline 2 and the results were combined with pipeline 1 as described in the methods sections. Topology predictions from pipeline 4 outperformed those of pipeline 3, and double knock-out predictions were on par with those of pipeline 3.CC BY · PubMed Central
Figure 2 Area under precision recall curve for each ranking scheme. For each pipeline we evaluated the performance in predicting topology using area under the precision recall curve (AUPR). We see that pipeline 4 generally outperforms all other methods, followed by MCZ, pipeline 3, and pipeline 2.
Figure 2 Area under precision recall curve for each ranking scheme. For each pipeline we evaluated the performance in predicting topology using area under the precision recall curve (AUPR). We see that pipeline 4 generally outperforms all other methods, followed by MCZ, pipeline 3, and pipeline 2.CC BY · PubMed Central
Figure 3 Trends in performance over the five networks. For panels A,B,C we consider only the performance of MCZ, and use relative rank as an estimate of error. We compute relative rank in the following way. Denote by the total number of possible regulatory interactions, and by the rank that was given to each regulatory interaction, . The relative rank of is defined to be . Error distributions of the predictions for the five networks are shown as black boxplots in panels A,B,C. Distributions of in-degree of the regulators, out-degree of the regulators, and median expression of the regulators are shown as gray boxplots in panels A,B,C, respectively. A) There is no apparent relationship between relative rank (Error) and the in-degree of the regulators. B) There is no apparent relationship between relative rank (Error) and the out-degree of the regulators. C) Relative rank (Error) in network prediction increases as the median expression of the regulators decreases. D) we show the relationship between median expression of the regulators and the performance in ranking regulatory interactions, in terms of AUPR, across all five networks. For MCZ a correlation of () exists between the TFs median expression and AUPR (shown in red), while for Resampling+MCZ there is a smaller correlation of () between the TFs median expression and AUPR (shown in purple).
Figure 3 Trends in performance over the five networks. For panels A,B,C we consider only the performance of MCZ, and use relative rank as an estimate of error. We compute relative rank in the following way. Denote by the total number of possible regulatory interactions, and by the rank that was given to each regulatory interaction, . The relative rank of is defined to be . Error distributions of the predictions for the five networks are shown as black boxplots in panels A,B,C. Distributions of in-degree of the regulators, out-degree of the regulators, and median expression of the regulators are shown as gray boxplots in panels A,B,C, respectively. A) There is no apparent relationship between relative rank (Error) and the in-degree of the regulators. B) There is no apparent relationship between relative rank (Error) and the out-degree of the regulators. C) Relative rank (Error) in network prediction increases as the median expression of the regulators decreases. D) we show the relationship between median expression of the regulators and the performance in ranking regulatory interactions, in terms of AUPR, across all five networks. For MCZ a correlation of () exists between the TFs median expression and AUPR (shown in red), while for Resampling+MCZ there is a smaller correlation of () between the TFs median expression and AUPR (shown in purple).CC BY · PubMed Central
Figure 4 Error as a function of binned median expression for all regulatory interactions. We further investigate the relationship between the median expression of the regulators and each pipeline's performance in predicting topology. We use relative rank as an estimate of error (as in Figure 3). We bin the regulators for all five networks based on their median expression (each of the seven bins has a roughly equal number of regulators). We show the distribution of relative ranks (Error) for each pipeline in each bin of regulator expression. We see that all of the pipelines that incorporate the predictions of tlCLR-Inferelator (pipelines 2,3, and 4) outperform MCZ for regulators with low median expression (bins , ).
Figure 4 Error as a function of binned median expression for all regulatory interactions. We further investigate the relationship between the median expression of the regulators and each pipeline's performance in predicting topology. We use relative rank as an estimate of error (as in Figure 3). We bin the regulators for all five networks based on their median expression (each of the seven bins has a roughly equal number of regulators). We show the distribution of relative ranks (Error) for each pipeline in each bin of regulator expression. We see that all of the pipelines that incorporate the predictions of tlCLR-Inferelator (pipelines 2,3, and 4) outperform MCZ for regulators with low median expression (bins , ).CC BY · PubMed Central
Figure 5 Performance on double knock-out prediction. We assess the accuracy of predicting the system's response to the simultaneous removal (knock-out) of two genes . In total, there were one-hundred pairs of genes that were knocked out. We bin these pairs of genes based on the average of their respective median expression in the single-gene knock-out data. We made two predictions, which differ only in the choice of initial conditions. We compare the error (as evaluated by the mean squared error) of our prediction to the error made by using the respective initial condition as a prediction. A) We use the wild-type expression, , as the set of initial conditions (green boxplots). We see that our predictions (black and red boxplots) are more accurate than if we used the initial conditions as a prediction (this is more apparent for TFs with a larger median expression). B) We use a combination of the single-gene knock-outs to compute our initial conditions (eq. 25). We do this because the single-gene knock-out data represents a system state that is closer to the state we are trying to predict than wild-type (as can be observed by comparing the green boxplots in panel A to those in panel B). We show the error distributions using parameters calculated by either pipeline 3 (tlCLR-Inferelator+MCZ) or pipeline 4 (Resampling+MCZ), gray and red boxplots, respectively, are smaller than the error distributions if we used the initial conditions as a prediction. Regardless of the choice of initial conditions, the error distributions using parameters calculated by pipeline 4 (red boxplots) are similar to the error distribution obtained by pipeline 3.
Figure 5 Performance on double knock-out prediction. We assess the accuracy of predicting the system's response to the simultaneous removal (knock-out) of two genes . In total, there were one-hundred pairs of genes that were knocked out. We bin these pairs of genes based on the average of their respective median expression in the single-gene knock-out data. We made two predictions, which differ only in the choice of initial conditions. We compare the error (as evaluated by the mean squared error) of our prediction to the error made by using the respective initial condition as a prediction. A) We use the wild-type expression, , as the set of initial conditions (green boxplots). We see that our predictions (black and red boxplots) are more accurate than if we used the initial conditions as a prediction (this is more apparent for TFs with a larger median expression). B) We use a combination of the single-gene knock-outs to compute our initial conditions (eq. 25). We do this because the single-gene knock-out data represents a system state that is closer to the state we are trying to predict than wild-type (as can be observed by comparing the green boxplots in panel A to those in panel B). We show the error distributions using parameters calculated by either pipeline 3 (tlCLR-Inferelator+MCZ) or pipeline 4 (Resampling+MCZ), gray and red boxplots, respectively, are smaller than the error distributions if we used the initial conditions as a prediction. Regardless of the choice of initial conditions, the error distributions using parameters calculated by pipeline 4 (red boxplots) are similar to the error distribution obtained by pipeline 3.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 51 references

Cited by

See all 255

Metadata

Authors
Greenfield, A., Madar, A., Ostrer, H., & Bonneau, R.
Affiliations
  • New York University (US)
  • Courant Institute of Mathematical Sciences (US)
Corresponding author
Bonneau, Richard
Venue
PLoS ONE
Published
2010-10-25
Volume
5
Issue
10
Pages
e13397
ISSN
1932-6203 (ISSN-L)
OpenAlex
W1986688807
Cited by
255
References
51
Citations by year
2026: 8 · 2025: 15 · 2024: 15 · 2023: 10 · 2022: 16 · 2021: 12 · 2020: 27 · 2019: 23 · 2018: 11 · 2017: 19 · 2016: 21 · 2015: 26 · 2014: 11 · 2013: 13 · 2012: 20
FWCI
5.34
Type
journal article
Language
en
Open access
Locations
  • PLoS ONE (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • National Science Foundation (0820757)
  • Université du Luxembourg
  • National Institutes of Health (5u54ca143907-05, U54CA143907)
  • National Eye Institute (PN2EY016586)
  • National Cancer Institute (IU54CA143907-01, U54-CA-143907)
Topics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene Regulatory Network Analysis
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Microbial Metabolic Engineering and Bioproduction
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Bioinformatics and Genomic Networks
Keywords
gene regulatory network inference, mutual information, ordinary differential equations, time series, method combination
MeSH terms
Models, Genetic, Likelihood Functions, Gene Regulatory Networks

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: DREAM4: Combining Genetic and Dynamic Information to Identify Biological Networks and Dynamical Models. https://app.cochat.ai/papers/10.1371/journal.pone.0013397
The paper
Greenfield, A., Madar, A., Ostrer, H., & Bonneau, R. (2010). DREAM4: Combining Genetic and Dynamic Information to Identify Biological Networks and Dynamical Models. PLoS ONE. https://doi.org/10.1371/journal.pone.0013397

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.