Towards a Rigorous Assessment of Systems Biology Models: The DREAM3 Challenges
Open accessThe DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Systems biology has embraced computational modeling in response to the quantitative nature and increasing scale of contemporary data sets. The onslaught of data is accelerating as molecular profiling technology evolves. The Dialogue for Reverse Engineering Assessments and Methods (DREAM) is a community effort to catalyze discussion about the design, application, and assessment of systems biology models through annual reverse-engineering challenges. METHODOLOGY AND PRINCIPAL FINDINGS: We describe our assessments of the four challenges associated with the third DREAM conference which came to be known as the DREAM3 challenges: signaling cascade identification, signaling response prediction, gene expression prediction, and the DREAM3 in silico network challenge. The challenges, based on anonymized data sets, tested participants in network inference and prediction of measurements. Forty teams submitted 413 predicted networks and measurement test sets. Overall, a handful of best-performer teams were identified, while a majority of teams made predictions that were equivalent to random. Counterintuitively, combining the predictions of multiple teams (including the weaker teams) can in some cases improve predictive power beyond that of any single method. CONCLUSIONS: DREAM provides valuable feedback to practitioners of systems biology modeling. Lessons learned from the predictions of the community provide much-needed context for interpreting claims of efficacy of algorithms described in the scientific literature.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures








Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Their results show that, in addition to the obstacles identified by Prill et al. [94] (lack of data and deficiencies in the inference algorithms), we must be also aware of fundamental problems related to the uniqueness and distinguishability of these networks (even for the utopian case of fully observed networks with no noise).”
From Reverse engineering and identification in systems biology: strategies, perspectives and challenges (2014) · cites this paper as [94]
In the citing paper
“The organizers identify two major hurdles to be surmounted: lack of data and deficiencies in the inference algorithms.”
From Reverse engineering and identification in systems biology: strategies, perspectives and challenges (2014) · cites this paper as [94]
In the citing paper
“In other words, reliable network inference remains an unsolved problem.”
From Reverse engineering and identification in systems biology: strategies, perspectives and challenges (2014) · cites this paper as [94]
In the citing paper
“There are several notable examples where collaborations have been very successful, such as SBML [238], BioModels Database [239] or the DREAM challenges [94].”
From Reverse engineering and identification in systems biology: strategies, perspectives and challenges (2014) · cites this paper as [94]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Regularization and Variable Selection Via the Elastic Net
Zou, H., & Hastie, T. (2005) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2005.00503.x · Cited by 20,957
Exploration, normalization, and summaries of high density oligonucleotide array probe level data
Irizarry, R. A. (2003) · Biostatistics · doi:10.1093/biostatistics/4.2.249 · Cited by 10,603
Journal of the Royal Statistical Society (B)
Fildes, R. (1993) · International Journal of Forecasting · doi:10.1016/0169-2070(93)90088-5 · Cited by 3,437
Margolin, A. A., Nemenman, I., Basso, K., Wiggins, C., Stolovitzky, G., Favera, R. D., & Califano, A. (2006) · BMC Bioinformatics · doi:10.1186/1471-2105-7-s1-s7 · Cited by 2,420
Self-citationMargolin, A. A., Nemenman, I. M., Basso, K., Wiggins, C. H., Stolovitzky, G. A., Favera, R. D., & Califano, A. (2006) · BMC Bioinformatics · doi:10.7916/d8g15zbp · Cited by 2,246
Self-citationReverse engineering of regulatory networks in human B cells
Basso, K., Margolin, A. A., Stolovitzky, G., Klein, U., Dalla-Favera, R., & Califano, A. (2005) · Nature Genetics · doi:10.1038/ng1532 · Cited by 1,404
Self-citationBUTTE, A. J., & KOHANE, I. S. (1999) · Biocomputing 2000 · doi:10.1142/9789814447331_0040 · Cited by 945
Natarajan, K., Meyer, M. R., Jackson, B. M., Slade, D., Roberts, C., Hinnebusch, A. G., & Marton, M. J. (2001) · Molecular and Cellular Biology · doi:10.1128/mcb.21.13.4347-4368.2001 · Cited by 748
Butte, A. J., Tamayo, P., Slonim, D., Golub, T. R., & Kohane, I. S. (2000) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.220392197 · Cited by 605
Bonneau, R., Reiss, D. J., Shannon, P., Facciotti, M., Hood, L., Baliga, N. S., & Thorsson, V. (2006) · Genome Biology · doi:10.1186/gb-2006-7-5-r36 · Cited by 510
Research works printed but not linked: 1
- [15]Alexopoulos L Saez-Rodriguez J Cosgrove B Lauffenburger D PK S 2009 Networks reconstructed from cell response data reveal profound dif ferences in signaling by tolllike receptors and nf-?b in normal and transformed human hepatocytes. Under review
Cited by
Inferring Regulatory Networks from Expression Data Using Tree-Based Methods
Huynh-Thu, V. A., Irrthum, A., Wehenkel, L., & Geurts, P. (2010) · PLoS ONE · doi:10.1371/journal.pone.0012776 · Cited by 2,131
“DREAM4 Datasets The DREAM (for ‘‘Dialogue for Reverse Engineering Assessments and Methods’’) initiative organizes an annual reverse engineering competition called the DREAM challenge [34–37].”
BackgroundWisdom of crowds for robust gene network inference
The DREAM5 Consortium, Marbach, D., Costello, J. C., Küffner, R., Vega, N. M., Prill, R. J., Camacho, D. M., Allison, K. R., Kellis, M., Collins, J. J., et al. (2012) · Nature Methods · doi:10.1038/nmeth.2016 · Cited by 1,667
Self-citationA Validated Regulatory Network for Th17 Cell Specification
Ciofani, M., Madar, A., Galan, C., Sellars, M., Mace, K., Pauli, F., Agarwal, A., Huang, W., Parkurst, C. N., Muratet, M., et al. (2012) · Cell · doi:10.1016/j.cell.2012.09.016 · Cited by 1,160
Critical assessment of automated flow cytometry data analysis techniques
The FlowCAP Consortium, Aghaeepour, N., The DREAM Consortium, Finak, G., Hoos, H., Mosmann, T. R., Brinkman, R., Gottardo, R., & Scheuermann, R. H. (2013) · Nature Methods · doi:10.1038/nmeth.2365 · Cited by 598
GeneNetWeaver: in silico benchmark generation and performance profiling of network inference methods
Schaffter, T., Marbach, D., & Floreano, D. (2011) · Bioinformatics · doi:10.1093/bioinformatics/btr373 · Cited by 535
Self-citation
Metadata
- Authors
- Prill, R. J., Marbach, D., Saez-Rodriguez, J., Sorger, P. K., Alexopoulos, L. G., Xue, X., Clarke, N. D., Altan-Bonnet, G., & Stolovitzky, G.
- Affiliations
- IBM (United States) (US)
- École Polytechnique Fédérale de Lausanne (CH)
- Harvard University (US)
- National Technical University of Athens (GR)
- Genome Institute of Singapore (SG)
- Corresponding author
- Stolovitzky, Gustavo
- ORCID
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2010-02-23
- Volume
- 5
- Issue
- 2
- Pages
- e9202
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC2826397
- OpenAlex
- W2143076232
- Cited by
- 439
- References
- 33
- Citations by year
- 2026: 16 · 2025: 16 · 2024: 15 · 2023: 11 · 2022: 21 · 2021: 18 · 2020: 17 · 2019: 17 · 2018: 28 · 2017: 27 · 2016: 23 · 2015: 50 · 2014: 40 · 2013: 38 · 2012: 47
- FWCI
- 18.23
- Type
- journal article
- Language
- en
- Update notices
- correction (2010-03-09)
- Open access
- gold
- doi.org
- Locations
- PLoS ONE (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Digital Access to Scholarship at Harvard (DASH) (Harvard University) (submittedVersion, cc-by)
- National University of Singapore (submittedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- DSpace - NTUA (National Technical University of Athens) (submittedVersion, other-oa)
- Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion)
- Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion, cc-by-nc-nd)
- Funders
- National Science Foundation (GM68762, 0848030)
- National Institutes of Health (3p50gm068762-07s2, NIH P50, GM68762)
- National Cancer Institute (U54CA112967)
- National Institute of General Medical Sciences (P50 GM68762, P50-GM068762)
- National Institute of Allergy and Infectious Diseases (R56AI083408, AI083408, R01AI083408)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene Regulatory Network Analysis
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Bioinformatics and Genomic Networks
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Microbial Metabolic Engineering and Bioproduction
- Keywords
- network inference, reverse engineering, gene expression prediction, ensemble forecasting, model validation
- MeSH terms
- Algorithms, Animals, Humans, Models, Biological, Software, Reproducibility of Results, Signal Transduction, Cluster Analysis, Computational Biology, Gene Expression Profiling, Protein Interaction Mapping, Systems Biology, Gene Regulatory Networks
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Towards a Rigorous Assessment of Systems Biology Models: The DREAM3 Challenges. https://app.cochat.ai/papers/10.1371/journal.pone.0009202
- The paper
- Prill, R. J., Marbach, D., Saez-Rodriguez, J., Sorger, P. K., Alexopoulos, L. G., Xue, X., Clarke, N. D., Altan-Bonnet, G., & Stolovitzky, G. (2010). Towards a Rigorous Assessment of Systems Biology Models: The DREAM3 Challenges. PLoS ONE. https://doi.org/10.1371/journal.pone.0009202
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.