Towards a Rigorous Assessment of Systems Biology Models: The DREAM3 Challenges
Open accessThe DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Systems biology has embraced computational modeling in response to the quantitative nature and increasing scale of contemporary data sets. The onslaught of data is accelerating as molecular profiling technology evolves. The Dialogue for Reverse Engineering Assessments and Methods (DREAM) is a community effort to catalyze discussion about the design, application, and assessment of systems biology models through annual reverse-engineering challenges. METHODOLOGY AND PRINCIPAL FINDINGS: We describe our assessments of the four challenges associated with the third DREAM conference which came to be known as the DREAM3 challenges: signaling cascade identification, signaling response prediction, gene expression prediction, and the DREAM3 in silico network challenge. The challenges, based on anonymized data sets, tested participants in network inference and prediction of measurements. Forty teams submitted 413 predicted networks and measurement test sets. Overall, a handful of best-performer teams were identified, while a majority of teams made predictions that were equivalent to random. Counterintuitively, combining the predictions of multiple teams (including the weaker teams) can in some cases improve predictive power beyond that of any single method. CONCLUSIONS: DREAM provides valuable feedback to practitioners of systems biology modeling. Lessons learned from the predictions of the community provide much-needed context for interpreting claims of efficacy of algorithms described in the scientific literature.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures








Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Their results show that, in addition to the obstacles identified by Prill et al. [94] (lack of data and deficiencies in the inference algorithms), we must be also aware of fundamental problems related to the uniqueness and distinguishability of these networks (even for the utopian case of fully observed networks with no noise).”
From Reverse engineering and identification in systems biology: strategies, perspectives and challenges (2014) · cites this paper as [94]
In the citing paper
“The organizers identify two major hurdles to be surmounted: lack of data and deficiencies in the inference algorithms.”
From Reverse engineering and identification in systems biology: strategies, perspectives and challenges (2014) · cites this paper as [94]
In the citing paper
“In other words, reliable network inference remains an unsolved problem.”
From Reverse engineering and identification in systems biology: strategies, perspectives and challenges (2014) · cites this paper as [94]
In the citing paper
“There are several notable examples where collaborations have been very successful, such as SBML [238], BioModels Database [239] or the DREAM challenges [94].”
From Reverse engineering and identification in systems biology: strategies, perspectives and challenges (2014) · cites this paper as [94]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Regularization and Variable Selection Via the Elastic Net
Zou, H., & Hastie, T. (2005) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2005.00503.x · Cited by 20,957
Exploration, normalization, and summaries of high density oligonucleotide array probe level data
Irizarry, R. A. (2003) · Biostatistics · doi:10.1093/biostatistics/4.2.249 · Cited by 10,603 · cited 1×
“Measurements were normalized using the RMA algorithm [18] within the commercial software package, GeneSpring.”
MethodJournal of the Royal Statistical Society (B)
Fildes, R. (1993) · International Journal of Forecasting · doi:10.1016/0169-2070(93)90088-5 · Cited by 3,437
Margolin, A. A., Nemenman, I., Basso, K., Wiggins, C., Stolovitzky, G., Favera, R. D., & Califano, A. (2006) · BMC Bioinformatics · doi:10.1186/1471-2105-7-s1-s7 · Cited by 2,420 · cited 1×
“…be applied off-the-shelf, a strategy for guarding against bogus predictions is to employ a wide variety of algorithms that train on different features of the data (e.g., correlation [28], mutual information [29–32], synergy [33], etc.), then merge the results to gain confidence in the predictions.”
ResultSelf-citationMargolin, A. A., Nemenman, I. M., Basso, K., Wiggins, C. H., Stolovitzky, G. A., Favera, R. D., & Califano, A. (2006) · BMC Bioinformatics · doi:10.7916/d8g15zbp · Cited by 2,246
Self-citationReverse engineering of regulatory networks in human B cells
Basso, K., Margolin, A. A., Stolovitzky, G., Klein, U., Dalla-Favera, R., & Califano, A. (2005) · Nature Genetics · doi:10.1038/ng1532 · Cited by 1,404 · cited 1×
“, correlation [28], mutual information [29–32], synergy [33], etc.”
BackgroundSelf-citationBUTTE, A. J., & KOHANE, I. S. (1999) · Biocomputing 2000 · doi:10.1142/9789814447331_0040 · Cited by 945 · cited 1×
“, correlation [28], mutual information [29–32], synergy [33], etc.”
BackgroundNatarajan, K., Meyer, M. R., Jackson, B. M., Slade, D., Roberts, C., Hinnebusch, A. G., & Marton, M. J. (2001) · Molecular and Cellular Biology · doi:10.1128/mcb.21.13.4347-4368.2001 · Cited by 748 · cited 1×
“Expression levels were assayed separately in the three mutant strains and in the wild type strain at times 0, 10, 20, 30, 45, 60, 90 and 120 minutes following the addition of 3-aminotriazole (3AT) as described [17].”
MethodButte, A. J., Tamayo, P., Slonim, D., Golub, T. R., & Kohane, I. S. (2000) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.220392197 · Cited by 605 · cited 1×
“…be applied off-the-shelf, a strategy for guarding against bogus predictions is to employ a wide variety of algorithms that train on different features of the data (e.g., correlation [28], mutual information [29–32], synergy [33], etc.), then merge the results to gain confidence in the predictions.”
ResultBonneau, R., Reiss, D. J., Shannon, P., Facciotti, M., Hood, L., Baliga, N. S., & Thorsson, V. (2006) · Genome Biology · doi:10.1186/gb-2006-7-5-r36 · Cited by 510
Research works printed but not linked: 1
- [15]Alexopoulos L Saez-Rodriguez J Cosgrove B Lauffenburger D PK S 2009 Networks reconstructed from cell response data reveal profound dif ferences in signaling by tolllike receptors and nf-?b in normal and transformed human hepatocytes. Under review
Cited by
Items 201–300 of 427
Abduallah, Y., Turki, T., Byron, K., Du, Z., Cervantes‐Cervantes, M., & Wang, J. T. L. (2017) · BioMed Research International · doi:10.1155/2017/6261802 · Cited by 13
“The array contains 5,744 probe sets and includes 10,928 Saccharomyces cerevisiae genes with 49 time points and transcriptional oscillations of about 4 hours.”
BackgroundInference on the structure of gene regulatory networks
Wang, Y., & Wang, Z. (2022) · Journal of Theoretical Biology · doi:10.1016/j.jtbi.2022.111055 · Cited by 19
“A common data type is to repeat the experiment for this scenario on different populations of the same cell type, such as a portion of the DREAM challenge data [53] and the M3D database [54].”
MethodShachaf, L., Roberts, E., Cahan, P., & Xiao, J. (2023) · BMC Bioinformatics · doi:10.1186/s12859-022-05047-5 · Cited by 13
“…different MI estimators and inference algorithms, we used a total of 15 different synthetic networks: ten synthetic networks from the DREAM3 (Dialogue for Reverse Engineering Assessments and Methods) competition [27] with 50 and 100 genes, respectively, and five networks from DREAM4 with 100 genes.”
MethodGSGS: A Computational Approach to Reconstruct Signaling Pathway Structures from Gene Sets
Acharya, L. R., Judeh, T., Duan, Z., Rabbat, M., & Zhu, D. (2011) · IEEE Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbb.2011.143 · Cited by 14
“coli network [17], [18], [30] from DREAM3 network challenges in the DREAM initiative.”
BackgroundMethodBelcastro, V., Gregoretti, F., Siciliano, V., Santoro, M. C., D’Angelo, G., Oliva, G., & di Bernardo, D. (2011) · IEEE Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbb.2011.60 · Cited by 15
Pinna, A., Heise, S., Flassig, R., de la Fuente, A., & Klamt, S. (2013) · BMC Systems Biology · doi:10.1186/1752-0509-7-73 · Cited by 23
“The different inference methodologies can be categorized based on the model formalism and the principle used for deriving interactions in a regulatory network: sparse regression [7], correlation-based approaches [8], z-score [9], mutual information [10,11], ANOVA-based analysis [12], Bayesian networks [13], Gaussian g…”
MethodTime-lagged Ordered Lasso for network inference
Nguyen, P., & Braun, R. (2018) · BMC Bioinformatics · doi:10.1186/s12859-018-2558-7 · Cited by 13
A Null Model for Pearson Coexpression Networks
Gobbi, A., & Jurman, G. (2015) · PLoS ONE · doi:10.1371/journal.pone.0128115 · Cited by 14
“In fact, in many practical cases, the performance of the reconstruction algorithms are poor, due to several factors limiting inference accuracy [3, 4] to the point of making it no better than a coin toss in some situations [5].”
BackgroundContinuous time Gaussian process dynamical models in gene regulatory network inferencePreprint
Aalto, A., Viitasaari, L., Ilmonen, P., Mombaerts, L., & Gonçalves, J. M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.08161 · Cited by 14
“We apply BINGO to the DREAM4 In Silico Network Challenge data [26, 27, 35, 39] using either the time series data only, or including also the steady state experiments.”
MethodZhu, K. N., & Zhao, C. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102811 · Cited by 13
SeqNet: An R Package for Generating Gene-Gene Networks and Simulating RNA-Seq Data
Grimes, T., & Datta, S. (2021) · Journal of Statistical Software · doi:10.18637/jss.v098.i12 · Cited by 14
“An early, organized effort to address this problem was the Dialogue for Reverse Engineering Assessment and Methods (DREAM; Stolovitzky et al. 2007; Marbach et al. 2010; Prill et al. 2010).”
BackgroundMethodInfluentialQi, J., & Michoel, T. (2012) · Bioinformatics · doi:10.1093/bioinformatics/bts434 · Cited by 12
“ks of known transcriptional regulatory interactions (Gama-Castro et al., 2008; Monteiro et al., 2008) in E. coli and yeast, following standard evaluation protocols established by the DREAM community (Prill et al., 2010). We found that double two-way t-testing performs as well as the best current methods, especially in…”
MethodInfluentialWalsh, C. J., Hu, P., Batt, J. A. E., & dos Santos, C. C. (2016) · Cancer Informatics · doi:10.4137/cin.s39369 · Cited by 12
Fan, Y., Wang, X., & Peng, Q. (2017) · Computational and Mathematical Methods in Medicine · doi:10.1155/2017/8307530 · Cited by 12
Structure Identifiability of an NDS With LFT Parametrized Subsystems
Zhou, T. (2022) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2022.3187371 · Cited by 12
Mahmud, M. I., & Banerjee, T. (2026) · Briefings in Bioinformatics · doi:10.1093/bib/bbag229 · Cited by 16
Reconstruction of gene regulatory networks using graph neural networks
Emma Paul, M., Jereesh, A. S., & Kumar, G. S. (2024) · Applied Soft Computing · doi:10.1016/j.asoc.2024.111899 · Cited by 15
Ciaccio, M. F., Chen, V. C., Jones, R. B., & Bagheri, N. (2015) · Integrative Biology · doi:10.1039/c5ib00065c · Cited by 13
The Distance Precision Matrix: computing networks from non-linear relationships
Ghanbari, M., Lasserre, J., & Vingron, M. (2018) · Bioinformatics · doi:10.1093/bioinformatics/bty724 · Cited by 15
“The DREAM (Dialogue for Reverse Engineering Assessments and Methods) challenge is an annual reverse engineering competition with the aim of fair comparison of network inference methods [15, 14, 22].”
BackgroundMethodLearning gene regulatory networks from gene expression data using weighted consensus
Fujii, C., Kuwahara, H., Yu, G., Guo, L., & Gao, X. (2016) · Neurocomputing · doi:10.1016/j.neucom.2016.02.087 · Cited by 12
Time to build on good design: Resolving the temporal dynamics of gene regulatory networks
Greenham, K., & McClung, C. R. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1807707115 · Cited by 11
Vashishtha, S., Broderick, G., Craddock, T. J. A., FLETCHER, M. A., & Klimas, N. G. (2015) · PLoS ONE · doi:10.1371/journal.pone.0127364 · Cited by 11
Quantifying the multi-scale performance of network inference algorithms
Oates, C. J., Amos, R., & Spencer, S. E. F. (2014) · Statistical Applications in Genetics and Molecular Biology · doi:10.1515/sagmb-2014-0012 · Cited by 13
Interpretable VAEs for nonlinear group factor analysisPreprint
Ainsworth, S., Foti, N. J., Lee, A. K. C., & Fox, E. B. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.06765 · Cited by 11
“However, it is well known that neural (Han et al., 2017), genomic (Prill et al., 2010), and financial data (Harvey et al., 1994), for example, exhibit complex nonlinearities.”
BackgroundPCAC: Causal discovery from low-dimensional small-scale time series
Sun, W., Zhang, Y., Liu, J., & Cai, B. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.114135 · Cited by 10
RMaNI: Regulatory Module Network Inference framework
Madhamshettiwar, P. B., Maetschke, S., Davis, M. J., & Ragan, M. A. (2013) · BMC Bioinformatics · doi:10.1186/1471-2105-14-s16-s14 · Cited by 10
“Although diverse computational and statistical approaches have been applied to this problem, the accuracy of edge-wise network inference methods remains poor [11-14].”
BackgroundStructured sparsity learning for large-scale fuzzy cognitive maps
Ding, F., & Luo, C. (2021) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2021.104444 · Cited by 11
Benchmarking time-series data discretization on inference methods
Li, Y., Jann, T., & Vera‐Licona, P. (2019) · Bioinformatics · doi:10.1093/bioinformatics/btz036 · Cited by 11
An Efficient Boolean Modelling Approach for Genetic Network Inference
Gamage, H. N., Chetty, M., Shatte, A. B. R., & Hallinan, J. S. (2021) · IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB) · doi:10.1109/cibcb49929.2021.9562881 · Cited by 10
Reconstruction of gene regulatory networks with multi-objective particle swarm optimisers
Hurtado, S., García-Nieto, J., Navas‐Delgado, I., Nebro, A. J., & Aldana-Montes, J. F. (2020) · Applied Intelligence · doi:10.1007/s10489-020-01891-1 · Cited by 10
Sambo, F., de, M. A. M., Camillo, B. D., TOFFOLO, G. M., & Stützle, T. (2012) · IEEE Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbb.2012.56 · Cited by 11
“Two candidate continuous optimization algorithms are considered from the state of the art: the NEW Unconstrained Optimization Algorithm (NEWUOA [32]) and Covariance Matrix Adaptation-Evolution Strategy (CMA-ES [33]).”
EnsInfer: a simple ensemble approach to network inference outperforms any single method
Shen, B., Coruzzi, G. M., & Shasha, D. E. (2023) · BMC Bioinformatics · doi:10.1186/s12859-023-05231-1 · Cited by 13
“Synthetic data from the DREAM3 and DREAM4 in silico challenges consists of ten datasets each with 100 genes and varying regulatory network structures [23, 24].”
MethodNetter: re-ranking gene network inference predictions using structural network properties
Ruyssinck, J., Demeester, P. M. A., Dhaene, T., & Saeys, Y. (2016) · BMC Bioinformatics · doi:10.1186/s12859-016-0913-0 · Cited by 11
Genetic Network Inference Using Hierarchical Structure
Kimura, S., Tokuhisa, M., & Okada, M. (2016) · Frontiers in Physiology · doi:10.3389/fphys.2016.00057 · Cited by 9
Global structure identifiability and reconstructibility of an NDS with descriptor subsystems
Zhou, T., & Yin, K. (2022) · Automatica · doi:10.1016/j.automatica.2022.110356 · Cited by 9
“For instance, on the basis of some biological knowledge about a gene regulation network, it is usually much easier to infer genes or other biochemical molecules that directly affect a specific gene, but rather hard and/or too expensive to infer an interaction strength [6, 12, 23].”
BackgroundInfluentialKimura, S., Sato, M., & Okada, M. (2013) · PLoS ONE · doi:10.1371/journal.pone.0083308 · Cited by 9
“Compared with the champion algorithm of the DREAM3 challenges [44], the performances of our method were worse.”
Construction of Cell Type-Specific Logic Models of Signaling Networks Using CellNOpt
Morris, M. K., Melas, I. N., & Sáez-Rodríguez, J. (2012) · Methods in molecular biology · doi:10.1007/978-1-62703-059-5_8 · Cited by 11
Hoeng, J., Talikka, M., Martin, F. P., Ansari, S., Drubin, D. A., Elamin, A., Gebel, S., Ivanov, N. V., Deehan, R., Kogel, U., et al. (2014) · CRC Press eBooks · doi:10.1201/b17359-9 · Cited by 13
Inference of gene regulatory networks with multi-objective cellular genetic algorithm
García-Nieto, J., Nebro, A. J., & Aldana-Montes, J. F. (2019) · Computational Biology and Chemistry · doi:10.1016/j.compbiolchem.2019.05.003 · Cited by 9
Variational Causal Networks: Approximate Bayesian Inference over Causal StructuresPreprint
Annadani, Y., Rothfuss, J., Lacoste, A., Scherrer, N., Goyal, A., Bengio, Y., & Bauer, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.07635 · Cited by 57
“Details of computing this is give in [30].”
BackgroundKadelka, C., & Hari, K. (2025) · PNAS Nexus · doi:10.1093/pnasnexus/pgaf228 · Cited by 10
“Around 2010, the Dialogue on Reverse Engineering Assessment and Methods (DREAM) challenges boosted the development of numerous GRN inference methods, as well as their comprehensive assessment using both synthetic and real bulk transcriptomic data (3 – 5).”
Wang, C., Xu, S., & Liu, Z. (2022) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2022.3199243 · Cited by 11
“The second simulation dataset is from DREAM3–In Silico Network Challenge [37].”
MethodUsing empirical biological knowledge to infer regulatory networks from multi-omics data
Pačínková, A., & Popovici, V. C. (2022) · BMC Bioinformatics · doi:10.1186/s12859-022-04891-9 · Cited by 8
Optimal structural inference of signaling pathways from unordered and overlapping gene sets
Acharya, L. R., Judeh, T., Wang, G., & Zhu, D. (2011) · Bioinformatics · doi:10.1093/bioinformatics/btr696 · Cited by 8
Asadi, B., Maurya, M. R., Tartakovsky, D. M., & Subramaniam, S. (2012) · IET Systems Biology · doi:10.1049/iet-syb.2011.0052 · Cited by 10
Zhu, F., Shi, L., Engel, J. D., & Guan, Y. (2015) · Bioinformatics · doi:10.1093/bioinformatics/btv186 · Cited by 9
“…2010, 2012; Margolin et al., 2013; Michoel et al., 2009; Mordelet and Vert, 2008; Park et al., 2010; Pique-Regi et al., 2011; Poultney et al., 2012; Prill et al., 2010, 2011; Yu et al., 2004; Zhu and Guan 2014; Zhu et al., 2014; Zou and Conzen, 2005) for modeling regulatory networks based on…”
BackgroundMethodInfluentialAn approach for reduction of false predictions in reverse engineering of gene regulatory networks
Khan, A., Saha, G. K., & Pal, R. K. (2018) · Journal of Theoretical Biology · doi:10.1016/j.jtbi.2018.02.015 · Cited by 11
Liu, Q., Li, J., Dong, M., Liu, M., & Chai, Y. (2022) · IEEE Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbb.2022.3144418 · Cited by 8
“DREAM4 [36], [37], [38] provides some gene expression data of some typical GRNs, and we can use these data to establish the network model and test the effectiveness of the proposed GRN inference method.”
MethodScalable reverse‐engineering of gene regulatory networks from time‐course measurements
Montefusco, F., Procopio, A., Bates, D. G., Amato, F., & Cosentino, C. (2022) · International Journal of Robust and Nonlinear Control · doi:10.1002/rnc.6044 · Cited by 9
“16-18 Although our approach is based on the use of linear models of biological interaction networks, the in silico networks used for its evaluation are nonlinear.”
MethodInfluentialGene Regulatory Network Inference from Systems Genetics Data Using Tree-Based Methods
Huynh‐Thu, V. A., Wehenkel, L. A., & Geurts, P. (2013) · Springer eBooks · doi:10.1007/978-3-642-45161-4_5 · Cited by 9
“tiative organizes an annual reverse engineering competition that comprises several challenges5 (Marbach 2012; Prill et al. 2010; Stolovitzky et al. 2009, 2007).”
BackgroundKim, D., Kang, M., Biswas, A. K., Liu, C., & Gao, J. (2016) · BMC Medical Genomics · doi:10.1186/s12920-016-0202-9 · Cited by 8
“In the experimental evaluation, we validate the proposed method on a dataset from the DREAM3 challenge [7].”
MethodPerkins, M., & Daniels, K. (2017) · International Conference on Information Visualisation (IV) · doi:10.1109/iv.2017.9 · Cited by 10
fastBMA: scalable network inference and transitive reduction
Hung, L., Shi, K., Wu, M., Young, W. C., Raftery, A. E., & Yeung, K. Y. (2017) · GigaScience · doi:10.1093/gigascience/gix078 · Cited by 8
“Simulated 10-gene and 100-gene time series data (5 sets of each) and the corresponding reference networks from DREAM4 [46–50].”
Integration of Multiple, Diverse Methods to Identify Biologically Significant Marker Genes
Chaney, C. P., Drake, K. A., & Carroll, T. J. (2022) · Journal of Molecular Biology · doi:10.1016/j.jmb.2022.167754 · Cited by 8
Segura-Ortiz, A., García-Nieto, J., Aldana-Montes, J. F., & Navas‐Delgado, I. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.108850 · Cited by 9
Issues impacting genetic network reverse engineering algorithm validation using small networks
Vinh, N. X., Chetty, M., Coppel, R. L., & Wangikar, P. P. (2012) · Biochimica et Biophysica Acta (BBA) - Proteins and Proteomics · doi:10.1016/j.bbapap.2012.05.017 · Cited by 8
Inference of dynamic biological networks based on responses to drug perturbations
Berlow, N., Davis, L. E., Keller, C., & Pal, R. (2014) · EURASIP Journal on Bioinformatics and Systems Biology · doi:10.1186/s13637-014-0014-1 · Cited by 7
Zhang, L., Wu, H. C., Ho, C. H., & Chan, S. C. (2018) · IEEE Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbb.2018.2828810 · Cited by 8
“…based on a synthetic time-varying GRN model in [33] and Dialogue for Reverse Engineering Assessments and Methods (DREAM) synthetic data-sets [34], [35], [36] show that the proposed method performs better than other conventional data-driven GRN identification methods such as the NIR and BANJO.”
MethodBayesian Model Averaging Methods and R Package for Gene Network Construction
Yeung, K. Y., Fraley, C., Young, C. D., Bumgarner, R. E., & Raftery, A. E. (2014) · University of Washington Tacoma Digital Commons (University of Washington Tacoma) · Cited by 7
CyNetworkBMA: a Cytoscape app for inferring gene regulatory networks
Fronczuk, M., Raftery, A. E., & Yeung, K. Y. (2015) · Source Code for Biology and Medicine · doi:10.1186/s13029-015-0043-5 · Cited by 9
“We will demonstrate the functionality of CyNetworkBMA by using one of the data sets from DREAM4 In Silico Network Challenge [32–34], specifically, the time series file for the first network of size 100.”
MethodInferring gene regulatory networks by an order independent algorithm using incomplete data sets
Aghdam, R., Ganjali, M., Niloofar, P., & Eslahchi, C. (2015) · Journal of Applied Statistics · doi:10.1080/02664763.2015.1079307 · Cited by 7
“The data set from DREAM3 challenge [44,46,47,56,59] is applied to investigate the performances of PCA–CMI–OI and ConMIT Score.”
MethodSemi-supervised network inference using simulated gene expression dynamics
Nguyen, P., & Braun, R. (2017) · Bioinformatics · doi:10.1093/bioinformatics/btx748 · Cited by 8
“Finally, in the DREAM4 in silico network challenge, GeneNetWeaver was used to apply various perturbations to 10- and 100-gene networks and the network response was measured before and after the perturbations were removed.”
Vatsa, D., & Agarwal, S. (2021) · PLoS ONE · doi:10.1371/journal.pone.0251666 · Cited by 9
“DREAM4 in silico network challenge provides biologically plausible simulated expression data of 5 networks of size 10-gene and 5 networks of size 100-genes with the aim to recover the underlying network [2, 26, 27].”
BackgroundExperimental Design in Dynamical System Identification: A Bandit-Based Active Learning Approach
Llamosi, A., Mezine, A., d’Alché–Buc, F., Letort, V., & Sébag, M. (2014) · Lecture notes in computer science · doi:10.1007/978-3-662-44851-9_20 · Cited by 9
“In this work we use the following metric to measure the quality of estimate θ̂, based on the DREAM 7 challenge [30]:”
BackgroundMethodScanning sample-specific miRNA regulation from bulk and single-cell RNA-sequencing data
Zhang, J., Liu, L., Wei, X., Zhao, C., Luo, Y., Li, J., & Le, T. D. (2024) · BMC Biology · doi:10.1186/s12915-024-02020-x · Cited by 7
“…[71], Kendall [72], Distance correlation (Dcor) [32], Random Dependence Coefficient (RDC) [73], Hoeffding’s D statistics (Hoeffding) [33], Z-score [74], Biweight midcorrelation (Bcor) [38], Weighted rank correlation (Wcor) [75], Cosine [76], Euclidean [77], Manhattan [78], Canberra [40],…”
BackgroundNiloofar, P., Aghdam, R., & Eslahchi, C. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.109238 · Cited by 8
Qiu, C., & Yang, K. (2024) · IEEE Transactions on Network Science and Engineering · doi:10.1109/tnse.2024.3487301 · Cited by 7
“Itremainstodateunder-explored todevelopamathematicalframeworkthatseamlesslyintegratesbothobservationalandinterventionaldataincausalstructure learning.”
Reconstructing ecological networks with hierarchical Bayesian regression and Mondrian processes
Aderhold, A., Husmeier, D., & Smith, V. A. (2013) · Enlighten: Publications (University of Glasgow) · Cited by 9
Dynamic modeling of the cellular senescence gene regulatory network
de Freitas, J. A. N. L. F., & Bischof, O. (2023) · Heliyon · doi:10.1016/j.heliyon.2023.e14007 · Cited by 7
Rapid Reconstruction of Time-Varying Gene Regulatory Networks
Pyne, S., Kumar, A. R., & Anand, A. (2018) · IEEE Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbb.2018.2861698 · Cited by 6
“These datasets are made publicly available as benchmarks for assessing and comparing the modelling algorithms through DREAM3 In Silico Network Challenge [27], [28].”
MethodCombining kinetic orders for efficient S-System modelling of gene regulatory network
Gill, J. K., Chetty, M., Shatte, A. B. R., & Hallinan, J. S. (2022) · Biosystems · doi:10.1016/j.biosystems.2022.104736 · Cited by 6
GSCA: Reconstructing biological pathway topologies using a cultural algorithms approach
Judeh, T., Jayyousi, T. W., Acharya, L. R., Reynolds, R. G., & Zhu, D. (2014) · IEEE Congress on Evolutionary Computation (CEC) · doi:10.1109/cec.2014.6900648 · Cited by 6
Identifying key genes in glaucoma based on a benchmarked dataset and the gene regulatory network
Chen, X., Wang, Q.-L., & Zhang, M. (2017) · Experimental and Therapeutic Medicine · doi:10.3892/etm.2017.4931 · Cited by 6
“Zscore executes knockout experiments more concretely and results in changes in other genes (22).”
Gaining confidence in inferred networks
Diaz, L., & Stumpf, M. P. H. (2022) · Scientific Reports · doi:10.1038/s41598-022-05402-9 · Cited by 9
Identification of direction in gene networks from expression and methylation
Simcha, D., Laurent Younes, B., Aryee, M. J., & Geman, D. (2013) · BMC Systems Biology · doi:10.1186/1752-0509-7-118 · Cited by 8
“The performance of the algorithm on synthetic data using the GeneNetWeaver simulator is excellent.”
MethodInfluentialScalable Inference of Gene Regulatory Networks with the Spark Distributed Computing Platform
Barba-González, C., García-Nieto, J., Benítez‐Hidalgo, A., Nebro, A. J., & Aldana-Montes, J. F. (2018) · Studies in computational intelligence · doi:10.1007/978-3-319-99626-4_6 · Cited by 6
An Interpretable Temporal Convolutional Framework for Granger Causality Analysis
Dong, A., Starr, A. G., & Zhao, Y. (2025) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2025.125396 · Cited by 5
Acampora, G., & Vitiello, A. (2015) · IEEE International Conference on Fuzzy Systems (FUZZ-IEEE) · doi:10.1109/fuzz-ieee.2015.7338070 · Cited by 5
“Foremost among the in silico networks are those released as part of the dialogue on reverse engineering assessment and methods (DREAM, see also the DREAM homepage1) [20][21].”
MethodBlum, C. F., Heramvand, N., Khonsari, A. S., & Kollmann, M. (2018) · Nature Communications · doi:10.1038/s41467-017-02489-x · Cited by 7
Detecting High-Dimensional Causal Networks Using Randomly Conditioned Granger Causality
Ma, H., Leng, S., & Chen, L. (2021) · CSIAM Transactions on Applied Mathematics · doi:10.4208/csiam-am.2020-0184 · Cited by 5
Hierarchical Probabilistic Interaction Modeling for Multiple Gene Expression Replicates
Patton, K. L., John, D. J., Norris, J. L., Lewis, D. R., & Muday, G. K. (2014) · IEEE Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbb.2014.2299804 · Cited by 7
ASDmiR: A Stepwise Method to Uncover miRNA Regulation Related to Autism Spectrum Disorder
Xiong, C., Sun, S., Jiang, W., Ma, L., & Zhang, J. (2020) · Frontiers in Genetics · doi:10.3389/fgene.2020.562971 · Cited by 9
Model-averaged ℓ1regularization using Markov chain Monte Carlo model composition
Fraley, C., & Percival, D. (2013) · Journal of Statistical Computation and Simulation · doi:10.1080/00949655.2013.861839 · Cited by 6
Clarke, N. D., & Bourque, G. (2010) · PLoS ONE · doi:10.1371/journal.pone.0008417 · Cited by 5
Self-citationLearning Causality: Synthesis of Large-Scale Causal Networks from High-Dimensional Time Series DataPreprint
Stehr, M.-O., Avar, P., Korte, A. R., Parvin, L., Sahab, Z. J., Bunin, D. I., Knapp, M., Nishita, D., Poggio, A., Talcott, C., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.02291 · Cited by 5
“cerevisiae, and in silico microarray data [24,28].”
BackgroundsREVEAL: Scalable extensions of REVEAL towards regulatory network inference
Chaitankar, V., Ghosh, P., Elasri, M. O., & Perkins, E. J. (2011) · International Conference on Intelligent Systems Design and Applications (ISDA) · doi:10.1109/isda.2011.6121850 · Cited by 4
Improvement of reliabilities of regulations using a hierarchical structure in a genetic network
Kimura, S., & Okada, M. (2015) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2015.7280362 · Cited by 4
“DREAM3, on the other hand, described these networks using a model different from the S-system model [14].”
MethodAn Integrative Approach to Infer Regulation Programs in a Transcription Regulatory Module Network
Qi, J., Michoel, T., & Butler, G. (2012) · BioMed Research International · doi:10.1155/2012/245968 · Cited by 4
“In addition, we are interested in performing experiments on synthetic datasets (e.g., DREAM [24]), where complete reference networks are available.”
On the Ensemble Prediction of Gene Regulatory Networks: A Comparative Study
Recamonde‐Mendoza, M., & Bazzan, A. L. C. (2012) · Brazilian Symposium on Neural Networks (SBRN) · doi:10.1109/sbrn.2012.27 · Cited by 4
“Given the set of predictions submitted by the participants of the DREAM3 challenge [7], which are given as ranked lists of predicted edges, authors have shown that inference methods have particular strengths and weaknesses, and that predictions drawn from the integration of multiple methods benefit from the variance i…”
BackgroundMargolin, A. A. (2013) · Cited by 4
Yin, Z., Wang, X., Liu, Y., Zhang, Z., Wang, S.-S., & Du, Y. (2017) · Journal of Cancer Research and Therapeutics · doi:10.4103/0973-1482.204842 · Cited by 4
Tree-Based Learning of Regulatory Network Topologies and Dynamics with Jump3
Huynh‐Thu, V. A., & Sanguinetti, G. (2018) · Methods in molecular biology · doi:10.1007/978-1-4939-8882-2_9 · Cited by 4
A Review of Mathematical Models for Muscular Dystrophy: A Systems Biology Approach
Houston, M. T., Cameron, A. N., & Gutiérrez, J. B. (2018) · PLoS Currents · doi:10.1371/currents.md.6af74d0cec0834554dac78f0045cfded · Cited by 6
Realistic simulation of time-course measurements in systems biology
Egert, J., & Kreutz, C. (2023) · Mathematical Biosciences & Engineering · doi:10.3934/mbe.2023467 · Cited by 4
Qualitative Reasoning for Biological Network Inference from Systematic Perturbation Experiments
Badaloni, S., Camillo, B. D., & Sambo, F. (2012) · IEEE Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbb.2012.69 · Cited by 4
“In this paper, we present Systematic Perturbation Qualitative Reasoning (SPQR), a novel Qualitative Reasoning approach to automate the analysis of systematic perturbation experiments.”
MethodScalable Probabilistic Causal Structure Discovery
Sridhar, D., Pujara, J., & Getoor, L. C. (2018) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/709 · Cited by 4
“Our second dataset from the DREAM4 challenge consists of a gold-standard yeast transcriptional regulatory network and simulated gene expression measurements [Marbach et al. , 2010; Prill et al. , 2010].”
MethodA Survey of Computational Approaches to Reconstruct and Partition Biological Networks
Acharya, L. R., Judeh, T., & Zhu, D. (2012) · Wiley eBooks · doi:10.1002/9781118346990.ch1 · Cited by 5
Robert, P. A., Jönsson, H., & Meyer‐Hermann, M. (2018) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/281188 · Cited by 4
Regenerative Engineering: Evolution and Its Modern Significance
Shi, Y., Zhang, L., Chen, M., Calle, E. A., & Langer, R. (2025) · Regenerative Engineering and Translational Medicine · doi:10.1007/s40883-025-00498-7 · Cited by 4
“Regenerative engineering involves dynamic interactions between cells, biomaterials, and environmental factors, which are difficult to fully capture using current AI models [151, 152].”
da Silva, J. E. H., Bernardino, H. S., de Oliveira, I. L., & Camata, J. J. (2025) · Biosystems · doi:10.1016/j.biosystems.2025.105464 · Cited by 6
Metadata
- Authors
- Prill, R. J., Marbach, D., Saez-Rodriguez, J., Sorger, P. K., Alexopoulos, L. G., Xue, X., Clarke, N. D., Altan-Bonnet, G., & Stolovitzky, G.
- Affiliations
- IBM (United States) (US)
- École Polytechnique Fédérale de Lausanne (CH)
- Harvard University (US)
- National Technical University of Athens (GR)
- Genome Institute of Singapore (SG)
- Corresponding author
- Stolovitzky, Gustavo
- ORCID
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2010-02-23
- Volume
- 5
- Issue
- 2
- Pages
- e9202
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC2826397
- OpenAlex
- W2143076232
- Cited by
- 439
- References
- 33
- Citations by year
- 2026: 16 · 2025: 16 · 2024: 15 · 2023: 11 · 2022: 21 · 2021: 18 · 2020: 17 · 2019: 17 · 2018: 28 · 2017: 27 · 2016: 23 · 2015: 50 · 2014: 40 · 2013: 38 · 2012: 47
- FWCI
- 18.23
- Type
- journal article
- Language
- en
- Update notices
- correction (2010-03-09)
- Open access
- gold
- doi.org
- Locations
- PLoS ONE (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Digital Access to Scholarship at Harvard (DASH) (Harvard University) (submittedVersion, cc-by)
- National University of Singapore (submittedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- DSpace - NTUA (National Technical University of Athens) (submittedVersion, other-oa)
- Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion)
- Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion, cc-by-nc-nd)
- Funders
- National Science Foundation (GM68762, 0848030)
- National Institutes of Health (3p50gm068762-07s2, NIH P50, GM68762)
- National Cancer Institute (U54CA112967)
- National Institute of General Medical Sciences (P50 GM68762, P50-GM068762)
- National Institute of Allergy and Infectious Diseases (R56AI083408, AI083408, R01AI083408)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene Regulatory Network Analysis
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Bioinformatics and Genomic Networks
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Microbial Metabolic Engineering and Bioproduction
- Keywords
- network inference, reverse engineering, gene expression prediction, ensemble forecasting, model validation
- MeSH terms
- Algorithms, Animals, Humans, Models, Biological, Software, Reproducibility of Results, Signal Transduction, Cluster Analysis, Computational Biology, Gene Expression Profiling, Protein Interaction Mapping, Systems Biology, Gene Regulatory Networks
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.