Wisdom of crowds for robust gene network inference
Open accessAbstract
Reconstructing gene regulatory networks from high-throughput data is a long-standing challenge. Through the Dialogue on Reverse Engineering Assessment and Methods (DREAM) project, we performed a comprehensive blind assessment of over 30 network inference methods on Escherichia coli, Staphylococcus aureus, Saccharomyces cerevisiae and in silico microarray data. We characterize the performance, data requirements and inherent biases of different inference approaches, and we provide guidelines for algorithm application and development. We observed that no single inference method performs optimally across all data sets. In contrast, integration of predictions from multiple inference methods shows robust and high performance across diverse data sets. We thereby constructed high-confidence networks for E. coli and S. aureus, each comprising ~1,700 transcriptional interactions at a precision of ~50%. We experimentally tested 53 previously unobserved regulatory interactions in E. coli, of which 23 (43%) were supported. Our results establish community-based methods as a powerful and robust tool for the inference of transcriptional gene regulatory networks.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Ensemble inference methodologies were identified as more robust for identification of gene-gene interactions from coexpression data.
“where ensemble inference methodologies were identified as more robust for identification of gene-gene interactions from coexpression data.”
From Meta-Analysis of the Alzheimer’s Disease Human Brain Transcriptome and Functional Dissection in Mouse Models (2020) · cites this paper as Marbach et al., 2012
To obtain module-level reproducibility, 20 independent samples are usually sufficient, but systematic and accurate reconstruction of specific edges may require hundreds of samples.
“To obtain module-level reproducibility, 20 independent samples are usually sufficient40, but systematic and accurate reconstruction of specific edges, particularly for systematic regulatory relationship discovery, may require hundreds of samples49.”
From Systems biology and gene networks in neurodevelopmental and neurodegenerative disorders (2015) · cites this paper as [49]
At currently available sample sizes, networks are most reproducible at module level, then hub gene level, and least at precise gene connectivity rankings or module memberships.
“At currently available sample sizes, networks are most reproducible at a module level35,38,39, then at the hub gene level41,223 and, last, at the level of precise gene connectivity rankings or precise module memberships of genes40,49.”
From Systems biology and gene networks in neurodevelopmental and neurodegenerative disorders (2015) · cites this paper as [49]
The DREAM challenge identified that integration of multiple network methods yields the most robust regulatory relationship predictions.
“The DREAM challenge identified that the integration of multiple network methods yields the most robust regulatory relationship predictions49.”
From Systems biology and gene networks in neurodevelopmental and neurodegenerative disorders (2015) · cites this paper as [49]
To obtain module-level reproducibility, 20 independent samples are typically considered adequate.
“To obtain module-level reproducibility, 20 independent samples are typically considered adequate; however, systematic consistency at the more granular hub and module membership levels may require hundreds of samples [61, 93, 94].”
From Systems-based proteomics to resolve the biology of Alzheimer’s disease beyond amyloid and tau (2021) · cites this paper as [94]
Network analyses of AD brain tissue demonstrate the most reproducibility at the module level, followed by the hub protein level, and then at the level of precise protein connectivity rankings and module protein membership.
“Network analyses of AD brain tissue demonstrate the most reproducibility at the module level, followed by the hub protein level, and then finally at the level of precise protein connectivity rankings and module protein membership [61, 89–94].”
From Systems-based proteomics to resolve the biology of Alzheimer’s disease beyond amyloid and tau (2021) · cites this paper as [94]
Systematic consistency at the more granular hub and module membership levels may require hundreds of samples.
“To obtain module-level reproducibility, 20 independent samples are typically considered adequate; however, systematic consistency at the more granular hub and module membership levels may require hundreds of samples [61, 93, 94].”
From Systems-based proteomics to resolve the biology of Alzheimer’s disease beyond amyloid and tau (2021) · cites this paper as [94]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482 · see citing works
Modularity and community structure in networks
Newman, M. E. J. (2006) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0601602103 · Cited by 12,256 · see citing works
Ensemble methods in machine learning
Dietterich, T. G. (2000) · Lecture Notes in Computer Science · doi:10.1007/3-540-45014-9_1 · Cited by 8,408 · see citing works
Bolstad, B. M., Irizarry, R. A., Åstrand, M., & Speed, T. P. (2003) · Bioinformatics · doi:10.1093/bioinformatics/19.2.185 · Cited by 8,185 · see citing works
Model Selection and Estimation in Regression with Grouped Variables
Yuan, M., & Lin, Y. (2006) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2005.00532.x · Cited by 7,261 · see citing works
Using Bayesian Networks to Analyze Expression Data
Friedman, N., Michal Linial, B., Nachman, I., & Pe’er, D. (2000) · Journal of Computational Biology · doi:10.1089/106652700750050961 · Cited by 3,248 · see citing works
Margolin, A. A., Nemenman, I., Basso, K., Wiggins, C., Stolovitzky, G., Favera, R. D., & Califano, A. (2006) · BMC Bioinformatics · doi:10.1186/1471-2105-7-s1-s7 · Cited by 2,420 · see citing works
Self-citationMeinshausen, N., & Bühlmann, P. (2010) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.1467-9868.2010.00740.x · Cited by 2,314 · see citing works
Inferring Regulatory Networks from Expression Data Using Tree-Based Methods
Huynh-Thu, V. A., Irrthum, A., Wehenkel, L., & Geurts, P. (2010) · PLoS ONE · doi:10.1371/journal.pone.0012776 · Cited by 2,131 · see citing works
Transcriptional regulatory code of a eukaryotic genome
Harbison, C. T., Gordon, D. B., Lee, T. I., Rinaldi, N. J., MacIsaac, K. D., Danford, T., Hannett, N. M., Tagne, J., Reynolds, D. B., Yoo, J. Y., et al. (2004) · Nature · doi:10.1038/nature02800 · Cited by 2,116 · see citing works
Self-citationReich, M. M., Liefeld, T., Gould, J., Lerner, J., Tamayo, P., & Mesirov, J. P. (2006) · Nature Genetics · doi:10.1038/ng0506-500 · Cited by 2,035 · see citing works
Faith, J. J., Hayete, B., Thaden, J. T., Mogno, I., Wierzbowski, J., Cottarel, G., Kasif, S., Collins, J. J., & Gardner, T. S. (2007) · PLoS Biology · doi:10.1371/journal.pbio.0050008 · Cited by 1,563 · see citing works
Self-citationNCBI GEO: archive for functional genomics data sets--10 years on
Barrett, T., Troup, D. B., Wilhite, S. E., Ledoux, P., Evangelista, C., Kim, I. F., Tomashevsky, M., Marshall, K. A., Phillippy, K., Sherman, P. M., et al. (2010) · Nucleic Acids Research · doi:10.1093/nar/gkq1184 · Cited by 1,127 · see citing works
Similar network activity from disparate circuit parameters
Prinz, A. A., Bucher, D., & Marder, E. (2004) · Nature Neuroscience · doi:10.1038/nn1352 · Cited by 1,091 · see citing works
Butte, A. J., & Kohane, I. S. (1999) · Pacific Symposium on Biocomputing (PSB) · doi:10.1142/9789814447331_0040 · Cited by 956 · see citing works
Revealing strengths and weaknesses of methods for gene network inference
Marbach, D., Prill, R. J., Schaffter, T., Mattiussi, C., Floreano, D., & Stolovitzky, G. A. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0913357107 · Cited by 755 · see citing works
Self-citationLessons from the Netflix prize challenge
Bell, R. M., & Koren, Y. (2007) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1345448.1345465 · Cited by 727 · see citing works
Computational discovery of gene modules and regulatory networks
Bar‐Joseph, Z., Gerber, G. K., Lee, T. I., Rinaldi, N. J., Yoo, J. Y., Robert, F., Gordon, D. B., Fraenkel, E., Jaakkola, T., Young, R. A., et al. (2003) · Nature Biotechnology · doi:10.1038/nbt890 · Cited by 606 · see citing works
Yeung, K., Bumgarner, R. E., & Raftery, A. (2005) · Bioinform. · doi:10.1093/bioinformatics/bti319 · Cited by 592 · see citing works
GeneNetWeaver: in silico benchmark generation and performance profiling of network inference methods
Schaffter, T., Marbach, D., & Floreano, D. (2011) · Bioinformatics · doi:10.1093/bioinformatics/btr373 · Cited by 538 · see citing works
Self-citationAdvantages and limitations of current network inference methods
Smet, R. D., & Marchal, K. (2010) · Nature Reviews Microbiology · doi:10.1038/nrmicro2419 · Cited by 510 · see citing works
A decade of CASP: progress, bottlenecks and prognosis in protein structure prediction
Moult, J. (2005) · Current Opinion in Structural Biology · doi:10.1016/j.sbi.2005.05.011 · Cited by 485 · see citing works
Multiple models to capture the variability in biological neurons and networks
Marder, E., & Taylor, A. L. (2011) · Nature Neuroscience · doi:10.1038/nn.2735 · Cited by 451 · see citing works
Towards a rigorous assessment of systems biology models: the DREAM3 challenges
Prill, R. J., Marbach, D., Saez-Rodriguez, J., Sorger, P. K., Alexopoulos, L. G., Xue, X., Clarke, N. D., Altan-Bonnet, G., & Stolovitzky, G. (2010) · PLoS ONE · doi:10.1371/journal.pone.0009202 · Cited by 439 · see citing works
Self-citationMarbach, D., Schaffter, T., Mattiussi, C., & Floreano, D. (2009) · Journal of Computational Biology · doi:10.1089/cmb.2008.09tt · Cited by 431 · see citing works
Self-citationGenetic reconstruction of a functional transcriptional regulatory network
Hu, Z., Killion, P. J., & Iyer, V. R. (2007) · Nature Genetics · doi:10.1038/ng2012 · Cited by 415 · see citing works
TIGRESS: Trustful Inference of Gene REgulation using Stability Selection
Haury, A.-C., Mordelet, F., Vera‐Licona, P., & Vert, J. (2012) · BMC Systems Biology · doi:10.1186/1752-0509-6-145 · Cited by 414 · see citing works
Dialogue on Reverse‐Engineering Assessment and Methods
Stolovitzky, G. A., Monroe, D., & Califano, A. (2007) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1407.021 · Cited by 377 · see citing works
Self-citationGama‐Castro, S., Salgado, H., Peralta-Gil, M., Santos-Zavaleta, A., Muñiz-Rascado, L., Solano-Lira, H., Jacinto, V. J., Weiss, V., García-Sotelo, J. S., López-Fuentes, A., et al. (2010) · Nucleic Acids Research · doi:10.1093/nar/gkq1110 · Cited by 353 · see citing works
Time and sample efficient discovery of Markov blankets and direct causal relations
Tsamardinos, I., Aliferis, C., & Statnikov, A. (2003) · ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/956750.956838 · Cited by 348 · see citing works
Chemogenomic profiling on a genome-wide scale using reverse-engineered gene networks
di Bernardo, D., Thompson, M. J., Gardner, T. S., Chobot, S. E., Eastwood, E. L., Wojtovich, A. P., Elliott, S. J., Schaus, S. E., & Collins, J. J. (2005) · Nature Biotechnology · doi:10.1038/nbt1075 · Cited by 347 · see citing works
Self-citationFaith, J. J., Driscoll, M. E., Fusaro, V. A., Cosgrove, E. J., Hayete, B., Juhn, F., Schneider, S. J., & Gardner, T. S. (2007) · Nucleic Acids Research · doi:10.1093/nar/gkm815 · Cited by 259 · see citing works
Greenfield, A., Madar, A., Ostrer, H., & Bonneau, R. (2010) · PLoS ONE · doi:10.1371/journal.pone.0013397 · Cited by 255 · see citing works
Reiss, D. J., Baliga, N. S., & Bonneau, R. (2006) · BMC Bioinformatics · doi:10.1186/1471-2105-7-280 · Cited by 253 · see citing works
Lessons from the DREAM2 Challenges
Stolovitzky, G. A., Prill, R. J., & Califano, A. (2009) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.2009.04497.x · Cited by 200 · see citing works
Self-citationSIRENE: supervised inference of regulatory networks
Mordelet, F., & Vert, J. (2008) · Bioinformatics · doi:10.1093/bioinformatics/btn273 · Cited by 191 · see citing works
Statistical inference of the time-varying structure of gene-regulation networks
Lèbre, S., Becq, J., Devaux, F., Stumpf, M. P. H., & Lelandais, G. (2010) · BMC Systems Biology · doi:10.1186/1752-0509-4-130 · Cited by 189 · see citing works
Ensemble modeling for analysis of cell signaling dynamics
Kuepfer, L., Peter, M., Sauer, U., & Stelling, J. (2007) · Nature Biotechnology · doi:10.1038/nbt1330 · Cited by 175 · see citing works
Marbach, D., Roy, S., Ay, F., Meyer, P. E., Candeias, R., Kahveci, T., Bristow, C. A., & Kellis, M. (2012) · Genome Research · doi:10.1101/gr.127191.111 · Cited by 119 · see citing works
Self-citationInferring gene regulatory networks by ANOVA
Küffner, R., Petri, T., Tavakkolkhah, P., Windhager, L., & Zimmer, R. (2012) · Bioinformatics · doi:10.1093/bioinformatics/bts143 · Cited by 117 · see citing works
Self-citationYip, K. Y.-L., Alexander, R. P., Yan, K., & Gerstein, M. (2010) · PLoS ONE · doi:10.1371/journal.pone.0008121 · Cited by 104 · see citing works
Inference of Regulatory Gene Interactions from Expression Data Using Three‐Way Mutual Information
Watkinson, J., Liang, K., Wang, X., Zheng, T., & Anastassiou, D. (2009) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.2008.03757.x · Cited by 84 · see citing works
Least absolute regression network analysis of the murine osteoblast differentiation network
van Someren, E. P., Vaes, B. L. T., Steegenga, W. T., Sijbers, A. M., Dechering, K. J., & Reinders, M. J. T. (2005) · Bioinformatics · doi:10.1093/bioinformatics/bti816 · Cited by 81 · see citing works
Analysis and Computational Dissection of Molecular Signature Multiplicity
Statnikov, A., & Aliferis, C. (2010) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000790 · Cited by 63 · see citing works
Lemmens, K., Bie, T. D., Dhollander, T., Keersmaecker, S. C. J. D., Thijs, I. M. V., Schoofs, G., Weerdt, A. D., Moor, B. D., Vanderleyden, J., Collado‐Vides, J., et al. (2009) · Genome biology · doi:10.1186/gb-2009-10-3-r27 · Cited by 55 · see citing works
Ravcheev, D. A., Best, A. A., Tintle, N., DeJongh, M., Osterman, A. L., Novichkov, P. S., & Rodionov, D. A. (2011) · Journal of Bacteriology · doi:10.1128/jb.00350-11 · Cited by 49 · see citing works
Systems analysis of cellular networks under uncertainty
Kaltenbach, H.-M., Dimopoulos, S., & Stelling, J. (2009) · FEBS Letters · doi:10.1016/j.febslet.2009.10.074 · Cited by 40 · see citing works
Karlebach, G., & Ron Shamir, A. (2012) · Journal of Computational Biology · doi:10.1089/cmb.2011.0100 · Cited by 32 · see citing works
Marbach, D., Mattiussi, C., & Floreano, D. (2009) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.2008.03945.x · Cited by 32 · see citing works
Self-citationSong, M. J., Lewis, C. K., Lance, E. R., Chesler, E. J., Yordanova, R., Langston, M. A., Lodowski, K. H., & Bergeson, S. E. (2009) · EURASIP Journal on Bioinformatics and Systems Biology · doi:10.1155/2009/545176 · Cited by 30 · see citing works
Stages of Gene Regulatory Network Inference: the Evolutionary Algorithm Role
Sîrbu, A., Heather, J., & Crane, M. (2011) · InTech eBooks · doi:10.5772/15182 · Cited by 11 · see citing works
An improved map of conserved regulatory sites for Saccharomyces cerevisiae
MacIsaac, K. D., Wang, T., Gordon, D. B., Gifford, D. K., Stormo, G. D., & Fraenkel, E. (2006) · BMC Bioinformatics · doi:10.1186/1471-2105-7-113 · Cited by 9 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- [39]Mani S Cooper GF A Bayesian Local Causal Discovery Algorithm Proceedings of the World Congress on Medical Informatics 731 735 2004 15360909
- [41]Aliferis CF Statnikov A Tsamardinos I Mani S Koutsoukos XD Local causal and Markov blanket induction for causal discovery and feature selection for classification. Part I: Algorithm and Empirical Evaluation Journal of Machine Learning Research 11 171 234 2010
Cited by
Sorted by citations
Scenic: single-cell regulatory network inference and clustering.
Aibar, S., González-Blas, C. B., Moerman, T., Huynh-Thu, V. A., Imrichova, H., Hulselmans, G., Rambow, F., Marine, J.-C., Geurts, P., Aerts, J., et al. (2017) · Nature Methods · doi:10.1038/nmeth.4463 · Cited by 6,520 · see citing works
Transcriptome-Based Network Analysis Reveals a Spectrum Model of Human Macrophage Activation
Xue, J., Schmidt, S. V., Sander, J., Draffehn, A., Krebs, W., Quester, I., De Nardo, D., Gohel, T. D., Emde, M., Schmidleithner, L., et al. (2014) · Immunity · doi:10.1016/j.immuni.2014.01.006 · Cited by 2,210 · see citing works
A comparison of single-cell trajectory inference methods
Saelens, W., Cannoodt, R., Todorov, H., & Saeys, Y. (2019) · Nature Biotechnology · doi:10.1038/s41587-019-0071-9 · Cited by 1,670 · see citing works
A Validated Regulatory Network for Th17 Cell Specification
Ciofani, M., Madar, A., Galan, C., Sellars, M., Mace, K., Pauli, F., Agarwal, A., Huang, W., Parkurst, C. N., Muratet, M., et al. (2012) · Cell · doi:10.1016/j.cell.2012.09.016 · Cited by 1,160 · see citing works
Gene co-expression analysis for functional classification and gene–disease predictions
van Dam, S., Võsa, U., van der Graaf, A., Franke, L., & de Magalhães, J. P. (2017) · Briefings in Bioinformatics · doi:10.1093/bib/bbw139 · Cited by 1,086 · see citing works
Benchmarking algorithms for gene regulatory network inference from single-cell transcriptomic data
Pratapa, A., Jalihal, A. P., Law, J. N., Bharadwaj, A., & Murali, T. M. (2020) · Nature Methods · doi:10.1038/s41592-019-0690-6 · Cited by 792 · see citing works
GRNBoost2 and Arboreto: efficient and scalable inference of gene regulatory networks
Moerman, T., Aibar Santos, S., Bravo González-Blas, C., Simm, J., Moreau, Y., Aerts, J., & Aerts, S. (2019) · Bioinformatics · doi:10.1093/bioinformatics/bty916 · Cited by 621 · see citing works
Gene Regulatory Network Inference from Single-Cell Data Using Multivariate Information Measures
Chan, T. E., Stumpf, M. P. H., & Babtie, A. C. (2017) · Cell Systems · doi:10.1016/j.cels.2017.08.014 · Cited by 578 · see citing works
Systems genetics approaches to understand complex traits
Civelek, M., & Lusis, A. J. (2014) · Nature Reviews Genetics · doi:10.1038/nrg3575 · Cited by 621 · see citing works
Systematic evaluation of spliced alignment programs for RNA-seq data
The RGASP Consortium, Engström, P. G., Steijger, T., Sipos, B., Grant, G. R., Kahles, A., Rätsch, G., Goldman, N., Hubbard, T. J., Harrow, J., et al. (2013) · Nature Methods · doi:10.1038/nmeth.2722 · Cited by 513 · see citing works
Self-citationTRRUST: a reference database of human transcriptional regulatory interactions
Han, H., Shim, H., Shin, D., Shim, J. E., Ko, Y., Shin, J., Kim, H., Cho, A., Kim, E., Lee, T., et al. (2015) · Scientific Reports · doi:10.1038/srep11432 · Cited by 462 · see citing works
Matsumoto, H., Kiryu, H., Furusawa, C., Ko, M. S. H., Ko, S. B. H., Gouda, N., Hayashi, T., & Nikaido, I. (2017) · Bioinformatics · doi:10.1093/bioinformatics/btx194 · Cited by 439 · see citing works
Systems biology and gene networks in neurodevelopmental and neurodegenerative disorders
Parikshak, N. N., Gandal, M. J., & Geschwind, D. H. (2015) · Nature Reviews Genetics · doi:10.1038/nrg3934 · Cited by 430 · see citing works
Tissue-specific regulatory circuits reveal variable modular perturbations across complex diseases
Marbach, D., Lamparter, D., Quon, G., Kellis, M., Kutalik, Z., & Bergmann, S. (2016) · Nature Methods · doi:10.1038/nmeth.3799 · Cited by 326 · see citing works
Self-citationPassing Messages between Biological Networks to Refine Predicted Interactions
Glass, K., Huttenhower, C., Quackenbush, J., & Yuan, G.-C. (2013) · PLoS ONE · doi:10.1371/journal.pone.0064832 · Cited by 266 · see citing works
A community computational challenge to predict the activity of pairs of compounds
Community, N.-D., Bansal, M., Yang, J., Karan, C., Menden, M. P., Costello, J. C., Tang, H., Xiao, G., Li, Y., Allen, J., et al. (2014) · Nature Biotechnology · doi:10.1038/nbt.3052 · Cited by 321 · see citing works
Self-citationEmmert-Streib, F., Dehmer, M., & Haibe-Kains, B. (2014) · Frontiers in Cell and Developmental Biology · doi:10.3389/fcell.2014.00038 · Cited by 282 · see citing works
Wan, Y.-W., Al-Ouran, R., Mangleburg, C. G., Perumal, T. M., Lee, T. V., Allison, K., Swarup, V., Funk, C. C., Gaiteri, C., Allen, M., et al. (2020) · Cell Reports · doi:10.1016/j.celrep.2020.107908 · Cited by 295 · see citing works
“The metanetwork inference methodology is inspired by the DREAM5 method (Marbach et al., 2012), where ensemble inference methodologies were identified as more robust for identification of gene-gene interactions from coexpression data.”
MethodSelf-citationComprehensive benchmarking and ensemble approaches for metagenomic classifiers
McIntyre, A. B. R., Ounit, R., Afshinnekoo, E., Prill, R. J., Hénaff, E., Alexander, N., Minot, S. S., Danko, D., Foox, J., Ahsanuddin, S., et al. (2017) · Genome Biology · doi:10.1186/s13059-017-1299-7 · Cited by 342 · see citing works
Self-citationMapping gene regulatory networks from single-cell omics data
Fiers, M. W. E. J., Minnoye, L., Aibar, S., Bravo González-Blas, C., Kalender Atak, Z., & Aerts, S. (2018) · Briefings in Functional Genomics · doi:10.1093/bfgp/elx046 · Cited by 235 · see citing works
Reverse engineering and identification in systems biology: strategies, perspectives and challenges
Villaverde, A. F., & Banga, J. R. (2014) · Journal of The Royal Society Interface · doi:10.1098/rsif.2013.0505 · Cited by 229 · see citing works
Chen, S., & Mar, J. C. (2018) · BMC Bioinformatics · doi:10.1186/s12859-018-2217-z · Cited by 235 · see citing works
Network inference with Granger causality ensembles on single-cell transcriptomics
Deshpande, A., Chu, L.-F., Stewart, R., & Gitter, A. (2022) · Cell Reports · doi:10.1016/j.celrep.2022.110333 · Cited by 130 · see citing works
Model selection in systems and synthetic biology
Kirk, P., Thorne, T., & Stumpf, M. P. (2013) · Current Opinion in Biotechnology · doi:10.1016/j.copbio.2013.03.012 · Cited by 118 · see citing works
Spearheading future omics analyses using dyngen, a multi-modal simulator of single cells
Cannoodt, R., Saelens, W., Deconinck, L., & Saeys, Y. (2021) · Nature Communications · doi:10.1038/s41467-021-24152-2 · Cited by 124 · see citing works
Systems-based proteomics to resolve the biology of Alzheimer’s disease beyond amyloid and tau
Rayaprolu, S., Higginbotham, L., Bagchi, P., Watson, C. M., Zhang, T., Levey, A. I., Rangaraju, S., & Seyfried, N. T. (2021) · Neuropsychopharmacology · doi:10.1038/s41386-020-00840-3 · Cited by 118 · see citing works
Aubin-Frankowski, P.-C., & Vert, J.-P. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btaa576 · Cited by 116 · see citing works
The need for an integrated multi-OMICs approach in microbiome science in the food system
Ferrocino, I., Rantsiou, K., McClure, R., Kostic, T., de Souza, R. S. C., Lange, L., FitzGerald, J., Kriaa, A., Cotter, P., Maguin, E., et al. (2023) · Comprehensive Reviews in Food Science and Food Safety · doi:10.1111/1541-4337.13103 · Cited by 110 · see citing works
Analytical tools and current challenges in the modern era of neuroepigenomics
Maze, I., Shen, L., Zhang, B., Garcia, B. A., Shao, N., Mitchell, A., Sun, H., Akbarian, S., Allis, C. D., & Nestler, E. J. (2014) · Nature Neuroscience · doi:10.1038/nn.3816 · Cited by 100 · see citing works
BTR: training asynchronous Boolean models using single-cell expression data
Lim, C. Y., Wang, H., Woodhouse, S., Piterman, N., Wernisch, L., Fisher, J., & Göttgens, B. (2016) · BMC Bioinformatics · doi:10.1186/s12859-016-1235-y · Cited by 81 · see citing works
Wang, H., Li, Y., Ryder, J. W., Hole, J. T., Ebert, P. J., Airey, D. C., Qian, H.-R., Logsdon, B., Fisher, A., Ahmed, Z., et al. (2018) · Molecular Neurodegeneration · doi:10.1186/s13024-018-0296-y · Cited by 83 · see citing works
CellOracle: Dissecting cell identity via network inference and in silico gene perturbation
Kamimoto, K., Hoffmann, C. M., & Morris, S. A. (2020) · doi:10.1101/2020.02.17.947416 · Cited by 81 · see citing works
Network modeling of single-cell omics data: challenges, opportunities, and progresses
Blencowe, M., Arneson, D., Ding, J., Chen, Y.-W., Saleem, Z., & Yang, X. (2019) · Emerging Topics in Life Sciences · doi:10.1042/etls20180176 · Cited by 73 · see citing works
Network Analysis Identifies Mitochondrial Regulation of Epidermal Differentiation by MPZL3 and FDXR
Bhaduri, A., Ungewickell, A., Boxer, L. D., Lopez-Pajares, V., Zarnegar, B. J., & Khavari, P. A. (2015) · Developmental Cell · doi:10.1016/j.devcel.2015.10.023 · Cited by 60 · see citing works
Data-driven reverse engineering of signaling pathways using ensembles of dynamic models
Henriques, D., Villaverde, A. F., Rocha, M., Saez-Rodriguez, J., & Banga, J. R. (2017) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1005379 · Cited by 44 · see citing works
Evaluating the Reproducibility of Single-Cell Gene Regulatory Network Inference Algorithms
Kang, Y., Thieffry, D., & Cantini, L. (2021) · Frontiers in Genetics · doi:10.3389/fgene.2021.617282 · Cited by 41 · see citing works
Prediction of Genetic Interactions Using Machine Learning and Network Properties
Madhukar, N. S., Elemento, O., & Pandey, G. (2015) · Frontiers in Bioengineering and Biotechnology · doi:10.3389/fbioe.2015.00172 · Cited by 39 · see citing works
How difficult is inference of mammalian causal gene regulatory networks?
Djordjevic, D., Yang, A., Zadoorian, A., Rungrugeecharoen, K., & Ho, J. W. K. (2014) · PLoS ONE · doi:10.1371/journal.pone.0111661 · Cited by 29 · see citing works
Ishikawa, M., Sugino, S., Masuda, Y., Tarumoto, Y., Seto, Y., Taniyama, N., Wagai, F., Yamauchi, Y., Kojima, Y., Kiryu, H., et al. (2023) · Communications Biology · doi:10.1038/s42003-023-05594-4 · Cited by 22 · see citing works
csuWGCNA: a combination of signed and unsigned WGCNA to capture negative correlations
Dai, R., Xia, Y., Liu, C., & Chen, C. (2018) · doi:10.1101/288225 · Cited by 13 · see citing works
Johnson, Z., Anderson, D., Cheung, M. S., & Bohutskyi, P. (2025) · Frontiers in Microbiology · doi:10.3389/fmicb.2025.1569559 · Cited by 12 · see citing works
Kuehne, A., Mayr, U., Sévin, D. C., Claassen, M., & Zamboni, N. (2017) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1005577 · Cited by 9 · see citing works
Zinati, Y., Takiddeen, A., & Emad, A. (2023) · doi:10.1101/2023.07.25.550225 · Cited by 36 · see citing works
- Authors
- The DREAM5 Consortium, Marbach, D., Costello, J. C., Küffner, R., Vega, N. M., Prill, R. J., Camacho, D. M., Allison, K. R., Kellis, M., Collins, J. J., & Stolovitzky, G.
- Affiliations
- MIT Computer Science and Artificial Intelligence Laboratory (US)
- Boston University (US)
- Ludwig-Maximilians-Universität München (DE)
- IBM (United States) (US)
- Broad Institute (US)
- Corresponding author
- Stolovitzky, Gustavo
- ORCID
- Venue
- Nature Methods
- Published
- 2012-07-15
- Volume
- 9
- Issue
- 8
- Pages
- 796-804
- ISSN
- 1548-7091 (ISSN-L)
- 1548-7105
- PMC
- PMC3512113
- OpenAlex
- W1543475572
- Cited by
- 1,667
- References
- 52
- Citations by year
- 2026: 76 · 2025: 70 · 2024: 90 · 2023: 102 · 2022: 112 · 2021: 136 · 2020: 147 · 2019: 133 · 2018: 140 · 2017: 147 · 2016: 154 · 2015: 142 · 2014: 118 · 2013: 82 · 2012: 18
- FWCI
- 46.38
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Nature Methods (publishedVersion)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- DSpace@MIT (Massachusetts Institute of Technology) (submittedVersion)
- Ghent University Academic Bibliography (Ghent University) (submittedVersion)
- CINECA IRIS Institutial research information system (University of Pisa) (submittedVersion)
- Archivio istituzionale della ricerca (Alma Mater Studiorum Università di Bologna) (submittedVersion)
- ENLIGHTEN (Jurnal Bimbingan dan Konseling Islam) (submittedVersion)
- UNICA IRIS Institutional Research Information System (University of Cagliari) (submittedVersion)
- Institutional Research Information System University of Turin (University of Turin) (submittedVersion)
- mediaTUM (Technical University of Munich) (submittedVersion)
- mediaTUM – the media and publications repository of the Technical University Munich (Technical University Munich) (submittedVersion, other-oa)
- ORBi (University of Liège) (submittedVersion)
- Use Siena air (University of Siena) (submittedVersion)
- PMC
- Funders
- National Science Foundation (U54CA132383, 0420407)
- Howard Hughes Medical Institute
- Agence Nationale de la Recherche (ANR-09-BLAN-0051, ANR-07-BLAN-0311)
- Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung (200767)
- Irish Research Council
- Fonds De La Recherche Scientifique - FNRS
- Fédération Wallonie-Bruxelles
- Fonds Wetenschappelijk Onderzoek
- Tel Aviv University
- Vlaamse regering
- Irish Centre for High-End Computing
- National Institutes of Health (DP1OD003644)
- National Human Genome Research Institute (R01HG004037)
- National Cancer Institute (U54CA121852, U54CA132383)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene Regulatory Network Analysis
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Microbial Metabolic Engineering and Bioproduction
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Bioinformatics and Genomic Networks
- Keywords
- gene regulatory network inference, wisdom of crowds, ensemble methods, DREAM challenge, Escherichia coli, Staphylococcus aureus, Saccharomyces cerevisiae, methodological bias
- MeSH terms
- Algorithms, Escherichia coli, Saccharomyces cerevisiae, Software, Staphylococcus aureus, Transcription, Genetic, Gene Expression Regulation, Bacterial, Computational Biology, Oligonucleotide Array Sequence Analysis, Gene Regulatory Networks
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
