Evaluating methods of inferring gene regulatory networks highlights their lack of performance for single cell gene expression data
Open accessAbstract
BACKGROUND: A fundamental fact in biology states that genes do not operate in isolation, and yet, methods that infer regulatory networks for single cell gene expression data have been slow to emerge. With single cell sequencing methods now becoming accessible, general network inference algorithms that were initially developed for data collected from bulk samples may not be suitable for single cells. Meanwhile, although methods that are specific for single cell data are now emerging, whether they have improved performance over general methods is unknown. In this study, we evaluate the applicability of five general methods and three single cell methods for inferring gene regulatory networks from both experimental single cell gene expression data and in silico simulated data. RESULTS: Standard evaluation metrics using ROC curves and Precision-Recall curves against reference sets sourced from the literature demonstrated that most of the methods performed poorly when they were applied to either experimental single cell data, or simulated single cell data, which demonstrates their lack of performance for this task. Using default settings, network methods were applied to the same datasets. Comparisons of the learned networks highlighted the uniqueness of some predicted edges for each method. The fact that different methods infer networks that vary substantially reflects the underlying mathematical rationale and assumptions that distinguish network methods from each other. CONCLUSIONS: This study provides a comprehensive evaluation of network modeling algorithms applied to experimental single cell gene expression data and in silico simulated datasets where the network structure is known. Comparisons demonstrate that most of these assessed network methods are not able to predict network structures from single cell expression data accurately, even if they are specifically developed for single cell methods. Also, single cell methods, which usually depend on more elaborative algorithms, in general have less similarity to each other in the sets of edges detected. The results from this study emphasize the importance for developing more accurate optimized network modeling methods that are compatible for single cell data. Newly-developed single cell methods may uniquely capture particular features of potential gene-gene relationships, and caution should be taken when we interpret these results.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
A recent comparison has shown both bulk and single-cell GRN inference methods to perform poorly on scRNA-seq data.
“While there exist GRN inference methods that were specifically developed for scRNA‐seq data (SCONE: Matsumoto et al, 2017; PIDC: Chan et al, 2017; SCENIC: Aibar et al, 2017), a recent comparison has shown both bulk and single‐cell methods to perform poorly on these data (Chen & Mar, 2018).”
From Current best practices in single-cell RNA-seq analysis: a tutorial (2019) · cites this paper as Chen & Mar, 2018
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Emergence of Scaling in Random Networks
Barabási, A.-L., & Albert, R. (1999) · Science · doi:10.1126/science.286.5439.509 · Cited by 36,135 · see citing works
STRING v10: protein–protein interaction networks, integrated over the tree of life
Szklarczyk, D., Franceschini, A., Wyder, S., Forslund, K., Heller, D., Huerta-Cepas, J., Simonovic, M., Roth, A., Santos, A., Tsafou, K. P., et al. (2015) · Nucleic Acids Research · doi:10.1093/nar/gku1003 · Cited by 10,436 · see citing works
The igraph software package for complex network research
Csárdi, G., & Nepusz, T. (2006) · Medical Entomology and Zoology · Cited by 10,078 · see citing works
Trapnell, C., Cacchiarelli, D., Grimsby, J., Pokharel, P., Li, S., Morse, M., Lennon, N. J., Livak, K. J., Mikkelsen, T. S., & Rinn, J. L. (2014) · Nature Biotechnology · doi:10.1038/nbt.2859 · Cited by 7,314 · see citing works
Scenic: single-cell regulatory network inference and clustering.
Aibar, S., González-Blas, C. B., Moerman, T., Huynh-Thu, V. A., Imrichova, H., Hulselmans, G., Rambow, F., Marine, J.-C., Geurts, P., Aerts, J., et al. (2017) · Nature Methods · doi:10.1038/nmeth.4463 · Cited by 6,520 · see citing works
Finak, G., McDavid, A., Yajima, M., Deng, J., Gersuk, V., Shalek, A. K., Slichter, C. K., Miller, H. W., McElrath, M. J., Prlic, M., et al. (2015) · Genome Biology · doi:10.1186/s13059-015-0844-5 · Cited by 3,159 · see citing works
More powerful procedures for multiple significance testing
Hochberg, Y., & Benjamini, Y. (1990) · Statistics in Medicine · doi:10.1002/sim.4780090710 · Cited by 2,715 · see citing works
Margolin, A. A., Nemenman, I., Basso, K., Wiggins, C., Stolovitzky, G., Favera, R. D., & Califano, A. (2006) · BMC Bioinformatics · doi:10.1186/1471-2105-7-s1-s7 · Cited by 2,420 · see citing works
Margolin, A. A., Nemenman, I. M., Basso, K., Wiggins, C. H., Stolovitzky, G. A., Favera, R. D., & Califano, A. (2006) · BMC Bioinformatics · doi:10.7916/d8g15zbp · Cited by 2,260 · see citing works
Inferring Regulatory Networks from Expression Data Using Tree-Based Methods
Huynh-Thu, V. A., Irrthum, A., Wehenkel, L., & Geurts, P. (2010) · PLoS ONE · doi:10.1371/journal.pone.0012776 · Cited by 2,131 · see citing works
Learning Bayesian Networks with the bnlearn R Package
Scutari, M. (2010) · Journal of Statistical Software · doi:10.18637/jss.v035.i03 · Cited by 1,978 · see citing works
Single-cell mRNA quantification and differential analysis with Census
Qiu, X., Hill, A. J., Packer, J. S., Lin, D., Ma, Y.-A., & Trapnell, C. (2017) · Nature Methods · doi:10.1038/nmeth.4150 · Cited by 1,885 · see citing works
Wisdom of crowds for robust gene network inference
The DREAM5 Consortium, Marbach, D., Costello, J. C., Küffner, R., Vega, N. M., Prill, R. J., Camacho, D. M., Allison, K. R., Kellis, M., Collins, J. J., et al. (2012) · Nature Methods · doi:10.1038/nmeth.2016 · Cited by 1,667 · see citing works
Faith, J. J., Hayete, B., Thaden, J. T., Mogno, I., Wierzbowski, J., Cottarel, G., Kasif, S., Collins, J. J., & Gardner, T. S. (2007) · PLoS Biology · doi:10.1371/journal.pbio.0050008 · Cited by 1,563 · see citing works
Estimation of entropy and mutual information
Paninski, L. (2003) · Neural Computation · doi:10.1162/089976603321780272 · Cited by 1,442 · see citing works
Bayesian approach to single-cell differential expression analysis
Kharchenko, P. V., Silberstein, L., & Scadden, D. T. (2014) · Nature Methods · doi:10.1038/nmeth.2967 · Cited by 1,348 · see citing works
Nonnegative Decomposition of Multivariate InformationPreprint
Williams, P. L., & Beer, R. (2010) · arXiv.org · doi:10.48550/arxiv.1004.2515 · Cited by 814 · see citing works
Buganim, Y., Faddah, D. A., Cheng, A. W., Itskovich, E., Markoulaki, S., Ganz, K., Klemm, S. L., van Oudenaarden, A., & Jaenisch, R. (2012) · Cell · doi:10.1016/j.cell.2012.08.023 · Cited by 800 · see citing works
Gene Regulatory Network Inference from Single-Cell Data Using Multivariate Information Measures
Chan, T. E., Stumpf, M. P. H., & Babtie, A. C. (2017) · Cell Systems · doi:10.1016/j.cels.2017.08.014 · Cited by 578 · see citing works
ZIFA: Dimensionality reduction for zero-inflated single-cell gene expression analysis
Pierson, E., & Yau, C. (2015) · Genome Biology · doi:10.1186/s13059-015-0805-z · Cited by 577 · see citing works
GeneNetWeaver: in silico benchmark generation and performance profiling of network inference methods
Schaffter, T., Marbach, D., & Floreano, D. (2011) · Bioinformatics · doi:10.1093/bioinformatics/btr373 · Cited by 538 · see citing works
CIDR: Ultrafast and accurate clustering through imputation for single-cell RNA-seq data
Lin, P., Troup, M., & Ho, J. W. K. (2017) · Genome biology · doi:10.1186/s13059-017-1188-0 · Cited by 529 · see citing works
Matsumoto, H., Kiryu, H., Furusawa, C., Ko, M. S. H., Ko, S. B. H., Gouda, N., Hayashi, T., & Nikaido, I. (2017) · Bioinformatics · doi:10.1093/bioinformatics/btx194 · Cited by 439 · see citing works
Moignard, V., Woodhouse, S., Haghverdi, L., Lilly, A. J., Tanaka, Y., Wilkinson, A. C., Buettner, F., Macaulay, I. C., Jawaid, W., Diamanti, E., et al. (2015) · Nature Biotechnology · doi:10.1038/nbt.3154 · Cited by 380 · see citing works
Gama‐Castro, S., Salgado, H., Peralta-Gil, M., Santos-Zavaleta, A., Muñiz-Rascado, L., Solano-Lira, H., Jacinto, V. J., Weiss, V., García-Sotelo, J. S., López-Fuentes, A., et al. (2010) · Nucleic Acids Research · doi:10.1093/nar/gkq1110 · Cited by 353 · see citing works
SINCERA: A Pipeline for Single-Cell RNA-Seq Profiling Analysis
Guo, M., Wang, H., Potter, S. S., Whitsett, J. A., & Xu, Y. J. (2015) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1004575 · Cited by 347 · see citing works
Cell fate inclination within 2-cell and 4-cell mouse embryos revealed by single-cell RNA sequencingPreprint
Biase, F. H., Cao, X., & Zhong, S. (2014) · Genome Research · doi:10.1101/gr.177725.114 · Cited by 309 · see citing works
BASiCS: Bayesian Analysis of Single-Cell Sequencing Data
Vallejos, C. A., Marioni, J. C., & Richardson, S. (2015) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1004333 · Cited by 302 · see citing works
A statistical approach for identifying differential distributions in single-cell RNA-seq experiments
Korthauer, K. D., Chu, L.-F., Newton, M. A., Li, Y., Thomson, J., Stewart, R., & Kendziorski, C. (2016) · Genome Biology · doi:10.1186/s13059-016-1077-y · Cited by 273 · see citing works
Greenfield, A., Madar, A., Ostrer, H., & Bonneau, R. (2010) · PLoS ONE · doi:10.1371/journal.pone.0013397 · Cited by 255 · see citing works
Mapping gene regulatory networks from single-cell omics data
Fiers, M. W. E. J., Minnoye, L., Aibar, S., Bravo González-Blas, C., Kalender Atak, Z., & Aerts, S. (2018) · Briefings in Functional Genomics · doi:10.1093/bfgp/elx046 · Cited by 235 · see citing works
Specht, A. T., & Li, J. (2017) · Bioinformatics · doi:10.1093/bioinformatics/btw729 · Cited by 193 · see citing works
Oscope identifies oscillatory genes in unsynchronized single-cell RNA-seq experiments
Leng, N., Chu, L., Barry, C., Li, Y., Choi, J., Li, X., Jiang, P., Stewart, R. M., Thomson, J. A., & Kendziorski, C. M. (2015) · Nature Methods · doi:10.1038/nmeth.3549 · Cited by 171 · see citing works
Single-cell RNA-seq reveals distinct injury responses in different types of DRG sensory neurons
Hu, G., Huang, K. Y., Hu, Y., Du, G., Xue, Z., Zhu, X., & Fan, G. (2016) · Scientific Reports · doi:10.1038/srep31851 · Cited by 159 · see citing works
Beta-Poisson model for single-cell RNA-seq data analyses
Vu, T. N., Wills, Q. F., Kalari, K. R., Niu, N., Wang, L., Rantalainen, M., & Pawitan, Y. (2016) · Bioinformatics · doi:10.1093/bioinformatics/btw202 · Cited by 157 · see citing works
Delmans, M., & Hemberg, M. (2016) · BMC Bioinformatics · doi:10.1186/s12859-016-0944-6 · Cited by 111 · see citing works
Delmans, M., & Hemberg, M. (2015) · doi:10.1101/020735 · Cited by 111 · see citing works
Single-Cell RNA-Sequencing: Assessment of Differential Expression Analysis Methods
Dal Molin, A., Baruzzo, G., & Di Camillo, B. (2017) · Frontiers in Genetics · doi:10.3389/fgene.2017.00062 · Cited by 107 · see citing works
Comparison of methods to detect differentially expressed genes between single-cell populations
Jaakkola, M. K., Seyednasrollah, F., Mehmood, A., & Elo, L. L. (2016) · Briefings in Bioinformatics · doi:10.1093/bib/bbw057 · Cited by 86 · see citing works
BTR: training asynchronous Boolean models using single-cell expression data
Lim, C. Y., Wang, H., Woodhouse, S., Piterman, N., Wernisch, L., Fisher, J., & Göttgens, B. (2016) · BMC Bioinformatics · doi:10.1186/s12859-016-1235-y · Cited by 81 · see citing works
Single-cell network analysis identifies DDIT3 as a nodal lineage regulator in hematopoiesis
Pina, C., Teles, J., Fugazza, C., May, G., Wang, D., Guo, Y., Soneji, S., Brown, J., Edén, P., Ohlsson, M., et al. (2015) · Cell Reports · doi:10.1016/j.celrep.2015.05.016 · Cited by 72 · see citing works
Campbell, K. R., & Yau, C. (2016) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1005212 · Cited by 66 · see citing works
Bellot, P., Olsen, C., Salembier, P., Oliveras, A., & Meyer, P. E. (2015) · BMC Bioinformatics · doi:10.1186/s12859-015-0728-4 · Cited by 50 · see citing works
Papatsenko, D., Darr, H., Kulakovskiy, I. V., Waghray, A., Makeev, V. J., MacArthur, B. D., & Lemischka, I. R. (2015) · Stem Cell Reports · doi:10.1016/j.stemcr.2015.07.004 · Cited by 43 · see citing works
Modeling the ascorbate-glutathione cycle in chloroplasts under light/dark conditions
Valero, E., Macià, H., la Fuente, I. M. D., Hernández, J. A., González-Sánchez, M.-I., & García‐Carmona, F. (2015) · BMC Systems Biology · doi:10.1186/s12918-015-0239-y · Cited by 40 · see citing works
Bayesian Inference for Single-cell Clustering and Imputing
Azizi, E., Prabhakaran, S., Carr, A., & Pe’er, D. (2017) · Genomics and Computational Biology · doi:10.18547/gcb.2017.vol3.iss1.e46 · Cited by 39 · see citing works
Guo, J., & Zheng, J. (2017) · Bioinformatics · doi:10.1093/bioinformatics/btx232 · Cited by 37 · see citing works
Integrated single cell data analysis reveals cell specific networks and novel coactivation markers
Ghazanfar, S., Bisogni, A. J., Ormerod, J. T., Lin, D. M., & Yang, J. Y. H. (2016) · BMC Systems Biology · doi:10.1186/s12918-016-0370-4 · Cited by 25 · see citing works
Processing, visualising and reconstructing network models from single‐cell data
Woodhouse, S., Moignard, V., Göttgens, B., & Fisher, J. (2015) · Immunology and Cell Biology · doi:10.1038/icb.2015.102 · Cited by 18 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Current best practices in single-cell RNA-seq analysis: a tutorial
Luecken, M. D., & Theis, F. J. (2019) · Molecular Systems Biology · doi:10.15252/msb.20188746 · Cited by 2,231 · see citing works
Benchmarking algorithms for gene regulatory network inference from single-cell transcriptomic data
Pratapa, A., Jalihal, A. P., Law, J. N., Bharadwaj, A., & Murali, T. M. (2020) · Nature Methods · doi:10.1038/s41592-019-0690-6 · Cited by 792 · see citing works
Single-cell multiomics: technologies and data analysis methods
Lee, J., Hyeon, D. Y., & Hwang, D. (2020) · Experimental & Molecular Medicine · doi:10.1038/s12276-020-0420-2 · Cited by 524 · see citing works
Concepts and limitations for learning developmental trajectories from single cell genomics
Tritschler, S., Büttner, M., Fischer, D. S., Lange, M., Bergen, V., Lickert, H., & Theis, F. J. (2019) · Development · doi:10.1242/dev.170506 · Cited by 237 · see citing works
The single-cell eQTLGen consortium
van der Wijst, M. G. P., de Vries, D., Groot, H. E., Trynka, G., Hon, C.-C., Bonder, M. J., Stegle, O., Nawijn, M. C., Idaghdour, Y., van der Harst, P., et al. (2020) · eLife · doi:10.7554/elife.52155 · Cited by 216 · see citing works
Single-cell transcriptomics unveils gene regulatory network plasticity
Iacono, G., Massoni-Badosa, R., & Heyn, H. (2019) · Genome Biology · doi:10.1186/s13059-019-1713-4 · Cited by 218 · see citing works
Computational methods for Gene Regulatory Networks reconstruction and analysis: A review
Delgado-Chaves, F. M., & Gómez-Vela, F. (2018) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2018.10.006 · Cited by 209 · see citing works
Jackson, C. A., Castro, D. M., Saldi, G.-A., Bonneau, R., & Gresham, D. (2020) · eLife · doi:10.7554/elife.51254 · Cited by 174 · see citing works
Genetic mapping of etiologic brain cell types for obesity
Timshel, P. N., Thompson, J. J., & Pers, T. H. (2020) · eLife · doi:10.7554/elife.55851 · Cited by 195 · see citing works
A guide to multi-omics data collection and integration for translational medicine
Athieniti, E., & Spyrou, G. M. (2022) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2022.11.050 · Cited by 185 · see citing works
A comprehensive survey of regulatory network inference methods using single cell RNA sequencing data
Nguyen, H., Tran, D., Tran, B., Pehlivan, B. A., & Nguyen, T. (2020) · Briefings in Bioinformatics · doi:10.1093/bib/bbaa190 · Cited by 181 · see citing works
Understanding cell‐cell communication and signaling in the colorectal cancer microenvironment
AlMusawi, S., Ahmed, M., & Nateri, A. S. (2021) · Clinical and Translational Medicine · doi:10.1002/ctm2.308 · Cited by 150 · see citing works
Evaluating measures of association for single-cell transcriptomics
Skinnider, M. A., Squair, J. W., & Foster, L. J. (2019) · Nature Methods · doi:10.1038/s41592-019-0372-4 · Cited by 145 · see citing works
Single-cell network biology for resolving cellular heterogeneity in human diseases
Cha, J., & Lee, I. (2020) · Experimental & Molecular Medicine · doi:10.1038/s12276-020-00528-0 · Cited by 139 · see citing works
Network inference with Granger causality ensembles on single-cell transcriptomics
Deshpande, A., Chu, L.-F., Stewart, R., & Gitter, A. (2022) · Cell Reports · doi:10.1016/j.celrep.2022.110333 · Cited by 130 · see citing works
Correlation AnalyzeR: functional predictions from gene co-expression correlations
Miller, H. E., & Bishop, A. J. R. (2021) · BMC Bioinformatics · doi:10.1186/s12859-021-04130-7 · Cited by 116 · see citing works
Statistical mechanics meets single-cell biology
Teschendorff, A. E., & Feinberg, A. P. (2021) · Nature Reviews Genetics · doi:10.1038/s41576-021-00341-z · Cited by 117 · see citing works
Inference of Gene Regulatory Network from Single-Cell Transcriptomic Data Using pySCENIC
Kumar, N., Mishra, B., Athar, M., & Mukhtar, S. M. (2021) · Methods in molecular biology · doi:10.1007/978-1-0716-1534-8_10 · Cited by 114 · see citing works
An integrative approach for building personalized gene regulatory networks for precision medicine
van der Wijst, M. G. P., de Vries, D. H., Brugge, H., Westra, H.-J., & Franke, L. (2018) · Genome Medicine · doi:10.1186/s13073-018-0608-4 · Cited by 106 · see citing works
SERGIO: A Single-Cell Expression Simulator Guided by Gene Regulatory Networks
Dibaeinia, P., & Sinha, S. (2020) · Cell Systems · doi:10.1016/j.cels.2020.08.003 · Cited by 104 · see citing works
Gene regulatory network reconstruction: harnessing the power of single-cell multi-omic data
Kim, D. J., Tran, A., Kim, H. J., Lin, Y., Yang, J. Y. H., & Yang, P. (2023) · npj Systems Biology and Applications · doi:10.1038/s41540-023-00312-6 · Cited by 110 · see citing works
Wang, M., Song, W., Chen, M., Wang, Q., Zhou, X., Xu, P., Krek, A., Yoon, Y., Ho, L., Orr, M. E., et al. (2022) · Molecular Neurodegeneration · doi:10.1186/s13024-022-00517-z · Cited by 100 · see citing works
Network Medicine in Pathobiology
Lee, L. Y.-H., & Loscalzo, J. (2019) · The American Journal of Pathology · doi:10.1016/j.ajpath.2019.03.009 · Cited by 95 · see citing works
Data analysis guidelines for single-cell RNA-seq in biomedical studies and clinical applications
Su, M., Pan, T., Chen, Q.-Z., Zhou, W., Gong, Y., Xu, G., Yan, H.-Y., Li, S., Shi, Q.-Z., Zhang, Y., et al. (2022) · Military Medical Research · doi:10.1186/s40779-022-00434-8 · Cited by 103 · see citing works
Xu, Q., Γεωργίου, Γ., Frölich, S., van der Sande, M., Veenstra, G. J. C., Zhou, H., & van Heeringen, S. J. (2021) · Nucleic Acids Research · doi:10.1093/nar/gkab598 · Cited by 88 · see citing works
Zhao, M., He, W., Tang, J., Zou, Q., & Guo, F. (2021) · Briefings in Bioinformatics · doi:10.1093/bib/bbab568 · Cited by 97 · see citing works
Computational Methods for Single-Cell RNA Sequencing
Hie, B., Peters, J. M., Nyquist, S. K., Shalek, A. K., Berger, B., & Bryson, B. D. (2020) · Annual Review of Biomedical Data Science · doi:10.1146/annurev-biodatasci-012220-100601 · Cited by 87 · see citing works
Bravo González‐Blas, C., Quan, X., Duran‐Romaña, R., Taskiran, I. I., Koldere, D., Davie, K., Christiaens, V., Makhzami, S., Hulselmans, G., de Waegeneer, M., et al. (2020) · Molecular Systems Biology · doi:10.15252/msb.20209438 · Cited by 88 · see citing works
Chen, J., Cheong, C., Lan, L., Zhou, X., Liu, J., Lü, A., Cheung, W. K., & Zhang, L. (2021) · Briefings in Bioinformatics · doi:10.1093/bib/bbab325 · Cited by 88 · see citing works
Investigating higher-order interactions in single-cell data with scHOT
Ghazanfar, S., Lin, Y., Su, X., Lin, D. M., Patrick, E., Han, Z., Marioni, J. C., & Yang, J. Y. H. (2020) · Nature Methods · doi:10.1038/s41592-020-0885-x · Cited by 73 · see citing works
CellOracle: Dissecting cell identity via network inference and in silico gene perturbation
Kamimoto, K., Hoffmann, C. M., & Morris, S. A. (2020) · doi:10.1101/2020.02.17.947416 · Cited by 81 · see citing works
Network modeling of single-cell omics data: challenges, opportunities, and progresses
Blencowe, M., Arneson, D., Ding, J., Chen, Y.-W., Saleem, Z., & Yang, X. (2019) · Emerging Topics in Life Sciences · doi:10.1042/etls20180176 · Cited by 73 · see citing works
High-performance single-cell gene regulatory network inference at scale: the Inferelator 3.0
Gibbs, C. S., Jackson, C. A., Saldi, G.-A., Tjärnberg, A., Shah, A., Watters, A., Veaux, N. D., Tchourine, K., Yi, R., Hamamsy, T., et al. (2022) · Bioinformatics · doi:10.1093/bioinformatics/btac117 · Cited by 71 · see citing works
Androvic, P., Schifferer, M., Perez Anderson, K., Cantuti-Castelvetri, L., Jiang, H., Ji, H., Liu, L., Gouna, G., Berghoff, S. A., Besson-Girard, S., et al. (2023) · Nature Communications · doi:10.1038/s41467-023-39447-9 · Cited by 67 · see citing works
From time-series transcriptomics to gene regulatory networks: A review on inference methods
Marku, M., & Pancaldi, V. (2023) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1011254 · Cited by 67 · see citing works
Osorio, D., Zhong, Y., Li, G., Huang, J. Z., & Cai, J. J. (2020) · Patterns · doi:10.1016/j.patter.2020.100139 · Cited by 65 · see citing works
Gene regulatory network inference in single-cell biology
Akers, K., & Murali, T. M. (2021) · Current Opinion in Systems Biology · doi:10.1016/j.coisb.2021.04.007 · Cited by 63 · see citing works
GRaNIE and GRaNPA: inference and evaluation of enhancer‐mediated gene regulatory networks
Kamal, A., Arnold, C., Claringbould, A., Moussa, R., Servaas, N. H., Kholmatov, M. M., Daga, N., Nogina, D., Müller‐Dott, S., Reyes‐Palomares, A., et al. (2023) · Molecular Systems Biology · doi:10.15252/msb.202311627 · Cited by 63 · see citing works
Schäfer, P. S. L., Dimitrov, D., Villablanca, E. J., & Sáez-Rodríguez, J. (2024) · Nature Immunology · doi:10.1038/s41590-024-01768-2 · Cited by 66 · see citing works
WASABI: a dynamic iterative framework for gene regulatory network inference
Bonnaffoux, A., Herbach, U., Richard, A., Guillemin, A., Gonin-Giraud, S., Gros, P.-A., & Gandrillon, O. (2019) · BMC Bioinformatics · doi:10.1186/s12859-019-2798-1 · Cited by 59 · see citing works
Graph representation learning for single-cell biology
Hetzel, L., Fischer, D. S., Günnemann, S., & Theis, F. J. (2021) · Current Opinion in Systems Biology · doi:10.1016/j.coisb.2021.05.008 · Cited by 55 · see citing works
Information Theory in Computational Biology: Where We Stand Today
Chanda, P., Costa, E., Hu, J. K., Sukumar, S., Hemert, J. V., & Walia, R. R. (2020) · Entropy · doi:10.3390/e22060627 · Cited by 55 · see citing works
Integration of single-cell multi-omics for gene regulatory network inference
Hu, X., Hu, Y., Wu, F., Leung, R. W. T., & Qin, J. (2020) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2020.06.033 · Cited by 53 · see citing works
Mar, J. C. (2019) · Biophysical Reviews · doi:10.1007/s12551-018-0494-4 · Cited by 53 · see citing works
Self-citationRecovering dynamic networks in big static datasets
Wu, R., & Jiang, L. (2021) · Physics Reports · doi:10.1016/j.physrep.2021.01.003 · Cited by 53 · see citing works
Network Approaches for Dissecting the Immune System
Shi, H., Yan, K., Ding, L., Qian, C., Chi, H., & Yu, J. (2020) · iScience · doi:10.1016/j.isci.2020.101354 · Cited by 51 · see citing works
McCalla, S. G., Siahpirani, A. F., Li, J., Pyne, S., Stone, M., Periyasamy, V., Shin, J., & Roy, S. (2023) · G3 Genes Genomes Genetics · doi:10.1093/g3journal/jkad004 · Cited by 50 · see citing works
Co-expression in Single-Cell Analysis: Saving Grace or Original Sin?
Crow, M., & Gillis, J. (2018) · Trends in Genetics · doi:10.1016/j.tig.2018.07.007 · Cited by 46 · see citing works
Das, S., Rai, A. K., & Rai, S. N. (2022) · Entropy · doi:10.3390/e24070995 · Cited by 45 · see citing works
Evaluating the Reproducibility of Single-Cell Gene Regulatory Network Inference Algorithms
Kang, Y., Thieffry, D., & Cantini, L. (2021) · Frontiers in Genetics · doi:10.3389/fgene.2021.617282 · Cited by 41 · see citing works
Zeng, L., Yang, K., Zhang, T., Zhu, X., Hao, W., Chen, H., & Ge, J. (2022) · Journal of Autoimmunity · doi:10.1016/j.jaut.2022.102919 · Cited by 45 · see citing works
scLink: Inferring Sparse Gene Co-Expression Networks from Single-Cell Expression Data
Li, W. V., & Li, Y. (2021) · Genomics Proteomics & Bioinformatics · doi:10.1016/j.gpb.2020.11.006 · Cited by 45 · see citing works
Constructing local cell-specific networks from single-cell data
Wang, X., Choi, D. S., & Roeder, K. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2113178118 · Cited by 43 · see citing works
Neumann, A., Jolicoeur‐Martineau, A., Székely, E., Sallis, H., O’Donnel, K., Greenwood, C. M. T., Levitan, R. D., Meaney, M. J., Wazana, A. D., Evans, J., et al. (2021) · Journal of Child Psychology and Psychiatry · doi:10.1111/jcpp.13501 · Cited by 47 · see citing works
The use of machine learning to discover regulatory networks controlling biological systems
Erbe, R., Gore, J., Gemmill, K., Gaykalova, D. A., & Fertig, E. J. (2022) · Molecular Cell · doi:10.1016/j.molcel.2021.12.011 · Cited by 40 · see citing works
Li, L., Sun, L., Chen, G., Wong, C.-W., Ching, W., & Liu, Z.-P. (2023) · Bioinformatics · doi:10.1093/bioinformatics/btad256 · Cited by 40 · see citing works
Zinati, Y., Takiddeen, A., & Emad, A. (2024) · Nature Communications · doi:10.1038/s41467-024-48516-6 · Cited by 36 · see citing works
Computational Stem Cell Biology: Open Questions and Guiding Principles
Cahan, P., Cacchiarelli, D., Dunn, S.-J., Hemberg, M., Lopes, S. M. C. D. S., Morris, S. A., Rackham, O. J. L., del Sol, A., & Wells, C. A. (2021) · Cell stem cell · doi:10.1016/j.stem.2020.12.012 · Cited by 36 · see citing works
Transcriptome Landscape of Cancer‐Associated Fibroblasts in Human PDAC
Tao, M., Liu, W., Chen, J., Liu, R., Zou, J., Yu, B., Wang, C., Huang, M., Chen, Q., Zhang, Z., et al. (2025) · Advanced Science · doi:10.1002/advs.202415196 · Cited by 35 · see citing works
Network Modeling Approaches and Applications to Unravelling Non-Alcoholic Fatty Liver Disease
Blencowe, M., Karunanayake, T., Wier, J. R., Hsu, N., & Yang, X. (2019) · Genes · doi:10.3390/genes10120966 · Cited by 35 · see citing works
Improved detection of tumor suppressor events in single-cell RNA-Seq data
Teschendorff, A. E., & Wang, N. (2020) · npj Genomic Medicine · doi:10.1038/s41525-020-00151-y · Cited by 33 · see citing works
Computational biology approaches for mapping transcriptional regulatory networks
Saint‐André, V. (2021) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2021.08.028 · Cited by 32 · see citing works
Kuciński, I., Wilson, N. K., Hannah, R., Kinston, S., Cauchy, P., Lenaerts, A., Grosschedl, R., & Göttgens, B. (2020) · The EMBO Journal · doi:10.15252/embj.2020104983 · Cited by 29 · see citing works
Maizels, R. J., Snell, D. M., & Briscoe, J. (2024) · Cell Systems · doi:10.1016/j.cels.2024.04.004 · Cited by 35 · see citing works
Loers, J. U., & Vermeirssen, V. (2024) · Briefings in Bioinformatics · doi:10.1093/bib/bbae382 · Cited by 33 · see citing works
Franchini, M., Pellecchia, S., Viscido, G., & Gambardella, G. (2023) · NAR Genomics and Bioinformatics · doi:10.1093/nargab/lqad024 · Cited by 29 · see citing works
Inferring better gene regulation networks from single-cell data
Stumpf, M. P. H. (2021) · Current Opinion in Systems Biology · doi:10.1016/j.coisb.2021.05.003 · Cited by 27 · see citing works
RVAgene: generative modeling of gene expression time series data
Mitra, R., & MacLean, A. L. (2021) · Bioinformatics · doi:10.1093/bioinformatics/btab260 · Cited by 25 · see citing works
Computational tools for inferring transcription factor activity
Hecker, D., Lauber, M., Ardakani, F. B., Ashrafiyan, S., Manz, Q., Kersting, J., Hoffmann, M., Schulz, M. H., & List, M. (2023) · PROTEOMICS · doi:10.1002/pmic.202200462 · Cited by 25 · see citing works
Single-cell transcriptomics in tissue engineering and regenerative medicine
Ruta, A., Krishnan, K., & Elisseeff, J. H. (2023) · Nature Reviews Bioengineering · doi:10.1038/s44222-023-00132-7 · Cited by 25 · see citing works
Stem-cell states converge in multistage cutaneous squamous cell carcinoma development
Taylor, M. A., Kandyba, E. E., Halliwill, K. D., Delrosario, R., Khoroshkin, M., Goodarzi, H., Quigley, D. A., Li, Y. R., Wu, D., Bollam, S., et al. (2024) · Science · doi:10.1126/science.adi7453 · Cited by 28 · see citing works
Effect of imputation on gene network reconstruction from single-cell RNA-seq data
Ly, L.-H., & Vingron, M. (2021) · Patterns · doi:10.1016/j.patter.2021.100414 · Cited by 22 · see citing works
Liu, T., Zhao, X., Lin, Y., Luo, Q., Zhang, S., Xi, Y., Chen, Y., Lin, L., Fan, W., Yang, J., et al. (2022) · Cancer Research · doi:10.1158/0008-5472.can-22-0668 · Cited by 23 · see citing works
Multiscale modelling of drug mechanism and safety
Zhang, J. D., Sach-Peltason, L., Krämer, C., Wang, K., & Ebeling, M. (2019) · Drug Discovery Today · doi:10.1016/j.drudis.2019.12.009 · Cited by 21 · see citing works
Identification of dynamic mass-action biochemical reaction networks using sparse Bayesian methods
Jiang, R., Singh, P., Wrede, F., Hellander, A., & Petzold, L. R. (2022) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1009830 · Cited by 22 · see citing works
Cha, J., Yu, J., Cho, J.-W., Hemberg, M., & Lee, I. (2022) · Nucleic Acids Research · doi:10.1093/nar/gkac1042 · Cited by 19 · see citing works
Quantum gene regulatory networks
Roman-Vicharra, C., & Cai, J. J. (2023) · npj Quantum Information · doi:10.1038/s41534-023-00740-6 · Cited by 29 · see citing works
Gao, S., Sun, C., Xiang, C., Qin, K., & Lee, T. H. (2020) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2020.3022430 · Cited by 22 · see citing works
Hui, E., Stafford, R., Matthews, I., & Smith, V. A. (2021) · Ecological Informatics · doi:10.1016/j.ecoinf.2021.101539 · Cited by 22 · see citing works
Escorcia-Rodríguez, J. M., Gaytan-Nuñez, E., Hernandez-Benitez, E. M., Zorro-Aranda, A., Tello‐Palencia, M. A., & Freyre-González, J. A. (2023) · Frontiers in Genetics · doi:10.3389/fgene.2023.1143382 · Cited by 22 · see citing works
Seiðr: Efficient calculation of robust ensemble gene networks
Schiffthaler, B., van Zalen, E. M., Serrano, A. R., Street, N. R., & Delhomme, N. (2023) · Heliyon · doi:10.1016/j.heliyon.2023.e16811 · Cited by 21 · see citing works
Su, E. Y., Spangler, A., Bian, Q., Kasamoto, J. Y., & Cahan, P. (2022) · Stem Cell Reports · doi:10.1016/j.stemcr.2021.12.018 · Cited by 15 · see citing works
Predictive network analysis identifies JMJD6 and other potential key drivers in Alzheimer’s disease
Merchant, J. P., Zhu, K., Henrion, M., Zaidi, S. S. A., Lau, B., Moein, S., Alamprese, M. L., Pearse, R. V., Bennett, D. A., Ertekin‐Taner, N., et al. (2023) · Communications Biology · doi:10.1038/s42003-023-04791-5 · Cited by 22 · see citing works
MICA: a multi-omics method to predict gene regulatory networks in early human embryos
Alanis‐Lobato, G., Bartlett, T. E., Huang, Q., Simon, C. S., McCarthy, A., Elder, K. T., Snell, P., Christie, L., & Niakan, K. K. (2023) · Life Science Alliance · doi:10.26508/lsa.202302415 · Cited by 21 · see citing works
Exploring Additional Valuable Information From Single-Cell RNA-Seq Data
Li, Y., Xu, Q., Wu, D., & Chen, G. (2020) · Frontiers in Cell and Developmental Biology · doi:10.3389/fcell.2020.593007 · Cited by 19 · see citing works
Understanding the Adult Mammalian Heart at Single-Cell RNA-Seq Resolution
Marín‐Sedeño, E., de Morentin, X. M., Pérez-Pomares, J. M., Gómez-Cabrero, D., & Ruiz‐Villalba, A. (2021) · Frontiers in Cell and Developmental Biology · doi:10.3389/fcell.2021.645276 · Cited by 18 · see citing works
Reconstructing gene regulatory networks in single-cell transcriptomic data analysis
Dai, H., 200031，中国, 国., Jin, Q., Li, L., Chen, L., 201210，中国, 张., 100049，中国, 中., 201210，中国, 上., 310024，中国, 中. 杭., & 650223，中国, 中. 昆. (2020) · 动物学研究 · doi:10.24272/j.issn.2095-8137.2020.215 · Cited by 17 · see citing works
GRNUlar: A Deep Learning Framework for Recovering Single-Cell Gene Regulatory Networks
Shrivastava, H., Zhang, X., Song, L., & Aluru, S. (2022) · Journal of Computational Biology · doi:10.1089/cmb.2021.0437 · Cited by 19 · see citing works
acorde unravels functionally interpretable networks of isoform co-usage from single cell data
Arzalluz-Luque, Á., Salguero, P., Tarazona, S., & Conesa, A. (2022) · Nature Communications · doi:10.1038/s41467-022-29497-w · Cited by 14 · see citing works
Seiðr: Efficient Calculation of Robust Ensemble Gene NetworksPreprint
Schiffthaler, B., van Zalen, E., Serrano, A. R., Street, N. R., & Delhomme, N. (2018) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/250696 · Cited by 17 · see citing works
Kashima, M., Shida, Y., Yamashiro, T., Hirata, H., & Kurosaka, H. (2021) · Frontiers in Bioinformatics · doi:10.3389/fbinf.2021.777299 · Cited by 16 · see citing works
Mohammed, M., Dziedziech, A., Macedo, D. H., Huppertz, F., Veith, Y., Postel, Z., Christ, E., Scheytt, R., Slotte, T., Henriksson, J., et al. (2024) · Nature Communications · doi:10.1038/s41467-024-51201-3 · Cited by 17 · see citing works
Multi-Omics Profiling of the Tumor Microenvironment
Oekelen, O. V., & Laganà, A. (2022) · Advances in experimental medicine and biology · doi:10.1007/978-3-030-91836-1_16 · Cited by 13 · see citing works
scSGL: kernelized signed graph learning for single-cell gene regulatory network inference
Karaaslanli, A., Saha, S., Aviyente, S., & Maiti, T. (2022) · Bioinformatics · doi:10.1093/bioinformatics/btac288 · Cited by 17 · see citing works
Bi, X.-P., 深圳518083，中国, 深., BGI-Shenzhen, S., Zhang, G.-J., 650223，中国, 中. 昆., 650223，中国, 中. 昆., Villum Center for Biodiversity Genomics, S. F. E. A. E., Center for Excellence in Animal Evolution and Genetics, C. A. O. S., State Key Laboratory of Genetic Resources and Evolution, K. I. O. Z., & Villum Center for Biodiversity Genomics, S. F. E. A. E. (2021) · Zoological Research · doi:10.24272/j.issn.2095-8137.2021.066 · Cited by 15 · see citing works
Gene regulatory networks: from correlative models to causal explanations
Maizels, R. J., & Briscoe, J. (2026) · Nature Reviews Genetics · doi:10.1038/s41576-026-00939-1 · Cited by 18 · see citing works
Sekula, M. N., Gaskins, J., & Datta, S. (2020) · BMC Bioinformatics · doi:10.1186/s12859-020-03707-y · Cited by 15 · see citing works
Forecasting cellular states: from descriptive to predictive biology via single-cell multiomics
Stein-O’Brien, G., Ainslie, M. C., & Fertig, E. J. (2021) · Current Opinion in Systems Biology · doi:10.1016/j.coisb.2021.03.008 · Cited by 13 · see citing works
High performance single-cell gene regulatory network inference at scale: The Inferelator 3.0Preprint
Gibbs, C. S., Jackson, C. A., Saldi, G.-A., Tjärnberg, A., Shah, A., Watters, A., Veaux, N. D., Tchourine, K., Yi, R., Hamamsy, T., et al. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.05.03.442499 · Cited by 14 · see citing works
SimiC enables the inference of complex gene regulatory dynamics across cell phenotypes
Peng, J., Serrano, G., Traniello, I. M., Calleja-Cervantes, M. E., Chembazhi, U. V., Bangru, S., Ezponda, T., Rodríguez-Madoz, J. R., Kalsotra, A., Prósper, F., et al. (2022) · Communications Biology · doi:10.1038/s42003-022-03319-7 · Cited by 12 · see citing works
- Authors
- Chen, S., & Mar, J. C.
- Affiliations
- Albert Einstein College of Medicine (US)
- Corresponding author
- Mar, Jessica C.
- Venue
- BMC Bioinformatics
- Published
- 2018-06-19
- Volume
- 19
- Issue
- 1
- Pages
- 232-232
- ISSN
- 1471-2105 (ISSN-L)
- PMC
- PMC6006753
- OpenAlex
- W2808743742
- Cited by
- 235
- References
- 49
- Citations by year
- 2026: 14 · 2025: 19 · 2024: 24 · 2023: 36 · 2022: 35 · 2021: 44 · 2020: 38 · 2019: 19 · 2018: 6
- FWCI
- 9.52
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- BMC Bioinformatics (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, other-oa)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- NYSTEM Training Grant (C030292GG)
- NYSTEM (C029154)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene Regulatory Network Analysis
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Single-cell and spatial transcriptomics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Bioinformatics and Genomic Networks
- Keywords
- gene regulatory network inference, single-cell gene expression, gene-gene interaction
- MeSH terms
- Algorithms, Hematopoietic Stem Cells, Humans, Computational Biology, Gene Expression Profiling, Gene Regulatory Networks, Embryonic Stem Cells, Single-Cell Analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
