The DOI matches its Crossref record; 67 of the 67 references it cites were found in a registry or an index.
Benchmarking algorithms for gene regulatory network inference from single-cell transcriptomic data
Open accessPublished version of a preprint
Abstract
We present a systematic evaluation of state-of-the-art algorithms for inferring gene regulatory networks from single-cell transcriptional data. As the ground truth for assessing accuracy, we use synthetic networks with predictable trajectories, literature-curated Boolean models and diverse transcriptional regulatory networks. We develop a strategy to simulate single-cell transcriptional data from synthetic and Boolean networks that avoids pitfalls of previously used methods. Furthermore, we collect networks from multiple experimental single-cell RNA-seq datasets. We develop an evaluation framework called BEELINE. We find that the area under the precision-recall curve and early precision of the algorithms are moderate. The methods are better in recovering interactions in synthetic networks than Boolean models. The algorithms with the best early precision values for Boolean models also perform well on experimental datasets. Techniques that do not require pseudotime-ordered cells are generally more accurate. Based on these results, we present recommendations to end users. BEELINE will aid the development of gene regulatory network inference algorithms.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Despite extensive efforts, GRN inference accuracy has remained disappointingly low, marginally exceeding random predictions39.”
From Inferring gene regulatory networks from single-cell multiome data using atlas-scale external data (2025) · cites this paper as [39]
In the citing paper
“Experiments with targeted perturbations can be modeled as affecting individual nodes of a regulatory network model, enabling investigation of mechanistic processes and inference of regulatory interactions and their directionality11.”
From scPerturb: harmonized single-cell perturbation data (2024) · cites this paper as [11]
In the citing paper
“Cell-type-specific ground-truth GRNs were generated in the same manner as in a previous benchmarking study55.”
From Dissecting cell identity via network inference and in silico gene perturbation (2023) · cites this paper as [55]
In the citing paper
“GENIE3 emerged as one of the best-performing algorithms in a previous benchmarking study55.”
From Dissecting cell identity via network inference and in silico gene perturbation (2023) · cites this paper as [55]
In the citing paper
“GRN inference performance was evaluated by the AUROC and the early precision ratio (EPR), following the evaluation method used in a previous benchmarking study55.”
From Dissecting cell identity via network inference and in silico gene perturbation (2023) · cites this paper as [55]
In the citing paper
“Existing GRN tools utilize scRNA-seq data with thousands of pre-select genes and cells, because GRNs from full transcriptomes in large scRNA-seq datasets are often computationally intensive and intractable.13,15”
From SCING: Inference of robust, interpretable gene regulatory networks from single cell and spatial transcriptomics (2023) · cites this paper as [13]
In the citing paper
“In addition, benchmarking studies have shown limited accuracy of existing methods on both synthetic and real data.13”
From SCING: Inference of robust, interpretable gene regulatory networks from single cell and spatial transcriptomics (2023) · cites this paper as [13]
In the citing paper
“Although there exist other GRN inference approaches based on pseudotime analysis,23,24,25 their performance is generally inferior to those that rely solely on the scRNA-seq data.13”
From SCING: Inference of robust, interpretable gene regulatory networks from single cell and spatial transcriptomics (2023) · cites this paper as [13]
In the citing paper
“Validation of GRN inference tools has remained challenging.13”
From SCING: Inference of robust, interpretable gene regulatory networks from single cell and spatial transcriptomics (2023) · cites this paper as [13]
In the citing paper
“Since ground truth networks are unknown for many conditions across cell types and tissues, current GRN accuracy evaluations rely on inferring synthetic ground truth networks that may not capture biological or curated interactions that forsake tissue and cell-type specificity.13”
From SCING: Inference of robust, interpretable gene regulatory networks from single cell and spatial transcriptomics (2023) · cites this paper as [13]
In the citing paper
“To benchmark the performance of SCING, we selected three existing methods, namely GRNBOOST2, ppcor, and PIDCm with default parameters. These methods were selected based on previous benchmarking studies where superior performance of these methods were supported.13”
From SCING: Inference of robust, interpretable gene regulatory networks from single cell and spatial transcriptomics (2023) · cites this paper as [13]
In the citing paper
“Single cell multiomics has become a powerful tool for identifying regulatory interactions between genes, but the performance of existing tools is limited in both accuracy and scalability.13”
From SCING: Inference of robust, interpretable gene regulatory networks from single cell and spatial transcriptomics (2023) · cites this paper as [13]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Szklarczyk, D., Gable, A. L., Lyon, D., Junge, A., Wyder, S., Huerta-Cepas, J., Simonovic, M., Doncheva, N. T., Morris, J. H., Bork, P., et al. (2019) · Nucleic Acids Research · doi:10.1093/nar/gky1131 · Cited by 17,749
Comprehensive Integration of Single-Cell Data
Stuart, T., Butler, A., Hoffman, P., Hafemeister, C., Papalexi, E., Mauck, W. M., Hao, Y., Stoeckius, M., Smibert, P., & Satija, R. (2019) · Cell · doi:10.1016/j.cell.2019.05.031 · Cited by 15,453
Scenic: single-cell regulatory network inference and clustering.
Aibar, S., González-Blas, C. B., Moerman, T., Huynh-Thu, V. A., Imrichova, H., Hulselmans, G., Rambow, F., Marine, J.-C., Geurts, P., Aerts, J., et al. (2017) · Nature Methods · doi:10.1038/nmeth.4463 · Cited by 6,520
Saito, T., & Rehmsmeier, M. (2015) · PLOS ONE · doi:10.1371/journal.pone.0118432 · Cited by 5,309
La Manno, G., Soldatov, R., Zeisel, A., Braun, E., Hochgerner, H., Petukhov, V., Lidschreiber, K., Kastriti, M. E., Lönnerberg, P., Furlan, A., et al. (2018) · Nature · doi:10.1038/s41586-018-0414-6 · Cited by 4,118
Slingshot: cell lineage and pseudotime inference for single-cell transcriptomics
Street, K., Risso, D., Fletcher, R. B., Das, D., Ngai, J., Yosef, N., Purdom, E., & Dudoit, S. (2018) · BMC Genomics · doi:10.1186/s12864-018-4772-0 · Cited by 3,159
Single-cell RNA-seq reveals new types of human blood dendritic cells, monocytes, and progenitors
Villani, A.-C., Satija, R., Reynolds, G., Sarkizova, S., Shekhar, K., Fletcher, J., Griesbeck, M., Butler, A., Zheng, S., Lazo, S., et al. (2017) · Science · doi:10.1126/science.aah4573 · Cited by 2,413
Current best practices in single-cell RNA-seq analysis: a tutorial
Luecken, M. D., & Theis, F. J. (2019) · Molecular Systems Biology · doi:10.15252/msb.20188746 · Cited by 2,231
Inferring Regulatory Networks from Expression Data Using Tree-Based Methods
Huynh-Thu, V. A., Irrthum, A., Wehenkel, L., & Geurts, P. (2010) · PLoS ONE · doi:10.1371/journal.pone.0012776 · Cited by 2,131
TRRUST v2: an expanded reference database of human and mouse transcriptional regulatory interactions
Han, H., Cho, J.-W., Lee, S., Yun, A., Kim, H., Bae, D., Yang, S., Kim, C. Y., Lee, M., Kim, E., et al. (2017) · Nucleic Acids Research · doi:10.1093/nar/gkx1013 · Cited by 1,986
Cited by
Sorted by citations
Items 801–816 of 816
Zhu, Y., Han, D., & Wang, Z. (2026) · Briefings in Bioinformatics · doi:10.1093/bib/bbag513
Artificial intelligence, network biology and multiomics data in drug discovery
Singh, M., Hafeez, A., Yadav, S. K., Srivastava, U., Singh, P., & Gupta, M. K. (2026) · Elsevier eBooks · doi:10.1016/b978-0-443-40338-5.00016-0
Zhang, X., Yang, X., Zhang, Y., Fan, Y., & Zhu, Y. (2026) · Current Molecular Pharmacology · doi:10.1016/j.cmp.2026.09.009
Yang, S., Li, Y., Yang, Y., Fu, X., Jiang, J., Wang, C., Fu, F., Zheng, M., He, X., He, Y., et al. (2026) · Frontiers in Endocrinology · doi:10.3389/fendo.2026.1924512
Xie, Y., Fu, B., Zhong, L., Wu, L., Li, H., Chang, X., & Liu, X. (2026) · Briefings in Bioinformatics · doi:10.1093/bib/bbag525
Artificial Intelligence for Plant Gene Regulatory Networks
Yu, H. (2026) · International Journal of Molecular Sciences · doi:10.3390/ijms27198819
gsGNA: A Universal Pipeline for Mining Plant Stress-Resistance RegulatorsPreprint
Chen, Z., Zan, Z., Lu, Q., Wang, J., Zhu, J., Zhao, Q., Wang, P., Hu, Z., Fan, S., Xuemei, W., et al. (2026) · Research Square · doi:10.21203/rs.3.rs-11125843/v1
Yan, S., Yu, Q., Dai, W., & Cheng, X. (2026) · PLoS ONE · doi:10.1371/journal.pone.0359164
MFGRN: Multi-Algorithm Fusion for Gene Regulatory Network Inference in Sickle Cell DiseasePreprint
Zan, Z., Chen, Z., Lu, Q., Wang, J., Zhu, J., Zhao, Q., Wang, P., Hu, Z., Fan, S., Xuemei, W., et al. (2026) · Research Square · doi:10.21203/rs.3.rs-10880995/v1
Emma Paul, M., Jereesh, A. S., & Kumar, G. S. (2026) · Discover Computing · doi:10.1007/s10791-026-10713-w
Wang, H., & Oba, K. (2026) · Frontiers in Bioinformatics · doi:10.3389/fbinf.2026.1924703
Wang, H., Wang, B., Duan, X., Jiao, G., Chen, M., Guan, X., Cao, J., Cui, L., Li, H., & Li, Z. (2026) · Molecular Carcinogenesis · doi:10.1002/mc.70198
Shahid, S. (2025) · International journal of medical and health research. · doi:10.61424/ijmhr.v3i3.1107
Noordijk, B., van Loo, T., van Hoek, M., Reinders, M. J. T., van Dijk, A. D. J., García‐Gómez, M. L., ten Tusscher, K., & de Ridder, D. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.10.20.758482
Zhang, T., Teng, X., Zhang, H., Zhao, Z., Li, Y., Dong, B., & Wang, G. (2026) · Functional & Integrative Genomics · doi:10.1007/s10142-026-02087-5
Metadata
- Authors
- Pratapa, A., Jalihal, A. P., Law, J. N., Bharadwaj, A., & Murali, T. M.
- Affiliations
- Virginia Tech (US)
- Corresponding author
- Murali, T. M.
- Venue
- Nature Methods
- Published
- 2020-01-06
- Volume
- 17
- Issue
- 2
- Pages
- 147-154
- ISSN
- 1548-7091 (ISSN-L)
- 1548-7105
- PMC
- PMC7098173
- OpenAlex
- W2997831639
- Cited by
- 792
- References
- 63
- Citations by year
- 2026: 135 · 2025: 146 · 2024: 135 · 2023: 142 · 2022: 101 · 2021: 94 · 2020: 36
- FWCI
- 31.76
- Type
- journal article
- Language
- en
- Open access
- green
- doi.org
- Locations
- Nature Methods (publishedVersion)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Funders
- National Science Foundation (CCF-1617678, DBI-1759858)
- U.S. Department of Health & Human Services | NIH | National Cancer Institute (UH2CA203768)
- ODNI | Intelligence Advanced Research Projects Activity (W911NF-17-2-0105)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene Regulatory Network Analysis
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Single-cell and spatial transcriptomics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Microbial Metabolic Engineering and Bioproduction
- Keywords
- gene regulatory network inference, single-cell transcriptomics, single-cell RNA sequencing, algorithm benchmarking, synthetic gene regulatory networks, Boolean model, data simulation, pseudotime, precision and recall
- MeSH terms
- Algorithms, Sequence Analysis, RNA, Gene Regulatory Networks, Single-Cell Analysis, Transcriptome, Datasets as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
