CoChat
Sign Up
Cited by 5Gene Regulatory Network AnalysisSingle-cell and spatial transcriptomicsBioinformatics and Genomic Networks

The DOI matches its Crossref record; 7 of the 7 references it cites were found in a registry or an index.

Benchmarking imputation methods for network inference using a novel method of synthetic scRNA-seq data generation

Open access
Lasri, Ayoub, Shahrezaei, Vahid, Sturrock, Marc·2022·BMC Bioinformatics
DOI 10.1186/s12859-022-04778-9PMC PMC9204969

Published version of a preprint

Open this paper in CoChat — free accountOpen PDF

Abstract

Background: Single cell RNA-sequencing (scRNA-seq) has very rapidly become the new workhorse of modern biology providing an unprecedented global view on cellular diversity and heterogeneity. In particular, the structure of gene-gene expression correlation contains information on the underlying gene regulatory networks. However, interpretation of scRNA-seq data is challenging due to specific experimental error and biases that are unique to this kind of data including drop-out (or technical zeros). Methods: To deal with this problem several methods for imputation of zeros for scRNA-seq have been developed. However, it is not clear how these processing steps affect inference of genetic networks from single cell data. Here, we introduce Biomodelling.jl, a tool for generation of synthetic scRNA-seq data using multiscale modelling of stochastic gene regulatory networks in growing and dividing cells. Results: Our tool produces realistic transcription data with a known ground truth network topology that can be used to benchmark different approaches for gene regulatory network inference. Using this tool we investigate the impact of different imputation methods on the performance of several network inference algorithms. Conclusions: Biomodelling.jl provides a versatile and useful tool for future development and benchmarking of network inference approaches using scRNA-seq data.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1186/s12859-022-04778-9

Cites

7 of 7 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Flagged first, then by citations

Research works printed but not linked: 76

  1. Davidson E, Levin M. Gene regulatory networks. In:Proceedings of the National Academy of Sciences. 2005;102(14):4935–4935. Publisher: National Academy of Sciences Section: Introduction. · DOI
  2. Hwang B, Lee JH, Bang D. Single-cell RNA sequencing technologies and bioinformatics pipelines. Exp Mol Med. 2018;50(8):1–14. · DOI
  3. Lambrechts D, Wauters E, Boeckx B, Aibar S, Nittner D, Burton O, Bassez AD, Pircher A, Van den Eynde K, Weynand B, Verbeken E, Leyn Paul D, Liston A, Vansteenkiste J, Carmeliet P, Aerts S, Thienpont B. Phenotype molding of stromal cells in the lung tumor microenvironment. Nat Med. 2018;24(8):1277–89. · DOI · Paper page
  4. La Gioele M, Ruslan S, Amit Z, Emelie B, Hannah H, Viktor P, Katja L, Maria EK, Peter L, Alessandro F, Jean F, Lars EB, Zehua L, Van David B, Jimin G, Xiaoling H, Roger B, Erik S, Gonçalo C-B, Patrick C, Igor A, Sten L, Peter VK. RNA velocity of single cells. Nature. 2018;560(7719):494–8. · DOI · Paper page
  5. Saint M, Bertaux F, Tang W, Sun X-M, Game L, Köferle A, Bähler J, Shahrezaei V, Marguerat S. Single-cell imaging and RNA sequencing reveal patterns of gene expression heterogeneity during fission yeast growth and adaptation. Nat Microbiol. 2019;4(3):480–91. · DOI
  6. Sara A, Carmen BG-B, Thomas M, Vân Anh H-T, Hana I, Gert H, Florian R, Jean-Christophe M, Pierre G, Jan A, van den Joost O, Zeynep KA, Jasper W, Stein A. SCENIC: single-cell regulatory network inference and clustering. Nat Methods. 2017;14(11):1083–6. · DOI · Paper page
  7. Chan TE, Stumpf Michael PH, Babtie Ann C. Gene Regulatory Network Inference from Single-Cell Data Using Multivariate Information Measures. Cell Systems. 2017;5(3):251–67. · DOI · Paper page
  8. Nguyen H, Tran D, Tran B, Pehlivan B, Nguyen T. A comprehensive survey of regulatory network inference methods using single cell rna sequencing data. Brief Bioinf. 2021;22(3):bbaa190. · DOI
  9. Huang M, Wang J, Torre E, Dueck H, Shaffer S, Bonasio R, Murray John I, Raj A, Li M, Zhang Nancy R. SAVER: gene expression recovery for single-cell RNA sequencing. Nat Methods. 2018;15(7):539–42. · DOI
  10. Li Wei V, Li Jingyi J. An accurate and robust imputation method scImpute for single-cell RNA-seq data. Nat Commun. 2018;9(1):997. · DOI
  11. Gong W, Kwak I-Y, Pota P, Koyano-Nakagawa N, Garry Daniel J. DrImpute: imputing dropout events in single cell RNA sequencing data. BMC Bioinf. 2018;19(1):220. · DOI
  12. Chen Mengjie, Zhou Xiang. VIPER: variability-preserving imputation for accurate gene expression recovery in single-cell RNA sequencing studies. Genome Biol. 2018;19(1):196. · DOI
  13. van David D, Roshan S, Juozas N, Kristina Y, Pooja K, Ambrose JC, Cassandra B, Kevin RM, Christine LC, Diwakar P, Brian B, Linas M, Guy W, Smita K, Dana P. Recovering gene interactions from single-cell data using data diffusion. Cell. 2018;174(3):716–29. · DOI · Paper page
  14. Tang W, Bertaux F, Thomas P, Stefanelli C, Saint M, Marguerat S, Shahrezaei V. bayNorm: Bayesian gene expression recovery, imputation and normalization for single-cell RNA-sequencing data. Bioinformatics. 2020;36(4):1174–81. · DOI
  15. Zhang Lihua, Zhang Shihua. Comparison of computational methods for imputing single-cell RNA-sequencing data. IEEE/ACM Trans Comput Biol Bioinf. 2020;17(2):376–89. · DOI
  16. Hou Wenpin, Ji Zhicheng, Ji Hongkai, Hicks Stephanie C. A systematic evaluation of single-cell RNA-sequencing imputation methods. Genome Biol. 2020;21(1):218. · DOI
  17. Lopez R, Regier J, Cole MB, Jordan MI, Yosef N. Deep generative modeling for single-cell transcriptomics. Nat Methods. 2018;15(12):1053–8. · DOI · Paper page
  18. Eraslan G, Simon Lukas M, Mircea M, Mueller Nikola S, Theis Fabian J. Single-cell RNA-seq denoising using a deep count autoencoder. Nat Commun. 2019;10(1):390. · DOI · Paper page
  19. Santos-Zavaleta Alberto, Gama-Castro Socorro, Pérez-Rueda Ernesto. A comparative genome analysis of the rpos sigmulon shows a high diversity of responses and origins. Microbiology. 2011;157(5):1393–401. · DOI
  20. Ly L-H, Vingron M. Effect of imputation on gene network reconstruction from single-cell rna-seq data. bioRxiv. 2021. · DOI
  21. Camacho Diogo M, Collins James J. Systems biology strikes gold. Cell. 2009;137(1):24–6. · DOI
  22. Cantone I, Marucci L, Iorio F, Ricci Maria A, Belcastro V, Bansal M, Santini S, Bernardo MD, Bernardo Diego D, Cosma Maria P. A yeast synthetic network for in vivo assessment of reverse-engineering and modeling approaches. Cell. 2009;137(1):172–81. · DOI
  23. Mendes P, Sha W, Ye K. Artificial gene networks for objective comparison of analysis algorithms. Bioinformatics. 2003;19(suppl 2):ii122–9. · DOI
  24. Van den Bulcke Tim, Van Leemput Koenraad, Naudts Bart, van Remortel Piet, Ma Hongwu, Verschoren Alain, De Moor Bart, Marchal Kathleen. Syntren: a generator of synthetic gene expression data for design and analysis of structure learning algorithms. BMC Bioinf. 2006;7(1):1–12. · DOI
  25. Li Yong, Zhu Yanming, Bai Xi, Cai Hua, Ji Wei, Guo Dianjing. Retrn: A retriever of real transcriptional regulatory network and expression data for evaluating structure learning algorithm. Genomics. 2009;94(5):349–54. · DOI
  26. Hache Hendrik, Lehrach Hans, Herwig Ralf. Reverse engineering of gene regulatory networks: a comparative study. EURASIP J Bioinf Syst Biol. 2009;1–12:2009.
  27. Payne Samuel H. The utility of protein and mrna correlation. Trends Biochem Sci. 2015;40(1):1–3. · DOI
  28. Roy S, Werner-Washburne M, Lane T. A system for generating transcription regulatory networks with combinatorial control of transcription. Bioinformatics. 2008;24(10):1318–20. · DOI
  29. Hache H, Wierling C, Lehrach H, Herwig R. Genge: systematic generation of gene regulatory networks. Bioinformatics. 2009;25(9):1205–7. · DOI
  30. Haynes Brian C, Brent Michael R. Benchmarking regulatory network reconstruction with grendel. Bioinformatics. 2009;25(6):801–7. · DOI
  31. Schaffter T, Marbach D, Floreano D. Genenetweaver: in silico benchmark generation and performance profiling of network inference methods. Bioinformatics. 2011;27(16):2263–70. · DOI
  32. Bellot P, Olsen C, Salembier P, Oliveras-Vergés A, Meyer Patrick E. Netbenchmark: a bioconductor package for reproducible benchmarks of gene regulatory network inference. BMC Bioinf. 2015;16(1):1–15. · DOI
  33. Marbach D, Costello James C, Küffner R, Vega Nicole M, Prill Robert J, Camacho Diogo M, Allison Kyle R, Kellis M, Collins James J, Stolovitzky G. Wisdom of crowds for robust gene network inference. Nature Methods. 2012;9(8):796–804. · DOI · Paper page
  34. Greenfield A, Madar A, Ostrer H, Bonneau R. Dream4: combining genetic and dynamic information to identify biological networks and dynamical models. PloS one. 2010;5(10): e13397. · DOI · Paper page
  35. Milkevych V, Karaman E, Sahana G, Janss L, Cai Z, Lund Mogens S. Mescot: the tool for quantitative trait simulation through the mechanistic modelling of genes’ regulatory interactions. G3 Genes Genomes Genetics. 2021;11:133. · DOI
  36. Bezanson J, Edelman A, Karpinski S, Shah Viral B. Julia: a fresh approach to numerical computing. SIAM Rev. 2017;59(1):65–98. · DOI · Paper page
  37. Sun X-M, Bowman A, Priestman M, Bertaux F, Martinez-Segura A, Tang W, Whilding C, Dormann D, Shahrezaei V, Marguerat S. Size-dependent increase in RNA polymerase II initiation rates mediates gene expression scaling with cell size. Curr Biol. 2020;30(7):1217–30. · DOI
  38. Broido Anna D, Clauset A. Scale-free networks are rare. Nature Commun. 2019;10(1):1017. · DOI · Paper page
  39. Khanin R, Ernst W. How scale-free are biological networks. J Comput Biol. 2006;13(3):810–8. · DOI
  40. Zachary OW, Pogacar K, Grotewold E. Topological and statistical analyses of gene regulatory networks reveal unifying yet quantitatively different emergent properties. PLoS Comput Biol. 2018;14:4.
  41. Espinosa-Soto C. On the role of sparseness in the evolution of modularity in gene regulatory networks. PLOS Comput Biol. 2018;14(5): e1006172. · DOI
  42. James GM, Sabatti C, Zhou N, Zhu J. Sparse regulatory networks. Ann. Appl Stat. 2010;4(2):663–86.
  43. Cai X, Bazerque Juan A, Giannakis Georgios B. Inference of gene regulatory networks with sparse structural equation models exploiting genetic perturbations. PLOS Comput Biol. 2013;9(5): e1003068. · DOI
  44. Gillespie Daniel T. Exact stochastic simulation of coupled chemical reactions. J Phys Chem. 1977;81(25):2340–61. · DOI
  45. Gillespie Daniel T. The chemical Langevin equation. J Chem Phy. 2000;113(1):297–306. · DOI
  46. Gillespie Daniel T. Approximate accelerated stochastic simulation of chemically reacting systems. J Chem Phys. 2001;115(4):1716–33. · DOI
  47. Kempe H, Schwabe A, Crémazy F, Verschure Pernette J, Bruggeman Frank J. The volumes and transcript counts of single cells reveal concentration homeostasis and capture biological noise. Mol Biol Cell. 2015;26(4):797–804. · DOI
  48. Couturier Charles P, Ayyadhury S, Le Phuong U, Nadaf J, Monlong J, Riva G, Allache R, Baig S, Yan X, Bourgey M, et al. Single-cell rna-seq reveals that glioblastoma recapitulates a normal neurodevelopmental hierarchy. Nat Commun. 2020;11(1):1–19. · DOI · Paper page
  49. Svensson V, Vento-Tormo R, Teichmann Sarah A. Exponential scaling of single-cell rna-seq in the past decade. Nat Protoc. 2018;13(4):599–604. · DOI · Paper page
  50. Barabasi N, Albert N. Emergence of scaling in random networks. Science. 1999;286(5439):509–12. · DOI · Paper page
  51. Tanouchi Y, Pai A, Park H, Huang S, Stamatov R, Buchler NE, You L. A noisy linear map underlies oscillations in cell size and gene expression in bacteria. Nature. 2015;523(7560):357–60. · DOI
  52. Bertaux François, Marguerat Samuel, Shahrezaei Vahid. Division rate, cell size and proteome allocation: impact on gene expression noise and implications for the dynamics of genetic circuits. Royal Soc Open Sci. 2018;5(3): 172234. · DOI
  53. Cadart C, Monnier S, Grilli J, Sáez Pablo J, Srivastava N, Attia R, Terriac E, Baum B, Cosentino-Lagomarsino M, Piel M. Size control in mammalian cells involves modulation of both growth rate and cell cycle duration. Nat Commun. 2018;9(1):1–15. · DOI
  54. Charlebois Daniel A, Balázsi G. Modeling cell population dynamics. In Silico Biol. 2019;13(1–2):21–39. · DOI
  55. Mora T, Walczak AM. Effect of phenotypic selection on stochastic gene expression. J Phys Chem. 2013;117(42):13194–205. · DOI
  56. Lee Kangtaek, Matsoukas Themis. Simultaneous coagulation and break-up using constant-n monte carlo. Powder Technol. 2000;110(1):82–9. · DOI
  57. Mantzaris Nikos V. Stochastic and deterministic simulations of heterogeneous cell population dynamics. J Theor Biol. 2006;241(3):690–706. · DOI
  58. Thomas Philipp. Intrinsic and extrinsic noise of gene expression in lineage trees. Sci Rep. 2019;9(1):474. · DOI
  59. Klein AM, Mazutis L, Akartuna I, Tallapragada N, Veres A, Li V, Peshkin L, Weitz DA, Kirschner MW. Droplet barcoding for single-cell transcriptomics applied to embryonic stem cells. Cell. 2015;161(5):1187–201. · DOI · Paper page
  60. Yamawaki TM, Lu DR, Ellwanger Daniel C, Bhatt D, Manzanill P, Arias V, Zhou H, Yoon OhK, Homann O, Wang S, Li C-M. Systematic comparison of high-throughput single-cell RNA-seq methods for immune cell profiling. BMC Genomics. 2021;22(1):66. · DOI
  61. Vallejos Catalina A, Risso D, Scialdone A, Dudoit S, Marioni John C. Normalizing single-cell RNA sequencing data: challenges and opportunities. Nat Methods. 2017;14(6):565–71. · DOI · Paper page
  62. Breda J, Zavolan M, van Nimwegen E. Bayesian inference of gene expression states from single-cell RNA-seq data. Nat Biotechnol. 2021;39(8):1008–16. · DOI
  63. Patruno L, Maspero D, Craighero F, Angaroni F, Antoniotti M, Graudenzi A. A review of computational strategies for denoising and imputation of single-cell transcriptomic data. Brief Bioinf. 2021;22(4):222.
  64. Chan Thalia E, Pallaseni Ananth V, Babtie Ann C, McEwen Kirsten R, Stumpf Michael PH. Empirical bayes meets information theoretical network reconstruction from single cell data. BioRxiv, 2018;pp. 264853. · DOI
  65. Huynh-Thu V, Irrthum A, Wehenkel L, Geurts P. Inferring regulatory networks from expression data using tree-based methods. PLOS ONE. 2010;5(9): e12776. · DOI · Paper page
  66. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143(1):29–36. · DOI · Paper page
  67. Boyd K, Eng Kevin H, David C. Page. area under the precision-recall curve: point estimates and confidence intervals. In: Hendrik B, Kristian K, Siegfried N, Filip V, editors. Machine learning and knowledge discovery in databases lecture notes in computer science. Berlin: Springer; 2013. p. 451–66.
  68. Haibo He. Imbalanced Learning: Foundations, Algorithms, and Applications, Wiley.
  69. Chan Thalia E, Stumpf Michael PH, Babtie Ann C. Gene regulatory network inference from single-cell data using multivariate information measures. Cell Syst. 2017;5(3):251–67. · DOI · Paper page
  70. Yamawaki Tracy M, Lu Daniel R, Ellwanger Daniel C, Bhatt D, Manzanillo P, Arias V, Zhou H, Yoon OhK, Homann O, Wang S, et al. Systematic comparison of high-throughput single-cell rna-seq methods for immune cell profiling. BMC Genomics. 2021;22(1):1–18.
  71. Barron M, Li J. Identifying and removing the cell-cycle effect from single-cell rna-sequencing data. Sci Rep. 2016;6(1):1–10. · DOI
  72. Liu J, Yang M, Zhao W and Xiaobo Z. Ccpe: Cell cycle pseudotime estimation for single cell rna-seq data. bioRxiv; 2021. · DOI
  73. Hsiao Chiaowen J, Tung P, Blischak John D, Burnett Jonathan E, Barr Kenneth A, Dey Kushal K, Stephens M, Gilad Y. Characterizing and inferring quantitative cell cycle phase in single-cell rna-seq data analysis. Genome Res. 2020;30(4):611–21. · DOI
  74. Ly L-H, Vingron M. Effect of imputation on gene network reconstruction from single-cell RNA-seq data. bioRxiv, pp. 439623, 2021. · DOI
  75. Steinheuer Lisa M, Canzler S, Hackermüller J. Benchmarking scRNA-seq imputation tools with respect to network inference highlights deficits in performance at high levels of sparsity. bioRxiv, pp. 438193, 2021. · DOI
  76. Marouf M, Machart P, Bansal V, Kilian C, Magruder Daniel S, Krebs Christian F, Bonn S. Realistic in silico generation and augmentation of single-cell rna-seq data using generative adversarial networks. Nat Commun. 2020;11(1):1–12. · DOI

Metadata

Authors
Lasri, A., Shahrezaei, V., & Sturrock, M.
Affiliations
  • Royal College of Surgeons in Ireland (IE)
  • Imperial College London (GB)
Corresponding author
Sturrock, Marc
ORCID
Venue
BMC Bioinformatics
Publisher
Springer Science and Business Media LLC
Published
2022-06-17
Volume
23
Issue
1
Pages
236-236
ISSN
1471-2105 (ISSN-L)
DOI
10.1186/s12859-022-04778-9
PMC
PMC9204969
OpenAlex
W4283009707
Cited by
5
References
7
Citations by year
2026: 1 · 2025: 1 · 2024: 2 · 2022: 1
FWCI
0.33
Type
journal article
Language
en
Open access
Licence
Locations
  • BMC Bioinformatics (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
  • Spiral (Imperial College London) (submittedVersion, cc-by)
Funders
Horizon 2020 Framework Programme (766069)
Topics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene Regulatory Network Analysis
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Single-cell and spatial transcriptomics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Bioinformatics and Genomic Networks
Keywords
single-cell RNA sequencing, gene regulatory network inference, zero imputation, dropout, synthetic data generation, benchmarking, multiscale modeling, gene expression correlation
MeSH terms
Software, Sequence Analysis, RNA, Benchmarking, Gene Expression Profiling, Gene Regulatory Networks, Single-Cell Analysis

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review