The DOI matches its Crossref record; 47 of the 47 references it cites were found in a registry or an index.
Comparison of clustering tools in R for medium-sized 10x Genomics single-cell RNA-sequencing data
Open accessPublished version of a preprint
Abstract
Background: The commercially available 10x Genomics protocol to generate droplet-based single-cell RNA-seq (scRNA-seq) data is enjoying growing popularity among researchers. Fundamental to the analysis of such scRNA-seq data is the ability to cluster similar or same cells into non-overlapping groups. Many competing methods have been proposed for this task, but there is currently little guidance with regards to which method to use. Methods: Here we use one gold standard 10x Genomics dataset, generated from the mixture of three cell lines, as well as three silver standard 10x Genomics datasets generated from peripheral blood mononuclear cells to examine not only the accuracy but also robustness of a dozen methods. Results: We found that some methods, including Seurat and Cell Ranger, outperform other methods, although performance seems to be dependent on the complexity of the studied system. Furthermore, we found that solutions produced by different methods have little in common with each other. Conclusions: In light of this, we conclude that the choice of clustering tool crucially determines interpretation of scRNA-seq data generated by 10x Genomics. Hence practitioners and consumers should remain vigilant about the outcome of 10x Genomics scRNA-seq analysis.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Cluster co-membership alone cannot establish that two cell populations from different organs are the same or similar, because unbiased clustering may group unrelated cells.
“Cluster co-membership alone, however, is insufficient to conclude that two cell populations from different organs represent the same or similar cell types; at any given resolution, unbiased clustering that groups related cells may also group unrelated cells21.”
From Single-cell transcriptomics of 20 mouse organs creates a Tabula Muris (2018) · cites this paper as [21]
The Results (Evaluation of clustering tools, Silver standard data; page unavailable) report that most clusters contained mixtures of different inferred cell types, supporting the caution that co-membership alone does not establish cell-type identity. The paper evaluates PBMC datasets, however, and does not test populations from different organs or establish that such populations were grouped together. (LLM, conf=0.84)
Experimental — accuracy not yet measuredCommunity detection based on graph modularity optimization via the Louvain algorithm works best for cluster identification.
“Independent benchmarks5,50,51 showed that community detection based on graph modularity optimization via the Louvain algorithm works best for cluster identification.”
From Best practices for single-cell analysis across modalities (2023) · cites this paper as [51]
Louvain has been shown to outperform other clustering methods for single-cell RNA-seq data, and flow and mass cytometry data.
“It has been shown to outperform other clustering methods for single‐cell RNA‐seq data (Duò et al, 2018; Freytag et al, 2018), and flow and mass cytometry data (Weber & Robinson, 2016).”
From Current best practices in single-cell RNA-seq analysis: a tutorial (2019) · cites this paper as Freytag et al, 2018
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
STAR: ultrafast universal RNA-seq aligner
Dobin, A., Davis, C. A., Schlesinger, F., Drenkow, J., Zaleski, C., Jha, S., Batut, P., Chaisson, M., & Gingeras, T. R. (2013) · Bioinformatics · doi:10.1093/bioinformatics/bts635 · Cited by 51,709
Integrating single-cell transcriptomic data across different conditions, technologies, and species
Butler, A., Hoffman, P., Smibert, P., Papalexi, E., & Satija, R. (2018) · Nature Biotechnology · doi:10.1038/nbt.4096 · Cited by 13,174
Hubert, L., & Arabie, P. (1985) · Journal of Classification · doi:10.1007/bf01908075 · Cited by 8,219
Massively parallel digital transcriptional profiling of single cells
Zheng, G. X. Y., Terry, J. M., Belgrader, P., Ryvkin, P., Bent, Z. W., Wilson, R., Ziraldo, S. B., Wheeler, T. D., McDermott, G. P., Zhu, J., et al. (2017) · Nature Communications · doi:10.1038/ncomms14049 · Cited by 7,353
Trapnell, C., Cacchiarelli, D., Grimsby, J., Pokharel, P., Li, S., Morse, M., Lennon, N. J., Livak, K. J., Mikkelsen, T. S., & Rinn, J. L. (2014) · Nature Biotechnology · doi:10.1038/nbt.2859 · Cited by 7,314
Highly parallel genome-wide expression profiling of individual cells using nanoliter droplets.
Macosko, E. Z., Basu, A., Satija, R., Nemesh, J., Shekhar, K., Goldman, M., Tirosh, I., Bialas, A. R., Kamitaki, N., Martersteck, E. M., et al. (2015) · Cell · doi:10.1016/j.cell.2015.05.002 · Cited by 7,110
The Subread aligner: fast, accurate and scalable read mapping by seed-and-vote
Liao, Y., Smyth, G. K., & Shi, W. (2013) · Nucleic Acids Research · doi:10.1093/nar/gkt214 · Cited by 3,064
An overlap invariant entropy measure of 3D medical image alignment
Studholme, C., Hill, D., & Hawkes, D. J. (1999) · Pattern Recognition · doi:10.1016/s0031-3203(98)00091-0 · Cited by 2,167
McCarthy, D. J., Campbell, K. R., Lun, A. T. L., & Wills, Q. F. (2017) · Bioinformatics · doi:10.1093/bioinformatics/btw777 · Cited by 1,852
A step-by-step workflow for low-level analysis of single-cell RNA-seq data
Lun, A. T. L., McCarthy, D. J., & Marioni, J. C. (2016) · F1000Research · doi:10.12688/f1000research.9501.1 · Cited by 1,538
SC3: consensus clustering of single-cell RNA-seq data
Kiselev, V. Y., Kirschner, K., Schaub, M. T., Andrews, T., Yiu, A., Chandra, T., Natarajan, K. N., Reik, W., Barahona, M., Green, A. R., et al. (2017) · Nature Methods · doi:10.1038/nmeth.4236 · Cited by 1,500
Comparative Analysis of Single-Cell RNA Sequencing Methods
Ziegenhain, C., Vieth, B., Parekh, S., Reinius, B., Guillaumet-Adkins, A., Smets, M., Leonhardt, H., Heyn, H., Hellmann, I., & Enard, W. (2017) · Molecular Cell · doi:10.1016/j.molcel.2017.01.023 · Cited by 1,494
Single-cell messenger RNA sequencing reveals rare intestinal cell types
Grün, D., Lyubimova, A., Kester, L., Wiebrands, K., Basak, O., Sasaki, N., Clevers, H., & van Oudenaarden, A. (2015) · Nature · doi:10.1038/nature14966 · Cited by 1,234
V-Measure: A Conditional Entropy-Based External Cluster Evaluation Measure
Rosenberg, A., & Hirschberg, J. (2007) · Cited by 1,216
Pooling across cells to normalize single-cell RNA sequencing data with many zero counts
L. Lun, A. T., Bach, K., & Marioni, J. C. (2016) · Genome Biology · doi:10.1186/s13059-016-0947-7 · Cited by 1,108
Multiplexed droplet single-cell RNA-sequencing using natural genetic variation
Kang, H. M., Subramaniam, M., Targ, S., Nguyen, M., Maliskova, L., McCarthy, E., Wan, E., Wong, S., Byrnes, L., Lanata, C. M., et al. (2018) · Nature Biotechnology · doi:10.1038/nbt.4042 · Cited by 1,101
Li, H., Courtois, E. T., Sengupta, D., Tan, Y., Chen, K. H., Goh, J. J. L., Kong, S. L., Chua, C., Hon, L. K., Tan, W. S., et al. (2017) · Nature Genetics · doi:10.1038/ng.3818 · Cited by 1,080
A practical guide to single-cell RNA-sequencing for biomedical research and clinical applications
Haque, A., Engel, J., Teichmann, S. A., & Lönnberg, T. (2017) · Genome Medicine · doi:10.1186/s13073-017-0467-4 · Cited by 1,071
SEQC/MAQC-III Consortium (2014) · Nature Biotechnology · doi:10.1038/nbt.2957 · Cited by 983
MAQC Consortium (2010) · Nature Biotechnology · doi:10.1038/nbt.1665 · Cited by 889
Scaling single-cell genomics from phenomenology to mechanism
Tanay, A., & Regev, A. (2017) · Nature · doi:10.1038/nature21350 · Cited by 792
Visualization and analysis of single-cell RNA-seq data by kernel-based similarity learning
Wang, B., Zhu, J., Pierson, E., Ramazzotti, D., & Batzoglou, S. (2017) · Nature Methods · doi:10.1038/nmeth.4207 · Cited by 699
TSCAN: Pseudo-time reconstruction and evaluation in single-cell RNA-seq analysis
Ji, Z., & Ji, H. (2016) · Nucleic Acids Research · doi:10.1093/nar/gkw430 · Cited by 643
De Novo Prediction of Stem Cell Identity using Single-Cell Transcriptome Data
Grün, D., Muraro, M. J., Boisset, J.-C., Wiebrands, K., Lyubimova, A., Dharmadhikari, G., van den Born, M., van Es, J., Jansen, E., Clevers, H., et al. (2016) · Cell Stem Cell · doi:10.1016/j.stem.2016.05.010 · Cited by 586
CIDR: Ultrafast and accurate clustering through imputation for single-cell RNA-seq data
Lin, P., Troup, M., & Ho, J. W. K. (2017) · Genome biology · doi:10.1186/s13059-017-1188-0 · Cited by 537
Single-cell mapping of lineage and identity in direct reprogramming
Biddy, B. A., Kong, W., Kamimoto, K., Guo, C., Waye, S. E., Sun, T., & Morris, S. A. (2018) · Nature · doi:10.1038/s41586-018-0744-4 · Cited by 390
Proportion of peripheral blood and decidual CD4+ CD25bright regulatory T cells in pre-eclampsia
Sasaki, Y., Darmochwał-Kolarz, D., Suzuki, D., Sakai, M., Ito, M., Shima, T., Shiozaki, A., Roliński, J., & Saito, S. (2007) · Clinical & Experimental Immunology · doi:10.1111/j.1365-2249.2007.03397.x · Cited by 353
Cell-type specific gene expression profiles of leukocytes in human peripheral blood
Palmer, C., Diehn, M., Alizadeh, A. A., & Brown, P. O. (2006) · BMC Genomics · doi:10.1186/1471-2164-7-115 · Cited by 331
Exploring the single-cell RNA-seq analysis landscape with the scRNA-tools database
Zappia, L., Phipson, B., & Oshlack, A. (2018) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1006245 · Cited by 319
Patterns of ribosomal protein expression specify normal and malignant human cells
Guimaraes, J. C., & Zavolan, M. (2016) · Genome biology · doi:10.1186/s13059-016-1104-z · Cited by 251
Visualizing the structure of RNA-seq expression data using grade of membership models
Dey, K. K., Hsiao, C. J., & Stephens, M. (2017) · PLoS Genetics · doi:10.1371/journal.pgen.1006599 · Cited by 161
Impact of similarity metrics on single-cell RNA-seq data clustering
Kim, T., Chen, I. R., Lin, Y., Wang, A. Y.-Y., Yang, J. Y. H., & Yang, P. (2019) · Briefings in Bioinformatics · doi:10.1093/bib/bby076 · Cited by 138
SIMLR: A Tool for Large‐Scale Genomic Analyses by Multi‐Kernel Learning
Wang, B., Ramazzotti, D., Sano, L. D., Zhu, J., Pierson, E., & Batzoglou, S. (2017) · PROTEOMICS · doi:10.1002/pmic.201700232 · Cited by 132
Linnorm: improved statistical analysis for single cell RNA-seq expression data
Yip, S. H., Wang, P., Kocher, J.-P. A., Sham, P. C., & Wang, J. (2017) · Nucleic Acids Research · doi:10.1093/nar/gkx828 · Cited by 121
scPipe: A flexible R/Bioconductor preprocessing pipeline for single-cell RNA-sequencing data
Tian, L., Su, S., Dong, X., Amann‐Zalcenstein, D., Biben, C., Seidi, A., Hilton, D. J., Naik, S. H., & Ritchie, M. E. (2018) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1006361 · Cited by 121
Self-citationJing, Y., Gravenstein, S., Chaganty, N. R., Chen, N., Lyerly, H. K., Joyce, S., & Deng, Y. (2007) · Experimental Gerontology · doi:10.1016/j.exger.2007.01.009 · Cited by 111
Fast and accurate single-cell RNA-seq analysis by clustering of transcript-compatibility counts
Ntranos, V., Kamath, G. M., Zhang, J. M., Pachter, L. S., & Tse, D. N. C. (2016) · Genome biology · doi:10.1186/s13059-016-0970-8 · Cited by 110
Evaluation of external RNA controls for the assessment of microarray performance
Tong, W., Lucas, A. B., Shippy, R., Fan, X., Fang, H., Hong, H., Orr, M. S., Chu, T., Guo, X., Collins, P. J., et al. (2006) · Nature Biotechnology · doi:10.1038/nbt1237 · Cited by 109
A systematic performance evaluation of clustering methods for single-cell RNA-seq data
Duò, A., Robinson, M. D., & Soneson, C. (2018) · F1000Research · doi:10.12688/f1000research.15666.1 · Cited by 97
Haemopedia: An Expression Atlas of Murine Hematopoietic Cells
de Graaf, C. A., Choi, J., Baldwin, T. M., Bolden, J. E., Fairfax, K. A., Robinson, A. J., Biben, C., Morgan, C., Ramsay, K. A., Ng, A. P., et al. (2016) · Stem Cell Reports · doi:10.1016/j.stemcr.2016.07.007 · Cited by 94
The Human Cell AtlasPreprint
Regev, A., Teichmann, S. A., Lander, E. S., Amit, I., Benoist, C. O., Birney, E., Bodenmiller, B., Campbell, P. J., Carninci, P., Clatworthy, M. R., et al. (2017) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/121202 · Cited by 83
RNA-seq mixology: designing realistic control experiments to compare protocols and analysis methods
Holik, A. Z., Law, C. W., Liu, R., Wang, Z., Wang, W., Ahn, J., Asselin-Labat, M.-L., Smyth, G. K., & Ritchie, M. E. (2016) · Nucleic Acids Research · doi:10.1093/nar/gkw1063 · Cited by 47
ascend : R package for analysis of single-cell RNA-seq data
Senabouth, A., Lukowski, S. W., Alquicira-Hernández, J., Andersen, S. B., Mei, X., Nguyen, Q. H., & Powell, J. E. (2019) · GigaScience · doi:10.1093/gigascience/giz087 · Cited by 46
SAVER: Gene expression recovery for UMI-based single cell RNA sequencingPreprint
Huang, M. C., Wang, J., Torre, E. A., Dueck, H., Shaffer, S., Bonasio, R., Murray, J. I., Raj, A., Li, M., & Zhang, N. R. (2017) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/138677 · Cited by 38
ascend : R package for analysis of single cell RNA-seq dataPreprint
Senabouth, A., Lukowski, S. W., Alquicira-Hernández, J., et al. (2017) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/207704 · Cited by 15
scPipe: a flexible R/Bioconductor preprocessing pipeline for single-cell RNA-sequencing dataPreprint
Tian, L., Su, S., Dong, X., Amann-Zalcenstein, D., Biben, C., Seidi, A., Hilton, D. J., Naik, S. H., & Ritchie, M. E. (2017) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/175927 · Cited by 14
Self-citationscRNA-seq mixology: towards better benchmarking of single cell RNA-seq analysis methodsPreprint
Tian, L., Dong, X., Freytag, S., Cao, K. L., Su, S., JalalAbadi, A., Amann‐Zalcenstein, D., Weber, T. S., Seidi, A., Jabbari, J. S., et al. (2018) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/433102 · Cited by 11
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Single-cell transcriptomics of 20 mouse organs creates a Tabula Muris
The Tabula Muris Consortium, coordination, O., coordination, L., processing, O. C. A., sequencing, L. P. A., analysis, C. D., annotation, C. T., Writing group, Supplemental text writing group, & Principal investigators (2018) · Nature · doi:10.1038/s41586-018-0590-4 · Cited by 2,842
Current best practices in single-cell RNA-seq analysis: a tutorial
Luecken, M. D., & Theis, F. J. (2019) · Molecular Systems Biology · doi:10.15252/msb.20188746 · Cited by 2,231
A single-cell atlas of the peripheral immune response in patients with severe COVID-19
Wilk, A. J., Rustagi, A., Zhao, N. Q., Roque, J., Martínez-Colón, G. J., McKechnie, J. L., Ivison, G. T., Ranganath, T., Vergara, R., Hollis, T., et al. (2020) · Nature Medicine · doi:10.1038/s41591-020-0944-y · Cited by 1,493
Eleven grand challenges in single-cell data science
Lähnemann, D., Köster, J., Szczurek, E., McCarthy, D. J., Hicks, S. C., Robinson, M. D., Vallejos, C. A., Campbell, K. R., Beerenwinkel, N., Mahfouz, A., et al. (2020) · Genome Biology · doi:10.1186/s13059-020-1926-6 · Cited by 1,327
Benchmarking atlas-level data integration in single-cell genomics
Luecken, M. D., Büttner, M., Chaichoompu, K., Danese, A., Interlandi, M., Mueller, M. F., Strobl, D. C., Zappia, L., Dugas, M., Colomé-Tatché, M., et al. (2022) · Nature Methods · doi:10.1038/s41592-021-01336-8 · Cited by 1,438
Challenges in unsupervised clustering of single-cell RNA-seq data
Kiselev, V. Y., Andrews, T. S., & Hemberg, M. (2019) · Nature Reviews Genetics · doi:10.1038/s41576-018-0088-9 · Cited by 1,184
Best practices for single-cell analysis across modalities
Heumos, L., Schaar, A. C., Lance, C., Litinetskaya, A., Drost, F., Zappia, L., Lücken, M. D., Strobl, D. C., Henao, J., Curion, F., et al. (2023) · Nature Reviews Genetics · doi:10.1038/s41576-023-00586-w · Cited by 1,051
“Differential gene expression (DGE) analysis is currently approached from two viewpoints.”
BackgroundOrchestrating single-cell analysis with Bioconductor
Amezquita, R. A., Lun, A. T. L., Becht, E., Carey, V. J., Carpp, L. N., Geistlinger, L., Marini, F., Rue-Albrecht, K., Risso, D., Soneson, C., et al. (2020) · Nature Methods · doi:10.1038/s41592-019-0654-x · Cited by 945
Mapping single-cell data to reference atlases by transfer learning
Lotfollahi, M., Naghipourfar, M., Luecken, M. D., Khajavi, M., Büttner, M., Wagenstetter, M., Avsec, Ž., Gayoso, A., Yosef, N., Interlandi, M., et al. (2022) · Nature Biotechnology · doi:10.1038/s41587-021-01001-7 · Cited by 602
Crowell, H. L., Soneson, C., Germain, P.-L., Calini, D., Collin, L., Raposo, C., Malhotra, D., & Robinson, M. D. (2020) · Nature Communications · doi:10.1038/s41467-020-19894-4 · Cited by 515
Krishna, C., DiNatale, R. G., Kuo, F., Srivastava, R. M., Vuong, L. H., Chowell, D., Gupta, S., Vanderbilt, C., Purohit, T. A., Liu, M., et al. (2021) · Cancer Cell · doi:10.1016/j.ccell.2021.03.007 · Cited by 439
Probabilistic cell-type assignment of single-cell RNA-seq for tumor microenvironment profiling
Zhang, A. W., O’Flanagan, C. H., Chavez, E. A., Lim, J. L., Ceglia, N., McPherson, A., Wiens, M., Walters, P., Chan, T., Hewitson, B., et al. (2019) · Nature Methods · doi:10.1038/s41592-019-0529-1 · Cited by 385
Benchmarking single cell RNA-sequencing analysis pipelines using mixture control experiments
Tian, L., Dong, X., Freytag, S., Cao, K. L., Su, S., JalalAbadi, A., Amann‐Zalcenstein, D., Weber, T. S., Seidi, A., Jabbari, J. S., et al. (2019) · Nature Methods · doi:10.1038/s41592-019-0425-8 · Cited by 342
Self-citationA systematic performance evaluation of clustering methods for single-cell RNA-seq data
Duò, A., Robinson, M. D., & Soneson, C. (2018) · F1000Research · doi:10.12688/f1000research.15666.2 · Cited by 296
Tutorial: guidelines for the computational analysis of single-cell RNA sequencing data
Andrews, T. S., Kiselev, V. Y., McCarthy, D. J., & Hemberg, M. (2020) · Nature Protocols · doi:10.1038/s41596-020-00409-w · Cited by 321
Rojahn, T. B., Vorstandlechner, V., Krausgruber, T., Bauer, W. M., Alkon, N., Bangert, C., Thaler, F. M., Sadeghyar, F., Fortelny, N., Fife, V. E. R., et al. (2020) · Journal of Allergy and Clinical Immunology · doi:10.1016/j.jaci.2020.03.041 · Cited by 233
Wilk, A. J., Lee, M. J., Wei, B., Parks, B. E., Pi, R., Martínez-Colón, G. J., Ranganath, T., Zhao, N. Q., Taylor, S., Becker, W. R., et al. (2021) · The Journal of Experimental Medicine · doi:10.1084/jem.20210582 · Cited by 214
Advances and Opportunities in Single-Cell Transcriptomics for Plant Research
Seyfferth, C., Renema, J., Wendrich, J. R., Eekhout, T., Seurinck, R., Vandamme, N., Blob, B., Saeys, Y., Helariutta, Y., Birnbaum, K. D., et al. (2021) · Annual Review of Plant Biology · doi:10.1146/annurev-arplant-081720-010120 · Cited by 214
Bangert, C., Rindler, K., Krausgruber, T., Alkon, N., Thaler, F. M., Kurz, H., Ayub, T., Demirtas, D., Fortelny, N., Vorstandlechner, V., et al. (2021) · Science Immunology · doi:10.1126/sciimmunol.abe2749 · Cited by 205
A Single-Cell Transcriptional Atlas of the Developing Murine Cerebellum
Carter, R. A., Bihannic, L., Rosencrance, C. D., Hadley, J. L., Tong, Y., Phoenix, T. N., Natarajan, S., Easton, J. B., Northcott, P. A., & Gawad, C. (2018) · Current Biology · doi:10.1016/j.cub.2018.07.062 · Cited by 190
Li, Z., & Zhou, X. (2022) · Genome biology · doi:10.1186/s13059-022-02734-7 · Cited by 176
Lin, Y., Cao, Y., Kim, H. J., Salim, A., Speed, T. P., Lin, D., Yang, P., & Yang, J. Y. H. (2020) · Molecular Systems Biology · doi:10.15252/msb.20199389 · Cited by 168
Over 1000 tools reveal trends in the single-cell RNA-seq analysis landscape
Zappia, L., & Theis, F. J. (2021) · Genome biology · doi:10.1186/s13059-021-02519-4 · Cited by 170
Interfacing Seurat with the R tidy universe
Mangiola, S., Doyle, M., & Papenfuss, A. T. (2021) · Bioinformatics · doi:10.1093/bioinformatics/btab404 · Cited by 153
Germain, P., Sonrel, A., & Robinson, M. D. (2020) · Genome biology · doi:10.1186/s13059-020-02136-7 · Cited by 148
Deciphering the functional heterogeneity of skin fibroblasts using single‐cell RNA sequencing
Vorstandlechner, V., Laggner, M., Kalinina, P., Haslik, W., Radtke, C., Shaw, L. E., Lichtenberger, B. M., Tschachler, E., Ankersmit, H. J., & Mildner, M. (2020) · The FASEB Journal · doi:10.1096/fj.201902001rr · Cited by 146
Comparative analysis of cell–cell communication at single-cell resolution
Wilk, A. J., Shalek, A. K., Holmes, S., & Blish, C. A. (2023) · Nature Biotechnology · doi:10.1038/s41587-023-01782-z · Cited by 141
Yu, L., Cao, Y., Yang, J. Y. H., & Yang, P. (2022) · Genome biology · doi:10.1186/s13059-022-02622-0 · Cited by 135
scMatch: a single-cell gene expression profile annotation tool using reference datasets
Hou, R., Denisenko, E., & Forrest, A. R. R. (2019) · Bioinformatics · doi:10.1093/bioinformatics/btz292 · Cited by 130
Demystifying “drop-outs” in single-cell UMI data
Kim, T. H., Zhou, X., & Chen, M. (2020) · Genome biology · doi:10.1186/s13059-020-02096-y · Cited by 130
SAFE-clustering: Single-cell Aggregated (from Ensemble) clustering for single-cell RNA-seq data
Yang, Y., Huh, R., Culpepper, H., Lin, Y., Love, M. I., & Li, Y. (2018) · Bioinformatics · doi:10.1093/bioinformatics/bty793 · Cited by 126
Review of single-cell RNA-seq data clustering for cell-type identification and characterization
Zhang, S., Li, X., Lin, J., Lin, Q., & Wong, K. (2023) · RNA · doi:10.1261/rna.078965.121 · Cited by 120
Nonparametric expression analysis using inferential replicate counts
Zhu, A., Srivastava, A., Ibrahim, J. G., Patro, R., & Love, M. I. (2019) · Nucleic Acids Research · doi:10.1093/nar/gkz622 · Cited by 112
Kansler, E. R., Dadi, S., Krishna, C., Nixon, B. G., Stamatiades, E. G., Liu, M., Kuo, F., Zhang, J., Zhang, X., Capistrano, K. J., et al. (2022) · Nature Immunology · doi:10.1038/s41590-022-01213-2 · Cited by 110
BREM-SC: a bayesian random effects mixture model for joint clustering single cell multi-omics data
Wang, X., Sun, Z., Zhang, Y., Xu, Z., Xin, H., Huang, H., Duerr, R. H., Chen, K., Ding, Y., & Chen, W. (2020) · Nucleic Acids Research · doi:10.1093/nar/gkaa314 · Cited by 99
Biologically informed deep learning to query gene programs in single-cell atlases
Lotfollahi, M., Rybakov, S., Hrovatin, K., Hediyeh-zadeh, S., Talavera-López, C., Misharin, A. V., & Theis, F. J. (2023) · Nature Cell Biology · doi:10.1038/s41556-022-01072-x · Cited by 100
Evaluation of single-cell classifiers for single-cell RNA sequencing data sets
Zhao, X. S., Wu, S., Fang, N., Sun, X. S., & Fan, J. (2019) · Briefings in Bioinformatics · doi:10.1093/bib/bbz096 · Cited by 97
Data analysis guidelines for single-cell RNA-seq in biomedical studies and clinical applications
Su, M., Pan, T., Chen, Q.-Z., Zhou, W., Gong, Y., Xu, G., Yan, H.-Y., Li, S., Shi, Q.-Z., Zhang, Y., et al. (2022) · Military Medical Research · doi:10.1186/s40779-022-00434-8 · Cited by 108
Contrastive self-supervised clustering of scRNA-seq data
Ciortan, M., & Defrance, M. (2021) · BMC Bioinformatics · doi:10.1186/s12859-021-04210-8 · Cited by 93
A systematic performance evaluation of clustering methods for single-cell RNA-seq data
Duò, A., Robinson, M. D., & Soneson, C. (2018) · F1000Research · doi:10.12688/f1000research.15666.1 · Cited by 97
Self-assembling manifolds in single-cell RNA sequencing data
Tarashansky, A. J., Xue, Y., Li, P., Quake, S. R., & Wang, B. (2019) · eLife · doi:10.7554/elife.48994 · Cited by 80
Thirant, C., Peltier, A., Durand, S., Kramdi, A., Louis‐Brennetot, C., Pierre‐Eugène, C., Gautier, M., Costa, A., Grelier, A., Zaïdi, S., et al. (2023) · Nature Communications · doi:10.1038/s41467-023-38239-5 · Cited by 93
What are the applications of single-cell RNA sequencing in cancer research: a systematic review
Li, L., Xiong, F., Wang, Y., Zhang, S., Gong, Z., Li, X., He, Y., Shi, L., Wang, F., Liao, Q., et al. (2021) · Journal of Experimental & Clinical Cancer Research · doi:10.1186/s13046-021-01955-1 · Cited by 88
SAME-clustering: Single-cell Aggregated Clustering via Mixture Model Ensemble
Huh, R., Yang, Y., Jiang, Y., Shen, Y., & Li, Y. (2019) · Nucleic Acids Research · doi:10.1093/nar/gkz959 · Cited by 84
Human intermediate progenitor diversity during cortical development
Pebworth, M.-P., Ross, J. M., Andrews, M. G., Bhaduri, A., & Kriegstein, A. R. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2019415118 · Cited by 90
Benchmark and Parameter Sensitivity Analysis of Single-Cell RNA Sequencing Clustering Methods
Krzak, M., Raykov, Y. P., Boukouvalas, A., Cutillo, L., & Angelini, C. (2019) · Frontiers in Genetics · doi:10.3389/fgene.2019.01253 · Cited by 74
Tosoni, G., Ayyildiz, D., Bryois, J., Macnair, W., Fitzsimons, C. P., Lucassen, P. J., & Salta, E. (2023) · Neuron · doi:10.1016/j.neuron.2023.03.010 · Cited by 77
Benchmarking UMI-based single-cell RNA-seq preprocessing workflows
You, Y., Tian, L., Su, S., Dong, X., Jabbari, J. S., Hickey, P. F., & Ritchie, M. E. (2021) · Genome biology · doi:10.1186/s13059-021-02552-3 · Cited by 76
Self-citationsccomp: Robust differential composition and variability analysis for single-cell data
Mangiola, S., Roth‐Schulze, A. J., Trussart, M., Zozaya‐Valdés, E., Ma, M., Gao, Z., Rubin, A. F., Speed, T. P., Shim, H., & Papenfuss, A. T. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2203828120 · Cited by 79
Sun, Z., Chen, L., Xin, H., Jiang, Y., Huang, Q., Cillo, A. R., Tabib, T., Kolls, J. K., Bruno, T. C., Lafyatis, R., et al. (2019) · Nature Communications · doi:10.1038/s41467-019-09639-3 · Cited by 71
Salas, D., Stacey, R. G., Akinlaja, M. O., & Foster, L. J. (2019) · Molecular & Cellular Proteomics · doi:10.1074/mcp.r119.001803 · Cited by 66
Wang, Q., Peng, C., Yang, M., Huang, F., Duan, X., Wang, S., Cheng, H., Yang, H., Zhao, H., & Qin, Q. (2021) · PLoS Pathogens · doi:10.1371/journal.ppat.1009665 · Cited by 64
Zhang, J., Liu, J., Yuan, Y., Huang, F., Ma, R., Luo, B., Xi, Z., Pan, T. S., Liu, B., Zhang, Y., et al. (2020) · PLoS Pathogens · doi:10.1371/journal.ppat.1008334 · Cited by 67
Huang, P., Zhao, Y., Zhong, J., Zhang, X., Liu, Q., Qiu, X., Chen, S., Yan, H., Hillyer, C. D., Mohandas, N., et al. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1915085117 · Cited by 65
Xie, J., Ma, A., Zhang, Y., Liu, B., Cao, S., Wang, C., Xu, J., Zhang, C., & Ma, Q. (2019) · Bioinformatics · doi:10.1093/bioinformatics/btz692 · Cited by 76
Autoencoder-based cluster ensembles for single-cell RNA-seq data analysis
Geddes, T. A., Kim, T.-Y., Nan, L., Burchfield, J. G., Yang, J. Y. H., Tao, D., & Yang, P. (2019) · BMC Bioinformatics · doi:10.1186/s12859-019-3179-5 · Cited by 60
scDC: single cell differential composition analysis
Cao, Y., Lin, Y., Ormerod, J. T., Yang, P., Yang, J. Y. H., & Lo, K. (2019) · BMC Bioinformatics · doi:10.1186/s12859-019-3211-9 · Cited by 55
Díaz-Mejía, J. J., Meng, E. C., Pico, A. R., MacParland, S. A., Ketela, T., Pugh, T. J., Bader, G. D., & Morris, J. H. (2019) · F1000Research · doi:10.12688/f1000research.18490.1 · Cited by 50
Díaz-Mejía, J. J., Meng, E. C., Pico, A. R., MacParland, S. A., Ketela, T., Pugh, T. J., Bader, G. D., & Morris, J. H. (2019) · F1000Research · doi:10.12688/f1000research.18490.3 · Cited by 49
ascend : R package for analysis of single-cell RNA-seq data
Senabouth, A., Lukowski, S. W., Alquicira-Hernández, J., Andersen, S. B., Mei, X., Nguyen, Q. H., & Powell, J. E. (2019) · GigaScience · doi:10.1093/gigascience/giz087 · Cited by 46
Chen, L., Zhai, Y., He, Q., Wang, W., & Deng, M. (2020) · Genes · doi:10.3390/genes11070792 · Cited by 47
Casey, M. J., Stumpf, P. S., & MacArthur, B. D. (2019) · WIREs Systems Biology and Medicine · doi:10.1002/wsbm.1471 · Cited by 43
Zeng, L., Yang, K., Zhang, T., Zhu, X., Hao, W., Chen, H., & Ge, J. (2022) · Journal of Autoimmunity · doi:10.1016/j.jaut.2022.102919 · Cited by 47
The Unmixing Problem: A Guide to Applying Single-Cell RNA Sequencing to Bone
Greenblatt, M. B., Ono, N., Ayturk, U. M., Debnath, S., & Lalani, S. (2019) · Journal of Bone and Mineral Research · doi:10.1002/jbmr.3802 · Cited by 43
Wang, J., Zou, Q., & Lin, C. (2021) · Briefings in Bioinformatics · doi:10.1093/bib/bbab345 · Cited by 45
A single-nucleus transcriptomic atlas of the adult Aedes aegypti mosquito
Goldman, O. V., DeFoe, A. E., Qi, Y., Jiao, Y., Weng, S., Wick, B. D., Houri-Zeevi, L., Lakhiani, P., Morita, T., Razzauti, J., et al. (2025) · Cell · doi:10.1016/j.cell.2025.10.008 · Cited by 46
Single-cell reference mapping to construct and extend cell-type hierarchies
Michielsen, L., Lotfollahi, M., Strobl, D. C., Sikkema, L., Reinders, M. J. T., Theis, F. J., & Mahfouz, A. E. (2023) · NAR Genomics and Bioinformatics · doi:10.1093/nargab/lqad070 · Cited by 43
RESCUE: imputing dropout events in single-cell RNA-sequencing data
Tracy, S., Yuan, G., & Dries, R. (2019) · BMC Bioinformatics · doi:10.1186/s12859-019-2977-0 · Cited by 41
Lack of evidence for increased transcriptional noise in aged tissues
Ibáñez-Solé, O., Ascensión, A. M., Araúzo-Bravo, M. J., & Izeta, A. (2022) · eLife · doi:10.7554/elife.80380 · Cited by 38
Latent cellular analysis robustly reveals subtle diversity in large-scale single-cell RNA-seq data
Cheng, C., Easton, J. B., Rosencrance, C. D., Li, Y., Ju, B., Williams, J. S., Mulder, H. L., Pang, Y., Chen, W., & Chen, X. '. (2019) · Nucleic Acids Research · doi:10.1093/nar/gkz826 · Cited by 35
Query to reference single-cell integration with transfer learningPreprint
Lotfollahi, M., Naghipourfar, M., Luecken, M. D., Khajavi, M., Büttner, M., Avsec, Ž., Misharin, A. V., & Theis, F. J. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.07.16.205997 · Cited by 38
Perspectives on rigor and reproducibility in single cell genomics
Gibson, G. C. (2022) · PLoS Genetics · doi:10.1371/journal.pgen.1010210 · Cited by 31
Shree, A., Pavan, M. K., & Zafar, H. (2023) · Nature Communications · doi:10.1038/s41467-023-43590-8 · Cited by 30
A comprehensive analysis framework for evaluating commercial single-cell RNA sequencing technologies
Simone, M. C. D., Hoover, J., Lau, J., Bennett, H. M., Wu, B., Chen, C., Menon, H., Au‐Yeung, A., Lear, S. P., Vaidya, S., et al. (2024) · Nucleic Acids Research · doi:10.1093/nar/gkae1186 · Cited by 57
Dynamic Alternative Splicing During Mouse Preimplantation Embryo Development
Xing, Y., Yang, W., Liu, G., Cui, X., Meng, H., Zhao, H., Zhao, X., Li, J., Liu, Z., Zhang, M. Q., et al. (2020) · Frontiers in Bioengineering and Biotechnology · doi:10.3389/fbioe.2020.00035 · Cited by 33
Single-cell transcriptomic analyses of dairy cattle ruminal epithelial cells during weaning
Gao, Y., Fang, L., Baldwin, R. L., Connor, E. E., Cole, J. B., Tassell, C. P. V., Ma, L., Li, C., & Liu, G. E. (2021) · Genomics · doi:10.1016/j.ygeno.2021.04.039 · Cited by 35
Probabilistic count matrix factorization for single cell expression data analysis
Durif, G., Modolo, L., Mold, J. E., Lambert‐Lacroix, S., & Picard, F. (2019) · Bioinformatics · doi:10.1093/bioinformatics/btz177 · Cited by 30
Applications for single-cell and spatial transcriptomics in plant research
Sang, Q., & Kong, F. (2024) · New Crops · doi:10.1016/j.ncrops.2024.100025 · Cited by 31
Bej, S., Galow, A.-M., David, R., Wolfien, M., & Olaf Wolkenhauer, A. (2021) · BMC Bioinformatics · doi:10.1186/s12859-021-04469-x · Cited by 28
Single-cell sequencing technology in tumor research
Bai, X., Li, Y., Zeng, X., Zhao, Q., & Zhang, Z. (2021) · Clinica Chimica Acta · doi:10.1016/j.cca.2021.03.013 · Cited by 30
Linear-time cluster ensembles of large-scale single-cell RNA-seq and multimodal dataPreprint
Do, V. H., Ringeling, F. R., & Canzar, S. (2021) · Genome Research · doi:10.1101/gr.267906.120 · Cited by 28
Carbonetto, P. S., Luo, K., Sarkar, A., Hung, A., Tayeb, K., Pott, S., & Stephens, M. (2023) · Genome biology · doi:10.1186/s13059-023-03067-9 · Cited by 26
Song, D., Li, K., Hemminger, Z., Wollman, R., & Li, J. J. (2021) · Bioinformatics · doi:10.1093/bioinformatics/btab273 · Cited by 27
Triku: a feature selection method based on nearest neighbors for single-cell data
Ascensión, A. M., Ibáñez-Solé, O., Inza, I., Izeta, A., & Araúzo-Bravo, M. J. (2022) · GigaScience · doi:10.1093/gigascience/giac017 · Cited by 27
Single-Cell Analysis of the Transcriptome and Epigenome
Mazan-Mamczarz, K., Ha, J., De, S., & Sen, P. (2022) · Methods in molecular biology · doi:10.1007/978-1-0716-1831-8_3 · Cited by 27
Single-cell sequencing in ovarian cancer: a new frontier in precision medicine
Winterhoff, B. J. N., Talukdar, S., Chang, Z., Wang, J., & Starr, T. K. (2018) · Current Opinion in Obstetrics & Gynecology · doi:10.1097/gco.0000000000000516 · Cited by 26
RCA2: a scalable supervised clustering algorithm that reduces batch effects in scRNA-seq data
Schmidt, F., Ranjan, B., Lin, Q. X. X., Krishnan, V., Joanito, I., Honardoost, M. A., Nawaz, Z., Venkatesh, P. N., Tan, J. H. J., Rayan, N. A., et al. (2021) · Nucleic Acids Research · doi:10.1093/nar/gkab632 · Cited by 25
Xia, J., Fei, S., Huang, Y., Lai, W., Yu, Y., Liang, L., Wu, H., Swevers, L., Sun, J., & Feng, M. (2023) · Insect Biochemistry and Molecular Biology · doi:10.1016/j.ibmb.2023.104043 · Cited by 23
Nie, J., Zhang, S., Guo, Y., Liu, C., Shi, J., Wu, H., Na, R., Liang, Y., Yu, S., Quan, F., et al. (2024) · Cancer Research · doi:10.1158/0008-5472.can-24-1173 · Cited by 22
Essential guidelines for computational method benchmarking
Weber, L. M., Saelens, W., Cannoodt, R., Soneson, C., Hapfelmeier, A., Gardner, P. P., Boulesteix, A.-L., Saeys, Y., & Robinson, M. D. (2019) · Genome Biology · doi:10.1186/s13059-019-1738-8 · Cited by 21
CReSCENT: CanceR Single Cell ExpressioN Toolkit
Mohanraj, S., Díaz-Mejía, J. J., Pham, M. D., Elrick, H., Husić, M., Rashid, S., Luo, P., Bal, P., Lu, K., Patel, S. S., et al. (2020) · Nucleic Acids Research · doi:10.1093/nar/gkaa437 · Cited by 18
Integrated Single-Cell Transcriptomic Atlas of Human Kidney Endothelial Cells
Zhang, K., Kan, H., Mao, A., Yu, F., Li, G., Zhou, T., Feng, L., & Ma, X. (2024) · Journal of the American Society of Nephrology · doi:10.1681/asn.0000000000000320 · Cited by 20
Application of single-cell RNA sequencing in embryonic development
Shangguan, Y., Li, C., Lin, H., Ou, M., Tang, D., Dai, Y., & Yan, Q. (2020) · Genomics · doi:10.1016/j.ygeno.2020.08.007 · Cited by 21
Comparative Analysis of Commercial Single-Cell RNA Sequencing TechnologiesPreprint
Simone, M. C. D., Hoover, J., Lau, J., Bennet, H., Wu, B., Chen, C., Menon, H., Au‐Yeung, A., Lear, S. P., Vaidya, S., et al. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.06.18.599579 · Cited by 21
ClustAssess: tools for assessing the robustness of single-cell clusteringPreprint
Shahsavari, A., Munteanu, A., & Mohorianu, I. I. (2022) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2022.01.31.478592 · Cited by 18
Feng, M., Xia, J., Fei, S., Huang, Y., Lai, W., Swevers, L., & Sun, J. (2023) · The Journal of Immunology · doi:10.4049/jimmunol.2300007 · Cited by 18
Single-cell RNA sequencing for engineering and studying human islets
Augsornworawat, P., & Millman, J. R. (2020) · Current Opinion in Biomedical Engineering · doi:10.1016/j.cobme.2020.06.003 · Cited by 17
Single cell RNA-sequencing: replicability of cell types
Crow, M., & Gillis, J. (2019) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2018.12.002 · Cited by 17
Shiga, M., Seno, S., Onizuka, M., & Matsuda, H. (2021) · PeerJ · doi:10.7717/peerj.12087 · Cited by 15
Advances in Single-Cell Sequencing Technology and Its Application in Poultry Science
Liu, Y., Liang, S., Wang, B., Zhao, J., Zi, X., Yan, S., Dou, T., Jia, J., Wang, K., & Ge, C. (2022) · Genes · doi:10.3390/genes13122211 · Cited by 14
Metadata
- Authors
- Freytag, S., Tian, L., Lönnstedt, I., Ng, M., & Bahlo, M.
- Affiliations
- The University of Melbourne (AU)
- Corresponding author
- Freytag, Saskia
- Venue
- F1000Research
- Publisher
- F1000 Research Ltd
- Published
- 2018-08-15
- Volume
- 7
- Pages
- 1297
- ISSN
- 2046-1402 (ISSN-L)
- PMC
- PMC6124389
- OpenAlex
- W4251287806
- Cited by
- 220
- References
- 46
- Citations by year
- 2026: 11 · 2025: 19 · 2024: 20 · 2023: 29 · 2022: 25 · 2021: 45 · 2020: 30 · 2019: 32 · 2018: 8
- FWCI
- 10.23
- Type
- journal article
- Language
- en
- Update notices
- new_version (2018-12-19)
- Open access
- gold
- doi.org
- Locations
- F1000Research (publishedVersion, cc-by)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nd)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Minerva Access (University of Melbourne) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- National Health and Medical Research Council (IRIIS, ProgramGrant1054618, SeniorResearchFellowship110297)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Single-cell and spatial transcriptomics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene expression and cancer classification
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Phylogenetic Studies
- Keywords
- single-cell RNA sequencing, cell clustering, 10x Genomics, clustering algorithm comparison, Seurat, Cell Ranger, peripheral blood mononuclear cells, method robustness, R package
- MeSH terms
- Leukocytes, Mononuclear, RNA, Software, Gene Expression Profiling, Genomics, Single-Cell Analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
