A comparison of automatic cell identification methods for single-cell RNA sequencing data
Open accessPublished version of a preprint
Abstract
Background: Single-cell transcriptomics is rapidly advancing our understanding of the cellular composition of complex tissues and organisms. A major limitation in most analysis pipelines is the reliance on manual annotations to determine cell identities, which are time-consuming and irreproducible. The exponential growth in the number of cells and samples has prompted the adaptation and development of supervised classification methods for automatic cell identification. Results: Here, we benchmarked 22 classification methods that automatically assign cell identities including single-cell-specific and general-purpose classifiers. The performance of the methods is evaluated using 27 publicly available single-cell RNA sequencing datasets of different sizes, technologies, species, and levels of complexity. We use 2 experimental setups to evaluate the performance of each method for within dataset predictions (intra-dataset) and across datasets (inter-dataset) based on accuracy, percentage of unclassified cells, and computation time. We further evaluate the methods’ sensitivity to the input features, number of cells per population, and their performance across different annotation levels and datasets. We find that most classifiers perform well on a variety of datasets with decreased accuracy for complex datasets with overlapping classes or deep annotations. The general-purpose support vector machine classifier has overall the best performance across the different experiments. Conclusions: We present a comprehensive evaluation of automatic cell identification methods for single-cell RNA sequencing data. All the code used for the evaluation is available on GitHub ( https://github.com/tabdelaal/scRNAseq_Benchmark ). Additionally, we provide a Snakemake workflow to facilitate the benchmarking and to support the extension of new methods and new datasets.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Annotation results obtained with pre-trained classifiers are strongly affected by the classifier type and the quality of training data.
“Annotation results obtained with pre-trained classifiers are strongly affected by the classifier type and the quality of the training data used to create the classifier54,55.”
From Best practices for single-cell analysis across modalities (2023) · cites this paper as [54]
Performance of automated classification depends partly on the number of cell types and the complexity (the relatedness between clusters) of a dataset.
“This analysis also found that performance depends partly on the number of cell types and the complexity (the relatedness between clusters) of a dataset, similar to what we observed.”
From A harmonized atlas of mouse spinal cord cell types and their spatial organization (2021) · cites this paper as [125]
A comparative analysis of automated classification approaches across diverse datasets found that SVM and neural network models showed the best performance on the Allen Brain Atlas dataset of 92 neuronal cell types.
“A comparative analysis of automated classification approaches across diverse datasets found that SVM and neural network models showed the best performance on the Allen Brain Atlas dataset of 92 neuronal cell types–a dataset similar in scale and complexity to the harmonized analysis here125.”
From A harmonized atlas of mouse spinal cord cell types and their spatial organization (2021) · cites this paper as [125]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016 · see citing works
Integrating single-cell transcriptomic data across different conditions, technologies, and species
Butler, A., Hoffman, P., Smibert, P., Papalexi, E., & Satija, R. (2018) · Nature Biotechnology · doi:10.1038/nbt.4096 · Cited by 13,174 · see citing works
UMAP: Uniform Manifold Approximation and Projection
McInnes, L., Healy, J., Saul, N., & Großberger, L. (2018) · Journal of Open Source Software · doi:10.21105/joss.00861 · Cited by 9,826 · see citing works
Massively parallel digital transcriptional profiling of single cells
Zheng, G. X. Y., Terry, J. M., Belgrader, P., Ryvkin, P., Bent, Z. W., Wilson, R., Ziraldo, S. B., Wheeler, T. D., McDermott, G. P., Zhu, J., et al. (2017) · Nature Communications · doi:10.1038/ncomms14049 · Cited by 7,353 · see citing works
UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction
McInnes, L., Healy, J., & Melville, J. (2018) · arXiv · doi:10.48550/arxiv.1802.03426 · Cited by 7,101 · see citing works
Aran, D., Looney, A. P., Liu, L., Wu, E., Fong, V., Hsu, A., Chak, S., Naikawadi, R. P., Wolters, P. J., Abate, A. R., et al. (2019) · Nature Immunology · doi:10.1038/s41590-018-0276-y · Cited by 4,984 · see citing works
The single-cell transcriptional landscape of mammalian organogenesis
Cao, J., Spielmann, M., Qiu, X., Huang, X., Ibrahim, D. M., Hill, A. J., Zhang, F., Mundlos, S., Christiansen, L., Steemers, F. J., et al. (2019) · Nature · doi:10.1038/s41586-019-0969-x · Cited by 4,490 · see citing works
Single-cell transcriptomics of 20 mouse organs creates a Tabula Muris
The Tabula Muris Consortium, coordination, O., coordination, L., processing, O. C. A., sequencing, L. P. A., analysis, C. D., annotation, C. T., Writing group, Supplemental text writing group, & Principal investigators (2018) · Nature · doi:10.1038/s41586-018-0590-4 · Cited by 3,326 · cited 1× · see citing works
“This approach proved very valuable in identifying novel cell populations and resulted in cellular maps of entire cell lineages, organs and even whole organisms [2–7].”
BackgroundFinak, G., McDavid, A., Yajima, M., Deng, J., Gersuk, V., Shalek, A. K., Slichter, C. K., Miller, H. W., McElrath, M. J., Prlic, M., et al. (2015) · Genome Biology · doi:10.1186/s13059-015-0844-5 · Cited by 3,159 · see citing works
Snakemake—a scalable bioinformatics workflow engine
Köster, J., & Rahmann, S. (2012) · Bioinformatics · doi:10.1093/bioinformatics/bts480 · Cited by 2,780 · see citing works
Deep generative modeling for single-cell transcriptomics
Lopez, R., Regier, J., Cole, M. B., Jordan, M. I., & Yosef, N. (2018) · Nature Methods · doi:10.1038/s41592-018-0229-2 · Cited by 2,530 · see citing works
Batch effects in single-cell RNA-sequencing data are corrected by matching mutual nearest neighbors
Haghverdi, L., Lun, A. T. L., Morgan, M. D., & Marioni, J. C. (2018) · Nature Biotechnology · doi:10.1038/nbt.4091 · Cited by 2,416 · see citing works
Current best practices in single-cell RNA-seq analysis: a tutorial
Luecken, M. D., & Theis, F. J. (2019) · Molecular Systems Biology · doi:10.15252/msb.20188746 · Cited by 2,231 · see citing works
Conserved cell types with divergent features in human versus mouse cortex
Hodge, R. D., Bakken, T. E., Miller, J. A., Smith, K. A., Barkan, E. R., Graybuck, L. T., Close, J. L., Long, B., Johansen, N., Penn, O., et al. (2019) · Nature · doi:10.1038/s41586-019-1506-7 · Cited by 1,775 · see citing works
Self-citationShared and distinct transcriptomic cell types across neocortical areas
Tasic, B., Yao, Z., Graybuck, L. T., Smith, K. A., Nguyen, T. N., Bertagnolli, D., Goldy, J., Garren, E., Economo, M. N., Viswanathan, S., et al. (2018) · Nature · doi:10.1038/s41586-018-0654-5 · Cited by 1,746 · see citing works
A comparison of single-cell trajectory inference methods
Saelens, W., Cannoodt, R., Todorov, H., & Saeys, Y. (2019) · Nature Biotechnology · doi:10.1038/s41587-019-0071-9 · Cited by 1,670 · see citing works
Mapping the Mouse Cell Atlas by Microwell-Seq
Han, X., Wang, R., Zhou, Y., Fei, L., Sun, H., Lai, S., Saadatpour, A., Zhou, Z., Chen, H., Ye, F., et al. (2018) · Cell · doi:10.1016/j.cell.2018.02.001 · Cited by 1,597 · see citing works
PanglaoDB: a web server for exploration of mouse and human single-cell RNA sequencing data
Franzén, O., Gan, L.-M., & Björkegren, J. L. M. (2019) · Database · doi:10.1093/database/baz046 · Cited by 1,551 · see citing works
Single-Cell Transcriptome Profiling of Human Pancreatic Islets in Health and Type 2 Diabetes
Segerstolpe, Å., Palasantza, A., Eliasson, P., Andersson, E.-M., Andréasson, A.-C., Sun, X., Picelli, S., Sabirsh, A., Clausen, M., Bjursell, M. K., et al. (2016) · Cell Metabolism · doi:10.1016/j.cmet.2016.08.020 · Cited by 1,550 · see citing works
CellMarker: a manually curated resource of cell markers in human and mouse
Zhang, X., Lan, Y., Xu, J., Quan, F., Zhao, E., Deng, C., Luo, T., Xu, L., Liao, G., Yan, M., et al. (2019) · Nucleic Acids Research · doi:10.1093/nar/gky900 · Cited by 1,524 · see citing works
Baron, M., Veres, A., Wolock, S. L., Faust, A. L., Gaujoux, R., Vetere, A., Ryu, J. H., Wagner, B. K., Shen-Orr, S. S., Klein, A. M., et al. (2016) · Cell Systems · doi:10.1016/j.cels.2016.08.011 · Cited by 1,508 · see citing works
Cao, J., Packer, J. S., Ramani, V., Cusanovich, D. A., Huynh, C., Daza, R., Qiu, X., Lee, C., Furlan, S. N., Steemers, F. J., et al. (2017) · doi:10.1101/104844 · Cited by 1,402 · see citing works
A Single-Cell Transcriptome Atlas of the Human Pancreas
Muraro, M. J., Dharmadhikari, G., Grün, D., Gröen, N., Dielen, T., Jansen, E., van Gurp, L., Engelse, M. A., Carlotti, F., de Koning, E. J. P., et al. (2016) · Cell Systems · doi:10.1016/j.cels.2016.09.002 · Cited by 1,288 · see citing works
Exponential scaling of single-cell RNA-seq in the past decade
Svensson, V., Vento‐Tormo, R., & Teichmann, S. A. (2018) · Nature Protocols · doi:10.1038/nprot.2017.149 · Cited by 936 · see citing works
Bias, robustness and scalability in single-cell differential expression analysis
Soneson, C., & Robinson, M. D. (2018) · Nature Methods · doi:10.1038/nmeth.4612 · Cited by 770 · see citing works
scmap: projection of single-cell RNA-seq data across data sets
(2018) · doi:10.1038/nmeth.4644 · Cited by 735 · see citing works
RNA Sequencing of Single Human Islet Cells Reveals Type 2 Diabetes Genes
Xin, Y., Kim, J., Okamoto, H., Ni, M., Wei, Y., Adler, C., Murphy, A. J., Yancopoulos, G. D., Lin, C., & Gromada, J. (2016) · Cell Metabolism · doi:10.1016/j.cmet.2016.08.018 · Cited by 677 · see citing works
Supervised classification enables rapid annotation of cell atlases
Pliner, H. A., Shendure, J., & Trapnell, C. (2019) · doi:10.1101/538652 · Cited by 498 · see citing works
Cell type atlas and lineage tree of a whole complex animal by single-cell transcriptomics
Plass, M., Solana, J., Wolf, F. A., Ayoub, S., Misios, A., Glažar, P., Obermayer, B., Theis, F. J., Kocks, C., & Rajewsky, N. (2018) · Science · doi:10.1126/science.aaq1723 · Cited by 474 · see citing works
Cell type transcriptome atlas for the planarian <i>Schmidtea mediterranea</i>
Fincher, C. T., Wurtzel, O., de Hoog, T., Kravarik, K. M., & Reddien, P. W. (2018) · Science · doi:10.1126/science.aaq1736 · Cited by 436 · see citing works
Mapping the Mouse Cell Atlas by Microwell-Seq
Han, X., Wang, R., Zhou, Y., Fei, L., Sun, H., Lai, S., Saadatpour, A., Zhou, Z., Chen, H., Ye, F., et al. (2018) · Cell · doi:10.1016/j.cell.2018.05.012 · Cited by 384 · see citing works
Crow, M., Paul, A., Ballouz, S., Huang, Z. J., & Gillis, J. (2018) · Nature Communications · doi:10.1038/s41467-018-03282-0 · Cited by 365 · see citing works
Tan, Y., & Cahan, P. (2019) · Cell Systems · doi:10.1016/j.cels.2019.06.004 · Cited by 352 · see citing works
Benchmarking single cell RNA-sequencing analysis pipelines using mixture control experiments
Tian, L., Dong, X., Freytag, S., Cao, K. L., Su, S., JalalAbadi, A., Amann‐Zalcenstein, D., Weber, T. S., Seidi, A., Jabbari, J. S., et al. (2019) · Nature Methods · doi:10.1038/s41592-019-0425-8 · Cited by 342 · see citing works
Snakemake—a scalable bioinformatics workflow engine
Köster, J., & Rahmann, S. (2018) · Bioinformatics · doi:10.1093/bioinformatics/bty350 · Cited by 310 · see citing works
Exploring the single-cell RNA-seq analysis landscape with the scRNA-tools database
Zappia, L., Phipson, B., & Oshlack, A. (2018) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1006245 · Cited by 304 · see citing works
A systematic performance evaluation of clustering methods for single-cell RNA-seq data
Duò, A., Robinson, M. D., & Soneson, C. (2018) · F1000Research · doi:10.12688/f1000research.15666.2 · Cited by 296 · see citing works
M3Drop: dropout-based feature selection for scRNASeq
Andrews, T. S., & Hemberg, M. (2018) · Bioinformatics · doi:10.1093/bioinformatics/bty1044 · Cited by 283 · see citing works
CHETAH: a selective, hierarchical cell type identification method for single-cell RNA sequencing
de Kanter, J. K., Lijnzaad, P., Candelli, T., Margaritis, T., & Holstege, F. C. P. (2019) · Nucleic Acids Research · doi:10.1093/nar/gkz543 · Cited by 274 · see citing works
ACTINN: automated identification of cell types in single cell RNA sequencing
Ma, F., & Pellegrini, M. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btz592 · Cited by 252 · see citing works
OMICtools: an informative directory for multi-omic data analysis
Henry, V., Bandrowski, A. E., Pepin, A.-S., Gonzalez, B. J., & Desfeux, A. (2014) · Database · doi:10.1093/database/bau069 · Cited by 236 · see citing works
A systematic performance evaluation of clustering methods for single-cell RNA-seq data
Duò, A., Robinson, M. D., & Soneson, C. (2020) · F1000Research · doi:10.12688/f1000research.15666.3 · Cited by 200 · see citing works
Exponential scaling of single-cell RNA-seq in the last decadePreprint
Svensson, V., Tormo, R. V., & Teichmann, S. A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.01379 · Cited by 166 · see citing works
Searching large-scale scRNA-seq databases via unbiased cell embedding with Cell BLAST
Cao, Z., Lin, W., Shen, L., Yang, D.-C., & Gao, G. (2020) · Nature Communications · doi:10.1038/s41467-020-17281-7 · Cited by 148 · see citing works
Lieberman, Y., Rokach, L., & Shay, T. (2018) · PLoS ONE · doi:10.1371/journal.pone.0205499 · Cited by 113 · see citing works
Systematic comparative analysis of single cell RNA-sequencing methodsPreprint
Ding, J., Adiconis, X., Simmons, S., Kowalczyk, M. S., Hession, C. C., Marjanovic, N. D., Hughes, T. K., Wadsworth, M. H., Burks, T., Nguyễn, L., et al. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/632216 · Cited by 106 · see citing works
Automated identification of Cell Types in Single Cell RNA SequencingPreprint
Ma, F.-Y., & Pellegrini, M. (2019) · bioRxiv · doi:10.1101/532093 · Cited by 75 · see citing works
Training Linear Discriminant Analysis in Linear Time
Cai, D., He, X., & Han, J. (2008) · Proceedings - International Conference on Data Engineering · doi:10.1109/icde.2008.4497429 · Cited by 73 · see citing works
Moana: A robust and scalable cell type classification framework for single-cell RNA-Seq dataPreprint
Wagner, F., & Yanai, I. (2018) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/456129 · Cited by 68 · see citing works
Díaz-Mejía, J. J., Meng, E. C., Pico, A. R., MacParland, S. A., Ketela, T., Pugh, T. J., Bader, G. D., & Morris, J. H. (2019) · F1000Research · doi:10.12688/f1000research.18490.1 · Cited by 50 · see citing works
Díaz-Mejía, J. J., Meng, E. C., Pico, A. R., MacParland, S. A., Ketela, T., Pugh, T. J., Bader, G. D., & Morris, J. H. (2019) · F1000Research · doi:10.12688/f1000research.18490.3 · Cited by 49 · see citing works
Johnson, T. S., Wang, T., Huang, Z., Yu, C. Y., Wu, Y., Han, Y., Zhang, Y., Huang, K., & Zhang, J. (2019) · Bioinformatics · doi:10.1093/bioinformatics/btz295 · Cited by 42 · see citing works
CHETAH: a selective, hierarchical cell type identification method for single-cell RNA sequencingPreprint
de Kanter, J. K., Lijnzaad, P., Candelli, T., et al. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/558908 · Cited by 36 · see citing works
Domanskyi, S., Szedlak, A., Hawkins, N. T., Wang, J., Paternostro, G., & Piermarocchi, C. (2019) · BMC Bioinformatics · doi:10.1186/s12859-019-2951-x · Cited by 35 · see citing works
Deep learning does not outperform classical machine learning for cell-type annotationPreprint
Köhler, N. D., Büttner, M., Andriamanga, N., & Theis, F. J. (2019) · bioRxiv · doi:10.1101/653907 · Cited by 32 · see citing works
Tan, Y., & Cahan, P. (2018) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/508085 · Cited by 22 · see citing works
Dynamic post-transcriptional regulation during embryonic stem cell differentiationPreprint
van den Berg, P. R., Budnik, B., Slavov, N., & Semrau, S. (2017) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/123497 · Cited by 16 · see citing works
scPred: Cell type prediction at single-cell resolutionPreprint
Alquicira-Hernández, J., Sathe, A., Ji, H. P., Nguyen, Q. H., & Powell, J. E. (2018) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/369538 · Cited by 15 · see citing works
Cell BLAST: Searching large-scale scRNA-seq databases via unbiased cell embeddingPreprint
Cao, Z., Lin, W., Shen, L., et al. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/587360 · Cited by 12 · see citing works
Boufea, K., Seth, S., & Batada, N. N. (2018) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/470203 · Cited by 12 · see citing works
Domanskyi, S., Szedlak, A., Hawkins, N. T., et al. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/539833 · Cited by 10 · see citing works
SCINA: Semi-Supervised Analysis of Single Cells in silicoPreprint
Zhang, Z., Luo, M. S. D., Zhong, X., Choi, J. H., Ma, Y., Mahrt, E., Guo, W., Stawiski, E. W., Wang, S., Modrusan, Z. D., et al. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/559872 · Cited by 10 · see citing works
Díaz-Mejía, J. J., Meng, E. C., Pico, A. R., et al. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/562082 · Cited by 8 · see citing works
scRNA-seq classification benchmarking
Abdelaal, T., Michielsen, L., Cats, D., et al. (2019) · Figshare · doi:10.5281/zenodo.3369158 · Cited by 1 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Eleven grand challenges in single-cell data science
Lähnemann, D., Köster, J., Szczurek, E., McCarthy, D. J., Hicks, S. C., Robinson, M. D., Vallejos, C. A., Campbell, K. R., Beerenwinkel, N., Mahfouz, A., et al. (2020) · Genome Biology · doi:10.1186/s13059-020-1926-6 · Cited by 1,327 · see citing works
Self-citationBest practices for single-cell analysis across modalities
Heumos, L., Schaar, A. C., Lance, C., Litinetskaya, A., Drost, F., Zappia, L., Lücken, M. D., Strobl, D. C., Henao, J., Curion, F., et al. (2023) · Nature Reviews Genetics · doi:10.1038/s41576-023-00586-w · Cited by 1,051 · see citing works
“Generally, pseudobulk methods with sum aggregation and mixed models such as MAST with random effect setting were found to be superior to naive methods, such as the popular Wilcoxon rank-sum test, which does not account for within-sample correlation 88 .”
MethodLarge-scale foundation model on single-cell transcriptomics
Hao, M., Gong, J., Zeng, X., Liu, C., Guo, Y., Cheng, X., Wang, T., Ma, J., Zhang, X., & Song, L. (2024) · Nature Methods · doi:10.1038/s41592-024-02305-7 · Cited by 628 · see citing works
Yang, F., Wang, W., Wang, F., Fang, Y., Tang, D., Huang, J., Lu, H., & Yao, J. (2021) · doi:10.1101/2021.12.05.471261 · Cited by 617 · see citing works
Mapping single-cell data to reference atlases by transfer learning
Lotfollahi, M., Naghipourfar, M., Luecken, M. D., Khajavi, M., Büttner, M., Wagenstetter, M., Avsec, Ž., Gayoso, A., Yosef, N., Interlandi, M., et al. (2022) · Nature Biotechnology · doi:10.1038/s41587-021-01001-7 · Cited by 602 · see citing works
Benchmarking of cell type deconvolution pipelines for transcriptomics data
Avila Cobos, F., Alquicira-Hernandez, J., Powell, J. E., Mestdagh, P., & De Preter, K. (2020) · Nature Communications · doi:10.1038/s41467-020-19015-1 · Cited by 446 · see citing works
Massively parallel Cas13 screens reveal principles for guide RNA design
Wessels, H.-H., Méndez-Mancilla, A., Guo, X., Legut, M., Daniloski, Z., & Sanjana, N. E. (2020) · Nature Biotechnology · doi:10.1038/s41587-020-0456-9 · Cited by 418 · see citing works
A harmonized atlas of mouse spinal cord cell types and their spatial organization
Russ, D. E., Cross, R. B. P., Li, L., Koch, S. C., Matson, K. J. E., Yadav, A., Alkaslasi, M. R., Lee, D. I., Le Pichon, C. E., Menon, V., et al. (2021) · Nature Communications · doi:10.1038/s41467-021-25125-1 · Cited by 255 · see citing works
Automated methods for cell type annotation on scRNA-seq data
Pasquini, G., Rojo Arias, J. E., Schäfer, P., & Busskamp, V. (2021) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2021.01.015 · Cited by 249 · see citing works
Transformer for one stop interpretable cell type annotation
Chen, J., Xu, H., Tao, W., Chen, Z., Zhao, Y., & Han, J.-D. J. (2023) · Nature Communications · doi:10.1038/s41467-023-35923-4 · Cited by 189 · see citing works
Statistical and machine learning methods for spatially resolved transcriptomics data analysis
Zeng, Z., Li, Y., Li, Y., & Luo, Y. (2022) · Genome Biology · doi:10.1186/s13059-022-02653-7 · Cited by 186 · see citing works
Shao, X., Yang, H., Zhuang, X., Liao, J., Yang, P., Cheng, J., Lu, X., Chen, H., & Fan, X. (2021) · Nucleic Acids Research · doi:10.1093/nar/gkab775 · Cited by 158 · see citing works
Evaluation of Cell Type Annotation R Packages on Single-Cell RNA-Seq Data
Huang, Q., Liu, Y., Du, Y., & Garmire, L. X. (2021) · Genomics, Proteomics & Bioinformatics · doi:10.1016/j.gpb.2020.07.004 · Cited by 149 · see citing works
Single-cell ATAC sequencing analysis: From data preprocessing to hypothesis generation
Baek, S., & Lee, I. (2020) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2020.06.012 · Cited by 151 · see citing works
clustifyr: an R package for automated single-cell RNA sequencing cluster classification
Fu, R., Gillen, A. E., Sheridan, R. M., Tian, C., Daya, M., Hao, Y., Hesselberth, J. R., & Riemondy, K. A. (2020) · F1000Research · doi:10.12688/f1000research.22969.2 · Cited by 135 · see citing works
Large Scale Foundation Model on Single-cell Transcriptomics
Hao, M., Gong, J., Zeng, X., Liu, C., Guo, Y., Cheng, X., Wang, T., Ma, J., Song, L., & Zhang, X. (2023) · doi:10.1101/2023.05.29.542705 · Cited by 639 · see citing works
Biologically informed deep learning to query gene programs in single-cell atlases
Lotfollahi, M., Rybakov, S., Hrovatin, K., Hediyeh-zadeh, S., Talavera-López, C., Misharin, A. V., & Theis, F. J. (2023) · Nature Cell Biology · doi:10.1038/s41556-022-01072-x · Cited by 100 · see citing works
A panoramic view of cell population dynamics in mammalian aging
Zhang, Z., Schaefer, C., Jiang, W., Lu, Z., Lee, J., Sziraki, A., Abdulraouf, A., Wick, B., Haeussler, M., Li, Z., et al. (2025) · Science · doi:10.1126/science.adn3949 · Cited by 95 · see citing works
Computational methods for the integrative analysis of single-cell data
Forcato, M., Romano, O., & Bicciato, S. (2021) · Briefings in Bioinformatics · doi:10.1093/bib/bbaa042 · Cited by 71 · see citing works
Automatic cell type identification methods for single-cell RNA sequencing
Xie, B., Jiang, Q., Mora, A., & Li, X. (2021) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2021.10.027 · Cited by 66 · see citing works
Prioritization of cell types responsive to biological perturbations in single-cell data with Augur
Squair, J. W., Skinnider, M. A., Gautier, M., Foster, L. J., & Courtine, G. (2021) · Nature Protocols · doi:10.1038/s41596-021-00561-x · Cited by 61 · see citing works
Wang, L., Nie, R., Yu, Z., Xin, R., Zheng, C., Zhang, Z., Zhang, J., & Cai, J. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-00244-4 · Cited by 62 · see citing works
Evaluating the Utilities of Foundation Models in Single-cell Data Analysis
Liu, T., Li, K., Wang, Y., Li, H., & Zhao, H. (2023) · doi:10.1101/2023.09.08.555192 · Cited by 62 · see citing works
Kannan, S., Farid, M., Lin, B. L., Miyamoto, M., & Kwon, C. (2021) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1009305 · Cited by 52 · see citing works
Johnson, N. D., Wu, X., Still, C. D., Chu, X., Petrick, A. T., Gerhard, G. S., Conneely, K. N., & DiStefano, J. K. (2021) · Clinical Epigenetics · doi:10.1186/s13148-021-01129-y · Cited by 43 · see citing works
CellO: comprehensive and hierarchical cell type classification of human cells with the Cell Ontology
Bernstein, M. N., Ma, Z., Gleicher, M., & Dewey, C. N. (2021) · iScience · doi:10.1016/j.isci.2020.101913 · Cited by 45 · see citing works
Sfaira accelerates data and model reuse in single cell genomics
Fischer, D. S., Dony, L., König, M., Moeed, A., Zappia, L., Heumos, L., Tritschler, S., Holmberg, O., Aliee, H., & Theis, F. J. (2021) · Genome Biology · doi:10.1186/s13059-021-02452-6 · Cited by 33 · see citing works
Single-Cell RNA Sequencing for Studying Human Cancers
Aran, D. (2023) · Annual Review of Biomedical Data Science · doi:10.1146/annurev-biodatasci-020722-091857 · Cited by 31 · see citing works
A novel strategy to dissect multifaceted macrophage function in human diseases
Karlinsey, K., Qu, L., Matz, A. J., & Zhou, B. (2022) · Journal of Leukocyte Biology · doi:10.1002/jlb.6mr0522-685r · Cited by 23 · see citing works
CRISPR/cas-mediated genome editing in sorghum — recent progress, challenges and prospects
Parikh, A., Brant, E. J., Baloglu, M. C., & Altpeter, F. (2021) · In Vitro Cellular & Developmental Biology - Plant · doi:10.1007/s11627-021-10215-y · Cited by 18 · see citing works
Capturing the multifaceted function of adipose tissue macrophages
Matz, A. J., Qu, L., Karlinsey, K., Vella, A. T., & Zhou, B. (2023) · Frontiers in Immunology · doi:10.3389/fimmu.2023.1148188 · Cited by 16 · see citing works
Xie, E., Cheng, L., Shireman, J., Cai, Y., Liu, J., Mohanty, C., Dey, M., & Kendziorski, C. (2024) · doi:10.1101/2024.12.04.626476 · Cited by 3 · see citing works
- Authors
- Abdelaal, T., Michielsen, L., Cats, D., Hoogduin, D., Mei, H., Reinders, M. J. T., & Mahfouz, A.
- Affiliations
- Leiden University Medical Center (NL)
- Corresponding author
- Mahfouz, Ahmed
- Venue
- Genome Biology
- Published
- 2019-09-09
- Volume
- 20
- Issue
- 1
- Pages
- 194-194
- ISSN
- 1465-6906 (ISSN-L)
- 1474-760X
- PMC
- PMC6734286
- OpenAlex
- W2971653850
- Cited by
- 628
- References
- 64
- Citations by year
- 2026: 65 · 2025: 75 · 2024: 94 · 2023: 91 · 2022: 102 · 2021: 117 · 2020: 68 · 2019: 14 · 2015: 1
- FWCI
- 25.82
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- Genome biology (publishedVersion, cc-by)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Research Repository (Delft University of Technology) (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- Leiden Repository (Leiden University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Data Archiving and Networked Services (DANS) (submittedVersion, cc-by)
- Funders
- European Commission H2020 MSCA (675743)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Single-cell and spatial transcriptomics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene expression and cancer classification
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Phylogenetic Studies
- Keywords
- single-cell RNA sequencing, cell type classification, classifier benchmarking, supervised classification, support vector machine, single-cell transcriptomics, Snakemake workflow
- MeSH terms
- Sequence Analysis, RNA, Gene Expression Profiling, Single-Cell Analysis, Support Vector Machine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
