Flexible comparison of batch correction methods for single-cell RNA-seq using BatchBench
Open accessPublished version of a preprint
Abstract
As the cost of single-cell RNA-seq experiments has decreased, an increasing number of datasets are now available. Combining newly generated and publicly accessible datasets is challenging due to non-biological signals, commonly known as batch effects. Although there are several computational methods available that can remove batch effects, evaluating which method performs best is not straightforward. Here, we present BatchBench (https://github.com/cellgeni/batchbench), a modular and flexible pipeline for comparing batch correction methods for single-cell RNA-seq data. We apply BatchBench to eight methods, highlighting their methodological differences and assess their performance and computational requirements through a compendium of well-studied datasets. This systematic comparison guides users in the choice of batch correction tool, and the pipeline makes it easy to evaluate other datasets.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Linear-embedding models such as canonical correlation analysis and Harmony perform well for batch correction on simpler integration tasks with distinct batch structures.
“Linear-embedding models such as canonical correlation analysis36 and Harmony37 were shown to perform well for batch correction on simpler integration tasks with distinct batch structures38,39.”
From Best practices for single-cell analysis across modalities (2023) · cites this paper as [39]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
limma powers differential expression analyses for RNA-sequencing and microarray studies
Ritchie, M. E., Phipson, B., Wu, D., Hu, Y., Law, C. W., Shi, W., & Smyth, G. K. (2015) · Nucleic Acids Research · doi:10.1093/nar/gkv007 · Cited by 40,458 · see citing works
Fast unfolding of communities in large networks
Blondel, V. D., Guillaume, J.-L., Lambiotte, R., & Lefebvre, E. (2008) · Journal of Statistical Mechanics: Theory and Experiment · doi:10.1088/1742-5468/2008/10/p10008 · Cited by 21,424 · see citing works
Comprehensive Integration of Single-Cell Data
Stuart, T., Butler, A., Hoffman, P., Hafemeister, C., Papalexi, E., Mauck, W. M., Hao, Y., Stoeckius, M., Smibert, P., & Satija, R. (2019) · Cell · doi:10.1016/j.cell.2019.05.031 · Cited by 15,453 · see citing works
Fast, sensitive and accurate integration of single-cell data with Harmony
Korsunsky, I., Millard, N., Fan, J., Slowikowski, K., Zhang, F., Wei, K., Baglaenko, Y., Brenner, M., Loh, P.-R., & Raychaudhuri, S. (2019) · Nature Methods · doi:10.1038/s41592-019-0619-0 · Cited by 10,488 · see citing works
UMAP: Uniform Manifold Approximation and Projection
McInnes, L., Healy, J., Saul, N., & Großberger, L. (2018) · Journal of Open Source Software · doi:10.21105/joss.00861 · Cited by 9,826 · see citing works
Leek, J. T., Johnson, W. E., Parker, H. S., Jaffe, A. E., & Storey, J. D. (2012) · Bioinformatics · doi:10.1093/bioinformatics/bts034 · Cited by 6,231 · see citing works
From Louvain to Leiden: guaranteeing well-connected communities
Traag, V. A., Waltman, L., & van Eck, N. J. (2019) · Scientific Reports · doi:10.1038/s41598-019-41695-z · Cited by 5,635 · see citing works
Nextflow enables reproducible computational workflows
Di Tommaso, P., Chatzou, M., Floden, E. W., Barja, P. P., Palumbo, E., & Notredame, C. (2017) · Nature Biotechnology · doi:10.1038/nbt.3820 · Cited by 4,001 · see citing works
Differential analysis of gene regulation at transcript resolution with RNA-seq
Trapnell, C., Hendrickson, D. G., Sauvageau, M., Goff, L., Rinn, J. L., & Pachter, L. (2013) · Nature Biotechnology · doi:10.1038/nbt.2450 · Cited by 3,492 · see citing works
Single-cell transcriptomics of 20 mouse organs creates a Tabula Muris
The Tabula Muris Consortium, coordination, O., coordination, L., processing, O. C. A., sequencing, L. P. A., analysis, C. D., annotation, C. T., Writing group, Supplemental text writing group, & Principal investigators (2018) · Nature · doi:10.1038/s41586-018-0590-4 · Cited by 3,326 · see citing works
Cell types in the mouse cortex and hippocampus revealed by single-cell RNA-seq
Zeisel, A., Muñoz-Manchado, A. B., Codeluppi, S., Lönnerberg, P., La Manno, G., Juréus, A., Marques, S., Munguba, H., He, L., Betsholtz, C., et al. (2015) · Science · doi:10.1126/science.aaa1934 · Cited by 3,124 · see citing works
Batch effects in single-cell RNA-sequencing data are corrected by matching mutual nearest neighbors
Haghverdi, L., Lun, A. T. L., Morgan, M. D., & Marioni, J. C. (2018) · Nature Biotechnology · doi:10.1038/nbt.4091 · Cited by 2,416 · see citing works
Current best practices in single-cell RNA-seq analysis: a tutorial
Luecken, M. D., & Theis, F. J. (2019) · Molecular Systems Biology · doi:10.15252/msb.20188746 · Cited by 2,231 · see citing works
Single-Cell Map of Diverse Immune Phenotypes in the Breast Tumor Microenvironment
Azizi, E., Carr, A. J., Plitas, G., Cornish, A. E., Konopacki, C., Prabhakaran, S., Nainys, J., Wu, K., Kiseliovas, V., Setty, M., et al. (2018) · Cell · doi:10.1016/j.cell.2018.05.060 · Cited by 1,976 · see citing works
Single-cell mRNA quantification and differential analysis with Census
Qiu, X., Hill, A. J., Packer, J. S., Lin, D., Ma, Y.-A., & Trapnell, C. (2017) · Nature Methods · doi:10.1038/nmeth.4150 · Cited by 1,885 · see citing works
Single-cell RNA sequencing technologies and bioinformatics pipelines
Hwang, B., Lee, J. H., & Bang, D. (2018) · Experimental & Molecular Medicine · doi:10.1038/s12276-018-0071-8 · Cited by 1,793 · see citing works
Mapping the Mouse Cell Atlas by Microwell-Seq
Han, X., Wang, R., Zhou, Y., Fei, L., Sun, H., Lai, S., Saadatpour, A., Zhou, Z., Chen, H., Ye, F., et al. (2018) · Cell · doi:10.1016/j.cell.2018.02.001 · Cited by 1,597 · see citing works
Single-Cell Transcriptome Profiling of Human Pancreatic Islets in Health and Type 2 Diabetes
Segerstolpe, Å., Palasantza, A., Eliasson, P., Andersson, E.-M., Andréasson, A.-C., Sun, X., Picelli, S., Sabirsh, A., Clausen, M., Bjursell, M. K., et al. (2016) · Cell Metabolism · doi:10.1016/j.cmet.2016.08.020 · Cited by 1,550 · see citing works
CellMarker: a manually curated resource of cell markers in human and mouse
Zhang, X., Lan, Y., Xu, J., Quan, F., Zhao, E., Deng, C., Luo, T., Xu, L., Liao, G., Yan, M., et al. (2019) · Nucleic Acids Research · doi:10.1093/nar/gky900 · Cited by 1,524 · see citing works
Baron, M., Veres, A., Wolock, S. L., Faust, A. L., Gaujoux, R., Vetere, A., Ryu, J. H., Wagner, B. K., Shen-Orr, S. S., Klein, A. M., et al. (2016) · Cell Systems · doi:10.1016/j.cels.2016.08.011 · Cited by 1,508 · see citing works
SC3: consensus clustering of single-cell RNA-seq data
Kiselev, V. Y., Kirschner, K., Schaub, M. T., Andrews, T., Yiu, A., Chandra, T., Natarajan, K. N., Reik, W., Barahona, M., Green, A. R., et al. (2017) · Nature Methods · doi:10.1038/nmeth.4236 · Cited by 1,500 · see citing works
Self-citationDiffusion pseudotime robustly reconstructs lineage branching
Haghverdi, L., Büttner, M., Wolf, F. A., Buettner, F., & Theis, F. J. (2016) · Nature Methods · doi:10.1038/nmeth.3971 · Cited by 1,385 · see citing works
Single-Cell RNA-Seq Reveals Dynamic, Random Monoallelic Gene Expression in Mammalian Cells
Deng, Q., Ramsköld, D., Reinius, B., & Sandberg, R. (2014) · Science · doi:10.1126/science.1245316 · Cited by 1,345 · see citing works
A Single-Cell Transcriptome Atlas of the Human Pancreas
Muraro, M. J., Dharmadhikari, G., Grün, D., Gröen, N., Dielen, T., Jansen, E., van Gurp, L., Engelse, M. A., Carlotti, F., de Koning, E. J. P., et al. (2016) · Cell Systems · doi:10.1016/j.cels.2016.09.002 · Cited by 1,288 · see citing works
Single-cell messenger RNA sequencing reveals rare intestinal cell types
Grün, D., Lyubimova, A., Kester, L., Wiebrands, K., Basak, O., Sasaki, N., Clevers, H., & van Oudenaarden, A. (2015) · Nature · doi:10.1038/nature14966 · Cited by 1,234 · see citing works
Single-cell transcriptomics reveals bimodality in expression and splicing in immune cells
Shalek, A. K., Satija, R., Adiconis, X., Gertner, R. S., Gaublomme, J. T., Raychowdhury, R., Schwartz, S., Yosef, N., Malboeuf, C., Lu, D., et al. (2013) · Nature · doi:10.1038/nature12172 · Cited by 1,190 · see citing works
A benchmark of batch-effect correction methods for single-cell RNA sequencing data
Tran, H. T. N., Ang, K. S., Chevrier, M., Zhang, X., Lee, N. Y. S., Goh, M., & Chen, J. (2020) · Genome Biology · doi:10.1186/s13059-019-1850-9 · Cited by 1,113 · see citing works
FUNDAMENTALS OF PLANARIAN REGENERATION
Reddien, P. W., & Alvarado, A. S. (2004) · Annual Review of Cell and Developmental Biology · doi:10.1146/annurev.cellbio.20.010403.095114 · Cited by 1,108 · see citing works
BBKNN: fast batch alignment of single cell transcriptomes
Polański, K., Young, M. D., Miao, Z., Meyer, K. B., Teichmann, S. A., & Park, J.-E. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btz625 · Cited by 932 · see citing works
Single-Cell RNA-Seq Technologies and Related Computational Data Analysis
Chen, G., Ning, B., & Shi, T. (2019) · Frontiers in Genetics · doi:10.3389/fgene.2019.00317 · Cited by 913 · see citing works
Bendall, S. C., Davis, K. L., Amir, E.-A. D., Tadmor, M. D., Simonds, E. F., Chen, T. J., Shenfeld, D. K., Nolan, G. P., & Pe’er, D. (2014) · Cell · doi:10.1016/j.cell.2014.04.005 · Cited by 906 · see citing works
Efficient integration of heterogeneous single-cell transcriptomes using Scanorama
Hie, B., Bryson, B. D., & Berger, B. (2019) · Nature Biotechnology · doi:10.1038/s41587-019-0113-3 · Cited by 853 · see citing works
scmap: projection of single-cell RNA-seq data across data sets
(2018) · doi:10.1038/nmeth.4644 · Cited by 735 · see citing works
From single-cell to cell-pool transcriptomes: Stochasticity in gene expression and RNA splicing
Marinov, G. K., Williams, B. A., McCue, K., Schroth, G. P., Gertz, J., Myers, R. M., & Wold, B. J. (2014) · Genome Research · doi:10.1101/gr.161034.113 · Cited by 468 · see citing works
Lönnberg, T., Svensson, V., James, K. R., Fernandez‐Ruiz, D., Sebina, I., Montandon, R., Soon, M. S. F., Fogg, L. G., Nair, A. S., Liligeto, U. N., et al. (2017) · Science Immunology · doi:10.1126/sciimmunol.aal2192 · Cited by 260 · see citing works
Kim, J. K., Kolodziejczyk, A. A., Ilicic, T., Teichmann, S. A., & Marioni, J. C. (2015) · Nature Communications · doi:10.1038/ncomms9687 · Cited by 251 · see citing works
Inferring the kinetics of stochastic gene expression from single-cell RNA-sequencing data
Kim, J. K., & Marioni, J. C. (2013) · Genome biology · doi:10.1186/gb-2013-14-1-r7 · Cited by 207 · see citing works
A multiple-site similarity measure
Diserud, O. H., & Ødegaard, F. (2006) · Biology Letters · doi:10.1098/rsbl.2006.0553 · Cited by 168 · see citing works
Analysis of allelic expression patterns in clonal somatic cells by single-cell RNA–seq
Reinius, B., Mold, J. E., Ramsköld, D., Deng, Q., Johnsson, P., Michaëlsson, J., Frisén, J., & Sandberg, R. (2016) · Nature Genetics · doi:10.1038/ng.3678 · Cited by 149 · see citing works
Computational approaches for interpreting sc<scp>RNA</scp>‐seq data
Rostom, R., Svensson, V., Teichmann, S. A., & Kar, G. (2017) · FEBS Letters · doi:10.1002/1873-3468.12684 · Cited by 107 · see citing works
Robust detection of alternative splicing in a population of single cells
Welch, J. D., Hu, Y., & Prins, J. F. (2016) · Nucleic Acids Research · doi:10.1093/nar/gkv1525 · Cited by 57 · see citing works
Temporal mixture modelling of single-cell RNA-seq data resolves a CD4 + T cell fate bifurcationPreprint
Lönnberg, T., Svensson, V., James, K. R., Fernandez‐Ruiz, D., Sebina, I., Montandon, R., Soon, M. S. F., Fogg, L. G., Stubbington, M. J. T., Bagger, F. O., et al. (2016) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/074971 · Cited by 5 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Benchmarking atlas-level data integration in single-cell genomics
Luecken, M. D., Büttner, M., Chaichoompu, K., Danese, A., Interlandi, M., Mueller, M. F., Strobl, D. C., Zappia, L., Dugas, M., Colomé-Tatché, M., et al. (2022) · Nature Methods · doi:10.1038/s41592-021-01336-8 · Cited by 1,438 · see citing works
Best practices for single-cell analysis across modalities
Heumos, L., Schaar, A. C., Lance, C., Litinetskaya, A., Drost, F., Zappia, L., Lücken, M. D., Strobl, D. C., Henao, J., Curion, F., et al. (2023) · Nature Reviews Genetics · doi:10.1038/s41576-023-00586-w · Cited by 1,051 · see citing works
Differential abundance testing on single-cell data using k-nearest neighbor graphs
Dann, E., Henderson, N. C., Teichmann, S. A., Morgan, M. D., & Marioni, J. C. (2022) · Nature Biotechnology · doi:10.1038/s41587-021-01033-z · Cited by 938 · see citing works
reComBat: batch-effect removal in large-scale multi-source gene-expression data integration
Adamer, M. F., Brüningk, S. C., Tejada-Arranz, A., Estermann, F., Basler, M., & Borgwardt, K. (2022) · Bioinformatics Advances · doi:10.1093/bioadv/vbac071 · Cited by 27 · see citing works
- Authors
- Chazarra-Gil, R., van Dongen, S., Kiselev, V. Y., & Hemberg, M.
- Affiliations
- Wellcome Sanger Institute (GB)
- Wellcome Sanger Institute, Wellcome Genome Campus, Hinxton CB10 1SA, UK
- Corresponding author
- Chazarra-Gil, Ruben
- Kiselev, Vladimir Yu
- Hemberg, Martin
- Venue
- Nucleic Acids Research
- Publisher
- Oxford University Press (OUP)
- Published
- 2021-02-01
- Volume
- 49
- Issue
- 7
- Pages
- e42-e42
- ISSN
- 0305-1048 (ISSN-L)
- 1362-4962
- PMC
- PMC8053088
- OpenAlex
- W3126492421
- Cited by
- 93
- References
- 42
- Citations by year
- 2026: 11 · 2025: 15 · 2024: 15 · 2023: 20 · 2022: 16 · 2021: 14 · 2020: 2
- FWCI
- 5.73
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Nucleic Acids Research (publishedVersion, cc-by)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Funders
- Wellcome Trust
- Polytechnic University of Valencia
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Single-cell and spatial transcriptomics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene expression and cancer classification
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Phylogenetic Studies
- Keywords
- batch effect correction, batch effects, single-cell RNA sequencing, benchmarking, method comparison, computational pipeline
- MeSH terms
- RNA-Seq, Animals, Humans, Software, Mice, Single-Cell Analysis, Datasets as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
