The DOI matches its Crossref record; 88 of the 88 references it cites were found in a registry or an index.
Benchmarking atlas-level data integration in single-cell genomics
Open accessPublished version of a preprint
Abstract
Single-cell atlases often include samples that span locations, laboratories and conditions, leading to complex, nested batch effects in data. Thus, joint analysis of atlas datasets requires reliable data integration. To guide integration method choice, we benchmarked 68 method and preprocessing combinations on 85 batches of gene expression, chromatin accessibility and simulation data from 23 publications, altogether representing >1.2 million cells distributed in 13 atlas-level integration tasks. We evaluated methods according to scalability, usability and their ability to remove batch effects while retaining biological variation using 14 evaluation metrics. We show that highly variable gene selection improves the performance of data integration methods, whereas scaling pushes methods to prioritize batch removal over conservation of biological variation. Overall, scANVI, Scanorama, scVI and scGen perform well, particularly on complex integration tasks, while single-cell ATAC-sequencing integration performance is strongly affected by choice of feature space. Our freely available Python module and benchmarking pipeline can identify optimal data integration methods for new data, benchmark new methods and improve method development.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures














Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Such integrative analysis is complicated by batch effects and biological differences between primary tissue and cell culture26,27.”
From scPerturb: harmonized single-cell perturbation data (2024) · cites this paper as [27]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Furthermore, 8 evaluation metrics (3 for batch effect removal and 5 for biological variation retention) are computed to verify the integration ability by scIB47, which ranks TOSICA the first out of all 13 applicable methods evaluated on combined effectiveness in batch effect removal and biological variation retention (Fig. 5b).”
From Transformer for one stop interpretable cell type annotation (2023) · cites this paper as [47]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In particular, Harmony22 is an algorithm well suited for correcting batch effects that may be present in a dimensionality-reduced single-cell expression dataset,79”
From hdWGCNA identifies co-expression networks in high-dimensional transcriptomics data (2023) · cites this paper as [79]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Although this work simplifies dataset access, joint analysis is limited by the complexity of data integration27.”
From scPerturb: harmonized single-cell perturbation data (2024) · cites this paper as [27]
The cited source says what the claim attributes to it.
In the citing paper
“Depending on the complexity of the integration tasks, such as atlas integration, deep-learning approaches such as scANVI40, scVI41 and scGen42 as well as linear-embedding models such as Scanorama43 performed best”
From Best practices for single-cell analysis across modalities (2023) · cites this paper as [35]
10 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016 · cited 12×
“Similar to NMI, we compared the cell-type labels with the NMI-optimized Louvain clustering computed on the integrated dataset.”
BackgroundInfluentialSciPy 1.0: fundamental algorithms for scientific computing in Python
Virtanen, P., Gommers, R., Oliphant, T. E., Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, P., Weckesser, W., Bright, J., et al. (2020) · Nature Methods · doi:10.1038/s41592-019-0686-2 · Cited by 38,430
A note on two problems in connexion with graphs
Dijkstra, E. W. (1959) · Numerische Mathematik · doi:10.1007/bf01386390 · Cited by 29,065
Silhouettes: A graphical aid to the interpretation and validation of cluster analysis
Rousseeuw, P. J. (1987) · Journal of Computational and Applied Mathematics · doi:10.1016/0377-0427(87)90125-7 · Cited by 21,392
Comprehensive Integration of Single-Cell Data
Stuart, T., Butler, A., Hoffman, P., Hafemeister, C., Papalexi, E., Mauck, W. M., Hao, Y., Stoeckius, M., Smibert, P., & Satija, R. (2019) · Cell · doi:10.1016/j.cell.2019.05.031 · Cited by 15,453
ggplot2: Elegant Graphics for Data Analysis
Wickham, H. (2009) · doi:10.1007/978-0-387-98141-3 · Cited by 14,447
Fast, sensitive and accurate integration of single-cell data with Harmony
Korsunsky, I., Millard, N., Fan, J., Slowikowski, K., Zhang, F., Wei, K., Baglaenko, Y., Brenner, M., Loh, P.-R., & Raychaudhuri, S. (2019) · Nature Methods · doi:10.1038/s41592-019-0619-0 · Cited by 10,488
SciPy 1.0: fundamental algorithms for scientific computing in Python
Virtanen, P., Gommers, R., Oliphant, T. E., Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, P., Weckesser, W., Bright, J., et al. (2019) · Apollo - University of Cambridge Repository · doi:10.17863/cam.64134 · Cited by 10,460
UMAP: Uniform Manifold Approximation and Projection
McInnes, L., Healy, J., Saul, N., & Großberger, L. (2018) · Journal of Open Source Software · doi:10.21105/joss.00861 · Cited by 9,826
SCANPY: large-scale single-cell gene expression data analysis
Wolf, F. A., Angerer, P., & Theis, F. J. (2018) · Genome Biology · doi:10.1186/s13059-017-1382-0 · Cited by 8,881 · cited 1×
“Metrics were run on the integrated and unintegrated AnnData 47 objects.”
MethodSelf-citation
Cited by
scGPT: Towards Building a Foundation Model for Single-Cell Multi-omics Using Generative AI
Cui, H., Wang, C., Maan, H., Pang, K., Luo, F., & Wang, B. (2023) · doi:10.1101/2023.04.30.538439 · Cited by 1,694
Methods and applications for single-cell and spatial multi-omics
Vandereyken, K., Sifrim, A., Thienpont, B., & Voet, T. (2023) · Nature Reviews Genetics · doi:10.1038/s41576-023-00580-2 · Cited by 1,244
A high-resolution transcriptomic and spatial atlas of cell types in the whole mouse brain
Yao, Z., van Velthoven, C. T. J., Kunst, M., Zhang, M., McMillen, D., Lee, C., Jung, W., Goldy, J., Abdelhak, A., Aitken, M., et al. (2023) · Nature · doi:10.1038/s41586-023-06812-z · Cited by 1,203
Best practices for single-cell analysis across modalities
Heumos, L., Schaar, A. C., Lance, C., Litinetskaya, A., Drost, F., Zappia, L., Lücken, M. D., Strobl, D. C., Henao, J., Curion, F., et al. (2023) · Nature Reviews Genetics · doi:10.1038/s41576-023-00586-w · Cited by 1,051
“Once TFs of interest have been identified, scATAC-seq data allow for additional validation of the TF impact through footprinting, which indicates whether the TF is binding in the given cell cluster.”
MethodInfluentialSelf-citationDifferential abundance testing on single-cell data using k-nearest neighbor graphs
Dann, E., Henderson, N. C., Teichmann, S. A., Morgan, M. D., & Marioni, J. C. (2022) · Nature Biotechnology · doi:10.1038/s41587-021-01033-z · Cited by 938
Metadata
- Authors
- Luecken, M. D., Büttner, M., Chaichoompu, K., Danese, A., Interlandi, M., Mueller, M. F., Strobl, D. C., Zappia, L., Dugas, M., Colomé-Tatché, M., & Theis, F. J.
- Affiliations
- Helmholtz Munich (DE)
- University of Münster (DE)
- University Hospital Heidelberg (DE)
- Corresponding author
- Theis, Fabian J.
- ORCID
- Venue
- Nature Methods
- Published
- 2021-12-23
- Volume
- 19
- Issue
- 1
- Pages
- 41-50
- ISSN
- 1548-7091 (ISSN-L)
- 1548-7105
- PMC
- PMC8748196
- OpenAlex
- W4225598893
- Cited by
- 1,438
- References
- 87
- Citations by year
- 2026: 380 · 2025: 358 · 2024: 304 · 2023: 210 · 2022: 115 · 2021: 53
- FWCI
- 55.88
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Nature Methods (publishedVersion, cc-by)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Site cant be reached (submittedVersion, cc-by)
- Funders
- Silicon Valley Community Foundation (182835)
- Wellcome Trust (108413 /A/15/D)
- Chan Zuckerberg Initiative (182835)
- European Commission (874656, 2020 ‘, HORIZON2020)
- Helmholtz Zentrum München
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Single-cell and spatial transcriptomics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene expression and cancer classification
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Bioinformatics and Genomic Networks
- Keywords
- single-cell data integration, batch effects, benchmarking, single-cell atlas, scANVI, scVI, single-cell ATAC-seq, chromatin accessibility, gene expression, feature space selection, scalability
- MeSH terms
- Animals, Humans, Immune System, Software, Sequence Analysis, RNA, Computational Biology, Benchmarking, Genomics, Databases, Genetic, Mice, Single-Cell Analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Benchmarking atlas-level data integration in single-cell genomics. https://app.cochat.ai/papers/10.1038/s41592-021-01336-8
- The paper
- Luecken, M. D., Büttner, M., Chaichoompu, K., Danese, A., Interlandi, M., Mueller, M. F., Strobl, D. C., Zappia, L., Dugas, M., Colomé-Tatché, M., & Theis, F. J. (2022). Benchmarking atlas-level data integration in single-cell genomics. Nature Methods. https://doi.org/10.1038/s41592-021-01336-8
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.