The DOI matches its Crossref record; 77 of the 77 references it cites were found in a registry or an index.
Missing data and technical variability in single-cell RNA-sequencing experiments
Open accessPublished version of a preprint
Abstract
Until recently, high-throughput gene expression technology, such as RNA-Sequencing (RNA-seq) required hundreds of thousands of cells to produce reliable measurements. Recent technical advances permit genome-wide gene expression measurement at the single-cell level. Single-cell RNA-Seq (scRNA-seq) is the most widely used and numerous publications are based on data produced with this technology. However, RNA-seq and scRNA-seq data are markedly different. In particular, unlike RNA-seq, the majority of reported expression levels in scRNA-seq are zeros, which could be either biologically-driven, genes not expressing RNA at the time of measurement, or technically-driven, genes expressing RNA, but not at a sufficient level to be detected by sequencing technology. Another difference is that the proportion of genes reporting the expression level to be zero varies substantially across single cells compared to RNA-seq samples. However, it remains unclear to what extent this cell-to-cell variation is being driven by technical rather than biological variation. Furthermore, while systematic errors, including batch effects, have been widely reported as a major challenge in high-throughput technologies, these issues have received minimal attention in published studies based on scRNA-seq technology. Here, we use an assessment experiment to examine data from published studies and demonstrate that systematic errors can explain a substantial percentage of observed cell-to-cell expression variability. Specifically, we present evidence that some of these reported zeros are driven by technical variation by demonstrating that scRNA-seq produces more zeros than expected and that this bias is greater for lower expressed genes. In addition, this missing data problem is exacerbated by the fact that this technical variation varies cell-to-cell. Then, we show how this technical cell-to-cell variability can be confused with novel biological results. Finally, we demonstrate and discuss how batch-effects and confounded experiments can intensify the problem.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“This is a serious drawback as the lack of biological replicates has also been shown to be a major cause of false discoveries (Hicks et al, 2018; Squair et al, 2021).”
From Evaluation of genetic demultiplexing of single-cell sequencing data from model species (2023) · cites this paper as Hicks et al, 2018
In the citing paper
“The existence of this natural biological variability coupled with technical variability in single-cell methods poses problems for studies without replicates or those that do not demultiplex pooled datasets (Hicks et al, 2018).”
From Evaluation of genetic demultiplexing of single-cell sequencing data from model species (2023) · cites this paper as Hicks et al, 2018
In the citing paper
“Increasing biological replicates through sample pooling prior to library preparation followed by demultiplexing (i.e., identifying and isolating the sequencing data related to each sample within the pool) to the original biological sample is an effective strategy to overcome these issues as well as reduce batch effects [5–9].”
From Benchmarking of computational demultiplexing methods for single-nucleus RNA sequencing data (2025) · cites this paper as [5]
In the citing paper
“while the now widespread use of unique molecular identifiers (UMI) in scRNA-seq removes technical variation associated with PCR, differences in cell lysis, reverse transcription efficiency, and stochastic molecular sampling during sequencing also contribute significantly, necessitating technical correction [4].”
From Normalization and variance stabilization of single-cell RNA-seq data using regularized negative binomial regression (2019) · cites this paper as [4]
In the citing paper
“These patterns correlate mostly with the RNA findings, except the weak but robust acinar expression around E18.5 which appears underestimated in scRNA sequencing data potentially attributed to technical limitations for weakly expressed genes [50–54].”
From TBX3 is dynamically expressed in pancreatic organogenesis and fine-tunes regeneration (2023) · cites this paper as [51]
4 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Moderated estimation of fold change and dispersion for RNA-seq data with DESeq2
Love, M. I., Huber, W., & Anders, S. (2014) · Genome Biology · doi:10.1186/s13059-014-0550-8 · Cited by 95,106
(2008) · Journal of Machine Learning Research · Cited by 33,051
RSEM: accurate transcript quantification from RNA-Seq data with or without a reference genome
Li, B., & Dewey, C. N. (2011) · BMC Bioinformatics · doi:10.1186/1471-2105-12-323 · Cited by 22,179
Trapnell, C., Williams, B. A., Pertea, G., Mortazavi, A., Kwan, G., van Baren, M. J., Salzberg, S. L., Wold, B. J., & Pachter, L. (2010) · Nature Biotechnology · doi:10.1038/nbt.1621 · Cited by 15,556
Mapping and quantifying mammalian transcriptomes by RNA-Seq
Mortazavi, A., Williams, B. A., McCue, K., Schaeffer, L., & Wold, B. (2008) · Nature Methods · doi:10.1038/nmeth.1226 · Cited by 13,696
Gene Expression Omnibus: NCBI gene expression and hybridization array data repository
Edgar, R. (2002) · Nucleic Acids Research · doi:10.1093/nar/30.1.207 · Cited by 13,288
LIII. On lines and planes of closest fit to systems of points in space
Pearson, K. (1901) · The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science · doi:10.1080/14786440109462720 · Cited by 11,934
Near-optimal probabilistic RNA-seq quantification
Bray, N. L., Pimentel, H., Melsted, P., & Pachter, L. (2016) · Nature Biotechnology · doi:10.1038/nbt.3519 · Cited by 9,723
Massively parallel digital transcriptional profiling of single cells
Zheng, G. X. Y., Terry, J. M., Belgrader, P., Ryvkin, P., Bent, Z. W., Wilson, R., Ziraldo, S. B., Wheeler, T. D., McDermott, G. P., Zhu, J., et al. (2017) · Nature Communications · doi:10.1038/ncomms14049 · Cited by 7,353
Trapnell, C., Cacchiarelli, D., Grimsby, J., Pokharel, P., Li, S., Morse, M., Lennon, N. J., Livak, K. J., Mikkelsen, T. S., & Rinn, J. L. (2014) · Nature Biotechnology · doi:10.1038/nbt.2859 · Cited by 7,314
Cited by
Fast, sensitive and accurate integration of single-cell data with Harmony
Korsunsky, I., Millard, N., Fan, J., Slowikowski, K., Zhang, F., Wei, K., Baglaenko, Y., Brenner, M., Loh, P.-R., & Raychaudhuri, S. (2019) · Nature Methods · doi:10.1038/s41592-019-0619-0 · Cited by 10,488
Hafemeister, C., & Satija, R. (2019) · Genome Biology · doi:10.1186/s13059-019-1874-1 · Cited by 4,479
“…unique molecular identifiers (UMI) in scRNA-seq removes technical variation associated with PCR, differences in cell lysis, reverse transcription efficiency, and stochastic molecular sampling during sequencing also contribute significantly, necessitating technical correction [Hicks et al., 2017].”
BackgroundDetermining cell type abundance and expression from bulk tissues with digital cytometry
Newman, A. M., Steen, C. B., Liu, C. L., Gentles, A. J., Chaudhuri, A. A., Scherer, F., Khodadoust, M. S., Esfahani, M. S., Luca, B. A., Steiner, D., et al. (2019) · Nature Biotechnology · doi:10.1038/s41587-019-0114-2 · Cited by 4,478
Finak, G., McDavid, A., Yajima, M., Deng, J., Gersuk, V., Shalek, A. K., Slichter, C. K., Miller, H. W., McElrath, M. J., Prlic, M., et al. (2015) · Genome Biology · doi:10.1186/s13059-015-0844-5 · Cited by 3,159
Batch effects in single-cell RNA-sequencing data are corrected by matching mutual nearest neighbors
Haghverdi, L., Lun, A. T. L., Morgan, M. D., & Marioni, J. C. (2018) · Nature Biotechnology · doi:10.1038/nbt.4091 · Cited by 2,416
Metadata
- Authors
- Hicks, S. C., Townes, F. W., Teng, M., & Irizarry, R. A.
- Affiliations
- Harvard University (US)
- Corresponding author
- Hicks, Stephanie C
- Venue
- Biostatistics
- Publisher
- Oxford University Press (OUP)
- Published
- 2017-11-06
- Volume
- 19
- Issue
- 4
- Pages
- 562-578
- ISSN
- 1465-4644 (ISSN-L)
- 1468-4357
- PMC
- PMC6215955
- OpenAlex
- W2794891507
- Cited by
- 635
- References
- 77
- Citations by year
- 2026: 63 · 2025: 72 · 2024: 57 · 2023: 64 · 2022: 67 · 2021: 87 · 2020: 65 · 2019: 56 · 2018: 45 · 2017: 36
- FWCI
- 22.56
- Type
- journal article
- Language
- en
- Open access
- green
- doi.org
- Licence
- Locations
- Biostatistics (publishedVersion)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- NIH (GM083084, RR021967/GM103552,, HG005220)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Single-cell and spatial transcriptomics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene expression and cancer classification
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Phylogenetic Studies
- Keywords
- single-cell RNA sequencing, missing data, technical variability, systematic errors, batch effects, confounding, low-expressed genes
- MeSH terms
- Animals, Humans, Sequence Analysis, RNA, Gene Expression Profiling, Single-Cell Analysis, High-Throughput Nucleotide Sequencing, Transcriptome
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Missing data and technical variability in single-cell RNA-sequencing experiments. https://app.cochat.ai/papers/10.1093/biostatistics/kxx053
- The paper
- Hicks, S. C., Townes, F. W., Teng, M., & Irizarry, R. A. (2018). Missing data and technical variability in single-cell RNA-sequencing experiments. Biostatistics. https://doi.org/10.1093/biostatistics/kxx053
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.