Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 79 of the 79 references it cites were found in a registry or an index.

Comparison and evaluation of statistical error models for scRNA-seq

Open access
Choudhary, Saket, Satija, Rahul · 2022 · Genome Biology

Published version of a preprint

Abstract

Background: Heterogeneity in single-cell RNA-seq (scRNA-seq) data is driven by multiple sources, including biological variation in cellular state as well as technical variation introduced during experimental processing. Deconvolving these effects is a key challenge for preprocessing workflows. Recent work has demonstrated the importance and utility of count models for scRNA-seq analysis, but there is a lack of consensus on which statistical distributions and parameter settings are appropriate. Results: Here, we analyze 59 scRNA-seq datasets that span a wide range of technologies, systems, and sequencing depths in order to evaluate the performance of different error models. We find that while a Poisson error model appears appropriate for sparse datasets, we observe clear evidence of overdispersion for genes with sufficient sequencing depth in all biological systems, necessitating the use of a negative binomial model. Moreover, we find that the degree of overdispersion varies widely across datasets, systems, and gene abundances, and argues for a data-driven approach for parameter estimation. Conclusions: Based on these analyses, we provide a set of recommendations for modeling variation in scRNA-seq data, particularly when using generalized linear models or likelihood-based approaches for preprocessing and downstream analysis.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1186/s13059-021-02584-9

Cites

79 of 79 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1 Shallow sequencing masks overdispersion in scRNA-seq data. A Proportion of genes that fail a goodness-of-fit test for a Poisson GLM (see the “Methods” section), as a function of gene abundance, for 59 scRNA-seq datasets. For visual clarity, both the color and diameter of each dot correspond to the fraction of genes that exhibit overdispersion. Y-axis represents non-cumulative gene abundance bins between two consecutive labels (for example, >1 refers to all genes with average abundance >1 UMI and ≤5 UMI). Values are listed in Additional file 1: Table S2. B Relationship between average gene abundance and quantile residual variance, after applying a Poisson GLM (see the “Methods” section). Results are shown for datasets profiling endogenous RNA (’‘technical controls”), a HEK293 cell line (’‘biological controls”), and human PBMC (’heterogeneous’). C In datasets profiling cell lines, the fraction of genes that exhibit overdispersion is correlated with average sequencing depth. D Distribution of molecular counts for highly expressed genes in the PBMC Smart-seq3 dataset after downsampling to two different sequencing depths. The expected density assuming a Poisson distribution is shown in red. E Same as (B) but after downsampling the PBMC Smart-seq3 dataset to five different sequencing depths
Fig. 1 Shallow sequencing masks overdispersion in scRNA-seq data. A Proportion of genes that fail a goodness-of-fit test for a Poisson GLM (see the “Methods” section), as a function of gene abundance, for 59 scRNA-seq datasets. For visual clarity, both the color and diameter of each dot correspond to the fraction of genes that exhibit overdispersion. Y-axis represents non-cumulative gene abundance bins between two consecutive labels (for example, >1 refers to all genes with average abundance >1 UMI and ≤5 UMI). Values are listed in Additional file 1: Table S2. B Relationship between average gene abundance and quantile residual variance, after applying a Poisson GLM (see the “Methods” section). Results are shown for datasets profiling endogenous RNA (’‘technical controls”), a HEK293 cell line (’‘biological controls”), and human PBMC (’heterogeneous’). C In datasets profiling cell lines, the fraction of genes that exhibit overdispersion is correlated with average sequencing depth. D Distribution of molecular counts for highly expressed genes in the PBMC Smart-seq3 dataset after downsampling to two different sequencing depths. The expected density assuming a Poisson distribution is shown in red. E Same as (B) but after downsampling the PBMC Smart-seq3 dataset to five different sequencing depthsCC BY · PubMed Central
Fig. 2 Overdispersion varies across datasets. A, B Distribution of per-gene values for the estimated inverse overdispersion θNB of a NB GLM across a range of cell lines (A) and heterogeneous datasets (B). We estimated parameters only for genes where the variance of counts exceeds the mean. Vertical bar indicates the median of the distribution, which varies substantially across datasets, but is concordant across replicate experiments within the same study. C Relationship between gene mean and the variance of Pearson residuals resulting from an NB GLM with θ=10 or θ=100. Each dot is a gene and the trendline (LOESS) is shown in red. D Same as (C), but shown for additional datasets and for θ=∞ (Poisson). Only trendlines are shown for visual clarity
Fig. 2 Overdispersion varies across datasets. A, B Distribution of per-gene values for the estimated inverse overdispersion θNB of a NB GLM across a range of cell lines (A) and heterogeneous datasets (B). We estimated parameters only for genes where the variance of counts exceeds the mean. Vertical bar indicates the median of the distribution, which varies substantially across datasets, but is concordant across replicate experiments within the same study. C Relationship between gene mean and the variance of Pearson residuals resulting from an NB GLM with θ=10 or θ=100. Each dot is a gene and the trendline (LOESS) is shown in red. D Same as (C), but shown for additional datasets and for θ=∞ (Poisson). Only trendlines are shown for visual clarityCC BY · PubMed Central
Fig. 3 A modified regularization procedure improves the robustness of parameter estimates. A Left: Estimated parameter estimates for θ on the Fetal sci-RNA-seq3 dataset [76], using the original regularization procedure from [9] (v1 regularization). Regularized estimates were learned using all cells (purple line), or downsampled cell subsets. Right: Same as (A), but using a modified procedure where the GLM slope was fixed, and genes where σ2≤μ and μ<0.001 were excluded from regularization (v2 regularization) which improves robustness, and enables us to learn parameter estimates from a subsample of 2,000 cells. B Correlation of Pearson residual variance after applying a NB GLM with v2 regularization where parameters were estimated from all 377,456 cells (x-axis), and a subsample of 2000 cells (y-axis). C Green curve: total sctransform run time as a function of dataset size, using all cells to estimate parameters. Orange curve: total runtime when using a subsample of 2000 cells, which increases computational efficiency for large datasets
Fig. 3 A modified regularization procedure improves the robustness of parameter estimates. A Left: Estimated parameter estimates for θ on the Fetal sci-RNA-seq3 dataset [76], using the original regularization procedure from [9] (v1 regularization). Regularized estimates were learned using all cells (purple line), or downsampled cell subsets. Right: Same as (A), but using a modified procedure where the GLM slope was fixed, and genes where σ2≤μ and μ<0.001 were excluded from regularization (v2 regularization) which improves robustness, and enables us to learn parameter estimates from a subsample of 2,000 cells. B Correlation of Pearson residual variance after applying a NB GLM with v2 regularization where parameters were estimated from all 377,456 cells (x-axis), and a subsample of 2000 cells (y-axis). C Green curve: total sctransform run time as a function of dataset size, using all cells to estimate parameters. Orange curve: total runtime when using a subsample of 2000 cells, which increases computational efficiency for large datasetsCC BY · PubMed Central
Fig. 4 Benchmarking on variable feature selection and differential expression analysis. A Proportion overlap (median) of cluster marker genes and identified variable features using fixed θ={100,10}, sctransform v1, and sctransform v2. Marker genes were identified using presto [65], based on unsupervised clustering of log normalized data. Additional plots are shown in Additional file 1: Figure S22. B Comparison of variable features selected by θ={100,10} and our v1 and v2 regularization procedure on a PBMC (ChromiumV3) dataset. The bottom sub-panel represents the top 3000 variable genes identified by four different methods, and groups genes into categories based on the methods where they were identified. Top sub-panel shows the distribution of logarithmic gene mean within each category, with the median value marked in red. Middle sub-panel shows the number of genes within each category, and their overlap with cluster markers. C Benchmarking differential expression analysis. Observed overall true-positive rate (TPR) and false discovery rate (FDR) values for DE genes at FDR cutoffs of 1%, 5%, and 10% using a Wilcoxon rank-sum test (Additional DE methods are indicated in Additional file 1: Figure S23). Dashed vertical lines indicate desired FDRs. Methods that control FDR at their desired level should fall to the left of the corresponding dashed line. Performances were averaged across three simulation replicates. Data was simulated with muscat [79] using three annotated cell types (CD4 T, monocytes, and natural killer cells) from a Smart-seq3 and a Drop-seq PBMC dataset. Titles indicate simulated proportion of DE genes. D Number of differentially expressed genes identified between two groups of biological identical NK cells (PBMC Smart-seq3) where one group was randomly downsampled to 20% sequencing depth. Additional DE methods are indicated in Additional file 1: Figure S24. SCT = sctransform; LogNorm = standard log-normalization
Fig. 4 Benchmarking on variable feature selection and differential expression analysis. A Proportion overlap (median) of cluster marker genes and identified variable features using fixed θ={100,10}, sctransform v1, and sctransform v2. Marker genes were identified using presto [65], based on unsupervised clustering of log normalized data. Additional plots are shown in Additional file 1: Figure S22. B Comparison of variable features selected by θ={100,10} and our v1 and v2 regularization procedure on a PBMC (ChromiumV3) dataset. The bottom sub-panel represents the top 3000 variable genes identified by four different methods, and groups genes into categories based on the methods where they were identified. Top sub-panel shows the distribution of logarithmic gene mean within each category, with the median value marked in red. Middle sub-panel shows the number of genes within each category, and their overlap with cluster markers. C Benchmarking differential expression analysis. Observed overall true-positive rate (TPR) and false discovery rate (FDR) values for DE genes at FDR cutoffs of 1%, 5%, and 10% using a Wilcoxon rank-sum test (Additional DE methods are indicated in Additional file 1: Figure S23). Dashed vertical lines indicate desired FDRs. Methods that control FDR at their desired level should fall to the left of the corresponding dashed line. Performances were averaged across three simulation replicates. Data was simulated with muscat [79] using three annotated cell types (CD4 T, monocytes, and natural killer cells) from a Smart-seq3 and a Drop-seq PBMC dataset. Titles indicate simulated proportion of DE genes. D Number of differentially expressed genes identified between two groups of biological identical NK cells (PBMC Smart-seq3) where one group was randomly downsampled to 20% sequencing depth. Additional DE methods are indicated in Additional file 1: Figure S24. SCT = sctransform; LogNorm = standard log-normalizationCC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 78 references

Research works printed but not linked: 2

  1. Wilcoxon F. Individual comparisons by ranking methods In: Kotz S, Johnson NL, editors. Breakthroughs in Statistics: Methodology and Distribution. New York: Springer: 1992. p. 196–202.
  2. Crowell HL, Soneson C, Germain P-L, Calini D, Collin L, Raposo C, Malhotra D, Robinson MD. Muscat detects subpopulation-specific state transitions from multi-sample multi-condition single-cell transcriptomics data. Nat Commun. 2020; 11(1):1–12. · DOI · Paper page

Cited by

See all 671

Metadata

Authors
Choudhary, S., & Satija, R.
Affiliations
New York Genome Center (US)
Corresponding author
Satija, Rahul
Venue
Genome Biology
Published
2022-01-18
Volume
23
Issue
1
Pages
27-27
ISSN
  • 1465-6906 (ISSN-L)
  • 1474-760X
OpenAlex
W4225343266
Cited by
634
References
78
Citations by year
2026: 168 · 2025: 209 · 2024: 156 · 2023: 75 · 2022: 25
FWCI
38.04
Type
journal article
Language
en
Open access
Licence
Locations
  • Genome biology (publishedVersion, cc-by)
  • bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • chan zuckerberg initiative (EOSS-0000000082)
  • national human genome research institute (RM1HG011014-02, DP2HG009623-01)
  • chan zuckerburg initiative (HCA-A-1704-01895)
  • nih office of the director (1OT2OD026673-01)
Topics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Single-cell and spatial transcriptomics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene expression and cancer classification
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Phylogenetic Studies
Keywords
single-cell RNA sequencing, count data models, overdispersion, negative binomial distribution, Poisson distribution, data preprocessing, generalized linear models, likelihood methods, parameter estimation, gene expression variation
MeSH terms
Likelihood Functions, Sequence Analysis, RNA, Gene Expression Profiling, Workflow, Single-Cell Analysis

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Comparison and evaluation of statistical error models for scRNA-seq. https://app.cochat.ai/papers/10.1186/s13059-021-02584-9
The paper
Choudhary, S., & Satija, R. (2022). Comparison and evaluation of statistical error models for scRNA-seq. Genome Biology. https://doi.org/10.1186/s13059-021-02584-9

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.