Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 53 of the 53 references it cites were found in a registry or an index; 27 of the 47 sources its claims cite were checked for support.

A practical solution to pseudoreplication bias in single-cell studies

Open access
Zimmerman, Kip D., Espeland, Mark A., Langefeld, Carl D. · 2021 · Nature Communications

Published version of a preprint

Abstract

Cells from the same individual share common genetic and environmental backgrounds and are not statistically independent; therefore, they are subsamples or pseudoreplicates. Thus, single-cell data have a hierarchical structure that many current single-cell methods do not address, leading to biased inference, highly inflated type 1 error rates, and reduced robustness and reproducibility. This includes methods that use a batch effect correction for individual as a means of accounting for within-sample correlation. Here, we document this dependence across a range of cell types and show that pseudo-bulk aggregation methods are conservative and underpowered relative to mixed models. To compute differential expression within a specific cell type across treatment groups, we propose applying generalized linear mixed models with a random effect for individual, to properly account for both zero inflation and the correlation structure among measures from cells within an individual. Finally, we provide power estimates across a range of experimental conditions to assist researchers in designing appropriately powered studies.

Overview

CoChat Summary

TL;DR

Single-cell differential expression analyses that treat cells from the same individual as independent suffer pseudoreplication, inflating type 1 error rates and reducing reproducibility. The authors recommend generalized linear mixed models with a random effect for individual, especially the two-part hurdle model as implemented in MAST with random effects, and provide power estimates to guide study design.

Key findings

  • Cells from the same individual are positively correlated across cell types: intra-individual Spearman correlations were always higher than inter-individual correlations, based on n = 40,775 cells from 43 individuals over four independent experiments.
  • In simulations, methods ignoring within-individual correlation had highly inflated type 1 error rates: MAST without random effects ranged from 0.561 to 0.888, and ComBat batch-corrected MAST ranged from 0.602 to 0.856, while MAST with a random effect ranged from 0.041 to 0.069 and Tweedie GLMM from 0.053 to 0.084.
  • Pseudo-bulk aggregation controlled type 1 error but was conservative: pseudo-bulk mean ranged from 0.021 to 0.026 and pseudo-bulk sum from 0.008 to 0.035 across conditions.
  • Power analyses showed increasing the number of independent experimental units is the best way to increase power; after sample sizes exceed 20, there were only marginal gains from sampling more than 100 cells per individual, and pseudo-bulk methods were only slightly underpowered when balanced but less well powered under increasing imbalance.

Methods

  • Literature review: PubMed search in January 2019 for “single-cell differential expression” returned 251 articles; 76 were deemed appropriate, including 10 reviews, 36 methods papers, and 30 implementation papers.
  • Correlation analyses: pairwise intra- and inter-individual Spearman correlations were computed across ten pancreatic cell types and a random sample of cell types, using publicly available datasets GSE81861, GSE72056, E-MTAB-5061, and EGAS00001004082.
  • Simulation and evaluation: a simulation engine reproduced hierarchical single-cell expression structure using TPM parameters from six cell types across three studies; type 1 error was evaluated over 250,000 iterations per condition with 5, 10, 20, 30, or 40 individuals per group and Poisson cell counts with λ = 50, 100, 250, or 500, comparing ten differential expression methods; power was computed over 1,000 simulations per fold-change increment of 0.01 between 1 and 5.

Limitations

  • Stated: the simulation engine was modeled using plate-based data, although droplet-based single-cell data also showed hierarchical correlation structure.
  • Stated: random-effects models are computationally expensive, with run times significantly higher than other tools, and estimating power with more than 1,000 cells per individual is computationally costly; mixed models may fail due to complete separation when only two individuals per treatment group are used, especially with fewer than 50 cells per individual.
  • Stated: inflated type 1 error rates at lower numbers of individuals per group are a consequence of the two-part hurdle model simultaneously testing two hypotheses and overabundance of subsampling with small sample sizes; evident from the text: the evaluation focuses on simulated data and differential expression within a specific cell type across treatment groups, rather than a broad empirical benchmark across all single-cell modalities.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1038/s41467-021-21038-1

Cites

53 of 53 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Fig. 1 Intra-individual correlation. Intra- and inter-individual Spearman’s correlations for gene expression values across ten different pancreatic cell types and a random sample of different cell types. The respective numbers of cells and individuals used for each cell type are listed in the Methods section. Median correlation among a donor’s own cells (intra-individual) is always greater than the mean correlation across individuals (inter-individual). The “random cell types” boxplots represent a random sampling of alpha, beta, and ductal cells. The center line represents the median. The lower and upper box limits represent the 25% and 75% quantiles, respectively. The whiskers extend to the largest observation within the box limit ±1.5 × interquartile range. n = 40,775 cells from 43 different individuals over four independent experiments. More details about each experiment are provided in Methods.
Fig. 1 Intra-individual correlation. Intra- and inter-individual Spearman’s correlations for gene expression values across ten different pancreatic cell types and a random sample of different cell types. The respective numbers of cells and individuals used for each cell type are listed in the Methods section. Median correlation among a donor’s own cells (intra-individual) is always greater than the mean correlation across individuals (inter-individual). The “random cell types” boxplots represent a random sampling of alpha, beta, and ductal cells. The center line represents the median. The lower and upper box limits represent the 25% and 75% quantiles, respectively. The whiskers extend to the largest observation within the box limit ±1.5 × interquartile range. n = 40,775 cells from 43 different individuals over four independent experiments. More details about each experiment are provided in Methods.CC BY · PubMed Central
Fig. 2 Simulation workflow. A gamma [Γ(α, β)] distribution was fit to the global mean transcript-per-million (TPM) value of each gene and used to obtain a grand mean, μi The variance of the individual-specific means (inter-individual variance) was modeled as a linear function of the grand mean, f1(μi). Using a normal N(μ, σ2) distribution with an expected value of zero and a variance computed by the linear relationship, f1(μi), a difference in means was drawn for each individual in the simulation. This difference was summed with the grand mean to obtain an individual mean, μij. Within-sample dispersion was simulated as a logarithmic function of the inter-individual mean, f2(μij). A Poisson (λ) distribution with a λ equal to the expected number of cells desired for each individual was then used to obtain the count of cells per individual. The probability of dropout was estimated as a gamma distribution. For each cell assigned to an individual, a count, Yijk, was drawn from a negative binomial distribution.
Fig. 2 Simulation workflow. A gamma [Γ(α, β)] distribution was fit to the global mean transcript-per-million (TPM) value of each gene and used to obtain a grand mean, μi The variance of the individual-specific means (inter-individual variance) was modeled as a linear function of the grand mean, f1(μi). Using a normal N(μ, σ2) distribution with an expected value of zero and a variance computed by the linear relationship, f1(μi), a difference in means was drawn for each individual in the simulation. This difference was summed with the grand mean to obtain an individual mean, μij. Within-sample dispersion was simulated as a logarithmic function of the inter-individual mean, f2(μij). A Poisson (λ) distribution with a λ equal to the expected number of cells desired for each individual was then used to obtain the count of cells per individual. The probability of dropout was estimated as a gamma distribution. For each cell assigned to an individual, a count, Yijk, was drawn from a negative binomial distribution.CC BY · PubMed Central
Fig. 3 Power calculations using MAST with a random effect for the individual. Power curves for various, but likely, single-cell scenarios using MAST with a random effect for individual. Fold-change is simulated by multiplying the global mean gene expression values by the fold-change value for one group. All power is computed at α = 0.05. a Differences in power when sample sizes range between 5 individuals per group to 100 when the number of cells per individual is held constant at 250. b–d Differences in power when increasing the number of cells per individual (100, 250, 500, 1000) for 10, 20, and 50 individuals per group, respectively. Additional power curves are supplied in the supplementary material.
Fig. 3 Power calculations using MAST with a random effect for the individual. Power curves for various, but likely, single-cell scenarios using MAST with a random effect for individual. Fold-change is simulated by multiplying the global mean gene expression values by the fold-change value for one group. All power is computed at α = 0.05. a Differences in power when sample sizes range between 5 individuals per group to 100 when the number of cells per individual is held constant at 250. b–d Differences in power when increasing the number of cells per individual (100, 250, 500, 1000) for 10, 20, and 50 individuals per group, respectively. Additional power curves are supplied in the supplementary material.CC BY · PubMed Central

Verification

Claims the paper makes

14 Valid · 3 Not valid · 7 Not checked · 61 Own results and interpretations

47 more claims · 4 with their cited sources checked

Create a free account to see all 85 claims and every source verdict.

See all claims — free account
  • Probably not a substantial claim. 15
  • It may state two separate claims as one. 22
  • The quote only partly states this claim. 1

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

1 more claim about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

2 of 2

Citations

Cites

Show all 51 references

Cited by

See all 356

Metadata

Authors
Zimmerman, K. D., Espeland, M. A., & Langefeld, C. D.
Affiliations
  • Wake Forest University Health Sciences (US)
  • Atrium Health Wake Forest Baptist (US)
Corresponding author
  • Zimmerman, Kip D.
  • Langefeld, Carl D.
Venue
Nature Communications
Published
2021-02-02
Volume
12
Issue
1
Pages
738-738
ISSN
2041-1723 (ISSN-L)
OpenAlex
W3126641974
Cited by
356
References
51
Citations by year
2026: 148 · 2025: 46 · 2024: 67 · 2023: 46 · 2022: 30 · 2021: 19
FWCI
11.70
Type
journal article
Language
en
Open access
Licence
Locations
  • Nature Communications (publishedVersion, cc-by)
  • bioRxiv (Cold Spring Harbor Laboratory) (submittedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • U.S. Department of Health & Human Services | National Institutes of Health (U01 NS036695)
  • U.S. Department of Health & Human Services | NIH | National Cancer Institute (P30CA012197)
Topics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Single-cell and spatial transcriptomics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene expression and cancer classification
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
Keywords
pseudoreplication, single-cell data, generalized linear mixed models, differential gene expression, excess zeros, type I error rate, statistical power, research design, batch effect correction, hierarchical data structures
MeSH terms
Computer Simulation, Quality Control, Sequence Analysis, RNA, Transcriptome

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: A practical solution to pseudoreplication bias in single-cell studies. https://app.cochat.ai/papers/10.1038/s41467-021-21038-1
The paper
Zimmerman, K. D., Espeland, M. A., & Langefeld, C. D. (2021). A practical solution to pseudoreplication bias in single-cell studies. Nature Communications. https://doi.org/10.1038/s41467-021-21038-1

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.