CoChat

Batch effects and the effective design of single-cell gene expression studies

Open access

Published version of a preprint

The DOI matches its Crossref record; 57 of the 57 references it cites were found in a registry or an index; 4 of the 18 sources its claims cite were checked for support.

Abstract

Single-cell RNA sequencing (scRNA-seq) can be used to characterize variation in gene expression levels at high resolution. However, the sources of experimental noise in scRNA-seq are not yet well understood. We investigated the technical variation associated with sample processing using the single-cell Fluidigm C1 platform. To do so, we processed three C1 replicates from three human induced pluripotent stem cell (iPSC) lines. We added unique molecular identifiers (UMIs) to all samples, to account for amplification bias. We found that the major source of variation in the gene expression data was driven by genotype, but we also observed substantial variation between the technical replicates. We observed that the conversion of reads to molecules using the UMIs was impacted by both biological and technical variation, indicating that UMI counts are not an unbiased estimator of gene expression levels. Based on our results, we suggest a framework for effective scRNA-seq studies.

Overview

CoChat Summary

TL;DR

Using three human iPSC lines processed across multiple Fluidigm C1 replicates with ERCC spike-ins and UMIs, this study finds that genotype is the largest source of expression variation but that C1 batch effects are substantial and affect both endogenous genes and ERCC controls. The authors recommend balanced mixed-individual C1 designs and at least 75 cells per condition at ≥1.5 million reads per cell, and show that ERCC spike-ins cannot fully correct batch effects.

Key findings

  • Individual/genotype was the major source of variation, but C1 batch effects were also substantial: median variance due to individual was 8% versus 4% due to C1 batch (Kruskal-Wallis, P < 0.001), and cells within a batch were more correlated than cells from the same individual in different batches (P < 0.001).
  • ERCC spike-ins were affected by both technical and biological variation and could not effectively account for C1 batch effects: total ERCC molecule counts differed between C1 batches (F-test, P < 0.001) and across individuals (LRT, P = 0.08); per-gene ERCC variance was larger for individual (median 9%) than batch (median 5%; P < 0.001). Read-to-molecule conversion efficiency showed batch effects for ERCC (P < 0.05) and endogenous genes (P < 0.001), and individual effects for endogenous (P < 0.01) and ERCC (P < 0.01).
  • UMI-based molecule counts were not an unbiased estimator of gene expression: read and molecule counts correlated at r = 0.99 for ERCC spike-ins but r = 0.92 for endogenous genes, with lower correlation for lowly expressed genes.
  • After batch correction, 560 genes had significantly different regulatory noise across individuals (empirical P < 0.0001), enriched for translation, protein disassembly, and biosynthetic processes; among them were pluripotency genes KLF4 and DPPA2. Only 95 of the 1,000 most highly variable genes were common to all three individuals, whereas 887 of the 1,000 most highly expressed genes were common.

Methods

  • scRNA-seq was performed on three Yoruba iPSC lines using the Fluidigm C1 platform, with three independent C1 collections per individual, matching bulk samples, ERCC spike-in controls, and 5-bp random-sequence UMIs. After quality control, 564 high-quality single cells were retained across eight replicates from three individuals (NA19098: 142, NA19101: 201, NA19239: 221).
  • Reads were mapped, filtered, and converted to molecule counts using UMI-tools; analyses included PCA, nested linear mixed models with individual fixed effects and batch random effects, variance component estimation, likelihood ratio tests, F-tests, Kruskal-Wallis tests, and subsampling of cells and reads.
  • Normalization included UMI collision correction, log2 counts-per-million transformation, ERCC-based Poisson GLM correction, and batch-effect removal using a limma-like linear mixed model. Regulatory noise was measured as adjusted coefficient of variation, with differential noise genes identified by median absolute deviation and permutation (300,000 permutations).

Limitations

  • Stated: the batch correction assumes the technical batch effect is identical across individuals with no interaction, and the small number of individuals prevented explicit testing of a first-order batch-by-individual interaction.
  • Stated: ERCC controls do not experience all processing steps of endogenous samples, have shorter polyA tails and are shorter than mammalian mRNAs, only about half of the 92 controls are typically detected, and they are affected by biological as well as technical variation; the authors recommend an alternative control such as Drosophila RNA for human single-cell studies.
  • Evident from the text: the study is a small pilot with only three individuals, one C1 replicate was excluded because of a pipetting error, and the conclusions about study design and noise are based on this limited sample.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1038/srep39921

Cites

57 of 57 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Figures

Figure 1 Experimental design and quality control of scRNA-seq. (a) Three C1 96 well-integrated fluidic circuit (IFC) replicates were collected from each of the three Yoruba individuals. A bulk sample was included in each batch. (b) Summary of the cutoffs used to remove data from low quality cells that might be ruptured or dead (See Supplementary Fig. S1 for details). (c–e) To assess the quality of the scRNA-seq data, the capture efficiency of cells and the faithfulness of mRNA fraction amplification were determined based on the proportion of unmapped reads, the number of detected genes, the numbers of total mapped reads, and the proportion of ERCC spike-in reads across cells. The dash lines indicate the cutoffs summarized in panel (b). The three colors represent the three individuals (NA19098 in red, NA19101 in green, and NA19239 in blue), and the numbers indicate the cell numbers observed in each capture site on C1 plate. (f) Scatterplots in log scale showing the mean read counts and the mean molecule counts of each endogenous gene (grey) and ERCC spike-ins (blue) from the 564 high quality single cell samples before removal of genes with low expression. (g) mRNA capture efficiency shown as observed molecule count versus number of molecules added to each sample, only including the 48 ERCC spike-in controls remaining after removal of genes with low abundance. Each red dot represents the mean +/− SEM of an ERCC spike-in across the 564 high quality single cell samples.
Figure 1 Experimental design and quality control of scRNA-seq. (a) Three C1 96 well-integrated fluidic circuit (IFC) replicates were collected from each of the three Yoruba individuals. A bulk sample was included in each batch. (b) Summary of the cutoffs used to remove data from low quality cells that might be ruptured or dead (See Supplementary Fig. S1 for details). (c–e) To assess the quality of the scRNA-seq data, the capture efficiency of cells and the faithfulness of mRNA fraction amplification were determined based on the proportion of unmapped reads, the number of detected genes, the numbers of total mapped reads, and the proportion of ERCC spike-in reads across cells. The dash lines indicate the cutoffs summarized in panel (b). The three colors represent the three individuals (NA19098 in red, NA19101 in green, and NA19239 in blue), and the numbers indicate the cell numbers observed in each capture site on C1 plate. (f) Scatterplots in log scale showing the mean read counts and the mean molecule counts of each endogenous gene (grey) and ERCC spike-ins (blue) from the 564 high quality single cell samples before removal of genes with low expression. (g) mRNA capture efficiency shown as observed molecule count versus number of molecules added to each sample, only including the 48 ERCC spike-in controls remaining after removal of genes with low abundance. Each red dot represents the mean +/− SEM of an ERCC spike-in across the 564 high quality single cell samples.CC BY · PubMed Central
Figure 2 The effect of sequencing depth and cell number on single cell UMI estimates. Sequencing reads from all the high quality single cells collected for NA19239 were subsampled to the indicated sequencing depth and cell number, and subsequently converted to molecules using the UMIs. Each point represents the mean +/− SEM of 10 random draws of the indicated cell number. The left panel displays the results for 6,097 (50% of detected) genes with lower expression levels and the right panel the results for 6,097 genes with higher expression levels. (a) Pearson correlation of aggregated gene expression level estimates from single cells compared to the bulk sequencing samples. (b) Total number of genes detected with at least one molecule in at least one of the single cells. (c) Pearson correlation of cell-to-cell gene expression variance estimates from subsets of single cells compared to the full single cell data set.
Figure 2 The effect of sequencing depth and cell number on single cell UMI estimates. Sequencing reads from all the high quality single cells collected for NA19239 were subsampled to the indicated sequencing depth and cell number, and subsequently converted to molecules using the UMIs. Each point represents the mean +/− SEM of 10 random draws of the indicated cell number. The left panel displays the results for 6,097 (50% of detected) genes with lower expression levels and the right panel the results for 6,097 genes with higher expression levels. (a) Pearson correlation of aggregated gene expression level estimates from single cells compared to the bulk sequencing samples. (b) Total number of genes detected with at least one molecule in at least one of the single cells. (c) Pearson correlation of cell-to-cell gene expression variance estimates from subsets of single cells compared to the full single cell data set.CC BY · PubMed Central
Figure 3 Batch effect of scRNA-seq data using the C1 platform. (a) Violin plots of the number of total ERCC spike-in molecule-counts in single cell samples per C1 replicate. (b) Scatterplot of the total ERCC molecule-counts and total gene molecule-counts. The colors represent the three individuals (NA19098 is in red, NA19101 in green, and NA19239 in blue). Data from different C1 replicates is plotted in different shapes. (c and d) Violin plots of the reads to molecule conversion efficiency (total molecule-counts divided by total read-counts per single cells) by C1 replicate. The endogenous genes and the ERCC spike-ins are shown separately in (c) and (d), respectively. There is significant difference across individuals of both endogenous genes (P < 0.001) and ERCC spike-ins (P < 0.05). The differences across C1 replicates per individual of endogenous genes and ERCC spike-ins were also evaluated (both P < 0.01).
Figure 3 Batch effect of scRNA-seq data using the C1 platform. (a) Violin plots of the number of total ERCC spike-in molecule-counts in single cell samples per C1 replicate. (b) Scatterplot of the total ERCC molecule-counts and total gene molecule-counts. The colors represent the three individuals (NA19098 is in red, NA19101 in green, and NA19239 in blue). Data from different C1 replicates is plotted in different shapes. (c and d) Violin plots of the reads to molecule conversion efficiency (total molecule-counts divided by total read-counts per single cells) by C1 replicate. The endogenous genes and the ERCC spike-ins are shown separately in (c) and (d), respectively. There is significant difference across individuals of both endogenous genes (P < 0.001) and ERCC spike-ins (P < 0.05). The differences across C1 replicates per individual of endogenous genes and ERCC spike-ins were also evaluated (both P < 0.01).CC BY · PubMed Central
Figure 4 Normalization and removal of technical variability. Principal component (PC) 1 versus PC2 of the (a) raw molecule counts, (b) log2 counts per million (cpm), (c) Poisson transformed expression levels (accounting for technical variability modeled by the ERCC spike-ins), and (d) batch-corrected expression levels. The colors represent the three individuals (NA19098 in red, NA19101 in green, and NA19239 in blue). Data from different C1 replicates is plotted in different shapes.
Figure 4 Normalization and removal of technical variability. Principal component (PC) 1 versus PC2 of the (a) raw molecule counts, (b) log2 counts per million (cpm), (c) Poisson transformed expression levels (accounting for technical variability modeled by the ERCC spike-ins), and (d) batch-corrected expression levels. The colors represent the three individuals (NA19098 in red, NA19101 in green, and NA19239 in blue). Data from different C1 replicates is plotted in different shapes.CC BY · PubMed Central
Figure 5 Cell-to-cell variation in gene expression. Adjusted CV plotted against average molecule counts across all cells in (a) and across only the cells in which the gene is expressed (b), including data from all three individuals. Each dot represents a gene, and the color indicates the corresponding gene-specific dropout rate (the proportion of cells in which the gene is undetected). (c and d) Venn diagrams showing the overlaps of top 1000 genes across individuals based on mean expression level in (c) and based on adjusted CV values in (d), considering only the cells in which the gene is expressed. (e and f ) Similarly, Venn diagrams showing the overlaps of top 1000 genes across individuals based on mean expression level in (e) and based on adjusted CV values in (f ), across all cells.
Figure 5 Cell-to-cell variation in gene expression. Adjusted CV plotted against average molecule counts across all cells in (a) and across only the cells in which the gene is expressed (b), including data from all three individuals. Each dot represents a gene, and the color indicates the corresponding gene-specific dropout rate (the proportion of cells in which the gene is undetected). (c and d) Venn diagrams showing the overlaps of top 1000 genes across individuals based on mean expression level in (c) and based on adjusted CV values in (d), considering only the cells in which the gene is expressed. (e and f ) Similarly, Venn diagrams showing the overlaps of top 1000 genes across individuals based on mean expression level in (e) and based on adjusted CV values in (f ), across all cells.CC BY · PubMed Central

Verification

Claims the paper makes

3 Valid · 0 Not valid · 7 Not checked · 88 Own results and interpretations

70 more claims

Create a free account to see all 98 claims and to verify the 14 references still open.

See all claims — free account
  • It may state two separate claims as one. 10
  • Probably not a substantial claim. 14

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

1 of 1

Citations

Cites

Show all 57 references

Cited by

Items 201–300 of 356

Metadata

Authors
Tung, P.-Y., Blischak, J. D., Hsiao, C. J., Knowles, D. A., Burnett, J. E., Pritchard, J. K., & Gilad, Y.
Affiliations
  • University of Chicago (US)
  • Stanford Medicine (US)
  • Howard Hughes Medical Institute (US)
Corresponding author
Gilad, Yoav
Venue
Scientific Reports
Published
2017-01-03
Volume
7
Issue
1
Pages
39921-39921
ISSN
2045-2322 (ISSN-L)
OpenAlex
W2951638683
Cited by
356
References
55
Citations by year
2026: 23 · 2025: 35 · 2024: 32 · 2023: 22 · 2022: 38 · 2021: 41 · 2020: 52 · 2019: 42 · 2018: 36 · 2017: 31
FWCI
18.09
Type
journal article
Language
en
Open access
Licence
Locations
  • Scientific Reports (publishedVersion, cc-by)
  • bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
Funders
  • Howard Hughes Medical Institute
  • National Institutes of Health
  • National Heart, Lung, and Blood Institute (R01HL092206, T32HL007381)
  • National Institute of General Medical Sciences (T32GM007197)
Topics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Single-cell and spatial transcriptomics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene expression and cancer classification
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Phylogenetic Studies
Keywords
single-cell RNA sequencing, batch effects, technical variability, unique molecular identifiers, amplification bias, gene expression quantification, induced pluripotent stem cells, genotypic variation, research design
MeSH terms
Humans, RNA, Gene Expression, Sequence Analysis, RNA, Principal Component Analysis, Induced Pluripotent Stem Cells, Single-Cell Analysis, High-Throughput Nucleotide Sequencing

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-04, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-04 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.