Comparative Analysis of Single-Cell RNA Sequencing Methods
Open accessPublished version of a preprint
Abstract
Single-cell RNA sequencing (scRNA-seq) offers new possibilities to address biological and medical questions. However, systematic comparisons of the performance of diverse scRNA-seq protocols are lacking. We generated data from 583 mouse embryonic stem cells to evaluate six prominent scRNA-seq methods: CEL-seq2, Drop-seq, MARS-seq, SCRB-seq, Smart-seq, and Smart-seq2. While Smart-seq2 detected the most genes per cell and across cells, CEL-seq2, Drop-seq, MARS-seq, and SCRB-seq quantified mRNA levels with less amplification noise due to the use of unique molecular identifiers (UMIs). Power simulations at different sequencing depths showed that Drop-seq is more cost-efficient for transcriptome quantification of large numbers of cells, while MARS-seq, SCRB-seq, and Smart-seq2 are more efficient when analyzing fewer cells. Our quantitative comparison offers the basis for an informed choice among six prominent scRNA-seq methods, and it provides a framework for benchmarking further improvements of scRNA-seq protocols.
Verified 2026-10-11 · 1 reference not checked yet
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
0 of 30 cited sources checked · 0 supported · 0 flagged
- Findingown data§ Precision of Amplified Genes Is Strongly Increased by UMIs
Amplification noise can be a major factor, as seen by the strong increase of extra Poisson variability when ignoring UMIs and considering read counts only.
“That amplification noise can be a major factor is seen by the strong increase of extra Poisson variability when ignoring UMIs and considering read counts only (Figure 5B, left; Figure S7A).”
Quote in the paperSubstantial claim - Findingown data§ Smart-Seq2 Has the Highest Sensitivity
A 50% ERCC detection probability was reached at approximately 7, 11, 14, 16, 17, and 28 ERCC molecules for Smart-seq2, Smart-seq/C1, CEL-seq2/C1, SCRB-seq, Drop-seq, and MARS-seq, respectively.
“A 50% detection probability was reached at 7, 11, 14, 16, 17, and 28 ERCC molecules for Smart-seq2, Smart-seq/C1, CEL-seq2/C1, SCRB-seq, Drop-seq, and MARS-seq, respectively (Figure S4C).”
Quote in the paperSubstantial claim - Findingown data§ Single-Cell Libraries Are Sequenced to a Reasonable Level of Saturation at One Million Reads For all six methods, >50% of the reads could be unambiguously mapped to the mouse genome (Figure 3A), which is comparable to previous results (Jaitin et al., 2014; Wu et al., 2014). Overall, between 48% (Smart-seq2) and 30% (Smart-seq/C1) of all reads were exonic and, thus, were used to quantify gene expression levels. However, the UMI data showed that only 14%, 5%, 7%, and 15% of the exonic reads were derived from independent mRNA molecules for CEL-seq2/C1, Drop-seq, MARSseq, and SCRB-seq, respectively (Figure 3A). To quantify the relationship between the number of detected genes or mRNA molecules and the number of reads in more detail, we downsampled reads to varying depths, and we estimated to what extent libraries were sequenced to saturation (Figure S2). The number of unique mRNA molecules plateaued at 56,760 UMIs per library for CEL-seq2/C1 and 26,210 UMIs per library for MARS-seq, was still marginally increasing at 17,210 UMIs per library for Drop-seq, and was considerably increasing at 49,980 UMIs per library for SCRB-seq (Figure S2C). Notably, CEL-seq2/C1 and MARS-seq showed a steeper slope at low sequencing depths than both Drop-seq and SCRB-seq, potentially due to a less biased amplification by in vitro transcription. Hence, among the UMI methods, CEL-seq2/C1 and SCRB-seq libraries had the highest complexity of mRNA molecules, and this complexity was sequenced to a reasonable level of saturation with one million reads. To investigate saturation also for non-UMI-based methods, we applied a similar approach at the gene level by counting the number of genes detected by at least one read. By fitting an asymptote to the downsampled data, we estimated that 90% (Drop-seq and SCRB-seq) to 100% (CEL-seq2/C1, MARS-seq, Smart-Seq/C1, and Smart-seq2) of all genes present in a library were detected at one million reads (Figure 3B; Figure S2A). In particular, the deep sequencing of Smart-seq2 libraries showed clearly that the number of detected genes did not change when increasing the sequencing depth from one million to five million reads per cell (Figure S2B).
Among the UMI methods, CEL-seq2/C1 and SCRB-seq libraries had the highest complexity of mRNA molecules.
“Hence, among the UMI methods, CEL-seq2/C1 and SCRB-seq libraries had the highest complexity of mRNA molecules”
Quote in the paperSubstantial claim - Findingown data§ Smart-Seq2 Has the Highest Sensitivity
The 3' counting methods showed a strong bias of reads mapped to the 3' end.
“As expected, the 3 0 counting methods showed a strong bias of reads mapped to the 3 0 end (Figure S3D).”
Quote in the paperNot a substantial claim - Findingown data§ Smart-Seq2 Has the Highest Sensitivity
Smart-seq2 detected the highest number of genes per cell, with a median of 9,138.
“cell (Figure 3C). Smart-seq2 detectedthe highestnumberofgenes per cell with a median of 9,138.”
Quote in the paperSubstantial claim
- Findingown data§ Smart-Seq2 Has the Highest Sensitivity
Smart-seq2 was the most sensitive method in both gene-based and ERCC-based comparisons.
“While Smartseq2 was the most sensitive method in both cases”
Quote in the paperSubstantial claim - Findingown data§ Precision of Amplified Genes Is Strongly Increased by UMIs
For 96.5% (MARS-seq) to 98.9% (Smart-seq2) of the 13,361 genes, the mean-variance relationship could be estimated.
“This was possible for 96.5% (MARS-seq) to 98.9% (Smart-seq2) of all the 13,361 genes.”
Quote in the paperSubstantial claim - Findingown data§ Single-Cell Libraries Are Sequenced to a Reasonable Level of Saturation at One Million Reads For all six methods, >50% of the reads could be unambiguously mapped to the mouse genome (Figure 3A), which is comparable to previous results (Jaitin et al., 2014; Wu et al., 2014). Overall, between 48% (Smart-seq2) and 30% (Smart-seq/C1) of all reads were exonic and, thus, were used to quantify gene expression levels. However, the UMI data showed that only 14%, 5%, 7%, and 15% of the exonic reads were derived from independent mRNA molecules for CEL-seq2/C1, Drop-seq, MARSseq, and SCRB-seq, respectively (Figure 3A). To quantify the relationship between the number of detected genes or mRNA molecules and the number of reads in more detail, we downsampled reads to varying depths, and we estimated to what extent libraries were sequenced to saturation (Figure S2). The number of unique mRNA molecules plateaued at 56,760 UMIs per library for CEL-seq2/C1 and 26,210 UMIs per library for MARS-seq, was still marginally increasing at 17,210 UMIs per library for Drop-seq, and was considerably increasing at 49,980 UMIs per library for SCRB-seq (Figure S2C). Notably, CEL-seq2/C1 and MARS-seq showed a steeper slope at low sequencing depths than both Drop-seq and SCRB-seq, potentially due to a less biased amplification by in vitro transcription. Hence, among the UMI methods, CEL-seq2/C1 and SCRB-seq libraries had the highest complexity of mRNA molecules, and this complexity was sequenced to a reasonable level of saturation with one million reads. To investigate saturation also for non-UMI-based methods, we applied a similar approach at the gene level by counting the number of genes detected by at least one read. By fitting an asymptote to the downsampled data, we estimated that 90% (Drop-seq and SCRB-seq) to 100% (CEL-seq2/C1, MARS-seq, Smart-Seq/C1, and Smart-seq2) of all genes present in a library were detected at one million reads (Figure 3B; Figure S2A). In particular, the deep sequencing of Smart-seq2 libraries showed clearly that the number of detected genes did not change when increasing the sequencing depth from one million to five million reads per cell (Figure S2B).
Between 48% (Smart-seq2) and 30% (Smart-seq/C1) of all reads were exonic.
“Overall, between 48% (Smart-seq2) and 30% (Smart-seq/C1) of all reads were exonic and, thus, were used to quantify gene expression levels.”
Quote in the paperNot a substantial claim - Findingown data§ Single-Cell Libraries Are Sequenced to a Reasonable Level of Saturation at One Million Reads For all six methods, >50% of the reads could be unambiguously mapped to the mouse genome (Figure 3A), which is comparable to previous results (Jaitin et al., 2014; Wu et al., 2014). Overall, between 48% (Smart-seq2) and 30% (Smart-seq/C1) of all reads were exonic and, thus, were used to quantify gene expression levels. However, the UMI data showed that only 14%, 5%, 7%, and 15% of the exonic reads were derived from independent mRNA molecules for CEL-seq2/C1, Drop-seq, MARSseq, and SCRB-seq, respectively (Figure 3A). To quantify the relationship between the number of detected genes or mRNA molecules and the number of reads in more detail, we downsampled reads to varying depths, and we estimated to what extent libraries were sequenced to saturation (Figure S2). The number of unique mRNA molecules plateaued at 56,760 UMIs per library for CEL-seq2/C1 and 26,210 UMIs per library for MARS-seq, was still marginally increasing at 17,210 UMIs per library for Drop-seq, and was considerably increasing at 49,980 UMIs per library for SCRB-seq (Figure S2C). Notably, CEL-seq2/C1 and MARS-seq showed a steeper slope at low sequencing depths than both Drop-seq and SCRB-seq, potentially due to a less biased amplification by in vitro transcription. Hence, among the UMI methods, CEL-seq2/C1 and SCRB-seq libraries had the highest complexity of mRNA molecules, and this complexity was sequenced to a reasonable level of saturation with one million reads. To investigate saturation also for non-UMI-based methods, we applied a similar approach at the gene level by counting the number of genes detected by at least one read. By fitting an asymptote to the downsampled data, we estimated that 90% (Drop-seq and SCRB-seq) to 100% (CEL-seq2/C1, MARS-seq, Smart-Seq/C1, and Smart-seq2) of all genes present in a library were detected at one million reads (Figure 3B; Figure S2A). In particular, the deep sequencing of Smart-seq2 libraries showed clearly that the number of detected genes did not change when increasing the sequencing depth from one million to five million reads per cell (Figure S2B).
Deep sequencing of Smart-seq2 libraries showed no change in the number of detected genes when increasing sequencing depth from one million to five million reads per cell.
“In particular, the deep sequencing of Smart-seq2 libraries showed clearly that the number of detected genes did not change when increasing the sequencing depth from one million to five million reads per cell (Figure S2B).”
Quote in the paperSubstantial claim - Findingown data§ Figure 5. Precision of scRNA-Seq Methods
Accuracy was worse for narrower and especially for lower concentration ranges.
“As expected, accuracy was worse for narrower and especially for lower concentration ranges (Figure S5C).”
Quote in the paperNot a substantial claim - Findingown data§ Single-Cell Libraries Are Sequenced to a Reasonable Level of Saturation at One Million Reads For all six methods, >50% of the reads could be unambiguously mapped to the mouse genome (Figure 3A), which is comparable to previous results (Jaitin et al., 2014; Wu et al., 2014). Overall, between 48% (Smart-seq2) and 30% (Smart-seq/C1) of all reads were exonic and, thus, were used to quantify gene expression levels. However, the UMI data showed that only 14%, 5%, 7%, and 15% of the exonic reads were derived from independent mRNA molecules for CEL-seq2/C1, Drop-seq, MARSseq, and SCRB-seq, respectively (Figure 3A). To quantify the relationship between the number of detected genes or mRNA molecules and the number of reads in more detail, we downsampled reads to varying depths, and we estimated to what extent libraries were sequenced to saturation (Figure S2). The number of unique mRNA molecules plateaued at 56,760 UMIs per library for CEL-seq2/C1 and 26,210 UMIs per library for MARS-seq, was still marginally increasing at 17,210 UMIs per library for Drop-seq, and was considerably increasing at 49,980 UMIs per library for SCRB-seq (Figure S2C). Notably, CEL-seq2/C1 and MARS-seq showed a steeper slope at low sequencing depths than both Drop-seq and SCRB-seq, potentially due to a less biased amplification by in vitro transcription. Hence, among the UMI methods, CEL-seq2/C1 and SCRB-seq libraries had the highest complexity of mRNA molecules, and this complexity was sequenced to a reasonable level of saturation with one million reads. To investigate saturation also for non-UMI-based methods, we applied a similar approach at the gene level by counting the number of genes detected by at least one read. By fitting an asymptote to the downsampled data, we estimated that 90% (Drop-seq and SCRB-seq) to 100% (CEL-seq2/C1, MARS-seq, Smart-Seq/C1, and Smart-seq2) of all genes present in a library were detected at one million reads (Figure 3B; Figure S2A). In particular, the deep sequencing of Smart-seq2 libraries showed clearly that the number of detected genes did not change when increasing the sequencing depth from one million to five million reads per cell (Figure S2B).
For UMI methods, only 14%, 5%, 7%, and 15% of exonic reads were derived from independent mRNA molecules for CEL-seq2/C1, Drop-seq, MARS-seq, and SCRB-seq, respectively.
“However, the UMI data showed that only 14%, 5%, 7%, and 15% of the exonic reads were derived from independent mRNA molecules for CEL-seq2/C1, Drop-seq, MARSseq, and SCRB-seq, respectively (Figure 3A).”
Quote in the paperSubstantial claim - Findingown data§ Single-Cell Libraries Are Sequenced to a Reasonable Level of Saturation at One Million Reads For all six methods, >50% of the reads could be unambiguously mapped to the mouse genome (Figure 3A), which is comparable to previous results (Jaitin et al., 2014; Wu et al., 2014). Overall, between 48% (Smart-seq2) and 30% (Smart-seq/C1) of all reads were exonic and, thus, were used to quantify gene expression levels. However, the UMI data showed that only 14%, 5%, 7%, and 15% of the exonic reads were derived from independent mRNA molecules for CEL-seq2/C1, Drop-seq, MARSseq, and SCRB-seq, respectively (Figure 3A). To quantify the relationship between the number of detected genes or mRNA molecules and the number of reads in more detail, we downsampled reads to varying depths, and we estimated to what extent libraries were sequenced to saturation (Figure S2). The number of unique mRNA molecules plateaued at 56,760 UMIs per library for CEL-seq2/C1 and 26,210 UMIs per library for MARS-seq, was still marginally increasing at 17,210 UMIs per library for Drop-seq, and was considerably increasing at 49,980 UMIs per library for SCRB-seq (Figure S2C). Notably, CEL-seq2/C1 and MARS-seq showed a steeper slope at low sequencing depths than both Drop-seq and SCRB-seq, potentially due to a less biased amplification by in vitro transcription. Hence, among the UMI methods, CEL-seq2/C1 and SCRB-seq libraries had the highest complexity of mRNA molecules, and this complexity was sequenced to a reasonable level of saturation with one million reads. To investigate saturation also for non-UMI-based methods, we applied a similar approach at the gene level by counting the number of genes detected by at least one read. By fitting an asymptote to the downsampled data, we estimated that 90% (Drop-seq and SCRB-seq) to 100% (CEL-seq2/C1, MARS-seq, Smart-Seq/C1, and Smart-seq2) of all genes present in a library were detected at one million reads (Figure 3B; Figure S2A). In particular, the deep sequencing of Smart-seq2 libraries showed clearly that the number of detected genes did not change when increasing the sequencing depth from one million to five million reads per cell (Figure S2B).
An estimated 90% (Drop-seq and SCRB-seq) to 100% (CEL-seq2/C1, MARS-seq, Smart-Seq/C1, and Smart-seq2) of all genes present in a library were detected at one million reads.
“By fitting an asymptote to the downsampled data, we estimated that 90% (Drop-seq and SCRB-seq) to 100% (CEL-seq2/C1, MARS-seq, Smart-Seq/C1, and Smart-seq2) of all genes present in a library were detected at one million reads (Figure 3B; Figure S2A).”
Quote in the paperSubstantial claim - Findingown data§ Precision of Amplified Genes Is Strongly Increased by UMIs
Extra Poisson variability could be calculated for 96.5% (MARS-seq) to 98.9% (Smart-seq2) of the 13,361 genes.
“This was possible for 96.5% (MARS-seq) to 98.9% (Smart-seq2) of all the 13,361 genes.”
Quote in the paperSubstantial claim - Findingown data§ Smart-Seq2 Has the Highest Sensitivity
Sensitivity estimated from the number of detected genes does not fully agree with the comparison based on ERCCs.
“Notably, the sensitivity estimated from the number of detected genes does not fully agree with the comparison based on ERCCs.”
Quote in the paperSubstantial claim - Findingcitation§ Smart-Seq2 Has the Highest Sensitivity
Smart-seq2 showed a more even coverage than Smart-seq.
“Smart-seq2 showed a more even coverage than Smart-seq, confirming previous findings (Picelli et al., 2013).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Picelli et al., 2013 source
Not checked in the free run.
- Findingown data§ PowerIs Determined by a Combination of Dropout Rates and Amplification Noise and Is Highest for SCRB-Seq To estimate the combined impact of sensitivity and precision on the power to detect differential gene expression, we simulated scRNA-seq data given the observed dropout rates and variance for the 13,361 genes. As these depend strongly on the expression level of a gene, it is important to retain the mean variance and mean dropout relationships. To this end, we estimated the mean, the variance (i.e., the dispersion parameter of the negative binomial distribution), and the dropout rate for each gene and method. We then fitted a cubic smoothing spline to the resulting pairs of mean and dispersion estimates to predict the dispersion of a gene given its mean (Figure S8A). Furthermore, we applied a local polynomial regression model to account for the dropout probability given a gene's mean expression (Figure S8B). When simulating data according to these fits, we recovered distributions of dropout rates and variance closely matching the observed data (Figures S8C and S8D). To compare the power for differential gene expression among the methods, we simulated read counts for two groups of n cells and added log-fold changes to 5% of the 13,361 genes in one group. To mimic a biologically realistic scenario, these log-fold changes were drawn from observed differences between microglial subpopulations from a previously published dataset (Zeisel et al., 2015). Simulated datasets were tested for differential expression using limma (Ritchie et al., 2015), and the true positive rate (TPR) and the false discovery rate (FDR) were calculated. Of note, this does include undetected genes, i.e., the 2.5% (SCRB-seq) to 6.8% (MARS-seq) of the 13,361 genes that had fewer than two measurements in a particular method (Figure S6B) and for which we could not estimate the variance. In our simulations, these genes could be drawn as differentially expressed, and in our TPR they were then counted as false negatives for the particular method. Hence, our power simulation framework considers the full range of dropout rates and is not biased against more sensitive methods. First, we analyzed how the number of cells affects TPR and FDR by running 100 simulations each for a range of 16 to 512 cells per group (Figure 6A). FDRs were similar in all methods ranging from 3.9% to 8.7% (Figure S9A). TPRs differed considerably among methods and SCRB-seq performed best, reaching a median TPR of 80% with 64 cells. CEL-seq2/C1, Drop-seq, MARS-seq, and Smart-seq2 performed slightly worse, reaching 80% power with 86, 99, 110, and 95 cells per group, respectively, while Smart-seq/C1 needed 150 cells to reach 80% power (Figure 6A). When disregarding UMIs, Smart-seq2 performed best (Figure 6B), as expected from its low dropout rate and its low amplification noise when considering reads only (Figure 5B). Furthermore, power dropped especially for Drop-seq and SCRB-seq (Figure 6B), as expected from the strong increase in amplification noise of these two methods when considering reads only (Figure 5B). When we stratified our analysis (considering UMIs) across five bins of expression levels, the ranking of methods was recapitulated and showed that the lowest expression bin strongly limited the TPR in all methods (Figure S9B). This ranking also was recapitulated when we analyzed a set of 19 genes previously reported to contain cell-cycle variation in the 2i/LIF culture condition (Kolodziejczyk et al., 2015b). The variance of these cell-cycle genes was clearly higher than the variance of 19 pluripotency and housekeeping (ribosomal) genes in all methods. The p value of that difference was lowest for SCRB-seq, the most powerful method, and highest for Smartseq/C1, the least powerful method (Figure S10D). Notably, this power analysis, as well as the sensitivity, accuracy, and precision parameters analyzed above, includes the variation that is generated in the two technical replicates (batches) per method that we performed (Figure 1). These estimates were very similar among our technical replicates, and, hence, our method comparison is valid with respect to batch variations (Figures S10B-S10D). In addition, as batch effects are known to be highly relevant for interpreting scRNA-seq data (Hicks et al., 2015), we gauged the magnitude of batch effects with respect to identifying differentially expressed genes. To this end, we used limma to identify differentially expressed genes between batches (FDR < 1%), using 25 randomly selected cells per batch and method. All methods had significantly more genes differentially expressed between batches than expected from permutations (zero to four genes), with a median of 119 (Dropseq) to 1,135 (CEL-seq2/C1) differentially expressed genes (Figure S10A). Notably, genes were affected at random across methods, as there was no significant overlap among them (extended hypergeometric test [Kalinka, 2013], p > 0.84). Hence, this analysis once more emphasizes that batches are important to consider in the design of scRNA-seq experiments (Hicks et al., 2015). While a quantitative comparison of the magnitude of batch effects among methods would require substantially more technical replicates per method, the methods differ in their flexibility to incorporate batch effect into the experimental design, which is an important aspect to consider as discussed below. <!-- image -->
The ranking of methods was recapitulated when analyzing a set of 19 genes previously reported to contain cell-cycle variation in the 2i/LIF culture condition.
“This ranking also was recapitulated when we analyzed a set of 19 genes previously reported to contain cell-cycle variation in the 2i/LIF culture condition (Kolodziejczyk et al., 2015b).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Kolodziejczyk et al., 2015b
Not checked in the free run.
- Findingown data§ PowerIs Determined by a Combination of Dropout Rates and Amplification Noise and Is Highest for SCRB-Seq To estimate the combined impact of sensitivity and precision on the power to detect differential gene expression, we simulated scRNA-seq data given the observed dropout rates and variance for the 13,361 genes. As these depend strongly on the expression level of a gene, it is important to retain the mean variance and mean dropout relationships. To this end, we estimated the mean, the variance (i.e., the dispersion parameter of the negative binomial distribution), and the dropout rate for each gene and method. We then fitted a cubic smoothing spline to the resulting pairs of mean and dispersion estimates to predict the dispersion of a gene given its mean (Figure S8A). Furthermore, we applied a local polynomial regression model to account for the dropout probability given a gene's mean expression (Figure S8B). When simulating data according to these fits, we recovered distributions of dropout rates and variance closely matching the observed data (Figures S8C and S8D). To compare the power for differential gene expression among the methods, we simulated read counts for two groups of n cells and added log-fold changes to 5% of the 13,361 genes in one group. To mimic a biologically realistic scenario, these log-fold changes were drawn from observed differences between microglial subpopulations from a previously published dataset (Zeisel et al., 2015). Simulated datasets were tested for differential expression using limma (Ritchie et al., 2015), and the true positive rate (TPR) and the false discovery rate (FDR) were calculated. Of note, this does include undetected genes, i.e., the 2.5% (SCRB-seq) to 6.8% (MARS-seq) of the 13,361 genes that had fewer than two measurements in a particular method (Figure S6B) and for which we could not estimate the variance. In our simulations, these genes could be drawn as differentially expressed, and in our TPR they were then counted as false negatives for the particular method. Hence, our power simulation framework considers the full range of dropout rates and is not biased against more sensitive methods. First, we analyzed how the number of cells affects TPR and FDR by running 100 simulations each for a range of 16 to 512 cells per group (Figure 6A). FDRs were similar in all methods ranging from 3.9% to 8.7% (Figure S9A). TPRs differed considerably among methods and SCRB-seq performed best, reaching a median TPR of 80% with 64 cells. CEL-seq2/C1, Drop-seq, MARS-seq, and Smart-seq2 performed slightly worse, reaching 80% power with 86, 99, 110, and 95 cells per group, respectively, while Smart-seq/C1 needed 150 cells to reach 80% power (Figure 6A). When disregarding UMIs, Smart-seq2 performed best (Figure 6B), as expected from its low dropout rate and its low amplification noise when considering reads only (Figure 5B). Furthermore, power dropped especially for Drop-seq and SCRB-seq (Figure 6B), as expected from the strong increase in amplification noise of these two methods when considering reads only (Figure 5B). When we stratified our analysis (considering UMIs) across five bins of expression levels, the ranking of methods was recapitulated and showed that the lowest expression bin strongly limited the TPR in all methods (Figure S9B). This ranking also was recapitulated when we analyzed a set of 19 genes previously reported to contain cell-cycle variation in the 2i/LIF culture condition (Kolodziejczyk et al., 2015b). The variance of these cell-cycle genes was clearly higher than the variance of 19 pluripotency and housekeeping (ribosomal) genes in all methods. The p value of that difference was lowest for SCRB-seq, the most powerful method, and highest for Smartseq/C1, the least powerful method (Figure S10D). Notably, this power analysis, as well as the sensitivity, accuracy, and precision parameters analyzed above, includes the variation that is generated in the two technical replicates (batches) per method that we performed (Figure 1). These estimates were very similar among our technical replicates, and, hence, our method comparison is valid with respect to batch variations (Figures S10B-S10D). In addition, as batch effects are known to be highly relevant for interpreting scRNA-seq data (Hicks et al., 2015), we gauged the magnitude of batch effects with respect to identifying differentially expressed genes. To this end, we used limma to identify differentially expressed genes between batches (FDR < 1%), using 25 randomly selected cells per batch and method. All methods had significantly more genes differentially expressed between batches than expected from permutations (zero to four genes), with a median of 119 (Dropseq) to 1,135 (CEL-seq2/C1) differentially expressed genes (Figure S10A). Notably, genes were affected at random across methods, as there was no significant overlap among them (extended hypergeometric test [Kalinka, 2013], p > 0.84). Hence, this analysis once more emphasizes that batches are important to consider in the design of scRNA-seq experiments (Hicks et al., 2015). While a quantitative comparison of the magnitude of batch effects among methods would require substantially more technical replicates per method, the methods differ in their flexibility to incorporate batch effect into the experimental design, which is an important aspect to consider as discussed below. <!-- image -->
The variance of the 19 cell-cycle genes was clearly higher than the variance of 19 pluripotency and housekeeping (ribosomal) genes in all methods.
“The variance of these cell-cycle genes was clearly higher than the variance of 19 pluripotency and housekeeping (ribosomal) genes in all methods.”
Quote in the paperNot a substantial claim - Findingown data§ DISCUSSION
Amplification noise and power strongly depend on the use of UMIs, especially for the PCR-based methods.
“Furthermore, amplification noise and power strongly depend on the use of UMIs, especially for the PCRbased methods (Figures 5B and 6B; Figure S7).”
Quote in the paperSubstantial claim - Findingreasoning§ DISCUSSION
For extra Poisson variability, the four UMI methods perform better.
“For the second factor (extra Poisson variability), we find the four UMI methods to perform better (Figure 5B), as expected from their ability to eliminate variation introduced by amplification.”
Quote in the paperSubstantial claim - Findingown data§ DISCUSSION
For dropout probability, Smart-seq2 is the best method.
“For the first factor (dropout probability), we find Smart-seq2 to be the best method (Figure 5A), as expected from its high gene detection sensitivity.”
Quote in the paperSubstantial claim - Findingnone given§ DISCUSSION
Drop-seq, MARS-seq, SCRB-seq, and Smart-seq2 (using in-house transposase) are 2- to 13-fold more cost efficient than CEL-seq2/C1, Smart-seq/C1, and Smart-seq2 (using commercial transposase).
“We find that Drop-seq, MARS-seq, SCRB-seq, and Smartseq2 (using in-house transposase) are 2- to 13-fold more cost efficient than CEL-seq2/C1, Smart-seq/C1, and Smart-seq2 (using commercial transposase).”
Quote in the paperSubstantial claim - Findingown data§ PowerIs Determined by a Combination of Dropout Rates and Amplification Noise and Is Highest for SCRB-Seq To estimate the combined impact of sensitivity and precision on the power to detect differential gene expression, we simulated scRNA-seq data given the observed dropout rates and variance for the 13,361 genes. As these depend strongly on the expression level of a gene, it is important to retain the mean variance and mean dropout relationships. To this end, we estimated the mean, the variance (i.e., the dispersion parameter of the negative binomial distribution), and the dropout rate for each gene and method. We then fitted a cubic smoothing spline to the resulting pairs of mean and dispersion estimates to predict the dispersion of a gene given its mean (Figure S8A). Furthermore, we applied a local polynomial regression model to account for the dropout probability given a gene's mean expression (Figure S8B). When simulating data according to these fits, we recovered distributions of dropout rates and variance closely matching the observed data (Figures S8C and S8D). To compare the power for differential gene expression among the methods, we simulated read counts for two groups of n cells and added log-fold changes to 5% of the 13,361 genes in one group. To mimic a biologically realistic scenario, these log-fold changes were drawn from observed differences between microglial subpopulations from a previously published dataset (Zeisel et al., 2015). Simulated datasets were tested for differential expression using limma (Ritchie et al., 2015), and the true positive rate (TPR) and the false discovery rate (FDR) were calculated. Of note, this does include undetected genes, i.e., the 2.5% (SCRB-seq) to 6.8% (MARS-seq) of the 13,361 genes that had fewer than two measurements in a particular method (Figure S6B) and for which we could not estimate the variance. In our simulations, these genes could be drawn as differentially expressed, and in our TPR they were then counted as false negatives for the particular method. Hence, our power simulation framework considers the full range of dropout rates and is not biased against more sensitive methods. First, we analyzed how the number of cells affects TPR and FDR by running 100 simulations each for a range of 16 to 512 cells per group (Figure 6A). FDRs were similar in all methods ranging from 3.9% to 8.7% (Figure S9A). TPRs differed considerably among methods and SCRB-seq performed best, reaching a median TPR of 80% with 64 cells. CEL-seq2/C1, Drop-seq, MARS-seq, and Smart-seq2 performed slightly worse, reaching 80% power with 86, 99, 110, and 95 cells per group, respectively, while Smart-seq/C1 needed 150 cells to reach 80% power (Figure 6A). When disregarding UMIs, Smart-seq2 performed best (Figure 6B), as expected from its low dropout rate and its low amplification noise when considering reads only (Figure 5B). Furthermore, power dropped especially for Drop-seq and SCRB-seq (Figure 6B), as expected from the strong increase in amplification noise of these two methods when considering reads only (Figure 5B). When we stratified our analysis (considering UMIs) across five bins of expression levels, the ranking of methods was recapitulated and showed that the lowest expression bin strongly limited the TPR in all methods (Figure S9B). This ranking also was recapitulated when we analyzed a set of 19 genes previously reported to contain cell-cycle variation in the 2i/LIF culture condition (Kolodziejczyk et al., 2015b). The variance of these cell-cycle genes was clearly higher than the variance of 19 pluripotency and housekeeping (ribosomal) genes in all methods. The p value of that difference was lowest for SCRB-seq, the most powerful method, and highest for Smartseq/C1, the least powerful method (Figure S10D). Notably, this power analysis, as well as the sensitivity, accuracy, and precision parameters analyzed above, includes the variation that is generated in the two technical replicates (batches) per method that we performed (Figure 1). These estimates were very similar among our technical replicates, and, hence, our method comparison is valid with respect to batch variations (Figures S10B-S10D). In addition, as batch effects are known to be highly relevant for interpreting scRNA-seq data (Hicks et al., 2015), we gauged the magnitude of batch effects with respect to identifying differentially expressed genes. To this end, we used limma to identify differentially expressed genes between batches (FDR < 1%), using 25 randomly selected cells per batch and method. All methods had significantly more genes differentially expressed between batches than expected from permutations (zero to four genes), with a median of 119 (Dropseq) to 1,135 (CEL-seq2/C1) differentially expressed genes (Figure S10A). Notably, genes were affected at random across methods, as there was no significant overlap among them (extended hypergeometric test [Kalinka, 2013], p > 0.84). Hence, this analysis once more emphasizes that batches are important to consider in the design of scRNA-seq experiments (Hicks et al., 2015). While a quantitative comparison of the magnitude of batch effects among methods would require substantially more technical replicates per method, the methods differ in their flexibility to incorporate batch effect into the experimental design, which is an important aspect to consider as discussed below. <!-- image -->
All methods had significantly more genes differentially expressed between batches than expected from permutations, with a median of 119 (Drop-seq) to 1,135 (CEL-seq2/C1) differentially expressed genes.
“All methods had significantly more genes differentially expressed between batches than expected from permutations (zero to four genes), with a median of 119 (Dropseq) to 1,135 (CEL-seq2/C1) differentially expressed genes (Figure S10A).”
Quote in the paperSubstantial claim - Findingown data§ PowerIs Determined by a Combination of Dropout Rates and Amplification Noise and Is Highest for SCRB-Seq To estimate the combined impact of sensitivity and precision on the power to detect differential gene expression, we simulated scRNA-seq data given the observed dropout rates and variance for the 13,361 genes. As these depend strongly on the expression level of a gene, it is important to retain the mean variance and mean dropout relationships. To this end, we estimated the mean, the variance (i.e., the dispersion parameter of the negative binomial distribution), and the dropout rate for each gene and method. We then fitted a cubic smoothing spline to the resulting pairs of mean and dispersion estimates to predict the dispersion of a gene given its mean (Figure S8A). Furthermore, we applied a local polynomial regression model to account for the dropout probability given a gene's mean expression (Figure S8B). When simulating data according to these fits, we recovered distributions of dropout rates and variance closely matching the observed data (Figures S8C and S8D). To compare the power for differential gene expression among the methods, we simulated read counts for two groups of n cells and added log-fold changes to 5% of the 13,361 genes in one group. To mimic a biologically realistic scenario, these log-fold changes were drawn from observed differences between microglial subpopulations from a previously published dataset (Zeisel et al., 2015). Simulated datasets were tested for differential expression using limma (Ritchie et al., 2015), and the true positive rate (TPR) and the false discovery rate (FDR) were calculated. Of note, this does include undetected genes, i.e., the 2.5% (SCRB-seq) to 6.8% (MARS-seq) of the 13,361 genes that had fewer than two measurements in a particular method (Figure S6B) and for which we could not estimate the variance. In our simulations, these genes could be drawn as differentially expressed, and in our TPR they were then counted as false negatives for the particular method. Hence, our power simulation framework considers the full range of dropout rates and is not biased against more sensitive methods. First, we analyzed how the number of cells affects TPR and FDR by running 100 simulations each for a range of 16 to 512 cells per group (Figure 6A). FDRs were similar in all methods ranging from 3.9% to 8.7% (Figure S9A). TPRs differed considerably among methods and SCRB-seq performed best, reaching a median TPR of 80% with 64 cells. CEL-seq2/C1, Drop-seq, MARS-seq, and Smart-seq2 performed slightly worse, reaching 80% power with 86, 99, 110, and 95 cells per group, respectively, while Smart-seq/C1 needed 150 cells to reach 80% power (Figure 6A). When disregarding UMIs, Smart-seq2 performed best (Figure 6B), as expected from its low dropout rate and its low amplification noise when considering reads only (Figure 5B). Furthermore, power dropped especially for Drop-seq and SCRB-seq (Figure 6B), as expected from the strong increase in amplification noise of these two methods when considering reads only (Figure 5B). When we stratified our analysis (considering UMIs) across five bins of expression levels, the ranking of methods was recapitulated and showed that the lowest expression bin strongly limited the TPR in all methods (Figure S9B). This ranking also was recapitulated when we analyzed a set of 19 genes previously reported to contain cell-cycle variation in the 2i/LIF culture condition (Kolodziejczyk et al., 2015b). The variance of these cell-cycle genes was clearly higher than the variance of 19 pluripotency and housekeeping (ribosomal) genes in all methods. The p value of that difference was lowest for SCRB-seq, the most powerful method, and highest for Smartseq/C1, the least powerful method (Figure S10D). Notably, this power analysis, as well as the sensitivity, accuracy, and precision parameters analyzed above, includes the variation that is generated in the two technical replicates (batches) per method that we performed (Figure 1). These estimates were very similar among our technical replicates, and, hence, our method comparison is valid with respect to batch variations (Figures S10B-S10D). In addition, as batch effects are known to be highly relevant for interpreting scRNA-seq data (Hicks et al., 2015), we gauged the magnitude of batch effects with respect to identifying differentially expressed genes. To this end, we used limma to identify differentially expressed genes between batches (FDR < 1%), using 25 randomly selected cells per batch and method. All methods had significantly more genes differentially expressed between batches than expected from permutations (zero to four genes), with a median of 119 (Dropseq) to 1,135 (CEL-seq2/C1) differentially expressed genes (Figure S10A). Notably, genes were affected at random across methods, as there was no significant overlap among them (extended hypergeometric test [Kalinka, 2013], p > 0.84). Hence, this analysis once more emphasizes that batches are important to consider in the design of scRNA-seq experiments (Hicks et al., 2015). While a quantitative comparison of the magnitude of batch effects among methods would require substantially more technical replicates per method, the methods differ in their flexibility to incorporate batch effect into the experimental design, which is an important aspect to consider as discussed below. <!-- image -->
FDRs were similar in all methods, ranging from 3.9% to 8.7%.
“FDRs were similar in all methods ranging from 3.9% to 8.7% (Figure S9A).”
Quote in the paperNot a substantial claim - Findingown data§ PowerIs Determined by a Combination of Dropout Rates and Amplification Noise and Is Highest for SCRB-Seq To estimate the combined impact of sensitivity and precision on the power to detect differential gene expression, we simulated scRNA-seq data given the observed dropout rates and variance for the 13,361 genes. As these depend strongly on the expression level of a gene, it is important to retain the mean variance and mean dropout relationships. To this end, we estimated the mean, the variance (i.e., the dispersion parameter of the negative binomial distribution), and the dropout rate for each gene and method. We then fitted a cubic smoothing spline to the resulting pairs of mean and dispersion estimates to predict the dispersion of a gene given its mean (Figure S8A). Furthermore, we applied a local polynomial regression model to account for the dropout probability given a gene's mean expression (Figure S8B). When simulating data according to these fits, we recovered distributions of dropout rates and variance closely matching the observed data (Figures S8C and S8D). To compare the power for differential gene expression among the methods, we simulated read counts for two groups of n cells and added log-fold changes to 5% of the 13,361 genes in one group. To mimic a biologically realistic scenario, these log-fold changes were drawn from observed differences between microglial subpopulations from a previously published dataset (Zeisel et al., 2015). Simulated datasets were tested for differential expression using limma (Ritchie et al., 2015), and the true positive rate (TPR) and the false discovery rate (FDR) were calculated. Of note, this does include undetected genes, i.e., the 2.5% (SCRB-seq) to 6.8% (MARS-seq) of the 13,361 genes that had fewer than two measurements in a particular method (Figure S6B) and for which we could not estimate the variance. In our simulations, these genes could be drawn as differentially expressed, and in our TPR they were then counted as false negatives for the particular method. Hence, our power simulation framework considers the full range of dropout rates and is not biased against more sensitive methods. First, we analyzed how the number of cells affects TPR and FDR by running 100 simulations each for a range of 16 to 512 cells per group (Figure 6A). FDRs were similar in all methods ranging from 3.9% to 8.7% (Figure S9A). TPRs differed considerably among methods and SCRB-seq performed best, reaching a median TPR of 80% with 64 cells. CEL-seq2/C1, Drop-seq, MARS-seq, and Smart-seq2 performed slightly worse, reaching 80% power with 86, 99, 110, and 95 cells per group, respectively, while Smart-seq/C1 needed 150 cells to reach 80% power (Figure 6A). When disregarding UMIs, Smart-seq2 performed best (Figure 6B), as expected from its low dropout rate and its low amplification noise when considering reads only (Figure 5B). Furthermore, power dropped especially for Drop-seq and SCRB-seq (Figure 6B), as expected from the strong increase in amplification noise of these two methods when considering reads only (Figure 5B). When we stratified our analysis (considering UMIs) across five bins of expression levels, the ranking of methods was recapitulated and showed that the lowest expression bin strongly limited the TPR in all methods (Figure S9B). This ranking also was recapitulated when we analyzed a set of 19 genes previously reported to contain cell-cycle variation in the 2i/LIF culture condition (Kolodziejczyk et al., 2015b). The variance of these cell-cycle genes was clearly higher than the variance of 19 pluripotency and housekeeping (ribosomal) genes in all methods. The p value of that difference was lowest for SCRB-seq, the most powerful method, and highest for Smartseq/C1, the least powerful method (Figure S10D). Notably, this power analysis, as well as the sensitivity, accuracy, and precision parameters analyzed above, includes the variation that is generated in the two technical replicates (batches) per method that we performed (Figure 1). These estimates were very similar among our technical replicates, and, hence, our method comparison is valid with respect to batch variations (Figures S10B-S10D). In addition, as batch effects are known to be highly relevant for interpreting scRNA-seq data (Hicks et al., 2015), we gauged the magnitude of batch effects with respect to identifying differentially expressed genes. To this end, we used limma to identify differentially expressed genes between batches (FDR < 1%), using 25 randomly selected cells per batch and method. All methods had significantly more genes differentially expressed between batches than expected from permutations (zero to four genes), with a median of 119 (Dropseq) to 1,135 (CEL-seq2/C1) differentially expressed genes (Figure S10A). Notably, genes were affected at random across methods, as there was no significant overlap among them (extended hypergeometric test [Kalinka, 2013], p > 0.84). Hence, this analysis once more emphasizes that batches are important to consider in the design of scRNA-seq experiments (Hicks et al., 2015). While a quantitative comparison of the magnitude of batch effects among methods would require substantially more technical replicates per method, the methods differ in their flexibility to incorporate batch effect into the experimental design, which is an important aspect to consider as discussed below. <!-- image -->
Dropout rates and variance depend strongly on the expression level of a gene.
“As these depend strongly on the expression level of a gene, it is important to retain the mean variance and mean dropout relationships.”
Quote in the paperSubstantial claim - Study design and conductown data§ Processing of scRNA-Seq Data
After processing and filtering, 583 high-quality scRNA-seq libraries were available for a fair comparison of the methods.
“In summary, we processed and filtered our data so that we ended up with a total of 583 high-quality scRNA-seq libraries that could be used for a fair comparison of the sensitivity, accuracy, precision, power, and efficiency of the methods.”
Quote in the paperSubstantial claim - Premisecitation§ Precision of Amplified Genes Is Strongly Increased by UMIs
UMIs should remove amplification noise.
“This is expected, as UMIs should remove amplification noise, which has been described previously for CEL-seq (Gr € un et al., 2014).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Gr € un et al., 2014
Not checked in the free run.
- Premisecitation§ INTRODUCTION
The small amount of starting material in scRNA-seq results in substantial technical variation.
“As the amount of starting material is so small, this process results in substantial technical variation (Kolodziejczyk et al., 2015a; Wagner et al., 2016).”
Quote in the paperSubstantial claim0 of 2 sources checked- cites Kolodziejczyk et al., 2015a
Not checked in the free run.
- cites Wagner et al., 2016 source
Not checked in the free run.
- Interpretationown data§ Single-Cell Libraries Are Sequenced to a Reasonable Level of Saturation at One Million Reads For all six methods, >50% of the reads could be unambiguously mapped to the mouse genome (Figure 3A), which is comparable to previous results (Jaitin et al., 2014; Wu et al., 2014). Overall, between 48% (Smart-seq2) and 30% (Smart-seq/C1) of all reads were exonic and, thus, were used to quantify gene expression levels. However, the UMI data showed that only 14%, 5%, 7%, and 15% of the exonic reads were derived from independent mRNA molecules for CEL-seq2/C1, Drop-seq, MARSseq, and SCRB-seq, respectively (Figure 3A). To quantify the relationship between the number of detected genes or mRNA molecules and the number of reads in more detail, we downsampled reads to varying depths, and we estimated to what extent libraries were sequenced to saturation (Figure S2). The number of unique mRNA molecules plateaued at 56,760 UMIs per library for CEL-seq2/C1 and 26,210 UMIs per library for MARS-seq, was still marginally increasing at 17,210 UMIs per library for Drop-seq, and was considerably increasing at 49,980 UMIs per library for SCRB-seq (Figure S2C). Notably, CEL-seq2/C1 and MARS-seq showed a steeper slope at low sequencing depths than both Drop-seq and SCRB-seq, potentially due to a less biased amplification by in vitro transcription. Hence, among the UMI methods, CEL-seq2/C1 and SCRB-seq libraries had the highest complexity of mRNA molecules, and this complexity was sequenced to a reasonable level of saturation with one million reads. To investigate saturation also for non-UMI-based methods, we applied a similar approach at the gene level by counting the number of genes detected by at least one read. By fitting an asymptote to the downsampled data, we estimated that 90% (Drop-seq and SCRB-seq) to 100% (CEL-seq2/C1, MARS-seq, Smart-Seq/C1, and Smart-seq2) of all genes present in a library were detected at one million reads (Figure 3B; Figure S2A). In particular, the deep sequencing of Smart-seq2 libraries showed clearly that the number of detected genes did not change when increasing the sequencing depth from one million to five million reads per cell (Figure S2B).
scRNA-seq libraries were sequenced to a reasonable level of saturation at one million reads.
“All in all, these analyses show that scRNA-seq libraries were sequenced to a reasonable level of saturation at one million reads, a cutoff that also has been suggested previously for scRNA-seq datasets (Wu et al., 2014).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Wu et al., 2014
Not checked in the free run.
- Premisecitation§ Smart-Seq2 Has the Highest Sensitivity
ERCCs can be problematic for modeling endogenous mRNAs, potentially due to their shorter length, shorter poly-A tail, and missing 5' cap.
“It has been noted before that ERCCs can be problematic for modeling endogenous mRNAs (Risso et al., 2014), potentially due to their shorter length, shorter poly-A tail, and their missing 5 0 cap (Gr € un and van Oudenaarden, 2015; Stegle et al., 2015).”
Quote in the paperSubstantial claim0 of 3 sources checked- cites Risso et al., 2014
Not checked in the free run.
- cites Gr € un and van Oudenaarden, 2015
Not checked in the free run.
- cites Stegle et al., 2015 source
Not checked in the free run.
- Interpretationown data§ Precision of Amplified Genes Is Strongly Increased by UMIs
UMIs strongly increase the precision of amplified genes.
“Precision of Amplified Genes Is Strongly Increased by UMIs”
Quote in the paperNot a substantial claim- The authors state this as an assumption: Differences in total variation among methods are due to differences in their technical variation.
- Rests on an assumption: The amount of biological variation is the same in cells analyzed by each of the six methods.
- Interpretationreasoning§ Smart-Seq2 Has the Highest Sensitivity
Endogenous mRNAs are likely a more reliable estimate for comparing sensitivities than ERCCs, because the same cell type was used for all methods.
“While ERCCs are still useful to gauge the absolute range of sensitivities, the thousands of endogenous mRNAs are likely to be a more reliable estimate for comparing sensitivities as we used the same cell type for all methods.”
Quote in the paperSubstantial claim - Interpretationown data§ Single-Cell Libraries Are Sequenced to a Reasonable Level of Saturation at One Million Reads For all six methods, >50% of the reads could be unambiguously mapped to the mouse genome (Figure 3A), which is comparable to previous results (Jaitin et al., 2014; Wu et al., 2014). Overall, between 48% (Smart-seq2) and 30% (Smart-seq/C1) of all reads were exonic and, thus, were used to quantify gene expression levels. However, the UMI data showed that only 14%, 5%, 7%, and 15% of the exonic reads were derived from independent mRNA molecules for CEL-seq2/C1, Drop-seq, MARSseq, and SCRB-seq, respectively (Figure 3A). To quantify the relationship between the number of detected genes or mRNA molecules and the number of reads in more detail, we downsampled reads to varying depths, and we estimated to what extent libraries were sequenced to saturation (Figure S2). The number of unique mRNA molecules plateaued at 56,760 UMIs per library for CEL-seq2/C1 and 26,210 UMIs per library for MARS-seq, was still marginally increasing at 17,210 UMIs per library for Drop-seq, and was considerably increasing at 49,980 UMIs per library for SCRB-seq (Figure S2C). Notably, CEL-seq2/C1 and MARS-seq showed a steeper slope at low sequencing depths than both Drop-seq and SCRB-seq, potentially due to a less biased amplification by in vitro transcription. Hence, among the UMI methods, CEL-seq2/C1 and SCRB-seq libraries had the highest complexity of mRNA molecules, and this complexity was sequenced to a reasonable level of saturation with one million reads. To investigate saturation also for non-UMI-based methods, we applied a similar approach at the gene level by counting the number of genes detected by at least one read. By fitting an asymptote to the downsampled data, we estimated that 90% (Drop-seq and SCRB-seq) to 100% (CEL-seq2/C1, MARS-seq, Smart-Seq/C1, and Smart-seq2) of all genes present in a library were detected at one million reads (Figure 3B; Figure S2A). In particular, the deep sequencing of Smart-seq2 libraries showed clearly that the number of detected genes did not change when increasing the sequencing depth from one million to five million reads per cell (Figure S2B).
CEL-seq2/C1 and SCRB-seq complexity was sequenced to a reasonable level of saturation with one million reads.
“and this complexity was sequenced to a reasonable level of saturation with one million reads.”
Quote in the paperSubstantial claim - Study design and conductnone given§ INTRODUCTION
Data processing was identical across the six scRNA-seq methods.
“Data processing was identical across methods, and the given cell numbers per method and replicate were used to compare sensitivity, accuracy, precision, power, and cost efficiency.”
Quote in the paperSubstantial claim - Study design and conductown data§ PowerIs Determined by a Combination of Dropout Rates and Amplification Noise and Is Highest for SCRB-Seq To estimate the combined impact of sensitivity and precision on the power to detect differential gene expression, we simulated scRNA-seq data given the observed dropout rates and variance for the 13,361 genes. As these depend strongly on the expression level of a gene, it is important to retain the mean variance and mean dropout relationships. To this end, we estimated the mean, the variance (i.e., the dispersion parameter of the negative binomial distribution), and the dropout rate for each gene and method. We then fitted a cubic smoothing spline to the resulting pairs of mean and dispersion estimates to predict the dispersion of a gene given its mean (Figure S8A). Furthermore, we applied a local polynomial regression model to account for the dropout probability given a gene's mean expression (Figure S8B). When simulating data according to these fits, we recovered distributions of dropout rates and variance closely matching the observed data (Figures S8C and S8D). To compare the power for differential gene expression among the methods, we simulated read counts for two groups of n cells and added log-fold changes to 5% of the 13,361 genes in one group. To mimic a biologically realistic scenario, these log-fold changes were drawn from observed differences between microglial subpopulations from a previously published dataset (Zeisel et al., 2015). Simulated datasets were tested for differential expression using limma (Ritchie et al., 2015), and the true positive rate (TPR) and the false discovery rate (FDR) were calculated. Of note, this does include undetected genes, i.e., the 2.5% (SCRB-seq) to 6.8% (MARS-seq) of the 13,361 genes that had fewer than two measurements in a particular method (Figure S6B) and for which we could not estimate the variance. In our simulations, these genes could be drawn as differentially expressed, and in our TPR they were then counted as false negatives for the particular method. Hence, our power simulation framework considers the full range of dropout rates and is not biased against more sensitive methods. First, we analyzed how the number of cells affects TPR and FDR by running 100 simulations each for a range of 16 to 512 cells per group (Figure 6A). FDRs were similar in all methods ranging from 3.9% to 8.7% (Figure S9A). TPRs differed considerably among methods and SCRB-seq performed best, reaching a median TPR of 80% with 64 cells. CEL-seq2/C1, Drop-seq, MARS-seq, and Smart-seq2 performed slightly worse, reaching 80% power with 86, 99, 110, and 95 cells per group, respectively, while Smart-seq/C1 needed 150 cells to reach 80% power (Figure 6A). When disregarding UMIs, Smart-seq2 performed best (Figure 6B), as expected from its low dropout rate and its low amplification noise when considering reads only (Figure 5B). Furthermore, power dropped especially for Drop-seq and SCRB-seq (Figure 6B), as expected from the strong increase in amplification noise of these two methods when considering reads only (Figure 5B). When we stratified our analysis (considering UMIs) across five bins of expression levels, the ranking of methods was recapitulated and showed that the lowest expression bin strongly limited the TPR in all methods (Figure S9B). This ranking also was recapitulated when we analyzed a set of 19 genes previously reported to contain cell-cycle variation in the 2i/LIF culture condition (Kolodziejczyk et al., 2015b). The variance of these cell-cycle genes was clearly higher than the variance of 19 pluripotency and housekeeping (ribosomal) genes in all methods. The p value of that difference was lowest for SCRB-seq, the most powerful method, and highest for Smartseq/C1, the least powerful method (Figure S10D). Notably, this power analysis, as well as the sensitivity, accuracy, and precision parameters analyzed above, includes the variation that is generated in the two technical replicates (batches) per method that we performed (Figure 1). These estimates were very similar among our technical replicates, and, hence, our method comparison is valid with respect to batch variations (Figures S10B-S10D). In addition, as batch effects are known to be highly relevant for interpreting scRNA-seq data (Hicks et al., 2015), we gauged the magnitude of batch effects with respect to identifying differentially expressed genes. To this end, we used limma to identify differentially expressed genes between batches (FDR < 1%), using 25 randomly selected cells per batch and method. All methods had significantly more genes differentially expressed between batches than expected from permutations (zero to four genes), with a median of 119 (Dropseq) to 1,135 (CEL-seq2/C1) differentially expressed genes (Figure S10A). Notably, genes were affected at random across methods, as there was no significant overlap among them (extended hypergeometric test [Kalinka, 2013], p > 0.84). Hence, this analysis once more emphasizes that batches are important to consider in the design of scRNA-seq experiments (Hicks et al., 2015). While a quantitative comparison of the magnitude of batch effects among methods would require substantially more technical replicates per method, the methods differ in their flexibility to incorporate batch effect into the experimental design, which is an important aspect to consider as discussed below. <!-- image -->
When simulating data according to the fitted mean/dispersion and mean/dropout relationships, the recovered distributions of dropout rates and variance closely matched the observed data.
“When simulating data according to these fits, we recovered distributions of dropout rates and variance closely matching the observed data (Figures S8C and S8D).”
Quote in the paperSubstantial claim - Interpretationown data§ PowerIs Determined by a Combination of Dropout Rates and Amplification Noise and Is Highest for SCRB-Seq To estimate the combined impact of sensitivity and precision on the power to detect differential gene expression, we simulated scRNA-seq data given the observed dropout rates and variance for the 13,361 genes. As these depend strongly on the expression level of a gene, it is important to retain the mean variance and mean dropout relationships. To this end, we estimated the mean, the variance (i.e., the dispersion parameter of the negative binomial distribution), and the dropout rate for each gene and method. We then fitted a cubic smoothing spline to the resulting pairs of mean and dispersion estimates to predict the dispersion of a gene given its mean (Figure S8A). Furthermore, we applied a local polynomial regression model to account for the dropout probability given a gene's mean expression (Figure S8B). When simulating data according to these fits, we recovered distributions of dropout rates and variance closely matching the observed data (Figures S8C and S8D). To compare the power for differential gene expression among the methods, we simulated read counts for two groups of n cells and added log-fold changes to 5% of the 13,361 genes in one group. To mimic a biologically realistic scenario, these log-fold changes were drawn from observed differences between microglial subpopulations from a previously published dataset (Zeisel et al., 2015). Simulated datasets were tested for differential expression using limma (Ritchie et al., 2015), and the true positive rate (TPR) and the false discovery rate (FDR) were calculated. Of note, this does include undetected genes, i.e., the 2.5% (SCRB-seq) to 6.8% (MARS-seq) of the 13,361 genes that had fewer than two measurements in a particular method (Figure S6B) and for which we could not estimate the variance. In our simulations, these genes could be drawn as differentially expressed, and in our TPR they were then counted as false negatives for the particular method. Hence, our power simulation framework considers the full range of dropout rates and is not biased against more sensitive methods. First, we analyzed how the number of cells affects TPR and FDR by running 100 simulations each for a range of 16 to 512 cells per group (Figure 6A). FDRs were similar in all methods ranging from 3.9% to 8.7% (Figure S9A). TPRs differed considerably among methods and SCRB-seq performed best, reaching a median TPR of 80% with 64 cells. CEL-seq2/C1, Drop-seq, MARS-seq, and Smart-seq2 performed slightly worse, reaching 80% power with 86, 99, 110, and 95 cells per group, respectively, while Smart-seq/C1 needed 150 cells to reach 80% power (Figure 6A). When disregarding UMIs, Smart-seq2 performed best (Figure 6B), as expected from its low dropout rate and its low amplification noise when considering reads only (Figure 5B). Furthermore, power dropped especially for Drop-seq and SCRB-seq (Figure 6B), as expected from the strong increase in amplification noise of these two methods when considering reads only (Figure 5B). When we stratified our analysis (considering UMIs) across five bins of expression levels, the ranking of methods was recapitulated and showed that the lowest expression bin strongly limited the TPR in all methods (Figure S9B). This ranking also was recapitulated when we analyzed a set of 19 genes previously reported to contain cell-cycle variation in the 2i/LIF culture condition (Kolodziejczyk et al., 2015b). The variance of these cell-cycle genes was clearly higher than the variance of 19 pluripotency and housekeeping (ribosomal) genes in all methods. The p value of that difference was lowest for SCRB-seq, the most powerful method, and highest for Smartseq/C1, the least powerful method (Figure S10D). Notably, this power analysis, as well as the sensitivity, accuracy, and precision parameters analyzed above, includes the variation that is generated in the two technical replicates (batches) per method that we performed (Figure 1). These estimates were very similar among our technical replicates, and, hence, our method comparison is valid with respect to batch variations (Figures S10B-S10D). In addition, as batch effects are known to be highly relevant for interpreting scRNA-seq data (Hicks et al., 2015), we gauged the magnitude of batch effects with respect to identifying differentially expressed genes. To this end, we used limma to identify differentially expressed genes between batches (FDR < 1%), using 25 randomly selected cells per batch and method. All methods had significantly more genes differentially expressed between batches than expected from permutations (zero to four genes), with a median of 119 (Dropseq) to 1,135 (CEL-seq2/C1) differentially expressed genes (Figure S10A). Notably, genes were affected at random across methods, as there was no significant overlap among them (extended hypergeometric test [Kalinka, 2013], p > 0.84). Hence, this analysis once more emphasizes that batches are important to consider in the design of scRNA-seq experiments (Hicks et al., 2015). While a quantitative comparison of the magnitude of batch effects among methods would require substantially more technical replicates per method, the methods differ in their flexibility to incorporate batch effect into the experimental design, which is an important aspect to consider as discussed below. <!-- image -->
Batches are important to consider in the design of scRNA-seq experiments.
“Hence, this analysis once more emphasizes that batches are important to consider in the design of scRNA-seq experiments (Hicks et al., 2015).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Hicks et al., 2015
Not checked in the free run.
- Interpretationnone given§ Cost Efficiency Is Similarly High for Drop-Seq, MARS-Seq, SCRB-Seq, and Smart-Seq2
Smartseq/C1, CEL-seq2/C1, and Drop-seq are less flexible in distributing scRNA-seq libraries across replicates than the other three methods that use PCR plates.
“Smartseq/C1, CEL-seq2/C1, and Drop-seq are less flexible in distributing scRNA-seq libraries across replicates than the other three methods that use PCR plates.”
Quote in the paperNot a substantial claim - Interpretationreasoning§ DISCUSSION
Smart-seq2 is the most suitable method when an annotation of single-cell transcriptomes is the focus.
“Hence, Smart-seq2 is certainly the most suitable method when an annotation of single-cell transcriptomes is the focus.”
Quote in the paperSubstantial claim
- May combine two claims. 32
- The quote only partly states this claim. 2
- Classified as probably not a substantial claim. 12
May combine two claims.
Findingown data§ Smart-Seq2 Has the Highest SensitivityDrop-seq and MARS-seq had the lowest sensitivity, with medians of 4,811 and 4,763 genes detected per cell, respectively.
“Taking the number of detected genes per cell as a measure of sensitivity, we found that Drop-seq and MARS-seq had the lowest sensitivity, with a median of 4,811 and 4,763 genes detected per cell, respectively”
Quote in the paperSubstantial claimMay combine two claims.
Findingown data§ Smart-Seq2 Has the Highest SensitivityA general difference in expression values between 3' counting and full-length methods was reflected in their strong separation by the first principal component, explaining 37% of the total variance.
“A general difference in expression values between 3 0 counting and full-length methods also was reflected in their strong separation by the first principal component, explaining 37% of the total variance”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown data§ Smart-Seq2 Has the Highest SensitivityCEL-seq2/C1, SCRB-seq, and Smart-seq/C1 detected median numbers of 7,536, 7,906, and 7,572 genes per cell, respectively.
“while CEL-seq2/C1, SCRB-seq, and Smartseq/C1 detected a median of 7,536, 7,906, and 7,572 genes per”
Quote in the paperSubstantial claimMay combine two claims.
Findingown data§ Smart-Seq2 Has the Highest SensitivityThe majority of genes (~13,000) were detected by all methods; ~400 genes were specific to each 3' counting method, and ~1,000 genes were specific to each full-length method.
“While the majority of genes ( 13,000) were detected by all methods, 400 genes were specific to each of the 3 0 counting methods, and 1,000 genes were specific to each of the two full-length methods (Figure S3A).”
Quote in the paperSubstantial claimMay combine two claims.
Findingown data§ Smart-Seq2 Has the Highest SensitivityDrop-seq performed better and SCRB-seq and MARS-seq performed worse when using ERCCs.
“Drop-seq performed better and SCRB-seq and MARS-seq performed worse when using ERCCs.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown data§ Smart-Seq2 Has the Highest SensitivityPooling sequence data from 65 cells per method detected approximately 19,000 genes for CEL-seq2/C1, 17,000 for MARS-seq, 18,000 for Drop-seq and SCRB-seq, 20,000 for Smart-seq/C1, and 21,000 for Smart-seq2.
“To compare the total number of genes detected across many cells, we pooled the sequence data of 65 cells per method, and we detected 19,000 genes for CEL-Seq2/C1, 17,000 for MARS-seq, 18,000 for Drop-seq and SCRB-Seq, 20,000 for Smart-seq/C1, and 21,000 for Smart-seq2 (Figure 3D).”
Quote in the paperSubstantial claimMay combine two claims.
Findingown data§ Precision of Amplified Genes Is Strongly Increased by UMIsMARS-seq had the highest median dropout probability (74%) and Smart-seq2 had the lowest (26%).
“As expected from the number of detected genes per cell (Figure 3C), MARS-seq had the highest median dropout probability (74%) and Smart-seq2 had the lowest (26%) (Figure 5A).”
Quote in the paperSubstantial claimMay combine two claims.
Findingown data§ PowerIs Determined by a Combination of Dropout Rates and Amplification Noise and Is Highest for SCRB-Seq To estimate the combined impact of sensitivity and precision on the power to detect differential gene expression, we simulated scRNA-seq data given the observed dropout rates and variance for the 13,361 genes. As these depend strongly on the expression level of a gene, it is important to retain the mean variance and mean dropout relationships. To this end, we estimated the mean, the variance (i.e., the dispersion parameter of the negative binomial distribution), and the dropout rate for each gene and method. We then fitted a cubic smoothing spline to the resulting pairs of mean and dispersion estimates to predict the dispersion of a gene given its mean (Figure S8A). Furthermore, we applied a local polynomial regression model to account for the dropout probability given a gene's mean expression (Figure S8B). When simulating data according to these fits, we recovered distributions of dropout rates and variance closely matching the observed data (Figures S8C and S8D). To compare the power for differential gene expression among the methods, we simulated read counts for two groups of n cells and added log-fold changes to 5% of the 13,361 genes in one group. To mimic a biologically realistic scenario, these log-fold changes were drawn from observed differences between microglial subpopulations from a previously published dataset (Zeisel et al., 2015). Simulated datasets were tested for differential expression using limma (Ritchie et al., 2015), and the true positive rate (TPR) and the false discovery rate (FDR) were calculated. Of note, this does include undetected genes, i.e., the 2.5% (SCRB-seq) to 6.8% (MARS-seq) of the 13,361 genes that had fewer than two measurements in a particular method (Figure S6B) and for which we could not estimate the variance. In our simulations, these genes could be drawn as differentially expressed, and in our TPR they were then counted as false negatives for the particular method. Hence, our power simulation framework considers the full range of dropout rates and is not biased against more sensitive methods. First, we analyzed how the number of cells affects TPR and FDR by running 100 simulations each for a range of 16 to 512 cells per group (Figure 6A). FDRs were similar in all methods ranging from 3.9% to 8.7% (Figure S9A). TPRs differed considerably among methods and SCRB-seq performed best, reaching a median TPR of 80% with 64 cells. CEL-seq2/C1, Drop-seq, MARS-seq, and Smart-seq2 performed slightly worse, reaching 80% power with 86, 99, 110, and 95 cells per group, respectively, while Smart-seq/C1 needed 150 cells to reach 80% power (Figure 6A). When disregarding UMIs, Smart-seq2 performed best (Figure 6B), as expected from its low dropout rate and its low amplification noise when considering reads only (Figure 5B). Furthermore, power dropped especially for Drop-seq and SCRB-seq (Figure 6B), as expected from the strong increase in amplification noise of these two methods when considering reads only (Figure 5B). When we stratified our analysis (considering UMIs) across five bins of expression levels, the ranking of methods was recapitulated and showed that the lowest expression bin strongly limited the TPR in all methods (Figure S9B). This ranking also was recapitulated when we analyzed a set of 19 genes previously reported to contain cell-cycle variation in the 2i/LIF culture condition (Kolodziejczyk et al., 2015b). The variance of these cell-cycle genes was clearly higher than the variance of 19 pluripotency and housekeeping (ribosomal) genes in all methods. The p value of that difference was lowest for SCRB-seq, the most powerful method, and highest for Smartseq/C1, the least powerful method (Figure S10D). Notably, this power analysis, as well as the sensitivity, accuracy, and precision parameters analyzed above, includes the variation that is generated in the two technical replicates (batches) per method that we performed (Figure 1). These estimates were very similar among our technical replicates, and, hence, our method comparison is valid with respect to batch variations (Figures S10B-S10D). In addition, as batch effects are known to be highly relevant for interpreting scRNA-seq data (Hicks et al., 2015), we gauged the magnitude of batch effects with respect to identifying differentially expressed genes. To this end, we used limma to identify differentially expressed genes between batches (FDR < 1%), using 25 randomly selected cells per batch and method. All methods had significantly more genes differentially expressed between batches than expected from permutations (zero to four genes), with a median of 119 (Dropseq) to 1,135 (CEL-seq2/C1) differentially expressed genes (Figure S10A). Notably, genes were affected at random across methods, as there was no significant overlap among them (extended hypergeometric test [Kalinka, 2013], p > 0.84). Hence, this analysis once more emphasizes that batches are important to consider in the design of scRNA-seq experiments (Hicks et al., 2015). While a quantitative comparison of the magnitude of batch effects among methods would require substantially more technical replicates per method, the methods differ in their flexibility to incorporate batch effect into the experimental design, which is an important aspect to consider as discussed below. <!-- image -->When stratified across five bins of expression levels, the ranking of methods was recapitulated, and the lowest expression bin strongly limited TPR in all methods.
“When we stratified our analysis (considering UMIs) across five bins of expression levels, the ranking of methods was recapitulated and showed that the lowest expression bin strongly limited the TPR in all methods (Figure S9B).”
Quote in the paperSubstantial claimMay combine two claims.
Findingown data§ DISCUSSIONDrop-seq is the most cost-effective method, closely followed by SCRB-seq, MARS-seq, and Smart-seq2.
“Using minimal cost calculations, we find that Drop-seq is the most cost-effective method, closely followed by SCRB-seq, MARS-seq, and Smart-seq2.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown data§ Cost Efficiency Is Similarly High for Drop-Seq, MARS-Seq, SCRB-Seq, and Smart-Seq2Smart-seq2 and CEL-seq2/C1 needed 1.3-fold more cells at 0.25 million reads than at one million reads to reach 80% power, while SCRB-seq and Drop-seq required 2.6-fold more cells.
“While Smart-seq2 and CEL-seq2/C1 needed just 1.3-fold more cells at 0.25 million reads than at one million reads to reach 80% power, SCRB-seq and Drop-seq required 2.6-fold morecells (Table 1).”
Quote in the paperSubstantial claimMay combine two claims.
Findingreasoning§ Cost Efficiency Is Similarly High for Drop-Seq, MARS-Seq, SCRB-Seq, and Smart-Seq2Drop-seq, SCRB-seq, and MARS-seq are the most cost-effective methods, closely followed by Smart-seq2, if using an in-house-produced transposase.
“In summary, we find that Drop-seq, SCRB-seq, and MARS-seq are the most cost-effective methods, closely followed by Smart-seq2, if using an in-house-produced transposase.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown data§ Cost Efficiency Is Similarly High for Drop-Seq, MARS-Seq, SCRB-Seq, and Smart-Seq2SCRB-seq is the most powerful method at one million reads and half a million reads, but CEL-seq2/C1 is the most powerful method at a sequencing depth of 250,000 reads.
“In summary, SCRB-seq is the most powerful method at one million reads and half a million reads, but CELseq2/C1 is the most powerful method at a sequencing depth of 250,000 reads.”
Quote in the paperSubstantial claimMay combine two claims.
Findingreasoning§ Cost Efficiency Is Similarly High for Drop-Seq, MARS-Seq, SCRB-Seq, and Smart-Seq2Drop-seq ($690) is the most cost-effective method when sequencing 254 cells at a depth of 250,000 reads, and SCRB-seq ($810), MARS-seq ($820), and Smart-seq2 ($1,090) are slightly more expensive at the same performance.
“Calculating minimal costs this way, Drop-seq (690$) is the most cost-effective method when sequencing 254 cells at a depth of 250,000 reads, and SCRB-seq (810$), MARS-seq (820$), and Smart-seq2 (1,090$) are slightly more expensive at the same performance (Table 1).”
Quote in the paperSubstantial claim- The authors state this as an assumption: Sequencing costs are the same for all methods.
May combine two claims.
Findingown data§ PowerIs Determined by a Combination of Dropout Rates and Amplification Noise and Is Highest for SCRB-Seq To estimate the combined impact of sensitivity and precision on the power to detect differential gene expression, we simulated scRNA-seq data given the observed dropout rates and variance for the 13,361 genes. As these depend strongly on the expression level of a gene, it is important to retain the mean variance and mean dropout relationships. To this end, we estimated the mean, the variance (i.e., the dispersion parameter of the negative binomial distribution), and the dropout rate for each gene and method. We then fitted a cubic smoothing spline to the resulting pairs of mean and dispersion estimates to predict the dispersion of a gene given its mean (Figure S8A). Furthermore, we applied a local polynomial regression model to account for the dropout probability given a gene's mean expression (Figure S8B). When simulating data according to these fits, we recovered distributions of dropout rates and variance closely matching the observed data (Figures S8C and S8D). To compare the power for differential gene expression among the methods, we simulated read counts for two groups of n cells and added log-fold changes to 5% of the 13,361 genes in one group. To mimic a biologically realistic scenario, these log-fold changes were drawn from observed differences between microglial subpopulations from a previously published dataset (Zeisel et al., 2015). Simulated datasets were tested for differential expression using limma (Ritchie et al., 2015), and the true positive rate (TPR) and the false discovery rate (FDR) were calculated. Of note, this does include undetected genes, i.e., the 2.5% (SCRB-seq) to 6.8% (MARS-seq) of the 13,361 genes that had fewer than two measurements in a particular method (Figure S6B) and for which we could not estimate the variance. In our simulations, these genes could be drawn as differentially expressed, and in our TPR they were then counted as false negatives for the particular method. Hence, our power simulation framework considers the full range of dropout rates and is not biased against more sensitive methods. First, we analyzed how the number of cells affects TPR and FDR by running 100 simulations each for a range of 16 to 512 cells per group (Figure 6A). FDRs were similar in all methods ranging from 3.9% to 8.7% (Figure S9A). TPRs differed considerably among methods and SCRB-seq performed best, reaching a median TPR of 80% with 64 cells. CEL-seq2/C1, Drop-seq, MARS-seq, and Smart-seq2 performed slightly worse, reaching 80% power with 86, 99, 110, and 95 cells per group, respectively, while Smart-seq/C1 needed 150 cells to reach 80% power (Figure 6A). When disregarding UMIs, Smart-seq2 performed best (Figure 6B), as expected from its low dropout rate and its low amplification noise when considering reads only (Figure 5B). Furthermore, power dropped especially for Drop-seq and SCRB-seq (Figure 6B), as expected from the strong increase in amplification noise of these two methods when considering reads only (Figure 5B). When we stratified our analysis (considering UMIs) across five bins of expression levels, the ranking of methods was recapitulated and showed that the lowest expression bin strongly limited the TPR in all methods (Figure S9B). This ranking also was recapitulated when we analyzed a set of 19 genes previously reported to contain cell-cycle variation in the 2i/LIF culture condition (Kolodziejczyk et al., 2015b). The variance of these cell-cycle genes was clearly higher than the variance of 19 pluripotency and housekeeping (ribosomal) genes in all methods. The p value of that difference was lowest for SCRB-seq, the most powerful method, and highest for Smartseq/C1, the least powerful method (Figure S10D). Notably, this power analysis, as well as the sensitivity, accuracy, and precision parameters analyzed above, includes the variation that is generated in the two technical replicates (batches) per method that we performed (Figure 1). These estimates were very similar among our technical replicates, and, hence, our method comparison is valid with respect to batch variations (Figures S10B-S10D). In addition, as batch effects are known to be highly relevant for interpreting scRNA-seq data (Hicks et al., 2015), we gauged the magnitude of batch effects with respect to identifying differentially expressed genes. To this end, we used limma to identify differentially expressed genes between batches (FDR < 1%), using 25 randomly selected cells per batch and method. All methods had significantly more genes differentially expressed between batches than expected from permutations (zero to four genes), with a median of 119 (Dropseq) to 1,135 (CEL-seq2/C1) differentially expressed genes (Figure S10A). Notably, genes were affected at random across methods, as there was no significant overlap among them (extended hypergeometric test [Kalinka, 2013], p > 0.84). Hence, this analysis once more emphasizes that batches are important to consider in the design of scRNA-seq experiments (Hicks et al., 2015). While a quantitative comparison of the magnitude of batch effects among methods would require substantially more technical replicates per method, the methods differ in their flexibility to incorporate batch effect into the experimental design, which is an important aspect to consider as discussed below. <!-- image -->The p value of the difference in variance between cell-cycle and pluripotency/housekeeping genes was lowest for SCRB-seq and highest for Smartseq/C1.
“The p value of that difference was lowest for SCRB-seq, the most powerful method, and highest for Smartseq/C1, the least powerful method (Figure S10D).”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown data§ Cost Efficiency Is Similarly High for Drop-Seq, MARS-Seq, SCRB-Seq, and Smart-Seq2The decrease in power with reduced sequencing depth was moderate and followed the drop in sensitivity at different sequencing depths.
“Overall, the decrease in power was moderate (Figure 6C; Table 1) and followed the drop in sensitivity at different sequencing depths (Figure 3B).”
Quote in the paperSubstantial claimThe quote only partly states this claim.
Findingown data§ PowerIs Determined by a Combination of Dropout Rates and Amplification Noise and Is Highest for SCRB-Seq To estimate the combined impact of sensitivity and precision on the power to detect differential gene expression, we simulated scRNA-seq data given the observed dropout rates and variance for the 13,361 genes. As these depend strongly on the expression level of a gene, it is important to retain the mean variance and mean dropout relationships. To this end, we estimated the mean, the variance (i.e., the dispersion parameter of the negative binomial distribution), and the dropout rate for each gene and method. We then fitted a cubic smoothing spline to the resulting pairs of mean and dispersion estimates to predict the dispersion of a gene given its mean (Figure S8A). Furthermore, we applied a local polynomial regression model to account for the dropout probability given a gene's mean expression (Figure S8B). When simulating data according to these fits, we recovered distributions of dropout rates and variance closely matching the observed data (Figures S8C and S8D). To compare the power for differential gene expression among the methods, we simulated read counts for two groups of n cells and added log-fold changes to 5% of the 13,361 genes in one group. To mimic a biologically realistic scenario, these log-fold changes were drawn from observed differences between microglial subpopulations from a previously published dataset (Zeisel et al., 2015). Simulated datasets were tested for differential expression using limma (Ritchie et al., 2015), and the true positive rate (TPR) and the false discovery rate (FDR) were calculated. Of note, this does include undetected genes, i.e., the 2.5% (SCRB-seq) to 6.8% (MARS-seq) of the 13,361 genes that had fewer than two measurements in a particular method (Figure S6B) and for which we could not estimate the variance. In our simulations, these genes could be drawn as differentially expressed, and in our TPR they were then counted as false negatives for the particular method. Hence, our power simulation framework considers the full range of dropout rates and is not biased against more sensitive methods. First, we analyzed how the number of cells affects TPR and FDR by running 100 simulations each for a range of 16 to 512 cells per group (Figure 6A). FDRs were similar in all methods ranging from 3.9% to 8.7% (Figure S9A). TPRs differed considerably among methods and SCRB-seq performed best, reaching a median TPR of 80% with 64 cells. CEL-seq2/C1, Drop-seq, MARS-seq, and Smart-seq2 performed slightly worse, reaching 80% power with 86, 99, 110, and 95 cells per group, respectively, while Smart-seq/C1 needed 150 cells to reach 80% power (Figure 6A). When disregarding UMIs, Smart-seq2 performed best (Figure 6B), as expected from its low dropout rate and its low amplification noise when considering reads only (Figure 5B). Furthermore, power dropped especially for Drop-seq and SCRB-seq (Figure 6B), as expected from the strong increase in amplification noise of these two methods when considering reads only (Figure 5B). When we stratified our analysis (considering UMIs) across five bins of expression levels, the ranking of methods was recapitulated and showed that the lowest expression bin strongly limited the TPR in all methods (Figure S9B). This ranking also was recapitulated when we analyzed a set of 19 genes previously reported to contain cell-cycle variation in the 2i/LIF culture condition (Kolodziejczyk et al., 2015b). The variance of these cell-cycle genes was clearly higher than the variance of 19 pluripotency and housekeeping (ribosomal) genes in all methods. The p value of that difference was lowest for SCRB-seq, the most powerful method, and highest for Smartseq/C1, the least powerful method (Figure S10D). Notably, this power analysis, as well as the sensitivity, accuracy, and precision parameters analyzed above, includes the variation that is generated in the two technical replicates (batches) per method that we performed (Figure 1). These estimates were very similar among our technical replicates, and, hence, our method comparison is valid with respect to batch variations (Figures S10B-S10D). In addition, as batch effects are known to be highly relevant for interpreting scRNA-seq data (Hicks et al., 2015), we gauged the magnitude of batch effects with respect to identifying differentially expressed genes. To this end, we used limma to identify differentially expressed genes between batches (FDR < 1%), using 25 randomly selected cells per batch and method. All methods had significantly more genes differentially expressed between batches than expected from permutations (zero to four genes), with a median of 119 (Dropseq) to 1,135 (CEL-seq2/C1) differentially expressed genes (Figure S10A). Notably, genes were affected at random across methods, as there was no significant overlap among them (extended hypergeometric test [Kalinka, 2013], p > 0.84). Hence, this analysis once more emphasizes that batches are important to consider in the design of scRNA-seq experiments (Hicks et al., 2015). While a quantitative comparison of the magnitude of batch effects among methods would require substantially more technical replicates per method, the methods differ in their flexibility to incorporate batch effect into the experimental design, which is an important aspect to consider as discussed below. <!-- image -->To reach 80% power at one million reads, CEL-seq2/C1, Drop-seq, MARS-seq, Smart-seq2, and Smart-seq/C1 required 86, 99, 110, 95, and 150 cells per group, respectively.
“CEL-seq2/C1, Drop-seq, MARS-seq, and Smart-seq2 performed slightly worse, reaching 80% power with 86, 99, 110, and 95 cells per group, respectively, while Smart-seq/C1 needed 150 cells to reach 80% power (Figure 6A).”
Quote in the paperSubstantial claimMay combine two claims.
Findingown data§ DISCUSSIONSmart-seq2 is the most sensitive method, closely followed by SCRB-seq, Smart-seq/C1, and CEL-seq2/C1, while Drop-seq and MARS-seq detect nearly 50% fewer genes per cell.
“In our comparison, we clearly find that Smart-seq2 is the most sensitive method, closely followed by SCRB-seq, Smart-seq/C1, and CEL-seq2/C1, while Drop-seq and MARSseq detect nearly 50% fewer genes per cell (Figures 3B and 3C).”
Quote in the paperSubstantial claimMay combine two claims.
Findingown data§ PowerIs Determined by a Combination of Dropout Rates and Amplification Noise and Is Highest for SCRB-Seq To estimate the combined impact of sensitivity and precision on the power to detect differential gene expression, we simulated scRNA-seq data given the observed dropout rates and variance for the 13,361 genes. As these depend strongly on the expression level of a gene, it is important to retain the mean variance and mean dropout relationships. To this end, we estimated the mean, the variance (i.e., the dispersion parameter of the negative binomial distribution), and the dropout rate for each gene and method. We then fitted a cubic smoothing spline to the resulting pairs of mean and dispersion estimates to predict the dispersion of a gene given its mean (Figure S8A). Furthermore, we applied a local polynomial regression model to account for the dropout probability given a gene's mean expression (Figure S8B). When simulating data according to these fits, we recovered distributions of dropout rates and variance closely matching the observed data (Figures S8C and S8D). To compare the power for differential gene expression among the methods, we simulated read counts for two groups of n cells and added log-fold changes to 5% of the 13,361 genes in one group. To mimic a biologically realistic scenario, these log-fold changes were drawn from observed differences between microglial subpopulations from a previously published dataset (Zeisel et al., 2015). Simulated datasets were tested for differential expression using limma (Ritchie et al., 2015), and the true positive rate (TPR) and the false discovery rate (FDR) were calculated. Of note, this does include undetected genes, i.e., the 2.5% (SCRB-seq) to 6.8% (MARS-seq) of the 13,361 genes that had fewer than two measurements in a particular method (Figure S6B) and for which we could not estimate the variance. In our simulations, these genes could be drawn as differentially expressed, and in our TPR they were then counted as false negatives for the particular method. Hence, our power simulation framework considers the full range of dropout rates and is not biased against more sensitive methods. First, we analyzed how the number of cells affects TPR and FDR by running 100 simulations each for a range of 16 to 512 cells per group (Figure 6A). FDRs were similar in all methods ranging from 3.9% to 8.7% (Figure S9A). TPRs differed considerably among methods and SCRB-seq performed best, reaching a median TPR of 80% with 64 cells. CEL-seq2/C1, Drop-seq, MARS-seq, and Smart-seq2 performed slightly worse, reaching 80% power with 86, 99, 110, and 95 cells per group, respectively, while Smart-seq/C1 needed 150 cells to reach 80% power (Figure 6A). When disregarding UMIs, Smart-seq2 performed best (Figure 6B), as expected from its low dropout rate and its low amplification noise when considering reads only (Figure 5B). Furthermore, power dropped especially for Drop-seq and SCRB-seq (Figure 6B), as expected from the strong increase in amplification noise of these two methods when considering reads only (Figure 5B). When we stratified our analysis (considering UMIs) across five bins of expression levels, the ranking of methods was recapitulated and showed that the lowest expression bin strongly limited the TPR in all methods (Figure S9B). This ranking also was recapitulated when we analyzed a set of 19 genes previously reported to contain cell-cycle variation in the 2i/LIF culture condition (Kolodziejczyk et al., 2015b). The variance of these cell-cycle genes was clearly higher than the variance of 19 pluripotency and housekeeping (ribosomal) genes in all methods. The p value of that difference was lowest for SCRB-seq, the most powerful method, and highest for Smartseq/C1, the least powerful method (Figure S10D). Notably, this power analysis, as well as the sensitivity, accuracy, and precision parameters analyzed above, includes the variation that is generated in the two technical replicates (batches) per method that we performed (Figure 1). These estimates were very similar among our technical replicates, and, hence, our method comparison is valid with respect to batch variations (Figures S10B-S10D). In addition, as batch effects are known to be highly relevant for interpreting scRNA-seq data (Hicks et al., 2015), we gauged the magnitude of batch effects with respect to identifying differentially expressed genes. To this end, we used limma to identify differentially expressed genes between batches (FDR < 1%), using 25 randomly selected cells per batch and method. All methods had significantly more genes differentially expressed between batches than expected from permutations (zero to four genes), with a median of 119 (Dropseq) to 1,135 (CEL-seq2/C1) differentially expressed genes (Figure S10A). Notably, genes were affected at random across methods, as there was no significant overlap among them (extended hypergeometric test [Kalinka, 2013], p > 0.84). Hence, this analysis once more emphasizes that batches are important to consider in the design of scRNA-seq experiments (Hicks et al., 2015). While a quantitative comparison of the magnitude of batch effects among methods would require substantially more technical replicates per method, the methods differ in their flexibility to incorporate batch effect into the experimental design, which is an important aspect to consider as discussed below. <!-- image -->TPRs differed considerably among methods, and SCRB-seq performed best, reaching a median TPR of 80% with 64 cells.
“TPRs differed considerably among methods and SCRB-seq performed best, reaching a median TPR of 80% with 64 cells.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown data§ DISCUSSIONAt a sequencing depth of one million reads, SCRB-seq has the highest power, probably due to a good balance of high sensitivity and low amplification noise.
“Our simulations show that, at a sequencing depth of one million reads, SCRB-seq has the highest power, probably due to a good balance of high sensitivity and low amplification noise.”
Quote in the paperSubstantial claimMay combine two claims.
Premisecitation§ Generation of scRNA-Seq LibrariesmESCs cultured under 2i/LIF conditions are a relatively homogeneous cell population, so biological variation was similar among experiments.
“We used mESCs cultured under two inhibitor/leukemia inhibitory factor (2i/LIF) conditions to obtain a relatively homogeneous cell population (Gr € un et al., 2014; Kolodziejczyk et al., 2015b), so that biological variation was similar among experiments and, hence, we mainly compared technical variation.”
Quote in the paperSubstantial claim0 of 2 sources checked- cites Gr € un et al., 2014
Not checked in the free run.
- cites Kolodziejczyk et al., 2015b
Not checked in the free run.
May combine two claims.
Premisecitation§ Precision of Amplified Genes Is Strongly Increased by UMIsExtra Poisson variability includes biological variation and technical variation, and technical variation includes noise introduced by amplification.
“This extra Poisson variability includes biological variation (assumed to be the same across methods in our data) and technical variation, and the latter includes noise introduced by amplification (Brennecke et al., 2013; Gr € un et al., 2014; Stegle et al., 2015).”
Quote in the paperSubstantial claim0 of 3 sources checkedProbably not a substantial claim: may not pass S3 (load-bearing).
Premisecitation§ Precision of Amplified Genes Is Strongly Increased by UMIsA mixture model including a dropout probability and a negative binomial distribution represents scRNA-seq data better than the negative binomial alone.
“Indeed, a mixture model including a dropout probability and a negative binomial distribution, modeling the overdispersion in the count data, have been shown to represent scRNA-seq data better than the negative binomial alone (Finak et al., 2015; Kharchenko et al., 2014).”
Quote in the paperNot a substantial claim0 of 2 sources checkedMay combine two claims.
Interpretationreasoning§ Smart-Seq2 Has the Highest SensitivityFull-length methods turn a slightly higher fraction of transcripts into sequenceable molecules than 3' counting methods and are more sensitive in this respect.
“Hence, it seems that full-length methods turn a slightly higher fraction of transcripts into sequenceable molecules than 3 0 counting methods and are more sensitive in this respect.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationown data§ Smart-Seq2 Has the Highest SensitivityAmong the 3' counting methods, CEL-seq2/C1 and SCRB-seq detect about as many genes per cell as Smart-seq/C1, whereas Drop-seq and MARS-seq detect considerably fewer genes.
“Among the 3 0 counting methods, CEL-seq2/C1 and SCRB-seq detect about as many genes per cell as Smart-seq/C1, whereas Drop-seq and MARS-seq detect considerably fewer genes.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationcitation§ Smart-Seq2 Has the Highest SensitivityMethod-specific genes are unlikely to remain method specific at higher expression levels, and their impact on conclusions drawn from scRNA-seq data is rather limited.
“This suggests that they are unlikely to remain method specific at higher expression levels and that their impact on conclusions drawn from scRNAseq data is rather limited (Lun et al., 2016).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Lun et al., 2016 source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Premisecitation§ INTRODUCTIONThe Smart-seq protocol has been optimized for sensitivity, more even full-length coverage, accuracy, and cost.
“Furthermore, the Smart-seq protocol has been optimized for sensitivity, more even full-length coverage, accuracy, and cost (Picelli et al., 2013)”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Picelli et al., 2013 source
Not checked in the free run.
May combine two claims.
Interpretationown data§ Smart-Seq2 Has the Highest SensitivitySmart-seq2 is the most sensitive method, detecting the highest number of genes per cell and the most genes in total across cells and having the most even coverage across transcripts.
“In summary, we find that Smart-seq2 is the most sensitive method, as it detects the highest number of genes per cell and the most genes in total across cells and has the most even coverage across transcripts.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Premisecitation§ INTRODUCTIONUMI information improves quantification of mRNA molecules.
“Utilization of UMI information improves quantification of mRNA molecules (Gr € un et al., 2014; Islam et al., 2014)”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites Gr € un et al., 2014
Not checked in the free run.
- cites Islam et al., 2014 source
Not checked in the free run.
May combine two claims.
Interpretationown data§ Smart-Seq2 Has the Highest SensitivitySmart-seq/C1 is slightly less sensitive per cell and detects almost the same number of genes across cells with slightly less even coverage.
“Smart-seq/C1 is slightly less sensitive per cell and detects almost the same number of genes across cells with slightly less even coverage.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Premisecitation§ INTRODUCTIONOther protocols have sacrificed full-length coverage to sequence part of the cDNA primer, enabling early cell-specific barcoding and increasing scRNA-seq library generation throughput by one to three orders of magnitude.
“Other protocols have sacrificed full-length coverage in order to sequence part of the primer used for cDNA generation. This enables early barcoding of libraries (i.e., the incorporation of cell-specific barcodes), allowing for multiplexing the cDNA amplification and thereby increasing the throughput of scRNAseq library generation by one to three orders of magnitude (Hashimshony et al., 2012; Jaiti”
Quote in the paperNot a substantial claim0 of 3 sources checkedProbably not a substantial claim: may not pass S3 (load-bearing).
Premisecitation§ INTRODUCTIONThe improved Smart-seq2 protocol has become widely used.
“and this improved Smart-seq2 protocol (Picelli et al., 2014b) has also become widely used (Gokce et al., 2016; Reinius et al., 2016; Tirosh et al., 2016).”
Quote in the paperNot a substantial claim0 of 3 sources checkedProbably not a substantial claim: may not pass S3 (load-bearing).
Premisecitation§ INTRODUCTIONscRNA-seq conducted in the small volumes of the Fluidigm C1 platform outperforms CEL-seq2, Smart-seq, or other commercially available kits in microliter volumes.
“For example, it has previously been shown that scRNA-seq conducted in the small volumes available in the automated microfluidic platform from Fluidigm (C1 platform) outperforms CEL-seq2, Smart-seq, or other commercially available kits in microliter volumes (Hashimshony et al., 2016; Wu et al., 2014).”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites Hashimshony et al., 2016 source
Not checked in the free run.
- cites Wu et al., 2014
Not checked in the free run.
May combine two claims.
Study design and conductown data§ DISCUSSIONThe study generated data for all six compared methods from the same cells, cultured under the same condition in the same laboratory.
“To this end, we generated data for all six compared methods from the same cells, cultured under the same condition in the same laboratory.”
Quote in the paperSubstantial claimMay combine two claims.
Study design and conductreasoning§ PowerIs Determined by a Combination of Dropout Rates and Amplification Noise and Is Highest for SCRB-Seq To estimate the combined impact of sensitivity and precision on the power to detect differential gene expression, we simulated scRNA-seq data given the observed dropout rates and variance for the 13,361 genes. As these depend strongly on the expression level of a gene, it is important to retain the mean variance and mean dropout relationships. To this end, we estimated the mean, the variance (i.e., the dispersion parameter of the negative binomial distribution), and the dropout rate for each gene and method. We then fitted a cubic smoothing spline to the resulting pairs of mean and dispersion estimates to predict the dispersion of a gene given its mean (Figure S8A). Furthermore, we applied a local polynomial regression model to account for the dropout probability given a gene's mean expression (Figure S8B). When simulating data according to these fits, we recovered distributions of dropout rates and variance closely matching the observed data (Figures S8C and S8D). To compare the power for differential gene expression among the methods, we simulated read counts for two groups of n cells and added log-fold changes to 5% of the 13,361 genes in one group. To mimic a biologically realistic scenario, these log-fold changes were drawn from observed differences between microglial subpopulations from a previously published dataset (Zeisel et al., 2015). Simulated datasets were tested for differential expression using limma (Ritchie et al., 2015), and the true positive rate (TPR) and the false discovery rate (FDR) were calculated. Of note, this does include undetected genes, i.e., the 2.5% (SCRB-seq) to 6.8% (MARS-seq) of the 13,361 genes that had fewer than two measurements in a particular method (Figure S6B) and for which we could not estimate the variance. In our simulations, these genes could be drawn as differentially expressed, and in our TPR they were then counted as false negatives for the particular method. Hence, our power simulation framework considers the full range of dropout rates and is not biased against more sensitive methods. First, we analyzed how the number of cells affects TPR and FDR by running 100 simulations each for a range of 16 to 512 cells per group (Figure 6A). FDRs were similar in all methods ranging from 3.9% to 8.7% (Figure S9A). TPRs differed considerably among methods and SCRB-seq performed best, reaching a median TPR of 80% with 64 cells. CEL-seq2/C1, Drop-seq, MARS-seq, and Smart-seq2 performed slightly worse, reaching 80% power with 86, 99, 110, and 95 cells per group, respectively, while Smart-seq/C1 needed 150 cells to reach 80% power (Figure 6A). When disregarding UMIs, Smart-seq2 performed best (Figure 6B), as expected from its low dropout rate and its low amplification noise when considering reads only (Figure 5B). Furthermore, power dropped especially for Drop-seq and SCRB-seq (Figure 6B), as expected from the strong increase in amplification noise of these two methods when considering reads only (Figure 5B). When we stratified our analysis (considering UMIs) across five bins of expression levels, the ranking of methods was recapitulated and showed that the lowest expression bin strongly limited the TPR in all methods (Figure S9B). This ranking also was recapitulated when we analyzed a set of 19 genes previously reported to contain cell-cycle variation in the 2i/LIF culture condition (Kolodziejczyk et al., 2015b). The variance of these cell-cycle genes was clearly higher than the variance of 19 pluripotency and housekeeping (ribosomal) genes in all methods. The p value of that difference was lowest for SCRB-seq, the most powerful method, and highest for Smartseq/C1, the least powerful method (Figure S10D). Notably, this power analysis, as well as the sensitivity, accuracy, and precision parameters analyzed above, includes the variation that is generated in the two technical replicates (batches) per method that we performed (Figure 1). These estimates were very similar among our technical replicates, and, hence, our method comparison is valid with respect to batch variations (Figures S10B-S10D). In addition, as batch effects are known to be highly relevant for interpreting scRNA-seq data (Hicks et al., 2015), we gauged the magnitude of batch effects with respect to identifying differentially expressed genes. To this end, we used limma to identify differentially expressed genes between batches (FDR < 1%), using 25 randomly selected cells per batch and method. All methods had significantly more genes differentially expressed between batches than expected from permutations (zero to four genes), with a median of 119 (Dropseq) to 1,135 (CEL-seq2/C1) differentially expressed genes (Figure S10A). Notably, genes were affected at random across methods, as there was no significant overlap among them (extended hypergeometric test [Kalinka, 2013], p > 0.84). Hence, this analysis once more emphasizes that batches are important to consider in the design of scRNA-seq experiments (Hicks et al., 2015). While a quantitative comparison of the magnitude of batch effects among methods would require substantially more technical replicates per method, the methods differ in their flexibility to incorporate batch effect into the experimental design, which is an important aspect to consider as discussed below. <!-- image -->The power simulation framework considers the full range of dropout rates and is not biased against more sensitive methods.
“Hence, our power simulation framework considers the full range of dropout rates and is not biased against more sensitive methods.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductown data§ Drop-seqThe Drop-seq method was successfully established in the authors' laboratory, as confirmed by a species-mixing experiment.
“successful establishment of the method in our lab was confirmed by a species-mixing experiment (Figure S1A).”
Quote in the paperNot a substantial claimMay combine two claims.
Study design and conductown data§ PowerIs Determined by a Combination of Dropout Rates and Amplification Noise and Is Highest for SCRB-Seq To estimate the combined impact of sensitivity and precision on the power to detect differential gene expression, we simulated scRNA-seq data given the observed dropout rates and variance for the 13,361 genes. As these depend strongly on the expression level of a gene, it is important to retain the mean variance and mean dropout relationships. To this end, we estimated the mean, the variance (i.e., the dispersion parameter of the negative binomial distribution), and the dropout rate for each gene and method. We then fitted a cubic smoothing spline to the resulting pairs of mean and dispersion estimates to predict the dispersion of a gene given its mean (Figure S8A). Furthermore, we applied a local polynomial regression model to account for the dropout probability given a gene's mean expression (Figure S8B). When simulating data according to these fits, we recovered distributions of dropout rates and variance closely matching the observed data (Figures S8C and S8D). To compare the power for differential gene expression among the methods, we simulated read counts for two groups of n cells and added log-fold changes to 5% of the 13,361 genes in one group. To mimic a biologically realistic scenario, these log-fold changes were drawn from observed differences between microglial subpopulations from a previously published dataset (Zeisel et al., 2015). Simulated datasets were tested for differential expression using limma (Ritchie et al., 2015), and the true positive rate (TPR) and the false discovery rate (FDR) were calculated. Of note, this does include undetected genes, i.e., the 2.5% (SCRB-seq) to 6.8% (MARS-seq) of the 13,361 genes that had fewer than two measurements in a particular method (Figure S6B) and for which we could not estimate the variance. In our simulations, these genes could be drawn as differentially expressed, and in our TPR they were then counted as false negatives for the particular method. Hence, our power simulation framework considers the full range of dropout rates and is not biased against more sensitive methods. First, we analyzed how the number of cells affects TPR and FDR by running 100 simulations each for a range of 16 to 512 cells per group (Figure 6A). FDRs were similar in all methods ranging from 3.9% to 8.7% (Figure S9A). TPRs differed considerably among methods and SCRB-seq performed best, reaching a median TPR of 80% with 64 cells. CEL-seq2/C1, Drop-seq, MARS-seq, and Smart-seq2 performed slightly worse, reaching 80% power with 86, 99, 110, and 95 cells per group, respectively, while Smart-seq/C1 needed 150 cells to reach 80% power (Figure 6A). When disregarding UMIs, Smart-seq2 performed best (Figure 6B), as expected from its low dropout rate and its low amplification noise when considering reads only (Figure 5B). Furthermore, power dropped especially for Drop-seq and SCRB-seq (Figure 6B), as expected from the strong increase in amplification noise of these two methods when considering reads only (Figure 5B). When we stratified our analysis (considering UMIs) across five bins of expression levels, the ranking of methods was recapitulated and showed that the lowest expression bin strongly limited the TPR in all methods (Figure S9B). This ranking also was recapitulated when we analyzed a set of 19 genes previously reported to contain cell-cycle variation in the 2i/LIF culture condition (Kolodziejczyk et al., 2015b). The variance of these cell-cycle genes was clearly higher than the variance of 19 pluripotency and housekeeping (ribosomal) genes in all methods. The p value of that difference was lowest for SCRB-seq, the most powerful method, and highest for Smartseq/C1, the least powerful method (Figure S10D). Notably, this power analysis, as well as the sensitivity, accuracy, and precision parameters analyzed above, includes the variation that is generated in the two technical replicates (batches) per method that we performed (Figure 1). These estimates were very similar among our technical replicates, and, hence, our method comparison is valid with respect to batch variations (Figures S10B-S10D). In addition, as batch effects are known to be highly relevant for interpreting scRNA-seq data (Hicks et al., 2015), we gauged the magnitude of batch effects with respect to identifying differentially expressed genes. To this end, we used limma to identify differentially expressed genes between batches (FDR < 1%), using 25 randomly selected cells per batch and method. All methods had significantly more genes differentially expressed between batches than expected from permutations (zero to four genes), with a median of 119 (Dropseq) to 1,135 (CEL-seq2/C1) differentially expressed genes (Figure S10A). Notably, genes were affected at random across methods, as there was no significant overlap among them (extended hypergeometric test [Kalinka, 2013], p > 0.84). Hence, this analysis once more emphasizes that batches are important to consider in the design of scRNA-seq experiments (Hicks et al., 2015). While a quantitative comparison of the magnitude of batch effects among methods would require substantially more technical replicates per method, the methods differ in their flexibility to incorporate batch effect into the experimental design, which is an important aspect to consider as discussed below. <!-- image -->The estimates of sensitivity, accuracy, precision, and power were very similar among the two technical replicates per method, so the method comparison is valid with respect to batch variations.
“These estimates were very similar among our technical replicates, and, hence, our method comparison is valid with respect to batch variations (Figures S10B-S10D).”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductown data§ EXPERIMENTAL MODEL AND SUBJECT DETAILSThe J1 embryonic stem cells used in the study were mycoplasma free, as determined by a PCR-based test.
“J1 embryonic stem cells were obtained from E. Li and T. Chen and mycoplasma free determined by a PCR-based test.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable).
Interpretationcitation§ DISCUSSIONExperiments need to be designed in a way that does not confound batches with biological factors.
“Furthermore, the presence of batch effects shows that experiments need to be designed in a way that does not confound batches with biological factors (Hicks et al., 2015).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Hicks et al., 2015
Not checked in the free run.
May combine two claims.
Interpretationown data§ DISCUSSIONIn summary, Drop-seq is preferable when quantifying transcriptomes of large numbers of cells with low sequencing depth, SCRB-seq and MARS-seq are preferable when quantifying transcriptomes of fewer cells, and Smart-seq2 is preferable when annotating and/or quantifying transcriptomes of fewer cells as long as one can use in-house-produced transposase.
“In summary, we systematically compared six prominent scRNA-seq methods and found that Drop-seq is preferable when quantifying transcriptomes of large numbers of cells with low sequencing depth, SCRB-seq and MARS-seq is preferable when quantifying transcriptomes of fewer cells, and Smart-seq2 is preferable when annotating and/or quantifying transcriptomes of fewer cells as long one can use in-house”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). The quote only partly states this claim.
Study design and conductnone given§ EXPERIMENTAL MODEL AND SUBJECT DETAILSCell line authentication was not recently performed for the J1 embryonic stem cells.
“Cell line authentication was not recently performed.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim. May combine two claims.
Study design and conductown data§ Published dataThe external Drop-seq ERCC data were trimmed to the same read length and processed in the same way as the data sequenced for this study.
“We trimmed cDNA reads to the same length and processed raw reads in the same way as data sequenced for this study.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductown data§ DNA sequencingSequencing qualities were similar across the libraries, as confirmed by FastQC v0.10.1.
“Similar sequencing qualities were confirmed by FastQC v0.10.1 (Figure S1B).”
Quote in the paperNot a substantial claim
The paper relies on this without support.
A thorough and systematic comparison of relevant parameters across scRNA-seq methods is still lacking.
“However, a thorough and systematic comparison of relevant parameters across scRNA-seq methods is still lacking.”
“However, systematic comparisons of the performance of diverse scRNA-seq protocols are lacking.”
The authors state this as an assumption.
Differences in total variation among methods are due to differences in their technical variation.
“Hence, we can assume that differences in the total variation among methods are due to differences in their technical variation.”
The paper relies on this without support.
The amount of biological variation is the same in cells analyzed by each of the six methods.
“As we used the same cell type under the same culture conditions for all methods, the amount of biological variation should be the same in the cells analyzed by each of the six methods.”
The authors state this as an assumption.
Sequencing costs are the same for all methods.
“we assume that sequencing costs are the same for all methods.”
The authors state this as an assumption.
Detections of ERCC molecules are independent from each other and can be regarded as independent Bernoulli trials.
“Detections are independent from each other and are thus regarded as independent Bernoulli trials.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Combinatorial indexing and droplet-based methods have higher throughput and lower cost per cell, whereas plate-based methods provide more sensitive gene detection and are more customizable.
“Combinatorial indexing and droplet-based methods have higher throughput and lower cost per cell, whereas plate-based methods provide more sensitive gene detection and are more customizable (Ziegenhain et al., 2017).”
From Single-Cell Sequencing of Brain Cell Transcriptomes and Epigenomes (2021) · cites this paper as Ziegenhain et al., 2017
In scRNA-seq experiments, individual cells are typically not sequenced to saturation, leading to sparse count matrices, where only highly expressed genes are detected with counts greater than zero.
“In scRNA-seq experiments, individual cells are typically not sequenced to saturation, leading to sparse count matrices, where only highly expressed genes are detected with counts greater than zero.”
From scPower accelerates and optimizes the design of multi-sample single cell transcriptomic studies (2021) · cites this paper as [36]
The optimal read depths were ~10,000 in most evaluations, relatively low compared to previous recommendations.
“Specifically, we found the optimal read depths to be ~10,000 in most evaluations, which is relatively low compared to previous recommendations34–36,80,81.”
From scPower accelerates and optimizes the design of multi-sample single cell transcriptomic studies (2021) · cites this paper as [36]
scRNA-seq techniques can be divided into full-length and 3′ enrichment methods.
“Furthermore, scRNA‐seq techniques can be divided into full‐length and 3′ enrichment methods (Svensson et al, 2017; Ziegenhain et al, 2017).”
From Current best practices in single-cell RNA-seq analysis: a tutorial (2019) · cites this paper as Ziegenhain et al, 2017
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Commercial Smart-seq2 total conflicts with the stated cost calculation
MajorAt one million reads, 95 cells per group means 190 libraries. At the listed $30 per library and $5 per million reads, the stated calculation gives $6,650, not the table's $10,470; the other two commercial Smart-seq2 totals are also too high under that calculation.
“| Smart-seq2 (commercial) | 0.8 | 5.2 | 95/105/128 | 30 | 10,470/11,040/13,160 |”
“c Assuming $5 per one million reads.”
Cites
STAR: ultrafast universal RNA-seq aligner
Dobin, A., Davis, C. A., Schlesinger, F., Drenkow, J., Zaleski, C., Jha, S., Batut, P., Chaisson, M., & Gingeras, T. R. (2013) · Bioinformatics · doi:10.1093/bioinformatics/bts635 · Cited by 51,709 · see citing works
limma powers differential expression analyses for RNA-sequencing and microarray studies
Ritchie, M. E., Phipson, B., Wu, D., Hu, Y., Law, C. W., Shi, W., & Smyth, G. K. (2015) · Nucleic Acids Research · doi:10.1093/nar/gkv007 · Cited by 40,458 · see citing works
Cutadapt removes adapter sequences from high-throughput sequencing reads
Martin, M. (2011) · EMBnet.journal · doi:10.14806/ej.17.1.200 · Cited by 33,889 · see citing works
An integrated encyclopedia of DNA elements in the human genome
The ENCODE Project Consortium (2012) · Nature · doi:10.1038/nature11247 · Cited by 18,072 · see citing works
Highly parallel genome-wide expression profiling of individual cells using nanoliter droplets.
Macosko, E. Z., Basu, A., Satija, R., Nemesh, J., Shekhar, K., Goldman, M., Tirosh, I., Bialas, A. R., Kamitaki, N., Martersteck, E. M., et al. (2015) · Cell · doi:10.1016/j.cell.2015.05.002 · Cited by 7,110 · see citing works
voom: precision weights unlock linear model analysis tools for RNA-seq read counts
Law, C. W., Chen, Y., Shi, W., & Smyth, G. K. (2014) · Genome Biology · doi:10.1186/gb-2014-15-2-r29 · Cited by 6,155 · see citing works
Dissecting the multicellular ecosystem of metastatic melanoma by single-cell RNA-seq
Tirosh, I., Izar, B., Prakadan, S. M., Wadsworth, M. H., Treacy, D., Trombetta, J. J., Rotem, A., Rodman, C., Lian, C., Murphy, G., et al. (2016) · Science · doi:10.1126/science.aad0501 · Cited by 4,944 · see citing works
Full-length RNA-seq from single cells using Smart-seq2
Picelli, S., Faridani, O. R., Björklund, Å. K., Winberg, G., Sagasser, S., & Sandberg, R. (2014) · Nature Protocols · doi:10.1038/nprot.2014.006 · Cited by 4,137 · see citing works
Targeted mutation of the DNA methyltransferase gene results in embryonic lethality
Li, E., Bestor, T. H., & Jaenisch, R. (1992) · Cell · doi:10.1016/0092-8674(92)90611-f · Cited by 3,984 · see citing works
Droplet barcoding for single-cell transcriptomics applied to embryonic stem cells
Klein, A. M., Mazutis, L., Akartuna, I., Tallapragada, N., Veres, A., Li, V., Peshkin, L., Weitz, D. A., & Kirschner, M. W. (2015) · Cell · doi:10.1016/j.cell.2015.04.044 · Cited by 3,356 · see citing works
Finak, G., McDavid, A., Yajima, M., Deng, J., Gersuk, V., Shalek, A. K., Slichter, C. K., Miller, H. W., McElrath, M. J., Prlic, M., et al. (2015) · Genome Biology · doi:10.1186/s13059-015-0844-5 · Cited by 3,159 · see citing works
Cell types in the mouse cortex and hippocampus revealed by single-cell RNA-seq
Zeisel, A., Muñoz-Manchado, A. B., Codeluppi, S., Lönnerberg, P., La Manno, G., Juréus, A., Marques, S., Munguba, H., He, L., Betsholtz, C., et al. (2015) · Science · doi:10.1126/science.aaa1934 · Cited by 3,124 · see citing works
The Subread aligner: fast, accurate and scalable read mapping by seed-and-vote
Liao, Y., Smyth, G. K., & Shi, W. (2013) · Nucleic Acids Research · doi:10.1093/nar/gkt214 · Cited by 3,064 · see citing works
Smart-seq2 for sensitive full-length transcriptome profiling in single cells
Picelli, S., Björklund, Å. K., Faridani, O. R., Sagasser, S., Winberg, G., & Sandberg, R. (2013) · Nature Methods · doi:10.1038/nmeth.2639 · Cited by 2,358 · see citing works
Normalization of RNA-seq data using factor analysis of control genes or samples
Risso, D., Ngai, J., Speed, T. P., & Dudoit, S. (2014) · Nature Biotechnology · doi:10.1038/nbt.2931 · Cited by 2,169 · see citing works
Massively parallel single-cell RNA-seq for marker-free decomposition of tissues into cell types
Jaitin, D. A., Kenigsberg, E., Keren-Shaul, H., Elefant, N., Paul, F., Zaretsky, I., Mildner, A., Cohen, N., Jung, S., Tanay, A., et al. (2014) · Science · doi:10.1126/science.1247651 · Cited by 1,795 · see citing works
Full-length mRNA-Seq from single-cell levels of RNA and individual circulating tumor cells
Ramsköld, D., Luo, S., Wang, Y.-C., Li, R., Deng, Q., Faridani, O. R., Daniels, G. A., Khrebtukova, I., Loring, J. F., Laurent, L. C., et al. (2012) · Nature Biotechnology · doi:10.1038/nbt.2282 · Cited by 1,609 · see citing works
The Technology and Biology of Single-Cell RNA Sequencing
Kolodziejczyk, A. A., Kim, J. K., Svensson, V., Marioni, J. C., & Teichmann, S. A. (2015) · Molecular Cell · doi:10.1016/j.molcel.2015.04.005 · Cited by 1,487 · see citing works
Bayesian approach to single-cell differential expression analysis
Kharchenko, P. V., Silberstein, L., & Scadden, D. T. (2014) · Nature Methods · doi:10.1038/nmeth.2967 · Cited by 1,348 · see citing works
Single-Cell RNA-Seq Reveals Dynamic, Random Monoallelic Gene Expression in Mammalian Cells
Deng, Q., Ramsköld, D., Reinius, B., & Sandberg, R. (2014) · Science · doi:10.1126/science.1245316 · Cited by 1,345 · see citing works
Self-citationCEL-seq: single-cell RNA-seq by multiplexed linear amplification
Hashimshony, T., Wagner, F., Sher, N., & Yanai, I. (2012) · Cell Reports · doi:10.1016/j.celrep.2012.08.003 · Cited by 1,231 · see citing works
Quantitative single-cell RNA-seq with unique molecular identifiers
Islam, S., Zeisel, A., Joost, S., La Manno, G., Zajac, P., Kasper, M., Lönnerberg, P., & Linnarsson, S. (2014) · Nature Methods · doi:10.1038/nmeth.2772 · Cited by 1,183 · see citing works
Computational and analytical challenges in single-cell transcriptomics
Stegle, O., Teichmann, S. A., & Marioni, J. C. (2015) · Nature Reviews Genetics · doi:10.1038/nrg3833 · Cited by 1,170 · see citing works
Buettner, F., Natarajan, K. N., Casale, F. P., Proserpio, V., Scialdone, A., Theis, F. J., Teichmann, S. A., Marioni, J. C., & Stegle, O. (2015) · Nature Biotechnology · doi:10.1038/nbt.3102 · Cited by 1,136 · see citing works
CEL-seq2: sensitive highly-multiplexed single-cell RNA-Seq
Hashimshony, T., Senderovich, N., Avital, G., Klochendler, A., de Leeuw, Y., Anavy, L., Gennert, D., Li, S., Livak, K. J., Rozenblatt-Rosen, O., et al. (2016) · Genome Biology · doi:10.1186/s13059-016-0938-8 · Cited by 1,125 · see citing works
Pooling across cells to normalize single-cell RNA sequencing data with many zero counts
L. Lun, A. T., Bach, K., & Marioni, J. C. (2016) · Genome Biology · doi:10.1186/s13059-016-0947-7 · Cited by 1,108 · see citing works
Single-Cell RNA-Seq Reveals Lineage and X Chromosome Dynamics in Human Preimplantation Embryos
Petropoulos, S., Edsgärd, D., Reinius, B., Deng, Q., Panula, S. P., Codeluppi, S., Plaza Reyes, A., Linnarsson, S., Sandberg, R., & Lanner, F. (2016) · Cell · doi:10.1016/j.cell.2016.03.023 · Cited by 1,063 · see citing works
Self-citationCounting absolute numbers of molecules using unique molecular identifiers
Kivioja, T., Vähärautio, A., Karlsson, K., Bonke, M., Enge, M., Linnarsson, S., & Taipale, J. (2012) · Nature Methods · doi:10.1038/nmeth.1778 · Cited by 1,010 · see citing works
Single-cell RNA-seq reveals dynamic paracrine control of cellular variation
Shalek, A. K., Satija, R., Shuga, J., Trombetta, J. J., Gennert, D., Lu, D., Chen, P., Gertner, R. S., Gaublomme, J. T., Yosef, N., et al. (2014) · Nature · doi:10.1038/nature13437 · Cited by 1,000 · see citing works
Accounting for technical noise in single-cell RNA-seq experiments
Brennecke, P., Anders, S., Kim, J. K., Kołodziejczyk, A. A., Zhang, X., Proserpio, V., Baying, B., Benes, V., Teichmann, S. A., Marioni, J. C., et al. (2013) · Nature Methods · doi:10.1038/nmeth.2645 · Cited by 979 · see citing works
Single-cell RNA-seq: advances and future challenges
Saliba, A.-E., Westermann, A. J., Gorski, S. A., & Vogel, J. (2014) · Nucleic Acids Research · doi:10.1093/nar/gku555 · Cited by 847 · see citing works
Tn5 transposase and tagmentation procedures for massively scaled sequencing projectsPreprint
Picelli, S., Björklund, Å. K., Reinius, B., Sagasser, S., Winberg, G., & Sandberg, R. (2014) · Genome Research · doi:10.1101/gr.177881.114 · Cited by 829 · see citing works
Self-citationIn vivo interrogation of gene function in the mammalian brain using CRISPR-Cas9
Swiech, L., Heidenreich, M., Banerjee, A., Habib, N., Li, Y., Trombetta, J., Sur, M., & Zhang, F. (2015) · Nature Biotechnology · doi:10.1038/nbt.3055 · Cited by 772 · see citing works
G&T-seq: parallel sequencing of single-cell genomes and transcriptomes
Macaulay, I. C., Haerty, W., Kumar, P., Li, Y. I., Hu, T. X., Teng, M. J., Goolam, M., Saurat, N., Coupland, P., Shirley, L. M., et al. (2015) · Nature Methods · doi:10.1038/nmeth.3370 · Cited by 750 · see citing works
Validation of noise models for single-cell transcriptomics
Grün, D., Kester, L., & van Oudenaarden, A. (2014) · Nature Methods · doi:10.1038/nmeth.2930 · Cited by 721 · see citing works
Quantitative assessment of single-cell RNA-sequencing methods
Wu, A. R., Neff, N. F., Kalisky, T., Dalerba, P., Treutlein, B., Rothenberg, M. E., Mburu, F. M., Mantalas, G. L., Sim, S., Clarke, M. F., et al. (2014) · Nature Methods · doi:10.1038/nmeth.2694 · Cited by 696 · see citing works
Synthetic spike-in standards for RNA-seq experiments
Jiang, L., Schlesinger, F., Davis, C. A., Zhang, Y., Li, R., Salit, M., Gingeras, T. R., & Oliver, B. (2011) · Genome Research · doi:10.1101/gr.121095.111 · Cited by 653 · see citing works
Revealing the vectors of cellular identity with single-cell genomics
Wagner, A., Regev, A., & Yosef, N. (2016) · Nature Biotechnology · doi:10.1038/nbt.3711 · Cited by 632 · see citing works
Missing Data and Technical Variability in Single-Cell RNA-Sequencing Experiments
Hicks, S. C., Townes, F. W., Teng, M., & Irizarry, R. A. (2015) · doi:10.1101/025528 · Cited by 625 · see citing works
Single Cell RNA-Sequencing of Pluripotent States Unlocks Modular Transcriptional Variation
Kolodziejczyk, A. A., Kim, J. K., Tsang, J. C. H., Ilicic, T., Henriksson, J., Natarajan, K. N., Tuck, A. C., Gao, X., Bühler, M., Liu, P., et al. (2015) · Cell Stem Cell · doi:10.1016/j.stem.2015.09.011 · Cited by 570 · see citing works
Power analysis of single-cell RNA-sequencing experiments
Svensson, V., Natarajan, K. N., Ly, L.-H., Miragaia, R. J., Labalette, C., Macaulay, I. C., Cvejic, A., & Teichmann, S. A. (2017) · Nature Methods · doi:10.1038/nmeth.4220 · Cited by 568 · see citing works
Design and Analysis of Single-Cell Sequencing Experiments
Grün, D., & van Oudenaarden, A. (2015) · Cell · doi:10.1016/j.cell.2015.10.039 · Cited by 536 · see citing works
From single-cell to cell-pool transcriptomes: Stochasticity in gene expression and RNA splicing
Marinov, G. K., Williams, B. A., McCue, K., Schroth, G. P., Gertz, J., Myers, R. M., & Wold, B. J. (2014) · Genome Research · doi:10.1101/gr.161034.113 · Cited by 468 · see citing works
Cellular Taxonomy of the Mouse Striatum as Revealed by Single-Cell RNA-Seq
Gokce, O., Stanley, G. M., Treutlein, B., Neff, N. F., Camp, J. G., Malenka, R. C., Rothwell, P. E., Fuccillo, M. V., Südhof, T. C., & Quake, S. R. (2016) · Cell Reports · doi:10.1016/j.celrep.2016.06.059 · Cited by 430 · see citing works
<i>Polyester</i> : simulating RNA-seq datasets with differential transcript expression
Frazee, A. C., Jaffe, A. E., Langmead, B., & Leek, J. T. (2015) · Bioinformatics · doi:10.1093/bioinformatics/btv272 · Cited by 313 · see citing works
The promise of single-cell sequencing
Eberwine, J. H., Sul, J., Bartfai, T., & Kim, J. (2013) · Nature Methods · doi:10.1038/nmeth.2769 · Cited by 305 · see citing works
The impact of amplification on differential expression analyses by RNA-seq
Parekh, S., Ziegenhain, C., Vieth, B., Enard, W., & Hellmann, I. (2016) · Scientific Reports · doi:10.1038/srep25533 · Cited by 252 · see citing works
Self-citationKim, J. K., Kolodziejczyk, A. A., Ilicic, T., Teichmann, S. A., & Marioni, J. C. (2015) · Nature Communications · doi:10.1038/ncomms9687 · Cited by 251 · see citing works
Entering the era of single-cell transcriptomics in biology and medicine
Sandberg, R. (2013) · Nature Methods · doi:10.1038/nmeth.2764 · Cited by 242 · see citing works
Fixed single-cell transcriptomic characterization of human radial glial diversity
Thomsen, E. R., Mich, J. K., Yao, Z., Hodge, R. D., Doyle, A. M., Jang, S., Shehata, S. I., Nelson, A. M., Shapovalova, N. V., Levi, B. P., et al. (2015) · Nature Methods · doi:10.1038/nmeth.3629 · Cited by 227 · see citing works
Single-Cell RNA-Seq Reveals Lineage and X Chromosome Dynamics in Human Preimplantation Embryos
Sophie Petropoulos, A., Edsgärd, D., Reinius, B., Deng, Q., Panula, S., Codeluppi, S., Reyes, Á. P., Linnarsson, S., Sandberg, R., & Lanner, F. (2016) · Cell · doi:10.1016/j.cell.2016.08.009 · Cited by 227 · see citing works
Self-citationCharacterization of directed differentiation by high-throughput single-cell RNA-SeqPreprint
Soumillon, M., Cacchiarelli, D., Semrau, S., van Oudenaarden, A., & Mikkelsen, T. S. (2014) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/003236 · Cited by 217 · see citing works
deML: robust demultiplexing of Illumina sequences using a likelihood-based approach
Renaud, G., Stenzel, U., Maričić, T., Wiebe, V., & Kelso, J. F. (2014) · Bioinformatics · doi:10.1093/bioinformatics/btu719 · Cited by 200 · see citing works
Nam, D., Lee, S., Zhou, G., Cao, X., Wang, C., Clark, T. W., Chen, J., Rowley, J. D., & Wang, S. M. (2002) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.092140899 · Cited by 193 · see citing works
Single-Cell RNA-Sequencing Reveals a Continuous Spectrum of Differentiation in Hematopoietic Cells
Macaulay, I. C., Svensson, V., Labalette, C., Ferreira, L., Hamey, F., Voet, T., Teichmann, S. A., & Cvejic, A. (2016) · Cell Reports · doi:10.1016/j.celrep.2015.12.082 · Cited by 191 · see citing works
Transcriptional Regulatory Circuits: Predicting Numbers from Alphabets
Kim, H. D., Shay, T., O’Shea, E. K., & Regev, A. (2009) · Science · doi:10.1126/science.1171347 · Cited by 179 · see citing works
Lamble, S., Batty, E. M., Attar, M., Buck, D., Bowden, R. J., Lunter, G. A., Crook, D. W., El‐Fahmawi, B., & Piazza, P. (2013) · BMC Biotechnology · doi:10.1186/1472-6750-13-104 · Cited by 149 · see citing works
Analysis of allelic expression patterns in clonal somatic cells by single-cell RNA–seq
Reinius, B., Mold, J. E., Ramsköld, D., Deng, Q., Johnsson, P., Michaëlsson, J., Frisén, J., & Sandberg, R. (2016) · Nature Genetics · doi:10.1038/ng.3678 · Cited by 149 · see citing works
Self-citationCounting individual DNA molecules by the stochastic attachment of diverse labels
Fu, G. K., Hu, J., Wang, P., & Fodor, S. P. A. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1017621108 · Cited by 142 · see citing works
Beyond comparisons of means: understanding changes in gene expression at the single-cell level
Vallejos, C. A., Richardson, S. T., & Marioni, J. C. (2016) · Genome biology · doi:10.1186/s13059-016-0930-3 · Cited by 94 · see citing works
Mora-Castilla, S., To, C. C., Vaezeslami, S., Morey, R. E., Srinivasan, S., Chousal, J. N., Cook‐Andersen, H., Jenkins, J., & Laurent, L. C. (2016) · SLAS TECHNOLOGY · doi:10.1177/2211068216630741 · Cited by 61 · see citing works
The probability of drawing intersections: extending the hypergeometric distributionPreprint
Kalinka, A. T. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1305.0717 · Cited by 11 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- Hicks, 2015, On the widespread and critical impact of systematic bias and batch effects in single-cell RNA-Seq data, bioRxiv
Cited by
Sorted by citations
Integrating single-cell transcriptomic data across different conditions, technologies, and species
Butler, A., Hoffman, P., Smibert, P., Papalexi, E., & Satija, R. (2018) · Nature Biotechnology · doi:10.1038/nbt.4096 · Cited by 13,174 · see citing works
McGinnis, C. S., Murrow, L. M., & Gartner, Z. J. (2019) · Cell Systems · doi:10.1016/j.cels.2019.03.003 · Cited by 4,521 · see citing works
Determining cell type abundance and expression from bulk tissues with digital cytometry
Newman, A. M., Steen, C. B., Liu, C. L., Gentles, A. J., Chaudhuri, A. A., Scherer, F., Khodadoust, M. S., Esfahani, M. S., Luca, B. A., Steiner, D. F., et al. (2019) · Nature Biotechnology · doi:10.1038/s41587-019-0114-2 · Cited by 4,528 · see citing works
Wound Healing: A Cellular Perspective.
Rodrigues, M., Kosaric, N., Bonham, C., & Gurtner, G. (2019) · Physiological Reviews · doi:10.1152/physrev.00067.2017 · Cited by 2,752 · see citing works
Regev, A., Teichmann, S. A., Lander, E. S., Amit, I., Benoist, C., Birney, E., Bodenmiller, B., Campbell, P., Carninci, P., Clatworthy, M., et al. (2017) · eLife · doi:10.7554/elife.27041 · Cited by 2,232 · see citing works
Self-citationCurrent best practices in single-cell RNA-seq analysis: a tutorial
Luecken, M. D., & Theis, F. J. (2019) · Molecular Systems Biology · doi:10.15252/msb.20188746 · Cited by 2,231 · see citing works
RNA sequencing: the teenage years
Stark, R., Grzelak, M., & Hadfield, J. (2019) · Nature Reviews Genetics · doi:10.1038/s41576-019-0150-2 · Cited by 1,906 · see citing works
Mapping the Mouse Cell Atlas by Microwell-Seq
Han, X., Wang, R., Zhou, Y., Fei, L., Sun, H., Lai, S., Saadatpour, A., Zhou, Z., Chen, H., Ye, F., et al. (2018) · Cell · doi:10.1016/j.cell.2018.02.001 · Cited by 1,597 · see citing works
Landscape and Dynamics of Single Immune Cells in Hepatocellular Carcinoma
Zhang, Q., He, Y., Luo, N., Patel, S. J., Han, Y., Gao, R., Modak, M., Carotta, S., Haslinger, C., Kind, D., et al. (2019) · Cell · doi:10.1016/j.cell.2019.10.003 · Cited by 1,624 · see citing works
Single-cell transcriptional diversity is a hallmark of developmental potential
Gulati, G. S., Sikandar, S. S., Wesche, D. J., Manjunath, A., Bharadwaj, A., Berger, M. J., Ilagan, F., Kuo, A. H., Hsieh, R. W., Cai, S., et al. (2020) · Science · doi:10.1126/science.aax0249 · Cited by 1,571 · see citing works
Regev, A., Teichmann, S. A., Lander, E. S., Amit, I., Benoist, C. O., Birney, E., Bodenmiller, B., & al, E. (2017) · Zurich Open Repository and Archive (University of Zurich) · doi:10.5167/uzh-162597 · Cited by 1,457 · see citing works
Multiplexed droplet single-cell RNA-sequencing using natural genetic variation
Kang, H. M., Subramaniam, M., Targ, S., Nguyen, M., Maliskova, L., McCarthy, E., Wan, E., Wong, S., Byrnes, L., Lanata, C. M., et al. (2018) · Nature Biotechnology · doi:10.1038/nbt.4042 · Cited by 1,339 · see citing works
Hu, C., Li, T., Xu, Y., Zhang, X., Li, F., Bai, J., Chen, J., Jiang, W., Yang, K., Ou, Q., et al. (2022) · Nucleic Acids Research · doi:10.1093/nar/gkac947 · Cited by 1,284 · see citing works
Single‐cell RNA sequencing technologies and applications: A brief overview
Jovic, D., Liang, X., Zeng, H., Lin, L., Xu, F., & Luo, Y. (2022) · Clinical and Translational Medicine · doi:10.1002/ctm2.694 · Cited by 1,307 · see citing works
Massively parallel single-nucleus RNA-seq with DroNc-seq
Habib, N., Avraham-Davidi, I., Basu, A., Burks, T., Shekhar, K., Hofree, M., Choudhury, S. R., Aguet, F., Gelfand, E., Ardlie, K., et al. (2017) · Nature Methods · doi:10.1038/nmeth.4407 · Cited by 1,069 · see citing works
A practical guide to single-cell RNA-sequencing for biomedical research and clinical applications
Haque, A., Engel, J., Teichmann, S. A., & Lönnberg, T. (2017) · Genome Medicine · doi:10.1186/s13073-017-0467-4 · Cited by 1,071 · see citing works
Li, Q., Cheng, Z., Zhou, L., Darmanis, S., Neff, N. F., Okamoto, J., Gulati, G., Bennett, M. L., Sun, L. O., Clarke, L. E., et al. (2019) · Neuron · doi:10.1016/j.neuron.2018.12.006 · Cited by 1,129 · see citing works
Moncada, R., Barkley, D., Wagner, F., Chiodin, M., Devlin, J. C., Baron, M., Hajdu, C. H., Simeone, D. M., & Yanai, I. (2020) · Nature Biotechnology · doi:10.1038/s41587-019-0392-8 · Cited by 1,045 · see citing works
Systematic comparison of single-cell and single-nucleus RNA-sequencing methods
Ding, J., Adiconis, X., Simmons, S. K., Kowalczyk, M. S., Hession, C. C., Marjanovic, N. D., Hughes, T. K., Wadsworth, M. H., Burks, T., Nguyen, L. T., et al. (2020) · Nature Biotechnology · doi:10.1038/s41587-020-0465-8 · Cited by 871 · see citing works
Guidelines for the use of flow cytometry and cell sorting in immunological studies (second edition)
Cossarizza, A., Chang, H., Radbruch, A., Acs, A., Adam, D., Adam‐Klages, S., Agace, W. W., Aghaeepour, N., Akdiş, M., Allez, M., et al. (2019) · European Journal of Immunology · doi:10.1002/eji.201970107 · Cited by 982 · see citing works
Neuronal cell-type classification: challenges, opportunities and the path forward
Zeng, H., & Sanes, J. R. (2017) · Nature Reviews Neuroscience · doi:10.1038/nrn.2017.85 · Cited by 841 · see citing works
Single-Cell RNA-Seq Technologies and Related Computational Data Analysis
Chen, G., Ning, B., & Shi, T. (2019) · Frontiers in Genetics · doi:10.3389/fgene.2019.00317 · Cited by 913 · see citing works
Single-cell transcriptomes from human kidneys reveal the cellular identity of renal tumors
Young, M. D., Mitchell, T. J., Vieira Braga, F. A., Tran, M. G. B., Stewart, B. J., Ferdinand, J. R., Collord, G., Botting, R. A., Popescu, D.-M., Loudon, K. W., et al. (2018) · Science · doi:10.1126/science.aat1699 · Cited by 762 · see citing works
Wu, H., Kirita, Y., Donnelly, E. L., & Humphreys, B. D. (2019) · Journal of the American Society of Nephrology · doi:10.1681/asn.2018090912 · Cited by 752 · see citing works
Comparison and evaluation of statistical error models for scRNA-seq
Choudhary, S., & Satija, R. (2022) · Genome Biology · doi:10.1186/s13059-021-02584-9 · Cited by 634 · see citing works
Single-cell RNA counting at allele and isoform resolution using Smart-seq3
Hagemann-Jensen, M., Ziegenhain, C., Chen, P., Ramsköld, D., Hendriks, G.-J., Larsson, A. J. M., Faridani, O. R., & Sandberg, R. (2020) · Nature Biotechnology · doi:10.1038/s41587-020-0497-0 · Cited by 700 · see citing works
Self-citationAn atlas of the aging lung mapped by single cell transcriptomics and deep tissue proteomics
Angelidis, I., Simon, L. M., Fernandez, I. E., Strunz, M., Mayr, C. H., Greiffo, F. R., Tsitsiridis, G., Ansari, M., Graf, E., Strom, T.-M., et al. (2019) · Nature Communications · doi:10.1038/s41467-019-08831-9 · Cited by 637 · see citing works
Xu, C., Lopez, R., Mehlman, E., Regier, J., Jordan, M. I., & Yosef, N. (2021) · Molecular Systems Biology · doi:10.15252/msb.20209620 · Cited by 691 · see citing works
“A second scenario is that of similar tissue but different technologies, which we expect to be more challenging as each technology comes with its own characteristics and biases (Ziegenhain et al , 2017).”
Decontamination of ambient RNA in single-cell RNA-seq with DecontX
Yang, S., Corbett, S. E., Koga, Y., Wang, Z., Johnson, W. E., Yajima, M., & Campbell, J. D. (2020) · Genome Biology · doi:10.1186/s13059-020-1950-6 · Cited by 635 · see citing works
Missing data and technical variability in single-cell RNA-sequencing experiments
Hicks, S. C., Townes, F. W., Teng, M., & Irizarry, R. A. (2018) · Biostatistics · doi:10.1093/biostatistics/kxx053 · Cited by 635 · see citing works
Applications of single-cell sequencing in cancer research: progress and perspectives
Lei, Y., Tang, R., Xu, J., Wang, W., Zhang, B., Liu, J., Yu, X., & Shi, S. (2021) · Journal of Hematology & Oncology · doi:10.1186/s13045-021-01105-2 · Cited by 597 · see citing works
Power analysis of single-cell RNA-sequencing experiments
Svensson, V., Natarajan, K. N., Ly, L.-H., Miragaia, R. J., Labalette, C., Macaulay, I. C., Cvejic, A., & Teichmann, S. A. (2017) · Nature Methods · doi:10.1038/nmeth.4220 · Cited by 568 · see citing works
Single-Cell RNA Sequencing Resolves Molecular Relationships Among Individual Plant Cells
Ryu, K. H., Huang, L., Kang, H. M., & Schiefelbein, J. (2019) · PLANT PHYSIOLOGY · doi:10.1104/pp.18.01482 · Cited by 529 · see citing works
Lineage Tracing in Humans Enabled by Mitochondrial Mutations and Single-Cell Genomics
Ludwig, L. S., Lareau, C. A., Ulirsch, J. C., Christian, E., Muus, C., Li, L. H., Pelka, K., Ge, W. H., Oren, Y., Brack, A., et al. (2019) · Cell · doi:10.1016/j.cell.2019.01.022 · Cited by 534 · see citing works
Genetic insights into the neurodevelopmental origins of schizophrenia
Birnbaum, R., & Weinberger, D. R. (2017) · Nature reviews. Neuroscience · doi:10.1038/nrn.2017.125 · Cited by 493 · see citing works
Benchmarking single-cell RNA-sequencing protocols for cell atlas projects
Mereu, E., Lafzi, A., Moutinho, C., Ziegenhain, C., McCarthy, D. J., Álvarez-Varela, A., Batlle, E., Sagar, Grün, D., Lau, J. K., et al. (2020) · Nature Biotechnology · doi:10.1038/s41587-020-0469-4 · Cited by 499 · see citing works
Self-citationMyeloid-derived suppressor cells: an emerging target for anticancer immunotherapy
Wu, Y., Yi, M., Niu, M., Mei, Q., & Wu, K. (2022) · Molecular Cancer · doi:10.1186/s12943-022-01657-y · Cited by 463 · see citing works
SABER amplifies FISH: enhanced multiplexed imaging of RNA and DNA in cells and tissues
Kishi, J. Y., Lapan, S. W., Beliveau, B. J., West, E. R., Zhu, A., Sasaki, H. M., Saka, S. K., Wang, Y., Cepko, C. L., & Yin, P. (2019) · Nature Methods · doi:10.1038/s41592-019-0404-0 · Cited by 440 · see citing works
Jew, B., Alvarez, M., Rahmani, E., Miao, Z., Ko, A., Garske, K. M., Sul, J. H., Pietiläinen, K. H., Pajukanta, P., & Halperin, E. (2020) · Nature Communications · doi:10.1038/s41467-020-15816-6 · Cited by 427 · see citing works
The emerging landscape of spatial profiling technologies
Moffitt, J. R., Lundberg, E., & Heyn, H. (2022) · Nature Reviews Genetics · doi:10.1038/s41576-022-00515-3 · Cited by 419 · see citing works
Self-citationByrne, A., Beaudin, A. E., Olsen, H. E., Jain, M., Cole, C., Palmer, T., DuBois, R. M., Forsberg, E. C., Akeson, M., & Vollmers, C. (2017) · Nature Communications · doi:10.1038/ncomms16027 · Cited by 379 · see citing works
Cell-cell communication: New insights and clinical implications
Su, J., Song, Y., Zhu, Z., Huang, X., Fan, J., Qiao, J., & Mao, F. (2024) · Signal Transduction and Targeted Therapy · doi:10.1038/s41392-024-01888-z · Cited by 433 · see citing works
MetaCell: analysis of single-cell RNA-seq data using K-nn graph partitions
Baran, Y., Bercovich, A., Sebé-Pedrós, A., Lubling, Y., Giladi, A., Chomsky, E., Meir, Z., Hoichman, M., Lifshitz, A., & Tanay, A. (2019) · Genome biology · doi:10.1186/s13059-019-1812-2 · Cited by 407 · see citing works
Single-cell RNA sequencing: Technical advancements and biological applications
Hedlund, E., & Deng, Q. (2017) · Molecular Aspects of Medicine · doi:10.1016/j.mam.2017.07.003 · Cited by 396 · see citing works
Bernard, V., Semaan, A., Huang, J. H., Lucas, F. A. S., Mulu, F. C., Stephens, B. M., Guerrero, P. A., Huang, Y., Zhao, J., Kamyabi, N., et al. (2018) · Clinical Cancer Research · doi:10.1158/1078-0432.ccr-18-1955 · Cited by 387 · see citing works
Comparative Analysis of Droplet-Based Ultra-High-Throughput Single-Cell RNA-Seq Systems
Zhang, X., Li, T., Liu, F., Chen, Y., Yao, J., Li, Z., Huang, Y., & Wang, J. (2018) · Molecular Cell · doi:10.1016/j.molcel.2018.10.020 · Cited by 380 · see citing works
Unravelling biology and shifting paradigms in cancer with single-cell sequencing
Baslan, T., & Hicks, J. B. (2017) · Nature reviews. Cancer · doi:10.1038/nrc.2017.58 · Cited by 377 · see citing works
Sárvári, A. K., Hauwaert, E. L. V., Markussen, L. K., Gammelmark, E., Marcher, A.-B., Ebbesen, M. F., Nielsen, R., Brewer, J. R., Madsen, J. G. S., & Mandrup, S. (2020) · Cell Metabolism · doi:10.1016/j.cmet.2020.12.004 · Cited by 368 · see citing works
zUMIs - A fast and flexible pipeline to process RNA sequencing data with UMIs
Parekh, S., Ziegenhain, C., Vieth, B., Enard, W., & Hellmann, I. (2018) · GigaScience · doi:10.1093/gigascience/giy059 · Cited by 349 · see citing works
Self-citationMeta-Analysis of Leukocyte Diversity in Atherosclerotic Mouse Aortas
Zernecke, A., Winkels, H., Cochain, C., Williams, J. W., Wolf, D., Soehnlein, O., Robbins, C. S., Monaco, C. G., Park, I., McNamara, C. A., et al. (2020) · Circulation Research · doi:10.1161/circresaha.120.316903 · Cited by 373 · see citing works
FateID infers cell fate bias in multipotent progenitors from single-cell RNA-seq data
Herman, J. S., Sagar, & Grün, D. (2018) · Nature Methods · doi:10.1038/nmeth.4662 · Cited by 363 · see citing works
Beyond bulk: a review of single cell transcriptomics methodologies and applications
Kulkarni, A., Anderson, A. G., Merullo, D. P., & Konopka, G. (2019) · Current Opinion in Biotechnology · doi:10.1016/j.copbio.2019.03.001 · Cited by 344 · see citing works
Cellular diversity in the Drosophila midbrain revealed by single-cell transcriptomics
Croset, V., Treiber, C. D., & Waddell, S. (2018) · eLife · doi:10.7554/elife.34550 · Cited by 299 · see citing works
Interpretable dimensionality reduction of single cell transcriptome data with deep generative models
Ding, J., Condon, A., & Shah, S. P. (2018) · Nature Communications · doi:10.1038/s41467-018-04368-5 · Cited by 337 · see citing works
Alternative splicing and related RNA binding proteins in human health and disease
Tao, Y., Zhang, Q., Wang, H., Yang, X., & Mu, H. (2024) · Signal Transduction and Targeted Therapy · doi:10.1038/s41392-024-01734-2 · Cited by 342 · see citing works
A single-cell analysis of the Arabidopsis vegetative shoot apex
Zhang, T., Chen, Y., & Wang, J. (2021) · Developmental Cell · doi:10.1016/j.devcel.2021.02.021 · Cited by 317 · see citing works
Direct Comparative Analyses of 10X Genomics Chromium and Smart-Seq2
Wang, X., He, Y., Zhang, Q., Ren, X., & Zhang, Z. (2021) · Genomics Proteomics & Bioinformatics · doi:10.1016/j.gpb.2020.02.005 · Cited by 316 · see citing works
Scaling by shrinking: empowering single-cell 'omics' with microfluidic devices
Prakadan, S. M., Shalek, A. K., & Weitz, D. A. (2017) · Nature Reviews Genetics · doi:10.1038/nrg.2017.15 · Cited by 327 · see citing works
The epigenetic basis of cellular heterogeneity
Carter, B. C., & Zhao, K. (2020) · Nature Reviews Genetics · doi:10.1038/s41576-020-00300-0 · Cited by 314 · see citing works
Singh, M., Al-Eryani, G., Carswell, S., Ferguson, J. M., Blackburn, J., Barton, K., Roden, D., Luciani, F., Giang Phan, T., Junankar, S., et al. (2019) · Nature Communications · doi:10.1038/s41467-019-11049-4 · Cited by 308 · see citing works
scds: computational annotation of doublets in single-cell RNA sequencing data
Bais, A. S., & Kostka, D. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btz698 · Cited by 308 · see citing works
Tutorial: guidelines for the computational analysis of single-cell RNA sequencing data
Andrews, T. S., Kiselev, V. Y., McCarthy, D. J., & Hemberg, M. (2020) · Nature Protocols · doi:10.1038/s41596-020-00409-w · Cited by 305 · see citing works
Profiling cell identity and tissue architecture with single-cell and spatial transcriptomics
Gulati, G. S., D'Silva, J. P., Liu, Y., Wang, L., & Newman, A. M. (2024) · Nature Reviews Molecular Cell Biology · doi:10.1038/s41580-024-00768-2 · Cited by 328 · see citing works
Dobnikar, L., Taylor, A. L., Chappell, J., Oldach, P., Harman, J. J., Oerton, E., Dzierzak, E. A., Bennett, M. R., Spivakov, M., & Jørgensen, H. F. (2018) · Nature Communications · doi:10.1038/s41467-018-06891-x · Cited by 303 · see citing works
Davis, R. T., Blake, K., Ma, D., Gabra, M. B. I., Hernandez, G. A., Phung, A. T., Yang, Y., Maurer, D., Lefebvre, A. E. Y. T., Alshetaiwi, H. S., et al. (2020) · Nature Cell Biology · doi:10.1038/s41556-020-0477-0 · Cited by 298 · see citing works
Prospects and challenges of multi-omics data integration in toxicology
Canzler, S., Schor, J., Busch, W., Schubert, K. A., Rolle-Kampczyk, U. E., Seitz, H., Kamp, H., von Bergen�, M., Buesen, R., & Hackermüller, J. (2020) · Archives of Toxicology · doi:10.1007/s00204-020-02656-y · Cited by 291 · see citing works
Cell type prioritization in single-cell data
Skinnider, M. A., Squair, J. W., Kathe, C., Anderson, M. A., Gautier, M., Matson, K. J. E., Milano, M., Hutson, T. H., Barraud, Q., Phillips, A. A., et al. (2021) · Nature Biotechnology · doi:10.1038/s41587-020-0605-1 · Cited by 274 · see citing works
Saviano, A., Henderson, N. C., & Baumert, T. F. (2020) · Journal of Hepatology · doi:10.1016/j.jhep.2020.06.004 · Cited by 269 · see citing works
Single-Cell Sequencing of Brain Cell Transcriptomes and Epigenomes
Armand, E. J., Li, J., Xie, F., Luo, C., & Mukamel, E. A. (2021) · Neuron · doi:10.1016/j.neuron.2020.12.010 · Cited by 241 · see citing works
A systematic evaluation of single cell RNA-seq analysis pipelines
Vieth, B., Parekh, S., Ziegenhain, C., Enard, W., & Hellmann, I. (2019) · Nature Communications · doi:10.1038/s41467-019-12266-7 · Cited by 265 · see citing works
Self-citationMapping gene regulatory networks from single-cell omics data
Fiers, M. W. E. J., Minnoye, L., Aibar, S., Bravo González-Blas, C., Kalender Atak, Z., & Aerts, S. (2018) · Briefings in Functional Genomics · doi:10.1093/bfgp/elx046 · Cited by 235 · see citing works
Ecker, J. R., Geschwind, D. H., Kriegstein, A. R., Ngai, J., Osten, P., Polioudakis, D., Regev, A., Šestan, N., Wickersham, I. R., & Zeng, H. (2017) · Neuron · doi:10.1016/j.neuron.2017.10.007 · Cited by 254 · see citing works
Tumor Cell Invasion in Glioblastoma
Vollmann‐Zwerenz, A., Leidgens, V., Feliciello, G., Klein, C. A., & Hau, P. (2020) · International Journal of Molecular Sciences · doi:10.3390/ijms21061932 · Cited by 256 · see citing works
Clonal Decomposition and DNA Replication States Defined by Scaled Single-Cell Genome Sequencing
Laks, E., McPherson, A., Zahn, H., Lai, D., Steif, A., Brimhall, J., Biele, J., Wang, B., Masud, T., Ting, J., et al. (2019) · Cell · doi:10.1016/j.cell.2019.10.026 · Cited by 247 · see citing works
Sensitive and powerful single-cell RNA sequencing using mcSCRB-seq
Bagnoli, J., Ziegenhain, C., Janjic, A., Wange, L. E., Vieth, B., Parekh, S., Geuder, J., Hellmann, I., & Enard, W. (2018) · Nature Communications · doi:10.1038/s41467-018-05347-6 · Cited by 242 · see citing works
Self-citationThe impact of amplification on differential expression analyses by RNA-seq
Parekh, S., Ziegenhain, C., Vieth, B., Enard, W., & Hellmann, I. (2016) · Scientific Reports · doi:10.1038/srep25533 · Cited by 252 · see citing works
Self-citationSCDC: bulk gene expression deconvolution by multiple single-cell RNA sequencing references
Dong, M., Thennavan, A., Urrutia, E., Li, Y., Perou, C. M., Zou, F., & Jiang, Y. (2019) · Briefings in Bioinformatics · doi:10.1093/bib/bbz166 · Cited by 255 · see citing works
Gene expression variability across cells and species shapes innate immunity
Hagai, T., Chen, X., Miragaia, R. J., Rostom, R., Gomes, T., Kunowska, N., Henriksson, J., Park, J.-E., Proserpio, V., Donati, G., et al. (2018) · Nature · doi:10.1038/s41586-018-0657-2 · Cited by 220 · see citing works
Single-Cell RNA Sequencing Analysis: A Step-by-Step Overview
Slovin, S., Carissimo, A., Panariello, F., Grimaldi, A., Bouchè, V., Gambardella, G., & Cacchiarelli, D. (2021) · Methods in molecular biology · doi:10.1007/978-1-0716-1307-8_19 · Cited by 254 · see citing works
Identifying cell populations with scRNASeq
Andrews, T. S., & Hemberg, M. (2017) · Molecular Aspects of Medicine · doi:10.1016/j.mam.2017.07.002 · Cited by 240 · see citing works
Alpern, D., Gardeux, V., Russeil, J., Mangeat, B., Meireles-Filho, A. C. A., Breysse, R., Hacker, D., & Deplancke, B. (2019) · Genome Biology · doi:10.1186/s13059-019-1671-x · Cited by 216 · see citing works
Live-seq enables temporal transcriptomic recording of single cells
Chen, W., Guillaume-Gentil, O. J., Rainer, P. Y., Gäbelein, C. G., Saelens, W., Gardeux, V., Klaeger, A., Dainese, R., Zachara, M., Zambelli, T., et al. (2022) · Nature · doi:10.1038/s41586-022-05046-9 · Cited by 257 · see citing works
Macrophage functional diversity in NAFLD — more than inflammation
Barreby, E., Chen, P., & Aouadi, M. (2022) · Nature Reviews Endocrinology · doi:10.1038/s41574-022-00675-6 · Cited by 246 · see citing works
Identification, discrimination and heterogeneity of fibroblasts
Lendahl, U., Muhl, L., & Betsholtz, C. (2022) · Nature Communications · doi:10.1038/s41467-022-30633-9 · Cited by 242 · see citing works
Revolutionizing immunology with single-cell RNA sequencing
Chen, H., Ye, F., & Guo, G. (2019) · Cellular and Molecular Immunology · doi:10.1038/s41423-019-0214-4 · Cited by 237 · see citing works
Multiplexed single-cell proteomics using SCoPE2
Petelski, A. A., Emmott, E., Leduc, A., Huffman, R. G., Specht, H., Perlman, D. H., & Slavov, N. (2021) · Nature Protocols · doi:10.1038/s41596-021-00616-z · Cited by 230 · see citing works
Observation weights unlock bulk RNA-seq tools for zero inflation and single-cell applications
Van den Berge, K., Perraudeau, F., Soneson, C., Love, M. I., Risso, D., Vert, J.-P., Robinson, M. D., Dudoit, S., & Clement, L. (2018) · Genome Biology · doi:10.1186/s13059-018-1406-4 · Cited by 228 · see citing works
Phenotypic convergence: distinct transcription factors regulate common terminal features
Konstantinides, N., Kapuralin, K., Fadil, C., Barboza, L., Satija, R., & Desplan, C. (2018) · Cell · doi:10.1016/j.cell.2018.05.021 · Cited by 226 · see citing works
Clarke, Z. A., Andrews, T. S., Atif, J., Pouyabahar, D., Innes, B. T., MacParland, S. A., & Bader, G. D. (2021) · Nature Protocols · doi:10.1038/s41596-021-00534-0 · Cited by 219 · see citing works
Microfluidics-free single-cell genomics with templated emulsification
Clark, I. C., Fontanez, K. M., Meltzer, R. H., Xue, Y., Hayford, C. E., May-Zhang, A. A., D’Amato, C., Osman, A., Zhang, J. Q., Hettige, P., et al. (2023) · Nature Biotechnology · doi:10.1038/s41587-023-01685-z · Cited by 225 · see citing works
Sun, S., Zhu, J., Ma, Y., & Zhou, X. (2019) · Genome biology · doi:10.1186/s13059-019-1898-6 · Cited by 225 · see citing works
Tutorial: guidelines for the experimental design of single-cell RNA sequencing studies
Lafzi, A., Moutinho, C., Picelli, S., & Heyn, H. (2018) · Nature Protocols · doi:10.1038/s41596-018-0073-y · Cited by 218 · see citing works
Self-citationGuo, M., Du, Y., Gokey, J. J., Ray, S., Bell, S. M., Adam, M., Sudha, P., Perl, A. T., Deshmukh, H. S., Potter, S. S., et al. (2018) · Nature Communications · doi:10.1038/s41467-018-07770-1 · Cited by 201 · see citing works
Single-cell RNA-seq reveals hidden transcriptional variation in malaria parasites
Reid, A. J., Talman, A. M., Bennett, H. M., Gomes, A. R., Sanders, M. J., Illingworth, C. J. R., Billker, O., Berriman, M., & Lawniczak, M. (2018) · eLife · doi:10.7554/elife.33105 · Cited by 209 · see citing works
Advances and Opportunities in Single-Cell Transcriptomics for Plant Research
Seyfferth, C., Renema, J., Wendrich, J. R., Eekhout, T., Seurinck, R., Vandamme, N., Blob, B., Saeys, Y., Helariutta, Y., Birnbaum, K. D., et al. (2021) · Annual Review of Plant Biology · doi:10.1146/annurev-arplant-081720-010120 · Cited by 216 · see citing works
Gao, S., Yan, L., Wang, R., Li, J., Yong, J., Zhou, X., Wei, Y., Wu, X., Wang, X., Fan, X., et al. (2018) · Nature Cell Biology · doi:10.1038/s41556-018-0105-4 · Cited by 208 · see citing works
Single-Cell (Multi)omics Technologies
Chappell, L., Russell, A. J. C., & Voet, T. (2018) · Annual Review of Genomics and Human Genetics · doi:10.1146/annurev-genom-091416-035324 · Cited by 206 · see citing works
Comparison of clustering tools in R for medium-sized 10x Genomics single-cell RNA-sequencing data
Freytag, S., Tian, L., Lönnstedt, I., Ng, M., & Bahlo, M. (2018) · F1000Research · doi:10.12688/f1000research.15809.1 · Cited by 220 · see citing works
Hu, Z., Artibani, M., Alsaadi, A., Wietek, N. C., Morotti, M., Shi, T., Zhong, Z., González, L. S., El-Sahhar, S., Carrami, E. M., et al. (2020) · Cancer Cell · doi:10.1016/j.ccell.2020.01.003 · Cited by 188 · see citing works
Barcoded solid-phase RNA capture for Spatial Transcriptomics profiling in mammalian tissue sections
Salmén, F., Ståhl, P. L., Mollbrink, A., Navarro, J. F., Vicković, S., Frisén, J., & Lundeberg, J. (2018) · Nature Protocols · doi:10.1038/s41596-018-0045-2 · Cited by 193 · see citing works
- Authors
- Ziegenhain, C., Vieth, B., Parekh, S., Reinius, B., Guillaumet-Adkins, A., Smets, M., Leonhardt, H., Heyn, H., Hellmann, I., & Enard, W.
- Affiliations
- Ludwig-Maximilians-Universität München (DE)
- Ludwig Cancer Research (BE)
- Universitat Pompeu Fabra (ES)
- Center for Integrated Protein Science Munich (DE)
- Corresponding author
- Enard, Wolfgang
- ORCID
- Venue
- Molecular Cell
- Publisher
- Elsevier BV
- Published
- 2017-02
- Volume
- 65
- Issue
- 4
- Pages
- 631-643.e4
- ISSN
- 1097-2765 (ISSN-L)
- OpenAlex
- W2887910031
- Cited by
- 1,494
- References
- 62
- Citations by year
- 2026: 86 · 2025: 156 · 2024: 158 · 2023: 152 · 2022: 160 · 2021: 219 · 2020: 197 · 2019: 174 · 2018: 125 · 2017: 60 · 2016: 3 · 2015: 1 · 2012: 1
- FWCI
- 63.41
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- Molecular Cell (publishedVersion)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- Repositori digital de la UPF (Universitat Pompeu Fabra) (publishedVersion, other-oa)
- RECERCAT (Consorci de Serveis Universitaris de Catalunya) (submittedVersion, other-oa)
- Annals of Hematology
- Funders
- Deutsche Forschungsgemeinschaft (DFG) (SFB1243, A01/A14/A15)
- Boehringer Ingelheim Fonds
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Single-cell and spatial transcriptomics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Phylogenetic Studies
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene expression and cancer classification
- Keywords
- single-cell RNA sequencing, mouse embryonic stem cells, Drop-seq, SmartSeq2, SMART-seq, unique molecular identifiers, sequencing depth, power simulation, cost efficiency, transcriptome quantification
- MeSH terms
- Animals, Base Sequence, Cell Line, Computer Simulation, Cost-Benefit Analysis, RNA, Sequence Analysis, RNA, Models, Economic, Mice, Embryonic Stem Cells, Single-Cell Analysis, High-Throughput Nucleotide Sequencing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-11.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-11.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
