scPower accelerates and optimizes the design of multi-sample single cell transcriptomic studies

May combine two claims.

For large mean values, eQTL power is estimated analytically; for small mean values, the approximation can be imprecise and simulations are used that take the discrete nature of scRNA-seq into account.

“For large mean values, the power is estimated analytically, for small mean values, this approximation can be imprecise and instead simulations are used that take the discrete nature of scRNA-seq into account.”

May combine two claims.

Both the expression probability and the DE/eQTL power depend on the mean and dispersion of expression levels of the gene.

“In addition, both the expression probability and the DE/eQTL power depend on the mean and dispersion of expression levels of gene .”

May combine two claims.

For this parameter combination and prior, the DE power would reach 98% for all DE genes of the study, but only 74% are likely to be expressed.

“For this parameter combination and prior, the DE power would reach even 98% for all DE genes of the study, however, only 74% are likely to be expressed.”

May combine two claims.

Similar detection ranges are found for the comparison of other CLL subtypes in the same study, while the detection power in a study of systemic sclerosis vs control were much lower with values up to 30%.

“Similar detection ranges are found for the comparison of other CLL subtypes in the same study, while the detection power in a study of systemic sclerosis vs control were much lower with values up to 30% (Supplementary Fig. S10).”

May combine two claims.

Smart-seq2 data showed zero-inflation on the single cell level, but aggregation to pseudobulk removed the excess of zero values.

“In contrast to the other technologies, the Smart-seq2 data showed zero-inflation on the single cell level (see also68,69), but aggregation to pseudobulk removed the excess of zero values.”

• cites [68] Chen, W. et al. UMI-count modeling and differential expression analysis for single-cell RNA sequencing. Genome Biol. 19, 70 (2018). sourceNot checked yet

  Not checked in the free run.

• cites [69] Svensson, V. Droplet scRNA-seq is not zero-inflated. https://doi.org/10.1101/582064. sourceNot checked yet

  Not checked in the free run.

May combine two claims.

This maximum overall detection power of 74% is reached with 3000 cells per cell type and individual, a total balanced sample size of 20, and FDR adjusted p values.

“This power is reached with the experimental parameters of 3000 cells per cell type and individual, a total balanced sample size of 20, i.e., 10 individuals per group, and FDR adjusted p values.”

May combine two claims.

scPower's analytic method can efficiently test a multitude of design options, making it suitable for the optimization of experimental parameters.

“In contrast to simulation-based methods, our analytic method can efficiently test a multitude of design options, making it suitable for the optimization of experimental parameters.”

May combine two claims.

Overall, DE power increases with higher number of measured cells and higher sample sizes, while expression probability is mainly influenced by the number of measured cells.

“Overall, the DE power increases with higher number of measured cells and higher sample sizes, while the expression probability is mainly influenced by the number of measured cells.”

Probably not a substantial claim: may not pass S3 (load-bearing).

powsimR and muscat differ slightly, caused by different modelling assumptions.

“Furthermore, powsimR and muscat differ slightly, caused by different modelling assumptions.”

Probably not a substantial claim: may not pass S3 (load-bearing).

For the DE example, the percentage threshold is 50% due to the balanced DE design.

“For our DE example, this results in a percentage threshold of 50% due to the balanced DE design.”

May combine two claims.

The observed power also depends on analysis choices such as the DE method, with scPower estimates being most accurate when using edgeR.

“Running simulations with different DE methods showed that the observed power also depends on analysis choices such as the DE method with scPower estimates being most accurate when using edgeR (Supplementary Fig. S13).”

Probably not a substantial claim: may not pass S3 (load-bearing).

The installation of scPower is easier due to less dependencies: 11 dependencies of scPower vs. 82 dependencies of powsimR and 28 dependencies of muscat.

“In addition the installation of scPower is easier due to less dependencies: 11 dependencies of scPower vs. 82 dependencies of powsimR and 28 dependencies of muscat.”

The quote only partly states this claim.

For the DE study with the specific prior combination and a budget of 10,000€, the optimal parameters are measuring 1200 cells per individual in 4 samples with a read depth of 30,000.

“For the DE study with this specific prior combination, the optimal parameters are measuring 1200 cells in 4 samples with a read depth of 30,000.”

• AssumptionThe authors state this as an assumption: The counts of each gene in each cell type follow a negative binomial distribution.
• AssumptionThe authors state this as an assumption: The sum of gene counts follows a negative binomial distribution with parameters altered by the number of cells per cell type and donor.
• AssumptionThe authors state this as an assumption: The distribution of mean values over all genes in a cell type is a mixture of a zero component and two left-censored gamma distributions.
• AssumptionThe authors state this as an assumption: The relationship between mean UMI counts per cell and the gamma parameters is linear.
• AssumptionThe authors state this as an assumption: The number of transcriptome mapped reads is linearly related to the logarithm of the mean UMI counts per cell, with increasing read depth leading to saturation of UMIs.
• AssumptionThe authors state this as an assumption: Overall detection power is the product of expression probability and power to detect DEGs/eQTLs, as both are conditionally independent given expression mean.

May combine two claims.

For simulation-based methods, memory requirements increase with larger sample size and numbers of cells, leading for example for 20 samples and 3000 cells per sample to 48GB used memory for powsimR and 35GB used memory for muscat compared to the parameter-independent requirements of scPower of few MB.

“For the simulation-based methods the memory requirements increase with larger sample size and numbers of cells, leading for example for 20 samples and 3000 cells per sample to 48GB used memory for powsimR and 35GB used memory for muscat compared to the parameter-independent requirements of scPower of few MB (Fig. 3h).”

May combine two claims.

Due to the fixed budget, measuring more cells per individual goes hand in hand with measuring fewer samples, which decreases DE power.

“but due to the fixed budget this goes hand in hand with measuring less samples which decreases the DE power.”

May combine two claims.

The Blueprint eQTL data set also contains eQTLs from monocytes where the same trend is observed and a maximal detection power of 65% was found.

“The Blueprint eQTL data set also contains eQTLs from monocytes where we observe the same trend and found a maximal detection power of 65% (Supplementary Fig. S11).”

May combine two claims.

Increasing the number of cells per individual increases both the expression probability and the eQTL power by shifting the expression mean of the pseudobulk counts to higher values.

“Increasing the number of cells per individual increases both the expression probability and the eQTL power by shifting the expression mean of the pseudo bulk counts to higher values.”

May combine two claims.

The analytic calculations of scPower are orders of magnitude faster than the simulation-based approaches: calculations for Fig. 3c-e took 8 days for powsimR, 3 days for muscat and less than a minute for scPower.

“The analytic calculations of scPower are orders of magnitude faster than the simulation-based approaches: calculations for Fig. 3c–e took 8 days for powsimR, 3 days for muscat and less than a minute for scPower (Fig. 3g).”

The quote only partly states this claim.

For the eQTL study with the specific prior combination and a budget of 30,000€, the optimal parameters are measuring 1500 cells per individual in 242 samples with a read depth of 10,000.

“For the eQTL study with this specific prior combination, the optimal parameters are measuring 1500 cells in 242 samples with a read depth of 10,000.”

• AssumptionThe authors state this as an assumption: The counts of each gene in each cell type follow a negative binomial distribution.
• AssumptionThe authors state this as an assumption: The sum of gene counts follows a negative binomial distribution with parameters altered by the number of cells per cell type and donor.
• AssumptionThe authors state this as an assumption: The distribution of mean values over all genes in a cell type is a mixture of a zero component and two left-censored gamma distributions.
• AssumptionThe authors state this as an assumption: The relationship between mean UMI counts per cell and the gamma parameters is linear.
• AssumptionThe authors state this as an assumption: The number of transcriptome mapped reads is linearly related to the logarithm of the mean UMI counts per cell, with increasing read depth leading to saturation of UMIs.
• AssumptionThe authors state this as an assumption: Overall detection power is the product of expression probability and power to detect DEGs/eQTLs, as both are conditionally independent given expression mean.

May combine two claims.

For Drop-seq lung data, similar trends as for 10X PBMC are observed, with cells per individual being the major determinant of power.

“For the Drop-seq lung data we observe similar trends as for the 10X PBMC data set, with the number of cells per individual being the major determinant of power.”

May combine two claims.

Using FWER adjustment for DE power changes observed overall power but leads to very similar optimal parameter combinations and the same trends overall.

“Using FWER adjustment for DE power changes the observed overall power, but leads to very similar optimal parameter combinations and the same trends overall (Supplementary Fig. S17).”

May combine two claims.

For Smart-seq2, in contrast to 10X Genomics and Drop-seq, the optimal number of reads per cell is much higher, and cells per individual and sample size are only increased at higher budgets for both prototypic and data-driven priors.

“In contrast to 10X Genomics and Drop-seq, the optimal number of reads per cell is much higher and the number of cells per individual and sample size is only increased at higher budgets for both the prototypic and data driven priors.”

Classified as probably not a substantial claim.

Small effect sizes require large sample sizes.

“where small effect sizes require large sample sizes.”

May combine two claims.

For eQTL scenarios, increasing sample size first is more beneficial than increasing read depth, which remains relatively low at 10,000 reads per cell.

“For eQTL scenarios, increasing the sample size first is more beneficial than increasing the read depth (Fig. 5b,d), which remains relatively low (10,000 reads per cell).”

May combine two claims.

Analytical models require only a small amount of memory independent of assessed parameters, while simulation of larger sample sizes leads to increasing memory usage.

“In addition, analytical models require only a small amount of memory independent of the assessed experimental parameters, while simulation of data sets with larger sample sizes lead to increasing memory usage.”

May combine two claims.

The optimal sample size is mostly dependent on effect size; low effect sizes require large sample sizes, and optimal settings with high sample size typically lead to low sequencing depth and relatively low number of cells.

“The optimal sample size is mostly dependent on the effect size, with low effect sizes requiring large sample sizes and consequently optimal setting with high sample size typically lead to low sequencing depth and relatively low number of cells.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Priors affect the optimal design.

“In general, priors affect the optimal design and should therefore be selected carefully.”

May combine two claims.

The identified optimal thresholds are low and increase the number of detectable genes.

“The identified optimal thresholds are low and increase the number of detectable genes.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The choice of threshold on reads required for a gene to be called expressed influences the choice of optimal parameters.

“The choice of a threshold on the number of reads required for a gene to be called expressed also influences the choice of optimal parameters.”

May combine two claims.

These percentages correspond to 1-10 reads per million sequenced, translating to 1-5 UMI counts for a median of around 5000 UMI counts per cell in our data set.

“These percentages correspond to 1–10 reads per million sequenced, which translates to 1–5 UMI counts for a median of around 5000 UMI counts per cell in our data set.”

May combine two claims.

For genes with small mean values, normalization might not be effective and analytical F-test power is overestimated.

“However, for genes with small mean values, i.e., only very few non-zero counts, this normalization might not be effective and the power is overestimated by the analytical power calculation based on the F-test.”

May combine two claims.

Taylor approximation of dispersion parameter was not accurate enough, so numerical optimization was used.

“Since the Taylor approximation of the dispersion parameter45 was not accurate enough, we used instead a numerical optimization.”

• cites [45] Law, C. W., Chen, Y., Shi, W. & Smyth, G. K. voom: Precision weights unlock linear model analysis tools for RNA-seq read counts. Genome Biol. 15, R29 (2014). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

A similar standard deviation was also observed in the pilot data.

“A similar standard deviation was also observed in the pilot data.”

May combine two claims.

For the DEseq dispersion model, a linear relationship with read depth was observed, which was not visible for Drop-seq and 10X Genomics.

“We observed for the parameter of the DEseq dispersion model a linear relationship with the read depth, which was not visible for Drop-seq and 10X Genomics.”

May combine two claims.

In scRNA-seq experiments, individual cells are typically not sequenced to saturation, leading to sparse count matrices, where only highly expressed genes are detected with counts greater than zero.

“In scRNA-seq experiments, individual cells are typically not sequenced to saturation, leading to sparse count matrices, where only highly expressed genes are detected with counts greater than zero.”

• cites [34] Wu, A. R. et al. Quantitative assessment of single-cell RNA-sequencing methods. Nat. Methods 11, 41–46 (2014). sourceNot checked yet

  Not checked in the free run.

• cites [35] Svensson, V. et al. Power analysis of single-cell RNA-sequencing experiments. Nat. Methods 14, 381–387 (2017). sourceNot checked yet

  Not checked in the free run.

• cites [36] Ziegenhain, C. et al. Comparative analysis of single-cell RNA sequencing methods. Mol. Cell 65, 631–643.e4 (2017). sourceNot checked yet

  Not checked in the free run.

May combine two claims.

The overall number of transcripts as well as the number of transcripts of individual genes can be highly cell type specific.

“In addition, the overall number of transcripts as well as the number of transcripts of individual genes can be highly cell type specific33.”

• cites [33] Monaco, G. et al. RNA-Seq signatures normalized by mRNA abundance allow absolute deconvolution of human immune cell types. Cell Rep. 26, 1627–1640.e7 (2019). sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim. May combine two claims.

A first analytic exploration of different experimental designs for single cell eQTL studies showed the importance of optimizing parameters for a restricted budget, as shallow sequencing of more samples can increase the effective sample size.

“A first analytic exploration of different experimental designs for single cell eQTL studies showed the importance of the optimizing parameters for a restricted budget50, as shallow sequencing of more samples can increase the effective sample size.”

• cites [50] Mandric, I. et al. Optimized design of single-cell RNA sequencing experiments for cell-type-specific eQTL analysis. Nat. Commun. 11, 5504 (2020). sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Potential weaker effect sizes that would be identified with larger sample sizes could not be considered in the priors, which leads to a low required sample size for the power estimation.

“Potential weaker effect sizes that would be identified with larger sample sizes could not be considered in the priors, which leads to a low required sample size for the power estimation.”

May combine two claims.

Smaller absolute fold changes in the systemic sclerosis study decrease the DE power and therefore also the overall detection power.

“Smaller absolute fold changes in this study decrease the DE power and therefore also the overall detection power.”

May combine two claims.

The first analytic exploration provided no generalizable tool that can be applied on other data sets, is missing an exact power estimation based on effect size priors, and is not applicable for DEG analysis.

“However, the analysis provided no generalizable tool that can be applied on other data sets and is missing an exact power estimation based on effect size priors. Furthermore, it is not applicable for DEG analysis.”

May combine two claims.

All training data sets could be modelled by negative binomial distributions after pseudobulk transformation and showed no evidence of zero inflation.

“We verified that all our training data sets could be modelled by negative binomial distributions after pseudobulk transformation and found no evidence of zero inflation (Supplementary Table S4).”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

In order to identify a gene as an DEG/eQTL gene, it must be both expressed and exceeding the significance cutoff.

“In order to identify a gene as an DEG/eQTL gene, it must be both expressed and exceeding the significance cutoff.”

May combine two claims.

This introduces a dependency between eQTL power and the expression mean, and thus eQTL power is considered conditional on the mean.

“This introduces a dependency between the eQTL power and the expression mean and thus eQTL power is considered conditional on the mean.”

May combine two claims.

Handling more complex designs is not easily accomplished with simulation based methods, but could be achieved with analytical power analysis methods.

“In addition, handling more complex designs is not easily accomplished with simulation based methods, but could be achieved with analytical power analysis methods.”

Probably not a substantial claim: may not pass S4 (based).

There currently exists no power estimation method for single cell eQTL that explicitly accounts for specific effect size priors.

“Contrary to DE analysis, there currently exists no power estimation method for single cell eQTL that explicitly accounts for specific effect size priors.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The expression cutoff should correspond to the percentage of samples in the smaller group.

“We followed the recommendation of edgeR75 that the expression cutoff should correspond to the percentage of samples in the smaller group.”

• cites [75] Chen, Y., Lun, A. T. L. & Smyth, G. K. From reads to genes to pathways: differential expression analysis of RNA-Seq experiments using Rsubread and the edgeR quasi-likelihood pipeline. F1000Res. 5, 1438 (2016).Not checked yet

  Not checked in the free run.

May combine two claims.

Under an unconfounded experimental design with batches containing both controls and cases, batch effects can be removed by adding a batch covariate to the regression model, and this increases power compared to non-batch corrected analyses.

“However, under the assumptions of an unconfounded experimental design with batches containing both controls and cases, batch effects can be removed by adding a batch covariate to the regression model71. This increases the power compared to non-batch corrected analyses72–74.”

• cites [71] Chen, W. et al. A comparison of methods accounting for batch effects in differential expression analysis of UMI count based single cell RNA sequencing. Comput. Struct. Biotechnol. J. 18, 861–873 (2020). sourceNot checked yet

  Not checked in the free run.

• cites [72] Hernández, A. V., Steyerberg, E. W. & Habbema, J. D. F. Covariate adjustment in randomized controlled trials with dichotomous outcomes increases statistical power and reduces sample size requirements. J. Clin. Epidemiol. 57, 454–460 (2004). sourceNot checked yet

  Not checked in the free run.

• cites [73] Stegle, O., Parts, L., Piipari, M., Winn, J. & Durbin, R. Using probabilistic estimation of expression residuals (PEER) to obtain increased power and interpretability of gene expression analyses. Nat. Protoc. 7, 500–507 (2012). sourceNot checked yet

  Not checked in the free run.

• cites [74] Kahan, B. C., Jairath, V., Doré, C. J. & Morris, T. P. The risks and rewards of covariate adjustment in randomized trials: an assessment of 12 outcomes from 8 studies. Trials 15, 139 (2014). sourceNot checked yet

  Not checked in the free run.

• AssumptionThe authors state this as an assumption: An unconfounded experimental design with batches containing both controls and cases is assumed for batch effect correction.

May combine two claims.

A conservative power estimation can be achieved by setting the expected cell type frequency to the frequency of the smaller group, which represents a good lower bound estimation, especially in cases with small sample sizes.

“In this case, a conservative power estimation can be achieved by setting the expected cell type frequency to the frequency of the smaller group. This represents a good lower bound estimation, especially in cases with small sample sizes.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

For the eQTL example, an eQTL with a minor allele frequency of 0.05 is considered, which is a common lower threshold for genetic variants tested for associations.

“For the eQTL example, we consider an eQTL with a minor allele frequency of 0.05, which is a common lower threshold for genetic variants tested for associations.”

May combine two claims.

The higher cost of Smart-seq2 allows measuring much fewer cells, even though a higher number of samples and cells would be beneficial.

“This allows only to measure much fewer cells, even though a higher number of samples and cells would be beneficial.”

May combine two claims.

Smart-seq2 experiments are not less powerful per se; their significantly higher cost in multi-sample settings leads to less powerful designs when restricting budget.

“In general, we observe that Smart-seq2 experiments are not less powerful per se, but the significantly higher cost in the multi-sample setting leads to less powerful designs when restricting the budget.”

May combine two claims.

scPower's power analysis framework is applicable to data sets for other tissues besides PBMC and other single cell technologies besides 10X Genomics.

“Our power analysis framework is applicable to data sets for other tissues besides PBMC and to other single cell technologies besides 10X Genomics.”

May combine two claims.

scPower generalizes across different tissues and scRNAseq technologies and provides means to design experiments maximizing biological discoveries.

“Our model generalizes across different tissues and scRNAseq technologies and provides the means to easily design experiments that maximize the number of biological discoveries.”

May combine two claims.

Strong effects can be detected with relatively few samples, while detection of expression requires deeper sequencing.

“indicating that strong effects can be detected with relatively few samples, while the detection of expression requires deeper sequencing.”

May combine two claims.

The doublet detection and donor identification of Demuxlet was validated using PBMC data by comparing sex-specific gene expression with assigned donor sex, showing high concordance after doublet removal, including for overloaded run 5.

“We validated the doublet detection and donor identification of Demuxlet using our PBMC data set by comparing the expression of sex specific genes with the sex of the assigned donor (Supplementary Fig. S2b) and found high concordance after doublet removal, also for run 5, which was overloaded with 25,000 cells.”

May combine two claims.

The speed of analytical models enables evaluation of many experimental design options in a short time and identification of optimal parameters for a limited budget.

“This transformatively enables the evaluation of many experimental design options in a short time and thus to identify optimal experimental parameters for a limited budget.”

Probably not a substantial claim: may not pass S3 (load-bearing).

These recommendations match our optimally determined parameters.

“These recommendations match our optimal determined parameters.”

May combine two claims.

In a systematic analysis of spike-in expression, measurement accuracy is not strongly dependent on sequencing depth and is consistently high for 10,000 reads per cell.

“However, in a systematic analysis of spike-in expression it has been shown that the accuracy of the measurements is not strongly dependent on the sequencing depth and consistently high for a read depth of 10,000 reads per cell35.”

• cites [35] Svensson, V. et al. Power analysis of single-cell RNA-sequencing experiments. Nat. Methods 14, 381–387 (2017). sourceNot checked yet

  Not checked in the free run.

May combine two claims.

scPower provides gene-level and overall power estimates based on prior data and is a generalizable tool for analytic power analysis of single cell DE and eQTL studies.

“In contrast, our approach provides gene level and overall power estimates based on prior data and we provide a generalizable tool for analytic power analysis of single cell DE and eQTL studies.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The negative binomial regression model for pseudobulk is currently the most powerful method for assessing individual-level differential expression.

“While the (negative binomial) regression model for pseudobulk is currently the most powerful method for assessing individual level differential expression43, it requires a discrete cell type definition and our approach is tightly linked to it.”

• cites [43] Crowell, H. L. et al. muscat detects subpopulation-specific state transitions from multi-sample multi-condition single-cell transcriptomics data. Nat. Commun. 11, 6077 (2020). sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

Shallow sequencing of more cells has been recommended for extracting gene expression programs required for annotations, because it achieves equal accuracy as deeper sequencing of fewer cells.

“Shallow sequencing of more cells has been recommended for extracting the gene expression programs required for annotations, because it has achieved equal accuracy as deeper sequencing of fewer cells50,82.”

• cites [50] Mandric, I. et al. Optimized design of single-cell RNA sequencing experiments for cell-type-specific eQTL analysis. Nat. Commun. 11, 5504 (2020). sourceNot checked yet

  Not checked in the free run.

• cites [82] Heimberg, G., Bhatnagar, R., El-Samad, H. & Thomson, M. Low dimensionality in gene expression data enables the accurate extraction of transcriptional programs from shallow sequencing. Cell Syst. 2, 239–250 (2016). sourceNot checked yet

  Not checked in the free run.

May combine two claims.

A first analytic investigation of power optimization in single cell eQTL studies has been done but suffered from several limitations.

“A first analytic investigation of power optimization in single cell eQTL studies50 has been done, but suffered from several limitations.”

• cites [50] Mandric, I. et al. Optimized design of single-cell RNA sequencing experiments for cell-type-specific eQTL analysis. Nat. Commun. 11, 5504 (2020). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

scPower can be combined with specific power analysis tools for cell type annotation to ensure sufficient power.

“To ensure sufficient power for cell type annotation, our framework scPower can be combined with specific power analysis tools for cell type annotation38,82.”

• cites [38] Abrams, D., Kumar, P., Karuturi, R. K. M. & George, J. A computational method to aid the design and analysis of single cell RNA-seq experiments for cell type identification. BMC Bioinforma. 20, 275 (2019). sourceNot checked yet

  Not checked in the free run.

• cites [82] Heimberg, G., Bhatnagar, R., El-Samad, H. & Thomson, M. Low dimensionality in gene expression data enables the accurate extraction of transcriptional programs from shallow sequencing. Cell Syst. 2, 239–250 (2016). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Best practice guidelines for differential gene expression with RNA-seq recommend cutoffs that remove between 19 and 33% of lowly expressed genes, depending on the analysis pipeline.

“For this reason, best practice guidelines for differential gene expression with RNA-seq recommend cutoffs that remove between 19 and 33% of lowly expressed genes, depending on the analysis pipeline84.”

• cites [84] SEQC/MAQC-III Consortium. A comprehensive assessment of RNA-seq accuracy, reproducibility and information content by the Sequencing Quality Control Consortium. Nat. Biotechnol. 32, 903–914 (2014). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The human cell atlas project aims to build a reference map of healthy human cells by iteratively sampling cells with increasing resolution.

“The human cell atlas project has made it its goal to build a reference map of healthy human cells by iteratively sampling the cells with increasing resolution61,85.”

• cites [61] Regev, A. et al. The human cell atlas. Elife 6, e27041 (2017). sourceNot checked yet

  Not checked in the free run.

• cites [85] Regev, A. et al. The human cell atlas white paper. arXiv [q-bio.TO] (2018).Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

The gene expression probability model is cell type specific and has to be fitted based on realistic pilot data.

“Our gene expression probability model is cell type specific and has to be fitted based on realistic pilot data.”

Probably not a substantial claim: may not pass S3 (load-bearing).

We did not address power for detection of variance QTLs from scRNAseq data due to lack of data-driven priors for effect sizes.

“Lastly, we did not address the power for the detection of variance QTLs (quantitative trait loci associated with gene expression variance across cells) from scRNAseq data48 due to the lack of data driven priors for the effect sizes.”

• cites [48] Sarkar, A. K. et al. Discovery and characterization of variance QTLs in human induced pluripotent stem cells. PLoS Genet 15, e1008045 (2019). sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

scPower's experimental design recommendations are optimized for differential expression between individuals.

“When using our approach, the user should keep in mind that our experimental design recommendations are optimized for differential expression between individuals.”

May combine two claims.

The pseudobulk negative binomial regression model requires a discrete cell type definition, and scPower is tightly linked to it.

“While the (negative binomial) regression model for pseudobulk is currently the most powerful method for assessing individual level differential expression43, it requires a discrete cell type definition and our approach is tightly linked to it.”

• cites [43] Crowell, H. L. et al. muscat detects subpopulation-specific state transitions from multi-sample multi-condition single-cell transcriptomics data. Nat. Commun. 11, 6077 (2020). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Some eQTL analyses of bulk RNAseq data advocate using >0 reads.

“In our examples we used a threshold of >10 and >3 reads, however, some eQTL analyses of bulk RNAseq data advocate using >0 reads70.”

• cites [70] Lappalainen, T. et al. Transcriptome and genome sequencing uncovers functional variation in humans. Nature 501, 506–511 (2013). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Co-expression analysis requires a high number of quantified genes per cell, especially for cell type specific co-expression and comparison between individuals.

“For instance, co-expression analysis requires a high number of quantified genes per cell, especially when one is interested in cell type specific co-expression and comparison of such co-expression relations between individuals.”

May combine two claims.

Actual experiments might result in higher doublet rates and lower number of usable cells.

“Thus, actual experiments might result in higher doublet rates and lower number of usable cells.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

The human cell atlas will create high-quality priors that will broaden the applicability of scPower.

“This will create high quality priors that will further broaden the applicability of scPower.”

May combine two claims.

The model can be applied to data generated with 10X Genomics, Drop-seq, and Smart-seq2, and is likely applicable to similar technology platforms.

“We have shown that our model can be applied to data generated with 10X Genomics, Drop-seq and Smart-seq2 and we are confident that it is applicable to similar technology platforms.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The power to annotate new rare cell types by clustering analysis of scRNA-seq data might have different optimal parameters.

“Furthermore, the power to annotate new rare cell types by clustering analysis of scRNA-seq data might have different optimal parameters38.”

• cites [38] Abrams, D., Kumar, P., Karuturi, R. K. M. & George, J. A computational method to aid the design and analysis of single cell RNA-seq experiments for cell type identification. BMC Bioinforma. 20, 275 (2019). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

Other applications might result in very different optimal experimental designs.

“Other applications might result in very different optimal experimental designs.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

scPower will provide the foundation for building rational experimental design of interindividual gene expression comparisons with cell type resolution across a wide range of organ systems.

“We are convinced that scPower will provide the foundation for building rational experimental design of interindividual gene expression comparisons with cell type resolution across a wide range of organ systems.”

Classified as probably not a substantial claim.

DE/eQTL power is computed using a significance threshold corrected for multiple testing, controlling either the family-wise error rate (FWER) or the false discovery rate (FDR).

“DE/eQTL power is computed using a significance threshold corrected for multiple testing, controlling either the family-wise error rate (FWER) or the false discovery rate (FDR).”

Classified as probably not a substantial claim.

Lung cell type specific priors were obtained from a DE study of freshly isolated airway epithelial cells of asthma patients and healthy controls55.

“Lung cell type specific priors were obtained from a DE study of freshly isolated airway epithelial cells of asthma patients and healthy controls55.”

• cites [55] Nicodemus-Johnson, J. et al. DNA methylation in lung cells is associated with asthma endotypes and genetic risk. JCI Insight 1, e90151 (2016). sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

A DE study analyzing age-dependent gene regulation in human pancreas56 was used to get pancreas cell type specific priors.

“A DE study analyzing age-dependent gene regulation in human pancreas56 was used to get pancreas cell type specific priors.”

• cites [56] Arda, H. E. et al. Age-dependent pancreatic gene regulation reveals mechanisms governing human β cell function. Cell Metab. 23, 909–920 (2016). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The authors used FDR adjustment for DE power and followed the GTEx consortium approach based on FWER adjustment for eQTL power.

“We used FDR adjustment for the DE power and followed the approach of the GTEx consortium63 based on FWER adjustment for the eQTL power.”

• cites GTEx consortium63Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

For eQTLs, the GTEx consortium approach assumes that for each gene on average 10 independent (uncorrelated) SNPs are tested in a genome-wide cis eQTL analysis.

“For eQTLs we followed the approach of the GTEX consortium63, which assumes that for each gene on average 10 independent (uncorrelated) SNPs are tested in a genome-wide cis eQTL analysis.”

• cites GTEx consortium63Not checked yet

  Not checked in the free run.

May combine two claims.

The expected number of true positives is derived from the power, and summing gene-wise power yields the expected number of significant DEGs/eQTLs.

“The expected number of true positives is directly derived from the power we reach for . Summing up the gene-wise power (at ) yields the expected number of significant DEGs/eQTLs .”

May combine two claims.

FDR correction depends on the p value distribution of all genes, unlike Bonferroni correction which depends only on the number of tests.

“In contrast to the Bonferroni correction, which depends only on the number of tests, the FDR correction depends on the p value distribution of all genes.”

Probably not a substantial claim: may not pass S3 (load-bearing).

edgeR recommends that the expression cutoff should correspond to the percentage of samples in the smaller group.

“For the population level, we followed in most of our analyses the recommendation of edgeR75 that the expression cutoff should correspond to the percentage of samples in the smaller group.”

• cites [75] Chen, Y., Lun, A. T. L. & Smyth, G. K. From reads to genes to pathways: differential expression analysis of RNA-Seq experiments using Rsubread and the edgeR quasi-likelihood pipeline. F1000Res. 5, 1438 (2016).Not checked yet

  Not checked in the free run.

May combine two claims.

Because the analytic method outputs power without computing p values, the FDR correction cannot be applied directly and the method of Jung is used.

“As our analytic method outputs the power without computing the p values, we can not apply the FDR correction directly and use the method of Jung.”

Probably not a substantial claim: may not pass S3 (load-bearing).

More complex experimental designs can be addressed using the method of reference 95.

“More complex experimental designs can be addressed using the method of95.”

• cites [95] Lyles, R. H., Lin, H.-M. & Williamson, J. M. A practical approach to computing power for generalized linear models with nominal, count, or ordinal responses. Stat. Med. 26, 1632–1648 (2007). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The three main components of the statistical framework evaluated in the comparison were expression probability, power, and overall detection power.

“The three main components of our statistical framework were evaluated in the comparison, the expression probability (by comparing the number of expressed genes), the power (here according to the definition of powsimR, i.e., the power of all genes expressed in the simulated data) and the overall detection power.”

May combine two claims.

An important limitation of simulation-based frameworks is that the number of expressed genes in the simulation tools can never be larger than the number of expressed genes in the training data set, while scPower can approximate expression of unseen genes with smaller mean values and estimate more expressed genes than seen in pilot data.

“An important limitation of the simulation based frameworks is here that the number of expressed genes in the simulation tools can never be larger than the number of expressed genes in the training data set, while scPower can also approximate expression of unseen genes with smaller mean values and so estimate more expressed genes than seen in the pilot data.”

Classified as probably not a substantial claim.

To validate the model, the DE power estimations of the framework were compared with two simulation-based tools, powsimR and muscat40,43.

“To validate our model, we compared the DE power estimations of our framework with two simulation-based tools, called powsimR and muscat40,43.”

• cites powsimRNot checked yet

  Not checked in the free run.

• cites muscat40,43Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

eQTL effect and sample sizes were used from the Blueprint study on bulk RNA-seq of FACS sorted Monocytes and T cells57.

“We used eQTL effect and sample sizes from the Blueprint study on bulk RNA-seq of FACS sorted Monocytes and T cells57.”

• cites [57] Chen, L. et al. Genetic drivers of epigenetic and transcriptional variation in human immune. Cells Cell 167, 1398–1414.e24 (2016). sourceNot checked yet

  Not checked in the free run.

May combine two claims.

We chose a cutoff of mean count < 5 and estimate power for genes with smaller mean values based on simulation instead of F-test to increase accuracy.

“We choose a cut-off of mean count < 5 and estimate the power for genes with smaller mean values based on simulation instead of the F-test to increase accuracy for small count values.”

May combine two claims.

For genes with high expression, eQTL power is calculated analytically using an F-test and depends on sample size, coefficient of determination, and significance threshold.

“For genes with high expression level, the power to detect an eQTL is calculated analytically using an F-test and depends on the sample size , the coefficient of determination of the locus and the chosen significance threshold .”

• cites [20] Cohen, J. Statistical power analysis for the behavioral sciences. (Hillsdale, 1989).Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Simulation-based methods perform random sampling of their count matrices, so simulation was repeated 25 times for each parameter combination to generate stable results.

“The simulation-based methods perform random sampling of their count matrices and therefore the simulation was repeated 25 times for each parameter combination to generate stable results.”

Probably not a substantial claim: may not pass S3 (load-bearing).

In eQTL case, definition of groups depends on genotype and is not directly chosen by user.

“In the eQTL case, the definition of groups depends on the genotype and is therefore not directly chosen by the user.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The reported power of powsimR includes only genes expressed in the simulated data set (count > 0).

“The reported power of powsimR includes only genes, which are expressed in the simulated data set (count > 0).”

• cites powsimRNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

For DE analysis, the significance threshold can be adjusted for FDR using the method of Jung22.

“Alternatively, for DE analysis the significance threshold can be adjusted for the false discovery rate using the method of Jung22.”

• cites Jung22Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

powsimR is not designed for multi-sample comparison and for both methods the option to apply a vector of log-fold changes with matching expression ranks was not available.

“powsimR is not designed for multi-sample comparison and for both methods the option to apply a vector of log-fold changes with matching expression ranks was not available.”

• cites powsimRNot checked yet

  Not checked in the free run.

• cites muscat40,43Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

The framework allows for any combination of power analysis and multiple testing method.

“However, our framework allows for any combination of power analysis and multiple testing method.”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

Overall detection power is the product of expression probability and power to detect DEGs/eQTLs, as both are conditionally independent given expression mean.

“The overall detection power for DEGs/eQTLs is the product of the expression probability and power to detect DEGs/eQTLs, as both probabilities are conditionally independent given the expression mean of the gene.”

• AssumptionThe authors state this as an assumption: Overall detection power is the product of expression probability and power to detect DEGs/eQTLs, as both are conditionally independent given expression mean.

Classified as probably not a substantial claim.

To get realistic estimates for effect sizes (fold changes), data sets from FACS sorted bulk RNA-seq studies were taken53,54.

“To get realistic estimates for effect sizes (fold changes), data sets from FACS sorted bulk RNA-seq studies were taken53,54.”

• cites [53] Rendeiro, A. F. et al. Chromatin accessibility maps of chronic lymphocytic leukaemia identify subtype-specific epigenome signatures and transcription regulatory networks. Nat. Commun. 7, 11938 (2016). sourceNot checked yet

  Not checked in the free run.

• cites [54] Moreno-Moral, A. et al. Changes in macrophage transcriptome associate with systemic sclerosis and mediate GSDMA contribution to disease risk. Ann. Rheum. Dis. 77, 596–601 (2018). sourceNot checked yet

  Not checked in the free run.

May combine two claims.

For large count values, normalized log-transformed counts have constant variance independent of mean and can be analyzed with linear models.

“For large count values, it has been shown that normalized log transformed counts have a constant variance independent of the mean value and can be analyzed with linear models45.”

• cites [45] Law, C. W., Chen, Y., Shi, W. & Smyth, G. K. voom: Precision weights unlock linear model analysis tools for RNA-seq read counts. Genome Biol. 15, R29 (2014). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

As no data was available to model the linear increase of the doublet rate during overloading correctly, the doublet rate was modeled as a constant factor and library preparation costs were estimated per cell.

“However, as there was no data available to model the linear increase of the doublet rate during overloading correctly, the doublet rate was modeled instead as a constant factor and the library preparation costs were estimated per cell.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Drop-seq is also a droplet-based technique, similar to 10X Genomics.

“The Drop-seq technology is also a droplet-based technique, similar to 10X Genomics.”

Classified as probably not a substantial claim.

A second effect of doublets is that the read distribution is shifted, as doublets contain more reads than singlets.

“A second effect of doublets is that the read distribution is shifted, as doublets contain more reads than singlets.”

Probably not a substantial claim: may not pass S3 (load-bearing).

muscat fits one negative binomial distribution separately for each sample and subpopulation in the training data set.

“It fits one negative binomial distribution separately for each sample and subpopulation in the training data set.”

• cites muscat40,43Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

muscat provides different scenarios for simulating differential expression besides the shift of mean expression (DE), including DP, DM, and DB.

“Another difference to powsimR is that muscat provides different scenarios for simulating differential expression besides the shift of the mean expression (called DE in muscat). Additionally, they simulate genes with different proportions of low and high expression-state (DP), differential modality (DM) or changes in both proportions and modality (DB).”

• cites muscat40,43Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim. May combine two claims.

Smart-seq2 is a plate-based technique, which produces full length transcripts and read counts instead of UMI counts.

“Smart-seq2 is a plate-based technique, which produces full length transcripts and read counts instead of UMI counts.”

Probably not a substantial claim: may not pass S3 (load-bearing).

muscat was specifically implemented for multi-sample comparisons, in contrast to powsimR.

“muscat: was specifically implemented for multi-sample comparisons, in contrast to powsimR.”

• cites muscat40,43Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Both data sets were subsampled to investigate the effect of the read depth.

“Both data sets were subsampled to investigate the effect of the read depth.”

Classified as probably not a substantial claim.

Budget restricted power was compared to PBMC 10X Genomics results, using the same simulated effect sizes and distribution ranks as well as matched observed priors from FACS sorted bulk studies.

“We compared the budget restricted power to our PBMC 10X Genomics results, using the same simulated effect sizes and distribution ranks as well as matched observed priors from FACS sorted bulk studies.”

Classified as probably not a substantial claim.

The simulation-based methods were applied to further validate scPower estimations for two real-life scenarios that can affect power: batch effects and differences of cell proportions between groups.

“We applied the simulation-based methods to further validate the scPower estimations in regards to two different real life scenarios that can affect the power: batch effects and differences of cell proportions between the groups.”

Classified as probably not a substantial claim.

Model priors (effect sizes and gene rank distributions) were derived from FACS sorted bulk RNA-seq to get realistic assumptions.

“Model priors, i.e., effect sizes and gene rank distributions, were derived from FACS sorted bulk RNA-seq to get realistic assumptions.”

Classified as probably not a substantial claim.

Following the approach of37, the doublet rate is modeled as linear dependent on the number of recovered cells, using values from the 10X User guide of V377.

“Following the approach of37, we model the doublet rate linear dependent on the number of recovered cells, using the values from the 10X User guide of V377.”

• cites [37] Hafemeister, C. How Many Cells. https://satijalab.org/howmanycells (2019).Not checked yet

  Not checked in the free run.

• cites V377Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Different extreme prior distributions were simulated to evaluate their influence on the optimal experimental parameters.

“In addition, we simulated different extreme prior distributions to evaluate their influence on the optimal experimental parameters.”

Classified as probably not a substantial claim.

Overall experimental cost for a 10X Genomics experiment is the sum of library preparation cost and sequencing cost.

“The overall experimental cost for a 10X Genomics experiment is the sum of the library preparation cost and the sequencing cost.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

Doublets also appear in Smart-seq2, but as a constant factor, not increasing with a higher number of cells per individual.

“Doublets also appear in Smart-seq2, but as a constant factor, not increasing with a higher number of cells per individual.”

Probably not a substantial claim: may not pass S3 (load-bearing).

powsimR uses training data to fit parameters of expression distributions for each cell type and gene.

“powsimR: uses training data to fit the parameters of the expression distributions for each cell type and gene.”

• cites powsimRNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim. May combine two claims.

Two data sets of the human cell atlas were used: a Drop-seq data set measured in lung tissue52 and a Smart-seq2 data set measured in pancreas tissue51.

“Two data sets of the human cell atlas were used for that, a Drop-seq data set measured in lung tissue52 and a Smart-seq2 data set measured in pancreas tissue51.”

• cites [52] Vieira Braga, F. A. et al. A cellular census of human lungs identifies novel cell states in health and in asthma. Nat. Med. 25, 1153–1163 (2019). sourceNot checked yet

  Not checked in the free run.

• cites [51] Enge, M. et al. Single-cell analysis of human pancreas reveals transcriptional signatures of aging and somatic mutation patterns. Cell 171, 321–330.e14 (2017). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Log fold changes for the DE studies were modeled as normally distributed.

“The log fold changes for the DE studies were modeled as normally distributed.”