Differential gene expression analysis for multi-subject single-cell RNA-sequencing studies with aggregateBioVar
Open accessAbstract
Motivation: Single-cell RNA-sequencing (scRNA-seq) provides more granular biological information than bulk RNA-sequencing; bulk RNA sequencing remains popular due to lower costs which allows processing more biological replicates and design more powerful studies. As scRNA-seq costs have decreased, collecting data from more than one biological replicate has become more feasible, but careful modeling of different layers of biological variation remains challenging for many users. Here, we propose a statistical model for scRNA-seq gene counts, describe a simple method for estimating model parameters and show that failing to account for additional biological variation in scRNA-seq studies can inflate false discovery rates (FDRs) of statistical tests. Results: First, in a simulation study, we show that when the gene expression distribution of a population of cells varies between subjects, a naïve approach to differential expression analysis will inflate the FDR. We then compare multiple differential expression testing methods on scRNA-seq datasets from human samples and from animal models. These analyses suggest that a naïve approach to differential expression testing could lead to many false discoveries; in contrast, an approach based on pseudobulk counts has better FDR control. Availability and implementation: A software package, aggregateBioVar, is freely available on Bioconductor (https://www.bioconductor.org/packages/release/bioc/html/aggregateBioVar.html) to accommodate compatibility with upstream and downstream methods in scRNA-seq data analysis pipelines. Supplementary information: Raw gene-by-cell count matrices for pig scRNA-seq data are available as GEO accession GSE150211. Supplementary data are available at Bioinformatics online.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Failing to account for multiple sources of biological variability inflates the false discovery rate.
“Failing to account for multiple sources of biological variability for experiments will inflate the FDR90,91.”
From Best practices for single-cell analysis across modalities (2023) · cites this paper as [90]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482 · see citing works
Moderated estimation of fold change and dispersion for RNA-seq data with DESeq2
Love, M. I., Huber, W., & Anders, S. (2014) · Genome Biology · doi:10.1186/s13059-014-0550-8 · Cited by 95,106 · see citing works
Robinson, M. D., McCarthy, D. J., & Smyth, G. K. (2010) · Bioinformatics · doi:10.1093/bioinformatics/btp616 · Cited by 41,364 · see citing works
Comprehensive Integration of Single-Cell Data
Stuart, T., Butler, A., Hoffman, P., Hafemeister, C., Papalexi, E., Mauck, W. M., Hao, Y., Stoeckius, M., Smibert, P., & Satija, R. (2019) · Cell · doi:10.1016/j.cell.2019.05.031 · Cited by 15,453 · see citing works
Data Analysis Using Regression and Multilevel/Hierarchical Models
Gelman, A., & Hill, J. (2006) · doi:10.1017/cbo9780511790942 · Cited by 13,597 · see citing works
Integrating single-cell transcriptomic data across different conditions, technologies, and species
Butler, A., Hoffman, P., Smibert, P., Papalexi, E., & Satija, R. (2018) · Nature Biotechnology · doi:10.1038/nbt.4096 · Cited by 13,174 · see citing works
Trapnell, C., Cacchiarelli, D., Grimsby, J., Pokharel, P., Li, S., Morse, M., Lennon, N. J., Livak, K. J., Mikkelsen, T. S., & Rinn, J. L. (2014) · Nature Biotechnology · doi:10.1038/nbt.2859 · Cited by 7,314 · see citing works
Highly parallel genome-wide expression profiling of individual cells using nanoliter droplets.
Macosko, E. Z., Basu, A., Satija, R., Nemesh, J., Shekhar, K., Goldman, M., Tirosh, I., Bialas, A. R., Kamitaki, N., Martersteck, E. M., et al. (2015) · Cell · doi:10.1016/j.cell.2015.05.002 · Cited by 7,110 · see citing works
Spatial reconstruction of single-cell gene expression data
Satija, R., Farrell, J. A., Gennert, D., Schier, A. F., & Regev, A. (2015) · Nature Biotechnology · doi:10.1038/nbt.3192 · Cited by 7,055 · see citing works
Reversed graph embedding resolves complex single-cell trajectories
Qiu, X., Mao, Q., Tang, Y., Wang, L., Chawla, R., Pliner, H. A., & Trapnell, C. (2017) · Nature Methods · doi:10.1038/nmeth.4402 · Cited by 5,070 · see citing works
Droplet barcoding for single-cell transcriptomics applied to embryonic stem cells
Klein, A. M., Mazutis, L., Akartuna, I., Tallapragada, N., Veres, A., Li, V., Peshkin, L., Weitz, D. A., & Kirschner, M. W. (2015) · Cell · doi:10.1016/j.cell.2015.04.044 · Cited by 3,356 · see citing works
Finak, G., McDavid, A., Yajima, M., Deng, J., Gersuk, V., Shalek, A. K., Slichter, C. K., Miller, H. W., McElrath, M. J., Prlic, M., et al. (2015) · Genome Biology · doi:10.1186/s13059-015-0844-5 · Cited by 3,159 · see citing works
Current best practices in single-cell RNA-seq analysis: a tutorial
Luecken, M. D., & Theis, F. J. (2019) · Molecular Systems Biology · doi:10.15252/msb.20188746 · Cited by 2,231 · see citing works
Single-cell mRNA quantification and differential analysis with Census
Qiu, X., Hill, A. J., Packer, J. S., Lin, D., Ma, Y.-A., & Trapnell, C. (2017) · Nature Methods · doi:10.1038/nmeth.4150 · Cited by 1,885 · see citing works
McCarthy, D. J., Campbell, K. R., Lun, A. T. L., & Wills, Q. F. (2017) · Bioinformatics · doi:10.1093/bioinformatics/btw777 · Cited by 1,852 · see citing works
Single-cell RNA sequencing technologies and bioinformatics pipelines
Hwang, B., Lee, J. H., & Bang, D. (2018) · Experimental & Molecular Medicine · doi:10.1038/s12276-018-0071-8 · Cited by 1,793 · see citing works
A step-by-step workflow for low-level analysis of single-cell RNA-seq data with Bioconductor
Lun, A. T. L., McCarthy, D. J., & Marioni, J. C. (2016) · F1000Research · doi:10.12688/f1000research.9501.2 · Cited by 1,586 · see citing works
PanglaoDB: a web server for exploration of mouse and human single-cell RNA sequencing data
Franzén, O., Gan, L.-M., & Björkegren, J. L. M. (2019) · Database · doi:10.1093/database/baz046 · Cited by 1,551 · see citing works
Comparative Analysis of Single-Cell RNA Sequencing Methods
Ziegenhain, C., Vieth, B., Parekh, S., Reinius, B., Guillaumet-Adkins, A., Smets, M., Leonhardt, H., Heyn, H., Hellmann, I., & Enard, W. (2017) · Molecular Cell · doi:10.1016/j.molcel.2017.01.023 · Cited by 1,494 · see citing works
Single-cell profiling of the developing mouse brain and spinal cord with split-pool barcoding
Rosenberg, A. B., Roco, C. M., Muscat, R. A., Kuchina, A., Sample, P., Yao, Z., Graybuck, L. T., Peeler, D. J., Mukherjee, S., Chen, W., et al. (2018) · Science · doi:10.1126/science.aam8999 · Cited by 1,422 · see citing works
Cao, J., Packer, J. S., Ramani, V., Cusanovich, D. A., Huynh, C., Daza, R., Qiu, X., Lee, C., Furlan, S. N., Steemers, F. J., et al. (2017) · doi:10.1101/104844 · Cited by 1,402 · see citing works
A First Course in Probability.
Sherman, M. J., & Ross, S. M. (1984) · Journal of the American Statistical Association · doi:10.2307/2288753 · Cited by 1,355 · see citing works
Bayesian approach to single-cell differential expression analysis
Kharchenko, P. V., Silberstein, L., & Scadden, D. T. (2014) · Nature Methods · doi:10.1038/nmeth.2967 · Cited by 1,348 · see citing works
Multiplexed droplet single-cell RNA-sequencing using natural genetic variation
Kang, H. M., Subramaniam, M., Targ, S., Nguyen, M., Maliskova, L., McCarthy, E., Wan, E., Wong, S., Byrnes, L., Lanata, C. M., et al. (2018) · Nature Biotechnology · doi:10.1038/nbt.4042 · Cited by 1,339 · see citing works
Eleven grand challenges in single-cell data science
Lähnemann, D., Köster, J., Szczurek, E., McCarthy, D. J., Hicks, S. C., Robinson, M. D., Vallejos, C. A., Campbell, K. R., Beerenwinkel, N., Mahfouz, A., et al. (2020) · Genome Biology · doi:10.1186/s13059-020-1926-6 · Cited by 1,327 · see citing works
EBSeq: an empirical Bayes hierarchical model for inference in RNA-seq experiments
Leng, N., Dawson, J. A., Thomson, J. A., Ruotti, V., Rissman, A. I., Smits, B. M. G., Haag, J. D., Gould, M. N., Stewart, R. M., & Kendziorski, C. M. (2013) · Bioinformatics · doi:10.1093/bioinformatics/btt087 · Cited by 1,304 · see citing works
Reyfman, P. A., Walter, J. M., Joshi, N., Anekalla, K. R., McQuattie-Pimentel, A. C., Chiu, S., Fernandez, R., Akbarpour, M., Chen, C.-I., Ren, Z., et al. (2019) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.201712-2410oc · Cited by 1,203 · see citing works
Single-Cell RNA-Seq Technologies and Related Computational Data Analysis
Chen, G., Ning, B., & Shi, T. (2019) · Frontiers in Genetics · doi:10.3389/fgene.2019.00317 · Cited by 913 · see citing works
Seq-Well: portable, low-cost RNA sequencing of single cells at high throughput
Gierahn, T. M., Wadsworth, M. H., Hughes, T. K., Bryson, B. D., Butler, A., Satija, R., Fortune, S. M., Love, J. C., & Shalek, A. K. (2017) · Nature Methods · doi:10.1038/nmeth.4179 · Cited by 855 · see citing works
baySeq: Empirical Bayesian methods for identifying differential expression in sequence count data
Hardcastle, T. J., & Kelly, K. A. (2010) · BMC Bioinformatics · doi:10.1186/1471-2105-11-422 · Cited by 755 · see citing works
Disruption of the CFTR Gene Produces a Model of Cystic Fibrosis in Newborn Pigs
Rogers, C. S., Stoltz, D. A., Meyerholz, D. K., Ostedgaard, L. S., Rokhlina, T., Taft, P. J., Rogan, M. P., Pezzulo, A. A., Karp, P. H., Itani, O. A., et al. (2008) · Science · doi:10.1126/science.1163600 · Cited by 752 · see citing works
Self-citationSingle-cell transcriptomes of the human skin reveal age-related loss of fibroblast priming
Solé-Boldo, L., Raddatz, G., Schütz, S., Mallm, J.-P., Rippe, K., Lonsdorf, A. S., Rodríguez-Paredes, M., & Lyko, F. (2020) · Communications Biology · doi:10.1038/s42003-020-0922-4 · Cited by 559 · see citing works
The dynamics of gene expression in vertebrate embryogenesis at single-cell resolution
Briggs, J., Weinreb, C., Wagner, D. E., Megason, S. G., Peshkin, L., Kirschner, M. W., & Klein, A. M. (2018) · Science · doi:10.1126/science.aar5780 · Cited by 548 · see citing works
Crowell, H. L., Soneson, C., Germain, P.-L., Calini, D., Collin, L., Raposo, C., Malhotra, D., & Robinson, M. D. (2020) · Nature Communications · doi:10.1038/s41467-020-19894-4 · Cited by 515 · see citing works
Cystic Fibrosis Pigs Develop Lung Disease and Exhibit Defective Bacterial Eradication at Birth
Stoltz, D. A., Meyerholz, D. K., Pezzulo, A. A., Ramachandran, S. S., Rogan, M. P., Davis, G. J., Hanfland, R. A., Wohlford-Lenane, C., Dohrn, C. L., Bartlett, J. A., et al. (2010) · Science Translational Medicine · doi:10.1126/scitranslmed.3000928 · Cited by 496 · see citing works
Self-citationComparative Analysis of Droplet-Based Ultra-High-Throughput Single-Cell RNA-Seq Systems
Zhang, X., Li, T., Liu, F., Chen, Y., Yao, J., Li, Z., Huang, Y., & Wang, J. (2018) · Molecular Cell · doi:10.1016/j.molcel.2018.10.020 · Cited by 380 · see citing works
A practical solution to pseudoreplication bias in single-cell studies
Zimmerman, K. D., Espeland, M. A., & Langefeld, C. D. (2021) · Nature Communications · doi:10.1038/s41467-021-21038-1 · Cited by 362 · see citing works
SINCERA: A Pipeline for Single-Cell RNA-Seq Profiling Analysis
Guo, M., Wang, H., Potter, S. S., Whitsett, J. A., & Xu, Y. J. (2015) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1004575 · Cited by 347 · see citing works
Sequencing thousands of single-cell genomes with combinatorial indexing
Vitak, S. A., Torkenczy, K. A., Rosenkrantz, J. L., Fields, A. J., Christiansen, L., Wong, M. H., Carbone, L., Steemers, F. J., & Adey, A. (2017) · Nature Methods · doi:10.1038/nmeth.4154 · Cited by 294 · see citing works
Wang, T., Li, B., Nelson, C. E., & Nabavi, S. (2019) · BMC Bioinformatics · doi:10.1186/s12859-019-2599-6 · Cited by 286 · see citing works
A statistical approach for identifying differential distributions in single-cell RNA-seq experiments
Korthauer, K. D., Chu, L.-F., Newton, M. A., Li, Y., Thomson, J., Stewart, R., & Kendziorski, C. (2016) · Genome Biology · doi:10.1186/s13059-016-1077-y · Cited by 273 · see citing works
DEsingle for detecting three types of differential expression in single-cell RNA-seq data
Miao, Z., Deng, K., Wang, X., & Zhang, X. (2018) · Bioinformatics · doi:10.1093/bioinformatics/bty332 · Cited by 272 · see citing works
A systematic evaluation of single cell RNA-seq analysis pipelines
Vieth, B., Parekh, S., Ziegenhain, C., Enard, W., & Hellmann, I. (2019) · Nature Communications · doi:10.1038/s41467-019-12266-7 · Cited by 265 · see citing works
A First Course in Probability.
Doney, R. A., & Ross, S. M. (1984) · Journal of the Royal Statistical Society Series A (General) · doi:10.2307/2981861 · Cited by 228 · see citing works
Carraro, G., Mulay, A., Yao, C., Mizuno, T., Konda, B. M., Petrov, M., Lafkas, D., Arron, J. R., Hogaboam, C. M., Chen, P., et al. (2020) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.201904-0792oc · Cited by 156 · see citing works
RNA Sequencing Data: Hitchhiker's Guide to Expression Analysis
den Berge, K. V., Hembach, K., Soneson, C., Tiberi, S., Clement, L., Love, M. I., Patro, R., & Robinson, M. D. (2019) · Annual Review of Biomedical Data Science · doi:10.1146/annurev-biodatasci-072018-021255 · Cited by 144 · see citing works
Wang, H., He, P., Pan, H., long, J., Wang, J., Li, Z., Liu, H., Jiang, W., & Zheng, Z. (2018) · Experimental & Molecular Medicine · doi:10.1038/s12276-018-0056-7 · Cited by 132 · see citing works
Highly multiplexed single-cell RNA-seq by DNA oligonucleotide tagging of cellular proteins
Gehring, J., Hwee Park, J., Chen, S., Thomson, M., & Pachter, L. (2020) · Nature Biotechnology · doi:10.1038/s41587-019-0372-z · Cited by 122 · see citing works
Comparative analysis of sequencing technologies for single-cell transcriptomics
Natarajan, K. N., Miao, Z., Jiang, M., Huang, X., Zhou, H., Xie, J., Wang, C., Qin, S., Zhao, Z., Wu, L. L., et al. (2019) · Genome biology · doi:10.1186/s13059-019-1676-5 · Cited by 114 · see citing works
Delmans, M., & Hemberg, M. (2016) · BMC Bioinformatics · doi:10.1186/s12859-016-0944-6 · Cited by 111 · see citing works
Delmans, M., & Hemberg, M. (2015) · doi:10.1101/020735 · Cited by 111 · see citing works
Using single-cell RNA sequencing to unravel cell lineage relationships in the respiratory tract
Zaragosi, L., Deprez, M., & Barbry, P. (2020) · Biochemical Society Transactions · doi:10.1042/bst20191010 · Cited by 67 · see citing works
Wang, T., & Nabavi, S. (2018) · Methods · doi:10.1016/j.ymeth.2018.04.017 · Cited by 56 · see citing works
Aicher, T. P., Carroll, S. L., Raddi, G., Gierahn, T. M., Wadsworth, M. H., Hughes, T. K., Love, C., & Shalek, A. K. (2019) · Methods in molecular biology · doi:10.1007/978-1-4939-9240-9_8 · Cited by 47 · see citing works
Newborn Cystic Fibrosis Pigs Have a Blunted Early Response to an Inflammatory Stimulus
Bartlett, J. A., Ramachandran, S. S., Wohlford-Lenane, C., Barker, C. K., Pezzulo, A. A., Zabner, J., Welsh, M. J., Meyerholz, D. K., Stoltz, D. A., & McCray, P. B. (2016) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.201510-2112oc · Cited by 37 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Best practices for single-cell analysis across modalities
Heumos, L., Schaar, A. C., Lance, C., Litinetskaya, A., Drost, F., Zappia, L., Lücken, M. D., Strobl, D. C., Henao, J., Curion, F., et al. (2023) · Nature Reviews Genetics · doi:10.1038/s41576-023-00586-w · Cited by 1,051 · see citing works
“3).”
Junttila, S., Smolander, J., & Elo, L. L. (2022) · Briefings in Bioinformatics · doi:10.1093/bib/bbac286 · Cited by 109 · see citing works
- Authors
- Thurman, A. L., Ratcliff, J. A., Chimenti, M. S., & Pezzulo, A. A.
- Affiliations
- University of Iowa (US)
- Corresponding author
- Thurman, Andrew L
- Pezzulo, Alejandro A
- Venue
- Bioinformatics
- Publisher
- Oxford University Press (OUP)
- Published
- 2021-05-10
- Volume
- 37
- Issue
- 19
- Pages
- 3243-3251
- ISSN
- 1367-4803 (ISSN-L)
- 1367-4811
- PMC
- PMC8504643
- OpenAlex
- W3162888433
- Cited by
- 27
- References
- 55
- Citations by year
- 2026: 4 · 2025: 5 · 2024: 5 · 2023: 6 · 2022: 7
- FWCI
- 1.41
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Bioinformatics (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Funders
- National Institutes of Health (NHLBI K01HL140261)
- NIH (NIDDK DK54759, NIEHS ES005605)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Single-cell and spatial transcriptomics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene expression and cancer classification
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Phylogenetic Studies
- Keywords
- single-cell RNA sequencing, differential gene expression, false discovery rate
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
