Robustly detecting differential expression in RNA sequencing data using observation weights
Open accessAbstract
A popular approach for comparing gene expression levels between (replicated) conditions of RNA sequencing data relies on counting reads that map to features of interest. Within such count-based methods, many flexible and advanced statistical approaches now exist and offer the ability to adjust for covariates (e.g. batch effects). Often, these methods include some sort of 'sharing of information' across features to improve inferences in small samples. It is important to achieve an appropriate tradeoff between statistical power and protection against outliers. Here, we study the robustness of existing approaches for count-based differential expression analysis and propose a new strategy based on observation weights that can be used within existing frameworks. The results suggest that outliers can have a global effect on differential analyses. We demonstrate the effectiveness of our new approach with real data and simulated data that reflects properties of real datasets (e.g. dispersion-mean trend) and develop an extensible framework for comprehensive testing of current and future methods. In addition, we explore the origin of such outliers, in some cases highlighting additional biological or technical factors within the experiment. Further details can be downloaded from the project website: http://imlspenticton.uzh.ch/robinson_lab/edgeR_robust/.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
A negative binomial model in edgeR was used to account for the sparse single-cell count matrix.
“We used the negative binomial (NB) model implemented in edgeR (3.10.0)79–83 for our single cell differential expression analysis to explicitly account for the sparse count matrix”
From Epigenetic regulation of brain region-specific microglia clearance activity (2018) · cites this paper as [83]
The quasi-likelihood framework in edgeR was used to handle the pronounced variability of single-cell expression states.
“and the quasi-likelihood framework within edgeR to handle the pronounced variability of the single cell expression states.”
From Epigenetic regulation of brain region-specific microglia clearance activity (2018) · cites this paper as [83]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482 · see citing works
Robinson, M. D., McCarthy, D. J., & Smyth, G. K. (2010) · Bioinformatics · doi:10.1093/bioinformatics/btp616 · Cited by 41,364 · see citing works
Self-citationRSEM: accurate transcript quantification from RNA-Seq data with or without a reference genome
Li, B., & Dewey, C. N. (2011) · BMC Bioinformatics · doi:10.1186/1471-2105-12-323 · Cited by 22,179 · see citing works
Full-length transcriptome assembly from RNA-Seq data without a reference genome
Grabherr, M. G., Haas, B. J., Yassour, M., Levin, J. Z., Thompson, D. A., Amit, I., Adiconis, X., Fan, L., Raychowdhury, R., Zeng, Q., et al. (2011) · Nature Biotechnology · doi:10.1038/nbt.1883 · Cited by 20,770 · see citing works
Differential expression analysis for sequence count data
Anders, S., & Huber, W. (2010) · Genome Biology · doi:10.1186/gb-2010-11-10-r106 · Cited by 15,581 · see citing works
RNA-Seq: a revolutionary tool for transcriptomics
Wang, Z., Gerstein, M., & Snyder, M. (2009) · Nature Reviews Genetics · doi:10.1038/nrg2484 · Cited by 13,174 · see citing works
Bioconductor: open software development for computational biology and bioinformatics
Gentleman, R., Carey, V., Bates, D., Bolstad, B., Dettling, M., Dudoit, S., Ellis, B., Gautier, L., Ge, Y., Gentry, J., et al. (2004) · Genome Biology · doi:10.1186/gb-2004-5-10-r80 · Cited by 13,133 · see citing works
A scaling normalization method for differential expression analysis of RNA-seq data
Robinson, M. D., & Oshlack, A. (2010) · Genome Biology · doi:10.1186/gb-2010-11-3-r25 · Cited by 7,876 · see citing works
Self-citationvoom: precision weights unlock linear model analysis tools for RNA-seq read counts
Law, C. W., Chen, Y., Shi, W., & Smyth, G. K. (2014) · Genome Biology · doi:10.1186/gb-2014-15-2-r29 · Cited by 6,155 · see citing works
Rue, H., Martino, S., & Chopin, N. (2009) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2008.00700.x · Cited by 5,279 · see citing works
McCarthy, D. J., Chen, Y., & Smyth, G. K. (2012) · Nucleic Acids Research · doi:10.1093/nar/gks042 · Cited by 5,195 · see citing works
Empirical Bayes analysis of a microarray experiment
Efron, B., Tibshirani, R., Storey, J. D., & Tusher, V. (2001) · Journal of the American Statistical Association · doi:10.1198/016214501753382129 · Cited by 1,733 · see citing works
Parameter Orthogonality and Approximate Conditional Inference
Cox, D. R., & Reid, N. M. (1987) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.2517-6161.1987.tb01422.x · Cited by 1,370 · see citing works
EBSeq: an empirical Bayes hierarchical model for inference in RNA-seq experiments
Leng, N., Dawson, J. A., Thomson, J. A., Ruotti, V., Rissman, A. I., Smits, B. M. G., Haag, J. D., Gould, M. N., Stewart, R. M., & Kendziorski, C. M. (2013) · Bioinformatics · doi:10.1093/bioinformatics/btt087 · Cited by 1,304 · see citing works
Count-based differential expression analysis of RNA sequencing data using R and Bioconductor
Anders, S., McCarthy, D. J., Chen, Y., Okoniewski, M. J., Smyth, G. K., Huber, W. K., & Robinson, M. D. (2013) · Nature Protocols · doi:10.1038/nprot.2013.099 · Cited by 1,129 · see citing works
Self-citationSmall-sample estimation of negative binomial dispersion, with applications to SAGE data
Robinson, M. D., & Smyth, G. K. (2007) · Biostatistics · doi:10.1093/biostatistics/kxm030 · Cited by 1,073 · see citing works
Self-citationA comparison of methods for differential expression analysis of RNA-seq data
Soneson, C., & Delorenzi, M. (2013) · BMC Bioinformatics · doi:10.1186/1471-2105-14-91 · Cited by 887 · see citing works
Moderated statistical tests for assessing differences in tag abundance
Robinson, M. D., & Smyth, G. K. (2007) · Bioinformatics · doi:10.1093/bioinformatics/btm453 · Cited by 856 · see citing works
Self-citationTranscriptome genetics using second generation sequencing in a Caucasian population
Montgomery, S. B., Sammeth, M., Gutierrez-Arcelus, M., Lach, R. P., Ingle, C., Nisbett, J., Guigo, R., & Dermitzakis, E. T. (2010) · Nature · doi:10.1038/nature08903 · Cited by 797 · see citing works
baySeq: Empirical Bayesian methods for identifying differential expression in sequence count data
Hardcastle, T. J., & Kelly, K. A. (2010) · BMC Bioinformatics · doi:10.1186/1471-2105-11-422 · Cited by 755 · see citing works
Comprehensive evaluation of differential gene expression analysis methods for RNA-seq data
Rapaport, F., Khanin, R., Liang, Y., Pirun, M., Krek, A., Zumbo, P., Mason, C. E., Socci, N. D., & Betel, D. (2013) · Genome Biology · doi:10.1186/gb-2013-14-9-r95 · Cited by 733 · see citing works
Removing technical variability in RNA-seq data using conditional quantile normalization
Hansen, K. D., Irizarry, R. A., & WU, Z. (2012) · Biostatistics · doi:10.1093/biostatistics/kxr054 · Cited by 607 · see citing works
Bayesian computing with INLA: New features
Martins, T. G., Simpson, D., Lindgren, F., & Rue, H. (2013) · Computational Statistics & Data Analysis · doi:10.1016/j.csda.2013.04.014 · Cited by 603 · see citing works
Li, J., & Tibshirani, R. J. (2011) · Statistical Methods in Medical Research · doi:10.1177/0962280211428386 · Cited by 429 · see citing works
Robust Inference for Generalized Linear Models
Cantoni, E., & Ronchetti, E. (2001) · Journal of the American Statistical Association · doi:10.1198/016214501753209004 · Cited by 404 · see citing works
Behavior research methods and instrumentation
Sidowski, J. B. (1968) · Behavior Research Methods · doi:10.3758/bf03209857 · Cited by 350 · see citing works
Sex-specific and lineage-specific alternative splicing in primatesPreprint
Blekhman, R., Marioni, J. C., Zumbo, P., Stephens, M., & Gilad, Y. (2009) · Genome Research · doi:10.1101/gr.099226.109 · Cited by 266 · see citing works
A new shrinkage estimator for dispersion improves differential expression detection in RNA-seq data
Wu, H., Wang, C., & Wu, Z. J. (2012) · Biostatistics · doi:10.1093/biostatistics/kxs033 · Cited by 241 · see citing works
Witten, D., Tibshirani, R. J., Gu, S. G., Fire, A. Z., & Lui, W. (2010) · BMC Biology · doi:10.1186/1741-7007-8-58 · Cited by 231 · see citing works
Tissue inhibitor of metalloproteinase-1 and -2 RNA expression in rat and human liver fibrosis.
Herbst, H., Wege, T., Milani, S., Pellegrini, G., Orzechowski, H., Bechstein, W. O., Neuhaus, P., Gressner, A. M., & Schuppan, D. (1997) · Europe PMC (PubMed Central) · Cited by 210 · see citing works
On Robustness in the Logistic Regression Model
Carroll, R. J., & Pederson, S. P. (1993) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.2517-6161.1993.tb01934.x · Cited by 195 · see citing works
Polymorphic Cis- and Trans-Regulation of Human Gene Expression
Cheung, V. G., Nayak, R. R., Wang, I. X., Elwyn, S., Cousins, S. M., Morley, M. P., & Spielman, R. S. (2010) · PLoS Biology · doi:10.1371/journal.pbio.1000480 · Cited by 165 · see citing works
Liver Gene Expression Signature of Mild Fibrosis in Patients With Chronic Hepatitis C
Asselah, T., Bièche, I., Laurendeau, I., Paradis, V., Vidaud, D., Degott, C., Martinot-Peignoux, M., Bédossa, P., Valla, D. C., Vidaud, M., et al. (2005) · Gastroenterology · doi:10.1053/j.gastro.2005.09.010 · Cited by 163 · see citing works
ReCount: A multi-experiment resource of analysis-ready RNA-seq gene count datasets
Frazee, A. C., Langmead, B., & Leek, J. T. (2011) · BMC Bioinformatics · doi:10.1186/1471-2105-12-449 · Cited by 155 · see citing works
Bayesian analysis of RNA sequencing data by estimating multiple shrinkage priors
van de Wiel, M. A., Leday, G. G. R., Pardo, L., Rue, H., van der Vaart, A., & van Wieringen, W. N. (2012) · Biostatistics · doi:10.1093/biostatistics/kxs031 · Cited by 132 · see citing works
Sample richness and genetic diversity as drivers of chimera formation in nSSU metagenetic analyses
Fonseca, V. G., Nichols, B., Lallias, D., et al. (2012) · Nucleic Acids Research · doi:10.1093/nar/gks002 · Cited by 131 · see citing works
Robust model selection in regression via weighted likelihood methodology
Agostinelli, C. (2002) · Statistics & Probability Letters · doi:10.1016/s0167-7152(01)00193-6 · Cited by 53 · see citing works
Evaluating statistical analysis models for RNA sequencing experiments
Reeb, P., & Steibel, J. P. (2013) · Frontiers in Genetics · doi:10.3389/fgene.2013.00178 · Cited by 36 · see citing works
Robust Inference in Generalized Linear Models
Alqallaf, F., & Agostinelli, C. (2015) · Communications in Statistics - Simulation and Computation · doi:10.1080/03610918.2014.911896 · Cited by 13 · see citing works
Higher order asymptotics for negative binomial regression inferences from RNA-sequencing data
Di, Y., Emerson, S. C., Schafer, D. W., Kimbrel, J. A., & Chang, J. H. (2013) · Statistical Applications in Genetics and Molecular Biology · doi:10.1515/sagmb-2012-0071 · Cited by 12 · see citing works
Robust Inference in Generalized Linear Models
Alqallaf, F., & Agostinelli, C. (2014) · Figshare · doi:10.6084/m9.figshare.1456236 · Cited by 2 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Moderated estimation of fold change and dispersion for RNA-seq data with DESeq2
Love, M. I., Huber, W., & Anders, S. (2014) · Genome Biology · doi:10.1186/s13059-014-0550-8 · Cited by 95,106 · see citing works
Bias, robustness and scalability in single-cell differential expression analysis
Soneson, C., & Robinson, M. D. (2018) · Nature Methods · doi:10.1038/nmeth.4612 · Cited by 770 · see citing works
Self-citationEpigenetic regulation of brain region-specific microglia clearance activity
Ayata, P., Badimon, A., Strasburger, H. J., Duff, M. K., Montgomery, S. E., Loh, Y.-H. E., Ebert, A., Pimenova, A. A., Ramirez, B. R., Chan, A. T., et al. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0192-3 · Cited by 460 · see citing works
Methods for ChIP-seq analysis: A practical workflow and advanced applications
Nakato, R., & Sakata, T. (2021) · Methods · doi:10.1016/j.ymeth.2020.03.005 · Cited by 267 · see citing works
A broken promise: microbiome differential abundance methods do not control the false discovery rate
Hawinkel, S., Mattiello, F., Bijnens, L., & Thas, O. (2019) · Briefings in Bioinformatics · doi:10.1093/bib/bbx104 · Cited by 199 · see citing works
Observation weights unlock bulk RNA-seq tools for zero inflation and single-cell applications
Van den Berge, K., Perraudeau, F., Soneson, C., Love, M. I., Risso, D., Vert, J.-P., Robinson, M. D., Dudoit, S., & Clement, L. (2018) · Genome Biology · doi:10.1186/s13059-018-1406-4 · Cited by 228 · see citing works
Self-citationMoisiadis, V. G., Constantinof, A., Kostaki, A., Szyf, M., & Matthews, S. G. (2017) · Scientific Reports · doi:10.1038/s41598-017-11635-w · Cited by 122 · see citing works
- Authors
- Zhou, X., Lindsay, H., & Robinson, M. D.
- Affiliations
- SIB Swiss Institute of Bioinformatics (CH)
- Corresponding author
- Robinson, Mark D.
- Venue
- Nucleic Acids Research
- Publisher
- Oxford University Press (OUP)
- Published
- 2014-04-20
- Volume
- 42
- Issue
- 11
- Pages
- e91-e91
- ISSN
- 0305-1048 (ISSN-L)
- 1362-4962
- PMC
- PMC4066750
- OpenAlex
- W2112058982
- Cited by
- 387
- References
- 41
- Citations by year
- 2026: 8 · 2025: 21 · 2024: 31 · 2023: 36 · 2022: 28 · 2021: 22 · 2020: 33 · 2019: 54 · 2018: 56 · 2017: 37 · 2016: 38 · 2015: 17 · 2014: 6
- FWCI
- 13.32
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- Nucleic Acids Research (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Zurich Open Repository and Archive (University of Zurich) (acceptedVersion, other-oa)
- reroDoc Digital Library (submittedVersion)
- Zurich Open Repository and Archive (University of Zurich) (submittedVersion, other-oa)
- Zurich Open Repository and Archive (University of Zurich)
- Funders
- European Commission (305626)
- Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung (143883)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene expression and cancer classification
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Phylogenetic Studies
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Single-cell and spatial transcriptomics
- Keywords
- differential gene expression, RNA sequencing, observation weighting, robust inference, outliers, edgeR, batch effects, small sample size, statistical power
- MeSH terms
- Computer Simulation, Sequence Analysis, RNA, Gene Expression Profiling
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
