How data analysis affects power, reproducibility and biological insight of RNA-seq studies in complex datasets
Open accessThe DOI matches its Crossref record; 46 of the 46 references it cites were found in a registry or an index.
Abstract
The sequencing of the full transcriptome (RNA-seq) has become the preferred choice for the measurement of genome-wide gene expression. Despite its widespread use, challenges remain in RNA-seq data analysis. One often-overlooked aspect is normalization. Despite the fact that a variety of factors or 'batch effects' can contribute unwanted variation to the data, commonly used RNA-seq normalization methods only correct for sequencing depth. The study of gene expression is particularly problematic when it is influenced simultaneously by a variety of biological factors in addition to the one of interest. Using examples from experimental neuroscience, we show that batch effects can dominate the signal of interest; and that the choice of normalization method affects the power and reproducibility of the results. While commonly used global normalization methods are not able to adequately normalize the data, more recently developed RNA-seq normalization can. We focus on one particular method, RUVSeq and show that it is able to increase power and biological insight of the results. Finally, we provide a tutorial outlining the implementation of RUVSeq normalization that is applicable to a broad range of studies as well as meta-analysis of publicly available data.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures








Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In large studies, some technical artifacts cannot be avoided, and in these cases it is crucial to understand to what extent those artifacts limit our ability to draw conclusions from observations”
From Multi-omics approaches to disease (2017) · cites this paper as [11]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Robinson, M. D., McCarthy, D. J., & Smyth, G. K. (2010) · Bioinformatics · doi:10.1093/bioinformatics/btp616 · Cited by 41,364
HTSeq—a Python framework to work with high-throughput sequencing data
Anders, S., Pyl, P. T., & Huber, W. (2015) · Bioinformatics · doi:10.1093/bioinformatics/btu638 · Cited by 21,464
Mapping and quantifying mammalian transcriptomes by RNA-Seq
Mortazavi, A., Williams, B. A., McCue, K., Schaeffer, L., & Wold, B. (2008) · Nature Methods · doi:10.1038/nmeth.1226 · Cited by 13,696
Bioconductor: open software development for computational biology and bioinformatics
Gentleman, R. C., Carey, V. J., Bates, D. M., Bolstad, B., Dettling, M., Dudoit, S., Ellis, B., Gautier, L., Ge, Y., Gentry, J., et al. (2004) · Genome Biology · doi:10.1186/gb-2004-5-10-r80 · Cited by 12,172
DAVID: Database for Annotation, Visualization, and Integrated Discovery
Dennis, G., Sherman, B. T., Hosack, D. A., Yang, J., Gao, W., Lane, H. C., & Lempicki, R. A. (2003) · Genome Biology · doi:10.1186/gb-2003-4-5-p3 · Cited by 9,327
A scaling normalization method for differential expression analysis of RNA-seq data
Robinson, M. D., & Oshlack, A. (2010) · Genome Biology · doi:10.1186/gb-2010-11-3-r25 · Cited by 7,876
limma: Linear Models for Microarray Data
Smyth, G. K. (2005) · Statistics for Biology and Health · doi:10.1007/0-387-29362-0_23 · Cited by 5,796
Bioinformatics and Computational Biology Solutions Using R and Bioconductor
(2005) · Statistics for Biology and Health · doi:10.1007/0-387-29362-0 · Cited by 3,289
GMAP: A genomic mapping and alignment program for mRNA and EST sequences
Wu, T. D., & Watanabe, C. K. (2005) · Bioinformatics · doi:10.1093/bioinformatics/bti310 · Cited by 2,354
Tackling the widespread and critical impact of batch effects in high-throughput data
Leek, J. T., Scharpf, R. B., Bravo, H. C., Simcha, D., Langmead, B., Johnson, W. E., Geman, D., Baggerly, K., & Irizarry, R. A. (2010) · Nature Reviews Genetics · doi:10.1038/nrg2825 · Cited by 2,196
Normalization of RNA-seq data using factor analysis of control genes or samples
Risso, D., Ngai, J., Speed, T. P., & Dudoit, S. (2014) · Nature Biotechnology · doi:10.1038/nbt.2931 · Cited by 2,052
Self-citationCapturing Heterogeneity in Gene Expression Studies by Surrogate Variable Analysis
Leek, J. T., & Storey, J. D. (2007) · PLoS Genetics · doi:10.1371/journal.pgen.0030161 · Cited by 1,930
Bullard, J. H., Purdom, E., Hansen, K. D., & Dudoit, S. (2010) · BMC Bioinformatics · doi:10.1186/1471-2105-11-94 · Cited by 1,680 · cited 2×
“Similar results were obtained using TMM, while FPKM is the worst performing normalization method as previously shown (2,3) (Supplementary Figure S1A).”
MethodResultDillies, M.-A., Rau, A., Aubert, J., Hennequet-Antier, C., Jeanmougin, M., Servant, N., Keime, C., Marot, G., Castel, D., Estelle, J., et al. (2013) · Briefings in Bioinformatics · doi:10.1093/bib/bbs046 · Cited by 1,298
Count-based differential expression analysis of RNA sequencing data using R and Bioconductor
Anders, S., McCarthy, D. J., Chen, Y., Okoniewski, M., Smyth, G. K., Huber, W., & Robinson, M. D. (2013) · Nature Protocols · doi:10.1038/nprot.2013.099 · Cited by 1,146
Memory Reconsolidation and Extinction Have Distinct Temporal and Biochemical Signatures
Suzuki, A., Josselyn, S. A., Frankland, P. W., Masushige, S., Silva, A. J., & Kida, S. (2004) · The Journal of Neuroscience · doi:10.1523/jneurosci.5491-03.2004 · Cited by 1,127
The MAPK cascade is required for mammalian associative learning
Atkins, C. M., Selcher, J. C., Petraitis, J. J., Trzaskos, J. M., & Sweatt, J. D. (1998) · Nature Neuroscience · doi:10.1038/2836 · Cited by 1,083 · cited 1×
“Combining the datasets identifies as upregulated the MAPK, Jak/STAT and Insulin signaling pathways, all of which are known to be involved in memory and synaptic plasticity (35,37,39).”
BackgroundGC-Content Normalization for RNA-Seq Data
Risso, D., Schwartz, K., Sherlock, G., & Dudoit, S. (2011) · BMC Bioinformatics · doi:10.1186/1471-2105-12-480 · Cited by 867 · cited 4×
“Step 1 is to explore the distribution of variation in the data both before normalization and following normalization for sequencing depth only (UQ, TMM), which can be done by constructing the RLE and PCA plots previously shown, using a publicly available package such as EDASeq (13).”
MethodInfluentialSelf-citationConserved epigenomic signals in mice and humans reveal immune basis of Alzheimer's disease
Gjoneska, E., Pfenning, A. R., Mathys, H., Quon, G., Kundaje, A., Tsai, L.-H., & Kellis, M. (2015) · Nature · doi:10.1038/nature14252 · Cited by 576 · cited 2×
“As expected no difference is observed in time without induction of neurodegeneration, proper separation of samples by treatment is observed in the AD mouse model (44).”
Backgroundsvaseq: removing batch effects and other unwanted noise from sequencing data
Leek, J. T. (2014) · Nucleic Acids Research · doi:10.1093/nar/gku864 · Cited by 546
Sakamoto, K., Karelina, K., & Obrietan, K. (2011) · Journal of Neurochemistry · doi:10.1111/j.1471-4159.2010.07080.x · Cited by 543 · cited 1×
“This is consistent with the previously established role of these pathways in learning and memory (33–37).”
ResultStegle, O., Parts, L., Durbin, R., & Winn, J. (2010) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000770 · Cited by 497 · cited 4×
“RNA-seq normalization methods that are able to correct for factors other than sequencing depth are available and include: RUV, a normalization method we have recently published that uses factor analysis to remove systematic artifacts (6,26), PEER (27,28)”
MethodInfluentialPromoter features related to tissue specificity as measured by Shannon entropy
Schug, J., Schuller, W.-P., Kappen, C., Salbaum, J. M., Bucan, M., & Stoeckert, C. J. (2005) · Genome Biology · doi:10.1186/gb-2005-6-4-r33 · Cited by 447
Using control genes to correct for unwanted variation in microarray data
Gagnon-Bartsch, J. A., & Speed, T. P. (2012) · Biostatistics · doi:10.1093/biostatistics/kxr034 · Cited by 442 · cited 1×
“RNA-seq normalization methods that are able to correct for factors other than sequencing depth are available and include: RUV, a normalization method we have recently published that uses factor analysis to remove systematic artifacts (6,26), PEER (27,28) Figure 1.”
MethodSelf-citationExamining Object Location and Object Recognition Memory in Mice
Vogel‐Ciernia, A., & Wood, M. A. (2014) · Current Protocols in Neuroscience · doi:10.1002/0471142301.ns0831s69 · Cited by 342
Self-citationIgaz, L. M., Vianna, M. R. M., Medina, J. H., & Izquierdo, I. (2002) · The Journal of Neuroscience · doi:10.1523/jneurosci.22-15-06781.2002 · Cited by 303
Modulation of learning and memory by cytokines: Signaling mechanisms and long term consequences
Donzis, E. J., & Tronson, N. C. (2014) · Neurobiology of Learning and Memory · doi:10.1016/j.nlm.2014.08.008 · Cited by 254
Reproducibility of high-throughput mRNA and small RNA sequencing across laboratories
The GEUVADIS Consortium, 't Hoen, P. A. C., Friedländer, M. R., Almlöf, J., Sammeth, M., Pulyakhina, I., Anvar, S. Y., Laros, J. F. J., Buermans, H. P. J., Karlberg, O., et al. (2013) · Nature Biotechnology · doi:10.1038/nbt.2702 · Cited by 250 · cited 1×
“RNA-seq normalization methods that are able to correct for factors other than sequencing depth are available and include: RUV, a normalization method we have recently published that uses factor analysis to remove systematic artifacts (6,26), PEER (27,28) Figure 1.”
MethodThe Jak/STAT pathway is involved in synaptic plasticity
Nicolas, C. S., Peineau, S., Amici, M., Csaba, Z., Fafouri, A., Javalet, C., Collett, V. J., Hildebrandt, L., Seaton, G., Choi, S.-L., et al. (2012) · Neuron · doi:10.1016/j.neuron.2011.11.024 · Cited by 246 · cited 1×
“Combining the datasets identifies as upregulated the MAPK, Jak / STAT and Insulin signaling pathways, all of which are known to be involved in memory and synaptic plasticity (35,37,39).”
BackgroundVogel-Ciernia, A., Matheos, D. P., Barrett, R. M., Kramár, E. A., Azzawi, S., Chen, Y., Magnan, C. N., Zeller, M., Sylvain, A., Haettig, J., et al. (2013) · Nature Neuroscience · doi:10.1038/nn.3359 · Cited by 236
Self-citationQuality Assessment of Affymetrix GeneChip Data
Bolstad, B. M., Collin, F., Brettschneider, J., Simpson, K., Cope, L., Irizarry, R. A., & Speed, T. P. (2005) · Statistics for Biology and Health · doi:10.1007/0-387-29362-0_3 · Cited by 190
Self-citationMemory Reconsolidation Engages Only a Subset of Immediate-Early Genes Induced during Consolidation
von Hertzen, L. S. J., & Giese, K. P. (2005) · The Journal of Neuroscience · doi:10.1523/jneurosci.4707-04.2005 · Cited by 187
Histone H2A.Z subunit exchange controls consolidation of recent and remote memory
Zovkic, I. B., Paulukaitis, B. S., Day, J. J., Etikala, D. M., & Sweatt, J. D. (2014) · Nature · doi:10.1038/nature13707 · Cited by 179 · cited 1×
“A batch effect can be observed in the controls, and there’s no separation between FC and naı̈ve injected animals (41).”
BackgroundLevenson, J. M., Choi, S., Lee, S.-Y., Cao, Y. A., Ahn, H. J., Worley, K. C., Pizzi, M., Liou, H.-C., & Sweatt, J. D. (2004) · The Journal of Neuroscience · doi:10.1523/jneurosci.5646-03.2004 · Cited by 171
Actinomycin D Blocks Formation of Memory of Shock-Avoidance in Goldfish
Agranoff, B. W., Davis, R. E., Casola, L., & Lim, R. (1967) · Science · doi:10.1126/science.158.3808.1600 · Cited by 154
NR4A nuclear receptors support memory enhancement by histone deacetylase inhibitors
Hawk, J. D., Bookout, A. L., Poplawski, S. G., Bridi, M., Rao, A. J., Sulewski, M. E., Kroener, B. T., Manglesdorf, D. J., & Abel, T. (2012) · Journal of Clinical Investigation · doi:10.1172/jci64145 · Cited by 147
Self-citationStilling, R. M., Benito, E., Barth, J., Gertig, M., Capece, V., Burckhardt, S., Bonn, S., & Fischer, A. (2014) · Frontiers in Cellular Neuroscience · doi:10.3389/fncel.2014.00373 · Cited by 116
Stern, S. A., Chen, D. Y., & Alberini, C. M. (2014) · Learning & Memory · doi:10.1101/lm.029348.112 · Cited by 89
Memory acquisition and retrieval impact different epigenetic processes that regulate gene expression
Peixoto, L. L., Wimmer, M. E., Poplawski, S. G., Tudor, J. C., Kenworthy, C. A., Liu, S., Mizuno, K., Garcia, B. A., Zhang, N. R., Giese, K. P., et al. (2015) · BMC Genomics · doi:10.1186/1471-2164-16-s5-s5 · Cited by 87
Self-citationFunctionally diverse dendritic mRNAs rapidly associate with ribosomes following a novel experience
Ainsley, J. A., Drane, L., Jacobs, J., Kittelberger, K. A., & Reijmers, L. G. (2014) · Nature Communications · doi:10.1038/ncomms5510 · Cited by 77
Stilling, R. M., Rönicke, R., Benito, E., Urbanke, H., Capece, V., Burkhardt, S., Bahari‐Javan, S., Barth, J., Sananbenesi, F., Schütz, A. L., et al. (2014) · The EMBO Journal · doi:10.15252/embj.201487870 · Cited by 71
Keeley, M. B., Wood, M. A., Isiegas, C., Stein, J., Hellman, K., Hannenhalli, S., & Abel, T. (2006) · Learning & Memory · doi:10.1101/lm.86906 · Cited by 58
Self-citationBarnes, P., Kirtley, A., & Thomas, K. L. (2012) · Hippocampus · doi:10.1002/hipo.20879 · Cited by 57
NF-κB p50 subunit knockout impairs late LTP and alters long term memory in the mouse hippocampus
Oikawa, K., Odero, G. L., Platt, E., Neuendorff, M., Hatherell, A., Bernstein, M. J., & Albensi, B. C. (2012) · BMC Neuroscience · doi:10.1186/1471-2202-13-45 · Cited by 52
Fiorenza, A., Lopez-Atalaya, J. P., Rovira, V., Scandaglia, M., Geijo-Barrientos, E., & Barco, A. (2016) · Cerebral Cortex · doi:10.1093/cercor/bhu332 · Cited by 50
Crowder, M. J., & Hand, D. J. (2017) · Routledge eBooks · doi:10.1201/9781315137421-4 · Cited by 19
Cited by
Multi-omics approaches to disease
Hasin, Y., Seldin, M., & Lusis, A. (2017) · Genome Biology · doi:10.1186/s13059-017-1215-1 · Cited by 2,953
Araport11: a complete reannotation of the Arabidopsis thaliana reference genome
Cheng, C., Krishnakumar, V., Chan, A. P., Thibaud‐Nissen, F., Schobel, S., & Town, C. D. (2017) · The Plant Journal · doi:10.1111/tpj.13415 · Cited by 1,317
Swarm Intelligence-Enhanced Detection of Non-Small-Cell Lung Cancer Using Tumor-Educated Platelets
Best, M. G., Sol, N., In ‘t Veld, S. G. J. G., Vancura, A., Muller, M., Niemeijer, A.-L. N., Fejes, A. V., Tjon Kon Fat, L.-A., Huis In ‘t Veld, A. E., Leurs, C., et al. (2017) · Cancer Cell · doi:10.1016/j.ccell.2017.07.004 · Cited by 343
Resolving host–pathogen interactions by dual RNA-seq
Westermann, A. J., Barquist, L., & Vogel, J. (2017) · PLOS Pathogens · doi:10.1371/journal.ppat.1006033 · Cited by 289
Deliu, E., Arecco, N., Morandell, J., Dotter, C. P., Contreras, X., Girardot, C., Käsper, E.-L., Kozlova, A., Kishi, K., Chiaradia, I., et al. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0266-2 · Cited by 89
Metadata
- Authors
- Peixoto, L., Risso, D., Poplawski, S. G., Wimmer, M. E., Speed, T. P., Wood, M. A., & Abel, T.
- Affiliations
- University of California, Berkeley (US)
- University of Pennsylvania (US)
- The University of Melbourne (AU)
- University of California, Irvine (US)
- Corresponding author
- Abel, Ted
- Venue
- Nucleic Acids Research
- Publisher
- Oxford University Press (OUP)
- Published
- 2015-07-21
- Volume
- 43
- Issue
- 16
- Pages
- 7664-7674
- ISSN
- 0305-1048 (ISSN-L)
- 1362-4962
- PMC
- PMC4652761
- OpenAlex
- W1910607297
- Cited by
- 105
- References
- 46
- Citations by year
- 2026: 6 · 2025: 9 · 2024: 3 · 2023: 15 · 2022: 10 · 2021: 15 · 2020: 9 · 2019: 9 · 2018: 12 · 2017: 5 · 2016: 10 · 2015: 2
- FWCI
- 4.19
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- Nucleic Acids Research (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Minerva Access (University of Melbourne) (submittedVersion, other-oa)
- Padua Research Archive (University of Padova) (submittedVersion, cc-by)
- eScholarship (California Digital Library) (submittedVersion)
- Funders
- National Institutes of Health
- Defense Advanced Research Projects Agency (58077LSDRP, 58077, DARPA 58077 LSDRP)
- National Heart, Lung, and Blood Institute (T32HL007953)
- National Institute of Mental Health (R01MH101491, R01MH087463)
- National Institute on Drug Abuse (DA 036984, R01DA036984)
- National Institute of General Medical Sciences (T32GM008076)
- National Institute of Neurological Disorders and Stroke (T32NS007413)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene expression and cancer classification
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Phylogenetic Studies
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Single-cell and spatial transcriptomics
- Keywords
- RNA-seq normalization, batch effects, statistical power, reproducibility, gene expression, sequencing depth, global normalization, transcriptome sequencing
- MeSH terms
- Animals, Male, Mice, Inbred C57BL, Neurosciences, Genetic Variation, Reproducibility of Results, Sequence Analysis, RNA, Gene Expression Profiling
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.