Using control genes to correct for unwanted variation in microarray data
Open accessAbstract
Microarray expression studies suffer from the problem of batch effects and other unwanted variation. Many methods have been proposed to adjust microarray data to mitigate the problems of unwanted variation. Several of these methods rely on factor analysis to infer the unwanted variation from the data. A central problem with this approach is the difficulty in discerning the unwanted variation from the biological variation that is of interest to the researcher. We present a new method, intended for use in differential expression studies, that attempts to overcome this problem by restricting the factor analysis to negative control genes. Negative control genes are genes known a priori not to be differentially expressed with respect to the biological factor of interest. Variation in the expression levels of these genes can therefore be assumed to be unwanted variation. We name this method "Remove Unwanted Variation, 2-step" (RUV-2). We discuss various techniques for assessing the performance of an adjustment method and compare the performance of RUV-2 with that of other commonly used adjustment methods such as Combat and Surrogate Variable Analysis (SVA). We present several example studies, each concerning genes differentially expressed with respect to gender in the brain and find that RUV-2 performs as well or better than other methods. Finally, we discuss the possibility of adapting RUV-2 for use in studies not concerned with differential expression and conclude that there may be promise but substantial challenges remain.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In large-scale RNA-seq studies, commonly used normalization methods (cqn, EDASeq, RUV2, sva, PEER) differ in how well they remove systematic biases, depending on sample complexity and initial data quality.
“commonly used methods fornormalization (cqn, EDASeq, RUV2, sva, PEER) varied in their ability to remove these systematic biases, depending on sample complexity and initial data quality.”
From Detecting and correcting systematic variation in large-scale RNA sequencing data (2014) · cites this paper as [31]
RUV2 with ERCC spike-ins removed on average about 20% of false positives and was more effective at sites with already low false positive rates (ILM4, ILM5).
“The application of RUV2 with ERCC spike-ins (RUV2-ERCC), which tries to remove confounding factors based on a control set of synthetic RNAs assessed across sites, on average removes just 20% of false positives on average, but is more effective for sites that already have relatively low false positive rates (ILM4, ILM5).”
From Detecting and correcting systematic variation in large-scale RNA sequencing data (2014) · cites this paper as [31]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Modern Applied Statistics with S
Venables, W., & Ripley, B. (2010) · Statisctics and computing/Statistics and computing · doi:10.1007/978-0-387-21706-2 · Cited by 16,188 · see citing works
Smyth, G. K. (2004) · Statistical Applications in Genetics and Molecular Biology · doi:10.2202/1544-6115.1027 · Cited by 11,829 · see citing works
Exploration, normalization, and summaries of high density oligonucleotide array probe level data
Irizarry, R. A. (2003) · Biostatistics · doi:10.1093/biostatistics/4.2.249 · Cited by 10,603 · see citing works
Principal components analysis corrects for stratification in genome-wide association studies
Price, A. L., Patterson, N. J., Plenge, R. M., Weinblatt, M. E., Shadick, N. A., & Reich, D. (2006) · Nature Genetics · doi:10.1038/ng1847 · Cited by 10,248 · see citing works
Adjusting batch effects in microarray expression data using empirical Bayes methods
Johnson, W. E., Li, C., & Rabinovic, A. (2007) · Biostatistics · doi:10.1093/biostatistics/kxj037 · Cited by 8,658 · see citing works
Bolstad, B. M., Irizarry, R. A., Åstrand, M., & Speed, T. P. (2003) · Bioinformatics · doi:10.1093/bioinformatics/19.2.185 · Cited by 8,185 · see citing works
Self-citationSummaries of Affymetrix GeneChip probe level data
Irizarry, R. A. (2003) · Nucleic Acids Research · doi:10.1093/nar/gng015 · Cited by 4,906 · see citing works
Yu, J., Pressoir, G., Briggs, W. H., Bi, I. V., Yamasaki, M., Doebley, J., McMullen, M. D., Gaut, B. S., Nielsen, D. M., Holland, J. B., et al. (2005) · Nature Genetics · doi:10.1038/ng1702 · Cited by 4,155 · see citing works
Bioinformatics and Computational Biology Solutions Using R and Bioconductor
Gentleman, R. C., Carey, V. J., Huber, W. K., Irizarry, R. A., & Dudoit, S. (2005) · Statistics in the health sciences · doi:10.1007/0-387-29362-0 · Cited by 3,470 · see citing works
Variance component model to account for sample structure in genome-wide association studies
Kang, H. M., Sul, J. H., Service, S. K., Zaitlen, N. A., Kong, S.-Y., Freimer, N. B., Sabatti, C., & Eskin, E. (2010) · Nature Genetics · doi:10.1038/ng.548 · Cited by 2,817 · see citing works
Tackling the widespread and critical impact of batch effects in high-throughput data
Leek, J. T., Scharpf, R. B., Bravo, H. C., Simcha, D., Langmead, B., Johnson, W. E., Geman, D., Baggerly, K., & Irizarry, R. A. (2010) · Nature Reviews Genetics · doi:10.1038/nrg2825 · Cited by 2,196 · see citing works
MAQC Consortium, Shi, L., Shi, L., Reid, L. H., Jones, W. D., Shippy, R., Warrington, J. A., Baker, S. C., Collins, P. J., de Longueville, F., et al. (2006) · Nature Biotechnology · doi:10.1038/nbt1239 · Cited by 2,126 · see citing works
Singular value decomposition for genome-wide expression data processing and modeling
Alter, O., Brown, P. O., & Botstein, D. (2000) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.97.18.10101 · Cited by 1,989 · see citing works
Capturing Heterogeneity in Gene Expression Studies by Surrogate Variable Analysis
Leek, J. T., & Storey, J. D. (2007) · PLoS Genetics · doi:10.1371/journal.pgen.0030161 · Cited by 1,930 · see citing works
Efficient Control of Population Structure in Model Organism Association Mapping
Kang, H. M., Zaitlen, N., Wade, C. M., Kirby, A. W., Heckerman, D. E., Daly, M. J., & Eskin, E. (2008) · Genetics · doi:10.1534/genetics.107.080101 · Cited by 1,848 · see citing works
ROBPCA: A New Approach to Robust Principal Component Analysis
Hubert, M., Rousseeuw, P. J., & Vanden Branden, K. (2005) · Technometrics · doi:10.1198/004017004000000563 · Cited by 1,052 · see citing works
Blalock, E. M., Geddes, J. W., Chen, K. C., Porter, N. M., Markesbery, W. R., & Landfield, P. W. (2004) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0308512100 · Cited by 1,025 · see citing works
Human housekeeping genes are compact
Eisenberg, E., & Levanon, E. Y. (2003) · Trends in Genetics · doi:10.1016/s0168-9525(03)00140-9 · Cited by 699 · see citing works
Book Review: Pattern Recognition and Machine Learning
Nasrabadi, N. M. (2007) · Journal of Electronic Imaging · doi:10.1117/1.2819119 · Cited by 636 · see citing works
Molecular characterisation of soft tissue tumours: a gene expression study
Nielsen, T. O., West, R. B., Linn, S. C., Alter, O., Knowling, M. A., OʼConnell, J. X., Zhu, S., Fero, M., Sherlock, G. J., Pollack, J. R., et al. (2002) · The Lancet · doi:10.1016/s0140-6736(02)08270-3 · Cited by 574 · see citing works
A general framework for multiple testing dependence
Leek, J. T., & Storey, J. D. (2008) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0808709105 · Cited by 365 · see citing works
Shankavaram, U. T., Reinhold, W. C., Nishizuka, S., Major, S. M., Morita, D., Chary, K., Reimers, M. A., Scherf, U., Kahn, A., Dolginow, D., et al. (2007) · Molecular Cancer Therapeutics · doi:10.1158/1535-7163.mct-06-0650 · Cited by 330 · see citing works
Gender-Specific Gene Expression in Post-Mortem Human Brain: Localization to Sex Chromosomes
Vawter, M. P., Evans, S. J., Choudary, P. V., Tomita, H., Meador-Woodruff, J., Molnar, M., Li, J. Z., López, J. F., Myers, R. M., Cox, D. R., et al. (2003) · Neuropsychopharmacology · doi:10.1038/sj.npp.1300337 · Cited by 204 · see citing works
Quality Assessment of Affymetrix GeneChip Data
Bolstad, B. M., Collin, F., Brettschneider, J. A., Simpson, K., Cope, L., Irizarry, R. A., & Speed, T. P. (2005) · Springer eBooks · doi:10.1007/0-387-29362-0_3 · Cited by 197 · see citing works
Self-citationBengtsson, H. L., Simpson, K., Bullard, J. W., & Hansen, K. D. (2008) · UC Berkeley · Cited by 149 · see citing works
Kang, H. M., Ye, C., & Eskin, E. (2008) · Genetics · doi:10.1534/genetics.108.094201 · Cited by 142 · see citing works
Correction for hidden confounders in the genetic analysis of gene expression
Listgarten, J., Kadie, C. M., Schadt, E. E., & Heckerman, D. E. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1002425107 · Cited by 141 · see citing works
Quality Assessment for Short Oligonucleotide Microarray Data
Brettschneider, J. A., Collin, F., Bolstad, B. M., & Speed, T. P. (2008) · Technometrics · doi:10.1198/004017008000000334 · Cited by 132 · see citing works
Self-citationStamova, B. S., Apperson, M., Walker, W. L., Tian, Y., Xu, H., Adamczy, P., Zhan, X., Liu, D., Ander, B. P., Liao, I. H., et al. (2009) · BMC Medical Genomics · doi:10.1186/1755-8794-2-49 · Cited by 132 · see citing works
Supervised normalization of microarrays
Mecham, B. H., Nelson, P. S., & Storey, J. D. (2010) · Bioinformatics · doi:10.1093/bioinformatics/btq118 · Cited by 130 · see citing works
Sparse Statistical Modelling in Gene Expression Genomics
Lucas, J. E., Carvalho, C. M., Wang, Q., Bild, A., Nevins, J. R., & West, M. (2006) · Cambridge University Press eBooks · doi:10.1017/cbo9780511584589.009 · Cited by 103 · see citing works
Batch Effects and Noise in Microarray Experiments: Sources and Solutions
Scherer, A. (2009) · Cited by 98 · see citing works
Bayesian Inference for Gene Expression and Proteomics
Vannucci, M. (2006) · Cambridge University Press eBooks · doi:10.1017/cbo9780511584589 · Cited by 94 · see citing works
Accounting for Non-genetic Factors Improves the Power of eQTL Studies
Stegle, O., Kannan, A., Durbin, R. M., & Winn, J. M. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-78839-3_35 · Cited by 29 · see citing works
Exploring the use of internal and externalcontrols for assessing microarray technical performance
Lippa, K. A., Duewer, D. L., Salit, M., Gamé, L., & Causton, H. C. (2010) · BMC Research Notes · doi:10.1186/1756-0500-3-349 · Cited by 15 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- BISHOP, C. M. (2006). Pattern Recognition and Machine Learning. New York: Springer.
Cited by
Sorted by citations
Leek, J. T., Johnson, W. E., Parker, H. S., Jaffe, A. E., & Storey, J. D. (2012) · Bioinformatics · doi:10.1093/bioinformatics/bts034 · Cited by 6,231 · see citing works
Harmonization of multi-site diffusion tensor imaging data
Fortin, J.-P., Parker, D., Tunç, B., Watanabe, T., Elliott, M. A., Ruparel, K., Roalf, D. R., Satterthwaite, T. D., Gur, R. C., Gur, R. E., et al. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2017.08.047 · Cited by 1,261 · see citing works
Buettner, F., Natarajan, K. N., Casale, F. P., Proserpio, V., Scialdone, A., Theis, F. J., Teichmann, S. A., Marioni, J. C., & Stegle, O. (2015) · Nature Biotechnology · doi:10.1038/nbt.3102 · Cited by 1,136 · see citing works
Accounting for cellular heterogeneity is critical in epigenome-wide association studies
Jaffe, A. E., & Irizarry, R. A. (2014) · Genome Biology · doi:10.1186/gb-2014-15-2-r31 · Cited by 1,067 · see citing works
DNA methylation in newborns and maternal smoking in pregnancy: genome-wide consortium meta-analysis
Joubert, B. R., Felix, J. F., Yousefi, P., Bakulski, K. M., Just, A. C., Breton, C., Reese, S. E., Markunas, C. A., Richmond, R. C., Xu, C.-J., et al. (2016) · The American Journal of Human Genetics · doi:10.1016/j.ajhg.2016.02.019 · Cited by 886 · see citing works
Revealing the vectors of cellular identity with single-cell genomics
Wagner, A., Regev, A., & Yosef, N. (2016) · Nature Biotechnology · doi:10.1038/nbt.3711 · Cited by 632 · see citing works
A general and flexible method for signal extraction from single-cell RNA-seq data
Risso, D., Perraudeau, F., Gribkova, S., Dudoit, S., & Vert, J.-P. (2018) · Nature Communications · doi:10.1038/s41467-017-02554-5 · Cited by 718 · see citing works
the BIOS Consortium, van Iterson, M., van Zwet, E. W., & Heijmans, B. T. (2017) · Genome biology · doi:10.1186/s13059-016-1131-9 · Cited by 421 · see citing works
Ratnapriya, R., Sosina, O. A., Starostik, M. R., Kwicklis, M., Kapphahn, R. J., Fritsche, L. G., Walton, A., Arvanitis, M., Gieser, L., Pietraszkiewicz, A., et al. (2019) · Nature Genetics · doi:10.1038/s41588-019-0351-9 · Cited by 312 · see citing works
Batch effect removal methods for microarray gene expression data integration: a survey
Lazar, C., Meganck, S., Taminau, J., Steenhoff, D., Coletta, A., Molter, C., Weiss-Solis, D. Y., Duque, R., Bersini, H., & Nowe, A. (2013) · Briefings in Bioinformatics · doi:10.1093/bib/bbs037 · Cited by 333 · see citing works
Observation weights unlock bulk RNA-seq tools for zero inflation and single-cell applications
Van den Berge, K., Perraudeau, F., Soneson, C., Love, M. I., Risso, D., Vert, J.-P., Robinson, M. D., Dudoit, S., & Clement, L. (2018) · Genome Biology · doi:10.1186/s13059-018-1406-4 · Cited by 228 · see citing works
Common dysregulation network in the human prefrontal cortex underlies two neurodegenerative diseases
Narayanan, M., Huynh, J. L., Wang, K., Yang, X., Yoo, S., McElwee, J., Zhang, B., Zhang, C., Lamb, J. R., Xie, T., et al. (2014) · Molecular Systems Biology · doi:10.15252/msb.20145304 · Cited by 208 · see citing works
Detecting and correcting systematic variation in large-scale RNA sequencing data
Li, S., Łabaj, P. P., Zumbo, P., Sykacek, P., Shi, W., Shi, L., Phan, J., Wu, P.-Y., Wang, M., Wang, C., et al. (2014) · Nature Biotechnology · doi:10.1038/nbt.3000 · Cited by 175 · see citing works
Evaluation of chromatin accessibility in prefrontal cortex of individuals with schizophrenia
Bryois, J., Garrett, M. E., Song, L., Safi, A., Giusti-Rodriguez, P., Johnson, G. D., Shieh, A. W., Buil, A., Fullard, J. F., Roussos, P., et al. (2018) · Nature Communications · doi:10.1038/s41467-018-05379-y · Cited by 167 · see citing works
Performance Assessment and Selection of Normalization Procedures for Single-Cell RNA-Seq
Cole, M. B., Risso, D., Wagner, A., DeTomaso, D., Ngai, J., Purdom, E., Dudoit, S., & Yosef, N. (2019) · Cell Systems · doi:10.1016/j.cels.2019.03.010 · Cited by 173 · see citing works
Predictability of human differential gene expression
Crow, M., Lim, N., Ballouz, S., Pavlidis, P., & Gillis, J. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1802973116 · Cited by 117 · see citing works
Peixoto, L., Risso, D., Poplawski, S. G., Wimmer, M. E., Speed, T. P., Wood, M. A., & Abel, T. (2015) · Nucleic Acids Research · doi:10.1093/nar/gkv736 · Cited by 105 · see citing works
Self-citationEvaluating stably expressed genes in single cells
Lin, Y., Ghazanfar, S., Strbenac, D., Wang, A., Patrick, E., Lin, D. M., Speed, T., Yang, J. Y. H., & Yang, P. (2019) · GigaScience · doi:10.1093/gigascience/giz106 · Cited by 84 · see citing works
Self-citationComputational methods for the integrative analysis of single-cell data
Forcato, M., Romano, O., & Bicciato, S. (2021) · Briefings in Bioinformatics · doi:10.1093/bib/bbaa042 · Cited by 71 · see citing works
The contribution of cell cycle to heterogeneity in single-cell RNA-seq data
McDavid, A., Finak, G., & Gottardo, R. (2016) · Nature Biotechnology · doi:10.1038/nbt.3498 · Cited by 57 · see citing works
Removing unwanted variation with CytofRUV to integrate multiple CyTOF datasets
Trussart, M., Teh, C. E., Tan, T., Leong, L., Gray, D. H., & Speed, T. P. (2020) · eLife · doi:10.7554/elife.59630 · Cited by 51 · see citing works
Self-citationPapiez, A., Marczyk, M., Polanska, J., & Polanski, A. (2019) · Bioinformatics · doi:10.1093/bioinformatics/bty900 · Cited by 26 · see citing works
Postmortem human brain genomics in neuropsychiatric disorders — how far can we go?
Jaffe, A. E. (2016) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2015.11.002 · Cited by 19 · see citing works
- Authors
- Gagnon-Bartsch, J. A., & Speed, T. P.
- Affiliations
- University of California, Berkeley (US)
- Walter and Eliza Hall Institute of Medical Research (AU)
- Corresponding author
- Gagnon-Bartsch, J. A.
- Venue
- Biostatistics
- Publisher
- Oxford University Press (OUP)
- Published
- 2011-11-17
- Volume
- 13
- Issue
- 3
- Pages
- 539-552
- ISSN
- 1465-4644 (ISSN-L)
- 1468-4357
- PMC
- PMC3577104
- OpenAlex
- W2116650193
- Cited by
- 442
- References
- 35
- Citations by year
- 2026: 15 · 2025: 23 · 2024: 34 · 2023: 30 · 2022: 18 · 2021: 33 · 2020: 58 · 2019: 43 · 2018: 39 · 2017: 46 · 2016: 36 · 2015: 21 · 2014: 28 · 2013: 10 · 2012: 7
- FWCI
- 5.84
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Biostatistics (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- UC Berkeley (submittedVersion)
- Funders
- National Institutes of Health
- National Cancer Institute (U24CA126551)
- National Institute of General Medical Sciences (R01GM083084)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene expression and cancer classification
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Bioinformatics and Genomic Networks
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Single-cell and spatial transcriptomics
- Keywords
- batch effects, microarray normalization, factor analysis, differential gene expression, combat, surrogate variable analysis, brain gene expression, sex differences
- MeSH terms
- Data Interpretation, Statistical, Female, Humans, Male, Oligonucleotide Array Sequence Analysis, Gene Expression Profiling
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
