Capturing Heterogeneity in Gene Expression Studies by Surrogate Variable Analysis
Open accessAbstract
It has unambiguously been shown that genetic, environmental, demographic, and technical factors may have substantial effects on gene expression levels. In addition to the measured variable(s) of interest, there will tend to be sources of signal due to factors that are unknown, unmeasured, or too complicated to capture through simple models. We show that failing to incorporate these sources of heterogeneity into an analysis can have widespread and detrimental effects on the study. Not only can this reduce power or induce unwanted dependence across genes, but it can also introduce sources of spurious signal to many genes. This phenomenon is true even for well-designed, randomized studies. We introduce "surrogate variable analysis" (SVA) to overcome the problems caused by heterogeneity in expression studies. SVA can be applied in conjunction with standard analysis techniques to accurately capture the relationship between expression and any modeled variables of interest. We apply SVA to disease class, time course, and genetics of gene expression studies. We show that SVA increases the biological accuracy and reproducibility of analyses in genome-wide expression studies.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Surrogate variable analyses revealed 4 significant SVA vectors that were used as covariates in the model to correct for technical artefacts and hidden confounds.
“Surrogate variable analyses (SVAs) revealed 4 significant SVA vectors that were used as covariates in the model to correct for technical artefacts and hidden confounds (Leek & Storey, 2007).”
From DNA methylation of NR3C1 and FKBP5 is associated with posttraumatic stress disorder, posttraumatic growth, and resilience. (2020) · cites this paper as Leek & Storey, 2007
The Results section (“Surrogate Variables”; page not available) says SVA identifies significant vectors and incorporates the resulting surrogate variables as covariates in subsequent analyses. The paper supports using them to account for unmodeled variation, but it does not report that this citing analysis found four significant vectors or used four in its model. (LLM, conf=0.88)
Experimental — accuracy not yet measuredsva and PEER, which use data across all sites, removed 85.1% to 87.7% of the original total false positive DEGs.
“Some methods (specifically sva and PEER) that leveraged all data across all sites were quite successful at ameliorating the high rate of false positives (Fig. 1, Supplementary Fig. 2), removing 85.1% to 87.7% of the original total false positive DEGs.”
From Detecting and correcting systematic variation in large-scale RNA sequencing data (2014) · cites this paper as [27]
More than 85% of RNA-seq false positives could be removed by subtracting latent variable effects identified by sva or PEER through joint analysis of all genes across multiple sites, without decreasing sensitivity or specificity of DEG detection.
“However, the majority of RNA-seq false positives (>85%) could successfully be removed by subtracting the effects of latent variables identified by either sva26,27 or PEER30, which could be achieved by jointly analyzing the set of measurements of all genes across multiple sites, without a decrease in the sensitivity or specificity of DEG detection at each site or across sites.”
From Detecting and correcting systematic variation in large-scale RNA sequencing data (2014) · cites this paper as [27]
Surrogate variable correction was used to account for unmeasured confounders, a standard approach.
“We used surrogate variable correction in our analyses to account for such unmeasured confounders (43), which is a standard approach (44).”
From Transcriptome-wide isoform-level dysregulation in ASD, schizophrenia, and bipolar disorder (2018) · cites this paper as [43]
After adjusting for surrogate variables, similar tissue and cell types clustered together across experiments.
“We used surrogate variable analysis (SVA) and 26 loosely described “biologic clusters” of tissues and cells to clearly demonstrate that, after adjusting for surrogate variables, similar tissue and cell types clustered together across experiments (Supplemental Fig. S3; Leek and Storey 2007).”
From Toward the human cellular microRNAome (2017) · cites this paper as Leek and Storey 2007
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Cluster analysis and display of genome-wide expression patterns
Eisen, M. B., Spellman, P. T., Brown, P. O., & Botstein, D. (1998) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.95.25.14863 · Cited by 12,617 · see citing works
The control of the false discovery rate in multiple testing under dependency
Benjamini, Y., & Yekutieli, D. (2001) · The Annals of Statistics · doi:10.1214/aos/1013699998 · Cited by 10,490 · see citing works
Principal components analysis corrects for stratification in genome-wide association studies
Price, A. L., Patterson, N. J., Plenge, R. M., Weinblatt, M. E., Shadick, N. A., & Reich, D. (2006) · Nature Genetics · doi:10.1038/ng1847 · Cited by 10,248 · see citing works
Statistical significance for genomewide studies
Storey, J. D., & Tibshirani, R. (2003) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1530509100 · Cited by 9,692 · see citing works
Self-citationBrown, R. A., Hastie, T. J., & Tibshirani, R. J. (1991) · Biometrics · doi:10.2307/2532174 · Cited by 8,262 · see citing works
A Direct Approach to False Discovery Rates
Storey, J. D. (2002) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/1467-9868.00346 · Cited by 5,603 · see citing works
Self-citationLamb, J., Crawford, E. D., Peck, D., Modell, J. W., Blat, I. C., Wrobel, M. J., Lerner, J., Brunet, J.-P., Subramanian, A., Ross, K. N., et al. (2006) · Science · doi:10.1126/science.1132939 · Cited by 5,257 · see citing works
Testing Statistical Hypotheses
Niewiadomska-Bugaj, M., & Bartoszynski, R. (2021) · Wiley series in probability and statistics · doi:10.1002/9781119243830.ch12 · Cited by 5,188 · see citing works
Exploring the metabolic and genetic control of gene expression on a genomic scale.
Derisi, J., Iyer, V., & Brown, P. (1997) · Science · doi:10.1126/science.278.5338.680 · Cited by 4,832 · see citing works
Genomic Expression Programs in the Response of Yeast Cells to Environmental Changes
Gasch, A. P., Spellman, P. T., Kao, C. M., Carmel-Harel, O., Eisen, M. B., Storz, G., Botstein, D., & Brown, P. O. (2000) · Molecular Biology of the Cell · doi:10.1091/mbc.11.12.4241 · Cited by 4,815 · see citing works
Yang, Y., Dudoit, S., Luu, P., Lin, D., Peng, V., Ngai, J., & Speed, T. (2002) · Nucleic Acids Research · doi:10.1093/nar/30.4.e15 · Cited by 3,722 · see citing works
Hastie, T. J., & Tibshirani, R. J. (2014) · Wiley StatsRef: Statistics Reference Online · doi:10.1002/9781118445112.stat03141 · Cited by 3,156 · see citing works
Mathematical Statistics and Data Analysis
Rabinowitz, L., & Rice, J. A. (1989) · Technometrics · doi:10.2307/3556160 · Cited by 3,142 · see citing works
Singular value decomposition for genome-wide expression data processing and modeling
Alter, O., Brown, P. O., & Botstein, D. (2000) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.97.18.10101 · Cited by 1,989 · see citing works
Genetics of gene expression surveyed in maize, mouse and man
Schadt, E., Monks, S. A., Drake, T., Lusis, A., Che, N., Colinayo, V. V., Ruff, T., Milligan, S. B., Lamb, J., Cavet, G., et al. (2003) · Nature · doi:10.1038/nature01434 · Cited by 1,594 · see citing works
Genetic Dissection of Transcriptional Regulation in Budding Yeast
Brem, R. B., Yvert, G., Clinton, R., & Kruglyak, L. (2002) · Science · doi:10.1126/science.1069516 · Cited by 1,392 · see citing works
Analysis of Variance for Gene Expression Microarray Data
Kerr, K. F., Martin, M., & Churchill, G. A. (2000) · Journal of Computational Biology · doi:10.1089/10665270050514954 · Cited by 1,310 · see citing works
Generalized additive models for medical research
Hastie, T. J., & Tibshirani, R. J. (1995) · Statistical Methods in Medical Research · doi:10.1177/096228029500400302 · Cited by 1,275 · see citing works
Genetic analysis of genome-wide variation in human gene expression
Morley, M. P., Molony, C. M., Weber, T. M., Devlin, J. L., Ewens, K. G., Spielman, R. S., & Cheung, V. G. (2004) · Nature · doi:10.1038/nature02797 · Cited by 1,207 · see citing works
Large-Scale Simultaneous Hypothesis Testing: The Choice of a Null Hypothesis
Efron, B. (2004) · Journal of the American Statistical Association · doi:10.1198/016214504000000089 · Cited by 1,005 · see citing works
Gene-expression profiles in hereditary breast cancer.
Hedenfalk, I., Duggan, D., Chen, Y.-D., Radmacher, M., Bittner, M., Simon, R., Meltzer, P., Gusterson, B., Esteller, M., Raffeld, M., et al. (2001) · New England Journal of Medicine · doi:10.1056/nejm200102223440801 · Cited by 978 · see citing works
Experimental design for gene expression microarrays
Kerr, K. F. (2001) · Biostatistics · doi:10.1093/biostatistics/2.2.183 · Cited by 709 · see citing works
Trans-acting regulatory variation in Saccharomyces cerevisiae and the role of transcription factors
Yvert, G., Brem, R. B., Whittle, J., Akey, J. M., Foss, E. J., Smith, E. N., Mackelprang, R., & Kruglyak, L. (2003) · Nature Genetics · doi:10.1038/ng1222 · Cited by 629 · see citing works
Loss of silent-chromatin looping and impaired imprinting of DLX5 in Rett syndrome
Horike, S., Cai, S., Miyano, M., Cheng, J., & Kohwi-shigematsu, T. (2004) · Nature Genetics · doi:10.1038/ng1491 · Cited by 585 · see citing works
Tseng, G. C. (2001) · Nucleic Acids Research · doi:10.1093/nar/29.12.2549 · Cited by 574 · see citing works
Significance analysis of time course microarray experiments
Storey, J. D., Xiao, W., Leek, J. T., Tompkins, R. G., & Davis, R. W. (2005) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0504609102 · Cited by 567 · see citing works
Self-citationBuja, A., & Eyuboglu, N. (1992) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr2704_2 · Cited by 563 · see citing works
Integrative analysis of the cancer transcriptome
Rhodes, D. R., & Chinnaiyan, A. M. (2005) · Nature Genetics · doi:10.1038/ng1570 · Cited by 490 · see citing works
Fundamental patterns underlying gene expression profiles: Simplicity from complexity
Holter, N. S., Mitra, M., Maritan, A., Cieplak, M., Banavar, J. R., & Fedoroff, N. V. (2000) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.150242097 · Cited by 475 · see citing works
Correlation and Large-Scale Simultaneous Significance Testing
Efron, B. (2007) · Journal of the American Statistical Association · doi:10.1198/016214506000001211 · Cited by 405 · see citing works
A Transcriptional Profile of Aging in the Human Kidney
Rodwell, G. E. J., Sonu, R., Zahn, J. M., Lund, J., Wilhelmy, J., Wang, L., Xiao, W., Mindrinos, M., Crane, E., Segal, E., et al. (2004) · PLoS Biology · doi:10.1371/journal.pbio.0020427 · Cited by 338 · see citing works
Amundson, S. A., Bittner, M., Chen, Y., Trent, J. M., Meltzer, P. S., & Fornace, A. J. (1999) · Oncogene · doi:10.1038/sj.onc.1202676 · Cited by 338 · see citing works
Genetic interactions between polymorphisms that affect gene expression in yeast
Brem, R. B., Storey, J. D., Whittle, J., & Kruglyak, L. (2005) · Nature · doi:10.1038/nature03865 · Cited by 298 · see citing works
Self-citationMultiple Locus Linkage Analysis of Genomewide Expression in Yeast
Storey, J. D., Akey, J. M., & Kruglyak, L. (2005) · PLoS Biology · doi:10.1371/journal.pbio.0030267 · Cited by 195 · see citing works
Self-citationMolecular classification of familial non- BRCA1/BRCA2 breast cancer
Hedenfalk, I. A., Ringnér, M., Ben‐Dor, A., Zohar Yakhini, B., Chen, Y., Chebil, G., Ach, R. A., Loman, N., Olsson, H. L., Meltzer, P. S., et al. (2003) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0533805100 · Cited by 180 · see citing works
Preter, K. D., Vandesompele, J., Heimann, P., Yigit, N., Beckman, S., Schramm, A. R., Eggert, A., Stallings, R. L., Benoît, Y., Renard, M., et al. (2006) · Genome biology · doi:10.1186/gb-2006-7-9-r84 · Cited by 162 · see citing works
Variance of the Number of False Discoveries
Owen, A. B. (2005) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.1467-9868.2005.00509.x · Cited by 142 · see citing works
Qiu, X., Klebanov, L. B., & Yakovlev, A. (2005) · Statistical Applications in Genetics and Molecular Biology · doi:10.2202/1544-6115.1157 · Cited by 108 · see citing works
Molecular Heterogeneity of Inflammatory Breast Cancer: A Hyperproliferative Phenotype
Nguyen, D., Sam, K., Tsimelzon, A. I., Li, X., Wong, H. H., Mohsin, S. K., Clark, G. M., Hilsenbeck, S. G., Elledge, R. M., Allred, D. C., et al. (2006) · Clinical Cancer Research · doi:10.1158/1078-0432.ccr-05-2248 · Cited by 81 · see citing works
Assessing stability of gene selection in microarray data analysis
Qiu, X., Xiao, Y., Gordon, A. Y., & Yakovlev, A. (2006) · BMC Bioinformatics · doi:10.1186/1471-2105-7-50 · Cited by 78 · see citing works
A reanalysis of a published Affymetrix GeneChip control dataset.
Dabney, A. R., & Storey, J. D. (2006) · Genome biology · doi:10.1186/gb-2006-7-3-401 · Cited by 56 · see citing works
Self-citationEstimation of false discovery proportion under general dependence
Pawitan, Y., Calza, S., & Ploner, A. (2006) · Bioinformatics · doi:10.1093/bioinformatics/btl527 · Cited by 43 · see citing works
SOME COMMENTS ON INSTABILITY OF FALSE DISCOVERY RATE ESTIMATION
Qiu, X., & YAKOVLEV, A. (2006) · Journal of Bioinformatics and Computational Biology · doi:10.1142/s0219720006002338 · Cited by 38 · see citing works
A new approach to intensity-dependent normalization of two-channel microarrays
Dabney, A. R., & Storey, J. D. (2006) · Biostatistics · doi:10.1093/biostatistics/kxj038 · Cited by 29 · see citing works
Self-citationKlebanov, L. B., & Yakovlev, A. (2006) · Statistical Applications in Genetics and Molecular Biology · doi:10.2202/1544-6115.1185 · Cited by 22 · see citing works
Modified Simes’ critical values under positive dependence
Cai, G., & Sarkar, S. K. (2005) · Journal of Statistical Planning and Inference · doi:10.1016/j.jspi.2005.06.004 · Cited by 12 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- Mardia KV Kent JT Bibby JM 1980 Multivariate analysis London Academic Press
Cited by
Sorted by citations
Fast, sensitive and accurate integration of single-cell data with Harmony
Korsunsky, I., Millard, N., Fan, J., Slowikowski, K., Zhang, F., Wei, K., Baglaenko, Y., Brenner, M., Loh, P.-R., & Raychaudhuri, S. (2019) · Nature Methods · doi:10.1038/s41592-019-0619-0 · Cited by 10,488 · see citing works
Leek, J. T., Johnson, W. E., Parker, H. S., Jaffe, A. E., & Storey, J. D. (2012) · Bioinformatics · doi:10.1093/bioinformatics/bts034 · Cited by 6,231 · see citing works
Self-citationGenetic effects on gene expression across human tissues
GTEx Consortium (2017) · Nature · doi:10.1038/nature24277 · Cited by 4,036 · see citing works
DNA methylation arrays as surrogate measures of cell mixture distribution
Houseman, E. A., Accomando, W. P., Koestler, D. C., Christensen, B. C., Marsit, C. J., Nelson, H. H., Wiencke, J. K., & Kelsey, K. T. (2012) · BMC Bioinformatics · doi:10.1186/1471-2105-13-86 · Cited by 3,175 · see citing works
DNA methylation-based classification of central nervous system tumours
Capper, D., Jones, D., Sill, M., Hovestadt, V., Schrimpf, D., Sturm, D., Koelsche, C., Sahm, F., Chávez, L., Reuß, D., et al. (2018) · Nature · doi:10.1038/nature26000 · Cited by 3,046 · see citing works
Tackling the widespread and critical impact of batch effects in high-throughput data
Leek, J. T., Scharpf, R. B., Bravo, H. C., Simcha, D., Langmead, B., Johnson, W. E., Geman, D., Baggerly, K., & Irizarry, R. A. (2010) · Nature Reviews Genetics · doi:10.1038/nrg2825 · Cited by 2,196 · see citing works
Self-citationNormalization of RNA-seq data using factor analysis of control genes or samples
Risso, D., Ngai, J., Speed, T. P., & Dudoit, S. (2014) · Nature Biotechnology · doi:10.1038/nbt.2931 · Cited by 2,169 · see citing works
Matrix eQTL: ultra fast eQTL analysis via large matrix operations
Shabalin, A. A. (2012) · Bioinformatics · doi:10.1093/bioinformatics/bts163 · Cited by 1,694 · see citing works
McCarthy, D. J., Campbell, K. R., Lun, A. T. L., & Wills, Q. F. (2017) · Bioinformatics · doi:10.1093/bioinformatics/btw777 · Cited by 1,852 · see citing works
Challenges of Big Data analysis
Fan, J., Han, F., & Liu, H. (2014) · National Science Review · doi:10.1093/nsr/nwt032 · Cited by 1,482 · see citing works
Functional aspects of meningeal lymphatics in ageing and Alzheimer’s disease
Da Mesquita, S., Louveau, A., Vaccari, A., Smirnov, I., Cornelison, R. C., Kingsmore, K. M., Contarino, C., Onengut-Gumuscu, S., Farber, E., Raper, D., et al. (2018) · Nature · doi:10.1038/s41586-018-0368-8 · Cited by 1,445 · see citing works
Functional genomic landscape of acute myeloid leukaemia
Tyner, J. W., Tognon, C. E., Bottomly, D., Wilmot, B., Kurtz, S. E., Savage, S. L., Long, N., Schultz, A. R., Traer, E. A., Abel, M. L., et al. (2018) · Nature · doi:10.1038/s41586-018-0623-z · Cited by 1,330 · see citing works
Harmonization of multi-site diffusion tensor imaging data
Fortin, J.-P., Parker, D., Tunç, B., Watanabe, T., Elliott, M. A., Ruparel, K., Roalf, D. R., Satterthwaite, T. D., Gur, R. C., Gur, R. E., et al. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2017.08.047 · Cited by 1,261 · see citing works
Transcriptome-wide isoform-level dysregulation in ASD, schizophrenia, and bipolar disorder
Gandal, M. J., Zhang, P., Hadjimichael, E., Walker, R. L., Chen, C., Liu, S., Won, H., van Bakel, H., Varghese, M., Wang, Y., et al. (2018) · Science · doi:10.1126/science.aat8127 · Cited by 1,247 · see citing works
Landscape of X chromosome inactivation across human tissues
GTEx Consortium, Tukiainen, T., Villani, A.-C., Yen, A., Rivas, M. A., Marshall, J. L., Satija, R., Aguirre, M., Gauthier, L., Fleharty, M., et al. (2017) · Nature · doi:10.1038/nature24265 · Cited by 1,138 · see citing works
Computational and analytical challenges in single-cell transcriptomics
Stegle, O., Teichmann, S. A., & Marioni, J. C. (2015) · Nature Reviews Genetics · doi:10.1038/nrg3833 · Cited by 1,170 · see citing works
Count-based differential expression analysis of RNA sequencing data using R and Bioconductor
Anders, S., McCarthy, D. J., Chen, Y., Okoniewski, M. J., Smyth, G. K., Huber, W. K., & Robinson, M. D. (2013) · Nature Protocols · doi:10.1038/nprot.2013.099 · Cited by 1,129 · see citing works
Buettner, F., Natarajan, K. N., Casale, F. P., Proserpio, V., Scialdone, A., Theis, F. J., Teichmann, S. A., Marioni, J. C., & Stegle, O. (2015) · Nature Biotechnology · doi:10.1038/nbt.3102 · Cited by 1,136 · see citing works
Methods of integrating data to uncover genotype-phenotype interactions
Ritchie, M. D., Holzinger, E. R., Li, R., Pendergrass, S. A., & Kim, D. (2015) · Nature Reviews Genetics · doi:10.1038/nrg3868 · Cited by 1,085 · see citing works
A simple new approach to variable selection in regression, with application to genetic fine mapping
Wang, G., Sarkar, A., Carbonetto, P., & Stephens, M. (2020) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/rssb.12388 · Cited by 1,121 · see citing works
False discovery rates: a new deal
Stephens, M. (2016) · Biostatistics · doi:10.1093/biostatistics/kxw041 · Cited by 1,053 · see citing works
Stegle, O., Parts, L., Piipari, M., Winn, J., & Durbin, R. (2012) · Nature Protocols · doi:10.1038/nprot.2011.457 · Cited by 1,049 · see citing works
Accounting for cellular heterogeneity is critical in epigenome-wide association studies
Jaffe, A. E., & Irizarry, R. A. (2014) · Genome Biology · doi:10.1186/gb-2014-15-2-r31 · Cited by 1,067 · see citing works
The role of regulatory variation in complex traits and disease
Albert, F. W., & Kruglyak, L. (2015) · Nature Reviews Genetics · doi:10.1038/nrg3891 · Cited by 1,047 · see citing works
Liu, Y., Aryee, M. J., Padyukov, L., Fallin, M. D., Hesselberg, E., Rúnarsson, Á. R., Reinius, L. E., Acevedo, N., Taub, M. A., Ronninger, M., et al. (2013) · Nature Biotechnology · doi:10.1038/nbt.2487 · Cited by 974 · see citing works
DNA methylation in newborns and maternal smoking in pregnancy: genome-wide consortium meta-analysis
Joubert, B. R., Felix, J. F., Yousefi, P., Bakulski, K. M., Just, A. C., Breton, C., Reese, S. E., Markunas, C. A., Richmond, R. C., Xu, C.-J., et al. (2016) · The American Journal of Human Genetics · doi:10.1016/j.ajhg.2016.02.019 · Cited by 886 · see citing works
Age-dependent DNA methylation of genes that are suppressed in stem cells is a hallmark of cancer
Teschendorff, A. E., Menon, U., Gentry-Maharaj, A., Ramus, S. J., Weisenberger, D. J., Shen, H., Campan, M., Noushmehr, H., Bell, C. G., Maxwell, A. P., et al. (2010) · Genome Research · doi:10.1101/gr.103606.109 · Cited by 855 · see citing works
Functional normalization of 450k methylation array data improves replication in large cancer studies
Fortin, J.-P., Labbe, A., Lemire, M., Zanke, B. W., Hudson, T. J., Fertig, E. J., Greenwood, C. M. T., & Hansen, K. D. (2014) · Genome biology · doi:10.1186/s13059-014-0503-2 · Cited by 864 · see citing works
DNA methylation patterns associate with genetic and gene expression variation in HapMap cell lines
Bell, J. T., Pai, A. A., Pickrell, J. K., Gaffney, D. J., Pique-Regi, R., Degner, J. F., Gilad, Y., & Pritchard, J. K. (2011) · Genome Biology · doi:10.1186/gb-2011-12-1-r10 · Cited by 838 · see citing works
Ballgown bridges the gap between transcriptome assembly and expression analysis
Frazee, A. C., Pertea, G. M., Jaffe, A. E., Langmead, B., Salzberg, S. L., & Leek, J. T. (2015) · Nature Biotechnology · doi:10.1038/nbt.3172 · Cited by 853 · see citing works
Self-citationGenome-wide changes in lncRNA, splicing, and regional gene expression patterns in autism
Parikshak, N. N., Swarup, V., Belgard, T. G., Irimia, M., Ramaswami, G., Gandal, M. J., Hartl, C., Leppa, V., Ubieta, L. D. L. T., Huang, J., et al. (2016) · Nature · doi:10.1038/nature20612 · Cited by 685 · see citing works
Hirst, J. (2013) · Annual Review of Biochemistry · doi:10.1146/annurev-biochem-070511-103700 · Cited by 754 · see citing works
Revealing the vectors of cellular identity with single-cell genomics
Wagner, A., Regev, A., & Yosef, N. (2016) · Nature Biotechnology · doi:10.1038/nbt.3711 · Cited by 632 · see citing works
A general and flexible method for signal extraction from single-cell RNA-seq data
Risso, D., Perraudeau, F., Gribkova, S., Dudoit, S., & Vert, J.-P. (2018) · Nature Communications · doi:10.1038/s41467-017-02554-5 · Cited by 718 · see citing works
Temporal dynamics and genetic control of transcription in the human prefrontal cortex
Colantuoni, C., Lipska, B. K., Ye, T., Hyde, T. M., Tao, R., Leek, J. T., Colantuoni, E. A., Elkahloun, A. G., Herman, M. M., Weinberger, D. R., et al. (2011) · Nature · doi:10.1038/nature10524 · Cited by 685 · see citing works
Self-citationBump hunting to identify differentially methylated regions in epigenetic epidemiology studies
Jaffe, A. E., Murakami, P. N., Lee, H.-J., Leek, J. T., Fallin, M. D., Feinberg, A. P., & Irizarry, R. A. (2012) · International Journal of Epidemiology · doi:10.1093/ije/dyr238 · Cited by 687 · see citing works
Self-citationAssigning Significance to Peptides Identified by Tandem Mass Spectrometry Using Decoy Databases
Käll, L., Storey, J. D., MacCoss, M. J., & Noble, W. S. (2008) · Journal of Proteome Research · doi:10.1021/pr700600n · Cited by 607 · see citing works
Self-citationBroad defects in the energy metabolism of leukocytes underlie immunoparalysis in sepsis
Cheng, S.-C., Scicluna, B. P., Arts, R. J. W., Gresnigt, M. S., Lachmandas, E., Giamarellos-Bourboulis, E. J., Kox, M., Manjeri, G. R., Wagenaars, J. A. L., Cremer, O. L., et al. (2016) · Nature Immunology · doi:10.1038/ni.3398 · Cited by 637 · see citing works
Accessory subunits are integral for assembly and function of human mitochondrial complex I
Stroud, D. A., Surgenor, E. E., Formosa, L. E., Reljić, B., Frazier, A. E., Dibley, M. G., Osellame, L. D., Stait, T. L., Beilharz, T. H., Thorburn, D. R., et al. (2016) · Nature · doi:10.1038/nature19754 · Cited by 591 · see citing works
Stegle, O., Parts, L., Durbin, R., & Winn, J. (2010) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000770 · Cited by 497 · see citing works
svaseq: removing batch effects and other unwanted noise from sequencing data
Leek, J. T. (2014) · Nucleic Acids Research · doi:10.1093/nar/gku864 · Cited by 538 · see citing works
Self-citationThe rules and impact of nonsense-mediated mRNA decay in human cancers
Lindeboom, R. G. H., Supek, F., & Lehner, B. (2016) · Nature Genetics · doi:10.1038/ng.3664 · Cited by 549 · see citing works
Chen, C., Grennan, K. S., Badner, J. A., Zhang, D., Gershon, E. S., Li, J., & Liu, C. (2011) · PLoS ONE · doi:10.1371/journal.pone.0017238 · Cited by 543 · see citing works
Comprehensive literature review and statistical considerations for microarray meta-analysis
Tseng, G. C., Ghosh, D., & Feingold, E. (2012) · Nucleic Acids Research · doi:10.1093/nar/gkr1265 · Cited by 527 · see citing works
Using control genes to correct for unwanted variation in microarray data
Gagnon-Bartsch, J. A., & Speed, T. P. (2012) · Biostatistics · doi:10.1093/biostatistics/kxr034 · Cited by 442 · see citing works
Mostafavi, S., Gaiteri, C., Sullivan, S. E., White, C. C., Tasaki, S., Xu, J., Taga, M., Klein, H.-U., Patrick, E., Komashko, V., et al. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0154-9 · Cited by 509 · see citing works
High-dimensional semiparametric Gaussian copula graphical models
Liu, H., Han, F., Yuan, M., Lafferty, J., & Wasserman, L. (2012) · The Annals of Statistics · doi:10.1214/12-aos1037 · Cited by 509 · see citing works
Polarization of the Effects of Autoimmune and Neurodegenerative Risk Alleles in Leukocytes
Raj, T., Rothamel, K., Mostafavi, S., Ye, C., Lee, M. N., Replogle, J. M., Feng, T., Lee, M., Asinovski, N., Frohlich, I., et al. (2014) · Science · doi:10.1126/science.1249547 · Cited by 506 · see citing works
The Heterogeneity Problem: Approaches to Identify Psychiatric Subtypes
Feczko, E., Miranda-Dominguez, O., Marr, M., Graham, A. M., Nigg, J. T., & Fair, D. A. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2019.03.009 · Cited by 443 · see citing works
Protecting Human Health From Air Pollution
Dominici, F., Peng, R. D., Barr, C. D., & Bell, M. L. (2010) · Epidemiology · doi:10.1097/ede.0b013e3181cc86e8 · Cited by 487 · see citing works
the BIOS Consortium, van Iterson, M., van Zwet, E. W., & Heijmans, B. T. (2017) · Genome biology · doi:10.1186/s13059-016-1131-9 · Cited by 421 · see citing works
Wagner, J. R., Busche, S., Ge, B., Kwan, T., Pastinen, T., & Blanchette, M. (2014) · Genome biology · doi:10.1186/gb-2014-15-2-r37 · Cited by 446 · see citing works
Global Analyses of Human Immune Variation Reveal Baseline Predictors of Postvaccination Responses
Tsang, J. S., Schwartzberg, P. L., Kotliarov, Y., Biancotto, A., Xie, Z., Germain, R. N., Wang, E., Olnes, M. J., Narayanan, M., Golding, H., et al. (2014) · Cell · doi:10.1016/j.cell.2014.03.031 · Cited by 454 · see citing works
Systems biology and gene networks in neurodevelopmental and neurodegenerative disorders
Parikshak, N. N., Gandal, M. J., & Geschwind, D. H. (2015) · Nature Reviews Genetics · doi:10.1038/nrg3934 · Cited by 430 · see citing works
Common Genetic Variants Modulate Pathogen-Sensing Responses in Human Dendritic Cells
Lee, M. N., Ye, C., Villani, A.-C., Raj, T., Li, W., Eisenhaure, T. M., Imboywa, S. H., Chipendo, P. I., Ran, F. A., Slowikowski, K., et al. (2014) · Science · doi:10.1126/science.1246980 · Cited by 441 · see citing works
Reference-free cell mixture adjustments in analysis of DNA methylation data
Houseman, E. A., Molitor, J., & Marsit, C. J. (2014) · Bioinformatics · doi:10.1093/bioinformatics/btu029 · Cited by 441 · see citing works
Cronk, J. C., Filiano, A. J., Louveau, A., Marin, I., Marsh, R., Ji, E., Goldman, D. H., Smirnov, I., Geraci, N., Acton, S., et al. (2018) · Journal of Experimental Medicine · doi:10.1084/jem.20180247 · Cited by 412 · see citing works
LFMM 2: Fast and Accurate Inference of Gene-Environment Associations in Genome-Wide Studies
Caye, K., Jumentier, B., Lepeule, J., & François, O. (2019) · Molecular Biology and Evolution · doi:10.1093/molbev/msz008 · Cited by 410 · see citing works
DNA methylation contributes to natural human variation
Heyn, H., Moran, S., Hernando-Herraez, I., Sayols, S., Gomez, A., Sandoval, J., Monk, D., Hata, K., Marques-Bonet, T., Wang, L., et al. (2013) · Genome Research · doi:10.1101/gr.154187.112 · Cited by 408 · see citing works
DNA methylation in human diseases
Jin, Z., & Liu, Y. (2018) · Genes & Diseases · doi:10.1016/j.gendis.2018.01.002 · Cited by 405 · see citing works
Why Batch Effects Matter in Omics Data, and How to Avoid Them
Goh, W. W. B., Wang, W., & Wong, L. (2017) · Trends in biotechnology · doi:10.1016/j.tibtech.2017.02.012 · Cited by 397 · see citing works
Genetic Control of Individual Differences in Gene-Specific Methylation in Human Brain
Zhang, D., Cheng, L., Badner, J. A., Chen, C., Chen, Q., Luo, W., Craig, D. W., Redman, M., Gershon, E. S., & Liu, C. (2010) · The American Journal of Human Genetics · doi:10.1016/j.ajhg.2010.02.005 · Cited by 388 · see citing works
Osteoclasts control reactivation of dormant myeloma cells by remodelling the endosteal niche
Lawson, M. A., McDonald, M. M., Kovačić, N., Khoo, W. H., Terry, R. L., Down, J. M., Kaplan, W. H., Paton‐Hough, J., Fellows, C., Pettitt, J. A., et al. (2015) · Nature Communications · doi:10.1038/ncomms9983 · Cited by 385 · see citing works
Extensive Variation in Chromatin States Across Humans
Kasowski, M., Kyriazopoulou-Panagiotopoulou, S., Grubert, F., Zaugg, J. B., Kundaje, A., Liu, Y., Boyle, A. P., Zhang, Q. C., Zakharia, F., Spacek, D. V., et al. (2013) · Science · doi:10.1126/science.1242510 · Cited by 339 · see citing works
A general framework for multiple testing dependence
Leek, J. T., & Storey, J. D. (2008) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0808709105 · Cited by 365 · see citing works
Self-citationDNA methylation signatures in development and aging of the human prefrontal cortex
Numata, S., Ye, T., Hyde, T. M., Guitart-Navarro, X., Tao, R., Wininger, M., Colantuoni, C., Weinberger, D. R., Kleinman, J. E., & Lipska, B. K. (2012) · The American Journal of Human Genetics · doi:10.1016/j.ajhg.2011.12.020 · Cited by 364 · see citing works
Heritability and genomics of gene expression in peripheral blood
Wright, F. A., Sullivan, P. F., Brooks, A. I., Zou, F., Sun, W., Xia, K., Madar, V., Jansen, R., Chung, W., Zhou, Y.-H., et al. (2014) · Nature Genetics · doi:10.1038/ng.2951 · Cited by 362 · see citing works
Pearce, S. L., Clarke, D. F., East, P. D., Elfékih, S., Gordon, K. H. J., Jermiin, L. S., McGaughran, A., Oakeshott, J. G., Papanikolaou, A., Perera, O. P., et al. (2017) · BMC Biology · doi:10.1186/s12915-017-0402-6 · Cited by 356 · see citing works
An Epigenetic Signature in Peripheral Blood Predicts Active Ovarian Cancer
Teschendorff, A. E., Menon, U., Gentry‐Maharaj, A., Ramus, S. J., Gayther, S. A., Apostolidou, S., Jones, A., Lechner, M., Beck, S., Jacobs, I. J., et al. (2009) · PLoS ONE · doi:10.1371/journal.pone.0008274 · Cited by 357 · see citing works
From cohorts to molecules: adverse impacts of endocrine disrupting mixtures
Caporale, N., Leemans, M., Birgersson, L., Germain, P.-L., Cheroni, C., Borbély, G., Engdahl, E., Lindh, C., Bressan, R. B., Cavallo, F., et al. (2022) · Science · doi:10.1126/science.abe8244 · Cited by 368 · see citing works
Normalization and missing value imputation for label-free LC-MS analysis
Karpievitch, Y. V., Dabney, A. R., & Smith, R. D. (2012) · BMC Bioinformatics · doi:10.1186/1471-2105-13-s16-s5 · Cited by 345 · see citing works
Identifying and mitigating bias in next-generation sequencing methods for chromatin biology
Meyer, C. A., & Liu, X. S. (2014) · Nature Reviews Genetics · doi:10.1038/nrg3788 · Cited by 329 · see citing works
Batch effect removal methods for microarray gene expression data integration: a survey
Lazar, C., Meganck, S., Taminau, J., Steenhoff, D., Coletta, A., Molter, C., Weiss-Solis, D. Y., Duque, R., Bersini, H., & Nowe, A. (2013) · Briefings in Bioinformatics · doi:10.1093/bib/bbs037 · Cited by 333 · see citing works
Fehrmann, R. S. N., Jansen, R. C., Veldink, J. H., Westra, H.-J., Arends, D., Bonder, M. J., Fu, J., Deelen, P., Groen, H. J. M., Smolonska, A., et al. (2011) · PLoS Genetics · doi:10.1371/journal.pgen.1002197 · Cited by 330 · see citing works
Davis, L. K., Yu, D., Keenan, C. L., Gamazon, E. R., Konkashbaev, A. I., Derks, E. M., Neale, B. M., Yang, J., Lee, S., Evans, P. D., et al. (2013) · PLoS Genetics · doi:10.1371/journal.pgen.1003864 · Cited by 331 · see citing works
Meffil: efficient normalization and analysis of very large DNA methylation datasets
Min, J. L., Hemani, G., Smith, G. D., Relton, C., & Suderman, M. (2018) · Bioinform. · doi:10.1093/bioinformatics/bty476 · Cited by 301 · see citing works
Identification of differentially methylated cell types in epigenome-wide association studies
Zheng, S., Breeze, C. E., Beck, S., & Teschendorff, A. E. (2018) · Nature Methods · doi:10.1038/s41592-018-0213-x · Cited by 321 · see citing works
Sieberts, S. K., Perumal, T. M., Carrasquillo, M. M., Allen, M., Reddy, J. S., Hoffman, G. E., Dang, K. K., Calley, J., Ebert, P. J., Eddy, J., et al. (2020) · Scientific Data · doi:10.1038/s41597-020-00642-8 · Cited by 318 · see citing works
Dream: powerful differential expression analysis for repeated measures designs
Hoffman, G. E., & Roussos, P. (2021) · Bioinformatics · doi:10.1093/bioinformatics/btaa687 · Cited by 319 · see citing works
Age-associated epigenetic drift: implications, and a case of epigenetic thrift?
Teschendorff, A. E., West, J., & Beck, S. (2013) · Human Molecular Genetics · doi:10.1093/hmg/ddt375 · Cited by 318 · see citing works
Lopez, J. P., Lim, R., Cruceanu, C., Crapper, L., Fasano, C., Labonte, B., Maussion, G., Yang, J. P., Yerko, V., Vigneault, E., et al. (2014) · Nature Medicine · doi:10.1038/nm.3582 · Cited by 308 · see citing works
SomaticSignatures: inferring mutational signatures from single-nucleotide variants
Gehring, J. S., Fischer, B., Lawrence, M. F., & Huber, W. K. (2015) · Bioinformatics · doi:10.1093/bioinformatics/btv408 · Cited by 307 · see citing works
Genome-wide DNA methylation analysis for diabetic nephropathy in type 1 diabetes mellitus
Bell, C. G., Teschendorff, A. E., Rakyan, V. K., Maxwell, A. P., Beck, S., & Savage, D. A. (2010) · BMC Medical Genomics · doi:10.1186/1755-8794-3-33 · Cited by 302 · see citing works
Wan, E. S., Qiu, W., Baccarelli, A., Carey, V. J., Bacherman, H., Rennard, S. I., Agustí, A. G. N., Anderson, W. H., Lomas, D. A., & DeMeo, D. L. (2012) · Human Molecular Genetics · doi:10.1093/hmg/dds135 · Cited by 304 · see citing works
Erion, G., Janizek, J. D., Sturmfels, P., Lundberg, S. M., & Lee, S.-I. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00343-w · Cited by 300 · see citing works
Machine learning identifies candidates for drug repurposing in Alzheimer’s disease
Rodriguez, S., Hug, C., Todorov, P., Moret, N., Boswell, S. A., Evans, K., Zhou, G., Johnson, N. T., Hyman, B. T., Sorger, P. K., et al. (2021) · Nature Communications · doi:10.1038/s41467-021-21330-0 · Cited by 299 · see citing works
Statistical and integrative system-level analysis of DNA methylation data
Teschendorff, A. E., & Relton, C. L. (2017) · Nature Reviews Genetics · doi:10.1038/nrg.2017.86 · Cited by 292 · see citing works
The Public Repository of Xenografts Enables Discovery and Randomized Phase II-like Trials in Mice
Townsend, E. C., Murakami, M. A., Christodoulou, A., Christie, A. L., Köster, J., DeSouza, T. A., Morgan, E. A., Kallgren, S. P., Liu, H., Wu, S.-C., et al. (2016) · Cancer Cell · doi:10.1016/j.ccell.2016.03.008 · Cited by 295 · see citing works
den Boon, J. A., Pyeon, D., WANG, S. S., Horswill, M. A., SCHIFFMAN, M. H., Sherman, M. E., Zuna, R. E., Wang, Z., Hewitt, S. M., Pearson, R., et al. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1509322112 · Cited by 293 · see citing works
Gaiteri, C., Ding, Y., French, B., Tseng, G. C., & Sibille, E. (2014) · Genes, Brain and Behavior · doi:10.1111/gbb.12106 · Cited by 289 · see citing works
SCnorm: robust normalization of single-cell RNA-seq data
Bacher, R., Chu, L.-F., Leng, N., Gasch, A. P., Thomson, J. A., Stewart, R. M., Newton, M., & Kendziorski, C. (2017) · Nature Methods · doi:10.1038/nmeth.4263 · Cited by 282 · see citing works
Phagocytosis imprints heterogeneity in tissue-resident macrophages
A-González, N., Quintana, J. A., Garcı́a-Silva, S., Mazariegos, M. S., de la Aleja, A. G., Nicolás-Ávila, J. Á., Walter, W., Adrover, J. M., Crainiciuc, G., Kuchroo, V. K., et al. (2017) · The Journal of Experimental Medicine · doi:10.1084/jem.20161375 · Cited by 282 · see citing works
Navigating the pitfalls of applying machine learning in genomics
Whalen, S., Schreiber, J. M., Noble, W. S., & Pollard, K. S. (2021) · Nature Reviews Genetics · doi:10.1038/s41576-021-00434-9 · Cited by 278 · see citing works
Genetic Control of Expression and Splicing in Developing Human Brain Informs Disease Mechanisms
Walker, R. L., Ramaswami, G., Hartl, C., Mancuso, N., Gandal, M. J., de la Torre-Ubieta, L., Paşaniuc, B., Stein, J. L., & Geschwind, D. H. (2019) · Cell · doi:10.1016/j.cell.2019.09.021 · Cited by 259 · see citing works
Normalization of circulating microRNA expression data obtained by quantitative real-time RT-PCR
Marabita, F., de Candia, P., Torri, A., Tegner, J. N., Abrignani, S., & Rossi, R. L. (2015) · Briefings in Bioinformatics · doi:10.1093/bib/bbv056 · Cited by 270 · see citing works
Sex-specific and lineage-specific alternative splicing in primatesPreprint
Blekhman, R., Marioni, J. C., Zumbo, P., Stephens, M., & Gilad, Y. (2009) · Genome Research · doi:10.1101/gr.099226.109 · Cited by 266 · see citing works
Intersection of population variation and autoimmunity genetics in human T cell activation
Ye, C. J., Feng, T., Kwon, H.-K., Raj, T., Wilson, M. T., Asinovski, N., McCabe, C., Lee, M. H., Frohlich, I., Paik, H.-I., et al. (2014) · Science · doi:10.1126/science.1254665 · Cited by 236 · see citing works
Li, S., Tighe, S. W., Nicolet, C. M., Grove, D., Levy, S., Farmerie, W., Viale, A., Wright, C., Schweitzer, P. A., Gao, Y., et al. (2014) · Nature Biotechnology · doi:10.1038/nbt.2972 · Cited by 240 · see citing works
Breton, C. V., Marsit, C. J., Faustman, E. M., Nadeau, K. C., Goodrich, J. M., Dolinoy, D. C., Herbstman, J. B., Holland, N., LaSalle, J. M., Schmidt, R. J., et al. (2017) · Environmental Health Perspectives · doi:10.1289/ehp595 · Cited by 261 · see citing works
Extraction and analysis of signatures from the Gene Expression Omnibus by the crowd
Wang, Z., Monteiro, C. D., Jagodnik, K. M., Fernandez, N., Gundersen, G. W., Rouillard, A. D., Jenkins, S. L., Feldmann, A. S., Hu, K. S., McDermott, M. G., et al. (2016) · Nature Communications · doi:10.1038/ncomms12846 · Cited by 254 · see citing works
- Authors
- Leek, J. T., & Storey, J. D.
- Affiliations
- University of Washington (US)
- Corresponding author
- Storey, John D
- Venue
- PLoS Genetics
- Publisher
- Public Library of Science (PLoS)
- Published
- 2007-09-28
- Volume
- 3
- Issue
- 9
- Pages
- e161
- ISSN
- 1553-7390 (ISSN-L)
- 1553-7404
- PMC
- PMC1994707
- OpenAlex
- W2137499573
- Cited by
- 1,930
- References
- 46
- Citations by year
- 2026: 77 · 2025: 114 · 2024: 131 · 2023: 126 · 2022: 155 · 2021: 195 · 2020: 177 · 2019: 156 · 2018: 141 · 2017: 143 · 2016: 112 · 2015: 79 · 2014: 87 · 2013: 74 · 2012: 54
- FWCI
- 8.49
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS Genetics (publishedVersion, cc-by)
- PubMed Central (publishedVersion, other-oa)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Funders
- National Institutes of Health
- National Human Genome Research Institute (R01HG002913)
- National Institute of General Medical Sciences (U54GM062119, U54 GM2119)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene expression and cancer classification
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Bioinformatics and Genomic Networks
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Single-cell and spatial transcriptomics
- Keywords
- surrogate variable analysis, gene expression heterogeneity, unmeasured confounding, time-course expression, reproducibility, statistical power
- MeSH terms
- Algorithms, Breast Neoplasms, Computer Simulation, Data Interpretation, Statistical, Female, Humans, Kidney, Genetic Linkage, Mutation, Saccharomyces cerevisiae, Time Factors, Reproducibility of Results, Gene Expression, Genome, Human, Linear Models, Genome, Fungal, Genetic Heterogeneity, Genes, BRCA1, Quantitative Trait, Heritable, Oligonucleotide Array Sequence Analysis, Genes, BRCA2, Epigenesis, Genetic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
