Tackling the widespread and critical impact of batch effects in high-throughput data
Open accessAbstract
High-throughput technologies are widely used, for example to assay genetic variants, gene and protein expression, and epigenetic modifications. One often overlooked complication with such studies is batch effects, which occur because measurements are affected by laboratory conditions, reagent lots and personnel differences. This becomes a major problem when batch effects are correlated with an outcome of interest and lead to incorrect conclusions. Using both published studies and our own analyses, we argue that batch effects (as well as other technical and biological artefacts) are widespread and critical to address. We review experimental and computational approaches for doing so.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Batch effects make it difficult to draw conclusions from data across experimental batches.
“These batch effects make it difficult to draw conclusions from data across experimental batches (Leek et al., 2010; Parker and Leek, 2012; Soneson et al., 2014; Nygaard et al., 2016; Caicedo et al., 2017).”
From WILDS: A Benchmark of in-the-Wild Distribution Shifts (2020) · cites this paper as Leek et al., 2010
Several methods have been suggested to adjust for batch effects, including recommendations for experimental designs and statistical approaches.
“Several methods have been suggested to adjust for batch effects, including recommendations for experimental designs and statistical approaches (see for example Leek et al., 2010).”
From The applied implications of epigenetics in anxiety, affective and stress-related disorders - A review and synthesis on psychosocial stress, psychotherapy and prevention (2020) · cites this paper as Leek et al., 2010
Measurements from high-throughput screens are confounded by technical artifacts that arise from differences in the execution of each batch.
“Despite attempts to carefully control experimental variables such as temperature, humidity, and reagent concentration, measurements from these screens are confounded by technical artifacts that arise from differences in the execution of each batch.”
From WILDS: A Benchmark of in-the-Wild Distribution Shifts (2020) · cites this paper as Leek et al., 2010
Batch effects and technical variation due to unknown confounders are common in both scRNA-seq and bulk RNA-seq.
“Batch effects and technical variation due to unknown (unmodeled) confounders are common in both scRNA-seq and bulk RNA-seq.”
From Revealing the vectors of cellular identity with single-cell genomics (2016) · cites this paper as [45]
Network edges are susceptible to ascertainment biases and confounding factors that can induce spurious relationships.
“Notably, edges are susceptible to ascertainment biases52,214,215 and confounding factors that can induce spurious relationships178 (FIG. 2b).”
From Systems biology and gene networks in neurodevelopmental and neurodegenerative disorders (2015) · cites this paper as [178]
Optimal methodological choices and study designs ensure that biological signals from main factors are not confounded by unwanted variation.
“Optimal methodological choices and study designs ensure that the biological signals from the main factors are not confounded by variation from unwanted factors178.”
From Systems biology and gene networks in neurodevelopmental and neurodegenerative disorders (2015) · cites this paper as [178]
Unlike transcriptional or genetic meta-analyses, microbiome data lack batch normalization approaches.
“there are no batch normalization approaches for microbiome data, unlike in transcriptional or genetic meta-analyses26.”
From Assessment of variation in microbial community amplicon sequencing by the Microbiome Quality Control (MBQC) project consortium (2017) · cites this paper as [26]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Exploration, normalization, and summaries of high density oligonucleotide array probe level data
Irizarry, R. A. (2003) · Biostatistics · doi:10.1093/biostatistics/4.2.249 · Cited by 10,603 · see citing works
Self-citationAdjusting batch effects in microarray expression data using empirical Bayes methods
Johnson, W. E., Li, C., & Rabinovic, A. (2007) · Biostatistics · doi:10.1093/biostatistics/kxj037 · Cited by 8,658 · see citing works
Self-citationBolstad, B. M., Irizarry, R. A., Åstrand, M., & Speed, T. P. (2003) · Bioinformatics · doi:10.1093/bioinformatics/19.2.185 · Cited by 8,185 · see citing works
Self-citationComprehensive genomic characterization defines human glioblastoma genes and core pathways
The Cancer Genome Atlas Research Network (2008) · Nature · doi:10.1038/nature07385 · Cited by 7,735 · see citing works
The International HapMap Project
†The International HapMap Consortium, Genotyping centres: Baylor College of Medicine and ParAllele BioScience, Gibbs, R. A., Belmont, J. W., Hardenbol, P., Willis, T. D., Yu, F., Chinese HapMap Consortium, Yang, H., Ch'ang, L.-Y., et al. (2003) · Nature · doi:10.1038/nature02168 · Cited by 6,057 · see citing works
Use of proteomic patterns in serum to identify ovarian cancer
Petricoin, E. F., Ardekani, A. M., Hitt, B. A., Levine, P. J., Fusaro, V. A., Steinberg, S. M., Mills, G. B., Simone, C., Fishman, D. A., Kohn, E. C., et al. (2002) · The Lancet · doi:10.1016/s0140-6736(02)07746-2 · Cited by 3,236 · see citing works
Principles of Numerical Taxonomy
Eysenck, H. J., & Eysenck, S. B. G. (2013) · Routledge eBooks · doi:10.4324/9780203766804-11 · Cited by 3,001 · see citing works
Singular value decomposition for genome-wide expression data processing and modeling
Alter, O., Brown, P. O., & Botstein, D. (2000) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.97.18.10101 · Cited by 1,989 · see citing works
Capturing Heterogeneity in Gene Expression Studies by Surrogate Variable Analysis
Leek, J. T., & Storey, J. D. (2007) · PLoS Genetics · doi:10.1371/journal.pgen.0030161 · Cited by 1,930 · see citing works
Self-citationMicroarray data analysis: from disarray to consolidation and consensus
Allison, D. B., Cui, X., Page, G. P., & Sabripour, M. (2006) · Nature Reviews Genetics · doi:10.1038/nrg1749 · Cited by 1,339 · see citing works
Principles of Numerical Taxonomy
Long, C. A., Sokal, R. R., & Sneath, P. H. A. (1965) · Journal of Mammalogy · doi:10.2307/1377831 · Cited by 937 · see citing works
Multiple-laboratory comparison of microarray platforms
Irizarry, R. A., Warren, D., Spencer, F., Kim, I. F., Biswal, S., Frank, B. C., Gabrielson, E., Garcia, J. G. N., Geoghegan, J., Germino, G., et al. (2005) · Nature Methods · doi:10.1038/nmeth756 · Cited by 862 · see citing works
Self-citationCox, M. A. A., & Cox, T. F. (2007) · Springer eBooks · doi:10.1007/978-3-540-33037-0_14 · Cited by 581 · see citing works
Common genetic variants account for differences in gene expression among ethnic groups
Spielman, R. S., Bastone, L. A., Burdick, J. T., Morley, M. P., Ewens, W. J., & Cheung, V. G. (2007) · Nature Genetics · doi:10.1038/ng1955 · Cited by 516 · see citing works
Handbook of Data Visualization
Chen, C., Härdle, W. K., & Unwin, A. R. (2007) · Springer eBooks · doi:10.1007/978-3-540-33037-0 · Cited by 498 · see citing works
Gene Expression in the Urinary Bladder
Dyrskjøt, L., Kruhøffer, M., Thykjær, T., Marcussen, N., Jensen, J. L., Møller, K., & Orntoft, T. F. (2004) · Cancer Research · doi:10.1158/0008-5472.can-03-3620 · Cited by 452 · see citing works
Lessons from Controversy: Ovarian Cancer Screening and Serum Proteomics
Ransohoff, D. F. (2005) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/dji054 · Cited by 265 · see citing works
High-resolution serum proteomic features for ovarian cancer detection.
Conrads, T. P., Fusaro, V. A., Ross, S., Johann, D., Rajapakse, V. N., Hitt, B. A., Steinberg, S. M., Kohn, E. C., Fishman, D. A., Whitely, G., et al. (2004) · Endocrine Related Cancer · doi:10.1677/erc.0.0110163 · Cited by 256 · see citing works
Effects of Atmospheric Ozone on Microarray Data Quality
Fare, T. L., Coffey, E. M., Dai, H. A., He, Y. D., Kessler, D. A., Kilian, K. A., Koch, J. E., LeProust, E. M., Marton, M. J., Meyer, M. R., et al. (2003) · Analytical Chemistry · doi:10.1021/ac034241b · Cited by 229 · see citing works
Dick, D. M., Foroud, T., Wetherill, L. F., Bowman, E. S., Miller, M., Rau, N. L., Moe, P. R., Samavedy, N., El‐Mallakh, R. S., Manji, H. K., et al. (2003) · The American Journal of Human Genetics · doi:10.1086/376562 · Cited by 215 · see citing works
Hu, J. (2005) · Briefings in Functional Genomics and Proteomics · doi:10.1093/bfgp/3.4.322 · Cited by 175 · see citing works
On the design and analysis of gene expression studies in human populations
Akey, J. M., Biswas, S., Leek, J. T., & Storey, J. D. (2007) · Nature Genetics · doi:10.1038/ng0707-807 · Cited by 130 · see citing works
Self-citationSupervised normalization of microarrays
Mecham, B. H., Nelson, P. S., & Storey, J. D. (2010) · Bioinformatics · doi:10.1093/bioinformatics/btq118 · Cited by 130 · see citing works
A gene expression bar code for microarray data
Zilliox, M. J., & Irizarry, R. A. (2007) · Nature Methods · doi:10.1038/nmeth1102 · Cited by 128 · see citing works
Self-citationYouden, W. J. (1972) · Technometrics · doi:10.1080/00401706.1972.10488878 · Cited by 117 · see citing works
Batch Effects and Noise in Microarray Experiments
(2009) · Wiley series in probability and statistics · doi:10.1002/9780470685983 · Cited by 107 · see citing works
Run Batch Effects Potentially Compromise the Usefulness of Genomic Signatures for Ovarian Cancer
Baggerly, K., Coombes, K. R., & Neeley, E. S. (2008) · Journal of Clinical Oncology · doi:10.1200/jco.2007.15.1951 · Cited by 72 · see citing works
Self-citationHigh-resolution serum proteomic patterns for ovarian cancer detection
Baggerly, K., Edmonson, S. R., Morris, J. S., & Coombes, K. R. (2004) · Endocrine Related Cancer · doi:10.1677/erc.1.00868 · Cited by 64 · see citing works
Self-citationA multilevel model to address batch effects in copy number estimation using SNP arrays
Scharpf, R. B., Ruczinski, I., Carvalho, B. S., Doan, B. Q., Chakravarti, A., & Irizarry, R. A. (2010) · Biostatistics · doi:10.1093/biostatistics/kxq043 · Cited by 46 · see citing works
Self-citationCross-generation and cross-laboratory predictions of Affymetrix microarrays by rank-based methods
Liu, H., Chen, C., Liu, Y., Chu, C.-B., Liang, D., Shih, L., & Lin, C. (2007) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2007.11.005 · Cited by 22 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
MultiQC: summarize analysis results for multiple tools and samples in a single report
Ewels, P., Magnusson, M., Lundin, S., & Käller, M. (2016) · Bioinformatics · doi:10.1093/bioinformatics/btw354 · Cited by 10,143 · see citing works
Leek, J. T., Johnson, W. E., Parker, H. S., Jaffe, A. E., & Storey, J. D. (2012) · Bioinformatics · doi:10.1093/bioinformatics/bts034 · Cited by 6,231 · see citing works
Self-citationPredicting effective microRNA target sites in mammalian mRNAs
Agarwal, V., Bell, G. W., Nam, J.-W., & Bartel, D. P. (2015) · eLife · doi:10.7554/elife.05005 · Cited by 7,318 · see citing works
Aryee, M. J., Jaffe, A. E., Bravo, H. C., Ladd‐Acosta, C., Feinberg, A. P., Hansen, K. D., & Irizarry, R. A. (2014) · Bioinformatics · doi:10.1093/bioinformatics/btu049 · Cited by 4,423 · see citing works
Self-citationMetsalu, T., & Vilo, J. (2015) · Nucleic Acids Research · doi:10.1093/nar/gkv468 · Cited by 3,729 · see citing works
Revolutionizing healthcare: the role of artificial intelligence in clinical practice
Alowais, S. A., Alghamdi, S. S., Alsuhebany, N., Alqahtani, T., Alshaya, A. I., Almohareb, S. N., Aldairem, A., Alrashed, M., Bin Saleh, K., Badreldin, H. A., et al. (2023) · BMC Medical Education · doi:10.1186/s12909-023-04698-z · Cited by 3,146 · see citing works
The prognostic landscape of genes and infiltrating immune cells across human cancers
Gentles, A. J., Newman, A. M., Liu, C. L., Bratman, S. V., Feng, W., Kim, D., Nair, V. S., Xu, Y., Khuong, A., Hoang, C. D., et al. (2015) · Nature Medicine · doi:10.1038/nm.3909 · Cited by 3,097 · see citing works
Methods for interpreting and understanding deep neural networks
Montavon, G., Samek, W., & Müller, K.-R. (2018) · Digital Signal Processing · doi:10.1016/j.dsp.2017.10.011 · Cited by 2,665 · see citing works
McCarthy, D. J., Campbell, K. R., Lun, A. T. L., & Wills, Q. F. (2017) · Bioinformatics · doi:10.1093/bioinformatics/btw777 · Cited by 1,852 · see citing works
Reducing the Effects of PCR Amplification and Sequencing Artifacts on 16S rRNA-Based Studies
Schloss, P. D., Gevers, D., & Westcott, S. L. (2011) · PLoS ONE · doi:10.1371/journal.pone.0027310 · Cited by 2,112 · see citing works
ComBat-seq: batch effect adjustment for RNA-seq count data
Zhang, Y., Parmigiani, G., & Johnson, W. E. (2020) · NAR Genomics and Bioinformatics · doi:10.1093/nargab/lqaa078 · Cited by 1,566 · see citing works
Self-citationMicroRNA profiling: approaches and considerations
Pritchard, C. C., Cheng, H. H., & Tewari, M. (2012) · Nature Reviews Genetics · doi:10.1038/nrg3198 · Cited by 1,765 · see citing works
Harmonization of cortical thickness measurements across scanners and sites
Fortin, J.-P., Cullen, N., Sheline, Y. I., Taylor, W. D., Aselcioglu, I., Cook, P. A., Adams, P., Cooper, C., Fava, M., McGrath, P. J., et al. (2018) · NeuroImage · doi:10.1016/j.neuroimage.2017.11.024 · Cited by 1,580 · see citing works
Eleven grand challenges in single-cell data science
Lähnemann, D., Köster, J., Szczurek, E., McCarthy, D. J., Hicks, S. C., Robinson, M. D., Vallejos, C. A., Campbell, K. R., Beerenwinkel, N., Mahfouz, A., et al. (2020) · Genome Biology · doi:10.1186/s13059-020-1926-6 · Cited by 1,327 · see citing works
Robust decomposition of cell type mixtures in spatial transcriptomics
Cable, D. M., Murray, E., Zou, L. S., Goeva, A., Macosko, E. Z., Chen, F., & Irizarry, R. A. (2022) · Nature Biotechnology · doi:10.1038/s41587-021-00830-w · Cited by 1,487 · see citing works
Self-citationCount-based differential expression analysis of RNA sequencing data using R and Bioconductor
Anders, S., McCarthy, D. J., Chen, Y., Okoniewski, M. J., Smyth, G. K., Huber, W. K., & Robinson, M. D. (2013) · Nature Protocols · doi:10.1038/nprot.2013.099 · Cited by 1,129 · see citing works
Integrative functional genomic analyses implicate specific molecular pathways and circuits in autism
Parikshak, N. N., Luo, R., Zhang, A., Won, H., Lowe, J. K., Chandran, V., Horvath, S., & Geschwind, D. H. (2013) · Cell · doi:10.1016/j.cell.2013.10.031 · Cited by 1,039 · see citing works
Increased methylation variation in epigenetic domains across cancer types
Hansen, K. D., Timp, W., Bravo, H. C., Sabunciyan, S., Langmead, B., McDonald, O. G., Wen, B., Wu, H., Liu, Y., Diep, D. H., et al. (2011) · Nature Genetics · doi:10.1038/ng.865 · Cited by 1,053 · see citing works
Self-citationThe role of regulatory variation in complex traits and disease
Albert, F. W., & Kruglyak, L. (2015) · Nature Reviews Genetics · doi:10.1038/nrg3891 · Cited by 1,047 · see citing works
Liu, Y., Aryee, M. J., Padyukov, L., Fallin, M. D., Hesselberg, E., Rúnarsson, Á. R., Reinius, L. E., Acevedo, N., Taub, M. A., Ronninger, M., et al. (2013) · Nature Biotechnology · doi:10.1038/nbt.2487 · Cited by 974 · see citing works
Deep learning in cancer diagnosis, prognosis and treatment selection
Tran, K. A., Kondrashova, O., Bradley, A., Williams, E. D., Pearson, J. V., & Waddell, N. (2021) · Genome Medicine · doi:10.1186/s13073-021-00968-x · Cited by 932 · see citing works
Orchestrating single-cell analysis with Bioconductor
Amezquita, R. A., Lun, A. T. L., Becht, E., Carey, V. J., Carpp, L. N., Geistlinger, L., Marini, F., Rue-Albrecht, K., Risso, D., Soneson, C., et al. (2020) · Nature Methods · doi:10.1038/s41592-019-0654-x · Cited by 945 · see citing works
MRIQC: Advancing the automatic prediction of image quality in MRI from unseen sites
Esteban, O., Birman, D., Schaer, M., Koyejo, O. O., Poldrack, R. A., & Gorgolewski, K. J. (2017) · PLOS ONE · doi:10.1371/journal.pone.0184661 · Cited by 920 · see citing works
Single-Cell RNA-Seq Technologies and Related Computational Data Analysis
Chen, G., Ning, B., & Shi, T. (2019) · Frontiers in Genetics · doi:10.3389/fgene.2019.00317 · Cited by 913 · see citing works
Functional normalization of 450k methylation array data improves replication in large cancer studies
Fortin, J.-P., Labbe, A., Lemire, M., Zanke, B. W., Hudson, T. J., Fertig, E. J., Greenwood, C. M. T., & Hansen, K. D. (2014) · Genome biology · doi:10.1186/s13059-014-0503-2 · Cited by 893 · see citing works
Data-analysis strategies for image-based cell profiling
Caicedo, J. C., Cooper, S., Heigwer, F., Warchal, S., Qiu, P., Molnar, C., Vasilevich, A. S., Barry, J. D., Bansal, H. S., Kraus, O., et al. (2017) · Nature Methods · doi:10.1038/nmeth.4397 · Cited by 739 · see citing works
A Single-Cell Atlas of the Tumor and Immune Ecosystem of Human Breast Cancer
Wagner, J., Rapsomaniki, M., Chevrier, S., Anzeneder, T., Langwieder, C., Dykgers, A., Rees, M., Ramaswamy, A., Muenst, S. E., Soysal, S. D., et al. (2019) · Cell · doi:10.1016/j.cell.2019.03.005 · Cited by 784 · see citing works
Metagenome-wide analysis of antibiotic resistance genes in a large cohort of human gut microbiota
Hu, Y., Yang, X., Qin, J., Lü, N., Cheng, G., Wu, N., Pan, Y., Li, J., Zhu, L., Wang, X., et al. (2013) · Nature Communications · doi:10.1038/ncomms3151 · Cited by 786 · see citing works
Revealing the vectors of cellular identity with single-cell genomics
Wagner, A., Regev, A., & Yosef, N. (2016) · Nature Biotechnology · doi:10.1038/nbt.3711 · Cited by 632 · see citing works
Bump hunting to identify differentially methylated regions in epigenetic epidemiology studies
Jaffe, A. E., Murakami, P. N., Lee, H.-J., Leek, J. T., Fallin, M. D., Feinberg, A. P., & Irizarry, R. A. (2012) · International Journal of Epidemiology · doi:10.1093/ije/dyr238 · Cited by 687 · see citing works
Self-citationLibrary construction for next-generation sequencing: Overviews and challenges
Head, S. R., Komori, H. K., LaMere, S. A., Whisenant, T. C., Nieuwerburgh, F. V., Salomon, D. R., & Ordoukhanian, P. T. (2014) · BioTechniques · doi:10.2144/000114133 · Cited by 673 · see citing works
Erho, N., Crisan, A., Vergara, I. A., Mitra, A. P., Ghadessi, M., Buerki, C., Bergstralh, E. J., Kollmeyer, T. M., Fink, S. R., Haddad, Z., et al. (2013) · PLoS ONE · doi:10.1371/journal.pone.0066855 · Cited by 646 · see citing works
Oligotyping: differentiating between closely related microbial taxa using 16S rRNA gene data
Eren, A. M., Maignien, L., Sul, W. J., Murphy, L. G., Grim, S. L., Morrison, H. G., & Sogin, M. L. (2013) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.12114 · Cited by 620 · see citing works
The Microbiome Quality Control Project Consortium, Sinha, R., Abu-Ali, G., Vogtmann, E., Fodor, A. A., Ren, B., Amir, A., Schwager, E., Crabtree, J., Ma, S., et al. (2017) · Nature Biotechnology · doi:10.1038/nbt.3981 · Cited by 587 · see citing works
Evaluation of the Infinium Methylation 450K technology
Dedeurwaerder, S., Defrance, M., Calonne, E., Denis, H., Sotiriou, C., & Fuks, F. (2011) · Epigenomics · doi:10.2217/epi.11.105 · Cited by 569 · see citing works
Comprehensive analysis of DNA methylation data with RnBeads
Assenov, Y., Müller, F., Lutsik, P., Walter, J., Lengauer, T., & Bock, C. (2014) · Nature Methods · doi:10.1038/nmeth.3115 · Cited by 598 · see citing works
Missing data and technical variability in single-cell RNA-sequencing experiments
Hicks, S. C., Townes, F. W., Teng, M., & Irizarry, R. A. (2018) · Biostatistics · doi:10.1093/biostatistics/kxx053 · Cited by 635 · see citing works
Self-citationAnalysing and interpreting DNA methylation data
Bock, C. (2012) · Nature Reviews Genetics · doi:10.1038/nrg3273 · Cited by 580 · see citing works
svaseq: removing batch effects and other unwanted noise from sequencing data
Leek, J. T. (2014) · Nucleic Acids Research · doi:10.1093/nar/gku864 · Cited by 538 · see citing works
Self-citationA Review of Domain Adaptation without Target Labels
Kouw, W. M., & Loog, M. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2945942 · Cited by 613 · see citing works
SeSAMe: reducing artifactual detection of DNA methylation by Infinium BeadChips in genomic deletions
Zhou, W., Triche, T. J., Laird, P. W., & Shen, H. (2018) · Nucleic Acids Research · doi:10.1093/nar/gky691 · Cited by 536 · see citing works
Big Data Analytics in Healthcare
Belle, A., Thiagarajan, R., Soroushmehr, S. M. R., Navidi, F., Beard, D. A., & Najarian, K. (2015) · BioMed Research International · doi:10.1155/2015/370194 · Cited by 527 · see citing works
Genome-scale analysis of DNA methylation in lung adenocarcinoma and integration with mRNA expressionPreprint
Selamat, S. A., Chung, B. S. I., Girard, L., Zhang, W., Zhang, Y., Campan, M., Siegmund, K. D., Koss, M. N., Hagen, J. A., Lam, W. L., et al. (2012) · Genome Research · doi:10.1101/gr.132662.111 · Cited by 524 · see citing works
Using control genes to correct for unwanted variation in microarray data
Gagnon-Bartsch, J. A., & Speed, T. P. (2012) · Biostatistics · doi:10.1093/biostatistics/kxr034 · Cited by 442 · see citing works
the BIOS Consortium, van Iterson, M., van Zwet, E. W., & Heijmans, B. T. (2017) · Genome biology · doi:10.1186/s13059-016-1131-9 · Cited by 421 · see citing works
Systems biology and gene networks in neurodevelopmental and neurodegenerative disorders
Parikshak, N. N., Gandal, M. J., & Geschwind, D. H. (2015) · Nature Reviews Genetics · doi:10.1038/nrg3934 · Cited by 430 · see citing works
CMScaller: an R package for consensus molecular subtyping of colorectal cancer pre-clinical models
Eide, P. W., Bruun, J., Lothe, R. A., & Sveen, A. (2017) · Scientific Reports · doi:10.1038/s41598-017-16747-x · Cited by 439 · see citing works
Salgado, H., Peralta-Gil, M., Gama‐Castro, S., Santos-Zavaleta, A., Muñiz-Rascado, L., García-Sotelo, J. S., Weiss, V., Solano-Lira, H., Martínez‐Flores, I., Medina-Rivera, A., et al. (2012) · Nucleic Acids Research · doi:10.1093/nar/gks1201 · Cited by 425 · see citing works
Single-cell RNA sequencing: Technical advancements and biological applications
Hedlund, E., & Deng, Q. (2017) · Molecular Aspects of Medicine · doi:10.1016/j.mam.2017.07.003 · Cited by 396 · see citing works
Why Batch Effects Matter in Omics Data, and How to Avoid Them
Goh, W. W. B., Wang, W., & Wong, L. (2017) · Trends in biotechnology · doi:10.1016/j.tibtech.2017.02.012 · Cited by 397 · see citing works
Pinto, D., Darvishi, K., Shi, X. M., Rajan, D., Rigler, D., Fitzgerald, T. W., Lionel, A. C., Thiruvahindrapuram, B., MacDonald, J. R., Mills, R. E., et al. (2011) · Nature Biotechnology · doi:10.1038/nbt.1852 · Cited by 403 · see citing works
Sagawa, S., Koh, P. W., Hashimoto, T., & Liang, P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.08731 · Cited by 563 · see citing works
Dimension reduction techniques for the integrative analysis of multi-omics data
Meng, C., Zeleznik, O. A., Thallinger, G., Küster, B., Gholami, A. M., & Culhane, A. C. (2016) · Briefings in Bioinformatics · doi:10.1093/bib/bbv108 · Cited by 383 · see citing works
Nygaard, V., Rødland, E. A., & Hovig, E. J. (2015) · Biostatistics · doi:10.1093/biostatistics/kxv027 · Cited by 379 · see citing works
Recommendations for the design and analysis of epigenome-wide association studies
Michels, K. B., Binder, A. M., Dedeurwaerder, S., Epstein, C. B., Greally, J. M., Gut, I., Houseman, E. A., Izzi, B., Kelsey, K. T., Meissner, A., et al. (2013) · Nature Methods · doi:10.1038/nmeth.2632 · Cited by 378 · see citing works
Self-citationWGCNA Application to Proteomic and Metabolomic Data Analysis
Pei, G., Chen, L., & Zhang, W. (2016) · Methods in enzymology on CD-ROM/Methods in enzymology · doi:10.1016/bs.mie.2016.09.016 · Cited by 368 · see citing works
Quantitative analysis of fitness and genetic interactions in yeast on a genome scale
Baryshnikova, A., Costanzo, M., Kim, Y., Ding, H., Koh, J., Toufighi, K., Youn, J.-Y., Ou, J., San Luis, B.-J., Bandyopadhyay, S., et al. (2010) · Nature Methods · doi:10.1038/nmeth.1534 · Cited by 364 · see citing works
Heritability and genomics of gene expression in peripheral blood
Wright, F. A., Sullivan, P. F., Brooks, A. I., Zou, F., Sun, W., Xia, K., Madar, V., Jansen, R., Chung, W., Zhou, Y.-H., et al. (2014) · Nature Genetics · doi:10.1038/ng.2951 · Cited by 362 · see citing works
Evolution of Translational Omics: Lessons Learned and the Path Forward
Micheel, C., Nass, S. J., & Omenn, G. S. (2013) · Cited by 365 · see citing works
Differential immune system DNA methylation and cytokine regulation in post-traumatic stress disorder
Smith, A. K., Conneely, K. N., Kilaru, V., Mercer, K. B., Weiss, T. E., Bradley, B., Tang, Y., Gillespie, C. F., Cubells, J. F., & Ressler, K. J. (2011) · American Journal of Medical Genetics Part B: Neuropsychiatric Genetics · doi:10.1002/ajmg.b.31212 · Cited by 355 · see citing works
From promises to practical strategies in epigenetic epidemiology
Mill, J. S., & Heijmans, B. T. (2013) · Nature Reviews Genetics · doi:10.1038/nrg3405 · Cited by 347 · see citing works
Li, B., Cui, Y., Diehn, M., & Li, R. (2017) · JAMA Oncology · doi:10.1001/jamaoncol.2017.1609 · Cited by 514 · see citing works
Computational solutions for omics data
Berger, B., Peng, J., & Singh, M. (2013) · Nature Reviews Genetics · doi:10.1038/nrg3433 · Cited by 341 · see citing works
He, X., Sanders, S. J., Liu, L., De Rubeis, S., Lim, E. T., Sutcliffe, J. S., Schellenberg, G. D., Gibbs, R. A., Daly, M. J., Buxbaum, J. D., et al. (2013) · PLoS Genetics · doi:10.1371/journal.pgen.1003671 · Cited by 304 · see citing works
Identifying and mitigating bias in next-generation sequencing methods for chromatin biology
Meyer, C. A., & Liu, X. S. (2014) · Nature Reviews Genetics · doi:10.1038/nrg3788 · Cited by 329 · see citing works
Big Data in Public Health: Terminology, Machine Learning, and Privacy
Mooney, S. J., & Pejaver, V. (2018) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-040617-014208 · Cited by 325 · see citing works
Inflammatory Bowel Disease as a Model for Translating the Microbiome
Huttenhower, C., Kostic, A. D., & Xavier, R. J. (2014) · Immunity · doi:10.1016/j.immuni.2014.05.013 · Cited by 332 · see citing works
Reconstructing A/B compartments as revealed by Hi-C using long-range correlations in epigenetic data
Fortin, J.-P., & Hansen, K. D. (2015) · Genome Biology · doi:10.1186/s13059-015-0741-y · Cited by 333 · see citing works
Batch effect removal methods for microarray gene expression data integration: a survey
Lazar, C., Meganck, S., Taminau, J., Steenhoff, D., Coletta, A., Molter, C., Weiss-Solis, D. Y., Duque, R., Bersini, H., & Nowe, A. (2013) · Briefings in Bioinformatics · doi:10.1093/bib/bbs037 · Cited by 333 · see citing works
Transcriptomic and metabolomic data integration
Cavill, R., Jennen, D. G. J., Kleinjans, J. C. S., & Briedé, J. J. (2015) · Briefings in Bioinformatics · doi:10.1093/bib/bbv090 · Cited by 331 · see citing works
Cloud computing for genomic data analysis and collaboration
Langmead, B., & Nellore, A. (2018) · Nature Reviews Genetics · doi:10.1038/nrg.2017.113 · Cited by 310 · see citing works
“That is, researchers can use public data to boost the power available to analyse a locally generated data set, a paradigm that already prevails in micro-array data analysis 8,9,12,92 .”
BackgroundSelf-citationMeffil: efficient normalization and analysis of very large DNA methylation datasets
Min, J. L., Hemani, G., Smith, G. D., Relton, C., & Suderman, M. (2018) · Bioinform. · doi:10.1093/bioinformatics/bty476 · Cited by 301 · see citing works
Age-associated epigenetic drift: implications, and a case of epigenetic thrift?
Teschendorff, A. E., West, J., & Beck, S. (2013) · Human Molecular Genetics · doi:10.1093/hmg/ddt375 · Cited by 318 · see citing works
–Omic and Electronic Health Record Big Data Analytics for Precision Medicine
Wu, P.-Y., Cheng, C., Kaddi, C. D., Venugopalan, J., Hoffman, R., & Wang, M. D. (2016) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2016.2573285 · Cited by 308 · see citing works
Associating cellular epigenetic models with human phenotypes
Lappalainen, T., & Greally, J. M. (2017) · Nature Reviews Genetics · doi:10.1038/nrg.2017.32 · Cited by 282 · see citing works
The role of replicates for error mitigation in next-generation sequencing
Robasky, K., Lewis, N. E., & Church, G. M. (2013) · Nature Reviews Genetics · doi:10.1038/nrg3655 · Cited by 299 · see citing works
Common Disease Is More Complex Than Implied by the Core Gene Omnigenic Model
Wray, N. R., Wijmenga, C., Sullivan, P. F., Yang, J., & Visscher, P. M. (2018) · Cell · doi:10.1016/j.cell.2018.05.051 · Cited by 291 · see citing works
Use and abuse of correlation analyses in microbial ecology
Carr, A. V., Diener, C., Baliga, N. S., & Gibbons, S. M. (2019) · The ISME Journal · doi:10.1038/s41396-019-0459-z · Cited by 295 · see citing works
Resolving host–pathogen interactions by dual RNA-seq
Westermann, A. J., Barquist, L., & Vogel, J. (2017) · PLoS Pathogens · doi:10.1371/journal.ppat.1006033 · Cited by 286 · see citing works
QoRTs: a comprehensive toolset for quality control and data processing of RNA-Seq experiments
Hartley, S. W., & Mullikin, J. C. (2015) · BMC Bioinformatics · doi:10.1186/s12859-015-0670-5 · Cited by 259 · see citing works
Schloss, P. D. (2018) · mBio · doi:10.1128/mbio.00525-18 · Cited by 275 · see citing works
WILDS: A Benchmark of in-the-Wild Distribution Shifts
Koh, P. W., Sagawa, S., Marklund, H., Xie, S. M., Zhang, M., Balsubramani, A., Hu, W., Yasunaga, M., Phillips, R. L., Gao, I., et al. (2020) · arXiv · doi:10.48550/arxiv.2012.07421 · Cited by 518 · see citing works
“These batch effects make it difficult to draw conclusions from data across experimental batches (Leek et al., 2010; Parker & Leek, 2012; Soneson et al., 2014; Nygaard et al., 2016; Caicedo et al., 2017).”
Navigating the pitfalls of applying machine learning in genomics
Whalen, S., Schreiber, J. M., Noble, W. S., & Pollard, K. S. (2021) · Nature Reviews Genetics · doi:10.1038/s41576-021-00434-9 · Cited by 278 · see citing works
Kyttälä, A., Moraghebi, R., Valensisi, C., Kettunen, J., Andrus, C., Pasumarthy, K. K., Nakanishi, M., Nishimura, K., Ohtaka, M., Weltner, J., et al. (2016) · Stem Cell Reports · doi:10.1016/j.stemcr.2015.12.009 · Cited by 257 · see citing works
Normalization of circulating microRNA expression data obtained by quantitative real-time RT-PCR
Marabita, F., de Candia, P., Torri, A., Tegner, J. N., Abrignani, S., & Rossi, R. L. (2015) · Briefings in Bioinformatics · doi:10.1093/bib/bbv056 · Cited by 270 · see citing works
DrugComb: an integrative cancer drug combination data portal
Zagidullin, B., Aldahdooh, J., Zheng, S., Wang, W., Wang, Y., Saad, J. J., Malyutina, A., Jafari, M., Tanoli, Z., Pessia, A., et al. (2019) · Nucleic Acids Research · doi:10.1093/nar/gkz337 · Cited by 265 · see citing works
Biomarker Discovery and Validation: Statistical Considerations
Ou, F., Michiels, S., Shyr, Y., Adjei, A. A., & Oberg, A. L. (2021) · Journal of Thoracic Oncology · doi:10.1016/j.jtho.2021.01.1616 · Cited by 265 · see citing works
A Statistical Framework for Joint eQTL Analysis in Multiple Tissues
Flutre, T., Wen, X., Pritchard, J. K., & Stephens, M. (2013) · PLoS Genetics · doi:10.1371/journal.pgen.1003486 · Cited by 252 · see citing works
recount3: summaries and queries for large-scale RNA-seq expression and splicing
Wilks, C., Zheng, S., Chen, F. Y., Charles, R., Solomon, B., Ling, J. P., Imada, E. L., Zhang, D., Joseph, L., Leek, J. T., et al. (2021) · Genome biology · doi:10.1186/s13059-021-02533-6 · Cited by 264 · see citing works
Self-citationGünther, T., & Nettelblad, C. (2019) · PLOS Genetics · doi:10.1371/journal.pgen.1008302 · Cited by 228 · see citing works
Big data in basic and translational cancer research
Jiang, P., Sinha, S., Aldape, K., Hannenhalli, S., Sahinalp, C., & Ruppin, E. (2022) · Nature Reviews Cancer · doi:10.1038/s41568-022-00502-0 · Cited by 254 · see citing works
Obayashi, T., Aoki, Y., Tadaka, S., Kagaya, Y., & Kinoshita, K. (2017) · Plant and Cell Physiology · doi:10.1093/pcp/pcx191 · Cited by 252 · see citing works
Artificial intelligence for precision medicine in neurodevelopmental disorders
Uddin, M. J., Wang, Y., & Woodbury‐Smith, M. (2019) · npj Digital Medicine · doi:10.1038/s41746-019-0191-0 · Cited by 257 · see citing works
Heath, A. P., Ferretti, V., Agrawal, S., An, M., Angelakos, J. C., Arya, R., Bajari, R., Baqar, B., Barnowski, J. H. B., Burt, J., et al. (2021) · Nature Genetics · doi:10.1038/s41588-021-00791-5 · Cited by 248 · see citing works
Hong, X., Hao, K., Ladd‐Acosta, C., Hansen, K. D., Tsai, H., Liu, X., Xu, X. P., Thornton, T. A., Caruso, D., Keet, C. A., et al. (2015) · Nature Communications · doi:10.1038/ncomms7304 · Cited by 247 · see citing works
Criteria for the use of omics-based predictors in clinical trials
McShane, L. M., Cavenagh, M. M., Lively, T., Eberhard, D. A., Bigbee, W. L., Williams, P. M., Mesirov, J. P., Polley, M. C., Kim, K. Y., Tricoli, J. V., et al. (2013) · Nature · doi:10.1038/nature12564 · Cited by 245 · see citing works
A single-sample microarray normalization method to facilitate personalized-medicine workflows
Piccolo, S., Sun, Y., Campbell, J. D., Lenburg, M. E., Bild, A. H., & Johnson, W. E. (2012) · Genomics · doi:10.1016/j.ygeno.2012.08.003 · Cited by 244 · see citing works
Self-citationGut microbiome heritability is nearly universal but environmentally contingent
Grieneisen, L., Dasari, M., Gould, T. J., Björk, J. R., Grenier, J.-C., Yotova, V., Jansen, D., Gottel, N., Gordon, J. B., Learn, N. H., et al. (2021) · Science · doi:10.1126/science.aba5483 · Cited by 243 · see citing works
A novel multi-tissue RNA diagnostic of healthy ageing relates to cognitive health status
Sood, S., Gallagher, I. J., Lunnon, K., Rullman, E., Keohane, A., Crossland, H., Phillips, B. E., Cederholm, T., Jensen, T., van Loon, L. J. C., et al. (2015) · Genome biology · doi:10.1186/s13059-015-0750-x · Cited by 240 · see citing works
Rapid neurogenesis through transcriptional activation in human stem cells
Busskamp, V., Lewis, N. E., Guye, P., Ng, A. H. M., Shipman, S. L., Byrne, S. M., Sanjana, N. E., Murn, J., Li, Y., Li, S., et al. (2014) · Molecular Systems Biology · doi:10.15252/msb.20145508 · Cited by 222 · see citing works
- Authors
- Leek, J. T., Scharpf, R. B., Bravo, H. C., Simcha, D., Langmead, B., Johnson, W. E., Geman, D., Baggerly, K., & Irizarry, R. A.
- Affiliations
- Johns Hopkins University (US)
- Brigham Young University (US)
- The University of Texas MD Anderson Cancer Center (US)
- Corresponding author
- Irizarry, Rafael A.
- Venue
- Nature Reviews Genetics
- Published
- 2010-09-14
- Volume
- 11
- Issue
- 10
- Pages
- 733-739
- ISSN
- 1471-0056 (ISSN-L)
- 1471-0064
- PMC
- PMC3880143
- OpenAlex
- W2096192437
- Cited by
- 2,196
- References
- 30
- Citations by year
- 2026: 235 · 2025: 166 · 2024: 159 · 2023: 125 · 2022: 178 · 2021: 178 · 2020: 167 · 2019: 138 · 2018: 152 · 2017: 130 · 2016: 133 · 2015: 107 · 2014: 100 · 2013: 95 · 2012: 70
- FWCI
- 6.29
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Nature Reviews Genetics (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Funders
- Juvenile Diabetes Research Foundation International
- Wellcome Trust
- National Institute for Health and Care Research
- National Institutes of Health (5u41hg004059-15)
- Cambridge Institute for Medical Research, University of Cambridge (079895)
- NIHR Cambridge Biomedical Research Centre
- National Human Genome Research Institute (HG005220, U41-HG004059, HG004059, R00HG005015, R01HG005220, P41HG004059, U24HG004059)
- National Institute of General Medical Sciences (GM0083084, R01GM083084)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene expression and cancer classification
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Bioinformatics and Genomic Networks
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Single-cell and spatial transcriptomics
- Keywords
- batch effects, high-throughput omics data, confounding, gene expression, protein expression, epigenetic modifications, laboratory conditions
- MeSH terms
- Biotechnology, Periodicals as Topic, Research Design, Sequence Analysis, DNA, Computational Biology, Oligonucleotide Array Sequence Analysis, Genomics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
