The DOI matches its Crossref record; 30 of the 30 references it cites were found in a registry or an index.
Tackling the widespread and critical impact of batch effects in high-throughput data
Open accessAbstract
High-throughput technologies are widely used, for example to assay genetic variants, gene and protein expression, and epigenetic modifications. One often overlooked complication with such studies is batch effects, which occur because measurements are affected by laboratory conditions, reagent lots and personnel differences. This becomes a major problem when batch effects are correlated with an outcome of interest and lead to incorrect conclusions. Using both published studies and our own analyses, we argue that batch effects (as well as other technical and biological artefacts) are widespread and critical to address. We review experimental and computational approaches for doing so.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“These batch effects make it difficult to draw conclusions from data across experimental batches (Leek et al., 2010; Parker and Leek, 2012; Soneson et al., 2014; Nygaard et al., 2016; Caicedo et al., 2017).”
From WILDS: A Benchmark of in-the-Wild Distribution Shifts (2020) · cites this paper as Leek et al., 2010
The cited source says what the claim attributes to it.
In the citing paper
“Several methods have been suggested to adjust for batch effects, including recommendations for experimental designs and statistical approaches (see for example Leek et al., 2010).”
From The applied implications of epigenetics in anxiety, affective and stress-related disorders - A review and synthesis on psychosocial stress, psychotherapy and prevention (2020) · cites this paper as Leek et al., 2010
The cited source says what the claim attributes to it.
In the citing paper
“Despite attempts to carefully control experimental variables such as temperature, humidity, and reagent concentration, measurements from these screens are confounded by technical artifacts that arise from differences in the execution of each batch.”
From WILDS: A Benchmark of in-the-Wild Distribution Shifts (2020) · cites this paper as Leek et al., 2010
In the citing paper
“Batch effects and technical variation due to unknown (unmodeled) confounders are common in both scRNA-seq and bulk RNA-seq.”
From Revealing the vectors of cellular identity with single-cell genomics (2016) · cites this paper as [45]
In the citing paper
“Notably, edges are susceptible to ascertainment biases52,214,215 and confounding factors that can induce spurious relationships178 (FIG. 2b).”
From Systems biology and gene networks in neurodevelopmental and neurodegenerative disorders (2015) · cites this paper as [178]
In the citing paper
“Optimal methodological choices and study designs ensure that the biological signals from the main factors are not confounded by variation from unwanted factors178.”
From Systems biology and gene networks in neurodevelopmental and neurodegenerative disorders (2015) · cites this paper as [178]
In the citing paper
“there are no batch normalization approaches for microbiome data, unlike in transcriptional or genetic meta-analyses26.”
From Assessment of variation in microbial community amplicon sequencing by the Microbiome Quality Control (MBQC) project consortium (2017) · cites this paper as [26]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Exploration, normalization, and summaries of high density oligonucleotide array probe level data
Irizarry, R. A. (2003) · Biostatistics · doi:10.1093/biostatistics/4.2.249 · Cited by 10,603
Self-citationAdjusting batch effects in microarray expression data using empirical Bayes methods
Johnson, W. E., Li, C., & Rabinovic, A. (2007) · Biostatistics · doi:10.1093/biostatistics/kxj037 · Cited by 8,658
Self-citationBolstad, B. M., Irizarry, R. A., Åstrand, M., & Speed, T. P. (2003) · Bioinformatics · doi:10.1093/bioinformatics/19.2.185 · Cited by 8,328
Self-citationComprehensive genomic characterization defines human glioblastoma genes and core pathways
The Cancer Genome Atlas Research Network (2008) · Nature · doi:10.1038/nature07385 · Cited by 7,735
The International HapMap Project
†The International HapMap Consortium, Genotyping centres: Baylor College of Medicine and ParAllele BioScience, Gibbs, R. A., Belmont, J. W., Hardenbol, P., Willis, T. D., Yu, F., Chinese HapMap Consortium, Yang, H., Ch'ang, L.-Y., et al. (2003) · Nature · doi:10.1038/nature02168 · Cited by 6,057
Use of proteomic patterns in serum to identify ovarian cancer
Petricoin, E. F., Ardekani, A. M., Hitt, B. A., Levine, P. J., Fusaro, V. A., Steinberg, S. M., Mills, G. B., Simone, C., Fishman, D. A., Kohn, E. C., et al. (2002) · The Lancet · doi:10.1016/s0140-6736(02)07746-2 · Cited by 3,236
Principles of Numerical Taxonomy
Eysenck, H. J., & Eysenck, S. B. G. (2013) · Routledge eBooks · doi:10.4324/9780203766804-11 · Cited by 3,001
Singular value decomposition for genome-wide expression data processing and modeling
Alter, O., Brown, P. O., & Botstein, D. (2000) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.97.18.10101 · Cited by 1,989
Capturing Heterogeneity in Gene Expression Studies by Surrogate Variable Analysis
Leek, J. T., & Storey, J. D. (2007) · PLoS Genetics · doi:10.1371/journal.pgen.0030161 · Cited by 1,930
Self-citationMicroarray data analysis: from disarray to consolidation and consensus
Allison, D. B., Cui, X., Page, G. P., & Sabripour, M. (2006) · Nature Reviews Genetics · doi:10.1038/nrg1749 · Cited by 1,339
Principles of Numerical Taxonomy
Long, C. A., Sokal, R. R., & Sneath, P. H. A. (1965) · Journal of Mammalogy · doi:10.2307/1377831 · Cited by 937
Multiple-laboratory comparison of microarray platforms
Irizarry, R. A., Warren, D., Spencer, F., Kim, I. F., Biswal, S., Frank, B. C., Gabrielson, E., Garcia, J. G. N., Geoghegan, J., Germino, G., et al. (2005) · Nature Methods · doi:10.1038/nmeth756 · Cited by 862
Self-citationCox, M. A. A., & Cox, T. F. (2007) · Springer eBooks · doi:10.1007/978-3-540-33037-0_14 · Cited by 616
Common genetic variants account for differences in gene expression among ethnic groups
Spielman, R. S., Bastone, L. A., Burdick, J. T., Morley, M. P., Ewens, W. J., & Cheung, V. G. (2007) · Nature Genetics · doi:10.1038/ng1955 · Cited by 518
Handbook of Data Visualization
Chen, C., Härdle, W. K., & Unwin, A. R. (2007) · Springer eBooks · doi:10.1007/978-3-540-33037-0 · Cited by 498
Gene Expression in the Urinary Bladder
Dyrskjøt, L., Kruhøffer, M., Thykjær, T., Marcussen, N., Jensen, J. L., Møller, K., & Orntoft, T. F. (2004) · Cancer Research · doi:10.1158/0008-5472.can-03-3620 · Cited by 456
Lessons from Controversy: Ovarian Cancer Screening and Serum Proteomics
Ransohoff, D. F. (2005) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/dji054 · Cited by 265
High-resolution serum proteomic features for ovarian cancer detection.
Conrads, T. P., Fusaro, V. A., Ross, S., Johann, D., Rajapakse, V. N., Hitt, B. A., Steinberg, S. M., Kohn, E. C., Fishman, D. A., Whitely, G., et al. (2004) · Endocrine Related Cancer · doi:10.1677/erc.0.0110163 · Cited by 256
Effects of Atmospheric Ozone on Microarray Data Quality
Fare, T. L., Coffey, E. M., Dai, H. A., He, Y. D., Kessler, D. A., Kilian, K. A., Koch, J. E., LeProust, E. M., Marton, M. J., Meyer, M. R., et al. (2003) · Analytical Chemistry · doi:10.1021/ac034241b · Cited by 229
Dick, D. M., Foroud, T., Wetherill, L. F., Bowman, E. S., Miller, M., Rau, N. L., Moe, P. R., Samavedy, N., El‐Mallakh, R. S., Manji, H. K., et al. (2003) · The American Journal of Human Genetics · doi:10.1086/376562 · Cited by 215
Hu, J. (2005) · Briefings in Functional Genomics and Proteomics · doi:10.1093/bfgp/3.4.322 · Cited by 175
Youden, W. J. (1972) · Technometrics · doi:10.1080/00401706.1972.10488878 · Cited by 131
On the design and analysis of gene expression studies in human populations
Akey, J. M., Biswas, S., Leek, J. T., & Storey, J. D. (2007) · Nature Genetics · doi:10.1038/ng0707-807 · Cited by 130
Self-citationSupervised normalization of microarrays
Mecham, B. H., Nelson, P. S., & Storey, J. D. (2010) · Bioinformatics · doi:10.1093/bioinformatics/btq118 · Cited by 130
A gene expression bar code for microarray data
Zilliox, M. J., & Irizarry, R. A. (2007) · Nature Methods · doi:10.1038/nmeth1102 · Cited by 125
Self-citationBatch Effects and Noise in Microarray Experiments
(2009) · Wiley series in probability and statistics · doi:10.1002/9780470685983 · Cited by 107
Run Batch Effects Potentially Compromise the Usefulness of Genomic Signatures for Ovarian Cancer
Baggerly, K., Coombes, K. R., & Neeley, E. S. (2008) · Journal of Clinical Oncology · doi:10.1200/jco.2007.15.1951 · Cited by 72
Self-citationHigh-resolution serum proteomic patterns for ovarian cancer detection
Baggerly, K., Edmonson, S. R., Morris, J. S., & Coombes, K. R. (2004) · Endocrine Related Cancer · doi:10.1677/erc.1.00868 · Cited by 64
Self-citationA multilevel model to address batch effects in copy number estimation using SNP arrays
Scharpf, R. B., Ruczinski, I., Carvalho, B. S., Doan, B. Q., Chakravarti, A., & Irizarry, R. A. (2010) · Biostatistics · doi:10.1093/biostatistics/kxq043 · Cited by 46
Self-citationCross-generation and cross-laboratory predictions of Affymetrix microarrays by rank-based methods
Liu, H., Chen, C., Liu, Y., Chu, C.-B., Liang, D., Shih, L., & Lin, C. (2007) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2007.11.005 · Cited by 22
Flagged first, then by citations
Cited by
Sorted by citations
MultiQC: summarize analysis results for multiple tools and samples in a single report
Ewels, P., Magnusson, M., Lundin, S., & Käller, M. (2016) · Bioinformatics · doi:10.1093/bioinformatics/btw354 · Cited by 10,143
Predicting effective microRNA target sites in mammalian mRNAs
Agarwal, V., Bell, G. W., Nam, J.-W., & Bartel, D. P. (2015) · eLife · doi:10.7554/elife.05005 · Cited by 7,318
Leek, J. T., Johnson, W. E., Parker, H. S., Jaffe, A. E., & Storey, J. D. (2012) · Bioinformatics · doi:10.1093/bioinformatics/bts034 · Cited by 6,231
Self-citationAryee, M. J., Jaffe, A. E., Bravo, H. C., Ladd‐Acosta, C., Feinberg, A. P., Hansen, K. D., & Irizarry, R. A. (2014) · Bioinformatics · doi:10.1093/bioinformatics/btu049 · Cited by 4,389
Self-citationMetsalu, T., & Vilo, J. (2015) · Nucleic Acids Research · doi:10.1093/nar/gkv468 · Cited by 3,770
Metadata
- Authors
- Leek, J. T., Scharpf, R. B., Bravo, H. C., Simcha, D., Langmead, B., Johnson, W. E., Geman, D., Baggerly, K., & Irizarry, R. A.
- Affiliations
- Johns Hopkins University (US)
- Brigham Young University (US)
- The University of Texas MD Anderson Cancer Center (US)
- Corresponding author
- Irizarry, Rafael A.
- Venue
- Nature Reviews Genetics
- Published
- 2010-09-14
- Volume
- 11
- Issue
- 10
- Pages
- 733-739
- ISSN
- 1471-0056 (ISSN-L)
- 1471-0064
- PMC
- PMC3880143
- OpenAlex
- W2096192437
- Cited by
- 2,196
- References
- 30
- Citations by year
- 2026: 235 · 2025: 166 · 2024: 159 · 2023: 125 · 2022: 178 · 2021: 178 · 2020: 167 · 2019: 138 · 2018: 152 · 2017: 130 · 2016: 133 · 2015: 107 · 2014: 100 · 2013: 95 · 2012: 70
- FWCI
- 6.29
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Nature Reviews Genetics (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Funders
- Juvenile Diabetes Research Foundation International
- Wellcome Trust
- National Institute for Health and Care Research
- National Institutes of Health (5u41hg004059-15)
- Cambridge Institute for Medical Research, University of Cambridge (079895)
- NIHR Cambridge Biomedical Research Centre
- National Human Genome Research Institute (HG005220, U41-HG004059, HG004059, R00HG005015, R01HG005220, P41HG004059, U24HG004059)
- National Institute of General Medical Sciences (GM0083084, R01GM083084)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene expression and cancer classification
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Bioinformatics and Genomic Networks
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Single-cell and spatial transcriptomics
- Keywords
- batch effects, high-throughput omics data, confounding, gene expression, protein expression, epigenetic modifications, laboratory conditions
- MeSH terms
- Biotechnology, Periodicals as Topic, Research Design, Sequence Analysis, DNA, Computational Biology, Oligonucleotide Array Sequence Analysis, Genomics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
