The practical effect of batch on genomic prediction
Open accessAbstract
Measurements from microarrays and other high-throughput technologies are susceptible to non-biological artifacts like batch effects. It is known that batch effects can alter or obscure the set of significant results and biological conclusions in high-throughput studies. Here we examine the impact of batch effects on predictors built from genomic technologies. To investigate batch effects, we collected publicly available gene expression measurements with known outcomes, and estimated batches using date. Using these data we show (1) the impact of batch effects on prediction depends on the correlation between outcome and batch in the training data, and (2) removing expression measurements most affected by batch before building predictors may improve the accuracy of those predictors. These results suggest that (1) training sets should be designed to minimize correlation between batches and outcome, and (2) methods for identifying batch-affected probes should be developed to improve prediction results for studies with high correlation between batches and outcome.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Batch effects make it difficult to draw conclusions from data across experimental batches.
“These batch effects make it difficult to draw conclusions from data across experimental batches (Leek et al., 2010; Parker and Leek, 2012; Soneson et al., 2014; Nygaard et al., 2016; Caicedo et al., 2017).”
From WILDS: A Benchmark of in-the-Wild Distribution Shifts (2020) · cites this paper as Parker and Leek, 2012
Measurements from high-throughput screens are confounded by technical artifacts that arise from differences in the execution of each batch.
“Despite attempts to carefully control experimental variables such as temperature, humidity, and reagent concentration, measurements from these screens are confounded by technical artifacts that arise from differences in the execution of each batch.”
From WILDS: A Benchmark of in-the-Wild Distribution Shifts (2020) · cites this paper as Parker and Leek, 2012
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
The citation lists haven’t been fetched yet
References 23 · Cited by 34
Research works printed but not linked: 20
- Akey JM Biswas S Leek JT Storey JD On the design and analysis of gene expression studies in human populations Nature Genetics 2007 39 807 8 author reply 808–9, URL http://dx.doi.org/10.1038/ng0707-807 17597765 · DOI
- Baggerly KA Edmonson SR Morris JS Coombes KR High-resolution serum proteomic patterns for ovarian cancer detection Endocrine-Related Cancer 2004 11 583 4 author reply 585–7, URL http://dx.doi.org/10.1677/erc.1.00868 15613439 · DOI
- Carroll JS Meyer CA Song J Li W Geistlinger TR Eeckhoute J Brodsky AS Keeton EK Fertuck KC Hall GF Wang Q Bekiranov S Sementchenko V Fox EA Silver PA Gingeras TR Liu XS Brown M Genome-wide analysis of estrogen receptor binding sites Nature Genetics 2006 38 1289 97 URL http://dx.doi.org/10.1038/ng1901 17013392 · DOI
- Edgar R Domrachev M Lash AE Gene Expression Omnibus: NCBI gene expression and hybridization array data repository Nucleic Acids Research 2002 30 207 10 URL http://dx.doi.org/10.1093/nar/30.1.207 11752295 PMC99122 · DOI · Paper page
- Efron B Hastie T Johnstone I Tibshirani R Least angle regression The Annals of Statistics 2004 32 407 499 URL http://dx.doi.org/10.1214/009053604000000067 · DOI · Paper page
- Eisen MB Spellman PT Brown PO Botstein D Cluster analysis and display of genome-wide expression patterns Proceedings of the National Academy of Sciences 1998 95 14863 8 URL http://dx.doi.org/10.1073/pnas.95.25.14863 PMC24541 9843981 · DOI · Paper page
- Geman D D’Avignon C Naiman DQ Winslow RL Classifying gene expression profiles from pairwise mRNA comparisons Statistical Applications in Genetics and Molecular Biology 2004 3 19 URL http://dx.doi.org/10.2202/1544-6115.1071 PMC1989150 16646797 · DOI
- Irizarry RA Bolstad BM Collin F Cope LM Hobbs B Speed TP Summaries of Affymetrix GeneChip probe level data Nucleic Acids Research 2003 31 e15 URL http://dx.doi.org/10.1093/nar/gng015 12582260 PMC150247 · DOI
- Johnson WE Li C Rabinovic A Adjusting batch effects in microarray expression data using empirical Bayes methods Biostatistics 2007 8 118 27 URL http://dx.doi.org/10.1093/biostatistics/kxj037 16632515 · DOI · Paper page
- Johnson WE Li W Meyer C. a. Gottardo R Carroll JS Brown M Liu XS Model-based analysis of tiling-arrays for ChIP-chip Proceedings of the National Academy of Sciences of the United States of America 2006 103 12457 62 URL http://dx.doi.org/10.1073/pnas.0601180103 16895995 PMC1567901 · DOI
- Luo J Schumacher M Scherer A Sanoudou D Megherbi D Davison T Shi T Tong W Shi L Hong H Zhao C Elloumi F Shi W Thomas R Lin S Tillinghast G Liu G Zhou Y Herman D Li Y Deng Y Fang H Bushel P Woods M Zhang J A comparison of batch effect removal methods for enhancement of prediction performance using MAQC-II microarray gene expression data The Pharmacogenomics Journal 2010 10 278 91 URL http://dx.doi.org/10.1038/tpj.2010.57 20676067 PMC2920074 · DOI
- McCall MN Bolstad BM Irizarry RA Frozen Robust Multi-Array Analysis (fRMA) Biostatistics 2010 11 242 253 URL http://dx.doi.org/10.1093/biostatistics/kxp059 20097884 PMC2830579 · DOI
- Mecham BH Nelson PS Storey JD Supervised normalization of microarrays Bioinformatics 2010 26 1308 15 URL http://dx.doi.org/10.1093/bioinformatics/btq118 20363728 PMC2865860 · DOI
- Minn AJ Gupta GP Siegel PM Bos PD Shu W Giri DD Viale A Olshen AB Gerald WL Massagué J Genes that mediate breast cancer metastasis to lung Nature 2005 436 518 24 URL http://dx.doi.org/10.1038/nature03799 16049480 PMC1283098 · DOI · Paper page
- Parkinson H Kapushesky M Kolesnikov N Rustici G Shojatalab M Abeygunawardena N Berube H Dylag M Emam I Farne A Holloway E Lukk M Malone J Mani R Pilicheva E Rayner TF Rezwan F Sharma A Williams E Bradley XZ Adamusiak T Brandizi M Burdett T Coulson R Krestyaninova M Kurnosov P Maguire E Neogi SG Rocca-Serra P Sansone S-A Sklyar N Zhao M Sarkans U Brazma A ArrayExpress update–from an archive of functional genomics experiments to the atlas of gene expression Nucleic Acids Research 2009 37 D868 72 URL http://dx.doi.org/10.1093/nar/gkn889 19015125 PMC2686529 · DOI
- Smialowski P Frishman D Kramer S Pitfalls of supervised feature selection Bioinformatics (Oxford, England) 2010 26 440 3 URL http://dx.doi.org/10.1093/bioinformatics/btp621 PMC2815655 19880370 · DOI
- Tibshirani R Regression Shrinkage and Selection via the Lasso Journal of the Royal Statistical Society: Series B 1996 58 267 288 URL http://www.jstor.org/stable/2346178
- Tibshirani R Hastie T Narasimhan B Chu G Diagnosis of multiple cancer types by shrunken centroids of gene expression Proceedings of the National Academy of Sciences 2002 99 6567 72 URL http://dx.doi.org/10.1073/pnas.082099299 PMC124443 12011421 · DOI
- Wang Y Klijn JGM Zhang Y Sieuwerts AM Look MP Yang F Talantov D Timmermans M Meijer-van Gelder ME Yu J Jatkoe T Berns EMJJ Atkins D Foekens JA Gene-expression profiles to predict distant metastasis of lymph-node-negative primary breast cancer The Lancet 2005 365 671 9 URL http://dx.doi.org/10.1016/S0140-6736(05)17947-1 15721472 · DOI
- Wu Z Irizarry RA Gentleman R Martinez-Murillo F Spencer F A Model-Based Background Adjustment for Oligonucleotide Expression Arrays Journal of the American Statistical Association 2004 99 909 917 URL http://dx.doi.org/10.1198/016214504000000683 · DOI
- Authors
- Parker, H. S., & Leek, J. T.
- Affiliations
- Johns Hopkins University (US)
- ORCID
- Leek, Jeffrey T.
- Venue
- Statistical Applications in Genetics and Molecular Biology
- Publisher
- Walter de Gruyter GmbH
- Published
- 2012-04-16
- Volume
- 11
- Issue
- 3
- Pages
- Article 10-Article 10
- ISSN
- 1544-6115 (ISSN-L)
- PMC
- PMC3760371
- OpenAlex
- W2010400359
- Cited by
- 34
- References
- 23
- Citations by year
- 2026: 2 · 2025: 1 · 2024: 2 · 2023: 5 · 2022: 3 · 2021: 2 · 2020: 2 · 2019: 2 · 2018: 1 · 2016: 3 · 2015: 3 · 2014: 7 · 2013: 1
- FWCI
- 1.42
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Statistical Applications in Genetics and Molecular Biology (publishedVersion)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- Funders
- National Institute of General Medical Sciences (R01GM083084, R01GM103552)
- National Center for Research Resources (R01RR021967)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene expression and cancer classification
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Bioinformatics and Genomic Networks
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Phylogenetic Studies
- Keywords
- batch effects, genomic prediction, gene expression microarrays, probe selection, training set construction, prediction accuracy
- MeSH terms
- Algorithms, Computer Simulation, Models, Genetic, Reproducibility of Results, Gene Expression Profiling, Genomics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
