Should We Abandon the t-Test in the Analysis of Gene Expression Microarray Data: A Comparison of Variance Modeling Strategies
Open accessThe DOI matches its Crossref record; 4 of the 4 references it cites were found in a registry or an index.
Abstract
High-throughput post-genomic studies are now routinely and promisingly investigated in biological and biomedical research. The main statistical approach to select genes differentially expressed between two groups is to apply a t-test, which is subject of criticism in the literature. Numerous alternatives have been developed based on different and innovative variance modeling strategies. However, a critical issue is that selecting a different test usually leads to a different gene list. In this context and given the current tendency to apply the t-test, identifying the most efficient approach in practice remains crucial. To provide elements to answer, we conduct a comparison of eight tests representative of variance modeling strategies in gene expression data: Welch's t-test, ANOVA [1], Wilcoxon's test, SAM [2], RVM [3], limma [4], VarMixt [5] and SMVar [6]. Our comparison process relies on four steps (gene list analysis, simulations, spike-in data and re-sampling) to formulate comprehensive and robust conclusions about test performance, in terms of statistical power, false-positive rate, execution time and ease of use. Our results raise concerns about the ability of some methods to control the expected number of false positives at a desirable level. Besides, two tests (limma and VarMixt) show significant improvement compared to the t-test, in particular to deal with small sample sizes. In addition limma presents several practical advantages, so we advocate its application to analyze gene expression data.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482
Statistical significance for genomewide studies
Storey, J. D., & Tibshirani, R. (2003) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1530509100 · Cited by 9,692 · cited 1×
“It differs from the false-discovery rate (FDR) in the sense that it represents the rate that truly null features are called significant whereas the FDR is the rate that significant features are truly null [21].”
BackgroundGene expression profiling predicts clinical outcome of breast cancer
van 't Veer, L. J., Dai, H., van de Vijver, M. J., He, Y. D., Hart, A. A. M., Mao, M., Peterse, H. L., van der Kooy, K., Marton, M. J., Witteveen, A. T., et al. (2002) · Nature · doi:10.1038/415530a · Cited by 9,536
Microarray data analysis: from disarray to consolidation and consensus
Allison, D. B., Cui, X., Page, G. P., & Sabripour, M. (2006) · Nature Reviews Genetics · doi:10.1038/nrg1749 · Cited by 1,339 · cited 2×
“So defined, FC lacks of a solid statistical footing [9]: it does not take the variance of the samples into account.”
BackgroundMethod
Research works printed but not linked: 27
- M Kerr, 2000, Analysis of variance for gene expression microarray data., Journal of Computational Biology, 7, 819, 10.1089/10665270050514954 · DOI · Paper page
- V Tusher, 2001, Significance analysis of microarrays applied to the ionizing radiation response., Proceedings of the National Academy of Sciences of the United States of America, 98, 5116, 10.1073/pnas.091062498 · DOI · Paper page
- G Wright, 2003, A random variance model for detection of differential gene expression in small microarray experiments., Bioinformatics, 19, 2448, 10.1093/bioinformatics/btg345 · DOI · Paper page
- G Smyth, 2004, Linear models and empirical bayes methods for assessing differential expression in microarray experiments., Statistical applications in genetics and molecular biology, 3, 10.2202/1544-6115.1027 · DOI · Paper page
- P Delmar, 2005, Varmixt: efficient variance modelling for the differential analysis of replicated gene expression data., Bioinformatics, 21, 502, 10.1093/bioinformatics/bti023 · DOI
- F Jaffrezic, 2007, A structural mixed model for variances in differential gene expression studies., Genetics Research, 89, 19, 10.1017/S0016672307008646 · DOI
- T Sorlie, 2003, Repeated observation of breast tumor subtypes in independent gene expression data sets., Proceedings of the National Academy of Sciences of the United States of America, 100, 8418, 10.1073/pnas.0932692100 · DOI · Paper page
- L Zhou, 2005, An expression index for Affymetrix GeneChips based on the generalized logarithm., Bioinformatics, 21, 3983, 10.1093/bioinformatics/bti665 · DOI
- R Simon, 2003, Pitfalls in the use of dna microarray data for diagnostic and prognostic classification., Journal of the National Cancer Institute, 95, 14, 10.1093/jnci/95.1.14 · DOI · Paper page
- C Murie, 2009, Comparison of small n statistical tests of differential expression applied to microarrays., BMC Bioinformatics, 10, 45, 10.1186/1471-2105-10-45 · DOI
- G Marot, 2009, Moderated effect size and P-value combinations for microarray meta-analyses., Bioinformatics, 25, 2692, 10.1093/bioinformatics/btp444 · DOI · Paper page
- M Pirooznia, 2008, A comparative study of different machine learning methods on microarray gene expression data., BMC Genomics, 9, S13, 10.1186/1471-2164-9-S1-S13 · DOI · Paper page
- C Yauk, 2007, Review of the literature examining the correlation among dna microarray technologies., Environmental and Molecular Mutagenesis, 48, 380, 10.1002/em.20290 · DOI
- I Jeffery, 2006, Comparison and evaluation of methods for generating differentially expressed gene lists from microarray data., BMC Bioinformatics, 7, 359, 10.1186/1471-2105-7-359 · DOI · Paper page
- M Sullivan Pepe, 2003, Selecting differentially expressed genes from microarray experiments., Biometrics, 59, 133, 10.1111/1541-0420.00016 · DOI · Paper page
- M McCall, 2008, Consolidated strategy for the analysis of microarray spike-in data., Nucleic Acids Research Advance, 36, e108, 10.1093/nar/gkn430 · DOI
- N Jain, 2003, Local-pooled-error test for identifying differentially expressed genes with a small number of replicated microarrays., Bioinformatics, 19, 1945, 10.1093/bioinformatics/btg264 · DOI · Paper page
- X Huang, 2002, Comparing three methods for variance estimation with duplicated high density oligonucleotide arrays., Functional and Integrative Genomics, 2, 10.1007/s10142-002-0066-2 · DOI
- C Kooperberg, 2005, Significance testing for small microarray experiments., Statistics in medicine, 24, 2281, 10.1002/sim.2109 · DOI
- D Chessel, 2004, The ade4 package - I : One-table methods., R News, 4, 5
- L Lamant, 2007, Gene-expression profiling of systemic anaplastic large-cell lymphoma reveals differences based on ALK status and two distinct morphologic ALK+ subtypes., Blood, 109, 2156, 10.1182/blood-2006-06-028969 · DOI · Paper page
- S Boyault, 2007, Transcriptome classification of hcc is related to gene alterations and to new therapeutic targets., Hepatology, 45, 10.1002/hep.21467 · DOI · Paper page
- D Rickman, 2008, Prediction of future metastasis and molecular characterization of head and neck squamous-cell carcinoma based on transcriptome and genome analysis by microarrays., Oncogene, 27, 6607, 10.1038/onc.2008.251 · DOI
- J Soulier, 2005, HOXA genes are included in genetic and biologic networks defining human acute T-cell leukemia (T-ALL)., Blood, 106, 274, 10.1182/blood-2004-10-3900 · DOI · Paper page
- P Bertheau, 2007, Exquisite sensitivity of TP53 mutant and basal breast cancers to a dose-dense epirubicin-cyclophosphamide regimen., PLoS Med, 4, e90, 10.1371/journal.pmed.0040090 · DOI · Paper page
- B Wu, 2005, Differential gene expression detection using penalized linear regression models: the improved SAM statistics., Bioinformatics, 21, 1565, 10.1093/bioinformatics/bti217 · DOI
- S Zhang, 2007, A comprehensive evaluation of SAM, the SAM R-package and a simple modification to improve its performance., BMC Bioinformatics, 8, 230, 10.1186/1471-2105-8-230 · DOI
Cited by
Dillies, M.-A., Rau, A., Aubert, J., Hennequet-Antier, C., Jeanmougin, M., Servant, N., Keime, C., Marot, G., Castel, D., Estelle, J., et al. (2013) · Briefings in Bioinformatics · doi:10.1093/bib/bbs046 · Cited by 1,298
Self-citationPhipson, B., Lee, S., Majewski, I. J., Alexander, W. S., & Smyth, G. K. (2016) · The Annals of Applied Statistics · doi:10.1214/16-aoas920 · Cited by 917
Exercise training reduces resting heart rate via downregulation of the funny channel HCN4
D’Souza, A., Bucchi, A., Johnsen, A. B., Logantha, S. J. R. J., Monfredi, O., Yanni, J., Prehar, S., Hart, G., Cartwright, E., Wisloff, U., et al. (2014) · Nature Communications · doi:10.1038/ncomms4775 · Cited by 275
Repression of p63 and induction of EMT by mutant Ras in mammary epithelial cells
Yoh, K. E., Regunath, K., Guzman, A., Lee, S.-M., Pfister, N. T., Akanni, O., Kaufman, L. J., Prives, C., & Prywes, R. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1613417113 · Cited by 75
van Harten, A. C., Mulders, J., Scheltens, P., van der Flier, W. M., & Oudejans, C. B. M. (2015) · Journal of Alzheimer's Disease · doi:10.3233/jad-140075 · Cited by 41
Metadata
- Authors
- Jeanmougin, M., de Reynies, A., Marisa, L., Paccard, C., Nuel, G., & Guedj, M.
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2010-09-03
- Volume
- 5
- Issue
- 9
- Pages
- e12336
- ISSN
- 1932-6203 (ISSN-L)
- OpenAlex
- W2067931814
- Cited by
- 154
- References
- 4
- FWCI
- 4.73
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Should We Abandon the t-Test in the Analysis of Gene Expression Microarray Data: A Comparison of Variance Modeling Strategies. https://app.cochat.ai/papers/10.1371/journal.pone.0012336
- The paper
- Jeanmougin, M., de Reynies, A., Marisa, L., Paccard, C., Nuel, G., & Guedj, M. (2010). Should We Abandon the t-Test in the Analysis of Gene Expression Microarray Data: A Comparison of Variance Modeling Strategies. PLoS ONE. https://doi.org/10.1371/journal.pone.0012336
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.