Identifying differentially expressed genes using false discovery rate controlling procedures
Open accessAbstract
Motivation: DNA microarrays have recently been used for the purpose of monitoring expression levels of thousands of genes simultaneously and identifying those genes that are differentially expressed. The probability that a false identification (type I error) is committed can increase sharply when the number of tested genes gets large. Correlation between the test statistics attributed to gene co-regulation and dependency in the measurement errors of the gene expression levels further complicates the problem. In this paper we address this very large multiplicity problem by adopting the false discovery rate (FDR) controlling approach. In order to address the dependency problem, we present three resampling-based FDR controlling procedures, that account for the test statistics distribution, and compare their performance to that of the naïve application of the linear step-up procedure in Benjamini and Hochberg (1995). The procedures are studied using simulated microarray data, and their performance is examined relative to their ease of implementation. Results: Comparative simulation analysis shows that all four FDR controlling procedures control the FDR at the desired level, and retain substantially more power then the family-wise error rate controlling procedures. In terms of power, using resampling of the marginal distribution of each test statistics substantially improves the performance over the naïve one. The highest power is achieved, at the expense of a more sophisticated algorithm, by the resampling-based procedures that resample the joint distribution of the test statistics and estimate the level of FDR control. Availability: An R program that adjusts p-values using FDR controlling procedures is freely available over the Internet at www.math.tau.ac.il/~ybenja Contact: anatr@post.tau.ac.il * To whom correspondence should be addressed.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The false discovery rate of the differential expression test was estimated using the Benjamini-Hochberg method.
“The false discovery rate (FDR) of the differential expression test was estimated using the Benjamini-Hochberg (BH) method (Reiner et al., 2003).”
From ELAVL4, splicing, and glutamatergic dysfunction precede neuron loss in MAPT mutation cerebral organoids (2021) · cites this paper as Reiner et al., 2003
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 3
- Storey,JD and Tibshirani,R. (2003) SAM thresholding and false discovery rates for detecting differential gene expression in DNA microarrays. In Parmigiani,G, Garrett,E.S., Irizarry,R.A. and Zeger,S.L. (eds), To appear in The Analysis of Gene Expression Data: Methods and Software. Springer, New York.
- Tusher,V., Tibshirani,R. and Chu,G. (2001) Significance analysis of microarrays applied to transcriptional responses to ionizing radiation. Proc. Natl Acad. Sci. USA, 98, 5116-5121.
- Yekutieli,D. (2002) Theoretical results needed for applying the false discovery rate in statistical problems, Phd thesis, Depart- ment of Statistics and Operation Research, Tel-Aviv University.
Cited by
Sorted by citations
Subramanian, A., Tamayo, P., Mootha, V. K., Mukherjee, S., Ebert, B. L., Gillette, M. A., Paulovich, A., Pomeroy, S. L., Golub, T. R., Lander, E. S., et al. (2005) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0506580102 · Cited by 53,625 · see citing works
limma: Linear Models for Microarray Data
Smyth, G. K. (2005) · Statistics for Biology and Health · doi:10.1007/0-387-29362-0_23 · Cited by 5,796 · see citing works
Radiomics: the process and the challenges
Kumar, V., Gu, Y., Basu, S., Berglund, A., Eschrich, S. A., Schabath, M. B., Forster, K., Aerts, H. J. W. L., Dekker, A., Fenstermacher, D., et al. (2012) · Magnetic Resonance Imaging · doi:10.1016/j.mri.2012.06.010 · Cited by 2,258 · see citing works
“False discovery rates [82–84] have been developed to provide more reasonable error estimates.”
MethodMicroRNA-34a regulates cardiac ageing and function.
Boon, R. A., Iekushi, K., Lechner, S., Seeger, T., Fischer, A., Heydt, S., Kaluza, D., Tréguer, K., Carmona, G., Bonauer, A., et al. (2013) · Nature · doi:10.1038/nature11919 · Cited by 821 · see citing works
Segal, L. N., Clemente, J. C., Tsay, J.-C. J., Koralov, S. B., Keller, B. C., Wu, B. G., Li, Y., Shen, N., Ghedin, E., Morris, A., et al. (2016) · Nature Microbiology · doi:10.1038/nmicrobiol.2016.31 · Cited by 685 · see citing works
Segal, L. N., Alekseyenko, A. V., Clemente, J. C., Kulkarni, R., Wu, B., Chen, H., Berger, K. I., Goldring, R. M., Rom, W. N., Blaser, M. J., et al. (2013) · Microbiome · doi:10.1186/2049-2618-1-19 · Cited by 482 · see citing works
Aston, C., Jiang, L., & Sokolov, B. P. (2005) · Molecular Psychiatry · doi:10.1038/sj.mp.4001565 · Cited by 427 · see citing works
A Chemoproteomic Approach to Query the Degradable Kinome Using a Multi-kinase Degrader
Huang, H.-T., Dobrovolsky, D., Paulk, J., Yang, G., Weisberg, E. L., Doctor, Z. M., Buckley, D. L., Cho, J.-H., Ko, E., Jang, J., et al. (2018) · Cell Chemical Biology · doi:10.1016/j.chembiol.2017.10.005 · Cited by 437 · see citing works
Hu, T., Mills, K. T., Yao, L., Demanelis, K., Eloustaz, M., Yancy, W. S., Kelly, T. N., He, J., & Bazzano, L. A. (2012) · American Journal of Epidemiology · doi:10.1093/aje/kws264 · Cited by 417 · see citing works
Correlation and Large-Scale Simultaneous Significance Testing
Efron, B. (2007) · Journal of the American Statistical Association · doi:10.1198/016214506000001211 · Cited by 405 · see citing works
Cholesterol and matrisome pathways dysregulated in astrocytes and microglia
TCW, J., Qian, L., Pipalia, N. H., Chao, M. J., Liang, S. A., Shi, Y., Jain, B. R., Bertelsen, S. E., Kapoor, M., Marcora, E., et al. (2022) · Cell · doi:10.1016/j.cell.2022.05.017 · Cited by 411 · see citing works
Park, J. E., Park, S. Y., Kim, H. J., & Kim, H. S. (2019) · Korean Journal of Radiology · doi:10.3348/kjr.2018.0070 · Cited by 388 · see citing works
Golland, Y., Bentin, S., Gelbard, H., Benjamini, Y., Heller, R., Nir, Y., Hasson, U., & Malach, R. (2007) · Cerebral Cortex · doi:10.1093/cercor/bhk030 · Cited by 337 · see citing works
Self-citationLower Airway Dysbiosis Affects Lung Cancer Progression
Tsay, J.-C. J., Wu, B. G., Sulaiman, I., Gershner, K., Schluger, R., Li, Y., Yie, T.-A., Meyn, P., Olsen, E., Perez, L., et al. (2021) · Cancer Discovery · doi:10.1158/2159-8290.cd-20-0263 · Cited by 322 · see citing works
Diverse Genome-wide Association Studies Associate the IL12/IL23 Pathway with Crohn Disease
Wang, K., Zhang, H., Kugathasan, S., Annese, V., Bradfield, J. P., Russell, R. K., Sleiman, P. M. A., Imielinski, M., Glessner, J., Hou, C., et al. (2009) · The American Journal of Human Genetics · doi:10.1016/j.ajhg.2009.01.026 · Cited by 291 · see citing works
Locus coeruleus cellular and molecular pathology during the progression of Alzheimer’s disease
Kelly, S. C., He, B., Perez, S. E., Ginsberg, S. D., Mufson, E. J., & Counts, S. E. (2017) · Acta Neuropathologica Communications · doi:10.1186/s40478-017-0411-2 · Cited by 286 · see citing works
Precision medicine and radiogenomics in breast cancer: new approaches toward diagnosis and treatment
Pinker, K., Chin, J., Melsaether, A. N., Morris, E. A., & Moy, L. (2018) · Radiology · doi:10.1148/radiol.2018172171 · Cited by 275 · see citing works
Enhancement of cytotoxicity of artemisinins toward cancer cells by ferrous iron
Efferth, T., Benakis, A., Romero, M. R., Tomicic, M., Rauh, R., Steinbach, D., Häfer, R., Stamminger, T., Oesch, F., Kaina, B., et al. (2004) · Free Radical Biology and Medicine · doi:10.1016/j.freeradbiomed.2004.06.023 · Cited by 266 · see citing works
Pishvaian, M. J., Bender, R. J., Halverson, D., Rahib, L., Hendifar, A. E., Mikhail, S., Chung, V., Picozzi, V. J., Sohal, D., Blais, E. M., et al. (2018) · Clinical Cancer Research · doi:10.1158/1078-0432.ccr-18-0531 · Cited by 245 · see citing works
Effects of dependence in high-dimensional multiple testing problems
Kim, K. I., & van de Wiel, M. A. (2008) · BMC Bioinformatics · doi:10.1186/1471-2105-9-114 · Cited by 220 · see citing works
Glutathione S transferase deficiency and passive smoking increase childhood asthma
Kabesch, M. (2004) · Thorax · doi:10.1136/thx.2003.016667 · Cited by 209 · see citing works
EphrinB2 Regulation by PTH and PTHrP Revealed by Molecular Profiling in Differentiating Osteoblasts
Allan, E. H., Häusler, K. D., Wei, T., Gooi, J. H., Quinn, J. M., Crimeen-Irwin, B., Pompolo, S., Sims, N. A., Gillespie, M. T., Onyia, J. E., et al. (2008) · Journal of Bone and Mineral Research · doi:10.1359/jbmr.080324 · Cited by 199 · see citing works
Mischak, H., Coon, J. J., Novak, J., Weissinger, E. M., Schanstra, J. P., & Dominiczak, A. F. (2009) · Mass Spectrometry Reviews · doi:10.1002/mas.20205 · Cited by 193 · see citing works
Bowles, K. R., Silva, M. C., Whitney, K., Bertucci, T., Berlind, J. E., Lai, J. D., Garza, J. C., Boles, N. C., Mahali, S., Strang, K. H., et al. (2021) · Cell · doi:10.1016/j.cell.2021.07.003 · Cited by 175 · see citing works
Higami, Y., Pugh, T. D., Page, G. P., Allison, D. B., Prolla, T. A., & Weindruch, R. (2004) · The FASEB Journal · doi:10.1096/fj.03-0678fje · Cited by 173 · see citing works
Nrf2-dependent and -independent Responses to Nitro-fatty Acids in Human Endothelial Cells
Kansanen, E., Jyrkkänen, H.-K., Volger, O. L., Leinonen, H., Kivelä, A. M., Häkkinen, S.-K., Woodcock, S. R., Schopfer, F. J., Horrevoets, A. J., Ylä-Herttuala, S., et al. (2009) · Journal of Biological Chemistry · doi:10.1074/jbc.m109.064873 · Cited by 165 · see citing works
Muñoz, D. P., Yannone, S. M., Daemen, A., Sun, Y., Vakar-Lopez, F., Kawahara, M., Freund, A. M., Rodier, F., Wu, J. D., Desprez, P.-Y., et al. (2019) · JCI Insight · doi:10.1172/jci.insight.124716 · Cited by 162 · see citing works
Spatial Differentiation in the Vegetative Mycelium of <i>Aspergillus niger</i>
Levin, A. M., de Vries, R. P., Conesa, A., de Bekker, C., Talon, M., Menke, H. H., van Peij, N. N. M. E., & Wösten, H. A. B. (2007) · Eukaryotic Cell · doi:10.1128/ec.00244-07 · Cited by 129 · see citing works
Antitumorigenesis of antioxidants in a transgenic Rac1 model of Kaposi's sarcoma
Ma, Q., Cavallin, L. E., Yan, B., Zhu, S., Duran, E. M., Wang, H., Hale, L. P., Dong, C., Cesarman, E., Mesri, E. A., et al. (2009) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0812688106 · Cited by 105 · see citing works
Initiative, F. T. A. D. N., Ridge, P. G., Karch, C. M., Hsu, S., Arano, I., Teerlink, C. C., Ebbert, M. T. W., Gonzalez Murcia, J. D., Farnham, J. M., Damato, A. R., et al. (2017) · Genome Medicine · doi:10.1186/s13073-017-0486-1 · Cited by 83 · see citing works
MeCP2 modulates gene expression pathways in astrocytes
Yasui, D. H., Xu, H., Dunaway, K. W., LaSalle, J. M., Jin, L.-W., & Maezawa, I. (2013) · Molecular Autism · doi:10.1186/2040-2392-4-3 · Cited by 90 · see citing works
Age‐related alterations of gene expression patterns in human CD8<sup>+</sup> T cells
Cao, J., Gollapudi, S., Sharman, E. H., Jia, Z., & Gupta, S. (2010) · Aging Cell · doi:10.1111/j.1474-9726.2009.00534.x · Cited by 71 · see citing works
Bamberger, C., Pankow, S., Martínez-Bartolomé, S., Ma, M., Diedrich, J., Rissman, R. A., & Yates, J. R. (2021) · Journal of Proteome Research · doi:10.1021/acs.jproteome.0c00912 · Cited by 70 · see citing works
Conklin, Q. A., King, B. G., Zanesco, A. P., Lin, J., Hamidi, A. B., Pokorny, J. J., Álvarez-López, M. J., Cosín-Tomás, M., Huang, C., Kaliman, P., et al. (2018) · Brain, Behavior, and Immunity · doi:10.1016/j.bbi.2018.03.003 · Cited by 66 · see citing works
Wingo, A. P., & Gibson, G. (2015) · Brain, Behavior, and Immunity · doi:10.1016/j.bbi.2014.09.016 · Cited by 59 · see citing works
Verma, P., Sharma, A., Sodhi, M., Thakur, K., Kataria, R. S., Niranjan, S. K., Bharti, V. K., Kumar, P., Giri, A., Kalia, S., et al. (2018) · Scientific Reports · doi:10.1038/s41598-018-25736-7 · Cited by 63 · see citing works
Rdh12 activity and effects on retinoid processing in the murine retina
Chrispell, J. D., Feathers, K. L., Kane, M. A., Kim, C. Y., Brooks, M., Khanna, R., Kurth, I., Hübner, C. A., Gal, A., Mears, A. J., et al. (2009) · Journal of Biological Chemistry · doi:10.1074/jbc.m109.020966 · Cited by 56 · see citing works
Mitochondrial fat oxidation is essential for lipid-induced inflammation in skeletal muscle in mice
Warfel, J. D., Bermudez, E. M., Mendoza, T. M., Ghosh, S., Zhang, J., Elks, C. M., Mynatt, R., & Vandanmagsar, B. (2016) · Scientific Reports · doi:10.1038/srep37941 · Cited by 50 · see citing works
Guérin, E., Raffelsberger, W., Pencreach, E., Maier, A., Neuville, A., Schneider, A., Bachellier, P., Rohr, S., Petitprez, A., Poch, O., et al. (2012) · Molecular Medicine · doi:10.2119/molmed.2011.00120 · Cited by 44 · see citing works
Gene Expression Patterns Associated With Histopathology in Toxic Liver Fibrosis
Ippolito, D. L., AbdulHameed, M. D. M., Tawa, G. J., Baer, C. E., Permenter, M. G., McDyre, B. C., Dennis, W. E., Boyle, M. H., Hobbs, C. A., Streicker, M. A., et al. (2016) · Toxicological Sciences · doi:10.1093/toxsci/kfv214 · Cited by 43 · see citing works
Song, X., Siriwardhana, N., Rathore, K., Lin, D., & Wang, H. R. (2010) · Molecular Carcinogenesis · doi:10.1002/mc.20616 · Cited by 40 · see citing works
Identification of gene expression profiles predicting tumor cell response to L-alanosine
Efferth, T., Gebhart, E., Ross, D. D., & Sauerbrey, A. (2003) · Biochemical Pharmacology · doi:10.1016/s0006-2952(03)00341-1 · Cited by 29 · see citing works
Immune Response Gene Profiles in the Term Placenta Depend Upon Maternal Muscle Mass
O’Tierney, P. F., Lewis, R. M., McWeeney, S. K., Hanson, M. A., Inskip, H. M., Morgan, T. K., Barker, D. J., Bagby, G., Cooper, C., Godfrey, K. M., et al. (2012) · Reproductive Sciences · doi:10.1177/1933719112440051 · Cited by 24 · see citing works
A microglia clonal inflammatory disorder in Alzheimer’s disease
Vicario, R., Fragkogianni, S., Weber, L., Lazarov, T., Hu, Y., Hayashi, S. Y., Craddock, B., Socci, N. D., Alberdi, A., Baako, A., et al. (2025) · eLife · doi:10.7554/elife.96519.3 · Cited by 21 · see citing works
Abnormal DNA methylation within HPA-axis genes years after paediatric critical illness
Coppens, G., Vanhorebeek, I., Güiza, F., Derese, I., Wouters, P. J., Téblick, A., Dulfer, K., Joosten, K. F., Verbruggen, S. C., & Van den Berghe, G. (2024) · Clinical Epigenetics · doi:10.1186/s13148-024-01640-y · Cited by 15 · see citing works
- Authors
- Reiner, A., Yekutieli, D., & Benjamini, Y.
- Affiliations
- Department of Statistics and Operations Research, The Sackler Faculty of Exact Sciences, Tel-Aviv University, Tel-Aviv 69978, Israel
- Tel Aviv University (IL)
- Venue
- Bioinformatics
- Publisher
- Oxford University Press (OUP)
- Published
- 2003-02-12
- Volume
- 19
- Issue
- 3
- Pages
- 368-375
- ISSN
- 1367-4803 (ISSN-L)
- 1367-4811
- OpenAlex
- W2108819945
- Cited by
- 1,765
- References
- 1
- Citations by year
- 2026: 30 · 2025: 39 · 2024: 56 · 2023: 55 · 2022: 68 · 2021: 86 · 2020: 91 · 2019: 78 · 2018: 74 · 2017: 74 · 2016: 64 · 2015: 83 · 2014: 77 · 2013: 73 · 2012: 74
- FWCI
- 33.51
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Bioinformatics (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Institutes of Health
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene expression and cancer classification
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Bioinformatics and Genomic Networks
- Keywords
- false discovery rate, differentially expressed genes, DNA microarrays, multiple comparisons, gene co-regulation, measurement error correlation, Benjamini-Hochberg procedure, family-wise error rate, statistical power, p-value adjustment
- MeSH terms
- Algorithms, Animals, Computer Simulation, Gene Expression Regulation, Cholesterol, HDL, Liver, Mice, Inbred C57BL, Models, Genetic, Reproducibility of Results, Models, Statistical, DNA, Complementary, Sample Size, Oligonucleotide Array Sequence Analysis, Gene Expression Profiling, Mice
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
