The positive false discovery rate: a Bayesian interpretation and the q-value
Abstract
Multiple hypothesis testing is concerned with controlling the rate of false positives when testing several hypotheses simultaneously. One multiple hypothesis testing error measure is the false discovery rate (FDR), which is loosely defined to be the expected proportion of false positives among all significant hypotheses. The FDR is especially appropriate for exploratory analyses in which one is interested in finding several significant results among many tests. In this work, we introduce a modified version of the FDR called the "positive false discoveryrate" (pFDR). We discuss the advantages and disadvantages of the pFDR and investigate its statistical properties. When assuming the test statistics follow a mixture distribution, we show that the pFDR can be written as a Bayesian posterior probability and can be connected to classification theory. These properties remain asymptotically true under fairly general conditions, even under certain forms of dependence. Also, a new quantity called the "$q$-value" is introduced and investigated, which is a natural "Bayesian posterior p-value," or rather the pFDR analogue of the p-value.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Adjustment was made for multiple analyses using Benjamini and Hochberg's false discovery rate.
“Adjustment was made for multiple analyses according to Benja-mini and Hochberg’s theory of false discovery rate (FDR) [32, 33].”
From Psychometric assessment of a Chinese version of a mindfulness practice quality measurement tool (2024) · cites this paper as [33]
Raw p-values for each DEG analysis were adjusted via empirical false discovery rate (eFDR).
“Raw p-values for each DEG analysis were adjusted via empirical false discovery rate (eFDR) [102].”
Cites
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482 · see citing works
Convergence of Probability Measures
Billingsley, P. (1999) · Wiley series in probability and statistics · doi:10.1002/9780470316962 · Cited by 14,153 · see citing works
The control of the false discovery rate in multiple testing under dependency
Benjamini, Y., & Yekutieli, D. (2001) · The Annals of Statistics · doi:10.1214/aos/1013699998 · Cited by 10,490 · see citing works
A Direct Approach to False Discovery Rates
Storey, J. D. (2002) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/1467-9868.00346 · Cited by 5,603 · see citing works
Self-citationTesting Statistical Hypotheses
Niewiadomska-Bugaj, M., & Bartoszynski, R. (2021) · Wiley series in probability and statistics · doi:10.1002/9781119243830.ch12 · Cited by 5,188 · see citing works
Exploring the new world of the genome with DNA microarrays
Brown, P. O., & Botstein, D. (1999) · Nature Genetics · doi:10.1038/4462 · Cited by 2,430 · see citing works
An improved Bonferroni procedure for multiple tests of significance
SIMES, R. J. (1986) · Biometrika · doi:10.1093/biomet/73.3.751 · Cited by 2,396 · see citing works
Shaffer, J. P. (1995) · Annual Review of Psychology · doi:10.1146/annurev.ps.46.020195.003021 · Cited by 1,960 · see citing works
Testing Statistical Hypotheses
Lehmann, E. L. (1986) · Springer texts in statistics · doi:10.1007/978-1-4757-1923-9 · Cited by 1,876 · see citing works
Empirical Bayes analysis of a microarray experiment
Efron, B., Tibshirani, R., Storey, J. D., & Tusher, V. (2001) · Journal of the American Statistical Association · doi:10.1198/016214501753382129 · Cited by 1,733 · see citing works
Self-citationSequential tests for the detection of linkage.
Morton, N. E. (1955) · PubMed · Cited by 1,590 · see citing works
On the adaptive control of the false discovery rate in multiple testing with independent statistics
Benjamini, Y., & Hochberg, Y. (2000) · Journal of Educational and Behavioral Statistics · doi:10.3102/10769986025001060 · Cited by 1,349 · see citing works
Storey, J. D., Taylor, J., & Siegmund, D. (2003) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.1467-9868.2004.00439.x · Cited by 1,345 · see citing works
Self-citationLearning from Data: Concepts, Theory, and Methods
Cherkassky, V., & Mulier, F. M. (1998) · Cited by 1,285 · see citing works
Operating Characteristics and Extensions of the False Discovery Rate Procedure
Genovese, C. R., & Wasserman, L. (2002) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/1467-9868.00347 · Cited by 545 · see citing works
Some Results on False Discovery Rate in Stepwise multiple testing procedures
Sarkar, S. K. (2002) · The Annals of Statistics · doi:10.1214/aos/1015362192 · Cited by 269 · see citing works
A New Approach to the Problem of Multiple Comparisons in the Genetic Dissection of Complex Traits
Weller, J. I., Song, J. Z., Heyen, D. W., Lewin, H. A., & Ron, M. (1998) · Genetics · doi:10.1093/genetics/150.4.1699 · Cited by 182 · see citing works
An Improved Bonferroni Procedure for Multiple Tests of Significance
Simes, R. J. (1986) · Biometrika · doi:10.2307/2336545 · Cited by 157 · see citing works
Operating Characteristics and Extensions of the FDR Procedure
Genovese, C. R., & Wasserman, L. (2018) · Figshare · doi:10.1184/r1/6586796 · Cited by 64 · see citing works
Zaykin, D. V., Young, S. S., & Westfall, P. H. (2000) · Genetics · doi:10.1093/genetics/154.4.1917 · Cited by 43 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
A Direct Approach to False Discovery Rates
Storey, J. D. (2002) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/1467-9868.00346 · Cited by 5,603 · see citing works
Self-citationRadiomics: the process and the challenges
Kumar, V., Gu, Y., Basu, S., Berglund, A., Eschrich, S. A., Schabath, M. B., Forster, K., Aerts, H. J. W. L., Dekker, A., Fenstermacher, D., et al. (2012) · Magnetic Resonance Imaging · doi:10.1016/j.mri.2012.06.010 · Cited by 2,258 · see citing works
“False discovery rates [82–84] have been developed to provide more reasonable error estimates.”
Method… and the Cross-Section of Expected Returns
Harvey, C. R., Liu, Y., & Zhu, H. (2016) · Review of Financial Studies · doi:10.1093/rfs/hhv059 · Cited by 2,119 · see citing works
Irizarry, R. A., Ladd-Acosta, C., Wen, B., Wu, Z., Montano, C., Onyango, P., Cui, H., Gabo, K., Rongione, M., Webster, M., et al. (2009) · Nature Genetics · doi:10.1038/ng.298 · Cited by 2,177 · see citing works
Motulsky, H. J., & Brown, R. E. (2006) · BMC Bioinformatics · doi:10.1186/1471-2105-7-123 · Cited by 1,616 · see citing works
Challenges of Big Data analysis
Fan, J., Han, F., & Liu, H. (2014) · National Science Review · doi:10.1093/nsr/nwt032 · Cited by 1,482 · see citing works
Microarray data analysis: from disarray to consolidation and consensus
Allison, D. B., Cui, X., Page, G. P., & Sabripour, M. (2006) · Nature Reviews Genetics · doi:10.1038/nrg1749 · Cited by 1,339 · see citing works
Bretz, F., Hothorn, T., & Westfall, P. (2016) · doi:10.1201/9781420010909 · Cited by 1,233 · see citing works
A survey on filter techniques for feature selection in gene expression microarray analysis
Lazar, C., Taminau, J., Meganck, S., Steenhoff, D., Coletta, A., Molter, C., de Schaetzen, V., Duque, R., Bersini, H., & Nowe, A. (2012) · IEEE/ACM Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbb.2012.33 · Cited by 619 · see citing works
Miller, R., Wu, G., Deshpande, R. R., Vieler, A., Gärtner, K., Li, X., Moellering, E. R., Zäuner, S., Cornish, A. J., Liu, B., et al. (2010) · PLANT PHYSIOLOGY · doi:10.1104/pp.110.165159 · Cited by 511 · see citing works
Fletcher, B. R., Damery, S., Aiyegbusi, O. L., Anderson, N., Calvert, M., Cockwell, P., Ferguson, J., Horton, M., Paap, M. C. S., Sidey-Gibbons, C., et al. (2022) · PLOS Medicine · doi:10.1371/journal.pmed.1003954 · Cited by 414 · see citing works
The UK-PSCSC Consortium, Liu, J. Z., The International IBD Genetics Consortium, The International PSC Study Group, Hov, J. R., Folseraas, T., Ellinghaus, E., Rushbrook, S. M., Doncheva, N. T., Andreassen, O. A., et al. (2013) · Nature Genetics · doi:10.1038/ng.2616 · Cited by 391 · see citing works
Normalization, testing, and false discovery rate estimation for RNA-sequencing data
Li, J., Witten, D. M., Johnstone, I. M., & Tibshirani, R. (2012) · Biostatistics · doi:10.1093/biostatistics/kxr031 · Cited by 369 · see citing works
Park, J. E., Park, S. Y., Kim, H. J., & Kim, H. S. (2019) · Korean Journal of Radiology · doi:10.3348/kjr.2018.0070 · Cited by 388 · see citing works
Combinatorial CRISPR–Cas9 screens for de novo mapping of genetic interactions
Shen, J. P., Zhao, D., Sasik, R., Luebeck, J., Birmingham, A., Bojorquez-Gomez, A., Licon, K., Klepper, K., Pekin, D., Beckett, A. N., et al. (2017) · Nature Methods · doi:10.1038/nmeth.4225 · Cited by 359 · see citing works
An exploration of aspects of Bayesian multiple testing
Scott, J. G., & Berger, J. O. (2006) · Journal of Statistical Planning and Inference · doi:10.1016/j.jspi.2005.08.031 · Cited by 330 · see citing works
Effects of aging on circadian patterns of gene expression in the human prefrontal cortex
Chen, C.-Y., Logan, R. W., Ma, T., Lewis, D. A., Tseng, G. C., Sibille, E., & McClung, C. A. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1508249112 · Cited by 277 · see citing works
Lopes, K. D. P., Snijders, G. J. L., Humphrey, J., Allan, A., Sneeboer, M. A. M., Navarro, E., Schilder, B. M., Vialle, R. A., Parks, M., Missall, R., et al. (2022) · Nature Genetics · doi:10.1038/s41588-021-00976-y · Cited by 273 · see citing works
Technical trading revisited: False discoveries, persistence tests, and transaction costs
Bajgrowicz, P., & Scaillet, O. (2012) · Journal of Financial Economics · doi:10.1016/j.jfineco.2012.06.001 · Cited by 253 · see citing works
Aberg, K. A., McClay, J. L., Nerella, S., Clark, S., Kumar, G., Chen, W., Khachane, A. N., Xie, L., Hudson, A., Gao, G., et al. (2014) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2013.3730 · Cited by 248 · see citing works
Effects of dependence in high-dimensional multiple testing problems
Kim, K. I., & van de Wiel, M. A. (2008) · BMC Bioinformatics · doi:10.1186/1471-2105-9-114 · Cited by 220 · see citing works
“Storey [4] introduces the positive false discovery rate (pFDR) and the q-value.”
MultipleTesting.com: A tool for life science researchers for multiple hypothesis testing correction
Menyhart, O., Weltz, B., & Győrffy, B. (2021) · PLOS ONE · doi:10.1371/journal.pone.0245824 · Cited by 216 · see citing works
Impacts of Neanderthal-Introgressed Sequences on the Landscape of Human Gene Expression
McCoy, R. C., Wakefield, J., & Akey, J. M. (2017) · Cell · doi:10.1016/j.cell.2017.01.038 · Cited by 200 · see citing works
Rosenberger, G., Bludau, I., Schmitt, U., Heusel, M., Hunter, C. L., Liu, Y., MacCoss, M. J., MacLean, B. X., Nesvizhskii, A. I., Pedrioli, P. G. A., et al. (2017) · Nature Methods · doi:10.1038/nmeth.4398 · Cited by 203 · see citing works
Kennedy, J. J., Abbatiello, S. E., Kim, K., Yan, P., Whiteaker, J. R., Lin, C., Kim, J. S., Zhang, Y., Wang, X., Ivey, R. G., et al. (2014) · Nature Methods · doi:10.1038/nmeth.2763 · Cited by 184 · see citing works
Brain network connectivity assessed using graph theory in frontotemporal dementia
Agosta, F., Sala, S., Valsasina, P., Meani, A., Canu, E., Magnani, G., Cappa, S. F., Scola, E., Quatto, P., Horsfield, M. A., et al. (2013) · Neurology · doi:10.1212/wnl.0b013e31829a33f8 · Cited by 168 · see citing works
Humphrey, J., Venkatesh, S., Hasan, R., Herb, J. T., de Paiva Lopes, K., Küçükali, F., Byrska-Bishop, M., Evani, U. S., Narzisi, G., Fagegaltier, D., et al. (2023) · Nature Neuroscience · doi:10.1038/s41593-022-01205-3 · Cited by 138 · see citing works
Circadian clock gene polymorphisms in alcohol use disorders and alcohol consumption
Kovanen, L., Saarikoski, S. T., Haukka, J., Pirkola, S., Aromaa, A., Lonnqvist, J., & Partonen, T. (2010) · Alcohol and Alcoholism · doi:10.1093/alcalc/agq035 · Cited by 138 · see citing works
Oomen, D., Nijhof, A. D., & Wiersema, J. R. (2021) · Molecular Autism · doi:10.1186/s13229-021-00424-y · Cited by 136 · see citing works
Alzheimer’s disease: Ablating single master site abolishes tau hyperphosphorylation
Stefanoska, K., Gajwani, M., Tan, A. R. P., Ahel, H. I., Asih, P. R., Volkerling, A., Poljak, A., & Ittner, A. (2022) · Science Advances · doi:10.1126/sciadv.abl8809 · Cited by 128 · see citing works
McClay, J. L., Adkins, D. E., Åberg, K., Bukszár, J., Khachane, A. N., Keefe, R. S. E., Perkins, D. O., McEvoy, J. P., Stroup, T. S., Vann, R. E., et al. (2011) · Neuropsychopharmacology · doi:10.1038/npp.2010.193 · Cited by 124 · see citing works
Steiner, F. A., Talbert, P. B., Kasinathan, S., Deal, R. B., & Henikoff, S. (2012) · Genome Research · doi:10.1101/gr.131748.111 · Cited by 123 · see citing works
Adaptive, convergent origins of the pygmy phenotype in African rainforest hunter-gatherers
Perry, G. H., Foll, M., Grenier, J.-C., Patin, E., Nédélec, Y., Pacis, A., Barakatt, M., Gravel, S., Zhou, X., Nsobya, S. L., et al. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1402875111 · Cited by 105 · see citing works
Disrupted brain connectome in semantic variant of primary progressive aphasia
Agosta, F., Galantucci, S., Valsasina, P., Canu, E., Meani, A., Marcone, A., Magnani, G., Falini, A., Comi, G., & Filippi, M. (2014) · Neurobiology of Aging · doi:10.1016/j.neurobiolaging.2014.05.017 · Cited by 94 · see citing works
Altmaier, E., Kastenmüller, G., Römisch-Margl, W., Thorand, B., Weinberger, K. M., Illig, T., Adamski, J., Döring, A., & Suhre, K. (2011) · European Journal of Epidemiology · doi:10.1007/s10654-010-9524-7 · Cited by 84 · see citing works
Associations between maternal negative affect and adolescent's neural response to peer evaluation
Tan, P. Z., Lee, K. H., Dahl, R. E., Nelson, E. E., Stroud, L. J., Siegle, G. J., Morgan, J. K., & Silk, J. S. (2014) · Developmental Cognitive Neuroscience · doi:10.1016/j.dcn.2014.01.006 · Cited by 59 · see citing works
Kim, E., Kwak, S. H., Chung, H. R., Ohn, J. H., Bae, J. H., Choi, S. H., Park, K. S., Hong, J.-S., Sung, J., & Jang, H. C. (2017) · Epigenetics · doi:10.1080/15592294.2017.1370172 · Cited by 36 · see citing works
FX technical trading rules can be profitable sometimes!
Zarrabi, N., Snaith, S., & Coakley, J. (2017) · International Review of Financial Analysis · doi:10.1016/j.irfa.2016.12.010 · Cited by 32 · see citing works
Harper, J., Malone, S. M., Wilson, S., Hunt, R. H., Thomas, K. M., & Iacono, W. G. (2021) · Biological Psychiatry · doi:10.1016/j.biopsych.2021.01.006 · Cited by 26 · see citing works
Eastern national parks protect greater tree species diversity than unprotected matrix forests
Miller, K. M., McGill, B. J., Mitchell, B. R., Comiskey, J., Dieffenbach, F. W., Matthews, E. R., Perles, S. J., Paul Schmit, J., & Weed, A. S. (2018) · Forest Ecology and Management · doi:10.1016/j.foreco.2018.02.018 · Cited by 22 · see citing works
Pappa, E., Baah, F., Lynch, J., Shiel, L., Blackman, G., Raihani, N., & Bell, V. (2025) · Schizophrenia Bulletin · doi:10.1093/schbul/sbae225 · Cited by 19 · see citing works
Low-frequency intermediate penetrance variants in the ROCK1 gene predispose to Tetralogy of Fallot
Palomino Doza, J., Topf, A., Bentham, J., Bhattacharya, S., Cosgrove, C., Brook, J. D., Granados-Riveron, J., Bu’Lock, F. A., O’Sullivan, J., Stuart, A. G., et al. (2013) · BMC Genetics · doi:10.1186/1471-2156-14-57 · Cited by 14 · see citing works
Integrative Bayesian analysis of neuroimaging-genetic data with application to cocaine dependence
Azadeh, S., Hobbs, B. P., Ma, L., Nielsen, D. A., Gerard Moeller, F., & Baladandayuthapani, V. (2016) · NeuroImage · doi:10.1016/j.neuroimage.2015.10.033 · Cited by 12 · see citing works
A resource for transcriptomic analysis in the mouse brain
Plessy, C., Fagiolini, M., Wagatsuma, A., Harasawa, N., Kuji, T., Asaka-Oba, A., Kanzaki, Y., Fujishima, S., Waki, K., Nakahara, H., et al. (2008) · PLoS ONE · doi:10.1371/journal.pone.0003012 · Cited by 12 · see citing works
Burger, T. (2023) · PROTEOMICS · doi:10.1002/pmic.202200406 · Cited by 10 · see citing works
“, 1%), using adjusted p-values [19] (or similarly, q-values [20]).”
Milewski, T. M., Lee, W., Young, R. L., Hofmann, H. A., & Curley, J. P. (2025) · PLoS Genetics · doi:10.1371/journal.pgen.1011548 · Cited by 7 · see citing works
Psychometric assessment of a Chinese version of a mindfulness practice quality measurement tool
Wang, Y., Liu, X., Chen, J., Zhang, Y., Lin, X., Wang, N., Wang, J., & Luo, F. (2024) · BMC Psychology · doi:10.1186/s40359-024-02176-9 · Cited by 1 · see citing works
“Adjustment was made for multiple analyses according to Benja-mini and Hochberg’s theory of false discovery rate (FDR) [32, 33].”
- Authors
- Storey, J. D.
- Affiliations
- University of Washington (US)
- Corresponding author
- Storey, John D.
- ORCID
- Storey, John D.
- Venue
- The Annals of Statistics
- Publisher
- Institute of Mathematical Statistics
- Published
- 2003-12-01
- Volume
- 31
- Issue
- 6
- Pages
- 2013-2035
- ISSN
- 0090-5364 (ISSN-L)
- 2168-8966
- OpenAlex
- W2125905177
- Cited by
- 2,204
- References
- 20
- Citations by year
- 2026: 47 · 2025: 96 · 2024: 99 · 2023: 101 · 2022: 101 · 2021: 143 · 2020: 112 · 2019: 128 · 2018: 126 · 2017: 95 · 2016: 105 · 2015: 99 · 2014: 100 · 2013: 109 · 2012: 133
- FWCI
- 57.16
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- The Annals of Statistics (publishedVersion)
- Project Euclid (Cornell University) (submittedVersion)
- Funders
- National Science Foundation
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Inference
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Keywords
- false discovery rate, multiple comparisons, Q-value, Bayesian posterior probability, mixture distributions, classification theory, false positives, asymptotic properties
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
