The DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.
On the Adaptive Control of the False Discovery Rate in Multiple Testing With Independent Statistics
Abstract
A new approach to problems of multiple significance testing was presented in Benjamini and Hochberg (1995), which calls for controlling the expected ratio of the number of erroneous rejections to the number of rejections–the False Discovery Rate (FDR). The procedure given there was shown to control the FDR for independent test statistics. When some of the hypotheses are in fact false, that procedure is too conservative. We present here an adaptive procedure, where the number of true null hypotheses is estimated first as in Hochberg and Benjamini (1990), and this estimate is used in the procedure of Benjamini and Hochberg (1995). The result is still a simple stepwise procedure, to which we also give a graphical companion. The new procedure is used in several examples drawn from educational and behavioral studies, addressing problems in multi-center studies, subset analysis and meta-analysis. The examples vary in the number of hypotheses tested, and the implication of the new procedure on the conclusions. In a large simulation study of independent test statistics the adaptive procedure is shown to control the FDR and have substantially better power than the previously suggested FDR controlling method, which by itself is more powerful than the traditional family wise error-rate controlling methods. In cases where most of the tested hypotheses are far from being true there is hardly any penalty due to the simultaneous testing of many hypotheses.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
p-values were adjusted for multiple testing by controlling the false discovery rate according to the method of Benjamini and Hochberg.
“p-values were adjusted for multiple testing by controlling the false discovery rate (FDR) according to the method of Benjamini and Hochberg [44].”
From Shared and distinct microRNA profiles between HT22, N2A and SH-SY5Y cell lines and primary mouse hippocampal neurons (2025) · cites this paper as [44]
In mouse and rat hippocampal neurons, miR-103, miR-130a, miR-130b, miR-134, miR-376 and miR-449 are in the group with intermediate expression.
“and miR-103, miR-130a, miR-130b, miR-134, miR-376 and miR-449 are in the group with intermediate expression [44].”
From Shared and distinct microRNA profiles between HT22, N2A and SH-SY5Y cell lines and primary mouse hippocampal neurons (2025) · cites this paper as [44]
Citations
Cites
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482
Self-citationAn improved Bonferroni procedure for multiple tests of significance
SIMES, R. J. (1986) · Biometrika · doi:10.1093/biomet/73.3.751 · Cited by 2,396
Flagged first, then by citations
Research works printed but not linked: 26
- Benjamini Y, 1993, Series in Statistics 93.1, Technical Report of the Department of Statistics and OR
- 10.1080/01621459.1998.10474112 · DOI
- Cox DR, 1965, Technometrics, 2, 149
- Harvånek & Chytil, 1983, Bulletin of the International Statistical Institution, 50, l
- 10.1093/biomet/75.4.773 · DOI
- 10.1093/biomet/75.4.800 · DOI · Paper page
- 10.1002/sim.4780090710 · DOI · Paper page
- 10.1002/9780470316672 · DOI · Paper page
- Holm S, 1979, Scandinavian Journal of Statistics, 6, 65
- 10.1093/biomet/75.2.383 · DOI
- Hommel G, 1987, Multiple hypotheses testing, 154
- 10.1037/0033-2909.114.2.266 · DOI
- 10.1111/0162-895X.00104 · DOI
- 10.1093/biomet/69.3.493 · DOI
- 10.1080/00401706.1968.10490605 · DOI
- 10.1016/S0378-3758(99)00037-3 · DOI
- 10.1016/S0378-3758(99)00028-2 · DOI
- 10.1146/annurev.ps.46.020195.003021 · DOI · Paper page
- Soriç B, 1989, Journal of the American Statistical Association, 84, 608
- 10.1016/S0378-3758(99)00145-7 · DOI
- 10.1214/ss/1177011945 · DOI · Paper page
- 10.1126/science.333584 · DOI
- 10.1055/s-0038-1635393 · DOI
- Williams VSL, 1994, Technical Report #33
- 10.3102/10769986024001042 · DOI
- 10.1016/S0378-3758(99)00041-5 · DOI
Cited by
Sorted by citations
The control of the false discovery rate in multiple testing under dependency
Benjamini, Y., & Yekutieli, D. (2001) · The Annals of Statistics · doi:10.1214/aos/1013699998 · Cited by 10,490
Self-citationStatistical significance for genomewide studies
Storey, J. D., & Tibshirani, R. (2003) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1530509100 · Cited by 9,692
A Direct Approach to False Discovery Rates
Storey, J. D. (2002) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/1467-9868.00346 · Cited by 5,603
Controlling the false discovery rate in behavior genetics research
Benjamini, Y., Drai, D., Elmer, G., Kafkafi, N., & Golani, I. (2001) · Behavioural Brain Research · doi:10.1016/s0166-4328(01)00297-2 · Cited by 4,146
Self-citationThe positive false discovery rate: a Bayesian interpretation and the q-value
Storey, J. D. (2003) · The Annals of Statistics · doi:10.1214/aos/1074290335 · Cited by 2,204
Identifying differentially expressed genes using false discovery rate controlling procedures
Reiner, A., Yekutieli, D., & Benjamini, Y. (2003) · Bioinformatics · doi:10.1093/bioinformatics/btf877 · Cited by 1,765
Self-citationBretz, F., Hothorn, T., & Westfall, P. (2016) · doi:10.1201/9781420010909 · Cited by 1,233
Mao, S., Zhang, M., Liu, J., & Zhu, W. (2015) · Scientific Reports · doi:10.1038/srep16116 · Cited by 532
Laffel, L. M., Kanapka, L. G., Beck, R. W., Bergamo, K., Clements, M. A., Criego, A., DeSalvo, D. J., Goland, R., Hood, K., Liljenquist, D., et al. (2020) · JAMA · doi:10.1001/jama.2020.6940 · Cited by 443
Frazier, T. W., Georgiades, S., Bishop, S. L., & Hardan, A. Y. (2014) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1016/j.jaac.2013.12.004 · Cited by 409
Genetics of brain fiber architecture and intellectual performance
Chiang, M.-C., Barysheva, M., Shattuck, D. W., Lee, A. D., Madsen, S. K., Avedissian, C., Klunder, A. D., Toga, A. W., McMahon, K. L., de Zubicaray, G. I., et al. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.4184-08.2009 · Cited by 406
Lainhart, J. E., Bigler, E. D., Bocian, M., Coon, H., Dinh, E., Dawson, G., Deutsch, C. K., Dunn, M., Estes, A., Tager‐Flusberg, H., et al. (2006) · American Journal of Medical Genetics Part A · doi:10.1002/ajmg.a.31465 · Cited by 381
Pratley, R. E., Kanapka, L. G., Rickels, M. R., Ahmann, A., Aleppo, G., Beck, R., Bhargava, A., Bode, B. W., Carlson, A., Chaytor, N. S., et al. (2020) · JAMA · doi:10.1001/jama.2020.6928 · Cited by 375
Bergenstal, R. M., Nimri, R., Beck, R. W., Criego, A., Laffel, L., Schatz, D., Battelino, T., Danne, T., Weinzimer, S. A., Sibayan, J., et al. (2021) · The Lancet · doi:10.1016/s0140-6736(20)32514-9 · Cited by 344
Hosseini, S., Wilk, E., Michaelsen-Preusse, K., Gerhauser, I., Baumgärtner, W., Geffers, R., Schughart, K., & Korte, M. (2018) · The Journal of Neuroscience · doi:10.1523/jneurosci.1740-17.2018 · Cited by 236
Benjamini, Y., & Liu, W. (1999) · Journal of Statistical Planning and Inference · doi:10.1016/s0378-3758(99)00040-3 · Cited by 242
Self-citationClinical and neural effects of six-week administration of oxytocin on core symptoms of autism
Watanabe, T., Kuroda, M., Kuwabara, H., Aoki, Y., Iwashiro, N., Tatsunobu, N., Takao, H., Nippashi, Y., Kawakubo, Y., Kunimatsu, A., et al. (2015) · Brain · doi:10.1093/brain/awv249 · Cited by 222
Compton, S. N., Peris, T. S., Almirall, D., Birmaher, B., Sherrill, J., Kendall, P. C., March, J. S., Gosch, E. A., Ginsburg, G. S., Rynn, M. A., et al. (2014) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0035458 · Cited by 226
Anisimova, M., & Yang, Z. (2007) · Molecular Biology and Evolution · doi:10.1093/molbev/msm042 · Cited by 213
Hardan, A. Y., Libove, R. A., Keshavan, M. S., Melhem, N. M., & Minshew, N. J. (2009) · Biological Psychiatry · doi:10.1016/j.biopsych.2009.04.024 · Cited by 173
Large-scale brain network dynamics supporting adolescent cognitive control
Dwyer, D. B., Harrison, B. J., Yücel, M., Whittle, S., Zalesky, A., Pantelis, C., Allen, N. B., & Fornito, A. (2014) · The Journal of Neuroscience · doi:10.1523/jneurosci.1634-14.2014 · Cited by 143
A Clinical Trial of In-Home CBT for Depressed Mothers in Home Visitation
Ammerman, R. T., Putnam, F. W., Altaye, M., Stevens, J., Teeters, A. R., & Van Ginkel, J. B. (2013) · Behavior Therapy · doi:10.1016/j.beth.2013.01.002 · Cited by 123
Patisaul, H. B., Sullivan, A. W., Radford, M. E., Walker, D. M., Adewale, H. B., Winnik, B., Coughlin, J. L., Buckley, B., & Gore, A. C. (2012) · PLoS ONE · doi:10.1371/journal.pone.0043890 · Cited by 106
Sanchez‐Roige, S., Fontanillas, P., Elson, S. L., The 23andMe Research Team, Gray, J. C., de Wit, H., Davis, L. K., MacKillop, J., & Palmer, A. A. (2019) · Addiction Biology · doi:10.1111/adb.12574 · Cited by 101
Nozawa, T., Sakaki, K., Ikeda, S., Jeong, H., Yamazaki, S., Kawata, K. H. D. S., Kawata, N. Y. D. S., Sasaki, Y., Kulason, K., Hirano, K., et al. (2019) · Scientific Reports · doi:10.1038/s41598-019-49257-z · Cited by 95
Wolchik, S. A., Sandler, I. N., Tein, J.-Y., Mahrer, N. E., Millsap, R. E., Winslow, E., Vélez, C., Porter, M. M., Luecken, L. J., & Reed, A. (2013) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0033235 · Cited by 91
Panigrahy, A., Schmithorst, V. J., Wisnowski, J. L., Watson, C. G., Bellinger, D. C., Newburger, J. W., & Rivkin, M. J. (2015) · NeuroImage: Clinical · doi:10.1016/j.nicl.2015.01.013 · Cited by 86
Caracausi, M., Ghini, V., Locatelli, C., Mericio, M., Piovesan, A., Antonaros, F., Pelleri, M. C., Vitale, L., Vacca, R. A., Bedetti, F., et al. (2018) · Scientific Reports · doi:10.1038/s41598-018-20834-y · Cited by 84
Scholtens, D. M., Bain, J. R., Reisetter, A. C., Muehlbauer, M. J., Nodzenski, M., Stevens, R. D., Ilkayeva, O., Lowe, L. P., Metzger, B. E., Newgard, C. B., et al. (2016) · Diabetes · doi:10.2337/db15-1748 · Cited by 69
Walker, D. M., Goetz, B. M., & Gore, A. C. (2014) · Molecular Endocrinology · doi:10.1210/me.2013-1270 · Cited by 73
Miller, K. M., Kanapka, L. G., Rickels, M. R., Ahmann, A. J., Aleppo, G., Ang, L., Bhargava, A., Bode, B. W., Carlson, A., Chaytor, N. S., et al. (2022) · Diabetes Technology & Therapeutics · doi:10.1089/dia.2021.0503 · Cited by 73
Salanova, M., Gelfi, C., Moriggi, M., Vasso, M., Viganò, A., Minafra, L., Bonifacio, G., Schiffl, G., Gutsmann, M., Felsenberg, D., et al. (2014) · The FASEB Journal · doi:10.1096/fj.14-252825 · Cited by 68
Cox, S. R., Harris, M. A., Ritchie, S. J., Buchanan, C. R., Valdés Hernández, M. C., Corley, J., Taylor, A. M., Madole, J. W., Harris, S. E., Whalley, H. C., et al. (2021) · Molecular Psychiatry · doi:10.1038/s41380-020-00975-1 · Cited by 60
Strunz, T., Lauwen, S., Kiel, C., International AMD Genomics Consortium (IAMDGC), Fritsche, L. G., Igl, W., Bailey, J. N. C., Grassmann, F., Sengupta, S., Bragg-Gresham, J. L., et al. (2020) · Scientific Reports · doi:10.1038/s41598-020-58510-9 · Cited by 60
Prenatal cocaine effects on brain structure in early infancy
Grewen, K., Burchinal, M., Vachet, C., Gouttard, S., Gilmore, J. H., Lin, W., Johns, J., Elam, M., & Gerig, G. (2014) · NeuroImage · doi:10.1016/j.neuroimage.2014.06.070 · Cited by 65
Vercammen, A., Knegtering, H., Bruggeman, R., & Aleman, A. (2011) · Schizophrenia Bulletin · doi:10.1093/schbul/sbq007 · Cited by 63
Functional brain connectivity in ex utero premature infants compared to in utero fetuses
De Asis-Cruz, J., Kapse, K., Basu, S. K., Said, M., Scheinost, D., Murnick, J., Chang, T., du Plessis, A., & Limperopoulos, C. (2020) · NeuroImage · doi:10.1016/j.neuroimage.2020.117043 · Cited by 63
Karcher, N. R., Michelini, G., Kotov, R., & Barch, D. M. (2021) · Biological Psychiatry: Cognitive Neuroscience and Neuroimaging · doi:10.1016/j.bpsc.2020.09.008 · Cited by 55
Bionic Pancreas Research Group, Messer, L. H., Buckingham, B. A., Cogen, F., Daniels, M., Forlenza, G., Jafri, R. Z., Mauras, N., Muir, A., Wadwa, R. P., et al. (2022) · Diabetes Technology & Therapeutics · doi:10.1089/dia.2022.0201.pub · Cited by 59
Gupta, A., Mayer, E. A., Acosta, J. R., Hamadani, K., Torgerson, C., van Horn, J. D., Chang, L., Naliboff, B., Tillisch, K., & Labus, J. S. (2017) · Neurobiology of Stress · doi:10.1016/j.ynstr.2017.02.003 · Cited by 56
Chien, H., Lin, H., Lai, M., Gau, S. S., & Tseng, W. I. (2015) · Autism Research · doi:10.1002/aur.1457 · Cited by 54
Morphological brain measures of cortico‐limbic inhibition related to resilience
Gupta, A., Love, A., Kilpatrick, L. A., Labus, J. S., Bhatt, R., Chang, L., Tillisch, K., Naliboff, B., & Mayer, E. A. (2017) · Journal of Neuroscience Research · doi:10.1002/jnr.24007 · Cited by 54
Kwong, K. S., Holland, B., & Cheung, S. H. (2002) · Journal of Statistical Planning and Inference · doi:10.1016/s0378-3758(01)00252-x · Cited by 54
Bionic Pancreas Research Group, Beck, R. W., Russell, S. J., Damiano, E. R., El-Khatib, F. H., Ruedy, K. J., Balliro, C., Li, Z., & Calhoun, P. (2022) · Diabetes Technology & Therapeutics · doi:10.1089/dia.2022.0167 · Cited by 40
Bionic Pancreas Research Group, Kruger, D., Kass, A., Lonier, J., Pettus, J., Raskin, P., Salam, M., Trikudanathan, S., Zhou, K., Russell, S. J., et al. (2022) · Diabetes Technology & Therapeutics · doi:10.1089/dia.2022.0200 · Cited by 38
Levy, C. J., Raghinaru, D., Kudva, Y. C., Pandit, K., Blevins, T., Casaubon, L., Desjardins, D., Levister, C. M., O’Malley, G., Reid, C., et al. (2024) · Clinical Diabetes · doi:10.2337/cd23-0025 · Cited by 37
Intermittent Fasting Implementation and Association with Eating Disorder Symptomatology
Cuccolo, K., Kramer, R., Petros, T., & Thoennes, M. (2022) · Eating Disorders · doi:10.1080/10640266.2021.1922145 · Cited by 33
Mindfulness, Obsessive–Compulsive Symptoms, and Executive Dysfunction
Crowe, K., & McKay, D. (2016) · Cognitive Therapy and Research · doi:10.1007/s10608-016-9777-x · Cited by 32
Amyloid-beta induced retrograde axonal degeneration in a mouse tauopathy model
Nishioka, C., Liang, H.-F., Barsamian, B., & Sun, S.-W. (2019) · NeuroImage · doi:10.1016/j.neuroimage.2019.01.007 · Cited by 30
Implicit Cognitions as a Behavioral Marker of Suicide Attempts in Adolescents
Millner, A. J., Augenstein, T. M., Visser, K. H., Gallagher, K., Vergara, G. A., D’Angelo, E. J., & Nock, M. K. (2019) · Archives of Suicide Research · doi:10.1080/13811118.2017.1421488 · Cited by 29
Foster, M. E., Anthony, J. L., Zucker, T. A., & Branum-Martin, L. (2019) · Early Childhood Research Quarterly · doi:10.1016/j.ecresq.2018.02.007 · Cited by 26
Thissen, A. J. A. M., Luman, M., Hartman, C., Hoekstra, P., van Lieshout, M., Franke, B., Oosterlaan, J., Rommelse, N. N. J., & Buitelaar, J. K. (2014) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1016/j.jaac.2014.05.015 · Cited by 22
Van Name, M. A., Kanapka, L. G., DiMeglio, L. A., Miller, K. M., Albanese-O’Neill, A., Commissariat, P., Corathers, S. D., Harrington, K. R., Hilliard, M. E., Anderson, B. J., et al. (2023) · Journal of Diabetes Science and Technology · doi:10.1177/19322968221084667 · Cited by 20
Mikkilineni, L., Natrakul, D. A., Lam, N., Manasanch, E. E., Mann, J., Weissler, K. A., Wong, N., Brudno, J. N., Goff, S. L., Yang, J. C., et al. (2024) · Molecular Therapy · doi:10.1016/j.ymthe.2023.12.018 · Cited by 16
Chebyshev’s Inequality for Nonparametric Testing with Small N and α in Microarray Research
Beasley, T. M., Page, G. P., Brand, J. P. L., Gadbury, G. L., Mountz, J. D., & Allison, D. B. (2004) · Journal of the Royal Statistical Society Series C: Applied Statistics · doi:10.1111/j.1467-9876.2004.00428.x · Cited by 15
Integrative Data Analysis and the Study of Global Health
Hussong, A. M., Cole, V. T., Curran, P. J., Bauer, D. J., & Gottfredson, N. C. (2020) · ICSA Book Series in Statistics · doi:10.1007/978-3-030-35260-8_5 · Cited by 13
Child anxiety prevention study: impact on functional outcomes
Pella, J. E., Drake, K. L., Tein, J.-Y., & Ginsburg, G. S. (2017) · Child Psychiatry & Human Development · doi:10.1007/s10578-016-0667-y · Cited by 12
Crowl, S., Coleman, M. B., Chaphiv, A., Jordan, B. T., & Naegle, K. M. (2025) · Cell Systems · doi:10.1016/j.cels.2025.101318 · Cited by 10
Allergic tendencies are associated with larger gray matter volumes
Takeuchi, H., Taki, Y., Nouchi, R., Yokoyama, R., Kotozaki, Y., Nakagawa, S., Sekiguchi, A., Iizuka, K., Yamamoto, Y., Hanawa, S., et al. (2018) · Scientific Reports · doi:10.1038/s41598-018-21985-8 · Cited by 8
Koning, S. M., Kessler, C. L., Canli, T., Duman, E. A., Adam, E. K., Zinbarg, R., Craske, M. G., Stephens, J. E., & Vrshek-Schallhorn, S. (2024) · Psychoneuroendocrinology · doi:10.1016/j.psyneuen.2024.107181 · Cited by 8
Anitua, E., Murias-Freijo, A., Tierno, R., Tejero, R., & Alkhraisat, M. H. (2024) · BMC Oral Health · doi:10.1186/s12903-024-04675-y · Cited by 7
Boyd, M. C., Burdette, J. H., Miller, M. E., Lyday, R. G., Hugenschmidt, C. E., Jack Rejeski, W., Simpson, S. L., Baker, L. D., Tomlinson, C. E., Kritchevsky, S. B., et al. (2024) · GeroScience · doi:10.1007/s11357-024-01251-8 · Cited by 3
Cell signaling pathways discovery from multi-modal data
He, C., Simpson, C., Cossentino, I., Zhang, B., Tkachev, S., Eddins, D. J., Kosters, A., Yang, J., Sheth, S., Levy, T., et al. (2025) · doi:10.1101/2025.02.06.636961 · Cited by 3
Murphy, R., Villegas-Salmerón, J., Sanz-Rodriguez, A., Engel, T., Mooney, C., Henshall, D. C., & Jimenez-Mateos, E. M. (2025) · PLOS One · doi:10.1371/journal.pone.0326401 · Cited by 3
Gruner, N. G., Cullen, J. M., Crosby, J. M., DeCross, S. N., Mathes, B., Garner, L., Gironda, C., Hu, Y., Krompinger, J. W., Elias, J. A., et al. (2024) · Journal of Cognitive Psychotherapy · doi:10.1891/jcp-2022-0033
Metadata
- Authors
- Benjamini, Y., & Hochberg, Y.
- Affiliations
- Tel Aviv University (IL)
- ORCID
- Benjamini, Yoav
- Venue
- Journal of Educational and Behavioral Statistics
- Published
- 2000-03
- Volume
- 25
- Issue
- 1
- Pages
- 60-83
- ISSN
- 1076-9986 (ISSN-L)
- 1935-1054
- OpenAlex
- W2143079975
- Cited by
- 2,015
- References
- 2
- Citations by year
- 2026: 88 · 2025: 141 · 2024: 146 · 2023: 136 · 2022: 118 · 2021: 134 · 2020: 134 · 2019: 125 · 2018: 102 · 2017: 83 · 2016: 94 · 2015: 85 · 2014: 94 · 2013: 81 · 2012: 51
- FWCI
- 10.54
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Educational and Behavioral Statistics (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Inference
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Keywords
- false discovery rate, multiple comparisons, adaptive procedures, Benjamini-Hochberg procedure, stepwise procedures, statistical power, family-wise error rate, multicenter study, subgroup analysis, meta-analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review