CoChat
Cited by 225

The DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index.

A review of modern multiple hypothesis testing, with particular attention to the false discovery proportion

Farcomeni, Alessio · 2008 · Statistical Methods in Medical Research

Abstract

In the last decade a growing amount of statistical research has been devoted to multiple testing, motivated by a variety of applications in medicine, bioinformatics, genomics, brain imaging, etc. Research in this area is focused on developing powerful procedures even when the number of tests is very large. This paper attempts to review research in modern multiple hypothesis testing with particular attention to the false discovery proportion, loosely defined as the number of false rejections divided by the number of rejections. We review the main ideas, stepwise and augmentation procedures; and resampling based testing. We also discuss the problem of dependence among the test statistics. Simulations make a comparison between the procedures and with Bayesian methods. We illustrate the procedures in applications in DNA microarray data analysis. Finally, few possibilities for further research are highlighted.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1177/0962280206079046

Cites

23 of 23 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 118

  1. 10.1002/(SICI)1097-0193(1996)4:1<58::AID-HBM4>3.0.CO;2-O · doi:10.1002/(sici)1097-0193(1996)4:1<58::aid-hbm4>3.0.co;2-o
  2. 10.1016/S0925-4927(00)00051-2 · DOI
  3. 10.1016/S0896-6273(03)00393-3 · DOI · Paper page
  4. 10.1016/j.neuroimage.2003.12.047 · DOI
  5. 10.1002/(SICI)1098-2272(1997)14:6<779::AID-GEPI36>3.0.CO;2-L · DOI
  6. Weller JI, 1998, Genetics, 150, 1699, 10.1093/genetics/150.4.1699 · DOI · Paper page
  7. 10.1152/physiolgenomics.1999.1.3.165 · DOI · Paper page
  8. 10.3168/jds.S0022-0302(00)74868-5 · DOI
  9. Mosig MO, 2001, Genetics, 157, 1683, 10.1093/genetics/157.4.1683 · DOI · Paper page
  10. 10.1214/ss/1056397487 · DOI · Paper page
  11. 10.1214/ss/1056397486 · DOI
  12. 10.1016/S0003-4975(00)02350-X · DOI
  13. Schlaeppi M., 1996, British Journal of Clinical Practice, 50, 14
  14. 10.1093/oxfordjournals.aje.a009501 · DOI
  15. Vedantham K., 2001, Canadian bus drivers. Canadian Journal of Psychiatrics, 46, 149
  16. Schaffer CM, 1998, Journal of Market Research Sociology, 40, 155
  17. 10.1080/01621459.2000.10474336 · DOI · Paper page
  18. 10.1007/BF02295736 · DOI
  19. 10.1207/s15327906mbr3201_2 · DOI
  20. 10.1111/j.1467-9574.2006.00343.x · DOI
  21. 10.1016/0167-9473(96)00003-5 · DOI · Paper page
  22. 10.1214/009053606000000074 · DOI · Paper page
  23. Holm S., 1979, Scandinavian Journal of Statistics, 6, 65
  24. 10.1214/lnms/1196285632 · DOI
  25. 10.1080/00401706.1968.10490605 · DOI
  26. 10.2202/1544-6115.1042 · DOI · Paper page
  27. 10.1198/016214506000000339 · DOI
  28. 10.1002/gepi.1124 · DOI · Paper page
  29. 10.1111/1467-9868.00346 · DOI · Paper page
  30. Sarkar SK, 2005, Stepup procedures controlling generalized FWER and generalized FDR
  31. 10.1214/aos/1074290335 · DOI · Paper page
  32. 10.1111/1467-9469.00072 · DOI · Paper page
  33. 10.1093/biomet/93.3.509 · DOI · Paper page
  34. 10.1111/1467-9868.00347 · DOI · Paper page
  35. 10.1016/j.jspi.2003.06.019 · DOI
  36. Westfall PH, 1993, Resampling-based multiple testing: examples and methods for p-value adjustment
  37. 10.1007/978-1-4613-8122-8 · DOI · Paper page
  38. Diaconis P., 1985, Theories of data analysis: from magical thinking through classical statistics
  39. Ahmed SW Issues arising in the application of Bonferroni procedures in federal surveys . In 1991 ASA Proceedings of the Survey Research Methods Section , 1991. 344-49.
  40. 10.2202/1544-6115.1040 · DOI · Paper page
  41. 10.1214/aos/1015362191 · DOI
  42. Pesarin F., 2001, Multivariate permutation tests with applications to biostatistics
  43. 10.1007/978-1-4899-4541-9 · DOI · Paper page
  44. 10.1198/0003130042845 · DOI
  45. 10.1016/j.jspi.2003.07.019 · DOI
  46. 10.1007/BF02595811 · DOI · Paper page
  47. Farcomeni A., 2006, Scandinavian Journal of Statistics
  48. 10.1214/009053606000000425 · DOI · Paper page
  49. 10.1016/S0378-3758(99)00041-5 · DOI · Paper page
  50. 10.1093/acprof:oso/9780195035681.001.0001 · DOI · Paper page
  51. 10.1177/00928615980320S131 · DOI
  52. 10.1016/S1047-2797(98)00003-9 · DOI
  53. 10.1002/(SICI)1097-0258(20000330)19:6<767::AID-SIM518>3.0.CO;2-U · DOI
  54. 10.2307/2531158 · DOI · Paper page
  55. 10.1002/sim.4780141103 · DOI
  56. 10.2307/2533057 · DOI · Paper page
  57. 10.1080/01621459.1984.10478093 · DOI · Paper page
  58. 10.2307/2532142 · DOI · Paper page
  59. 10.1093/biomet/75.4.800 · DOI · Paper page
  60. 10.2202/1544-6115.1041 · DOI
  61. Sidak Z., 1967, Journal of the American Statistical Association, 62, 626
  62. 10.1214/aoms/1177693504 · DOI
  63. 10.1214/aos/1028144847 · DOI
  64. 10.1080/01621459.1992.10475188 · DOI · Paper page
  65. Seneta E., 1997, Theory of Stochastic Processes, 3, 393
  66. 10.1111/j.1751-5823.2005.tb00249.x · DOI
  67. 10.1111/j.1467-9868.2004.00439.x · DOI · Paper page
  68. 10.1214/009053604000000283 · DOI · Paper page
  69. 10.1214/aos/1015362192 · DOI · Paper page
  70. 10.1093/biomet/93.3.491 · DOI · Paper page
  71. Storey JD, 2001, false discovery rates under dependence, with applications to DNA microarrays
  72. 10.1093/biomet/69.3.493 · DOI · Paper page
  73. 10.1006/nimg.2001.0764 · DOI
  74. Swanepoel JWH., 1999, Annals of Statistics, 27, 24
  75. 10.1214/009053605000000741 · DOI · Paper page
  76. 10.1111/j.1467-9868.2005.00515.x · DOI · Paper page
  77. 10.1086/324109 · DOI
  78. 10.1111/j.1467-9868.2005.00509.x · DOI
  79. 10.2202/1544-6115.1143 · DOI
  80. R Development Core Team. R., 2004, A language and environment for statistical computing
  81. 10.1198/0162145000001655 · DOI
  82. 10.1007/978-1-4899-4473-3 · DOI · Paper page
  83. 10.1016/S0378-3758(99)00044-0 · DOI
  84. Berry DA Multiple comparisons, multiple tests, and data dredging: a Bayesian perspective . In Bernardo J, DeGroot M, Lindley D , Smith A, eds. Bayesian statistics. vol. 3. Oxford University Press, 1988: 79-94.
  85. 10.1080/00401706.1965.10490249 · DOI · Paper page
  86. 10.1016/S0378-3758(99)00042-7 · DOI
  87. Leibermann B., 1971, Contemporary problems in statistics
  88. 10.1214/aos/1018031111 · DOI
  89. 10.1037/1082-989X.7.3.356 · DOI
  90. 10.1007/978-1-4757-3071-5 · DOI · Paper page
  91. 10.1080/01621459.1995.10476592 · DOI · Paper page
  92. Berger JO, 1997, Statistical Science, 12, 133, 10.1214/ss/1030037904 · DOI · Paper page
  93. Jeffreys H., 1961, Theory of probability
  94. 10.1214/aos/1028144846 · DOI · Paper page
  95. 10.1080/01621459.1997.10473682 · DOI · Paper page
  96. 10.1214/aos/1176343846 · DOI
  97. van der Laan MJ, 2000, Biostatistics, 1, 1, 10.1093/biostatistics/1.1.1 · doi:10.1093/biostatistics/1.1.1
  98. Hommel G., 1983, Biomedical Journal, 25, 423
  99. 10.1007/s10260-006-0002-z · DOI
  100. Bickel DR, 2004, On `Strong control, conservative point estimation and simultaneous conservative consistency of false discovery rates': does a large number of tests obviate confidence intervals of the FDR?
  101. Neuhaus KL, 1992, JAmCollCard, 19, 885
  102. Amaratunga D., 2004, Exploration and analysis of DNA microarray and protein array data
  103. 10.1007/b97411 · DOI · Paper page
  104. 10.1038/4462 · DOI · Paper page
  105. 10.1038/4434 · DOI · Paper page
  106. Bolsover SR, 1997, From genes to cells
  107. Garret RH, 2002, Principles of byochemistry
  108. 10.1093/nar/29.12.2549 · DOI · Paper page
  109. 10.1093/bioinformatics/btg464 · DOI
  110. Zweiger G., 2001, Transducing the genome: information, anarchy and revolution in the biomedical sciences
  111. 10.1073/pnas.96.12.6745 · DOI · Paper page
  112. 10.1126/science.286.5439.531 · DOI · Paper page
  113. 10.1198/016214502753479248 · DOI · Paper page
  114. Cover T., 1967, IEEE Transactions on Information Theory, IT
  115. Schervish M., 1996, American Statistician, 50, 203
  116. Bayarri MJ, 2000, Journal of the American Statistical Association, 95, 1127
  117. Cabras S., 2004, Control of the false discovery rate with frequentist p-values in microarray data analysis
  118. 10.1198/016214504000001646 · DOI · Paper page

Cited by

See all 225

Metadata

Authors
Farcomeni, A.
Venue
Statistical Methods in Medical Research
Published
2007-08-14
Volume
17
Issue
4
Pages
347-388
ISSN
  • 0962-2802 (ISSN-L)
  • 1477-0334
OpenAlex
W2032136732
Cited by
225
References
23
FWCI
8.71
Type
journal article
Language
en
Open access
closed
Licence
journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.