Effects of dependence in high-dimensional multiple testing problems
Open accessAbstract
BACKGROUND: We consider effects of dependence among variables of high-dimensional data in multiple hypothesis testing problems, in particular the False Discovery Rate (FDR) control procedures. Recent simulation studies consider only simple correlation structures among variables, which is hardly inspired by real data features. Our aim is to systematically study effects of several network features like sparsity and correlation strength by imposing dependence structures among variables using random correlation matrices. RESULTS: We study the robustness against dependence of several FDR procedures that are popular in microarray studies, such as Benjamin-Hochberg FDR, Storey's q-value, SAM and resampling based FDR procedures. False Non-discovery Rates and estimates of the number of null hypotheses are computed from those methods and compared. Our simulation study shows that methods such as SAM and the q-value do not adequately control the FDR to the level claimed under dependence conditions. On the other hand, the adaptive Benjamini-Hochberg procedure seems to be most robust while remaining conservative. Finally, the estimates of the number of true null hypotheses under various dependence conditions are variable. CONCLUSION: We discuss a new method for efficient guided simulation of dependent data, which satisfy imposed network constraints as conditional independence structures. Our simulation set-up allows for a structural study of the effect of dependencies on multiple testing criterions and is useful for testing a potentially new method on pi0 or FDR estimation in a dependency context.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
FDR correction is appropriate and robust for biological data that inherently show dependencies.
“FDR correction was chosen as it has been shown to be appropriate and robust for biological data that inherently show dependencies, such as those in the present study [60, 61, 62].”
From Interpersonal Synchronization of Autonomic Physiology via Mere Visual Contact (2026) · cites this paper as [60]
The Abstract’s Results section reports that adaptive Benjamini–Hochberg was robust under dependence, supporting the choice of a particular FDR procedure. But it also reports that SAM and q-value did not adequately control FDR under dependence. The paper therefore does not support calling FDR correction generally robust for dependent biological data; the reported comparison is a simulation study. Page numbers were not provided in our copy. (LLM, conf=0.88)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482 · cited 3× · see citing works
“[2] first suggest an implementation of FDR by a linear step up approach.”
InfluentialSubramanian, A., Tamayo, P., Mootha, V. K., Mukherjee, S., Ebert, B. L., Gillette, M. A., Paulovich, A., Pomeroy, S. L., Golub, T. R., Lander, E. S., et al. (2005) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0506580102 · Cited by 53,625 · cited 3× · see citing works
“As a data-guided simulation using known gene sets [21,25], we introduce an algorithm for using real data in the Results section.”
Gentleman R: R: A language for data analysis and graphics
Ihaka, R., & Gentleman, R. (1996) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.1996.10474713 · Cited by 11,747 · cited 2× · see citing works
“Simulation program We developed R code [32] for this simulation studies.”
In silico prediction of protein-protein interactions in human macrophages
Souiai, O., Guerfali, F. Z., Miled, S. B., Brun, C., & Benkahla, A. (2014) · BMC Research Notes · doi:10.1186/1756-0500-7-157 · Cited by 10,785 · see citing works
Significance analysis of microarrays applied to the ionizing radiation response
Tusher, V. G., Tibshirani, R., & Chu, G. (2001) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.091062498 · Cited by 10,568 · see citing works
The control of the false discovery rate in multiple testing under dependency
Benjamini, Y., & Yekutieli, D. (2001) · The Annals of Statistics · doi:10.1214/aos/1013699998 · Cited by 10,490 · cited 6× · see citing works
“The reason behind these phenomena is hard to explain because the implementation of FDR-AME is modified from the algorithms of Yekutieli et al. [12].”
InfluentialStatistical significance for genomewide studies
Storey, J. D., & Tibshirani, R. (2003) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1530509100 · Cited by 9,692 · cited 1× · see citing works
“[19], respectively.”
A Direct Approach to False Discovery Rates
Storey, J. D. (2002) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/1467-9868.00346 · Cited by 5,603 · cited 6× · see citing works
“Storey [4] introduces the positive false discovery rate (pFDR) and the q -value. pFDR is known to control error rate under the clumpy dependency condition [5].”
InfluentialR: A Language for Data Analysis and Graphics
Ihaka, R., & Gentleman, R. C. (1996) · Journal of Computational and Graphical Statistics · doi:10.2307/1390807 · Cited by 4,245 · see citing works
Adaptive linear step-up procedures that control the false discovery rate
Benjamini, Y., Krieger, A. M., & Yekutieli, D. (2006) · Biometrika · doi:10.1093/biomet/93.3.491 · Cited by 3,228 · cited 3× · see citing works
“We found that in all our simulations, the simple adaptive Benjamini-Hochberg procedure [8] is most optimal under dependence, since it achieves relatively high power while remaining conservative.”
InfluentialLauritzen, S. L. (1996) · doi:10.1093/oso/9780198522195.001.0001 · Cited by 2,976 · see citing works
Resampling-Based Multiple Testing: Examples and Methods for p-Value Adjustment
Martín, M. A., Westfall, P. H., & Young, S. (1993) · Biometrics · doi:10.2307/2533464 · Cited by 2,308 · cited 1× · see citing works
“Resampling based approaches are immediate in utilizing sample correlation structure [11].”
The positive false discovery rate: a Bayesian interpretation and the q-value
Storey, J. D. (2003) · The Annals of Statistics · doi:10.1214/aos/1074290335 · Cited by 2,204 · cited 1× · see citing works
“Storey [4] introduces the positive false discovery rate (pFDR) and the q-value.”
Graphical Models in Applied Multivariate Statistics
Goodall, C. R., & Whittaker, J. (1991) · Technometrics · doi:10.2307/1269424 · Cited by 1,842 · see citing works
Identifying differentially expressed genes using false discovery rate controlling procedures
Reiner, A., Yekutieli, D., & Benjamini, Y. (2003) · Bioinformatics · doi:10.1093/bioinformatics/btf877 · Cited by 1,765 · see citing works
Multiple Hypothesis Testing in Microarray Experiments
Dudoit, S., Shaffer, J. P., & Boldrick, J. C. (2003) · Statistical Science · doi:10.1214/ss/1056397487 · Cited by 1,054 · cited 2× · see citing works
“SAM FDR is known to estimate the expected number of false discoveries over the observed number of total rejections under the complete null hypothesis [7].”
Resampling-Based Multiple Testing: Examples and Methods for p-Value Adjustment
Chaubey, Y. P., Westfall, P. H., & Young, S. S. (1993) · Technometrics · doi:10.2307/1270279 · Cited by 943 · see citing works
An empirical Bayes approach to inferring large-scale gene association networks
Schäfer, J., & Strimmer, K. (2004) · Bioinformatics · doi:10.1093/bioinformatics/bti062 · Cited by 786 · cited 1× · see citing works
“Nowadays, explaining high-dimensional data with conditional independence structures is quite active especially in microarray data analysis [1, 22-24 ].”
On testing the significance of sets of genes
Efron, B., & Tibshirani, R. J. (2007) · The Annals of Applied Statistics · doi:10.1214/07-aoas101 · Cited by 785 · cited 1× · see citing works
“As a data-guided simulation using known gene sets [21,25], we introduce an algorithm for using real data in the Results section.”
Yekutieli, D., & Benjamini, Y. (1999) · Journal of Statistical Planning and Inference · doi:10.1016/s0378-3758(99)00041-5 · Cited by 738 · see citing works
Wright, G. W. J., & Simon, R. M. (2003) · Bioinformatics · doi:10.1093/bioinformatics/btg345 · Cited by 662 · cited 1× · see citing works
“As an alternative to the fudge factor, the random variance model (RVM) by [26] can be used and simple replacement of the pooled variance of t-statistic by the RVM variance results in close control of the FDR to the nominal level under dependence in moderate to large sample size conditions.”
Operating Characteristics and Extensions of the False Discovery Rate Procedure
Genovese, C. R., & Wasserman, L. (2002) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/1467-9868.00347 · Cited by 545 · cited 1× · see citing works
“The FNR is introduced by Genovese et al. [17].”
Sparse graphical models for exploring gene expression data
Dobra, A., Hans, C., Jones, B., et al. (2004) · Journal of Multivariate Analysis · doi:10.1016/j.jmva.2004.02.009 · Cited by 462 · cited 1× · see citing works
“Nowadays, explaining high-dimensional data with conditional independence structures is quite active especially in microarray data analysis [1,22-24].”
Correlation and Large-Scale Simultaneous Significance Testing
Efron, B. (2007) · Journal of the American Statistical Association · doi:10.1198/016214506000001211 · Cited by 405 · cited 3× · see citing works
“The variance of pairwise correlations was previously described as an important parameter in FDR estimation [20].”
InfluentialSparse graphical Gaussian modeling of the isoprenoid gene network in Arabidopsis thaliana
Wille, A., Philip Zimmermann, B., Vranová, E., Fürholz, A., Laule, O., Stefan Bleuler, A., Hennig, L., Prelić, A., von Rohr, P., Thiele, L., et al. (2004) · Genome biology · doi:10.1186/gb-2004-5-11-r92 · Cited by 380 · see citing works
Estimating the Proportion of True Null Hypotheses, with application to DNA Microarray Data
Langaas, M., Lindqvist, B. H., & Ferkingstad, E. (2005) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.1467-9868.2005.00515.x · Cited by 316 · cited 4× · see citing works
“Careful simulation studies on the comparison of performance of π 0 estimation methods are done by Black [9] and Langaas et al. [10].”
InfluentialWagner, G. P. (1984) · Journal of Mathematical Biology · doi:10.1007/bf00275224 · Cited by 269 · cited 1× · see citing works
“A family of covariance matrices as a Gaussian ensemble can also be considered as described in [28].”
Controlling the number of false discoveries: application to high-dimensional genomic data
Korn, E. L., Troendle, J. F., McShane, L. M., & Simon, R. M. (2003) · Journal of Statistical Planning and Inference · doi:10.1016/s0378-3758(03)00211-8 · Cited by 248 · cited 2× · see citing works
“Typically in microarray data analysis, equicorrelated normal structures such as single pairwise correlation matrices or block diagonal matrices with a single pairwise correlation in each block are considered [14,15].”
Variance of the Number of False Discoveries
Owen, A. B. (2005) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.1467-9868.2005.00509.x · Cited by 142 · cited 2× · see citing works
“Other interesting works on the effects of dependence on the number of false discoveries rather than FDR are Owen [27] and Korn et al.”
Generating Correlation Matrices
Marsaglia, G., & Olkin, I. (1984) · SIAM Journal on Scientific and Statistical Computing · doi:10.1137/0905034 · Cited by 115 · cited 1× · see citing works
“Generating constrained random correlation matrices Unconstrained random correlation matrices are generated simply by products of matrix transposes and its standardizations [31].”
Qiu, X., Klebanov, L. B., & Yakovlev, A. (2005) · Statistical Applications in Genetics and Molecular Biology · doi:10.2202/1544-6115.1157 · Cited by 108 · cited 2× · see citing works
“[14] investigate the vulnerability of application of empirical Bayes theory under strong correlations.”
Covariance decomposition in undirected Gaussian graphical models
Jones, B., & West, M. (2005) · Biometrika · doi:10.1093/biomet/92.4.779 · Cited by 79 · cited 1× · see citing works
“Nowadays, explaining high-dimensional data with conditional independence structures is quite active especially in microarray data analysis [1,22-24].”
A Note on the Adaptive Control of False Discovery Rates
Black, M. A. (2004) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.1369-7412.2003.05527.x · Cited by 69 · cited 4× · see citing works
“Careful simulation studies on the comparison of performance of π 0 estimation methods are done by Black [9] and Langaas et al. [10].”
InfluentialHow accurately can we control the FDR in analyzing microarray data?
Jung, S., & Jang, W. (2006) · Bioinformatics · doi:10.1093/bioinformatics/btl161 · Cited by 29 · cited 1× · see citing works
“[16] who simulate multivariate normal test staFDR ≤ = ≤ m m 0 0 γ π γ γ .”
Statistical Applications in Genetics and Molecular Biology
Qiu, X., Klebanov, L. B., & Yakovlev, A. (2005) · Cited by 4 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Overexpression of lncRNA H19 enhances carcinogenesis and metastasis of gastric cancer
Li, H., Yu, B., Li, J., Su, L., Yan, M., Zhu, Z., & Liu, B. (2014) · Oncotarget · doi:10.18632/oncotarget.1913 · Cited by 509 · see citing works
“01) in the functional annotation categories based on genes identified in the microarray [21-22].”
BackgroundLarge-scale GWAS reveals insights into the genetic architecture of same-sex sexual behavior
Ganna, A., Verweij, K. J. H., Nivard, M. G., Maier, R., Wedow, R., Busch, A. S., Abdellaoui, A., Guo, S., Sathirapongsasuti, J. F., 23andMe Research Team, et al. (2019) · Science · doi:10.1126/science.aat7693 · Cited by 406 · see citing works
Cytokine signature associated with disease severity in chronic fatigue syndrome patients
Montoya, J. G., Holmes, T. H., Anderson, J. N., Maecker, H. T., Rosenberg‐Hasson, Y., Valencia, I. J., Chu, L., Younger, J., Tato, C. M., & Davis, M. M. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1710519114 · Cited by 397 · see citing works
“Therefore, nonspecific binding was included as a covariate to prevent introduction of bias (60).”
BackgroundMethodMultiple hypothesis testing in genomics
Goeman, J., & Solari, A. (2014) · Statistics in Medicine · doi:10.1002/sim.6082 · Cited by 377 · see citing works
“FDR control for the adaptive Benjamini, Krieger and Yekutieli procedure has only yet been proven under independence, although simulations suggest FDR control under positive dependence as well [31, 57, 58].”
BackgroundMethodInfluentialAcoustic Enhancement of Sleep Slow Oscillations and Concomitant Memory Improvement in Older Adults
Papalambros, N. A., Santostasi, G., Malkani, R. G., Braun, R., Weintraub, S., Paller, K. A., & Zee, P. C. (2017) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2017.00109 · Cited by 259 · see citing works
Mass univariate analysis of event‐related brain potentials/fields II: Simulation studies
Groppe, D. M., Urbach, T. P., & Marta Kutas, A. (2011) · Psychophysiology · doi:10.1111/j.1469-8986.2011.01272.x · Cited by 214 · see citing works
“This result is consistent with applications of the BKY procedure to simulated data with positive correlations (Benjamini, Krieger, & Yekutieli, 2006) and of the BKYand BH procedures to simulated data with positive and negative correlations (Kim & van de Wiel, 2008).”
Eirin, A., Zhu, X.-Y., Puranik, A. S., Woollard, J. R., Tang, H., Dasari, S., Lerman, A., van Wijnen, A. J., & Lerman, L. O. (2016) · Scientific Reports · doi:10.1038/srep36120 · Cited by 156 · see citing works
A Factor Model Approach to Multiple Testing Under Dependence
Friguet, C., Kloareg, M., & Causeur, D. (2009) · Journal of the American Statistical Association · doi:10.1198/jasa.2009.tm08332 · Cited by 155 · see citing works
Hollmén, M., Roudnicky, F., Karaman, S., & Detmar, M. J. (2015) · Scientific Reports · doi:10.1038/srep09188 · Cited by 147 · see citing works
Bo, T., Wen, J., Zhao, Y.-C., Tian, S.-J., Zhang, X., & Wang, D. (2020) · The Journal of Steroid Biochemistry and Molecular Biology · doi:10.1016/j.jsbmb.2020.105602 · Cited by 132 · see citing works
Steuer, A. E., Brockbals, L., & Kræmer, T. (2019) · Frontiers in Chemistry · doi:10.3389/fchem.2019.00319 · Cited by 125 · see citing works
Yin, P., & Xu, G. (2014) · Journal of Chromatography A · doi:10.1016/j.chroma.2014.11.050 · Cited by 121 · see citing works
Bo, T., Zhang, X., Kohl, K. D., Wen, J., Tian, S.-J., & Wang, D. (2020) · The ISME Journal · doi:10.1038/s41396-020-0711-6 · Cited by 114 · see citing works
“Significance values were corrected for multiple measurements using the Benjamin–Hochberg FDR method [40].”
Lun, A. T. L., & Smyth, G. K. (2014) · Nucleic Acids Research · doi:10.1093/nar/gku351 · Cited by 113 · see citing works
“However, it is known to be robust to correlations (31,32) which is consistent with the conservativeness in these results.”
ResultPika Population Density Is Associated with the Composition and Diversity of Gut Microbiota
Li, H., Qu, J., Li, T., Li, J., Lin, Q., & Li, X. (2016) · Frontiers in Microbiology · doi:10.3389/fmicb.2016.00758 · Cited by 107 · see citing works
Early Age-Related Functional Connectivity Decline in High-Order Cognitive Networks
Siman-Tov, T., Bosak, N., Sprecher, E., Paz, R., Eran, A., Aharon‐Peretz, J., & Kahn, I. (2017) · Frontiers in Aging Neuroscience · doi:10.3389/fnagi.2016.00330 · Cited by 108 · see citing works
Yi, Z., Jin, Y., Xiao, Y., Chen, L., Tan, L., Du, A., He, K., Liu, D., Luo, H., Fang, Y., et al. (2019) · Frontiers in Microbiology · doi:10.3389/fmicb.2019.00472 · Cited by 108 · see citing works
Eirin, A., Zhu, X.-Y., Puranik, A. S., Woollard, J. R., Tang, H., Dasari, S., Lerman, A., van Wijnen, A. J., & Lerman, L. O. (2017) · PLoS ONE · doi:10.1371/journal.pone.0174303 · Cited by 99 · see citing works
“Data was normalized by protein loading, and differential p-values FDR-corrected using the Benjamini-Hochberg-Yekutieli procedure [25].”
Involvement of prefrontal cortex in scalar implicatures: evidence from magnetoencephalography
Politzer‐Ahles, S., & Gwilliams, L. (2015) · Language Cognition and Neuroscience · doi:10.1080/23273798.2015.1027235 · Cited by 93 · see citing works
Systemic proteomics and miRNA profile analysis of exosomes derived from human pluripotent stem cells
Bi, Y., Qiao, X., Liu, Q., Song, S., Zhu, K., Qiu, X., Zhang, X., jia, C., Wang, H., Yang, Z., et al. (2022) · Stem Cell Research & Therapy · doi:10.1186/s13287-022-03142-1 · Cited by 94 · see citing works
“Differentially expressed proteins (DEPs) were considered valid after the data were normalized by protein loading and differential p-value false discovery rates (FDR) corrected [30].”
MethodWard, J., Lyall, L. M., Bethlehem, R. A. I., Ferguson, A., Strawbridge, R. J., Lyall, D. M., Cullen, B., Graham, N., Johnston, K. J. A., Bailey, M. E. S., et al. (2019) · Translational Psychiatry · doi:10.1038/s41398-019-0635-y · Cited by 83 · see citing works
Improved annotation of the domestic pig genome through integration of Iso-Seq and RNA-seq data
Beiki, H., Liu, H., Huang, J., Manchanda, N., Nonneman, D., Smith, T. P. L., Reecy, J. M., & Tuggle, C. K. (2019) · BMC Genomics · doi:10.1186/s12864-019-5709-y · Cited by 79 · see citing works
Kikugawa, S., Nishikata, K., Murakami, K., Sato, Y., Suzuki, M., Altaf‐Ul‐Amin, M., Kanaya, S., & Imanishi, T. (2012) · BMC Systems Biology · doi:10.1186/1752-0509-6-s2-s7 · Cited by 76 · see citing works
“FDR, the expected proportion of incorrectly rejected null hypotheses, is a widely used statistic for multiple testing [38].”
MethodBrokordt, K., Defranchi, Y., Espósito, I., et al. (2019) · Frontiers in Physiology · doi:10.3389/fphys.2019.00077 · Cited by 75 · see citing works
Carvajal‐Rodríguez, A., & de Uña‐Álvarez, J. (2011) · PLoS ONE · doi:10.1371/journal.pone.0024700 · Cited by 75 · see citing works
“However, in some cases high correlations could be found in the data and this could have great impact in the p-value distribution and consequently in the correction methods [16–17].”
MethodA method for generating realistic correlation matrices
Hardin, J., Garcia, S. R., & Golan, D. (2013) · The Annals of Applied Statistics · doi:10.1214/13-aoas638 · Cited by 75 · see citing works
Role of Cytokinin, Strigolactone, and Auxin Export on Outgrowth of Axillary Buds in Apple
Tan, M., Li, G., Chen, X., Xing, L., Ma, J., Zhang, D., Ge, H., Han, M., Sha, G., & An, N. (2019) · Frontiers in Plant Science · doi:10.3389/fpls.2019.00616 · Cited by 73 · see citing works
Jiang, S., Wen, N., Li, Z., Dube, U., Del Aguila, J., Budde, J., Martinez, R., Hsu, S., Fernandez, M. V., Cairns, N. J., et al. (2018) · Translational Psychiatry · doi:10.1038/s41398-018-0319-z · Cited by 73 · see citing works
Yen, A., & Kellis, M. (2015) · Nature Communications · doi:10.1038/ncomms8973 · Cited by 67 · see citing works
“…on one another due to the connection between each gene and its 15 features, Benjamini–Hochberg FDR correction is always valid for dependent tests that uphold positive regression dependency in a subset, and Benjamini–Hochberg also performs well in many practical cases and simulations 60–64 .”
BackgroundChung, M. K., Louis, G. M. B., Kannan, K., & Patel, C. J. (2018) · Environment International · doi:10.1016/j.envint.2018.11.037 · Cited by 64 · see citing works
“Alternatively, FDR controlling procedures are generally more powerful than those for FWER, but at the sake of a higher type I error rate [55,56].”
In vitro assessment of hepatotoxicity by metabolomics: a review
Cuykx, M., Rodrigues, R. M., Laukens, K., Vanhaecke, T., & Covaci, A. (2018) · Archives of Toxicology · doi:10.1007/s00204-018-2286-9 · Cited by 63 · see citing works
“The outcome and interpretation of this statistical approach is very clear, but the implementation of univariate tests for hundreds of variables inflates the risk of false positive features (Broadhurst and Kell 2006; Kim and Wiel 2008).”
BackgroundMethodFlowFP: A Bioconductor Package for Fingerprinting Flow Cytometric Data
Rogers, W. T., & Holyst, H. A. (2009) · Advances in Bioinformatics · doi:10.1155/2009/193947 · Cited by 61 · see citing works
Wang, C., Zhang, H., Wang, J., Chen, S., Wang, Z., Zhao, L., & Wang, X. (2020) · Microbiological Research · doi:10.1016/j.micres.2020.126527 · Cited by 62 · see citing works
Optimal weighting for false discovery rate control
Roquain, É., & van de Wiel, M. A. (2009) · Electronic Journal of Statistics · doi:10.1214/09-ejs430 · Cited by 60 · see citing works
Eirin, A., Meng, Y., Zhu, X.-Y., Li, Y., Saadiq, I. M., Jordan, K. L., Tang, H. R., Lerman, A., van Wijnen, A. J., & Lerman, L. O. (2021) · Frontiers in Cell and Developmental Biology · doi:10.3389/fcell.2021.660851 · Cited by 55 · see citing works
“Differential p-values were FDR-corrected using the Benjamini-Hochberg-Yekutieli procedure (Kim and van de Wiel, 2008).”
MethodSun, H., Liu, Y., Gai, Y., et al. (2015) · BMC Genomics · doi:10.1186/s12864-015-1842-4 · Cited by 55 · see citing works
“In our analysis, an estimated absolute value of log2-fold change of ≥2 and FDR adjusted P-value ≤ 0.001 were used as the threshold to judge the significance of DEGs.”
MethodInfluentialAlshabi, A. M., Vastrad, B. M., Shaikh, I. A., & Vastrad, C. M. (2019) · Biomolecules · doi:10.3390/biom9050201 · Cited by 54 · see citing works
“Multiple testing corrections were executed with the Benjamini-Hochberg method [20] to gain the adjusted p-value.”
MethodZhao, D., Li, Y., Liu, T., Voon, V., & Yuan, T. (2020) · Frontiers in Neuroscience · doi:10.3389/fnins.2020.00208 · Cited by 53 · see citing works
Correa, S., Meany, E. L., Gale, E. C., Klich, J. H., Saouaf, O. M., Mayer, A. T., Xiao, Z., Liong, C. S., Brown, R. A., Maikawa, C. L., et al. (2022) · Advanced Science · doi:10.1002/advs.202103677 · Cited by 48 · see citing works
The Plant Ontology Facilitates Comparisons of Plant Development Stages Across Species
Walls, R., Cooper, L., Elser, J. L., Gandolfo, M. A., Mungall, C., Smith, B., Stevenson, D. W., & Jaiswal, P. (2019) · Frontiers in Plant Science · doi:10.3389/fpls.2019.00631 · Cited by 51 · see citing works
Feng, W., Li, Y., Dao, P., Aburas, J., Islam, P., Elbaz, B., Kolarzyk, A., Brown, A. E., & Kratsios, P. (2020) · eLife · doi:10.7554/elife.50065 · Cited by 49 · see citing works
Xu, J., Xu, X., Shakeel, M., et al. (2017) · Frontiers in Microbiology · doi:10.3389/fmicb.2017.01421 · Cited by 49 · see citing works
Liu, H., Wang, R., Mao, B., et al. (2019) · BMC Genomics · doi:10.1186/s12864-019-5442-6 · Cited by 49 · see citing works
Test–Retest Reliability of the Effects of Continuous Theta-Burst Stimulation
Jannati, A., Fried, P. J., Block, G., Oberman, L. M., Rotenberg, A., & Pascual‐Leone, Á. (2019) · Frontiers in Neuroscience · doi:10.3389/fnins.2019.00447 · Cited by 44 · see citing works
Liao, X., Cai, B., Tian, B., et al. (2019) · Journal of Cellular and Molecular Medicine · doi:10.1111/jcmm.14328 · Cited by 46 · see citing works
“Correlation thresholding based on Benjamini‐Hochberg adjusted P‐values was show in Figure S1B.”
MethodEirin, A., Ferguson, C. M., Zhu, X.-Y., Saadiq, I. M., Tang, H., Lerman, A., & Lerman, L. O. (2020) · Stem Cell Research · doi:10.1016/j.scr.2020.101877 · Cited by 45 · see citing works
“…al., 2014) and protein group intensities of each sample log2 transformed, normalized, and modeled using a Gaussian-linked generalized linear model (Kim and van de Wiel, 2008). mRNAs and proteins with fold-change (MetS-EVs/Lean-EVs) > 1.4 and p < 0.05 (Lean-EVs vs. MetS-EVs) were classified as…”
InfluentialA Tutorial on Hunting Statistical Significance by Chasing N
Szűcs, D. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2016.01444 · Cited by 45 · see citing works
Sun, Y., Mu, C., Chen, Y., Kong, X., Xu, Y., Zheng, H., Zhang, H., Wang, Q., Xue, Y., Li, Z., et al. (2016) · Plant Physiology and Biochemistry · doi:10.1016/j.plaphy.2016.10.017 · Cited by 45 · see citing works
Urine Organic Acids as Potential Biomarkers for Autism-Spectrum Disorder in Chinese Children
Chen, Q., Qiao, Y., Xu, X., You, X., & Tao, Y. (2019) · Frontiers in Cellular Neuroscience · doi:10.3389/fncel.2019.00150 · Cited by 44 · see citing works
Nelson, S. K., Ariizumi, T., & Steber, C. M. (2017) · Frontiers in Plant Science · doi:10.3389/fpls.2017.02158 · Cited by 44 · see citing works
Eirin, A., Zhu, X.-Y., Woollard, J. R., Tang, H. R., Dasari, S., Lerman, A., & Lerman, L. O. (2019) · Stem Cells Translational Medicine · doi:10.1002/sctm.18-0171 · Cited by 43 · see citing works
Conditional calibration for false discovery rate control under dependence
Fithian, W., & Lei, L. (2022) · The Annals of Statistics · doi:10.1214/21-aos2137 · Cited by 43 · see citing works
“This “off-label” use of BH owes in part to a widely held belief that, under dependence typically arising in practice, BH is more often conservative than it is anti-conservative (e.g. Farcomeni, 2006; Kim and van de Wiel, 2008).”
BackgroundLi, D., Wang, X., Fu, Y., Zhang, C., Cao, Y., Wang, J., Zhang, Y., Li, Y., Chen, Y., Li, Z., et al. (2019) · Frontiers in Genetics · doi:10.3389/fgene.2019.00457 · Cited by 41 · see citing works
“The threshold P-value was determined by controlling the false discovery rate (FDR), and the corrected p-value was the q-value ( Kim and van de Wiel, 2008).”
Zhang, S., Yuan, C.-J., Chen, Y., et al. (2016) · Frontiers in Microbiology · doi:10.3389/fmicb.2016.01476 · Cited by 41 · see citing works
Pernas, L. F., Ramirez, R., Holmes, T. H., et al. (2014) · The Journal of Infectious Diseases · doi:10.1093/infdis/jiu189 · Cited by 40 · see citing works
“All P-value adjustments were via an adaptive 2-stage linear step-up procedure [30] for control of false discovery rate (FDR), a method that has been”
MethodBrain Responses to Emotional Faces in Natural Settings: A Wireless Mobile EEG Recording Study
Soto, V., Tyson‐Carr, J., Kokmotou, K., et al. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.02003 · Cited by 40 · see citing works
Wang, H., Fang, Y., Wang, L., et al. (2014) · Molecular Biology Reports · doi:10.1007/s11033-014-3481-2 · Cited by 40 · see citing works
Lee, B., Jones, E. K., Manohar, M., Li, L., Yadav, D. K., Conwell, D. L., Hart, P. A., Vege, S. S., Fogel, E. L., Serrano, J., et al. (2023) · Gastroenterology · doi:10.1053/j.gastro.2023.03.236 · Cited by 41 · see citing works
Sood, U., Hira, P., Kumar, R., Bajaj, A., Rao, D. L. N., Lal, R., & Shakarad, M. (2019) · Frontiers in Microbiology · doi:10.3389/fmicb.2019.00053 · Cited by 39 · see citing works
Epigenetic and immunological indicators of IPEX disease in subjects with FOXP3 gene mutation
Narula, M., Lakshmanan, U., Borna, Š., Schulze, J., Holmes, T. H., Harre, N., Kirkey, M., Ramachandran, A., Tagi, V. M., Barzaghi, F., et al. (2022) · Journal of Allergy and Clinical Immunology · doi:10.1016/j.jaci.2022.09.013 · Cited by 39 · see citing works
Energy Balance, the PI3K-AKT-mTOR Pathway Genes, and the Risk of Bladder Cancer
Lin, J. M., Wang, J., Greisinger, A. J., Grossman, H. B., Forman, M. R., Dinney, C. P., Hawk, E. T., & Wu, X. (2010) · Cancer Prevention Research · doi:10.1158/1940-6207.capr-09-0263 · Cited by 38 · see citing works
Yang, L.-G., Wang, Y., Wang, Y., et al. (2021) · Journal of Invertebrate Pathology · doi:10.1016/j.jip.2021.107665 · Cited by 38 · see citing works
Further results on controlling the false discovery proportion
Guo, W., He, L., & Sarkar, S. K. (2014) · The Annals of Statistics · doi:10.1214/14-aos1214 · Cited by 37 · see citing works
Moneghetti, K. J., Skhiri, M., Contrepois, K., Kobayashi, Y., Maecker, H. T., Davis, M. M., Snyder, M. P., Haddad, F., & Montoya, J. G. (2018) · Scientific Reports · doi:10.1038/s41598-018-20941-w · Cited by 36 · see citing works
Adrenal vein sampling: Substantial need for technical improvement at regional referral centres
Elliott, P., & Holmes, D. T. (2013) · Clinical Biochemistry · doi:10.1016/j.clinbiochem.2013.04.004 · Cited by 36 · see citing works
Hemerik, J., & Goeman, J. J. (2017) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/rssb.12238 · Cited by 35 · see citing works
Li, S., Xu, X., Shakeel, M., et al. (2018) · Frontiers in Physiology · doi:10.3389/fphys.2018.01478 · Cited by 35 · see citing works
Pawar, A. S., Eirin, A., Krier, J. D., Woollard, J. R., Zhu, X.-Y., Lerman, A., van Wijnen, A. J., & Lerman, L. O. (2019) · Stem Cell Research · doi:10.1016/j.scr.2019.101423 · Cited by 35 · see citing works
“Data was normalized by protein loading, and differential p-values FDR-corrected (32) .”
MethodDong, H., Ye, Y., Guo, Y., et al. (2020) · BMC Genetics · doi:10.1186/s12863-020-00926-3 · Cited by 35 · see citing works
“Genes with FDR ≤ 0.05 and fold-change ≥1 were considered as differentially expressed compared with the control.”
MethodInfluentialGenome-Wide Analyses Reveal Genes Subject to Positive Selection in Pasteurella multocida
Cao, P., Guo, D., Liu, J., Jiang, Q., Xu, Z., & Qu, L. (2017) · Frontiers in Microbiology · doi:10.3389/fmicb.2017.00961 · Cited by 33 · see citing works
Acaricidal Mechanism of Scopoletin Against Tetranychus cinnabarinus
Zhou, H., Zhang, Y., Lai, T. F., Liu, X., Guo, F.-Y., Guo, T., & Ding, W. (2019) · Frontiers in Physiology · doi:10.3389/fphys.2019.00164 · Cited by 32 · see citing works
Zhao, J., Fang, Y., Duan, C., et al. (2016) · Scientific Reports · doi:10.1038/srep36247 · Cited by 31 · see citing works
Empirical pathway analysis, without permutation
Zhou, Y., Barry, W. T., & Wright, F. A. (2013) · Biostatistics · doi:10.1093/biostatistics/kxt004 · Cited by 29 · see citing works
Vari, H. K., Roslund, M. I., Oikarinen, S., Nurminen, N., Puhakka, R., Parajuli, A., Grönroos, M., Siter, N., Laitinen, O. H., Hyöty, H., et al. (2020) · Chemosphere · doi:10.1016/j.chemosphere.2020.128965 · Cited by 29 · see citing works
Phenotypic and genetic associations between anhedonia and brain structure in UK Biobank
Zhu, X., Ward, J., Cullen, B., Lyall, D. M., Strawbridge, R. J., Lyall, L. M., & Smıth, D. J. (2021) · Translational Psychiatry · doi:10.1038/s41398-021-01522-4 · Cited by 28 · see citing works
Ou, B. S., Baillet, J., Interrante, M. V. F., Adamska, J., Zhou, X.-T., Saouaf, O. M., Yan, J., Klich, J., Jons, C., Meany, E. L., et al. (2024) · Science Advances · doi:10.1126/sciadv.adn7187 · Cited by 24 · see citing works
Temporal stability of bacterial symbionts in a temperate ascidian
López‐Legentil, S., Turón, X., Espluga, R., & Erwin, P. M. (2015) · Frontiers in Microbiology · doi:10.3389/fmicb.2015.01022 · Cited by 28 · see citing works
New procedures controlling the false discovery proportion via Romano–Wolf's heuristic
Delattre, S., & Roquain, É. (2015) · HAL · Cited by 28 · see citing works
Tang, L., Mo, J., Guo, T., Huang, S., Li, Q., Ning, P., & Hsiang, T. (2019) · World Journal of Microbiology and Biotechnology · doi:10.1007/s11274-019-2781-z · Cited by 28 · see citing works
Mitro, S. D., Liu, J., Jaacks, L. M., Fleisch, A. F., Williams, P. L., Knowler, W. C., Laferrère, B., Perng, W., Bray, G. A., Wallia, A., et al. (2020) · International Journal of Hygiene and Environmental Health · doi:10.1016/j.ijheh.2020.113680 · Cited by 27 · see citing works
Wang, J., Lippman, S. M., Lee, J. J., et al. (2010) · Carcinogenesis · doi:10.1093/carcin/bgq138 · Cited by 27 · see citing works
Intra‐ and interregional coregulation of opioid genes: broken symmetry in spinal circuits
Kononenko, O., Galatenko, V., Andersson, M., Bazov, I., Watanabe, H., Zhou, X. W., Iatsyshyna, A. P., Mityakina, I., Yakovleva, T., Sarkisyan, D., et al. (2017) · The FASEB Journal · doi:10.1096/fj.201601039r · Cited by 27 · see citing works
Gene Expression Profile of Bombyx mori Hemocyte under the Stress of Destruxin A
Gong, L., Chen, X., Liu, C., Jin, F., & Hu, Q. (2014) · PLoS ONE · doi:10.1371/journal.pone.0096170 · Cited by 26 · see citing works
“The false discovery rate (FDR) method was used to determine the P-value threshold in multiple testing [52].”
MethodMultiple comparisons in mass-spectrometry-based -omics technologies
Franceschi, P., Giordan, M., & Wehrens, R. (2013) · TrAC Trends in Analytical Chemistry · doi:10.1016/j.trac.2013.04.011 · Cited by 25 · see citing works
Caveolin-2 is regulated by BRD4 and contributes to cell growth in pancreatic cancer
Jiao, F., Han, T., Yuan, C., et al. (2020) · Cancer Cell International · doi:10.1186/s12935-020-1135-0 · Cited by 25 · see citing works
Li, M., Guo, Y. F., Feng, Y., & Zhang, N. (2019) · Frontiers in Genetics · doi:10.3389/fgene.2019.00180 · Cited by 23 · see citing works
Shan, C., Wang, C., Zhang, S., et al. (2020) · BMC Genomics · doi:10.1186/s12864-020-6454-y · Cited by 23 · see citing works
Limited evidence for reliability of low and high frequency rTMS over the motor cortex.
Prei, K., Kanig, C., Osnabrügge, M., Langguth, B., Mack, W., Abdelnaim, M., Schecklmann, M., & Schoisswohl, S. (2023) · Brain Research · doi:10.1016/j.brainres.2023.148534 · Cited by 21 · see citing works
Long, D., Zhu, L., Du, F.-L., et al. (2019) · BMC Genomics · doi:10.1186/s12864-019-5705-2 · Cited by 23 · see citing works
Pernaci, M., Mita, S. D., Andrieux, A., et al. (2014) · Frontiers in Plant Science · doi:10.3389/fpls.2014.00454 · Cited by 22 · see citing works
Ye, C., Xu, S., Hu, Q., Min-qiang, H., Zhou, L., Qin, X., Jia, J., & Hu, G. (2020) · Comparative Biochemistry and Physiology Part D Genomics and Proteomics · doi:10.1016/j.cbd.2019.100653 · Cited by 22 · see citing works
Wei, M., Liao, Y., Liu, J., Li, L., Huang, G., Huang, J., Li, D., Xiao, L., & Zhang, Z. (2020) · Journal of Clinical Neurophysiology · doi:10.1097/wnp.0000000000000758 · Cited by 22 · see citing works
HLA-disease association and pleiotropy landscape in over 235,000 Finns
Ritari, J., Koskela, S., Hyvärinen, K., et al. (2022) · Human Immunology · doi:10.1016/j.humimm.2022.02.003 · Cited by 22 · see citing works
Zhao, L., Shan, C., Shan, T., et al. (2022) · Food Research International · doi:10.1016/j.foodres.2022.111375 · Cited by 22 · see citing works
Li, Z., & Liu, J.-Z. (2017) · Frontiers in Microbiology · doi:10.3389/fmicb.2017.01987 · Cited by 22 · see citing works
Jia, J., Liu, X., Li, L., Lei, C., Dong, Y., Wu, G., & Hu, G. (2018) · Frontiers in Physiology · doi:10.3389/fphys.2018.00812 · Cited by 22 · see citing works
Luan, Y., Zuo, W., Li, C., et al. (2018) · International Journal of Molecular Sciences · doi:10.3390/ijms19123718 · Cited by 22 · see citing works
A Critical Review of Proteomic Studies in Gestational Diabetes Mellitus
Zhou, T., Huang, L., Wang, M., Chen, D., Chen, Z., & Jiang, S. (2020) · Journal of Diabetes Research · doi:10.1155/2020/6450352 · Cited by 22 · see citing works
Zhao, L., Zhang, S., Shan, C., et al. (2021) · Gene · doi:10.1016/j.gene.2021.145713 · Cited by 21 · see citing works
Steuer, A. E., Brockbals, L., & Kræmer, T. (2021) · Wiley Interdisciplinary Reviews Forensic Science · doi:10.1002/wfs2.1449 · Cited by 21 · see citing works
“…the need for multiple repetitions for hundreds of variables in metabolome investigations leads to an increased risk for detection of false-positive features, which should be taken into account by application of false discovery (FDR) corrections (Broadhurst & Kell, 2006; Kim & van de Wiel, 2008).”
Background
- Authors
- Kim, K. I., & van de Wiel, M. A.
- Affiliations
- Eindhoven University of Technology (NL)
- Corresponding author
- Kim, Kyung In
- Venue
- BMC Bioinformatics
- Published
- 2008-02-25
- Volume
- 9
- Issue
- 1
- Pages
- 114-114
- ISSN
- 1471-2105 (ISSN-L)
- PMC
- PMC2375137
- OpenAlex
- W2116319049
- Cited by
- 220
- References
- 34
- Citations by year
- 2026: 3 · 2025: 5 · 2024: 6 · 2023: 15 · 2022: 11 · 2021: 13 · 2020: 29 · 2019: 33 · 2018: 15 · 2017: 19 · 2016: 14 · 2015: 10 · 2014: 14 · 2013: 5 · 2012: 5
- FWCI
- 7.24
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- BMC Bioinformatics (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Amsterdam UMC (VU Amsterdam) - Institutional Repository (publishedVersion, other-oa)
- TU/e Research Portal (publishedVersion, other-oa)
- Pure Amsterdam UMC (submittedVersion, other-oa)
- TU/e Research Portal (submittedVersion, other-oa)
- TU/e Research Portal (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- VU Research Portal (submittedVersion, other-oa)
- Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene expression and cancer classification
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Inference
- Keywords
- multiple comparisons, false discovery rate, high-dimensional data, microarray data, conditional independence structure, sparsity
- MeSH terms
- Algorithms, Sensitivity and Specificity, Statistics as Topic, Reproducibility of Results, Artifacts, Oligonucleotide Array Sequence Analysis, Gene Expression Profiling
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
