The DOI matches its Crossref record; 94 of the 94 references it cites were found in a registry or an index.
Multiple Comparisons Using R
Abstract
Adopting a unifying theme based on maximum statistics, Multiple Comparisons Using R describes the common underlying theory of multiple comparison procedures through numerous examples. It also presents a detailed description of available software implementations in R. The R packages and source code for the analyses are available at https://CRAN.R-pro
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“After applying the Bonferroni correction for evaluating multiple probabilities (Bretz et al. 2010), only the projections of areas 7A and DP to area 10 were inconsistent despite a statistically very low probability of being so.”
From A Weighted and Directed Interareal Connectivity Matrix for Macaque Cerebral Cortex (2014) · cites this paper as Bretz et al. 2010
Citations
Cites
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482
Simultaneous Inference in General Parametric Models
Hothorn, T., Bretz, F., & Westfall, P. H. (2008) · Biometrical Journal · doi:10.1002/bimj.200810425 · Cited by 13,950
Self-citationGentleman R: R: A language for data analysis and graphics
Ihaka, R., & Gentleman, R. (1996) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.1996.10474713 · Cited by 11,747
The control of the false discovery rate in multiple testing under dependency
Benjamini, Y., & Yekutieli, D. (2001) · The Annals of Statistics · doi:10.1214/aos/1013699998 · Cited by 10,490
A Multiple Comparison Procedure for Comparing Several Treatments with a Control
Dunnett, C. W. (1955) · Journal of the American Statistical Association · doi:10.1080/01621459.1955.10501294 · Cited by 6,097
A Direct Approach to False Discovery Rates
Storey, J. D. (2002) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/1467-9868.00346 · Cited by 5,603
A Multiple Testing Procedure for Clinical Trials
O'Brien, P. C., & Fleming, T. R. (1979) · Biometrics · doi:10.2307/2530245 · Cited by 3,311
Rectangular Confidence Regions for the Means of Multivariate Normal Distributions
Šidák, Z. (1967) · Journal of the American Statistical Association · doi:10.1080/01621459.1967.10482935 · Cited by 2,904
An improved Bonferroni procedure for multiple tests of significance
SIMES, R. J. (1986) · Biometrika · doi:10.1093/biomet/73.3.751 · Cited by 2,396
Discrete sequential boundaries for clinical trials
GORDON LAN, K. K., & DEMETS, D. L. (1983) · Biometrika · doi:10.1093/biomet/70.3.659 · Cited by 2,310
Flagged first, then by citations
Cited by
Sorted by citations
Vrieze, A., Van Nood, E., Holleman, F., Salojärvi, J., Kootte, R. S., Bartelsman, J. F. W. M., Dallinga–Thie, G. M., Ackermans, M. T., Serlie, M. J., Oozeer, R., et al. (2012) · Gastroenterology · doi:10.1053/j.gastro.2012.06.031 · Cited by 2,913
Blanche, P., Dartigues, J., & Jacqmin‐Gadda, H. (2013) · Statistics in Medicine · doi:10.1002/sim.5958 · Cited by 1,917
A Weighted and Directed Interareal Connectivity Matrix for Macaque Cerebral Cortex
Markov, N. T., Ercsey-Ravasz, M. M., Ribeiro Gomes, A. R., Lamy, C., Magrou, L., Vezoli, J., Misery, P., Falchier, A., Quilodran, R., Gariel, M. A., et al. (2014) · Cerebral Cortex · doi:10.1093/cercor/bhs270 · Cited by 857
“ing multiple probabilities (Bretz et al. 2010), only the projections of areas 7A and DP to area 10 were inconsistent despite a statistically very low probability of being so.”
BackgroundMethodMicroplastic Shape, Polymer Type, and Concentration Affect Soil Properties and Plant Biomass
Lozano, Y. M., Lehnert, T., Linck, L. T., Lehmann, A., & Rillig, M. C. (2021) · Frontiers in Plant Science · doi:10.3389/fpls.2021.616645 · Cited by 697
“Then, we performed multiple comparisons “glht” among treatments by using the “Tukey” test and the function “sandwich” from the eponymous R package; this function provided a heteroscedasticity-consistent estimate of the covariance matrix (Zeileis, 2006; Bretz et al., 2011).”
MethodSimulation-Based Power Analysis for Factorial Analysis of Variance Designs
Lakens, D., & Caldwell, A. R. (2021) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245920951503 · Cited by 602
“The Holm procedure is slightly more powerful than the Bonferroni correction, without requiring additional assumptions (for other approaches, see Bretz et al., 2011).”
MethodRist, S., Baun, A., & Hartmann, N. B. (2017) · Environmental Pollution · doi:10.1016/j.envpol.2017.05.048 · Cited by 571
“Significant findings between the groups were compared using the glht function in R (Bretz et al., 2010).”
MethodBiomass Production and Carbon/Nitrogen Ratio Influence Ecosystem Services from Cover Crop Mixtures
Finney, D. M., White, C. M., & Kaye, J. P. (2015) · Agronomy Journal · doi:10.2134/agronj15.0182 · Cited by 627
“To determine if cover crop treatments provided each service, we used Dunnett’s test to compare measured values from each treatment to the control value within each year (Bretz et al., 2011).”
MethodCombination Antifungal Therapy for Cryptococcal Meningitis
Day, J., Chau, T. T., Wolbers, M., Mai, P. P., Dung, N. T., Mai, N. H., Phu, N. H., Nghia, H. D. T., Phong, N. D., Cao, Q. T., et al. (2013) · New England Journal of Medicine · doi:10.1056/nejmoa1110404 · Cited by 542
Forest resilience to drought varies across biomes
Gazol, A., Camarero, J. J., Vicente-Serrano, S. M., Sánchez‐Salguero, R., Gutiérrez, E., de Luis, M., Sangüesa‐Barreda, G., Novak, K., Rozas, V., Tíscar, P. A., et al. (2018) · Global Change Biology · doi:10.1111/gcb.14082 · Cited by 483
An introduction to multiplicity issues in clinical trials: the what, why, when and how
Li, G., Taljaard, M., Van den Heuvel, E. R., Levine, M. A., Cook, D. J., Wells, G. A., Devereaux, P. J., & Thabane, L. (2016) · International Journal of Epidemiology · doi:10.1093/ije/dyw320 · Cited by 381
“62 Herein we focus on commonly encountered questions about the need for multiplicity corrections, using a tutorial style.”
MethodExpDes: An R Package for ANOVA and Experimental Designs
Ferreira, E. B., Cavalcanti, P. P., & Nogueira, D. A. (2014) · Applied Mathematics · doi:10.4236/am.2014.519280 · Cited by 387
“Thus, according to [1] [2] [4] [7], when treatments are qualitative, several proposals of contrasts among means are available in literature, such as the test of Tukey, Duncan, Student-Newman-Keuls, t, Bonferroni, Dunn, Dunn-Šidák, Dunnett, Gomes-Howell, FisherHayter, Scheffé and others.”
BackgroundLinking anthropogenic resources to wildlife–pathogen dynamics: a review and meta‐analysis
Becker, D. J., Streicker, D. G., & Altizer, S. (2015) · Ecology Letters · doi:10.1111/ele.12428 · Cited by 384
“From each study, we recorded the relationship between provisioning and infection measures (effect size and directionality) along with the source and intention of provisioning, host and pathogen type, and transmission mode of the pathogen.”
MethodRubin, M. (2021) · Synthese · doi:10.1007/s11229-021-03276-4 · Cited by 371
“Berger (1982) proposed this approach as an intersection–union test (Berger, 1982; Berger & Hsu, 1996; Bretz et al., 2011, p. 22).”
BackgroundFrancisco, A., Mohammadi, N., & Taylor, J. E. (2019) · Journal of Management in Engineering · doi:10.1061/(asce)me.1943-5479.0000741 · Cited by 326
“Each hypothesis was tested for each building, introducing the multiple comparison problem, which increases the probability of committing a Type I error (Bretz et al. 2010).”
MethodKonietschke, F., Placzek, M., Schaarschmidt, F., & Hothorn, L. A. (2015) · Journal of Statistical Software · doi:10.18637/jss.v064.i09 · Cited by 316
“For a comprehensive overview of parametric multiple contrast tests and SCI we refer the reader to Bretz, Hothorn, and Westfall (2010) and references therein.”
MethodKaza, A. K., Wamala, I., Friehs, I., Kuebler, J. D., Rathod, R. H., Berra, I., Ericsson, M., Yao, R., Thedsanamoorthy, J. K., Zurakowski, D., et al. (2016) · Journal of Thoracic and Cardiovascular Surgery · doi:10.1016/j.jtcvs.2016.10.077 · Cited by 280
Kazdin, A. E. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108993647.010 · Cited by 279
Kenkel, C. D., Meyer, E., & Matz, M. V. (2013) · Molecular Ecology · doi:10.1111/mec.12390 · Cited by 263
“…was found to be significant at the P < 0.1 level following false discovery rate correction using the method of Benjamini & Hochberg (1995), a post hoc Tukey’s test was used to evaluate the significance of each pairwise comparison using the function ghlt of the multcomp package (Bretz et al. 2010).”
MethodInfluentialPlant Pollinator Networks along a Gradient of Urbanisation
Geslin, B., Gauzens, B., Thébault, É., & Dajoz, I. (2013) · PLoS ONE · doi:10.1371/journal.pone.0063421 · Cited by 252
“Post-hoc test were performed using glht function of the package multcomp [44].”
MethodElevated virulence of an emerging viral genotype as a driver of honeybee loss
McMahon, D. P., Natsopoulou, M. E., Doublet, V., Fürst, M. A., Weging, S., Brown, M. J. F., Gogol‐Döring, A., & Paxton, R. J. (2016) · Proceedings of the Royal Society B Biological Sciences · doi:10.1098/rspb.2016.0811 · Cited by 244
“The R package ‘multcomp’ [46] was used for comparing significant differences between treatment means, using Bonferroni correction to account for multiple testing.”
MethodSonbul, S., & Schmitt, N. (2012) · Language Learning · doi:10.1111/j.1467-9922.2012.00730.x · Cited by 250
opm: an R package for analysing OmniLog® phenotype microarray data
Vaas, L. A. I., Sikorski, J., Hofner, B., Fiebig, A., Buddruhs, N., Klenk, H., & Göker, M. (2013) · Bioinformatics · doi:10.1093/bioinformatics/btt291 · Cited by 216
“Since most addressed experimental questions require to statistically compare not only single curves, but distinct groups of curves, the package provides adapted functionality for performing simultaneous multiple comparisons of group means (Bretz, Hothorn, and Westfall 2010).”
MethodVickerstaff, V., Omar, R., & Ambler, G. (2019) · BMC Medical Research Methodology · doi:10.1186/s12874-019-0754-4 · Cited by 212
“In the context of multiple outcomes, the power of the study can be defined in a number of ways depending on the clinical objective of the trial: i) ‘disjunctive power’, ii) ‘conjunctive power’ or iii) ‘marginal power’ [12].”
BackgroundInfluentialJakobsen, J., Gluud, C., Winkel, P., Lange, T., & Wetterslev, J. (2014) · BMC Medical Research Methodology · doi:10.1186/1471-2288-14-34 · Cited by 191
Rapid publication-ready MS-Word tables for one-way ANOVA
Assaad, H., Zhou, L. F., Carroll, R. J., & Wu, G. (2014) · SpringerPlus · doi:10.1186/2193-1801-3-474 · Cited by 203
“Increasing the power can be done by extending single-step testing procedures into stepwise procedures via a technique known as the closure principle (Bretz et al. 2010).”
MethodBlomster, T., Salojärvi, J., Sipari, N., Brosché, M., Ahlfors, R., Keinänen, M., Overmyer, K., & Kangasjärvi, J. (2011) · PLANT PHYSIOLOGY · doi:10.1104/pp.111.181883 · Cited by 188
“Contrasts were computed with the multcomp package (Bretz et al., 2010) with single-step P value correction.”
MethodDeep ploughing increases agricultural soil organic matter stocks
Alcántara, V., Don, A., Well, R., & Nieder, R. (2016) · Global Change Biology · doi:10.1111/gcb.13289 · Cited by 198
“The P-values were computed with Tukey’s ‘honest significant difference’ of means on 95% family-wise confidence level with Bonferroni correction (Bretz et al., 2010).”
MethodSeasonal priority effects: implications for invasion and restoration in a semi‐arid system
Wainwright, C. E., Wolkovich, E. M., & Cleland, E. E. (2011) · Journal of Applied Ecology · doi:10.1111/j.1365-2664.2011.02088.x · Cited by 170
Simulating ideal assistive devices to reduce the metabolic cost of walking with heavy loads
Dembia, C. L., Silder, A., Uchida, T. K., Hicks, J. L., & Delp, S. L. (2017) · PLoS ONE · doi:10.1371/journal.pone.0180320 · Cited by 170
Vaas, L. A. I., Sikorski, J., Michael, V., Göker, M., & Klenk, H. (2012) · PLoS ONE · doi:10.1371/journal.pone.0034846 · Cited by 165
“Second, as a somewhat more sophisticated approach, we provide a simultaneous multiple comparison procedure of means [70].”
MethodStatistical Analysis and Modeling of Mass Spectrometry-Based Metabolomics Data
Xi, B., Gu, H., Baniasadi, H. R., & Raftery, D. (2014) · Methods in molecular biology · doi:10.1007/978-1-4939-1258-2_22 · Cited by 174
“multiple comparison procedures such as the Bonferroni correction are too conservative and do not work well [10]; they set a cutoff value that is too close to 0 as the number of tests becomes large.”
Role of somatostatin-positive cortical interneurons in the generation of sleep slow waves
Funk, C. M., Peelman, K., Bellesi, M., Marshall, W., Cirelli, C., & Tononi, G. (2017) · The Journal of Neuroscience · doi:10.1523/jneurosci.1303-17.2017 · Cited by 145
“All post hoc comparisons in the LME models were z tests, performed using the glht() function of the multcomp package in R, with p values adjusted for multiple comparisons using the single-step method (Bretz et al., 2011).”
MethodRank-based multiple test procedures and simultaneous confidence intervals
Konietschke, F., Hothorn, L. A., & Brunner, E. (2012) · Electronic Journal of Statistics · doi:10.1214/12-ejs691 · Cited by 158
“For a comprehensive overview of existing parametric methods we refer to Bretz, Hothorn, and Westfall (2010) [4].”
MethodMichl, S. C., Ratten, J.-M., Beyer, M., Hasler, M., LaRoche, J., & Schulz, C. (2017) · PLoS ONE · doi:10.1371/journal.pone.0177735 · Cited by 158
Traditional multiplicity adjustment methods in clinical trials
Dmitrienko, A. A., & D’Agostino, R. B. (2013) · Statistics in Medicine · doi:10.1002/sim.5990 · Cited by 153
Destructive disinfection of infected brood prevents systemic disease spread in ant colonies
Pull, C. D., Ugelvig, L. V., Wiesenhofer, F. M., Grasse, A. V., Tragust, S., Schmitt, T., Brown, M. J. F., & Cremer, S. (2017) · eLife · doi:10.7554/elife.32073 · Cited by 147
Fischer, C., Roscher, C., Jensen, B., Eisenhauer, N., Baade, J., Attinger, S., Scheu, S., Weisser, W. W., Schumacher, J., & Hildebrandt, A. (2014) · PLoS ONE · doi:10.1371/journal.pone.0098987 · Cited by 140
“As the effects of the presence (or absence) of certain plant functional groups on the response variable were considered as post-hoc tests, the respective p-values were adjusted according to the Holm procedure to avoid inflation of Type I error rates [52].”
MethodMultiple testing in statistical analysis of systems-based information retrieval experiments
Carterette, B. (2012) · TOIS · doi:10.1145/2094072.2094076 · Cited by 135
“A more powerful approach to adjusting p-values uses information about correlations between hypotheses [Hothorn et al. 2008; Bretz et al. 2010].”
MethodSimultaneous and selective inference: Current successes and future challenges
Benjamini, Y. (2010) · Biometrical Journal · doi:10.1002/bimj.200900299 · Cited by 169
“I was equally happy to learn that the book by Bretz et al. (2010) ‘‘Multiple Comparisons using R’’ has been just published, transferring into R realm the work and attitude of Westfall et al. (1999).”
MethodReward of labor coordination and hunting success in wild chimpanzees
Samuni, L., Preis, A., Deschner, T., Crockford, C., & Wittig, R. M. (2018) · Communications Biology · doi:10.1038/s42003-018-0142-3 · Cited by 139
Chadeau‐Hyam, M., Campanella, G., Jombart, T., Bottolo, L., Portengen, L., Vineis, P., Liquet, B., & Vermeulen, R. C. H. (2013) · Environmental and Molecular Mutagenesis · doi:10.1002/em.21797 · Cited by 141
“For instance, Holm’s procedure [Holm, 1979] is step-down and alternative step-up approaches starting from the largest p-value (e.g. Hochberg’s [Hochberg, 1988]) were shown to be more powerful [Bretz et al., 2011].”
MethodUse of acute and chronic ecotoxicity data in environmental risk assessment of pharmaceuticals
Vestel, J., Caldwell, D. J., Constantine, L. A., D’Aco, V. J., Davidson, T. L., Dolan, D. G., Millard, S. P., Murray‐Smith, R., Parke, N. J., Ryan, J. J., et al. (2015) · Environmental Toxicology and Chemistry · doi:10.1002/etc.3260 · Cited by 130
Cerin, E., Conway, T. L., Adams, M. A., Barnett, A. H., Cain, K. L., Owen, N. G., Christiansen, L. B., Dyck, D. V., Mitáš, J., Sarmiento, O. L., et al. (2018) · Social Science & Medicine · doi:10.1016/j.socscimed.2018.06.034 · Cited by 127
“All analyses were conducted in R (R Core Team, 2015) using the packages ‘mgcv’ (Wood, 2006) and ‘multcomp’ (Bretz et al., 2010).”
Regulation of cortical activity and arousal by the matrix cells of the ventromedial thalamic nucleus
Honjoh, S., Sasai, S., Schiereck, S. S., Nagai, H., Tononi, G., & Cirelli, C. (2018) · Nature Communications · doi:10.1038/s41467-018-04497-x · Cited by 118
Schuppli, C., Meulman, E. J. M., Forss, S., Aprilinayati, F., van Noordwijk, M. A., & van Schaik, C. P. (2016) · Animal Behaviour · doi:10.1016/j.anbehav.2016.06.014 · Cited by 129
Bunn, C., Läderach, P., Perez-Jimenez, J. G., Montagnon, C., & Schilling, T. (2015) · PLoS ONE · doi:10.1371/journal.pone.0140490 · Cited by 108
“We based the AEZ descriptions on the differences of the group means from the grand mean and calculated their confidence intervals using R’s multcomp package [34].”
MethodVogel, C., Heister, K., Buegger, F., Tanuwidjaja, I., Haug, S., Schloter, M., & Kögel‐Knabner, I. (2015) · Biology and Fertility of Soils · doi:10.1007/s00374-014-0987-7 · Cited by 122
“To compare the influence of different soil types and time levels, we conducted a Tukey’s test using the R package multcomp (Bretz 2011).”
Xu, J., Ou, J., Luo, S., Wang, Z., Chang, E. C., Novak, C. J., Shen, J., Zheng, S., & Wang, Y. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.566965 · Cited by 130
“Then, for perceived social support measured at all three time points, the Bonferroni-corrected post hoc test was conducted using the multcomp package in R (Bretz et al., 2016).”
MethodAssessing Distraction Potential of Augmented Reality Head-Up Displays for Vehicle Drivers
Kim, H., & Gabbard, J. L. (2019) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720819844845 · Cited by 123
“…we found significant main effects, we performed appropriate post hoc tests depending on the types of dependent variables, such as paired t -tests or McNemar tests with Tukey’s adjustment or false discovery rate adjustments to account for multiple comparisons (Bretz, Westfall, & Hothorn, 2016).”
MethodChang, A., Le, C. P., Walker, A. K., Creed, S. J., Pon, C. K., Albold, S., Carroll, D., Halls, M. L., Lane, J. R., Riedel, B., et al. (2016) · Brain Behavior and Immunity · doi:10.1016/j.bbi.2016.06.011 · Cited by 119
Advanced multiplicity adjustment methods in clinical trials
Alosh, M., Bretz, F., & Huque, M. F. (2013) · Statistics in Medicine · doi:10.1002/sim.5974 · Cited by 112
Self-citationIshibashi, H., Crittenden, D. B., Miyauchi, A., Libanati, C. R., Maddox, J., Fan, M., Chen, L., & Grauer, A. (2017) · Bone · doi:10.1016/j.bone.2017.07.005 · Cited by 110
Lozano, Y. M., Aguilar‐Trigueros, C. A., Roy, J., & Rillig, M. C. (2021) · New Phytologist · doi:10.1111/nph.17707 · Cited by 113
Zi, X.-M., Sheng, C.-L., Goodale, U. M., Shao, S., & Gao, J. (2014) · Mycorrhiza · doi:10.1007/s00572-014-0565-8 · Cited by 108
“05 (Bretz et al. 2010).”
Rahnenführer, J., Bin, R. D., Benner, A., Ambrogi, F., Lusa, L., Boulesteix, A., Migliavacca, E., Binder, H., Michiels, S., Sauerbrei, W., et al. (2023) · BMC Medicine · doi:10.1186/s12916-023-02858-y · Cited by 132
Correlated Evolution of Two Copulatory Organs via a Single Cis-Regulatory Nucleotide ChangePreprint
Nagy, O., Nuez, I., Savisaar, R., Peluffo, A. E., Yassin, A., Lang, M., Stern, D. L., Matute, D. R., David, J. R., & Courtier‐Orgogozo, V. (2018) · SSRN Electronic Journal · doi:10.2139/ssrn.3188440 · Cited by 109
Piepho, H., & Edmondson, R. (2018) · Journal of Agronomy and Crop Science · doi:10.1111/jac.12267 · Cited by 101
Madembo, C., Mhlanga, B., & Thierfelder, C. (2020) · Agricultural Systems · doi:10.1016/j.agsy.2020.102921 · Cited by 104
Carrino‐Kyker, S. R., Kluber, L. A., Petersen, S. M., Coyle, K. P., Hewins, C. R., DeForest, J. L., Smemo, K. A., & Burke, D. J. (2016) · FEMS Microbiology Ecology · doi:10.1093/femsec/fiw024 · Cited by 102
Rapid publication-ready MS-Word tables for two-way ANOVA
Assaad, H., Hou, Y., Zhou, L. F., Carroll, R. J., & Wu, G. (2015) · SpringerPlus · doi:10.1186/s40064-015-0795-z · Cited by 99
“In this section, we merely scratch the surface of the complex topic of multiple hypotheses testing; the interested reader may find the books by Westfall et al. (2011) and Bretz et al. (2010) extremely helpful.”
BackgroundA conditional error function approach for subgroup selection in adaptive clinical trials
Friede, T., Parsons, N., & Stallard, N. (2012) · Statistics in Medicine · doi:10.1002/sim.5541 · Cited by 97
Gamba, M., Torti, V., Estienne, V., Randrianarison, R. M., Valente, D., Rovara, P., Bonadonna, G., Friard, O., & Giacoma, C. (2016) · Frontiers in Neuroscience · doi:10.3389/fnins.2016.00249 · Cited by 97
“We analyzed pitch variability by setting a frequency range from theminimum to themaximum of each unit in a DP and then calculating the frequency value at the upper limit of the second (Q50) quartile of energy (Figure 2).”
MethodFeatures and distribution patterns of Chinese endemic seed plant species
Huang, J., Jian‐Hua, C. H. E. N., Jun‐Sheng, Y. I. N. G., & Ma, K. (2011) · Journal of Systematics and Evolution · doi:10.1111/j.1759-6831.2011.00119.x · Cited by 97
“ANOVA and Tukey’s multiple comparisons (Bretz et al., 2010) were carried out to assess endemism differences in floristic elements and taxonomic groups.”
MethodSpano, G. M., Banningh, S. W., Marshall, W., de Vivo, L., Bellesi, M., Loschky, S. S., Tononi, G., & Cirelli, C. (2019) · Journal of Neuroscience · doi:10.1523/jneurosci.0380-19.2019 · Cited by 99
“Post hoc tests were performed using the glht() function of the multcomp package in R, with p values adjusted for multiple comparisons using the single-step method (Bretz et al., 2011).”
MethodZhou, V., MUNSON, J. A., Greenson, J., Hou, Y. F., Rogers, S. J., & Estes, A. M. (2017) · Autism · doi:10.1177/1362361317743251 · Cited by 94
“Due to the exploratory nature of this project and the small sample sizes, Bonferroni correction was considered overly stringent, and the results were reported both with and without this correction (Bretz et al., 2011).”
Demanuele, C., Bartsch, U., Baran, B., Khan, S., Vangel, M. G., Cox, R., Hämäläinen, M., Jones, M. W., Stickgold, R., & Manoach, D. S. (2016) · SLEEP · doi:10.1093/sleep/zsw013 · Cited by 96
β2-adrenoceptor signaling regulates invadopodia formation to enhance tumor cell invasion
Creed, S. J., Le, C. P., Hassan, M., Pon, C. K., Albold, S., Chan, K. T., Berginski, M. E., Huang, Z., Bear, J. E., Lane, J. R., et al. (2015) · Breast Cancer Research · doi:10.1186/s13058-015-0655-3 · Cited by 93
Curiosity Killed the Cat, but Makes Crowdwork Better
Law, E., Yin, M., Goh, J., Chen, K., Terry, M. A., & Gajos, K. Z. (2016) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858144 · Cited by 92
Zhang, G., Chen, F., & Gan, Y. (2016) · Journal of Geophysical Research Atmospheres · doi:10.1002/2016jd024928 · Cited by 91
“Tukey's test [Tukey, 1949; Bretz et al., 2010] was employed to analyze whether the performances of any two schemes of the same subprocess differed.”
MethodCatuara‐Solarz, S., Espinosa‐Carrasco, J., Erb, I., Langohr, K., González, J. R., Notredame, C., & Dierssen, M. (2016) · eNeuro · doi:10.1523/eneuro.0103-16.2016 · Cited by 90
Myelin modifications after chronic sleep loss in adolescent mice
Bellesi, M., Haswell, J. D., de Vivo, L., Marshall, W., Roseboom, P. H., Tononi, G., & Cirelli, C. (2018) · SLEEP · doi:10.1093/sleep/zsy034 · Cited by 92
The Gut Microbiota Can Provide Viral Tolerance in the Honey Bee
Dosch, C., Manigk, A., Streicher, T., Tehel, A., Paxton, R. J., & Tragust, S. (2021) · Microorganisms · doi:10.3390/microorganisms9040871 · Cited by 91
Hahn, K. R., Myers, N. E., Prigarin, S. M., Rodenacker, K., Kurz, A., Förstl, H., Zimmer, C., Wohlschläger, A. M., & Sorg, C. (2013) · NeuroImage · doi:10.1016/j.neuroimage.2013.05.011 · Cited by 86
Impact of Invasive Tree Species on Natural Regeneration Species Composition, Diversity, and Density
Dyderski, M. K., & Jagodziński, A. M. (2020) · Forests · doi:10.3390/f11040456 · Cited by 96
“We evaluated influence of forest type using Tukey’s posteriori test, implemented in the multcomp package [82].”
BackgroundMethodSpalek, K., Gotzner, N., & Wartenburger, I. (2013) · Journal of Memory and Language · doi:10.1016/j.jml.2013.09.001 · Cited by 86
The shape of success in a turbulent world: wave exposure filtering of coral reef herbivory
Bejarano, S., Jouffray, J., Chollett, I., Allen, R., Roff, G., Marshell, A., Steneck, R. S., Ferse, S. C. A., & Mumby, P. J. (2017) · Functional Ecology · doi:10.1111/1365-2435.12828 · Cited by 85
STATISTICAL ANALYSIS OF THE MARIANA TRENCH GEOMORPHOLOGY USING R PROGRAMMING LANGUAGE
Lemenkova, P. (2019) · Geodesy and Cartography · doi:10.3846/gac.2019.3785 · Cited by 83
Diagnostic gene expression biomarkers of coral thermal stress
Kenkel, C. D., Sheridan, C., Leal, M. C., Bhagooli, R., Castillo, K. D., Kurata, N. P., McGinty, E. S., Goulet, T. L., & Matz, M. V. (2013) · Molecular Ecology Resources · doi:10.1111/1755-0998.12218 · Cited by 83
Research Design in Clinical Psychology
Kazdin, A. E. (2021) · doi:10.1017/9781108993647 · Cited by 240
Quantitative Designs and Statistical Techniques
Gries, S. Τ. (2015) · Cambridge University Press eBooks · doi:10.1017/9781009477888.004 · Cited by 81
Quantitative Methods in Archaeology Using R
Carlson, D. L. (2017) · Cambridge University Press eBooks · doi:10.1017/9781139628730 · Cited by 85
Tang, X., Pei, X., Lei, N., Luo, X., Liu, L., Shi, L., Chen, G., & Liang, J. (2020) · Geoderma · doi:10.1016/j.geoderma.2020.114339 · Cited by 80
Corlatti, L., Béthaz, S., von Hardenberg, A., Bassano, B., Palme, R., & Lovari, S. (2012) · Animal Behaviour · doi:10.1016/j.anbehav.2012.08.005 · Cited by 80
Longitudinal Analysis of Progression in Glaucoma Using Spectral-Domain Optical Coherence Tomography
Wessel, J. M., Horn, F. K., Tornow, R. P., Schmid, M. C., Mardin, C. Y., Kruse, F. E., Juenemann, A. G. M., & Lämmer, R. (2013) · Investigative Ophthalmology & Visual Science · doi:10.1167/iovs.12-9786 · Cited by 80
Deep soil flipping increases carbon stocks of New Zealand grasslands
Schiedung, M., Tregurtha, C. S., Beare, M., Thomas, S. M., & Don, A. (2019) · Global Change Biology · doi:10.1111/gcb.14588 · Cited by 82
The effects of land use characteristics on urban stormwater quality and watershed pollutant loads.
Yazdi, M. N., Sample, D., Scott, D., Wang, X.-X., & Ketabchy, M. (2021) · Science of the Total Environment · doi:10.1016/j.scitotenv.2021.145358 · Cited by 62
Interaction of methyl viologen-induced chloroplast and mitochondrial signalling in Arabidopsis
Cui, F., Brosché, M., Shapiguzov, A., He, X., Vainonen, J. P., Leppälä, J., Trotta, A., Kangasjärvi, S., Salojärvi, J., Kangasjärvi, J., et al. (2019) · Free Radical Biology and Medicine · doi:10.1016/j.freeradbiomed.2019.02.006 · Cited by 77
Dolný, A., & Harabiš, F. (2011) · Biological Conservation · doi:10.1016/j.biocon.2011.10.020 · Cited by 76
Recommendation domains to scale out climate change adaptation in cocoa production in Ghana
Bunn, C., Läderach, P., Quaye, A. K., Muilerman, S., Noponen, M., & Lundy, M. (2019) · Climate Services · doi:10.1016/j.cliser.2019.100123 · Cited by 77
Dolný, A., Harabiš, F., & Mižičová, H. (2014) · PLoS ONE · doi:10.1371/journal.pone.0100408 · Cited by 74
A Versatile Method for Confirmatory Evaluation of the Effects of a Covariate in Multiple Models
Pipper, C. B., Ritz, C., & Bisgaard, H. F. (2011) · Journal of the Royal Statistical Society Series C (Applied Statistics) · doi:10.1111/j.1467-9876.2011.01005.x · Cited by 74
“A very practical and popular solution to this problem is provided in current state of the art procedures (Hothorn et al., 2008; Herberich et al., 2010; Bretz et al., 2010).”
BackgroundMethodInfluentialMultimodal learning analytics to investigate cognitive load during online problem solving
Larmuseau, C., Cornelis, J. P., Lancieri, L., Desmet, P., & Depaepe, F. (2020) · British Journal of Educational Technology · doi:10.1111/bjet.12958 · Cited by 95
Pocnet, C., Antonietti, J., Strippoli, M. F., Glaus, J., Preisig, M., & Rossier, J. (2016) · Quality of Life Research · doi:10.1007/s11136-016-1296-4 · Cited by 78
“events, perceived social support, personality traits, and QoL (the p-values were corrected using Holm’s method to counteract the problem of multiple comparisons that occurs when one considers a set of statistical inference simultaneously [39, 40]).”
MethodHiatus Resolution in American English: The Case Against Glide Insertion
Davidson, L., & Erker, D. (2014) · Language · doi:10.1353/lan.2014.0028 · Cited by 72
“…be modal than fast tokens, and VBV is less likely to be modal than VGV, but there is no significant difference between VGV and VV. Post-hoc Tukey tests using the multcomp package in R (Bretz et al. 2010) show that VV is significantly more likely to be modal than VBV (β = 3.40, z = 5.99, p < 0.001).”
MethodInfluentialWang, J., Chatzidimitriou, E., Wood, L., Hasanaliyeva, G., Markellou, E., Iversen, P. O., Seal, C. J., Barański, M., Vigar, V., Ernst, L., et al. (2020) · Food Chemistry X · doi:10.1016/j.fochx.2020.100091 · Cited by 71
Quigley, K. M., Willis, B. L., & Bay, L. K. (2016) · Royal Society Open Science · doi:10.1098/rsos.160471 · Cited by 71
Nicotine in floral nectar pharmacologically influences bumblebee learning of floral features
Baracchi, D., Marples, A., Jenkins, A. J., Leitch, A. R., & Chıttka, L. (2017) · Scientific Reports · doi:10.1038/s41598-017-01980-1 · Cited by 71
Flavonoids as Putative Inducers of the Transcription Factors Nrf2, FoxO, and PPARγ
Pallauf, K., Duckstein, N., Hasler, M., Klotz, L., & Rimbach, G. H. (2017) · Oxidative Medicine and Cellular Longevity · doi:10.1155/2017/4397340 · Cited by 71
Husted, G. R., Thorsteinsson, B., Esbensen, B. A., Gluud, C. N., Winkel, P., Hommel, E. E., & Zoffmann, V. (2014) · Trials · doi:10.1186/1745-6215-15-321 · Cited by 72
“to control the family-wise error rate [35].”
BackgroundMethodInfluentialOn Using the Bootstrap for Multiple Comparisons
Westfall, P. H. (2011) · Journal of Biopharmaceutical Statistics · doi:10.1080/10543406.2011.607751 · Cited by 70
Self-citation
Metadata
- Authors
- Bretz, F., Hothorn, T., & Westfall, P.
- Affiliations
- Texas Tech University (US)
- Venue
- Chapman & Hall/CRC eBooks
- Publisher
- Chapman and Hall/CRC
- Published
- 2016-04-19
- OpenAlex
- W396053900
- Cited by
- 1,233
- References
- 93
- Citations by year
- 2026: 20 · 2025: 57 · 2024: 96 · 2023: 70 · 2022: 97 · 2021: 146 · 2020: 99 · 2019: 133 · 2018: 98 · 2017: 89 · 2016: 78
- FWCI
- 12.36
- Type
- book
- Language
- en
- Open access
- closed
- Locations
- Chapman & Hall/CRC eBooks (publishedVersion)
- Thammasat University Digital Collections (submittedVersion)
- TU Digital Collections (Thammasat University) (submittedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Physical Sciences › Mathematics › Statistics and Probability › Data Analysis with R
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Applications
- Keywords
- multiple comparisons, R software, statistical software implementation, hypothesis testing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
