Cited by 1,233Statistical Methods in Clinical TrialsData Analysis with RStatistical Methods and Applications

The DOI matches its Crossref record; 94 of the 94 references it cites were found in a registry or an index.

Multiple Comparisons Using R

Bretz, Frank, Hothorn, Torsten, Westfall, Peter·2016·Chapman & Hall/CRC eBooks

DOI 10.1201/9781420010909

Open this paper in CoChat — free account

Abstract

Adopting a unifying theme based on maximum statistics, Multiple Comparisons Using R describes the common underlying theory of multiple comparison procedures through numerous examples. It also presents a detailed description of available software implementations in R. The R packages and source code for the analyses are available at https://CRAN.R-pro

Integrity signals

Registry

Crossref · 10.1201/9781420010909

Cites

94 of 94 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482
Simultaneous Inference in General Parametric Models
Hothorn, T., Bretz, F., & Westfall, P. H. (2008) · Biometrical Journal · doi:10.1002/bimj.200810425 · Cited by 14,265
Self-citation
Gentleman R: R: A language for data analysis and graphics
Ihaka, R., & Gentleman, R. (1996) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.1996.10474713 · Cited by 11,747
The control of the false discovery rate in multiple testing under dependency
Benjamini, Y., & Yekutieli, D. (2001) · The Annals of Statistics · doi:10.1214/aos/1013699998 · Cited by 10,490
A Multiple Comparison Procedure for Comparing Several Treatments with a Control
Dunnett, C. W. (1955) · Journal of the American Statistical Association · doi:10.1080/01621459.1955.10501294 · Cited by 6,097
A Direct Approach to False Discovery Rates
Storey, J. D. (2002) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/1467-9868.00346 · Cited by 5,603
A Multiple Testing Procedure for Clinical Trials
O'Brien, P. C., & Fleming, T. R. (1979) · Biometrics · doi:10.2307/2530245 · Cited by 3,311
Rectangular Confidence Regions for the Means of Multivariate Normal Distributions
Šidák, Z. (1967) · Journal of the American Statistical Association · doi:10.1080/01621459.1967.10482935 · Cited by 2,904
An improved Bonferroni procedure for multiple tests of significance
SIMES, R. J. (1986) · Biometrika · doi:10.1093/biomet/73.3.751 · Cited by 2,396
Discrete sequential boundaries for clinical trials
GORDON LAN, K. K., & DEMETS, D. L. (1983) · Biometrika · doi:10.1093/biomet/70.3.659 · Cited by 2,310

Flagged first, then by citations

Show all 93 references

Sorted by citations

Items 501–600 of 1,000

The occurrence of invasive plant species differed significantly across three urban greenspace types of Metro Vancouver, Canada
Nguyen, N.-A., Eskelson, B. N. I., Gergel, S. E., & Murray, T. (2021) · Urban forestry & urban greening · doi:10.1016/j.ufug.2021.126999 · Cited by 17
Parent-offspring turn-taking dynamics influence parents’ song structure and elaboration in a singing primate
Gregorio, C. D., Zanoli, A., Carugati, F., Raimondi, T., Valente, D., Torti, V., Miaretsoa, L., Rajaonson, A., Gamba, M., & Giacoma, C. (2022) · Frontiers in Ecology and Evolution · doi:10.3389/fevo.2022.906322 · Cited by 16
“Finally, we used the Tukey test (within the multiple contrast package multcomp in R) to perform all pairwise comparisons for all levels of the interaction (Bretz et al., 2010).”
Method
Modification and oxidative degradation of β-lactoglobulin by UVB irradiation
Fitzner, L., Kühl, T., Hasler, M., Imhof, D., Schwarz, K., & Keppler, J. K. (2023) · Food Chemistry · doi:10.1016/j.foodchem.2023.136698 · Cited by 16
Assessing teratogenic risks of gadolinium in freshwater environments: Implications for environmental health
Cesarini, G., Spani, F., Patricelli, R., Quattrocchi, C. C., Colasanti, M., & Scalici, M. (2024) · Ecotoxicology and Environmental Safety · doi:10.1016/j.ecoenv.2024.116442 · Cited by 16
“All previous steps of the data analysis were conducted with R libraries car (Fox and Weisberg, 2018) and multcomp (Bretz et al., 2016) and related functions.”
Phenological stages of wheat modulate effects of phosphorus fertilization in plant-soil microbial interactions
García‐Díaz, C., Siles, J. A., Moreno, J. L., García, C., Ruiz‐Navarro, A., & Bastida, F. (2024) · Plant and Soil · doi:10.1007/s11104-024-06880-8 · Cited by 16
“1.4–23 (Bretz et al. 2016) and multcompView ver.”
Influential
Dominance of biologically produced nitrate in upland waters of Great Britain indicated by stable isotopes
Curtis, C. J., Heaton, T. H. E., Simpson, G. L., Evans, C., Shilland, J., & Turner, S. D. (2011) · Biogeochemistry · doi:10.1007/s10533-011-9686-8 · Cited by 16
“Post hoc pair-wise comparisons were conducted according to the method of Tukey’s Honest Significance Differences (Tukey’s range test) at a 95% family-wise confidence level (Bretz et al. 2010).”
Method
Land transitions from multivariate time series: using seasonal trend analysis and segmentation to detect land-cover changes
Parmentier, B., & Eastman, J. R. (2014) · International Journal of Remote Sensing · doi:10.1080/01431161.2013.871595 · Cited by 16
“Possible solutions to solve these problems may include examining the dependency structure among observations and among variables or performing Monte Carlo analyses in order to adjust the p-level of individual tests (Dudoit and Van Der Laan 2007; Bretz, Hothorn, and Westfall 2010).”
Method
Modulation of respiratory metabolism in response to nutrient changes along a soil chronosequence
Kornfeld, A., Atkin, O. K., Griffin, K. L., Horton, T. W., Yakir, D., & Turnbull, M. H. (2012) · Plant Cell & Environment · doi:10.1111/pce.12047 · Cited by 16
Restoring riparian forest in the Atlantic Forest: does planting seedlings make a difference?
Inhamuns, M. C., Rezende, R. D. S., & Coelho, G. C. (2021) · Restoration Ecology · doi:10.1111/rec.13356 · Cited by 16
Statistical methods for exploring spontaneous adverse event reporting databases for drug-host factor interactions
Lu, Z., Suzuki, A., & Wang, D. (2023) · BMC Medical Research Methodology · doi:10.1186/s12874-023-01885-w · Cited by 16
Determination of changes in the concentration and distribution of elements within olive drupes (cv. Leccino) from Se biofortified plants, using laser ablation inductively coupled plasma mass spectrometry
D’Amato, R., Petrelli, M., Proietti, P., Onofri, A., Regni, L., Perugini, D., & Businelli, D. (2018) · Journal of the Science of Food and Agriculture · doi:10.1002/jsfa.9030 · Cited by 16
Social and configural effects on the cognitive dynamics of perspective-taking
Galati, A., Dale, R. A. C., & Duran, N. D. (2018) · Journal of Memory and Language · doi:10.1016/j.jml.2018.08.007 · Cited by 16
Sex-specific hoarding behavior in North American red squirrels (Tamiasciurus hudsonicus)
Archibald, D. W., Fletcher, Q. E., Boutin, S. A., McAdam, A. G., Speakman, J. R., & Humphries, M. M. (2013) · Journal of Mammalogy · doi:10.1644/12-mamm-a-213.1 · Cited by 16
Potential local adaptation in populations of invasive reed canary grass (Phalaris arundinacea) across an urbanization gradient
Weston, L. M., Mattingly, K. Z., Day, C. T. C., & Hovick, S. M. (2021) · Ecology and Evolution · doi:10.1002/ece3.7938 · Cited by 15
To what extent does productive derivational knowledge of adult L1 speakers and L2 learners at two educational levels differ?
Iwaizumi, E., & Webb, S. (2021) · TESOL Journal · doi:10.1002/tesj.640 · Cited by 20
Wildfire promotes the invasion of Robinia pseudoacacia in the unmanaged Mediterranean Castanea sativa coppice forests
Saulino, L., Rita, A., Stinca, A., Liuzzi, G., Silvestro, R., Rossi, S., & Saracino, A. (2023) · Frontiers in Forests and Global Change · doi:10.3389/ffgc.2023.1177551 · Cited by 17
“Statistical differences among the estimated parameters were tested by applying the Wald test procedure using the Bonferroni multiplicity adjustment method, followed by Tukey’s multiple comparison test (Bretz et al., 2011).”
Method
Reciprocal indirect facilitation between an invasive macrophyte and an invasive crayfish
Thouvenot, L., Haury, J., Pottier, G., & Thiébaut, G. (2017) · Aquatic Botany · doi:10.1016/j.aquabot.2017.02.002 · Cited by 15
Effect of Physical Fatigue Elicited by Continuous and Intermittent Exercise on Motor Imagery Ability
Kanthack, T. F. D., Guillot, A., Clémençon, M., Debarnot, U., & Rienzo, F. D. (2020) · Research Quarterly for Exercise and Sport · doi:10.1080/02701367.2019.1691709 · Cited by 15
Reproductive state modulates testosterone-induced singing in adult female European starlings (Sturnus vulgaris)
Rouse, M. L., Stevenson, T. J., Fortune, E. S., & Ball, G. F. (2015) · Hormones and Behavior · doi:10.1016/j.yhbeh.2015.04.022 · Cited by 15
“For statistically significant interaction effects in the MANOVA, post-hoc pairwise comparisons were made and were corrected for multiple comparisons usingWestfall's procedure (Westfall, 1997; Bretz et al., 2010).”
Plasma Lipocalin 2 in Alzheimer’s disease: potential utility in the differential diagnosis and relationship with other biomarkers
Hermann, P., Villar‐Piqué, A., Schmitz, M., Schmidt, C., Varges, D., Goebel, S., Bunck, T., Lindemann, H., Bogner, C., Santana, I. J., et al. (2022) · Alzheimer s Research & Therapy · doi:10.1186/s13195-021-00955-9 · Cited by 16
“Multiple comparisons of means were performed with Tukey contrasts available in the multcomp R package [31].”
Method
Western Gull Foraging Behavior as an Ecosystem State Indicator in Coastal California
Cimino, M. A., Shaffer, S. A., Welch, H., Santora, J. A., Warzybok, P., Jahncke, J., Schroeder, I. D., Hazen, E. L., & Bograd, S. J. (2022) · Frontiers in Marine Science · doi:10.3389/fmars.2021.790559 · Cited by 15
“, 2008) with Tukey’s honestly significant difference pairwise multiple comparisons of means (Bretz et al., 2016) and adjusted p-values using the Bonferroni correction to account for multiple comparisons.”
Method
Neural processing of poems and songs is based on melodic properties
Scharinger, M., Knoop, C. A., Wagner, V., & Menninghaus, W. (2022) · NeuroImage · doi:10.1016/j.neuroimage.2022.119310 · Cited by 15
“Planned comparisons using the multcomp package (Bretz et al., 2011) revealed that original poems received slightly higher ratings than modified poems (t=2.”
MethodInfluential
Mixed Pine Forests in a Hotter and Drier World: The Great Resilience to Drought of Aleppo Pine Benefits It Over Other Coexisting Pine Species
Gazol, A., Oliva, J., Valeriano, C., Colangelo, M., & Camarero, J. J. (2022) · Frontiers in Forests and Global Change · doi:10.3389/ffgc.2022.899425 · Cited by 15
“When significant effect of species or interactions between species and year were found, we performed least-square means based on the Tukey’s honest significant difference (HSD) to test for the differences in growth trends and resilience components between species (Bretz et al., 2010).”
Method
Simultaneous inference for several quantiles of a normal population with applications
Liu, W., Bretz, F., Hayter, A. J., & Glimm, E. (2012) · Biometrical Journal · doi:10.1002/bimj.201100232 · Cited by 15
“Various simultaneous confidence intervals involving several normal means have been published in the statistical literature; see Hochberg and Tamhane (1987), Hsu (1996), and Bretz et al. (2010) for extensive reviews. The method of Satten (1995), which is based on a 1 − α confidence set for (μ, σ 2) and can be regarded…”
BackgroundMethodInfluentialSelf-citation
MRI vs. CT for orthodontic applications: comparison of two MRI protocols and three CT (multislice, cone-beam, industrial) technologies
Detterbeck, A., Hofmeister, M., Hofmann, E., Haddad, D., Weber, D. L., Hölzing, A., Zabler, S., Schmid, M. C., Hiller, K., Jakob, P. M., et al. (2016) · Journal of Orofacial Orthopedics / Fortschritte der Kieferorthopädie · doi:10.1007/s00056-016-0028-2 · Cited by 15
“5) were compared by pairwise statistical ‘‘general linear hypothesis’’ testing [35, 36].”
Method
Can neuropsychological testing facilitate differential diagnosis between at-risk mental state (ARMS) for psychosis and adult attention-deficit/hyperactivity disorder (ADHD)?
Studerus, E., Corbisiero, S., Mazzariello, N., Ittig, S., Leanza, L., Egloff, L., Beck, K., Heitz, U., Andreou, C., Stieglitz, R., et al. (2018) · European Psychiatry · doi:10.1016/j.eurpsy.2018.02.006 · Cited by 15
“We first tested for overall group differences using likelihood ratio tests and then performed all possible pairwise comparisons using the ghlt function in the R multcomp package [55].”
Method
Does winter oilseed rape grown from clothianidin-coated seeds affect experimental populations of mason bees and bumblebees? A semi-field and field study
Dietzsch, A. C., Kunz, N., Wirtz, I., Stähler, M., Heimbach, U., & Pistorius, J. (2019) · Journal of Consumer Protection and Food Safety · doi:10.1007/s00003-019-01225-5 · Cited by 18
“Multiple comparisons were adjusted using the multivariate t distribution (Bretz et al. 2010).”
Method
Magnitude Based Inference in Relation to One-sided Hypotheses Testing ProceduresPreprint
Aisbett, J., Lakens, D., & Sainani, K. L. (2020) · doi:10.31236/osf.io/pn9s3 · Cited by 15
Effect of management and age of ploughed out grass–clover on forage maize yield and residual soil nitrogen
Cougnon, M., Van Den Berge, K., D'Hose, T., Clement, L., & Reheul, D. (2018) · The Journal of Agricultural Science · doi:10.1017/s0021859618000631 · Cited by 14
T cell phenotype in paediatric heart transplant recipients
Mengrelis, K., Kučera, F., Shahid, N., Watt, E., Ross, S. E., Lau, C., Adams, S. P., Gilmour, K. C., Pils, D., Crompton, T., et al. (2020) · Pediatric Transplantation · doi:10.1111/petr.13930 · Cited by 14
All roads lead to Rome: inter-origin variation in metabolomics reprogramming of the northern shrimp exposed to global changes leads to a comparable physiological status
Guscelli, E., Chabot, D., Vermandele, F., Madeira, D., & Calosi, P. (2023) · Frontiers in Marine Science · doi:10.3389/fmars.2023.1170451 · Cited by 16
“When one of these tests was significant according to FDR, Tukey post-hoc tests were used to identify significant differences using the HSD.test or the glht function for isolated or interactive effects respectively (Hothorn et al., 2008; Bretz et al., 2016).”
BackgroundMethod
Brain wave modulation and EEG power changes during auditory beats stimulation
Anjos, T. D., Rienzo, F. D., Benoit, C.-É., Daligault, S., & Guillot, A. (2024) · Neuroscience · doi:10.1016/j.neuroscience.2024.07.014 · Cited by 14
Antibacterial and Antibiofilm Activity of Layers Enriched with Silver Nanoparticles on Orthodontic Microimplants
Sycińska-Dziarnowska, M., Ziąbka, M., Cholewa‐Kowalska, K., Klesiewicz, K., Spagnuolo, G., Lindauer, S. J., Park, H., & Woźniak, K. (2025) · Journal of Functional Biomaterials · doi:10.3390/jfb16030078 · Cited by 14
Epigenetic regulation of ABCG2 gene is associated with susceptibility to xenobiotic exposure
Babu, K., Zhang, J., Moloney, S., Pleasants, T., McLean, C. A., Phua, S. H., & Sheppard, A. M. (2012) · Journal of Proteomics · doi:10.1016/j.jprot.2012.04.040 · Cited by 14
Sizing clinical trials when comparing bivariate time‐to‐event outcomes
Sugimoto, T., Hamasaki, T., Evans, S. R., & Sozu, T. (2017) · Statistics in Medicine · doi:10.1002/sim.7225 · Cited by 14
Is tail biting in growing pigs reduced by a prolonged suckling period?
Naya, A., Traulsen, I., Gertz, M., Hasler, M., Burfeind, O., Beilage, E. G., & Krieter, J. (2018) · Applied Animal Behaviour Science · doi:10.1016/j.applanim.2018.10.020 · Cited by 14
Does extraction of orchids affect their population structure? Evidence from populations of Laelia autumnalis (Orchidaceae)
Emeterio-Lara, A., García-Franco, J. G., Hernández‐Apolinar, M., Toledo‐Hernández, V. H., Valencia-Díaz, S., & Flores‐Palacios, A. (2020) · Forest Ecology and Management · doi:10.1016/j.foreco.2020.118667 · Cited by 14
Comparing impacts of metal contamination on macroinvertebrate and fish assemblages in a northern Japanese river
Namba, H., Iwasaki, Y., Morita, K., Ogino, T., Mano, H., Shinohara, N., Yasutaka, T., Matsuda, H., & Kamo, M. (2021) · PeerJ · doi:10.7717/peerj.10808 · Cited by 14
Understanding the limits to the hydraulic leg mechanism: the effects of speed and size on limb kinematics in vagrant arachnids
Boehm, C., Schultz, J. T., & Clemente, C. J. (2021) · Journal of Comparative Physiology A · doi:10.1007/s00359-021-01468-4 · Cited by 14
Not all yawns tell the same story: The case of Tonkean macaques
Zannella, A., Stanyon, R., Maglieri, V., & Palagi, E. (2021) · American Journal of Primatology · doi:10.1002/ajp.23263 · Cited by 14
“We performed pairwise comparisons for the levels of the multilevel factor with the Tukey test (Bretz et al., 2010) by using the R package emmeans (Length et al.”
Method
Wild geladas (Theropithecus gelada) in crops—more than in pasture areas—reduce aggression and affiliation
Caselli, M., Zanoli, A., Dagradi, C., Gallo, A., Yazezew, D., Tadesse, A., Capasso, M., Ianniello, D., Rinaldi, L., Palagi, E., et al. (2021) · Primates · doi:10.1007/s10329-021-00916-8 · Cited by 14
“For multinomial predictors with a significant main effect, we used a multiple contrast package (multcomp) to perform all pairwise comparisons for each bonding level with the Tukey test (Bretz et al. 2010).”
Method
Rising temperatures threaten pollinators of fig trees—Keystone resources of tropical forests
van Kolfschoten, L., Dück, L., Lind, M. I., & Jandér, K. C. (2022) · Ecology and Evolution · doi:10.1002/ece3.9311 · Cited by 13
“Additionally, significant differences in lifespan between all pairwise temperature comparisons were investigated using Tukey contrasts with Bonferroni adjusted p - values, as implemented in the “multcomp” package (Bretz et al., 2010).”
A double-sampling approach to deriving training and validation data for remotely-sensed vegetation products
Karl, J. W., Taylor, J. J., & Bobo, M. (2014) · International Journal of Remote Sensing · doi:10.1080/01431161.2014.880820 · Cited by 14
“The ANOVAs were followed up with two-way multiple comparison tests using the Tukey–Kramer method to control for family-wise Type I error rate (Bretz, Hothorn, and Westfall 2010).”
Method
Beaches under pressure – effects of human access on vegetation at Baltic Sea beaches
Seer, F. K., Irmler, U., & Schrautzer, J. (2016) · Applied Vegetation Science · doi:10.1111/avsc.12221 · Cited by 14
Impact of US Brown Swiss genetics on milk quality from low-input herds in Switzerland: Interactions with season
Stergiadis, S., Bieber, A., Chatzidimitriou, E., Franceschin, E., Isensee, A., Rempelos, L., Barański, M., Maurer, V., Cozzi, G., Bapst, B., et al. (2018) · Food Chemistry · doi:10.1016/j.foodchem.2018.01.077 · Cited by 14
Screening dietary biochanin A, daidzein, equol and genistein for their potential to increase DHA biosynthesis in rainbow trout (Oncorhynchus mykiss)
Fickler, A., Staats, S., Rimbach, G. H., & Schulz, C. (2019) · PLoS ONE · doi:10.1371/journal.pone.0210197 · Cited by 14
Nuclear behavior and roles indicate that Codiolum phase is a sporophyte in Monostroma angicava (Ulotrichales, Ulvophyceae)
Horinouchi, Y., Yamaguchi, M., Chibana, H., & Togashi, T. (2019) · Journal of Phycology · doi:10.1111/jpy.12841 · Cited by 14
“For multiple comparisons, we corrected the significance level via the Bonferroni method (Bretz et al. 2016).”
Method
Geological boundary detection from well-logs: An efficient approach based on pattern recognition
Partovi, S. M. A., & Sadeghnejad, S. (2019) · Journal of Petroleum Science and Engineering · doi:10.1016/j.petrol.2019.01.069 · Cited by 14
COMBINING EXPLICIT AND SENSITIVE INDICES FOR MEASURING L2 VOCABULARY LEARNING THROUGH CONTEXTUALIZED INPUT AND WORD-FOCUSED INSTRUCTION
Vandenberghe, B., Perez, M. M., Reynvoet, B., & Desmet, P. (2021) · Studies in Second Language Acquisition · doi:10.1017/s0272263120000431 · Cited by 12
Acute‐phase proteins in cats with obstructive feline lower urinary tract disease
Dinallo, H. R., Giuffrida, R., de Azevedo, M. G. P., Santarém, C. L., Andrade, S. M. C. F., Brinholi, R. B., da Motta, Y. P., Schmidt, E. M. S., Ferreira, J. C. P., García, H. D. M., et al. (2022) · Veterinary Clinical Pathology · doi:10.1111/vcp.13047 · Cited by 13
Keeping an Eye on the Refutation Effect: The Role of Prior Knowledge and Text-Based Interest on Attention Allocation
Catrysse, L., Chauliac, M., Donche, V., & Gijbels, D. (2022) · Discourse Processes · doi:10.1080/0163853x.2022.2142459 · Cited by 13
“Additional post hoc comparisons were performed with the multcomp package (Bretz et al., 2011; version 1.4–18).”
MethodInfluential
Transcriptome alterations in peripheral blood B cells of patients with multiple sclerosis receiving immune reconstitution therapy
Hecker, M., Fitzner, B., Boxberger, N., Putscher, E., Engelmann, R., Bergmann, W., Müller, M. K., Ludwig‐Portugall, I., Schwartz, M., Meister, S., et al. (2023) · Journal of Neuroinflammation · doi:10.1186/s12974-023-02859-x · Cited by 13
“Pairwise comparisons between the study groups were performed with Tukey post hoc tests for linear mixed-effects models (LMM) using the R package multcomp [50].”
Method
Increasing grain selenium concentration via genetic and agronomic innovations
Daud, M. F. B., Rempelos, L., Çakmak, İ., Leifert, C., & Bilsborrow, P. E. (2023) · Plant and Soil · doi:10.1007/s11104-023-06293-z · Cited by 14
“…and fertiliser type for the QLIF trial and (iii) fertiliser type and variety for the HMC trial and including significant interactions between those factors, general linear hypothesis tests (Tukey contrasts) were performed using the “glht” function of the “multcomp” package (Bretz et al. 2011) in R.”
Application of biological early warning systems in wastewater treatment plants: Introducing a promising approach to monitor changing wastewater composition.
Kizgin, A., Schmidt, D., Joss, A., Hollender, J., Morgenroth, E., Kienle, C., & Langer, M. (2023) · Journal of Environmental Management · doi:10.1016/j.jenvman.2023.119001 · Cited by 10
“For comparison, the compact letter display was calculated by using the package “ multcomp ” (Bretz et al., 2016).”
Method
Leaf surface features of maize cultivars and response to foliar phosphorus application: effect of leaf stage and plant phosphorus status
Henningsen, J. N., Venturas, M. D., Quintero, J. M., Recena, R., Mühling, K. H., & Fernández, V. (2023) · Physiologia Plantarum · doi:10.1111/ppl.14093 · Cited by 13
High-Resolution Disease Phenotyping Reveals Distinct Resistance Mechanisms of Tomato Crop Wild Relatives against Sclerotinia sclerotiorum
Einspanier, S., Tominello-Ramirez, C., Hasler, M., Barbacci, A., Raffaele, S., & Stam, R. (2024) · Plant Phenomics · doi:10.34133/plantphenomics.0214 · Cited by 13
Evidence for the importance of litter as a co-substrate for MCPA dissipation in an agricultural soil
Saleh, O., Pagel, H., Enowashu, E., Devers‐Lamrani, M., Martin-Laurent, F. A., Streck, T., Kandeler, E., & Poll, C. (2015) · Environmental Science and Pollution Research · doi:10.1007/s11356-015-4633-1 · Cited by 13
“Significant differences between half-lives were estimated by a Tukey’s range test employing the glht function of the multcomp R package (Bretz et al. 2011).”
Method
Predation of the non-native Asian shore crab Hemigrapsus sanguineus by a native fish species, the cunner (Tautogolabrus adspersus)
Savaria, M. C., & O’Connor, N. J. (2013) · Journal of Experimental Marine Biology and Ecology · doi:10.1016/j.jembe.2013.10.003 · Cited by 13
Localized Glaucomatous Change Detection within the Proper Orthogonal Decomposition Framework
Balasubramanian, M., Kriegman, D. J., Bowd, C., Holst, M., Weinreb, R. N., Sample, P. A., & Zangwill, L. M. (2012) · Investigative Ophthalmology & Visual Science · doi:10.1167/iovs.11-8847 · Cited by 13
Dietary Buglossoides arvensis Oil as a Potential Candidate to Substitute Fish Oil in Rainbow Trout Diets
Fickler, A., Staats, S., Hasler, M., Rimbach, G. H., & Schulz, C. (2018) · Lipids · doi:10.1002/lipd.12092 · Cited by 13
The pink staircase of Sully-sur-Loire castle: Even bacteria like historic stonework
Leplat, J., Bousta, F., François, A., Guiavarc'H, M., Mertz, J., & Brissaud, D. (2019) · International Biodeterioration & Biodegradation · doi:10.1016/j.ibiod.2019.104805 · Cited by 13
Compositional data analysis of smoke emissions from debris piles with low-density polyethylene
Weise, D. R., Jung, H., Palarea‐Albaladejo, J., & Cocker, D. R. (2020) · Journal of the Air & Waste Management Association · doi:10.1080/10962247.2020.1784309 · Cited by 13
Direct and indirect trophic interactions of soil nematodes impact chickpea and oat nutrition
Gilarte, P., Plett, J. M., Pendall, E. G., Carrillo, Y., & Nielsen, U. N. (2020) · Plant and Soil · doi:10.1007/s11104-020-04735-6 · Cited by 13
“All analyses were completed in R version 3.6 (Bates et al. 2014; Bretz et al. 2016; Kuznetsova et al. 2017).”
MethodInfluential
An efficient method to produce segregating populations in quinoa (Chenopodium quinoa)
Emrani, N., Hasler, M., Patiranage, D. S. R., Maldonado‐Taipe, N., Rey, E., & Jung, C. (2020) · Plant Breeding · doi:10.1111/pbr.12873 · Cited by 13
“The ρ-values are based on appropriate multiple contrast tests (Bretz et al., 2011).”
MethodInfluential
Systematic Comparison of Commercial Hydrogels Revealed That a Synergy of Laminin and Strain-Stiffening Promotes Directed Migration of Neural Cells
Millesi, F., Mero, S., Semmler, L., Rad, A., Stadlmayr, S., Borger, A., Supper, P., Haertinger, M., Ploszczanski, L., Windberger, U. B., et al. (2023) · ACS Applied Materials & Interfaces · doi:10.1021/acsami.2c20040 · Cited by 13
ROC residuals in signal-detection models of recognition memory
Kellen, D., & Singmann, H. (2015) · Psychonomic Bulletin & Review · doi:10.3758/s13423-015-0888-2 · Cited by 13
“…the root-mean-squared error and MAD the median absolute deviation of the estimated marginal mean residuals from the LMM (i.e., estimates of the amount of deviation) into account (Bretz et al., 2010)3, restricting the overall Type I error probability to .05 for all tests conducted within each LMM.”
BackgroundMethod
A new class of powerful and informative simultaneous confidence intervals
Brannath, W., & Schmidt, S. (2014) · Statistics in Medicine · doi:10.1002/sim.6187 · Cited by 13
Cardiac autonomic activity during daytime nap in young adults
Cellini, N., Torre, J., Stegagno, L., & Sarlo, M. (2017) · Journal of Sleep Research · doi:10.1111/jsr.12539 · Cited by 13
“All statistical analyses were performed in R (R Core Team, 2015), using the libraries lme4 (Bates et al., 2015), multcomp (Bretz et al., 2011) and effects (Fox, 2003).”
Method
The CPCAT as a novel tool to overcome the shortcomings of NOEC/LOEC statistics in ecotoxicology: a simulation study to evaluate the statistical power
Lehmann, R., Bachmann, J., Karaoğlan, B., Lacker, J., Lurman, G. J., Polleichtner, C., Ratte, H. T., & Ratte, M. (2018) · Environmental Sciences Europe · doi:10.1186/s12302-018-0178-5 · Cited by 13
“CPCAT: a short overview The CPCAT is a combination of the closure principle (CP) [8] and the computational approach test (CAT) [9].”
BackgroundMethod
Singing behind the stage: thrush nightingales produce more variable songs on their wintering grounds
Souriau, A., Geberzahn, N., Иваницкий, В. В., Марова, И. М., Vokurková, J., Reifová, R., Reif, J., & Petrusková, T. (2019) · Behavioral Ecology and Sociobiology · doi:10.1007/s00265-019-2765-x · Cited by 13
“For LMMs that revealed a significant effect of any predictor variables, we used post hoc Tukey’s contrasts for multiple comparisons (multcomp package, Bretz et al. 2010), which provided P values adjusted by the single-step method.”
Method
Full greenhouse gas balance of silage maize cultivation following grassland: Are no-tillage practices favourable under highly productive soil conditions?
Struck, I. J. A., Taube, F., Hoffmann, M., Kluß, C., Herrmann, A., Loges, R., & Reinsch, T. (2020) · Soil and Tillage Research · doi:10.1016/j.still.2020.104615 · Cited by 13
Higher than expected N2O emissions from soybean crops in the Pampas Region of Argentina: Estimates from DayCent simulations and field measurements
Chiesa, T. D., Piñeiro, G., Grosso, S. J. D., Parton, W. J., Araujo, P. I., & Yahdjian, L. (2022) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2022.155408 · Cited by 17
Marginal Calluna populations are more resistant to climate change, but not under high-nitrogen loads
Meyer‐Grünefeldt, M., Belz, K., Calvo, L., Marcos, E., von Oheimb, G., & Härdtle, W. (2016) · Plant Ecology · doi:10.1007/s11258-015-0563-8 · Cited by 13
“2014), andmultcomp (Bretz et al. 2010).”
MethodInfluential
Effect of feed intake level and dietary protein content on the body temperature of pigs housed under thermo neutral conditions
Morales, A., Ibarra, N., Chávez, M., Gómez, T., Suárez, A. S. B., Valle, J. A., Morales, R. L. C., & Cervantes, M. (2017) · Journal of Animal Physiology and Animal Nutrition · doi:10.1111/jpn.12824 · Cited by 15
Increase in NREM sleep slow waves following injections of sodium oxybate in the mouse cerebral cortex and the role of somatostatin‐positive interneurons
Spano, G. M., Cavelli, M., Marshall, W., Tononi, G., & Cirelli, C. (2022) · European Journal of Neuroscience · doi:10.1111/ejn.15846 · Cited by 11
“We used a LRT to test for an effect of injection, and when appropriate, performed post hoc comparisons using the glht() function of the mult-comp package, and corrected for multiple comparisons using a single-step method (Bretz et al., 2011).”
Method
Examining the impact of agricultural drainage ditch management on in-stream bacterial communities involved in nitrogen cycling: insights from the Environmental Change One Health Observatory (ECO2)
Guo, G., Lapen, D. R., Khan, I. U. H., Sunohara, M. D., Craiovan, E., & Chen, W. (2024) · Frontiers in Sustainable Food Systems · doi:10.3389/fsufs.2024.1329422 · Cited by 12
Heatwave‐induced functional shifts in zooplankton communities result in weaker top‐down control on phytoplankton
Huỳnh, T., Horváth, Z., Pálffy, K., Kardos, V., Szabó, B., Dobosy, P., & Vad, C. F. (2024) · Ecology and Evolution · doi:10.1002/ece3.70096 · Cited by 15
“In case of a significant treatment ef - fect based on the GAMM, we performed a pairwise test for multiple comparisons with single-step p - value adjustment using the package “multcomp” (Bretz et al., 2010).”
Method
The Statistical Analysis of Small Data Sets
Neuhäuser, M., & Ruxton, G. D. (2024) · doi:10.1093/oso/9780198872979.001.0001 · Cited by 12
Effect of sample volume size and sampling method on feline longitudinal myocardial velocity profiles from color tissue Doppler imaging
Granström, S., Pipper, C. B., Møgelvang, R., Søgaard, P., Willesen, J. L., & Koch, J. (2012) · Journal of Veterinary Cardiology · doi:10.1016/j.jvc.2012.06.001 · Cited by 12
A factor-adjusted multiple testing procedure for ERP data analysis
Causeur, D., Chu, M. C., Hsieh, S., & Sheu, C. (2012) · Behavior Research Methods · doi:10.3758/s13428-012-0230-0 · Cited by 12
Flavonoids as putative modulators of Δ4‐, Δ5‐, and Δ6‐desaturases: Studies in cultured hepatocytes, myocytes, and adipocytes
Kühn, G., Pallauf, K., Schulz, C., & Rimbach, G. H. (2018) · BioFactors · doi:10.1002/biof.1443 · Cited by 12
Breathing with the mind: Effects of motor imagery on breath-hold performance
Kanthack, T. F. D., Guillot, A., Saboul, D., Debarnot, U., & Rienzo, F. D. (2019) · Physiology & Behavior · doi:10.1016/j.physbeh.2019.112583 · Cited by 12
In the corner of the eye: camouflaging motion in the peripheral visual field
Smart, I. E., Cuthill, I. C., & Scott‐Samuel, N. E. (2020) · Proceedings of the Royal Society B Biological Sciences · doi:10.1098/rspb.2019.2537 · Cited by 12
Aerobiological behavior of Paleolithic rock art sites in Dordogne (France): a comparative study in protected sites ranging from rock shelters to caves, with and without public access
Leplat, J., François, A., Touron, S., Frouin, M., Portais, J.-C., & Bousta, F. (2020) · Aerobiologia · doi:10.1007/s10453-020-09637-9 · Cited by 12
Pacing and Performance in Swimming: Differences Between Individual and Relay Events
McGibbon, K. E., Shephard, M. E., Osborne, M. A., Thompson, K., & Pyne, D. B. (2020) · International Journal of Sports Physiology and Performance · doi:10.1123/ijspp.2019-0381 · Cited by 12
Urinary Cortisol Increases During a Respiratory Outbreak in Wild Chimpanzees
Behringer, V., Preis, A., Wu, D. F., Crockford, C., Leendertz, F., Wittig, R., & Deschner, T. (2020) · Frontiers in Veterinary Science · doi:10.3389/fvets.2020.00485 · Cited by 15
“A post hoc comparison was then performed for each period (before, during, and after) using the function “glht” from the R-package “multcomp” (56, 57).”
Method
Similarity-induced interference or facilitation in language production reflects representation, not selection
Oppenheim, G. M., & Nozari, N. (2024) · Cognition · doi:10.1016/j.cognition.2024.105720 · Cited by 12
“For the analyses that compared multiple non-orthogonal contrasts, the mult-comp R package version 1.4 – 16 (Bretz, Hothorn, & Westfall, 2011) was used, which returns Tukey-corrected p-values for pairwise comparisons but does not tolerate interaction terms in the model.”
MethodInfluential
Ecological Efficiency of Maize-Based Cropping Systems for Biogas Production
Svoboda, N., Taube, F., Kluß, C., Wienforth, B., Sieling, K., Hasler, M., Kage, H., Ohl, S., Hartung, E., & Herrmann, A. (2015) · BioEnergy Research · doi:10.1007/s12155-015-9614-1 · Cited by 15
“, see [42]) to compare the levels of influence factors with respect to the parameters of the exponential function.”
Background
Intercropping fenugreek (Trigonella foenum‐graecum L.) and barley (Hordeum vulgare L.) with and without biochar: Tests along a competition gradient
Bitarafan, Z., Jensen, S. M., Liu, F., & Andreasen, C. (2018) · Journal of Agronomy and Crop Science · doi:10.1111/jac.12305 · Cited by 12
Con- and Heterospecific Shoaling Makes Invasive Guppies More Risk Taking
Santiago‐Arellano, A., Palomera-Hernández, V., & Camacho‐Cervantes, M. (2021) · Frontiers in Ecology and Evolution · doi:10.3389/fevo.2021.624245 · Cited by 13
“Frontiers in Ecology and Evolution | www.frontiersin.org 4 March 2021 | Volume 9 | Article 624245 p-value we carried out a Tukey post hoc (glht) analysis from the “multcomp” R package to identify the treatments that were significantly different (Bretz et al., 2016).”
Method
Spatial and temporal trends in the ecological risk posed by polycyclic aromatic hydrocarbons in Mediterranean Sea sediments using large-scale monitoring data
Rizzi, C., Villa, S., Chimera, C., Finizio, A., & Monti, G. S. (2021) · Ecological Indicators · doi:10.1016/j.ecolind.2021.107923 · Cited by 12
Alien vs. predator: influence of environmental variability and predation on the survival of ascidian recruits of a native and alien species
Gauff, R. P. M., Lejeusne, C., Arsenieff, L., Bohner, O., Coudret, J., Desbordes, F., Jandard, A., Loisel, S., Schires, G., Wafo, E., et al. (2022) · Biological Invasions · doi:10.1007/s10530-021-02720-3 · Cited by 12
Gradient Studies Reveal the True Drivers of Extreme Life in the Atacama Desert
Boy, D., Moeller, R., Sauheitl, L., Schaarschmidt, F., Rapp, S., van den Brink, L., Gschwendtner, S., Bórquez, R. G., Matus, F. J., Horn, M. A., et al. (2022) · Journal of Geophysical Research Biogeosciences · doi:10.1029/2021jg006714 · Cited by 12
“17) using one-way ANOVA followed by Tukey's post hoc test for significant differences along the gradients (Bretz et al., 2011).”
Method
Effects of forest canopy density and epixylic vegetation on nutrient concentrations in decaying logs of a subalpine fir forest
Wang, Z., Yang, W., Tan, B., Wang, Q., Wang, L., Chang, C., Cao, R., Jiang, Y., & Müller, J. C. (2022) · Forest Ecosystems · doi:10.1016/j.fecs.2022.100064 · Cited by 12
The effect of injury on whole-plant senescence: an experiment with two root-sproutingBarbareaspecies
Martínková, J., Šmilauer, P., Mihulka, S., Latzel, V., & Klimešová, J. (2016) · Annals of Botany · doi:10.1093/aob/mcw010 · Cited by 12
“…Core Team, 2013); LMMs were fitted using the ‘nlme’ package (Pinheiro and Bates, 2000); Cox proportional hazard models were fitted using the ‘survival’ package (Therneau and Grambsch, 2000); post-hoc tests and planned contrast tests were performed using the ‘multcomp’ package (Bretz et al., 2011).”
Method
Modelling response time in a mental rotation task by gender, physical activity, and task features
Fargier, P., Champely, S., Massarelli, R., Ammary, L., & Hoyek, N. (2022) · Scientific Reports · doi:10.1038/s41598-022-19054-2 · Cited by 12
The effect of tree decline over soil water content largely controls soil respiration dynamics in a Mediterranean woodland
Rodríguez, A., Durán, J., Yuste, J. C., Valladares, F., & Rey, A. (2023) · Agricultural and Forest Meteorology · doi:10.1016/j.agrformet.2023.109398 · Cited by 11
Tell-tale signals: faces reveal playful and aggressive mood in wolves
Maglieri, V., Mastrandrea, F., Galotti, A., Böye, M., Laurent, S., Marcolla, A., & Palagi, E. (2024) · Animal Behaviour · doi:10.1016/j.anbehav.2024.05.013 · Cited by 11
“We performed all pairwise comparisons for the levels of the multilevel factor with the Tukey test (Bretz et al., 2010) by using the R package emmeans (Lenth, 2023).”
Outcomes of collared uncemented stems in total hip arthroplasty for primary osteoarthritis: A Swiss National joint registry analysis
Hamoudi, C., Debordes, P.-A., Fils, J., Eberlová, L., Husmann, O., Fischer, J.-F., & Lunebourg, A. (2025) · Orthopaedics & Traumatology Surgery & Research · doi:10.1016/j.otsr.2025.104293 · Cited by 11
Three hulless oat varieties show economic potential as organic layer feed grain
Winkler, L. R., Murphy, K., & HERMES, J. C. (2017) · Renewable Agriculture and Food Systems · doi:10.1017/s1742170517000217 · Cited by 11
“All contrasts for a given dependent variable were pre-defined and evaluated simultaneously so that reported P values are adjusted for multiple comparisons (single-step procedure, Bretz et al., 2011).”
Method

Authors: Bretz, F., Hothorn, T., & Westfall, P.
Affiliations: Texas Tech University (US)
ORCID: Bretz, Frank
Hothorn, Torsten
Venue: Chapman & Hall/CRC eBooks
Publisher: Chapman and Hall/CRC
Published: 2016-04-19
DOI: 10.1201/9781420010909
OpenAlex: W396053900
Cited by: 1,233
References: 93
Citations by year: 2026: 20 · 2025: 57 · 2024: 96 · 2023: 70 · 2022: 97 · 2021: 146 · 2020: 99 · 2019: 133 · 2018: 98 · 2017: 89 · 2016: 78
FWCI: 12.36
Type: book
Language: en
Open access: closed
Locations: Chapman & Hall/CRC eBooks (publishedVersion)
Thammasat University Digital Collections (submittedVersion)
TU Digital Collections (Thammasat University) (submittedVersion)
Topics: Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
Physical Sciences › Mathematics › Statistics and Probability › Data Analysis with R
Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Applications
Keywords: multiple comparisons, R software, statistical software implementation, hypothesis testing

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Multiple Comparisons Using R

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks