The DOI matches its Crossref record; 94 of the 94 references it cites were found in a registry or an index.
Multiple Comparisons Using R
Abstract
Adopting a unifying theme based on maximum statistics, Multiple Comparisons Using R describes the common underlying theory of multiple comparison procedures through numerous examples. It also presents a detailed description of available software implementations in R. The R packages and source code for the analyses are available at https://CRAN.R-pro
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“After applying the Bonferroni correction for evaluating multiple probabilities (Bretz et al. 2010), only the projections of areas 7A and DP to area 10 were inconsistent despite a statistically very low probability of being so.”
From A Weighted and Directed Interareal Connectivity Matrix for Macaque Cerebral Cortex (2014) · cites this paper as Bretz et al. 2010
Citations
Cites
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482
Simultaneous Inference in General Parametric Models
Hothorn, T., Bretz, F., & Westfall, P. H. (2008) · Biometrical Journal · doi:10.1002/bimj.200810425 · Cited by 14,265
Self-citationGentleman R: R: A language for data analysis and graphics
Ihaka, R., & Gentleman, R. (1996) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.1996.10474713 · Cited by 11,747
The control of the false discovery rate in multiple testing under dependency
Benjamini, Y., & Yekutieli, D. (2001) · The Annals of Statistics · doi:10.1214/aos/1013699998 · Cited by 10,490
A Multiple Comparison Procedure for Comparing Several Treatments with a Control
Dunnett, C. W. (1955) · Journal of the American Statistical Association · doi:10.1080/01621459.1955.10501294 · Cited by 6,097
A Direct Approach to False Discovery Rates
Storey, J. D. (2002) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/1467-9868.00346 · Cited by 5,603
A Multiple Testing Procedure for Clinical Trials
O'Brien, P. C., & Fleming, T. R. (1979) · Biometrics · doi:10.2307/2530245 · Cited by 3,311
Rectangular Confidence Regions for the Means of Multivariate Normal Distributions
Šidák, Z. (1967) · Journal of the American Statistical Association · doi:10.1080/01621459.1967.10482935 · Cited by 2,904
An improved Bonferroni procedure for multiple tests of significance
SIMES, R. J. (1986) · Biometrika · doi:10.1093/biomet/73.3.751 · Cited by 2,396
Discrete sequential boundaries for clinical trials
GORDON LAN, K. K., & DEMETS, D. L. (1983) · Biometrika · doi:10.1093/biomet/70.3.659 · Cited by 2,310
The positive false discovery rate: a Bayesian interpretation and the q-value
Storey, J. D. (2003) · The Annals of Statistics · doi:10.1214/aos/1074290335 · Cited by 2,204
On the adaptive control of the false discovery rate in multiple testing with independent statistics
Benjamini, Y., & Hochberg, Y. (2000) · Journal of Educational and Behavioral Statistics · doi:10.3102/10769986025001060 · Cited by 2,015
Group sequential methods in the design and analysis of clinical trials
POCOCK, S. J. (1977) · Biometrika · doi:10.1093/biomet/64.2.191 · Cited by 1,831
On closed testing procedures with special reference to ordered analysis of variance
Marcus, R., Peritz, E., & Gabriel, K. R. (1976) · Biometrika · doi:10.1093/biomet/63.3.655 · Cited by 1,417
A Stagewise Rejective Multiple Test Procedure Based on a Modified Bonferroni Test
Hommel, G. (1988) · Biometrika · doi:10.1093/biomet/75.2.383 · Cited by 1,367
Adjusted P-Values for Simultaneous Inference
Wright, S. P. (1992) · Biometrics · doi:10.2307/2532694 · Cited by 1,026
Resampling-Based Multiple Testing.
Booth, J. G., Westfall, P. H., & Young, S. S. (1994) · Journal of the American Statistical Association · doi:10.2307/2291234 · Cited by 909
Self-citationA Lego System for Conditional Inference
Hothorn, T., Hornik, K., van de Wiel, M. A., & Zeileis, A. (2006) · The American Statistician · doi:10.1198/000313006x118430 · Cited by 894
Self-citationEvaluation of Experiments with Adaptive Interim Analyses
Bauer, P., & Kohne, K. (1994) · Biometrics · doi:10.2307/2533441 · Cited by 869
Modified Sequentially Rejective Multiple Test Procedures
Shaffer, J. P. (1986) · Journal of the American Statistical Association · doi:10.1080/01621459.1986.10478341 · Cited by 808
Williams, D. A. (1971) · Biometrics · doi:10.2307/2528930 · Cited by 780
Object-Oriented Computation of Sandwich Estimators
Zeileis, A. (2006) · Journal of Statistical Software · doi:10.18637/jss.v016.i09 · Cited by 699
On a Heuristic Method of Test Construction and its use in Multivariate Analysis
Roy, S. (1953) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177729029 · Cited by 631
Adaptive Sample Size Calculations in Group Sequential Trials
Lehmacher, W., & Wassmer, G. (1999) · Biometrics · doi:10.1111/j.0006-341x.1999.01286.x · Cited by 599
Modification of Sample Size in Group Sequential Clinical Trials
Cui, L., Hung, H. M. J., & Wang, S. (1999) · Biometrics · doi:10.1111/j.0006-341x.1999.00853.x · Cited by 574
Designed Extension of Studies Based on Conditional Power
Proschan, M. A., & Hunsberger, S. (1995) · Biometrics · doi:10.2307/2533262 · Cited by 536
An Algorithm for a Letter-Based Representation of All-Pairwise Comparisons
Piepho, H. (2004) · Journal of Computational and Graphical Statistics · doi:10.1198/1061860043515 · Cited by 492
Testing the Statistical Certainty of a Response to Increasing Doses of a Drug
Tukey, J. W., Ciminera, J. L., & Heyse, J. F. (1985) · Biometrics · doi:10.2307/2530666 · Cited by 484
Plots of P-Values to Evaluate Many Tests Simultaneously
Schweder, T., & Spjøtvoll, E. (1982) · Biometrika · doi:10.1093/biomet/69.3.493 · Cited by 455
Müller, H., & Schäfer, H. (2001) · Biometrics · doi:10.1111/j.0006-341x.2001.00886.x · Cited by 374
Simultaneous Test Procedures--Some Theory of Multiple Comparisons
Gabriel, K. R. (1969) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177697819 · Cited by 367
Approximately Optimal One-Parameter Boundaries for Group Sequential Trials
Wang, S. K., & Tsiatis, A. A. (1987) · Biometrics · doi:10.2307/2531959 · Cited by 341
The Simes Method for Multiple Hypothesis Testing With Positively Dependent Test Statistics
Sarkar, S. K., & Chang, C.-K. (1997) · Journal of the American Statistical Association · doi:10.1080/01621459.1997.10473682 · Cited by 336
On the Asymptotic Theory of Permutation Statistics
Strasser, H., & Weber, C. (1999) · ePubWU Institutional Repository (Wirtschaftsuniversität Wien) · doi:10.57938/ff565ba0-aa64-4fe0-a158-86fd331bee78 · Cited by 256
Generalizations of the Familywise Error Rate
Lehmann, E. L., & Romano, J. P. (2011) · Springer eBooks · doi:10.1007/978-1-4614-1412-4_57 · Cited by 253
The Simes Method for Multiple Hypothesis Testing With Positively Dependent Test Statistics
Sarkar, S. K., & Chang, C.-K. (1997) · Journal of the American Statistical Association · doi:10.2307/2965431 · Cited by 253
Group sequential designs using a family of type I error probability spending functions
Hwang, I. K., Shih, W. J., & de Cani, J. S. (1990) · Statistics in Medicine · doi:10.1002/sim.4780091207 · Cited by 247
Testing and estimation in flexible group sequential designs with adaptive treatment selection
Posch, M., König, F., Branson, M., Brannath, W., Dunger‐Baldauf, C., & Bauer, P. (2005) · Statistics in Medicine · doi:10.1002/sim.2389 · Cited by 244
Multiple Testing of General Contrasts Using Logical Constraints and Correlations
Westfall, P. H. (1997) · Journal of the American Statistical Association · doi:10.1080/01621459.1997.10473627 · Cited by 223
Self-citationvan der Laan, M. J., Dudoit, S., & Pollard, K. S. (2004) · Statistical Applications in Genetics and Molecular Biology · doi:10.2202/1544-6115.1042 · Cited by 180
Simultaneous and selective inference: Current successes and future challenges
Benjamini, Y. (2010) · Biometrical Journal · doi:10.1002/bimj.200900299 · Cited by 169
Improvements of General Multiple Test Procedures for Redundant Systems of Hypotheses
Bergmann, B., & Hommel, G. (1988) · Medizinische Informatik und Statistik · doi:10.1007/978-3-642-52307-6_8 · Cited by 168
Step‐down multiple tests for comparing treatments with a control in unbalanced one‐way layouts
Dunnett, C. W., & Tamhane, A. C. (1991) · Statistics in Medicine · doi:10.1002/sim.4780100614 · Cited by 156
A fixed sequence Bonferroni procedure for testing multiple endpoints
Wiens, B. L. (2003) · Pharmaceutical Statistics · doi:10.1002/pst.64 · Cited by 156
Genz, A. C., & Bretz, F. (1999) · Journal of Statistical Computation and Simulation · doi:10.1080/00949659908811962 · Cited by 154
Self-citation(1998) · Materials science and technology/Materials science & technology · doi:10.1201/9781420049909.ch6 · Cited by 153
Ordered tests in the analysis of variance
Bartholomew, D. J. (1961) · Biometrika · doi:10.1093/biomet/48.3-4.325 · Cited by 144
Straßburger, K., & Bretz, F. (2008) · Statistics in Medicine · doi:10.1002/sim.3338 · Cited by 128
Self-citationThe Fallback Procedure for Evaluating a Single Family of Hypotheses
Wiens, B. L., & Dmitrienko, A. A. (2005) · Journal of Biopharmaceutical Statistics · doi:10.1080/10543400500265660 · Cited by 127
Schmidli, H., Bretz, F., Racine, A., & Maurer, W. (2006) · Biometrical Journal · doi:10.1002/bimj.200510231 · Cited by 127
Self-citationA Theory of Some Multiple Decision Problems, I
Lehmann, E. L. (2011) · Springer eBooks · doi:10.1007/978-1-4614-1412-4_53 · Cited by 126
Stepwise Confidence Intervals without Multiplicity Adjustment for Dose—Response and Toxicity Studies
Hsu, J. C., & Berger, R. L. (1999) · Journal of the American Statistical Association · doi:10.1080/01621459.1999.10474141 · Cited by 125
The partitioning principle: a powerful tool in multiple decision theory
Finner, H., & Straßburger, K. (2002) · The Annals of Statistics · doi:10.1214/aos/1031689023 · Cited by 123
Multiple Testing with Minimal Assumptions
Westfall, P. H., & Troendle, J. F. (2008) · Biometrical Journal · doi:10.1002/bimj.200710456 · Cited by 117
Self-citationComparison of Methods for Estimating the Number of True Null Hypotheses in Multiplicity Testing
Hsueh, H., Chen, J. J., & Kodell, R. L. (2003) · Journal of Biopharmaceutical Statistics · doi:10.1081/bip-120024202 · Cited by 113
Power and sample size when multiple endpoints are considered
Senn, S. J., & Bretz, F. (2007) · Pharmaceutical Statistics · doi:10.1002/pst.301 · Cited by 111
Self-citationRegression models: calculating the confidence interval of effects in the presence of interactions
Figueiras, A., Domenech, J. M., & Cadarso-Suárez, C. (1998) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19980930)17:18 · Cited by 104
Self-citationOn Confidence Sets in Multiple Comparisons
Stefánsson, G., Kim, W.-C., & Hsu, J. C. (1988) · Springer eBooks · doi:10.1007/978-1-4612-3818-8_8 · Cited by 101
Methods for the Computation of Multivariate t-Probabilities ∗
Genz, A. C., Bretz, F., & Bioinformatik, L. (2002) · Cited by 91
Self-citationTests of the Overall Hypothesis for Arbitrary Dependence Structures
Hommel, G. (1983) · Biometrical Journal · doi:10.1002/bimj.19830250502 · Cited by 90
Improved Prediction of Body Fat by Measuring Skinfold Thickness, Circumferences, and Bone Breadths
García, A. L., Wagner, K., Hothorn, T., Koebnick, C., Zunft, H.-J. F., & Trippo, U. (2005) · Obesity Research · doi:10.1038/oby.2005.67 · Cited by 73
Self-citationFinner, H., & Gontscharuk, V. (2009) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.1467-9868.2009.00719.x · Cited by 69
Control of Directional Errors with Stagewise Multiple Test Procedures
Shaffer, J. P. (1980) · The Annals of Statistics · doi:10.1214/aos/1176345205 · Cited by 66
Pairwise Comparisons of Generally Correlated Means
Hayter, A. J. (1989) · Journal of the American Statistical Association · doi:10.1080/01621459.1989.10478757 · Cited by 65
Some selection rules for comparing p processes with a standard
Naik, U. D. (1975) · Communications in Statistics · doi:10.1080/03610927508827267 · Cited by 64
Rosenthal, R., & Rubin, D. B. (1983) · Psychological Bulletin · doi:10.1037/0033-2909.94.3.540 · Cited by 62
Stepwise multiple test procedures and control of directional errors
Finner, H. (1999) · The Annals of Statistics · doi:10.1214/aos/1018031111 · Cited by 61
Hommel, G., & Bernhard, G. (1993) · Contributions to statistics · doi:10.1007/978-3-642-99766-2_10 · Cited by 58
Analysis of Dose–Response Studies—Modeling Approaches
Pinheiro, J. C., Bretz, F., & Branson, M. (2006) · Springer eBooks · doi:10.1007/0-387-33706-7_10 · Cited by 57
Self-citationClosed procedures are better and often admit a shortcut
Grechanovsky, E., & Hochberg, Y. (1999) · Journal of Statistical Planning and Inference · doi:10.1016/s0378-3758(98)00125-6 · Cited by 44
General Solutions to Multiple Testing Problems
Sonnemann, E. (2008) · Biometrical Journal · doi:10.1002/bimj.200810462 · Cited by 44
Rosenthal, R., & Rubin, D. B. (1983) · Psychological Bulletin · doi:10.1037//0033-2909.94.3.540 · Cited by 44
Weighted FWE-controlling methods in high-dimensional situations
Westfall, P. H., Kropf, S., & Finos, L. (2004) · Lecture notes-monograph series · doi:10.1214/lnms/1196285632 · Cited by 38
Self-citationShortcuts for Locally Consonant Closed Test Procedures
Brannath, W., & Bretz, F. (2010) · Journal of the American Statistical Association · doi:10.1198/jasa.2010.tm08127 · Cited by 34
Self-citationMultiple Test Procedures for Identifying the Maximum Safe Dose
Tamhane, A. C., Dunnett, C. W., Green, J. W., & Wetherington, J. D. (2001) · Journal of the American Statistical Association · doi:10.1198/016214501753208546 · Cited by 34
Aesthetics and Power Considerations in Multiple Testing – A Contradiction?
Hommel, G., & Bretz, F. (2008) · Biometrical Journal · doi:10.1002/bimj.200710463 · Cited by 34
Self-citationMultiple testing of pairs of one-sided hypotheses
Bauer, P., Hackl, P., Hommel, G., & Sonnemann, E. (1986) · Metrika · doi:10.1007/bf01894737 · Cited by 33
Bonferroni procedures for logically related hypotheses
Hommel, G., & Bernhard, G. (1999) · Journal of Statistical Planning and Inference · doi:10.1016/s0378-3758(99)00035-x · Cited by 32
Generalized Maximum Range Tests for Pairwise Comparisons of Several Populations
Royen, T. (1989) · Biometrical Journal · doi:10.1002/bimj.4710310804 · Cited by 26
Step Down Procedures for Comparison With a Control
Bofinger, E. (1987) · Australian Journal of Statistics · doi:10.1111/j.1467-842x.1987.tb00751.x · Cited by 26
Exploratory Data Analysis and Clinical Research
Victor, N. (1982) · Methods of Information in Medicine · doi:10.1055/s-0038-1635393 · Cited by 26
Comparison of treatments in a combination therapy trial
Patel, H. I. (1991) · Journal of Biopharmaceutical Statistics · doi:10.1080/10543409108835016 · Cited by 25
Partitioning to Uncover Conditions for Permutation Tests to Control Multiple Testing Error Rates
Calian, V., Li, D., & Hsu, J. C. (2008) · Biometrical Journal · doi:10.1002/bimj.200710471 · Cited by 23
Blacker, E. (1960) · New Zealand Engineering · Cited by 23
On improving the min test for the analysis of combination drug trials
Sarkar, S. K., Snapinn, S., & Wang, W. (1995) · Journal of Statistical Computation and Simulation · doi:10.1080/00949659508811632 · Cited by 17
Westfall, P. H. (2005) · Encyclopedia of Biostatistics · doi:10.1002/0470011815.b2a15181 · Cited by 14
Self-citationAbgeschlossene multiple Spannweitentests
Finner, H. (1988) · Medizinische Informatik und Statistik · doi:10.1007/978-3-642-52307-6_2 · Cited by 10
Multiple Comparison Procedures in Dose Response Studies
Tamhane, A. C., & Logan, B. R. (2006) · Springer eBooks · doi:10.1007/0-387-33706-7_11 · Cited by 9
On δ-Equivalence With the Best in k-Sample Models
Finner, H., & Straßburger, K. (2006) · Journal of the American Statistical Association · doi:10.1198/016214505000001014 · Cited by 8
Mean–Mean Multiple Comparison Displays for Families of Linear Contrasts
Heiberger, R. M., & Holland, B. S. (2006) · Journal of Computational and Graphical Statistics · doi:10.1198/106186006x160500 · Cited by 6
Multiple comparisons with more than one control
Spurrier, J. D., & Solorzano, E. (2004) · Lecture notes-monograph series · doi:10.1214/lnms/1196285630 · Cited by 4
Wassmer, G. (2005) · Encyclopedia of Statistical Sciences · doi:10.1002/0471667196.ess7175 · Cited by 3
Christensen (1997) · doi:10.1002/(sici)1099-1166(199703)12:3
Flagged first, then by citations
Cited by
Sorted by citations
Vrieze, A., Van Nood, E., Holleman, F., Salojärvi, J., Kootte, R. S., Bartelsman, J. F. W. M., Dallinga–Thie, G. M., Ackermans, M. T., Serlie, M. J., Oozeer, R., et al. (2012) · Gastroenterology · doi:10.1053/j.gastro.2012.06.031 · Cited by 2,913
Blanche, P., Dartigues, J., & Jacqmin‐Gadda, H. (2013) · Statistics in Medicine · doi:10.1002/sim.5958 · Cited by 1,917
A Weighted and Directed Interareal Connectivity Matrix for Macaque Cerebral Cortex
Markov, N. T., Ercsey-Ravasz, M. M., Ribeiro Gomes, A. R., Lamy, C., Magrou, L., Vezoli, J., Misery, P., Falchier, A., Quilodran, R., Gariel, M. A., et al. (2014) · Cerebral Cortex · doi:10.1093/cercor/bhs270 · Cited by 857
“ing multiple probabilities (Bretz et al. 2010), only the projections of areas 7A and DP to area 10 were inconsistent despite a statistically very low probability of being so.”
BackgroundMethodMicroplastic Shape, Polymer Type, and Concentration Affect Soil Properties and Plant Biomass
Lozano, Y. M., Lehnert, T., Linck, L. T., Lehmann, A., & Rillig, M. C. (2021) · Frontiers in Plant Science · doi:10.3389/fpls.2021.616645 · Cited by 697
“Then, we performed multiple comparisons “glht” among treatments by using the “Tukey” test and the function “sandwich” from the eponymous R package; this function provided a heteroscedasticity-consistent estimate of the covariance matrix (Zeileis, 2006; Bretz et al., 2011).”
MethodSimulation-Based Power Analysis for Factorial Analysis of Variance Designs
Lakens, D., & Caldwell, A. R. (2021) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245920951503 · Cited by 602
“The Holm procedure is slightly more powerful than the Bonferroni correction, without requiring additional assumptions (for other approaches, see Bretz et al., 2011).”
Method
Metadata
- Authors
- Bretz, F., Hothorn, T., & Westfall, P.
- Affiliations
- Texas Tech University (US)
- Venue
- Chapman & Hall/CRC eBooks
- Publisher
- Chapman and Hall/CRC
- Published
- 2016-04-19
- OpenAlex
- W396053900
- Cited by
- 1,233
- References
- 93
- Citations by year
- 2026: 20 · 2025: 57 · 2024: 96 · 2023: 70 · 2022: 97 · 2021: 146 · 2020: 99 · 2019: 133 · 2018: 98 · 2017: 89 · 2016: 78
- FWCI
- 12.36
- Type
- book
- Language
- en
- Open access
- closed
- Locations
- Chapman & Hall/CRC eBooks (publishedVersion)
- Thammasat University Digital Collections (submittedVersion)
- TU Digital Collections (Thammasat University) (submittedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Physical Sciences › Mathematics › Statistics and Probability › Data Analysis with R
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Applications
- Keywords
- multiple comparisons, R software, statistical software implementation, hypothesis testing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
