The DOI matches its Crossref record; 44 of the 44 references it cites were found in a registry or an index.
Taking Parametric Assumptions Seriously: Arguments for the Use of Welch’s F-test instead of the Classical F-test in One-Way ANOVA
Open accessAbstract
Student’s t-test and classical F-test ANOVA rely on the assumptions that two or more samples are independent, and that independent and identically distributed residuals are normal and have equal variances between groups. We focus on the assumptions of normality and equality of variances, and argue that these assumptions are often unrealistic in the field of psychology. We underline the current lack of attention to these assumptions through an analysis of researchers’ practices. Through Monte Carlo simulations, we illustrate the consequences of performing the classic parametric F-test for ANOVA when the test assumptions are not met on the Type I error rate and statistical power. Under realistic deviations from the assumption of equal variances, the classic F-test can yield severely biased results and lead to invalid statistical inferences. We examine two common alternatives to the F-test, namely the Welch’s ANOVA (W-test) and the Brown-Forsythe test (F*-test). Our simulations show that under a range of realistic scenarios, the W-test is a better alternative and we therefore recommend using the W-test by default when comparing means. We provide a detailed example explaining how to perform the W-test in SPSS and R. We summarize our conclusions in practical recommendations that researchers can use to improve their statistical practices.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Henrich, J., Heine, S. J., & Norenzayan, A. (2010) · Nature · doi:10.1038/466029a · Cited by 3,441
Robust Tests for the Equality of Variances
Brown, M. B., & Forsythe, A. B. (1974) · Journal of the American Statistical Association · doi:10.1080/01621459.1974.10482955 · Cited by 3,216
Glass, G. V., Peckham, P. D., & Sanders, J. R. (1972) · Review of Educational Research · doi:10.3102/00346543042003237 · Cited by 1,826
On the meaning and use of kurtosis.
DeCarlo, L. T. (1997) · Psychological Methods · doi:10.1037//1082-989x.2.3.292 · Cited by 1,755
Box, G. E. P. (1954) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177728786 · Cited by 1,513
The unicorn, the normal curve, and other improbable creatures.
Micceri, T. (1989) · Psychological Bulletin · doi:10.1037//0033-2909.105.1.156 · Cited by 1,485
Cain, M. K., Zhang, Z., & Yuan, K.-H. (2017) · Behavior Research Methods · doi:10.3758/s13428-016-0814-1 · Cited by 1,164
Why Psychologists Should by Default Use Welch’s t-test Instead of Student’s t-test
Delacre, M., Lakens, D., & Leys, C. (2017) · International Review of Social Psychology · doi:10.5334/irsp.82 · Cited by 1,037
Self-citationHaar, J. M., Russo, M., Suñe, A., & Ollier-Malaterre, A. (2014) · Journal of Vocational Behavior · doi:10.1016/j.jvb.2014.08.010 · Cited by 965
Modern robust statistical methods: An easy way to maximize the accuracy and power of your research.
Erceg-Hurn, D. M., & Mirosevich, V. M. (2008) · American Psychologist · doi:10.1037/0003-066x.63.7.591 · Cited by 959
Attentional Routes to Conscious Perception
Chica, A. B., & Bartolomeo, P. (2012) · Frontiers in Psychology · doi:10.3389/fpsyg.2012.00001 · Cited by 921
The effects of violations of assumptions underlying the t test.
Boneau, C. A. (1960) · Psychological Bulletin · doi:10.1037/h0041412 · Cited by 790
Bradley, J. V. (1978) · British Journal of Mathematical and Statistical Psychology · doi:10.1111/j.2044-8317.1978.tb00581.x · Cited by 739
Keselman, H. J., Huberty, C. J., Lix, L. M., Olejnik, S., Cribbie, R. A., Donahue, B., Kowalchuk, R. K., Lowman, L. L., Petoskey, M. D., Keselman, J. C., et al. (1998) · Review of Educational Research · doi:10.2307/1170601 · Cited by 627
Kurtosis as Peakedness, 1905–2014. R.I.P.
Westfall, P. H. (2014) · The American Statistician · doi:10.1080/00031305.2014.917055 · Cited by 518
Harwell, M. R., Rubinstein, E. N., Hayes, W. S., & Olds, C. C. (1992) · Journal of Educational Statistics · doi:10.3102/10769986017004315 · Cited by 484
Lix, L. M., Keselman, J. C., & Keselman, H. J. (1996) · Review of Educational Research · doi:10.3102/00346543066004579 · Cited by 474
Lix, L. M., Keselman, J. C., & Keselman, H. J. (1996) · Review of Educational Research · doi:10.2307/1170654 · Cited by 367
Cohen, A. B., & Hill, P. C. (2007) · Journal of Personality · doi:10.1111/j.1467-6494.2007.00454.x · Cited by 359
Variation of Individualism and Collectivism within and between 20 Countries: A Typological Analysis
Green, E. G. T., Deschamps, J.-C., & Páez, D. (2005) · Journal of Cross-Cultural Psychology · doi:10.1177/0022022104273654 · Cited by 353
Comparison of ANOVA alternatives under variance heterogeneity and specific noncentrality structures.
Tomarken, A. J., & Serlin, R. C. (1986) · Psychological Bulletin · doi:10.1037//0033-2909.99.1.90 · Cited by 326
How many discoveries have been lost by ignoring modern statistical methods?
Wilcox, R. R. (1998) · American Psychologist · doi:10.1037//0003-066x.53.3.300 · Cited by 298
Statistical Assumptions of Substantive Analyses Across the General Linear Model: A Mini-Review
Nimon, K. F. (2012) · Frontiers in Psychology · doi:10.3389/fpsyg.2012.00322 · Cited by 225
Internal and external validity of the WHO Well‐Being Scale in the elderly general population
Heun, R., Burkart, M., Maier, W., & Bech, P. (1999) · Acta Psychiatrica Scandinavica · doi:10.1111/j.1600-0447.1999.tb00973.x · Cited by 160
Heterogeneity of variance in clinical data.
Grissom, R. J. (2000) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.68.1.155 · Cited by 156
Effects of Training on Interpretation of Emotional Ambiguity
Grey, S. J., & Mathews, A. (2000) · The Quarterly Journal of Experimental Psychology Section A · doi:10.1080/713755937 · Cited by 148
Hsu, T.-C., & Feldt, L. S. (1969) · American Educational Research Journal · doi:10.3102/00028312006004515 · Cited by 147
Are Assumptions of Well-Known Statistical Techniques Checked, and Why (Not)?
Hoekstra, R., Kiers, H. A. L., & Johnson, A. (2012) · Frontiers in Psychology · doi:10.3389/fpsyg.2012.00137 · Cited by 134
Heterogeneity of variance in experimental studies: A challenge to conventional interpretations.
Bryk, A. S., & Raudenbush, S. W. (1988) · Psychological Bulletin · doi:10.1037/0033-2909.104.3.396 · Cited by 134
THE EFFECT OF NON-NORMALITY ON THE POWER FUNCTION OF THE F-TEST IN THE ANALYSIS OF VARIANCE
David, F. N., & Johnson, N. L. (1951) · Biometrika · doi:10.1093/biomet/38.1-2.43 · Cited by 96
¿Qué? Quoi? Do Languages with Grammatical Gender Promote Sexist Attitudes?
Wasserman, B. D., & Weseley, A. J. (2009) · Sex Roles · doi:10.1007/s11199-009-9696-3 · Cited by 90
Power Function of the F -Test Under Non-Normal Situations
TIKU, M. L. (1971) · Journal of the American Statistical Association · doi:10.1080/01621459.1971.10482371 · Cited by 75
EFFECT OF NON-NORMALITY ON THE POWER OF THE ANALYSIS OF VARIANCE TEST
SRIVASTAVA, A. B. L. (1959) · Biometrika · doi:10.1093/biomet/46.1-2.114 · Cited by 75
Structural Equation Modeling with Heavy Tailed Distributions
Yuan, K., Bentler, P. M., & Chan, W. (2004) · Psychometrika · doi:10.1007/bf02295644 · Cited by 74
Liu, H. (2015) · VCU Scholars Compass (Virginia Commonwealth University) · doi:10.25772/bwfp-ye95 · Cited by 71
Power Function of the F-Test Under Non-Normal Situations
TIKU, M. L. (1971) · Journal of the American Statistical Association · doi:10.2307/2284254 · Cited by 57
Hsu, T.-C., & Feldt, L. S. (1969) · American Educational Research Journal · doi:10.2307/1162248 · Cited by 54
Promoting Gender-Fair Language: The Impact of Arguments on Language Use, Attitudes, and Cognitions
Koeser, S., & Sczesny, S. (2014) · Journal of Language and Social Psychology · doi:10.1177/0261927x14541280 · Cited by 52
Shared ethnicity effects on service encounters: A study across three U.S. subcultures
Montoya, D. Y., & Briggs, E. (2011) · Journal of Business Research · doi:10.1016/j.jbusres.2011.08.011 · Cited by 45
Tests That are Robust against Variance Heterogeneity in k × 2 Designs with Unequal Cell Frequencies
Overall, J. E., Atlas, R. S., & Gibson, J. M. (1995) · Psychological Reports · doi:10.2466/pr0.1995.76.3.1011 · Cited by 36
Identity in Descriptions of Others Across Ethnic Groups in South Africa
Adams, B. G., van de Vijver, F. J. R., de Bruin, G. P., & Torres, C. B. (2014) · Journal of Cross-Cultural Psychology · doi:10.1177/0022022114542466 · Cited by 35
Alexander and Govern's Approximation: Providing an Alternative to ANOVA Under Variance Heterogeneity
Schneider, P. J., & Penfield, D. A. (1997) · The Journal of Experimental Education · doi:10.1080/00220973.1997.9943459 · Cited by 34
Church, A. T., Wilmore, S. L., Anderson, A. T., Ochiai, M., Porter, N., Mateo, N. J., Reyes, J. A. S., Vargas‐Flores, J. D. J., Ibáñez‐Reyes, J., Alvarez, J. M., et al. (2011) · Journal of Cross-Cultural Psychology · doi:10.1177/0022022111428514 · Cited by 33
Beilmann, M., Mayer, B., Kasearu, K., & Realo, A. (2014) · Child Indicators Research · doi:10.1007/s12187-014-9232-z · Cited by 12
Cited by
Simulation-Based Power Analysis for Factorial Analysis of Variance Designs
Lakens, D., & Caldwell, A. R. (2021) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245920951503 · Cited by 602
Self-citationWagner, L., Corona, L. L., Weitlauf, A. S., Marsh, K. L., Berman, A. F., Broderick, N. A., Francis, S., Hine, J., Nicholson, A., Stone, C., et al. (2021) · Journal of Autism and Developmental Disorders · doi:10.1007/s10803-020-04767-y · Cited by 138
Validation and factor structure of the mindfulness-based self efficacy scale-revised
Cayoun, B., Elphinstone, B., Kasselis, N., Bilsborrow, G., & Skilbeck, C. (2022) · Mindfulness · doi:10.1007/s12671-022-01834-6 · Cited by 17
Metadata
- Authors
- Delacre, M., Leys, C., Mora, Y. L., & Lakens, D.
- Venue
- International Review of Social Psychology
- Publisher
- Ubiquity Press, Ltd.
- Published
- 2019-08-01
- Volume
- 32
- Issue
- 1
- Pages
- 13
- ISSN
- 0992-986X (ISSN-L)
- 2397-8570
- OpenAlex
- W2944604439
- Cited by
- 312
- References
- 44
- FWCI
- 20.14
- Type
- journal article
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.