The DOI matches its Crossref record; 57 of the 57 references it cites were found in a registry or an index.
Consequences of Assumption Violations Revisited: A Quantitative Review of Alternatives to the One-Way Analysis of Variance F Test
Abstract
The presence of variance heterogeneity and nonnormality in educational and psychological data may frequently invalidate the use of the analysis of variance (ANOVA) F test in one-way independent groups designs. This article offers recommendations to applied researchers on the use of various parametric and nonparametric alternatives to the F test under assumption violation conditions. Meta-analytic techniques were used to summarize the statistical robustness literature on the Type I error properties of the Brown-Forsythe (Brown & Forsythe, 1974 ), James (1951) second-order, Kruskal-Wallis ( Kruskal & Wallis, 1952 ), and Welch (1951) tests. Two variables, based on the theoretical work of Box (1954) , are shown to be highly effective in deciding when a particular alternative procedure should be adopted. Based on the meta-analysis findings, it is recommended that researchers gain a clear understanding of the nature of their data before conducting statistical analyses. Of all of the procedures, the James and Welch tests performed best under violations of the variance homogeneity assumption, although their sensitivity to certain types of nonnormality may preclude their use in all data-analytic situations. Opportunities for further methodological studies of ANOVA alternative procedures are also discussed.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Multiple regression: testing and interpreting interactions
(1992) · Choice Reviews Online · doi:10.5860/choice.29-3352 · Cited by 36,654
Use of Ranks in One-Criterion Variance Analysis
Kruskal, W. H., & Wallis, W. A. (1952) · Journal of the American Statistical Association · doi:10.1080/01621459.1952.10483441 · Cited by 13,912
From dissertation abstracts international
(1974) · ACM SIGGRAPH Computer Graphics · doi:10.1145/988616.988619 · Cited by 12,099
Statistical Methods for Meta-Analysis
Wasserman, S., Hedges, L. V., & Olkin, I. (1988) · Journal of Educational Statistics · doi:10.2307/1164953 · Cited by 10,326
SCHEFFE, H. (1960) · Soil Science · doi:10.1097/00010694-196006000-00016 · Cited by 5,516
Statistical Methods for Meta-Analysis.
Freeman, P. R., Hedges, L. V., & Olkin, I. (1986) · Biometrics · doi:10.2307/2531069 · Cited by 4,784
BMPD statistical software manual
Dixon, W. (1988) · University of California Press eBooks · Cited by 2,536
Glass, G. V., Peckham, P. D., & Sanders, J. R. (1972) · Review of Educational Research · doi:10.3102/00346543042003237 · Cited by 1,826 · cited 8×
“For example, until the 1990s the qualitative review of the statistical robustness litera ture by Glass et al. (1972) was often used as a benchmark for deciding under what conditions it was feasible to use the F test.”
BackgroundMethodInfluentialON THE COMPARISON OF SEVERAL MEAN VALUES: AN ALTERNATIVE APPROACH
WELCH, B. L. (1951) · Biometrika · doi:10.1093/biomet/38.3-4.330 · Cited by 1,727
Box, G. E. P. (1954) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177728786 · Cited by 1,513
The unicorn, the normal curve, and other improbable creatures.
Micceri, T. (1989) · Psychological Bulletin · doi:10.1037//0033-2909.105.1.156 · Cited by 1,485
The Handbook of Research Synthesis
Hanka, R. (1994) · BMJ · doi:10.1136/bmj.309.6952.488a · Cited by 1,052
Estimation of a Single Effect Size: Parametric and Nonparametric Methods
Hedges, L. V., & Olkin, I. (1985) · Statistical Methods for Meta-Analysis · doi:10.1016/b978-0-08-057065-5.50010-5 · Cited by 983
The effects of violations of assumptions underlying the t test.
Boneau, C. A. (1960) · Psychological Bulletin · doi:10.1037/h0041412 · Cited by 790
Bradley, J. V. (1978) · British Journal of Mathematical and Statistical Psychology · doi:10.1111/j.2044-8317.1978.tb00581.x · Cited by 739
The Small Sample Behavior of Some Statistics Which Test the Equality of Several Means
Brown, M. B., & Forsythe, A. B. (1974) · Technometrics · doi:10.1080/00401706.1974.10489158 · Cited by 663
The Assumptions Underlying the Analysis of Variance
Eisenhart, C. (1947) · Biometrics · doi:10.2307/3001534 · Cited by 634
Harwell, M. R., Rubinstein, E. N., Hayes, W. S., & Olds, C. C. (1992) · Journal of Educational Statistics · doi:10.3102/10769986017004315 · Cited by 484
Outliers and influential data points in regression analysis.
Stevens, J. P. (1984) · Psychological Bulletin · doi:10.1037//0033-2909.95.2.334 · Cited by 465
Hedges, L. V. (1994) · Cited by 397
Some Consequences When the Assumptions for the Analysis of Variance are not Satisfied
Cochran, W. G. (1947) · Biometrics · doi:10.2307/3001535 · Cited by 396
British Journal of Mathematical and Statistical Psychology
(2010) · British Journal of Mathematical and Statistical Psychology · doi:10.1111/(issn)2044-8317 · Cited by 385
Comparison of ANOVA alternatives under variance heterogeneity and specific noncentrality structures.
Tomarken, A. J., & Serlin, R. C. (1986) · Psychological Bulletin · doi:10.1037//0033-2909.99.1.90 · Cited by 326
New Designs in Analysis of Variance
Wilcox, R. R. (1987) · Annual Review of Psychology · doi:10.1146/annurev.ps.38.020187.000333 · Cited by 267
JAMES, G. S. (1951) · Biometrika · doi:10.1093/biomet/38.3-4.324 · Cited by 266
THE ANALYSIS OF VARIANCE IN CASES OF NON-NORMAL VARIATION
PEARSON, E. S. (1931) · Biometrika · doi:10.1093/biomet/23.1-2.114 · Cited by 263
Applied analysis of variance in behavioral science
Edwards, L. K. (1993) · M. Dekker eBooks · Cited by 233
Statistics for the Social Sciences
Reichardt, C. S., Novotny, J. A., & Wilcox, R. R. (1997) · Journal of the American Statistical Association · doi:10.2307/2965738 · Cited by 196
Neeleman, D. (1973) · Springer eBooks · doi:10.1007/978-94-011-7486-2_4 · Cited by 178
Nonparametric Tests of Interaction in Experimental Design
Sawilowsky, S. S. (1990) · Review of Educational Research · doi:10.2307/1170226 · Cited by 162
A New and Simpler Approximation for ANOVA Under Variance Heterogeneity
Alexander, R. A., & Govern, D. M. (1994) · Journal of Educational Statistics · doi:10.3102/10769986019002091 · Cited by 160
Kohr, R. L., & Games, P. A. (1974) · The Journal of Experimental Education · doi:10.1080/00220973.1974.10806305 · Cited by 146
Feir‐Walsh, B. J., & Toothaker, L. E. (1974) · Educational and Psychological Measurement · doi:10.1177/001316447403400406 · Cited by 145
New monte carlo results on the robustness of the anova f, w and f statistics
Wilcox, R. R., Charlin, V. L., & Thompson, K. L. (1986) · Communications in Statistics - Simulation and Computation · doi:10.1080/03610918608812553 · Cited by 142
Blair, R. C., & Higgins, J. J. (1980) · Journal of Educational Statistics · doi:10.3102/10769986005004309 · Cited by 113
ANOVA: A Paradigm for Low Power and Misleading Measures of Effect Size?
Wilcox, R. R. (1995) · Review of Educational Research · doi:10.3102/00346543065001051 · Cited by 108
A Review of Regression Diagnostics for Behavioral Research
Chätterjee, S., & Yılmaz, M. (1992) · Applied Psychological Measurement · doi:10.1177/014662169201600301 · Cited by 104
To Trim or Not to Trim: Tests of Location Equality Under Heteroscedasticityand Nonnormality
Lix, L. M., & Keselman, H. J. (1998) · Educational and Psychological Measurement · doi:10.1177/0013164498058003004 · Cited by 100
Self-citationAlgina, J. J., Oshima, T. C., & Lin, W.-Y. (1994) · Journal of Educational and Behavioral Statistics · doi:10.3102/10769986019003275 · Cited by 92
Parametric Alternatives to the Analysis of Variance
Clinch, J., & Keselman, H. J. (1982) · Journal of Educational Statistics · doi:10.3102/10769986007003207 · Cited by 81
Self-citationApproximate degrees of freedom tests: A unified perspective on testing for mean equality.
Lix, L. M., & Keselman, H. J. (1995) · Psychological Bulletin · doi:10.1037/0033-2909.117.3.547 · Cited by 79
Self-citationRobustness of the T Test: A Guide for Researchers on Effect of Violations of Assumptions
Havlicek, L. L., & Peterson, N. L. (1974) · Psychological Reports · doi:10.2466/pr0.1974.34.3c.1095 · Cited by 75
Summarizing Monte Carlo Results in Methodological Research
Harwell, M. R. (1992) · Journal of Educational Statistics · doi:10.3102/10769986017004297 · Cited by 72
A Monte Carlo Study Comparing Various Two-Sample Tests for Differences in Mean
Neave, H. R., & Granger, C. W. J. (1968) · Technometrics · doi:10.1080/00401706.1968.10490598 · Cited by 69
Power of the F test with skewed data: Should one transform or not?
Levine, D. W., & Dunlap, W. P. (1982) · Psychological Bulletin · doi:10.1037/0033-2909.92.1.272 · Cited by 69
A One-Way Random Effects Model for Trimmed Means
Wilcox, R. R. (1994) · Psychometrika · doi:10.1007/bf02296126 · Cited by 69
Wilcox, R. R. (1989) · Journal of Educational Statistics · doi:10.2307/1165019 · Cited by 59
Oshima, T. C., & Algina, J. J. (1992) · British Journal of Mathematical and Statistical Psychology · doi:10.1111/j.2044-8317.1992.tb00991.x · Cited by 51
Variance Stabilizing Transformations and the Power of the F Test
Budescu, D. V., & Appelbaum, M. I. (1981) · Journal of Educational Statistics · doi:10.3102/10769986006001055 · Cited by 51
Implementing the Welch-James Procedure with Factorial Designs
Algina, J. J., & Olejnik, S. F. (1984) · Educational and Psychological Measurement · doi:10.1177/0013164484441004 · Cited by 35
An evaluation of parametric and non‐parametric tests on modified and non‐modified data
Rasmussen, J. L. (1986) · British Journal of Mathematical and Statistical Psychology · doi:10.1111/j.2044-8317.1986.tb00858.x · Cited by 34
Some Alternative Approximate Tests for a Split Plot Design
Algina, J. J. (1994) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr2904_3 · Cited by 32
Levy, K. J. (1978) · Journal of Statistical Computation and Simulation · doi:10.1080/00949657808810247 · Cited by 30
Performance of Four Multivariate Tests Under Variance-Covariance Heteroscedasticity
Tang, K. L., & Algina, J. J. (1993) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr2804_1 · Cited by 27
Wilcox, R. R. (1993) · Cited by 22
The Power of Student's t and Wilcoxon W Statistics
Rasmussen, J. L. (1985) · Evaluation Review · doi:10.1177/0193841x8500900407 · Cited by 18
Comment on a Wilcox Test Statistic for Comparing Means When Variances Are Unequal
Hsiung, T., Olejnik, S. F., & Huberty, C. J. (1994) · Journal of Educational Statistics · doi:10.3102/10769986019002111 · Cited by 18
Cited by
Non-normal data: Is ANOVA still a valid option?
Blanca, M., Alarcón, R., Arnau, J., Bono, R., & Bendayan, R. (2017) · Psicothema · doi:10.7334/psicothema2016.383 · Cited by 1,624
“…rates were within the bounds of Bradley’s criterion under certain departures from normality (Black et al., 2010; Clinch & Keselman, 1982; Feir-Walsh & Thoothaker, 1974; Gamage & Weerahandi, 1998; Kanji, 1976; Lantz, 2013; Lix et al., 1996; Patrick, 2007; Schmider et al., 2010; Zijlstra, 2004).”
InfluentialSchmider, E., Ziegler, M., Danay, E., Beyer, L., & Bühner, M. (2010) · Methodology · doi:10.1027/1614-2241/a000016 · Cited by 1,290
Modern robust statistical methods: An easy way to maximize the accuracy and power of your research.
Erceg-Hurn, D. M., & Mirosevich, V. M. (2008) · American Psychologist · doi:10.1037/0003-066x.63.7.591 · Cited by 959
Lo, S., & Andrews, S. (2015) · Frontiers in Psychology · doi:10.3389/fpsyg.2015.01171 · Cited by 973
Workplace harassment: Double jeopardy for minority women.
Berdahl, J. L., & Moore, C. (2006) · Journal of Applied Psychology · doi:10.1037/0021-9010.91.2.426 · Cited by 643
Metadata
- Authors
- Lix, L. M., Keselman, J. C., & Keselman, H. J.
- Venue
- Review of Educational Research
- Published
- 1996-12
- Volume
- 66
- Issue
- 4
- Pages
- 579-619
- ISSN
- 0034-6543 (ISSN-L)
- 1935-1046
- OpenAlex
- W2142137317
- Cited by
- 474
- References
- 57
- FWCI
- 1.98
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.