The DOI matches its Crossref record; 93 of the 95 references it cites were found in a registry or an index.
Consequences of Failure to Meet Assumptions Underlying the Fixed Effects Analyses of Variance and Covariance
Abstract
The effects of violating the assumptions underlying the fixed-effects analyses of variance (ANOVA) and covariance (ANCOVA) on Type-I and Type-II error rates have been of great concern to researchers and statisticians. The major effects of violation of assumptions are now well known, after nearly four decades of research. Early summaries and reviews by Hey (1938), Garret and Zubin (1943), Grant (1944), and Gourlay (1955) and more recent reviews by Bradley (1963), Atiqullah (1967), Elashoff (1969) and Scheffe (1959, Ch. 10) can be extended and updated with recent research which provides closure to an area of active inquiry. (For a review of the effects of violation of the assumptions of the randomeffects ANOVA-a subject not reviewed here-the reader is directed to Scheffe, 1959, pp. 334-337 and Box & Anderson, 1962.) Asking whether ANOVA and ANCOVA assumptions are satisfied is not idle curiosity. The assumptions of most mathematical models are always false to a greater or lesser extent. The relevant question is not whether ANOVA assumptions are met exactly, but rather whether the plausible violations of the assumptions have serious consequences on the validity of probability statements based on the standard assumptions. Applied statistics in education and the social sciences experienced a largely unnecessary hegira to non-parametric statistics during the 1950s. Increasingly during the 1950s and early 1960s the fixed-effects, normal theory ANOVA was replaced by such comparable no…
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“We focus on fairly drastic non-normality because only little bias can be expected from minor violations (Bishara & Hittner, 2012; Glass et al., 1972; Hack, 1958; Puth et al., 2014).”
From Violating the normality assumption may be the lesser of two evils (2021) · cites this paper as Glass et al., 1972
In the citing paper
“For instance, when applying a t test that assumes equal variances in both groups to data that come from substantially different variances (e.g., σ12/ σ22 = 0.1), then high rates of type I errors (e.g., 23%) may be obtained in a situation where sample sizes are unbalanced (N1 = 15, N2 = 5), namely when the small sample comes from the more variable group (Glass et al., 1972; Miller, 1986).”
From Violating the normality assumption may be the lesser of two evils (2021) · cites this paper as Glass et al., 1972
In the citing paper
“Violating this assumption may result in more notable increases of type I errors (compared to what we examined here) at least when the violations are drastic.”
From Violating the normality assumption may be the lesser of two evils (2021) · cites this paper as Glass et al., 1972
Citations
Cites
Nonparametric Statistics for the Behavioral Sciences
Guitton, H., & Siegel, S. (1958) · Revue économique · doi:10.2307/3498751 · Cited by 35,279
Statistical principles in experimental design.
Winer, B. J. (1962) · McGraw-Hill Book Company · doi:10.1037/11774-000 · Cited by 27,137
Principles and Procedures of Statistics
Robinson, H. F. (1961) · Agronomy Journal · doi:10.2134/agronj1961.00021962005300050002x · Cited by 17,727
An Analysis of Transformations
Box, G. E. P., & Cox, D. R. (1964) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.2517-6161.1964.tb00553.x · Cited by 15,356
Statistical Methods for Research Workers
Fisher, R. A. (1945) · American Journal of Clinical Pathology · doi:10.1093/ajcp/15.11.552b · Cited by 10,883
Nonparametric statistics for the behavioral sciences
McGinn, L. C. (1957) · Journal of the Franklin Institute · doi:10.1016/0016-0032(57)90766-4 · Cited by 7,284
Journal of The American Statistical Association
Fienberg, S. E. (2004) · Encyclopedia of Statistical Sciences · doi:10.1002/0471667196.ess1317 · Cited by 6,289
Scheffé, H. (1960) · Soil Science · doi:10.1097/00010694-196006000-00016 · Cited by 5,625
On Methods in the Analysis of Profile Data
Greenhouse, S. W., & Geisser, S. (1959) · Psychometrika · doi:10.1007/bf02289823 · Cited by 4,946
THE GENERALIZATION OF ‘STUDENT'S’ PROBLEM WHEN SEVERAL DIFFERENT POPULATION VARLANCES ARE INVOLVED
WELCH, B. L. (1947) · Biometrika · doi:10.1093/biomet/34.1-2.28 · Cited by 4,135
Robust tests for equality of variances
Levene, H. (1961) · Medical Entomology and Zoology · Cited by 3,271
Principles and Procedures of Statistics.
David, F. N., Steel, R. G. D., & Torrie, J. H. (1961) · Biometrika · doi:10.2307/2333165 · Cited by 2,402
The Advanced Theory of Statistics
L., S., Kendall, M. G., & Stuart, A. (1963) · Population · doi:10.2307/1527192 · Cited by 1,840
ON THE COMPARISON OF SEVERAL MEAN VALUES: AN ALTERNATIVE APPROACH
Welch, B. L. (1951) · Biometrika · doi:10.1093/biomet/38.3-4.330 · Cited by 1,531
Box, G. E. P. (1954) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177728786 · Cited by 1,528
THE COMPARISON OF PERCENTAGES IN MATCHED SAMPLES
COCHRAN, W. G. (1950) · Biometrika · doi:10.1093/biomet/37.3-4.256 · Cited by 1,465
Biometrika Tables for Statisticians
Savage, I. R., Pearson, E. S., & Hartley, H. O. (1967) · Mathematics of Computation · doi:10.2307/2004190 · Cited by 1,440
Bartlett, M. S. (1947) · Biometrics · doi:10.2307/3001536 · Cited by 1,253
Pearson, K. (1897) · Proceedings of the Royal Society of London · doi:10.1098/rspl.1896.0076 · Cited by 1,166
(1958) · Medical Entomology and Zoology · Cited by 1,059
One Degree of Freedom for Non-Additivity
Tukey, J. W. (1949) · Biometrics · doi:10.2307/3001938 · Cited by 912
Statistical Methods for Research Workers.
Hotelling, H., & Fisher, R. A. (1933) · Journal of the American Statistical Association · doi:10.2307/2278451 · Cited by 861
The effects of violations of assumptions underlying the t test.
Boneau, C. A. (1960) · Psychological Bulletin · doi:10.1037/h0041412 · Cited by 791
Contributions to Probability and Statistics
Gleser, L. J., Perlman, M. D., Press, S. J., & Sampson, A. R. (1989) · Springer eBooks · doi:10.1007/978-1-4612-3678-8 · Cited by 612
THE FIDUCIAL ARGUMENT IN STATISTICAL INFERENCE
Fisher, R. A. (1935) · Annals of Eugenics · doi:10.1111/j.1469-1809.1935.tb02120.x · Cited by 580
Box, G. E. P. (1954) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177728717 · Cited by 562
USING ANALYSIS OF VARIANCE WITH A DICHOTOMOUS DEPENDENT VARIABLE: AN EMPIRICAL STUDY1
Lunney, G. H. (1970) · Journal of Educational Measurement · doi:10.1111/j.1745-3984.1970.tb00727.x · Cited by 536
Berkson, J. (1950) · Journal of the American Statistical Association · doi:10.2307/2280676 · Cited by 532
Analysis of Covariance: Its Nature and Uses
Cochran, W. G. (1957) · Biometrics · doi:10.2307/2527916 · Cited by 511
Lord, F. M. (1952) · Cited by 494
Errors of Measurement in Statistics
Cochran, W. G. (1968) · Technometrics · doi:10.2307/1267450 · Cited by 474
Some Consequences When the Assumptions for the Analysis of Variance are not Satisfied
Cochran, W. G. (1947) · Biometrics · doi:10.2307/3001535 · Cited by 392
Pitman, E. J. G. (1938) · Biometrika · doi:10.1093/biomet/29.3-4.322 · Cited by 314
THE ANALYSIS OF VARIANCE IN CASES OF NON-NORMAL VARIATION
PEARSON, E. S. (1931) · Biometrika · doi:10.1093/biomet/23.1-2.114 · Cited by 241
Baker, B. O., Hardyck, C. D., & Petrinovich, L. F. (1966) · Educational and Psychological Measurement · doi:10.1177/001316446602600204 · Cited by 218
Analysis of Covariance: A Delicate Instrument
ELASHOFF, J. D. (1969) · American Educational Research Journal · doi:10.3102/00028312006003383 · Cited by 177
Robustness to non-normality of regression tests
BOX, G. E. P., & WATSON, G. S. (1962) · Biometrika · doi:10.1093/biomet/49.1-2.93 · Cited by 149
Collier, R. O., Baker, F. B., Mandeville, G. K., & Hayes, T. F. (1967) · Psychometrika · doi:10.1007/bf02289596 · Cited by 147
Donaldson, T. S. (1968) · Journal of the American Statistical Association · doi:10.2307/2284037 · Cited by 140
Hsu, T.-C., & Feldt, L. S. (1969) · American Educational Research Journal · doi:10.3102/00028312006004515 · Cited by 136
“STUDENT” (1927) · Biometrika · doi:10.1093/biomet/19.1-2.151 · Cited by 133
Large-Sample Covariance Analysis When the Control Variable is Fallible
Lord, F. M. (1960) · Journal of the American Statistical Association · doi:10.2307/2281743 · Cited by 117
On Transformations Used in the Analysis of Variance
Curtiss, J. H. (1943) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177731452 · Cited by 114
Obustness of Some Procedures for the Two-Sample Location Problem
Pratt, J. W. (1964) · Journal of the American Statistical Association · doi:10.1080/01621459.1964.10480721 · Cited by 113
The Distribution of “Student'S” Ratio for Non-Normal Samples
Geary, R. C. (1936) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.2307/2983669 · Cited by 110
The Effect of Non-Normality on the t Distribution
Bartlett, M. S. (1935) · Mathematical Proceedings of the Cambridge Philosophical Society · doi:10.1017/s0305004100013311 · Cited by 109
THE DISTRIBUTION OF ‘STUDENT'S’ t IN RANDOM SAMPLES OF ANY SIZE DRAWN FROM NON-NORMAL UNIVERSES
GAYEN, A. K. (1949) · Biometrika · doi:10.1093/biomet/36.3-4.353 · Cited by 104
Large-Sample Covariance Analysis when the Control Variable is Fallible
Lord, F. M. (1960) · Journal of the American Statistical Association · doi:10.1080/01621459.1960.10482065 · Cited by 102
THE DISTRIBUTION OF THE VARIANCE RATIO IN RANDOM SAMPLES OF ANY SIZE DRAWN FROM NON-NORMAL UNIVERSES
GAYEN, A. K. (1950) · Biometrika · doi:10.1093/biomet/37.3-4.236 · Cited by 101
THE EFFECT OF NON-NORMALITY ON THE POWER FUNCTION OF THE F-TEST IN THE ANALYSIS OF VARIANCE
David, F. N., & Johnson, N. L. (1951) · Biometrika · doi:10.1093/biomet/38.1-2.43 · Cited by 96
Donaldson, T. S. (1968) · Journal of the American Statistical Association · doi:10.1080/01621459.1968.11009285 · Cited by 86
Control of spurious association and the reliability of the controlled variable.
Kahneman, D. J. (1965) · Psychological Bulletin · doi:10.1037/h0022529 · Cited by 85
Pearson, E. S., & Adyanthaya, N. K. (1929) · Biometrika · doi:10.1093/biomet/21.1-4.259 · Cited by 79
EFFECT OF NON-NORMALITY ON THE POWER OF THE ANALYSIS OF VARIANCE TEST
SRIVASTAVA, A. B. L. (1959) · Biometrika · doi:10.1093/biomet/46.1-2.114 · Cited by 75
A Monte Carlo Study Comparing Various Two-Sample Tests for Differences in Mean
Neave, H. R., & Granger, C. W. J. (1968) · Technometrics · doi:10.1080/00401706.1968.10490598 · Cited by 69
NON-PARAMETRIC METHODS IN PSYCHOLOGICAL RESEARCH
Gaito, J. M. (1959) · Psychological Reports · doi:10.2466/pr0.1959.5.g.115 · Cited by 66
A comparison of the power of the U and t tests.
Boneau, C. A. (1962) · Psychological Review · doi:10.1037/h0047269 · Cited by 62
The Robustness of the Covariance Analysis of a One-Way Classification
Atiqullah, M. (1964) · Biometrika · doi:10.1093/biomet/51.3-4.365 · Cited by 62
A Random Normal Number Generator for 32-Bit-Word Computers
Chen, E. H. (1971) · Journal of the American Statistical Association · doi:10.1080/01621459.1971.10482276 · Cited by 61
Atiqullah, M. (1962) · Biometrika · doi:10.1093/biomet/49.1-2.83 · Cited by 60
THE EFFECT OF UNEQUAL GROUP VARIANCES ON THE F-TEST FOR THE HOMOGENEITY OF GROUP MEANS
Horsnell, G. (1953) · Biometrika · doi:10.1093/biomet/40.1-2.128 · Cited by 55
The Effect of Correlation On the Repeated Measures Design
Lana, R. E., & Lubin, A. (1963) · Educational and Psychological Measurement · doi:10.1177/001316446302300408 · Cited by 49
Numerical transformations in the analysis of experimental data.
Mueller, C. G. (1949) · Psychological Bulletin · doi:10.1037/h0056381 · Cited by 47
EFFECT OF NON-NORMALITY ON THE POWER FUNCTION OF t-TEST
SRIVASTAVA, A. B. L. (1958) · Biometrika · doi:10.1093/biomet/45.3-4.421 · Cited by 44
SIGNIFICANCE OF DIFFERENCE BETWEEN THE MEANS OF TWO NON-NORMAL SAMPLES
GAYEN, A. K. (1950) · Biometrika · doi:10.1093/biomet/37.3-4.399 · Cited by 38
Power of the Analysis of Variance of Independent Groups on Non-Normal and Normally Transformed Data
Games, P. A., & Lucas, P. A. (1966) · Educational and Psychological Measurement · doi:10.1177/001316446602600205 · Cited by 38
Contribution to the Theory of “Student’s” t-Test as Applied to the Problem of Two Samples
Hsu, P. (1983) · Springer eBooks · doi:10.1007/978-1-4684-9324-5_8 · Cited by 37
Neyman, J., & Pearson, E. S. (1964) · Biometrika · doi:10.2307/2334245 · Cited by 34
Swanson, L. A., Vance, S. C., Ferrari, M. R., et al. (1969) · Academy of Management Journal · doi:10.5465/amj.1969.4498543 · Cited by 34
Rider, P. R. (1929) · Biometrika · doi:10.1093/biomet/21.1-4.124 · Cited by 33
The analysis of variance in psychological research.
Garrett, H. E., & Zubin, J. (1943) · Psychological Bulletin · doi:10.1037/h0063637 · Cited by 30
Some Scheffé-Type Tests for Some Behrens-Fisher-Type Regression Problems
Potthoff, R. F. (1965) · Journal of the American Statistical Association · doi:10.1080/01621459.1965.10480859 · Cited by 25
Gronow, D. G. C. (1951) · Biometrika · doi:10.1093/biomet/38.1-2.252 · Cited by 24
HACK, H. R. B. (1958) · Biometrika · doi:10.1093/biomet/45.1-2.260 · Cited by 23
A NEW METHOD OF EXPERIMENTAL SAMPLING ILLUSTRATED ON CERTAIN NON-NORMAL POPULATIONS
Hey, G. B. (1938) · Biometrika · doi:10.1093/biomet/30.1-2.68 · Cited by 22
Readings in Statistics for the Behavioral Sciences.
Knott, M. J., Heermann, E. F., & Braskamp, L. A. (1971) · Journal of the Royal Statistical Society Series A (General) · doi:10.2307/2344256 · Cited by 19
Heterogeneity and Skewness in Analysis of Variance
Young, R. K., & Veldman, D. J. (1963) · Perceptual and Motor Skills · doi:10.2466/pms.1963.16.2.588 · Cited by 19
Peckham, P. D. (1968) · Medical Entomology and Zoology · Cited by 16
Self-citationDaniels, H. E. (1938) · Mathematical Proceedings of the Cambridge Philosophical Society · doi:10.1017/s0305004100020247 · Cited by 13
On "The analysis of variance in psychological research."
Grant, D. A. (1944) · Psychological Bulletin · doi:10.1037/h0058670 · Cited by 10
The Validity of thez-Criterion when the Variates are taken from Different Normal Populations
Quensel, C.-E. (1947) · Scandinavian Actuarial Journal · doi:10.1080/03461238.1947.10419648 · Cited by 8
Robust tests for variances and effect of non-normality and variance heterogeneity on standard tests
Box, G. E. P., & Andersen, S. (1954) · NCSU Libraries Repository (North Carolina State University Libraries) · Cited by 7
A Distinction Between Exact and Approximate Nonparametric Methods
Sawrey, W. L. (1958) · Psychometrika · doi:10.1007/bf02289012 · Cited by 6
F-Test Bias for Experimental Designs in Educational Research
Gourlay, N. (1955) · Psychometrika · doi:10.1007/bf02289019 · Cited by 5
Non-normality and heterogeneity in two samplet-test
Bhattacharjee, G. P. (1968) · Annals of the Institute of Statistical Mathematics · doi:10.1007/bf02911638 · Cited by 5
BRADLEY, R. A. (1952) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177729488 · Cited by 5
DATA TRANSFORMATION IN TWO‐WAY ANALYSIS OF VARIANCE
Schlesselman, J. J. (1971) · ETS Research Bulletin Series · doi:10.1002/j.2333-8504.1971.tb01017.x · Cited by 4
Bradley, J. V. (1963) · PsycEXTRA Dataset · doi:10.1037/e523952009-001 · Cited by 4
Discussion on Paper by Professor Box and Professor Cox
(1964) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.2517-6161.1964.tb00554.x · Cited by 4
NOTE ON MR SRIVASTAVA'S PAPER ON THE POWER FUNCTION OF STUDENT'S TEST
Pearson, E. S. (1958) · Biometrika · doi:10.1093/biomet/45.3-4.429 · Cited by 2
Mallows, C. L., & Cox, D. R. (1959) · Biometrika · doi:10.2307/2333552 · Cited by 2
Bradley, J. V. (1966) · PsycEXTRA Dataset · doi:10.1037/e401702004-001 · Cited by 1
Research works printed but not linked: 16
- Atiqullah M, 1967, Bulletin of the Institute of Statistical Research and Training, 1, 77
- Behrens WU, 1929, Landweise Jahresbuch, 68, 807
- Gayen AK, 1949, Biometrika, 36, 353
- Glass GV, 1970, Statistical methods in education and psychology
- Horsnell G, 1953, Biometrika, 40, 28
- KohrRLA comparison of statistical procedures for testing μ 1 = μ 2 with unequal n ’s and variances1970Pennsylvania State UniversityDoctoral dissertation
- Lindquist EF, 1953, Design and analysis of experiments in education and psychology
- Lunney GH, 1969, Paper presented at the Annual Meeting of the American Educational Research Association
- Mandeville GK, 1969, A Monte Carlo investigation of the adequacy of standard analysis of variance test procedures for dependent binary variates
- Pitman EJG, 1937, Biometrika, 29, 322
- Porter AC, 1967, The effects of using fallible variables in the analysis of covariance, Ph.D
- Pratt JW, 1964, Journal of the American Statistical Association, 59, 665
- Quesnel CE, 1947, Skandanavian Aktuarietids, 30, 44
- van der Vaart HR, 1961, Quantative methods in pharmacology, 140
- Welch BL, 1947, Biometrika, 34, 28
- 10.2466/pms.1963.16.2.588 · DOI
Cited by
Sorted by citations
Statistical Methods for Environmental Pollution Monitoring.
Lambert, D., & Gilbert, R. O. (1988) · Journal of the American Statistical Association · doi:10.2307/2290180 · Cited by 2,066
Anderson, M. J., & Walsh, D. C. I. (2013) · Ecological Monographs · doi:10.1890/12-2010.1 · Cited by 2,050
Statistical Methods for Environmental Pollution Monitoring.
Gilbert, R. O. (1988) · Biometrics · doi:10.2307/2531935 · Cited by 1,728
Non-normal data: Is ANOVA still a valid option?
Blanca, M., Alarcón, R., Arnau, J., Bono, R., & Bendayan, R. (2017) · Psicothema · doi:10.7334/psicothema2016.383 · Cited by 1,624
“However, Glass, Peckham, and Sanders (1972) summarized these early studies and concluded that the procedure was affected by kurtosis, whereas skewness had very little effect.”
The unicorn, the normal curve, and other improbable creatures.
Micceri, T. (1989) · Psychological Bulletin · doi:10.1037/0033-2909.105.1.156 · Cited by 1,487
Metadata
- Authors
- Glass, G. V., Peckham, P. D., & Sanders, J. R.
- Affiliations
- University of Colorado System (US)
- University of Washington (US)
- Indiana University (US)
- ORCID
- Glass, Gene V
- Venue
- Review of Educational Research
- Published
- 1972-09
- Volume
- 42
- Issue
- 3
- Pages
- 237-288
- ISSN
- 0034-6543 (ISSN-L)
- 1935-1046
- OpenAlex
- W2120106329
- Semantic Scholar
- 96cd4ca99a137bf80e8c2dc5cb20ab6782df819e
- Cited by
- 1,826
- References
- 95
- Citations by year
- 2026: 91 · 2025: 69 · 2024: 55 · 2023: 75 · 2022: 72 · 2021: 98 · 2020: 85 · 2019: 96 · 2018: 137 · 2017: 100 · 2016: 106 · 2015: 74 · 2014: 65 · 2013: 70 · 2012: 69
- FWCI
- 10.37
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Review of Educational Research (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Statistical Methods and Models
- Physical Sciences › Mathematics › Statistics and Probability › Probability and Statistical Research
- Keywords
- covariance analysis, assumption violations, type I error, type II error, nonparametric statistics, Wilcoxon test, Mann-Whitney test, Kruskal-Wallis test, Friedman test, social science research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review