The DOI matches its Crossref record; 82 of the 82 references it cites were found in a registry or an index.
Modern robust statistical methods: An easy way to maximize the accuracy and power of your research.
Open accessAbstract
Classic parametric statistical significance tests, such as analysis of variance and least squares regression, are widely used by researchers in many disciplines, including psychology. For classic parametric tests to produce accurate results, the assumptions underlying them (e.g., normality and homoscedasticity) must be satisfied. These assumptions are rarely met when analyzing real data. The use of classic parametric methods with violated assumptions can result in the inaccurate computation of p values, effect sizes, and confidence intervals. This may lead to substantive errors in the interpretation of data. Many modern robust statistical methods alleviate the problems inherent in using parametric methods with violated assumptions, yet modern methods are rarely used by researchers. The authors examine why this is the case, arguing that most researchers are unaware of the serious limitations of classic methods and are unfamiliar with modern alternatives. A range of modern robust and rank-based significance tests suitable for analyzing a wide range of designs is introduced. Practical advice on conducting modern analyses using software such as SPSS, SAS, and R is provided. The authors conclude by discussing robust effect size indices.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“We examined the total, direct, and indirect effects of each predictor variable on outcomes using bias-corrected bootstrapped estimates (Efron & Tibshirani, 1993) based on 10,000 bootstrapped samples, which provides a powerful test of mediation (Fritz & MacKinnon, 2007) and is robust to small departures from normality (Erceg-Hurn & Mirosevich, 2008).”
From Five Facets of Mindfulness and Psychological Health: Evaluating a Psychological Model of the Mechanisms of Mindfulness (2015) · cites this paper as Erceg-Hurn & Mirosevich, 2008
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Beaton, A. E., & Keppel, G. (1975) · American Educational Research Journal · doi:10.2307/1162588 · Cited by 5,857
Rank transformations as a bridge between parametric and nonparametric statistics
Conover, W. J., & Iman, R. L. (1981) · The American Statistician · doi:10.1080/00031305.1981.10479327 · Cited by 3,922
Design and Analysis of Experiments in Psychology and Education.
Ehrenberg, A. S. C., & Lindquist, E. F. (1954) · Journal of the Royal Statistical Society Series A (General) · doi:10.2307/2342775 · Cited by 2,634
Design and Analysis: A Researcher's Handbook
Morgan, J. P. (2005) · Technometrics · doi:10.1198/tech.2005.s329 · Cited by 2,416
Introduction to Robust Estimation and Hypothesis Testing
Wilcox, R. R. (2012) · Elsevier eBooks · doi:10.1016/c2010-0-67044-1 · Cited by 2,086
Glass, G. V., Peckham, P. D., & Sanders, J. R. (1972) · Review of Educational Research · doi:10.3102/00346543042003237 · Cited by 1,826
The unicorn, the normal curve, and other improbable creatures.
Micceri, T. (1989) · Psychological Bulletin · doi:10.1037/0033-2909.105.1.156 · Cited by 1,487
Rank Transformations as a Bridge Between Parametric and Nonparametric Statistics
Conover, W. J., & Iman, R. L. (1981) · The American Statistician · doi:10.2307/2683975 · Cited by 1,481
NON-NORMALITY AND TESTS ON VARIANCES
BOX, G. E. P. (1953) · Biometrika · doi:10.1093/biomet/40.3-4.318 · Cited by 1,302
King, T., D'Agostino, R., & Stephens, M. A. (1987) · Technometrics · doi:10.2307/1269472 · Cited by 1,256
A Critique and Improvement of the CL Common Language Effect Size Statistics of McGraw and Wong
Vargha, A., & Delaney, H. D. (2000) · Journal of Educational and Behavioral Statistics · doi:10.3102/10769986025002101 · Cited by 1,164
A common language effect size statistic
McGraw, K. O., & Wong, S. P. (1992) · Psychological Bulletin · doi:10.1037/0033-2909.111.2.361 · Cited by 934
Size of Treatment Effects and Their Importance to Clinical Research and Practice
KRAEMER, H. C., & Kupfer, D. J. (2005) · Biological Psychiatry · doi:10.1016/j.biopsych.2005.09.014 · Cited by 850
Variability in Reaction Time Performance of Younger and Older Adults
Hultsch, D. F., MacDonald, S. W. S., & Dixon, R. A. (2002) · The Journals of Gerontology Series B · doi:10.1093/geronb/57.2.p101 · Cited by 838
A Critique and Improvement of the "CL" Common Language Effect Size Statistics of McGraw and Wong
Vargha, A., Delaney, H. D., & Vargha, A. (2000) · Journal of Educational and Behavioral Statistics · doi:10.2307/1165329 · Cited by 836
The effects of violations of assumptions underlying the t test.
Boneau, C. A. (1960) · Psychological Bulletin · doi:10.1037/h0041412 · Cited by 791
Effect sizes for research: A broad practical approach.
Grissom, R. J., & Kim, J. J. (2005) · Cited by 732
Robust Nonparametric Statistical Methods
Hettmansperger, T. P., & McKean, J. W. (2010) · CRC Press eBooks · doi:10.1201/b10451 · Cited by 649
Nonparametric analysis of longitudinal data in factorial experiments
Brunner, E., Domhof, S., & Langer, F. (2002) · Cited by 635
Keselman, H. J., Huberty, C. J., Lix, L. M., Olejnik, S. F., Cribbie, R. A., Donahue, B., Kowalchuk, R. K., Lowman, L. L., Petoskey, M. D., Keselman, J. C., et al. (1998) · Review of Educational Research · doi:10.3102/00346543068003350 · Cited by 630
Nonparametric Analysis of Longitudinal Data in Factorial Experiments
Ghosh, S. (2003) · Technometrics · doi:10.1198/tech.2003.s161 · Cited by 585
Nonparametric Analysis of Ordinal Data in Designed Factorial Experiments
Shah, D. A., & Madden, L. V. (2004) · Phytopathology · doi:10.1094/phyto.2004.94.1.33 · Cited by 535
Introduction to Robust Estimation and Hypothesis Testing
Horn, P. S., & Wilcox, R. R. (1998) · Technometrics · doi:10.2307/1271402 · Cited by 521
Modern Robust Data Analysis Methods: Measures of Central Tendency.
Wilcox, R. R., & Keselman, H. J. (2003) · Psychological Methods · doi:10.1037/1082-989x.8.3.254 · Cited by 505
Harwell, M. R., Rubinstein, E. N., Hayes, W. S., & Olds, C. C. (1992) · Journal of Educational Statistics · doi:10.3102/10769986017004315 · Cited by 485
Lix, L. M., Keselman, J. C., & Keselman, H. J. (1996) · Review of Educational Research · doi:10.3102/00346543066004579 · Cited by 477
Design and analysis of experiments in psychology and education.
indicated, N. A. (1953) · Journal of Consulting Psychology · doi:10.1037/h0051055 · Cited by 474
Applying Contemporary Statistical Techniques
(2003) · Elsevier eBooks · doi:10.1016/b978-0-12-751541-0.x5021-4 · Cited by 421
Fundamentals of Modern Statistical Methods: Substantially Improving Power and Accuracy
Wilcox, R. R. (2010) · doi:10.1007/978-1-4419-5525-8 · Cited by 404 · cited 1×
“Several prominent statisticians and researchers have described the use of classic parametric statistics in the face of assumption violations as invalid (e.g., Kezelman et al., 1998; Leech & Onwuegbuzie, 2002; Wilcox, 2001; Zimmerman, 1998).”
BackgroundSawilowsky, S. S., & Blair, R. C. (1992) · Psychological Bulletin · doi:10.1037/0033-2909.111.2.352 · Cited by 395
Statistical methods in education and psychology, 3rd ed.
Glass, G. V., & Hopkins, K. D. (1996) · Cited by 386
Lix, L. M., Keselman, J. C., & Keselman, H. J. (1996) · Review of Educational Research · doi:10.2307/1170654 · Cited by 371
Introduction to Modern Nonparametric Statistics
Higgins, J. J. (2003) · Cited by 313
Introduction to Modern Nonparametric Statistics
Fotopoulos, S. (2004) · Technometrics · doi:10.1198/tech.2004.s229 · Cited by 301
How many discoveries have been lost by ignoring modern statistical methods?
Wilcox, R. R. (1998) · American Psychologist · doi:10.1037/0003-066x.53.3.300 · Cited by 295
Algina, J. J., Keselman, H. J., & Penfield, R. D. (2005) · Psychological Methods · doi:10.1037/1082-989x.10.3.317 · Cited by 290
Nonparametric methods in factorial designs
Brunner, E., & PURI, M. L. (2001) · Statistical Papers · doi:10.1007/s003620000039 · Cited by 285
Probabilistic index: an intuitive non‐parametric approach to measuring the size of treatment effects
Ación, L., Peterson, J. J., TEMPLE, S. D., & Arndt, S. V. (2005) · Statistics in Medicine · doi:10.1002/sim.2256 · Cited by 270
Introduction to Robust Estimation and Hypothesis Testing (2nd ed.)
Ziegel, E. R. (2005) · Dialnet (Universidad de la Rioja) · Cited by 258
Jaccard, J., & Guilamo-Ramos, V. (2002) · Journal of Clinical Child & Adolescent Psychology · doi:10.1207/s15374424jccp3101_15 · Cited by 248
Fundamentals of Modern Statistical Methods
Wilcox, R. R. (2001) · Springer eBooks · doi:10.1007/978-1-4757-3522-2 · Cited by 227
Nonparametric Hypotheses and Rank Statistics for Unbalanced Factorial Designs
Akritas, M. G., Arnold, S. F., & Brunner, E. (1997) · Journal of the American Statistical Association · doi:10.1080/01621459.1997.10473623 · Cited by 220
Probability of the superior outcome of one treatment over another.
Grissom, R. J. (1994) · Journal of Applied Psychology · doi:10.1037/0021-9010.79.2.314 · Cited by 217
Robust Nonparametric Statistical Methods
Mi, J., Hettmansperger, T. P., & McKean, J. W. (1999) · Technometrics · doi:10.2307/1270999 · Cited by 210
Applying Contemporary Statistical Techniques
Nembhard, H. B. (2004) · Technometrics · doi:10.1198/tech.2004.s233 · Cited by 187
Non-Normality and Tests on Variances
Box, G. E. P. (1953) · Biometrika · doi:10.2307/2333350 · Cited by 186
Zimmerman, D. W. (1998) · The Journal of Experimental Education · doi:10.1080/00220979809598344 · Cited by 162
Nonparametric Tests of Interaction in Experimental Design
Sawilowsky, S. S. (1990) · Review of Educational Research · doi:10.3102/00346543060001091 · Cited by 162
A warning about median reaction time.
Miller, J. O. (1988) · Journal of Experimental Psychology Human Perception & Performance · doi:10.1037//0096-1523.14.3.539 · Cited by 159
Heterogeneity of variance in clinical data.
Grissom, R. J. (2000) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.68.1.155 · Cited by 154
New monte carlo results on the robustness of the anova f, w and f statistics
Wilcox, R. R., Charlin, V. L., & Thompson, K. L. (1986) · Communications in Statistics - Simulation and Computation · doi:10.1080/03610918608812553 · Cited by 134
Keselman, H. J., Algina, J. J., Lix, L. M., Wilcox, R. R., & Deering, K. (2008) · Psychological Methods · doi:10.1037/1082-989x.13.2.110 · Cited by 125
Statistical Reform in Psychology: Is Anything Changing?
Cumming, G., Fidler, F., Leonard, M., Kalinowski, P., Christiansen, A., Kleinig, A., Lo, J., McMenamin, N., & Wilson, S. (2007) · Psychological Science · doi:10.1111/j.1467-9280.2007.01881.x · Cited by 122
Tests for the Normal Distribution
D’Agostino, R. B. (2017) · Routledge eBooks · doi:10.1201/9780203753064-9 · Cited by 120
A warning about median reaction time.
Miller, J. O. (1988) · Journal of Experimental Psychology Human Perception & Performance · doi:10.1037/0096-1523.14.3.539 · Cited by 118
Heterogeneity of variance in clinical data.
Grissom, R. J. (2000) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.68.1.155 · Cited by 113
Harwell, M. R., Rubinstein, E. N., Hayes, W. S., & Olds, C. C. (1992) · Journal of Educational Statistics · doi:10.2307/1165127 · Cited by 105
Review of assumptions and problems in the appropriate conceptualization of effect size.
Grissom, R. J., & Kim, J. J. (2001) · Psychological Methods · doi:10.1037/1082-989x.6.2.135 · Cited by 104
Exact Type 1 Error Rates for Robustness of Student's t Test with Unequal Variances
Ramsey, P. H. (1980) · Journal of Educational Statistics · doi:10.3102/10769986005004337 · Cited by 99
Rank-Based Analyses of Linear Models UsingR
Terpstra, J., & McKean, J. W. (2005) · Journal of Statistical Software · doi:10.18637/jss.v014.i07 · Cited by 96
A Common Situation Conducive to Bizarre Distribution Shapes
Bradley, J. V. (1977) · The American Statistician · doi:10.1080/00031305.1977.10479224 · Cited by 88
A Call for Greater Use of Nonparametric Statistics
Leech, N. L., & Onwuegbuzie, A. J. (2002) · Cited by 84
Robust Analysis of Linear Models
McKean, J. W. (2004) · Statistical Science · doi:10.1214/088342304000000549 · Cited by 84
Summarizing Monte Carlo Results in Methodological Research
Harwell, M. R. (1992) · Journal of Educational Statistics · doi:10.3102/10769986017004297 · Cited by 62
Confidence Interval Coverage for Cohen's Effect Size Statistic
Algina, J. J., Keselman, H. J., & Penfield, R. D. (2006) · Educational and Psychological Measurement · doi:10.1177/0013164406288161 · Cited by 62
Nonparametric Hypotheses and Rank Statistics for Unbalanced Factorial Designs
Akritas, M. G., Arnold, S. F., & Brunner, E. (1997) · Journal of the American Statistical Association · doi:10.2307/2291470 · Cited by 62
Jaccard, J. J., & Guilamo‐Ramos, V. (2002) · Journal of Clinical Child & Adolescent Psychology · doi:10.1207/s15374424jccp3102_13 · Cited by 61
Nonrobustness in Z, t, and F tests at large sample sizes
Bradley, J. V. (1980) · Bulletin of the Psychonomic Society · doi:10.3758/bf03329558 · Cited by 56
Zimmerman, D. W. (2000) · The Journal of General Psychology · doi:10.1080/00221300009598589 · Cited by 53
Twenty Nonparametric Statistics And Their Large Sample Approximations
Fahoome, G. F. (2002) · Journal of Modern Applied Statistical Methods · doi:10.22237/jmasm/1036110540 · Cited by 51
Nonparametric Competitors to the Two-Way ANOVA
Toothaker, L. E., & Newman, D. (1994) · Journal of Educational and Behavioral Statistics · doi:10.3102/10769986019003237 · Cited by 46
Effect Sizes and their Intervals: The Two-Level Repeated Measures Case
Algina, J. J., Keselman, H. J., & Penfield, R. D. (2005) · Educational and Psychological Measurement · doi:10.1177/0013164404268675 · Cited by 43
Onwuegbuzie, A. J., & Levin, J. R. (2003) · Journal of Modern Applied Statistical Methods · doi:10.22237/jmasm/1051747920 · Cited by 34
Nonparametric Competitors to the Two-Way ANOVA
Toothaker, L. E., & Newman, D. (1994) · Journal of Educational and Behavioral Statistics · doi:10.2307/1165296 · Cited by 24
More Powerful Tests of Predictor Subsets in Regression Analysis Under Nonnormality.
Serlin, R. C., & Harwell, M. R. (2004) · Psychological Methods · doi:10.1037/1082-989x.9.4.492 · Cited by 23
Confidence Intervals For An Effect Size When Variances Are Not Equal
Algina, J. J., Keselman, H. J., & Penfield, R. D. (2006) · Journal of Modern Applied Statistical Methods · doi:10.22237/jmasm/1146456060 · Cited by 20
How many discoveries have been lost by ignoring modern statistical methods?
Wilcox, R. R. (1998) · American Psychologist · doi:10.1037//0003-066x.53.3.300 · Cited by 20
NONPARAMETRIC METHODS IN FACTORIAL DESIGNS
Brunner, E., & PURI, M. L. (2003) · De Gruyter eBooks · doi:10.1515/9783110917819.534 · Cited by 19
Rank-Based Procedures for Linear Models: Applications to Pharmaceutical Science Data
Abebe, A., Crimin, K. S., McKean, J. W., Haas, J. V., & Vidmar, T. J. (2001) · Drug Information Journal · doi:10.1177/009286150103500334 · Cited by 15
Latency Models for Reaction Time Distributions
Taylor, D. H. (1965) · Psychometrika · doi:10.1007/bf02289445 · Cited by 11
Robust General Linear Models and Graphics via a User Interface (Web RGLM)
Crimin, K. S., Abebe, A., & McKean, J. W. (2008) · Journal of Modern Applied Statistical Methods · doi:10.22237/jmasm/1209616020 · Cited by 5
Review of Twenty Nonparametric Statistics and Their Large Sample Approximations.
Fahoome, G. F., & Sawilowsky, S. S. (2000) · Cited by 4
Research works printed but not linked: 1
- Bradley, J. V. (1978). Robustness? British Journal of Mathematical and Statistical Psychology, 31, 144-152.
Cited by
Sorted by citations
The New Statistics: Why and How
Cumming, G. (2014) · Psychological Science · doi:10.1177/0956797613504966 · Cited by 3,200
Ellis, P. D. (2010) · doi:10.1017/cbo9780511761676 · Cited by 2,461
Noguchi, K., Gel, Y. R., Brunner, E., & Konietschke, F. (2012) · Journal of Statistical Software · doi:10.18637/jss.v050.i12 · Cited by 1,146
Why Psychologists Should by Default Use Welch’s t-test Instead of Student’s t-test
Delacre, M., Lakens, D., & Leys, C. (2017) · International Review of Social Psychology · doi:10.5334/irsp.82 · Cited by 1,037
Statistics in Corpus Linguistics
Březina, V. (2018) · Cambridge University Press eBooks · doi:10.1017/9781316410899 · Cited by 796
Metadata
- Authors
- Erceg-Hurn, D. M., & Mirosevich, V. M.
- Affiliations
- Government of Western Australia (AU)
- ORCID
- Erceg-Hurn, David M.
- Venue
- American Psychologist
- Published
- 2008
- Volume
- 63
- Issue
- 7
- Pages
- 591-601
- ISSN
- 0003-066X (ISSN-L)
- 1935-990X
- OpenAlex
- W2102493402
- Cited by
- 959
- References
- 76
- Citations by year
- 2026: 38 · 2025: 59 · 2024: 55 · 2023: 54 · 2022: 81 · 2021: 54 · 2020: 94 · 2019: 77 · 2018: 61 · 2017: 66 · 2016: 73 · 2015: 59 · 2014: 45 · 2013: 36 · 2012: 41
- FWCI
- 20.78
- Type
- journal article
- Language
- en
- Open access
- Locations
- American Psychologist (publishedVersion)
- PubMed (publishedVersion)
- UWA Profiles and Research Repository (UWA) (submittedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Statistical Methods and Models
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Physical Sciences › Mathematics › Statistics and Probability › Probability and Statistical Research
- Keywords
- robust statistics, rank tests, nonparametric tests, normality, homoscedasticity, statistical power, significance testing
- MeSH terms
- Data Interpretation, Statistical, Humans, Research, Software, Statistics, Nonparametric
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
