CoChat
Sign Up
Cited by 1,037Probability and Statistical ResearchAdvanced Statistical Methods and ModelsPsychology Research and Bibliometrics

Why Psychologists Should by Default Use Welch’s t-test Instead of Student’s t-test

Open access
Delacre, Marie, Lakens, Daniël, Leys, Christophe·2017·International Review of Social Psychology
DOI 10.5334/irsp.82
Open this paper in CoChat — free accountOpen PDF

Abstract

When comparing two independent groups, psychology researchers commonly use Student’s t-tests. Assumptions of normality and homogeneity of variance underlie this test. More often than not, when these conditions are not met, Student’s t-test can be severely biased and lead to invalid statistical inferences. Moreover, we argue that the assumption of equal variances will seldom hold in psychological research, and choosing between Student’s t-test and Welch’s t-test based on the outcomes of a test of the equality of variances often fails to provide an appropriate answer. We show that the Welch’s t-test provides a better control of Type 1 error rates when the assumption of homogeneity of variance is not met, and it loses little robustness compared to Student’s t-test when the assumptions are met. We argue that Welch’s t-test should be used as a default strategy. Publisher’s Note: A correction article relating to this paper has been published and can be found at https://www.rips-irsp.com/articles/10.5334/irsp.661/.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.5334/irsp.82

Cites

4 of 4 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 38

  1. Baguley, T. (2012). Serious stats: A guide to advanced statistics for the behavioral sciences. Palgrave Macmillan. Retrieved from https://books.google.fr/ books?hl=fr&lr=&id=ObUcBQAAQBAJ&oi=fnd&pg- =PP1&dq=baguley+2012&ots=-eiUlHiCYs&sig=YU UKZ7jiGF33wdo3WVO-8l-OUu8.
  2. Banerjee, A., Chitnis, U., Jadhav, S., Bhawalkar, J. & Chaudhury, S. (2009). Hypothesis testing, type I and type II errors. Industrial Psychiatry Journal, 18(2), 127. DOI: https://doi.org/10.4103/0972-6748.62274 · DOI
  3. Baumgartner, H. & Steenkamp, J.-B. E. (2001). Response styles in marketing research: A cross-national investigation. Journal of Marketing Research, 38(2), 143-156. DOI: https://doi.org/10.1509/ jmkr.38.2.143.18840 · DOI
  4. Bryk, A. S. & Raudenbush, S. W. (1988). Hetero- geneity of variance in experimental studies: A challenge to conventional interpretations. Psy- chological Bulletin, 104(3), 396. DOI: https://doi. org/10.1037/0033-2909.104.3.396 · DOI
  5. Carroll, R. J. & Schneider, H. (1985). A note on Levene's tests for equality of variances. Statistics & Probability Letters, 3(4), 191-194. DOI: https://doi. org/10.1016/0167-7152(85)90016-1 · DOI
  6. Cowdry, R. W., Gardner, D. L., O'Leary, K. M., Leibenluft, E. & Rubinow, D. R. (1991). Mood variability: A study of four groups. American Journal of Psychiatry, 148(11), 1505-1511. DOI: https://doi. org/10.1176/ajp.148.11.1505 · DOI
  7. Cumming, G. (2013). Understanding the new statis- tics: Effect sizes, confidence intervals, and meta- analysis. Routledge. Retrieved from https:// books.google.fr/books?hl=fr&lr=&id=1W6la Nc7Xt8C&oi=fnd&pg=PR1&dq=understandi ng+the+new+statistics:+effect+sizes,+confi dence+intervals,+and+meta-analysis&ots=P uJZVHb03Q&sig=lhSjkfzp4o5OXAKhZ_zYzP9nsr8.
  8. Feingold, A. (1992). Sex differences in variability in intel- lectual abilities: A new look at an old controversy. Review of Educational Research, 62(1), 61-84. DOI: https://doi.org/10.3102/00346543062001061 · DOI
  9. Grissom, R. J. (2000). Heterogeneity of variance in clinical data. Journal of Consulting and Clini- cal Psychology, 68(1), 155. DOI: https://doi. org/10.1037/0022-006X.68.1.155 · DOI
  10. Hanushek, E. A. & W ößmann, L. (2006). Does educa- tional tracking affect performance and inequality? Differences-in-differences evidence across countries*. Economic Journal, 116(510), C63-C76. DOI: https:// doi.org/10.1111/j.1468-0297.2006.01076.x · DOI
  11. Hayes, A. F. & Cai, L. (2007). Further evaluating the condi- tional decision rule for comparing two independent means. British Journal of Mathematical and Statisti- cal Psychology, 60(2), 217-244. DOI: https://doi. org/10.1348/000711005X62576 · DOI
  12. Hoekstra, R., Kiers, H. & Johnson, A. (2012). Are assumptions of well-known statistical techniques checked, and why (not)? Frontiers in Psychology, 3, 137. DOI: https://doi.org/10.3389/fpsyg.2012.00137 · DOI
  13. Keppel, G. (1991). Design and analysis: A researcher's handbook. Prentice-Hall, Inc. Retrieved from http:// psycnet.apa.org/psycinfo/1991-98751-000.
  14. Keselman, H. J., Huberty, C. J., Lix, L. M., Olejnik, S., Cribbie, R. A., Donahue, B., Levin, J. R., et al. (1998). Statistical practices of educational researchers: An analysis of their ANOVA, MANOVA, and ANCOVA anal- yses. Review of Educational Research, 68(3), 350-386. DOI: https://doi.org/10.3102/00346543068003350 · DOI
  15. Keselman, H. J., Othman, A. R., Wilcox, R. R. & Fradette, K. (2004). The new and improved two-sample t test. Psychological Science, 15(1), 47-51. DOI: https:// doi.org/10.1111/j.0963-7214.2004.01501008.x · DOI
  16. Lakens, D., Hilgard, J. & Staaks, J. (2016). On the repro- ducibility of meta-analyses: Six practical recommen- dations. BMC Psychology, 4(1), 1. DOI: https://doi. org/10.1186/s40359-016-0126-3 · DOI
  17. Lim, T.-S. & Loh, W.-Y. (1996). A comparison of tests of equality of variances. Computational Statistics & Data Analysis, 22(3), 287-301. DOI: https://doi. org/10.1016/0167-9473(95)00054-2 · DOI
  18. Luh, W.-M. & Guo, J.-H. (2007). Approximate sample size formulas for the two-sample trimmed mean test with unequal variances. British Journal of Math- ematical and Statistical Psychology, 60(1), 137-146. DOI: https://doi.org/10.1348/000711006X100491 · DOI
  19. Moser, B. K. & Stevens, G. R. (1992). Homogeneity of variance in the two-sample means test. American Statistician, 46(1), 19-21. DOI: https://doi.org/10.1 080/00031305.1992.10475839 · DOI
  20. Nachar, N. (2008). The Mann-Whitney U: A test for assess- ing whether two independent samples come from the same distribution. Tutorials in Quantitative Methods for Psychology, 4(1), 13-20. DOI: https:// doi.org/10.20982/tqmp.04.1.p013 Neuhäuser, M. & Ruxton, G. D. (2009). Distribution-free two-sample comparisons in the case of heterogene- ous variances. Behavioral Ecology and Sociobiology, 63(4), 617-623. DOI: https://doi.org/10.1007/ s00265-008-0683-4 · DOI
  21. Nimon, K. F. (2012). Statistical assumptions of substan- tive analyses across the general linear model: A mini-review. Frontiers in Psychology, 3, 322. DOI: https://doi.org/10.3389/fpsyg.2012.00322 · DOI
  22. Nordstokke, D. W. & Zumbo, B. D. (2007). A Cautionary Tale about Levene's Tests for Equal Variances. Jour- nal of Educational Research & Policy Studies, 7(1), 1-14.
  23. Overall, J. E., Atlas, R. S. & Gibson, J. M. (1995). Tests that are robust against variance heterogeneity in k × 2 designs with unequal cell frequencies. Psycho- logical Reports, 76(3), 1011-1017. DOI: https://doi. org/10.2466/pr0.1995.76.3.1011 · DOI
  24. Rakotomalala, R. (2008). Comparaison de populations. Tests Non Paramétriques, Université Lumiere Lyon, 2. Retrieved from http://www.academia.edu/down- load/44989200/Comp_Pop_Tests_Nonparametr- iques.pdf.
  25. Rasch, D., Kubinger, K. D. & Moder, K. (2011). The two-sample t test: Pre-testing its assumptions does not pay off. Statistical Papers, 52(1), 219-231. DO: https://doi.org/10.1007/s00362-009-0224-x · DOI
  26. Ruxton, G. D. (2006). The unequal variance t-test is an underused alternative to Student's t-test and the Mann-Whitney U test. Behavioral Ecology, 17(4), 688-690. DOI: https://doi.org/10.1093/beheco/ ark016 · DOI · Paper page
  27. Shaw, R. G. & Mitchell-Olds, T. (1993). ANOVA for unbal- anced data: An overview. Ecology, 74(6), 1638-1645. DOI: https://doi.org/10.2307/1939922 · DOI
  28. Shields, S. (1975). Functionalism, Darwinism, and the psychology of women. American Psychologist, 30(7), 739. DOI: https://doi.org/10.1037/h0076948 Student. (1908). The probable error of a mean. Biometrika, 1-25. DOI: https://doi.org/10.1093/biomet/6.1.1 · DOI · Paper page
  29. Wallenstein, S., Zucker, C. L. & Fleiss, J. L. (1980). Some statistical methods useful in circulation research. Circulation Research, 47(1), 1-9. DOI: https://doi. org/10.1161/01.RES.47.1.1 · DOI
  30. Wang, H., Smith, K. P., Combs, E., Blake, T., Horsley, R. D. & Muehlbauer, G. J. (2012). Effect of population size and unbalanced data sets on QTL detection using genome-wide association map- ping in barley breeding germplasm. Theoretical and Applied Genetics, 124(1), 111-124. DOI: https://doi. org/10.1007/s00122-011-1691-8 · DOI
  31. Welch, B. L. (1947). The generalization of Student's prob- lem when several different population variances are involved. Biometrika, 34(1/2), 28-35. DOI: https:// doi.org/10.2307/2332510 · DOI
  32. Wilcox, R. R., Granger, D. A. & Clark, F. (2013). Modern robust statistical methods: Basics with illustrations using psychobiological data. Universal Journal of Psychology, 1(2), 21-31.
  33. Wilcox, R. R. & Keselman, H. J. (2003). Modern robust data analysis methods: Measures of central tendency. Psychological Methods, 8(3), 254. DOI: https://doi. org/10.1037/1082-989X.8.3.254 · DOI
  34. Yuen, K. K. (1974). The two-sample trimmed t for unequal population variances. Biometrika, 61(1), 165-170. DOI: https://doi.org/10.1093/biomet/61.1.165 · DOI
  35. Zimmerman, D. W. (1996). Some properties of prelimi- nary tests of equality of variances in the two-sample location problem. Journal of General Psychology, 123(3), 217-231. DOI: https://doi.org/10.1080/00 221309.1996.9921274 · DOI
  36. Zimmerman, D. W. (2000). Statistical significance levels of nonparametric tests biased by heterogeneous variances of treatment groups. Journal of General Psychology, 127(4), 354-364. DOI: https://doi. org/10.1080/00221300009598589 · DOI
  37. Zimmerman, D. W. (2004). A note on preliminary tests of equality of variances. British Journal of Mathemati- cal and Statistical Psychology, 57(1), 173-181. DOI: https://doi.org/10.1348/000711004849222 · DOI
  38. Zumbo, B. D. & Coulombe, D. (1997). Investigation of the robust rank-order test for non-normal populations with unequal variances: The case of reaction time. Canadian Journal of Experimen- tal Psychology/Revue Canadienne de Psycholo- gie Expérimentale, 51(2), 139. DOI: https://doi. org/10.1037/1196-1961.51.2.139 · DOI

Cited by

Sorted by citations

Authors
Delacre, M., Lakens, D., & Leys, C.
Affiliations
  • Université Libre de Bruxelles (BE)
  • Eindhoven University of Technology (NL)
Corresponding author
  • Delacre, Marie
  • Leys, Christophe
ORCID
Venue
International Review of Social Psychology
Publisher
Ubiquity Press, Ltd.
Published
2017-04-05
Volume
30
Issue
1
Pages
92-101
ISSN
  • 0992-986X (ISSN-L)
  • 2397-8570
DOI
10.5334/irsp.82
OpenAlex
W2604452528
Cited by
1,037
References
4
Citations by year
2026: 178 · 2025: 147 · 2024: 151 · 2023: 123 · 2022: 132 · 2021: 101 · 2020: 87 · 2019: 67 · 2018: 41 · 2017: 10
FWCI
39.18
Type
journal article
Language
en
Open access
Licence
Locations
  • International Review of Social Psychology (publishedVersion, cc-by)
  • Dépôt institutionnel de l'Université libre de Bruxelles (Université Libre de Bruxelles) (submittedVersion, other-oa)
  • TU/e Research Portal (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • International Review of Social Psychology (Association pour la Diffusion de la Recherche Internationale en Psychologie Sociale) (submittedVersion, cc-by)
  • TU/e Research Portal (submittedVersion, cc-by)
Topics
  • Physical Sciences › Mathematics › Statistics and Probability › Probability and Statistical Research
  • Physical Sciences › Mathematics › Statistics and Probability › Advanced Statistical Methods and Models
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
Keywords
Welch's t-test, Student's t-test, heteroscedasticity, homoscedasticity, type I error rate, robustness

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review