The DOI matches its Crossref record; 20 of the 20 references it cites were found in a registry or an index.
How to Classify, Detect, and Manage Univariate and Multivariate Outliers, With Emphasis on Pre-Registration
Open accessAbstract
Researchers often lack knowledge about how to deal with outliers when analyzing their data. Even more frequently, researchers do not pre-specify how they plan to manage outliers. In this paper we aim to improve research practices by outlining what you need to know about outliers. We start by providing a functional definition of outliers. We then lay down an appropriate nomenclature/classification of outliers. This nomenclature is used to understand what kinds of outliers can be encountered and serves as a guideline to make appropriate decisions regarding the conservation, deletion, or recoding of outliers. These decisions might impact the validity of statistical inferences as well as the reproducibility of our experiments. To be able to make informed decisions about outliers you first need proper detection tools. We remind readers why the most common outlier detection methods are problematic and recommend the use of the median absolute deviation to detect univariate outliers, and of the Mahalanobis-MCD distance to detect multivariate outliers. An R package was created that can be used to easily perform these detection tests. Finally, we promote the use of pre-registration to avoid flexibility in data analysis when handling outliers. Publishers note: due to a typesetting error, this paper was originally published with incorrect table numbering, where tables 2, 3, and 4 were incorrectly labelled. This was corrected soon after publication.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
The Hopkins Symptom Checklist (HSCL): A self-report symptom inventory
Derogatis, L. R., Lipman, R. S., Rickels, K., Uhlenhuth, E. H., & Covi, L. (1974) · Behavioral Science · doi:10.1002/bs.3830190102 · Cited by 4,604
Leys, C., Ley, C., Klein, O., Bernard, P., & Licata, L. (2013) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2013.03.013 · Cited by 4,147
Self-citationMeasuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling
John, L. K., Loewenstein, G., & Prelec, D. (2012) · Psychological Science · doi:10.1177/0956797611430953 · Cited by 2,239
The preregistration revolution
Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1708274114 · Cited by 1,998
Choosing Prediction Over Explanation in Psychology: Lessons From Machine Learning
Yarkoni, T., & Westfall, J. (2017) · Perspectives on Psychological Science · doi:10.1177/1745691617693393 · Cited by 1,916
Increasing Transparency Through a Multiverse Analysis
Steegen, S., Tuerlinckx, F., Gelman, A., & Vanpaemel, W. (2016) · Perspectives on Psychological Science · doi:10.1177/1745691616658637 · Cited by 1,372
Best-practice recommendations for defining, identifying, and handling outliers
Aguinis, H., Gottfredson, R. K., & Joo, H. (2013) · Organizational Research Methods · doi:10.1177/1094428112470848 · Cited by 1,341
Wicherts, J. M., Veldkamp, C. L. S., Augusteijn, H. E. M., Bakker, M., van Aert, R. C. M., & van Assen, M. A. L. M. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2016.01832 · Cited by 754
Mathematics Anxiety, Working Memory, and Mathematics Performance in Secondary-School Children
Passolunghi, M. C., Caviola, S., De Agostini, R., Perin, C., & Mammarella, I. C. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2016.00042 · Cited by 696
Outliers detection and treatment: a review.
Cousineau, D., & Chartier, S. (2010) · International Journal of Psychological Research · doi:10.21500/20112084.844 · Cited by 672
Detecting multivariate outliers: Use a robust variant of the Mahalanobis distance
Leys, C., Klein, O., Dominicy, Y., & Ley, C. (2018) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2017.09.011 · Cited by 504
Self-citationPre-registration in social psychology—A discussion and suggested template
van 't Veer, A. E., & Giner-Sorolla, R. (2016) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2016.03.004 · Cited by 439
On the Bootstrap and Confidence Intervals
Hall, P. (1986) · The Annals of Statistics · doi:10.1214/aos/1176350168 · Cited by 321
A Practical Guide for Transparency in Psychological Science
Klein, O., Hardwicke, T. E., Aust, F., Breuer, J., Danielsson, H., Mohr, A. H., IJzerman, H., Nilsonne, G., Vanpaemel, W., & Frank, M. C. (2018) · Collabra: Psychology · doi:10.1525/collabra.158 · Cited by 233
Bakker, M., & Wicherts, J. M. (2014) · Psychological Methods · doi:10.1037/met0000014 · Cited by 209
CREATIVE HYPOTHESIS GENERATING IN PSYCHOLOGY: Some Useful Heuristics
McGuire, W. J. (1997) · Annual Review of Psychology · doi:10.1146/annurev.psych.48.1.1 · Cited by 203
A nonparametric method to analyze interactions: The adjusted rank transform test
Leys, C., & Schumann, S. (2010) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2010.02.007 · Cited by 164
Self-citationTukey, J. W., & McLaughlin, D. H. (2016) · Cited by 127
Chang, C., McAleer, M. J., & Wong, W. (2018) · Journal of risk and financial management · doi:10.3390/jrfm11010015 · Cited by 22
Research works printed but not linked: 12
- 1995, Statistics as principled argument
- 1987, Unraveling the mystery of health. How people manage stress and stay well
- 2003, Applied multiple correlation/regression analysis for the behavioral sciences
- 1983, A Festschrift for Erich L. Lehmann
- 1994, An introduction to the bootstrap
- 1997, Statistical methods for psychology
- 2015, Principles and practice of structural equation modeling
- 1930, On tests and measures of groups divergence, theoretical formulae, International Journal of the Asiatic Society of Bengal, 26, 541
- 2000, Handbook of research methods in social and personality psychology, 393
- 2000, Sensitivity analysis
- 2004, Handbook of parametric and nonparametric statistical procedures
- 2013, Using multivariate statistics
Cited by
Sahib, A., Chen, J., Cárdenas, D., Calear, A. L., & Wilson, C. (2024) · Journal of Affective Disorders · doi:10.1016/j.jad.2024.08.056 · Cited by 34
Lenhart, J., Richter, T., Appel, M., & Mar, R. A. (2023) · Scientific Reports · doi:10.1038/s41598-023-35346-7 · Cited by 29
Metadata
- Authors
- Leys, C., Delacre, M., Mora, Y. L., Lakens, D., & Ley, C.
- Venue
- International Review of Social Psychology
- Publisher
- Ubiquity Press, Ltd.
- Published
- 2019
- Volume
- 32
- Issue
- 1
- ISSN
- 0992-986X (ISSN-L)
- 2119-4130
- OpenAlex
- W2942533109
- Cited by
- 297
- References
- 20
- FWCI
- 20.14
- Type
- journal article
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.