Best-practice recommendations for defining, identifying, and handling outliers
Abstract
The presence of outliers, which are data points that deviate markedly from others, is one of the most enduring and pervasive methodological challenges in organizational science research. We provide evidence that different ways of defining, identifying, and handling outliers alter substantive research conclusions. Then, we report results of a literature review of 46 methodological sources (i.e., journal articles, book chapters, and books) addressing the topic of outliers, as well as 232 organizational science journal articles mentioning issues about outliers. Our literature review uncovered (a) 14 unique and mutually exclusive outlier definitions, 39 outlier identification techniques, and 20 different ways of handling outliers; (b) inconsistencies in how outliers are defined, identified, and handled in various methodological sources; and (c) confusion and lack of transparency in how outliers are addressed by substantive researchers. We offer guidelines, including decision-making trees, that researchers can follow to define, identify, and handle error, interesting, and influential (i.e., model fit and prediction) outliers. Although our emphasis is on regression, structural equation modeling, and multilevel modeling, our general framework forms the basis for a research agenda regarding outliers in the context of other data-analytic approaches. Our recommendations can be used by authors as well as journal editors and reviewers to improve the consistency and transparency of practices regarding the treatment of outliers in organizational science research.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Together with its extreme baseline score and qualitative differences from other sites, the sensitivity analyses support the exclusion of Site A from the analytic sample.
“Together with its extreme baseline score and qualitative differences from other sites in the study, these sensitivity analyses support the exclusion of Site A from the analytic sample (Aguinis, Gottfredson, & Joo, 2013).”
From Improving relationship quality in group care settings: The impact of implementing the CARE model (2020) · cites this paper as Aguinis, Gottfredson, & Joo, 2013
Cites
Regression Diagnostics: Identifying Influential Data and Sources of Collinearity
Belsley, D. A., Kuh, E., & Welsch, R. E. (1980) · Wiley Series in Probability and Statistics · doi:10.1002/0471725153 · Cited by 7,297 · see citing works
Subjective well-being. The science of happiness and a proposal for a national index
Diener, E. (2000) · American Psychologist · doi:10.1037/0003-066x.55.1.34 · Cited by 7,121 · see citing works
Methods of Meta-Analysis: Correcting Error and Bias in Research Findings
Schmidt, F. L., & Hunter, J. E. (2015) · doi:10.4135/9781483398105 · Cited by 5,542 · see citing works
Hox, J., Moerbeek, M., & van de Schoot, R. (2010) · doi:10.4324/9780203852279 · Cited by 3,469 · see citing works
Hunter, J., & Schmidt, F. (2004) · doi:10.4135/9781412985031 · Cited by 3,023 · see citing works
Engendering Change: Organizational Dynamics and Workplace Gender Desegregation, 1975–2005
Huffman, M. L., Cohen, P. N., & Pearlman, J. (2010) · Administrative Science Quarterly · doi:10.2189/asqu.2010.55.2.255 · Cited by 210 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Exploratory Factor Analysis: A Guide to Best Practice
Watkins, M. W. (2018) · Journal of Black Psychology · doi:10.1177/0095798418771807 · Cited by 2,218 · see citing works
A Review on Outlier/Anomaly Detection in Time Series Data
Blázquez-García, A., Conde, A., Mori, U., & Lozano, J. A. (2022) · ACM Computing Surveys · doi:10.1145/3444690 · Cited by 1,102 · see citing works
Abusive supervision: A meta-analysis and empirical review
Mackey, J. D., Frieder, R. E., Brees, J. R., & Martinko, M. J. (2017) · Journal of Management · doi:10.1177/0149206315573997 · Cited by 706 · see citing works
Emotional exhaustion and workload predict clinician-rated and objective patient safety
Welp, A., Meier, L. L., & Manser, T. (2015) · Frontiers in Psychology · doi:10.3389/fpsyg.2014.01573 · Cited by 353 · see citing works
Busch, T., & Friede, G. (2018) · Corporate Social Responsibility and Environmental Management · doi:10.1002/csr.1480 · Cited by 345 · see citing works
Mackey, J. D., Parker Ellen, B., McAllister, C. P., & Alexander, K. C. (2021) · Journal of Business Research · doi:10.1016/j.jbusres.2020.10.037 · Cited by 320 · see citing works
Brill, T. M., Munoz, L., & Miller, R. J. (2019) · Journal of Marketing Management · doi:10.1080/0267257x.2019.1687571 · Cited by 291 · see citing works
Steffens, N. K., LaRue, C. J., Haslam, C., Walter, Z. C., Cruwys, T., Munt, K. A., Haslam, S. A., Jetten, J., & Tarrant, M. (2021) · Health Psychology Review · doi:10.1080/17437199.2019.1669481 · Cited by 259 · see citing works
Fiction reading has a small positive impact on social cognition: A meta-analysis.
Dodell-Feder, D., & Tamir, D. I. (2018) · Journal of Experimental Psychology: General · doi:10.1037/xge0000395 · Cited by 223 · see citing works
How to conduct a meta-analysis in eight steps: A practical guide
Hansen, C., Steinmetz, H., & Block, J. (2022) · Management Review Quarterly · doi:10.1007/s11301-021-00247-4 · Cited by 218 · see citing works
Questions About Questionable Research Practices in the Field of Management: A Guest Commentary
Banks, G. C., O’Boyle, E. H., Pollack, J. M., White, C. D., Batchelor, J. H., Whelpley, C. E., Abston, K. A., Bennett, A. A., & Adkins, C. L. (2016) · Journal of Management · doi:10.1177/0149206315619011 · Cited by 163 · see citing works
Jukic, I., Castilla, A. P., Ramos, A. G., Van Hooren, B., McGuigan, M. R., & Helms, E. R. (2023) · Sports Medicine · doi:10.1007/s40279-022-01754-4 · Cited by 138 · see citing works
A meta-analysis on verbal working memory in children and adolescents with ADHD
Ramos, A. A., Hamdan, A. C., & Machado, L. (2020) · The Clinical Neuropsychologist · doi:10.1080/13854046.2019.1604998 · Cited by 125 · see citing works
Wu, Y., & Schunn, C. D. (2023) · Contemporary Educational Psychology · doi:10.1016/j.cedpsych.2023.102160 · Cited by 78 · see citing works
Engel, Y., Ramesh, A., & Steiner, N. (2020) · Journal of Business Venturing · doi:10.1016/j.jbusvent.2019.105986 · Cited by 76 · see citing works
Mindfulness and emotion regulation: promoting well-being during the transition to college
Finkelstein-Fox, L., Park, C. L., & Riley, K. E. (2018) · Anxiety, Stress, & Coping · doi:10.1080/10615806.2018.1518635 · Cited by 65 · see citing works
Sánchez, A. C., Jones, S. K., Purvis, A., Estrada-Carmona, N., & De Palma, A. (2022) · Agriculture, Ecosystems & Environment · doi:10.1016/j.agee.2022.107933 · Cited by 60 · see citing works
Geirhos, A., Domhardt, M., Lunkenheimer, F., Temming, S., Holl, R. W., Minden, K., Warschburger, P., Meissner, T., Mueller-Stierlin, A. S., & Baumeister, H. (2022) · BMC Pediatrics · doi:10.1186/s12887-022-03134-3 · Cited by 37 · see citing works
Demangeon, A., Claudel-Valentin, S., Aubry, A., & Tazouti, Y. (2023) · Contemporary Educational Psychology · doi:10.1016/j.cedpsych.2023.102182 · Cited by 55 · see citing works
Single-case experimental research designs.
Kazdin, A. E. (2015) · Methodological issues and strategies in clinical research (4th ed.). · doi:10.1037/14805-029 · Cited by 39 · see citing works
Abós, Á., Sevil-Serrano, J., Haerens, L., Aelterman, N., & García-González, L. (2019) · Learning and Individual Differences · doi:10.1016/j.lindif.2019.04.008 · Cited by 37 · see citing works
Posttraumatic growth and adjustment to spinal cord injury: moderated by posttraumatic depreciation?
Kunz, S., Joseph, S., Geyh, S., & Peter, C. (2017) · Psychological Trauma: Theory, Research, Practice, and Policy · doi:10.1037/tra0000164 · Cited by 34 · see citing works
Finkelstein-Fox, L., Pavlacic, J. M., Buchanan, E. M., Schulenberg, S. E., & Park, C. L. (2020) · Cognitive Therapy and Research · doi:10.1007/s10608-019-10062-7 · Cited by 33 · see citing works
Improving relationship quality in group care settings: The impact of implementing the CARE model
Izzo, C. V., Smith, E. G., Sellers, D. E., Holden, M. J., & Nunno, M. A. (2020) · Children and Youth Services Review · doi:10.1016/j.childyouth.2019.104623 · Cited by 33 · see citing works
Perceived stigma and erotic technology: From sex toys to erobots
Dubé, S., Santaguida, M., Anctil, D., Zhu, C. Y., Thomasse, L., Giaccari, L., Oassey, R., Vachon, D., & Johnson, A. (2023) · Psychology & Sexuality · doi:10.1080/19419899.2022.2067783 · Cited by 29 · see citing works
Kolokotroni, P., Anagnostopoulos, F., & Hantzi, A. (2018) · Journal of Clinical Psychology in Medical Settings · doi:10.1007/s10880-018-9555-x · Cited by 28 · see citing works
Binding Characteristics of Thyroid Hormone Distributor Proteins to Thyroid Hormone Metabolites
Jongejan, R. M. S., Meima, M. E., Visser, W. E., Korevaar, T. I. M., van den Berg, S. A. A., Peeters, R. P., & de Rijke, Y. B. (2022) · Thyroid® · doi:10.1089/thy.2021.0588 · Cited by 20 · see citing works
Investigating how Explicit Contextual Cues Affect Predictive Sensorimotor Control in Autistic Adults
Arthur, T., Brosnan, M., Harris, D., Buckingham, G., Wilson, M., Williams, G., & Vine, S. (2023) · Journal of Autism and Developmental Disorders · doi:10.1007/s10803-022-05718-5 · Cited by 20 · see citing works
McLellan, A. G., Ahmad, N. S. B., Acton, J. P., Punni, H. K., O'Donnell, E., Shepherd, A. I., Perissiou, M., James, L. J., Clifford, T., & Bailey, S. J. (2026) · Food & Function · doi:10.1039/d6fo02338j
- Authors
- Aguinis, H., Gottfredson, R. K., & Joo, H.
- Affiliations
- Indiana University Bloomington (US)
- Corresponding author
- Aguinis, Herman
- Venue
- Organizational Research Methods
- Publisher
- SAGE Publications
- Published
- 2013-01-14
- Volume
- 16
- Issue
- 2
- Pages
- 270-301
- ISSN
- 1094-4281 (ISSN-L)
- 1552-7425
- OpenAlex
- W2099506356
- Cited by
- 1,341
- References
- 6
- Citations by year
- 2026: 172 · 2025: 197 · 2024: 140 · 2023: 156 · 2022: 143 · 2021: 150 · 2020: 101 · 2019: 88 · 2018: 63 · 2017: 38 · 2016: 27 · 2015: 34 · 2014: 23 · 2013: 5 · 2012: 1
- FWCI
- 24.11
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Organizational Research Methods (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Statistical Methods and Models
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Physical Sciences › Computer Science › Artificial Intelligence › Anomaly Detection Techniques and Applications
- Keywords
- outliers, outlier detection, outlier handling, regression analysis, structural equation modeling, multilevel modeling, research transparency, methodological consistency, decision tree
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
