Integrating machine learning and explainable AI for employee attrition prediction in HR analytics

Abstract

Employee attrition poses significant challenges to organizations, impacting productivity, morale, and financial stability. Predicting attrition and understanding its underlying drivers are critical for implementing effective retention strategies. In this study, we propose a comprehensive framework that utilizes advanced machine learning techniques to predict employee attrition and job change likelihood. The framework integrates robust preprocessing pipelines, state-of-the-art predictive models, and explainability tools such as SHAP (SHapley Additive exPlanations) to ensure transparency and fairness in HR analytics. By addressing key challenges such as class imbalance, feature selection, and model interpretability, our approach provides actionable insights for proactive talent management. We evaluate the framework on multiple datasets (including the IBM HR Analytics Employee Attrition & Performance dataset and the HR Analytics: Job Change of Data Scientists dataset), achieving near-optimal performance metrics across diverse scenarios. Notably, the Adaptive Boosting (AB) and Histogram Gradient Boosting (HGB) models demonstrate superior performance, with high Precision, Recall, F1-score, and Accuracy. Global and local interpretability analyses using SHAP visualizations reveal critical predictors of attrition, such as OverTime, JobLevel, and JobSatisfaction, enabling targeted interventions. The results underscore the framework's adaptability, scalability, and potential for real-time deployment in organizational settings. This study contributes to advancing HR analytics by bridging gaps in predictive accuracy, interpretability, and generalizability; offering practical solutions for mitigating employee turnover and safeguarding human capital investments.