Explainability in action: A metric-driven assessment of local explanations for healthcare tabular models

Abstract

Explainable AI (XAI) is increasingly used in clinical machine learning, yet quantitative evaluation of explanation quality is often reported inconsistently across methods and datasets. We present a reproducible, metric-driven framework for evaluating XAI methods on healthcare tabular data. The framework consolidates six established, family-specific metrics, fidelity, simplicity, consistency, robustness, precision, and coverage, into explicit equations; pairs them with a pre-specified focal-model protocol; and releases open-source code with a method–metric applicability map. We evaluate LIME, SHAP, Anchors, EBM, and TabNet across four public healthcare tabular datasets. Post-hoc explainers are applied to a single selected Random Forest focal predictor to control model-induced variability, whereas EBM and TabNet are assessed through their native interpretability mechanisms. Global explanation summaries are reported descriptively only. The results show that SHAP/TreeSHAP provides exact score reconstruction for the Random Forest setting, while LIME produces simpler but lower-fidelity explanations with greater instance-level variability. LIME and SHAP show the strongest rank agreement among the evaluated pairs, although agreement varies across datasets. TabNet often yields compact native explanations, but these must be interpreted alongside its dataset-specific predictive performance. EBM and TabNet show low sensitivity under the fixed Gaussian-jitter robustness protocol, while Anchors produces high-precision rules with reduced coverage at stricter thresholds. Overall, the framework enables controlled comparison under explicit method–metric and focal-model assumptions, supporting more transparent XAI selection for tabular machine learning. Although demonstrated in healthcare, the framework is transferable to other high-stakes tabular domains. Source code: https://github.com/matifq/XAI_Tab_Health .