Shapley Values for Feature Selection: The Good, the Bad, and the Axioms
Open accessThe DOI matches its Crossref record; 63 of the 64 references it cites were found in a registry or an index.
Abstract
The Shapley value has become popular in the Explainable AI (XAI) literature, thanks, to a large extent, to a solid theoretical foundation, including four “favourable and fair” axioms for attribution in transferable utility games. The Shapley value is provably the only solution concept satisfying these axioms. In this paper, we introduce the Shapley value and draw attention to its recent uses as a feature selection tool. We call into question this use of the Shapley value, using simple, abstract “toy” counterexamples to illustrate that the axioms may work against the goals of feature selection. From this, we develop a number of insights that are then investigated in concrete simulation settings, with a variety of Shapley value formulations, including SHapley Additive exPlanations (SHAP) and Shapley Additive Global importancE (SAGE). The aim is not to encourage any use of the Shapley value for feature selection, but we aim to clarify various limitations around their current use in the literature. In so doing, we hope to help demystify certain aspects of the Shapley value axioms that are viewed as “favourable”. In particular, we wish to highlight that the favourability of the axioms depends non-trivially on the way in which the Shapley value is appropriated in the XAI application.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“we incorporated the use of interpretability tools (eg, SHAP 28 values, feature importance) to ensure transparency, a priority emphasized in recent ML reporting guidelines.”
From Linguistic markers for identifying post-traumatic stress disorder and associated symptoms: a systematic literature review (2025) · cites this paper as [28]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“To align with best practices 27 for systematic reviews of ML studies, we augmented PROBAST with criteria tailored to these challenges”
From Linguistic markers for identifying post-traumatic stress disorder and associated symptoms: a systematic literature review (2025) · cites this paper as [28]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707
Gauriat, C.-M., Pencolé, Y., Ribot, P., & Brouillet, G. (2024) · HAL (Le Centre pour la Communication Scientifique Directe) · doi:10.4230/oasics.dx.2024.27 · Cited by 12,586
From local explanations to global understanding with explainable AI for trees
Lundberg, S. M., Erion, G., Chen, H., DeGrave, A., Prutkin, J. M., Nair, B., Katz, R., Himmelfarb, J., Bansal, N., & Lee, S.-I. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-019-0138-9 · Cited by 10,551
17. A Value for n-Person Games
Shapley, L. S. (1953) · Contributions to the Theory of Games (AM-28), Volume II · doi:10.1515/9781400881970-018 · Cited by 6,950
A survey on feature selection methods
Chandrashekar, G., & Sahin, F. (2014) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2013.11.024 · Cited by 5,189
Shapley, L. S. (1988) · The Shapley Value · doi:10.1017/cbo9780511528446.003 · Cited by 4,954
Explaining prediction models and individual predictions with feature contributions
Štrumbelj, E., & Kononenko, I. (2014) · Knowledge and Information Systems · doi:10.1007/s10115-013-0679-x · Cited by 2,286
Tukey, J. W. (1962) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177704711 · Cited by 1,860
Consistent Individualized Feature Attribution for Tree Ensembles
Lundberg, S. M., Erion, G. G., & Lee, S.-I. (2018) · arXiv · doi:10.48550/arxiv.1802.03888 · Cited by 1,430
Budescu, D. V. (1993) · Psychological Bulletin · doi:10.1037/0033-2909.114.3.542 · Cited by 1,193
Variable Importance Assessment in Regression: Linear Regression versus Random Forest
Grömping, U. (2009) · The American Statistician · doi:10.1198/tast.2009.08199 · Cited by 1,103
Introduction to Bivariate and Multivariate Analysis.
Sen, P. K., Lindeman, R. H., Merenda, P. F., & Gold, R. Z. (1981) · Journal of the American Statistical Association · doi:10.2307/2287559 · Cited by 1,055
Aas, K., Jullum, M., & Løland, A. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2021.103502 · Cited by 1,023
Model selection and model averaging
Claeskens, G., & Hjort, N. L. (2008) · Cited by 872
Potential, Value, and Consistency
Hart, S., & Mas-Colell, A. (1989) · Econometrica · doi:10.2307/1911054 · Cited by 728
The Shapley value : essays in honor of Lloyd S. Shapley
Roth, A. E. (1991) · Cited by 716
Analysis of regression in game theory approach
Lipovetsky, S., & Conklin, M. (2001) · Applied Stochastic Models in Business and Industry · doi:10.1002/asmb.446 · Cited by 650
Conference structures and fair allocation rules
Myerson, R. B. (1980) · International Journal of Game Theory · doi:10.1007/bf01781371 · Cited by 611
From explanations to feature selection: assessing SHAP values as feature selection mechanism
Marcilio, W. E., & Eler, D. M. (2020) · 2020 33rd SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI) · doi:10.1109/sibgrapi51738.2020.00053 · Cited by 504
An Efficient Explanation of Individual Classifications using Game Theory
StrumbeljErik, & KononenkoIgor (2010) · Journal of Machine Learning Research · doi:10.5555/1756006.1756007 · Cited by 380
Computational Aspects of Cooperative Game Theory
Chalkiadakis, G., Elkind, E., & Wooldridge, M. (2012) · Synthesis Lectures on Artificial Intelligence and Machine Learning · doi:10.1007/978-3-031-01558-8 · Cited by 337
The explanation game: Explaining machine learning models using shapley values
Merrick, L., & Taly, A. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-57321-8_2 · Cited by 277
Sobol' Indices and Shapley Value
Owen, A. B. (2014) · SIAM/ASA Journal on Uncertainty Quantification · doi:10.1137/130936233 · Cited by 266
Computational Aspects of Cooperative Game Theory
Chalkiadakis, G., Elkind, E., & Wooldridge, M. (2011) · Synthesis lectures on artificial intelligence and machine learning · doi:10.2200/s00355ed1v01y201107aim016 · Cited by 265
Asymmetric Shapley values: incorporating causal knowledge into model-agnostic explainabilityPreprint
Frye, C., Rowat, C., & Feige, I. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.06358 · Cited by 261
Explainable AI for Trees: From Local Explanations to Global Understanding
Lundberg, S., Erion, G., Chen, H., DeGrave, A. J., Prutkin, J. M., Nair, B. G., Katz, R., Himmelfarb, J., Bansal, N., & Lee, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.04610 · Cited by 249
A Shapley-based decomposition of the R-Square of a linear regression
Israeli, O. (2007) · The Journal of Economic Inequality · doi:10.1007/s10888-006-9036-6 · Cited by 248
Shapley Effects for Global Sensitivity Analysis: Theory and Computation
Song, E., Nelson, B. L., & Staum, J. (2016) · SIAM/ASA Journal on Uncertainty Quantification · doi:10.1137/15m1048070 · Cited by 244 · cited 3×
“The ML methods that stand out in terms of popularity are SHapley Additive exPlanations (SHAP) [15], [17], Shapley Effects [6] and Shapley Additive Global importancE (SAGE) [24], though the Shapley value itself carries a rich history of investigation in the context of game theory – Lloyd Shapley’s 1953 seminal paper [2…”
Relative Importance by Averaging over Orderings
Kruskal, W. (1987) · The American Statistician · doi:10.1080/00031305.1987.10475432 · Cited by 237
Problems with Shapley-value-based explanations as feature importance measures
Kumar, I. E., Venkatasubramanian, S., Scheidegger, C., & Friedler, S. (2020) · arXiv · doi:10.48550/arxiv.2002.11097 · Cited by 228 · cited 1×
“There have, however, been a number of recent criticisms of the Shapley value in ML [37]–[39], suggesting to us that the Shapley value and its alternatives may be further developed considerably over the coming years.”
Axiomatic arguments for decomposing goodness of fit according to Shapley and Owen values
Huettner, F., & Sunder, M. (2012) · Electronic Journal of Statistics · doi:10.1214/12-ejs710 · Cited by 206
Feature selection based on the Shapley value
Cohen, S. B., Ruppin, E., & Dror, G. (2005) · Edinburgh Research Explorer (University of Edinburgh) · Cited by 197
Understanding Global Feature Contributions With Additive Importance Measures
Covert, I., Lundberg, S., & Lee, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.00668 · Cited by 191
Introduction to bivariate and multivariate analysis
Lindeman, R. H., Merenda, P. F., & Gold, R. Z. (1980) · Cited by 179
On Shapley Value for Measuring Importance of Dependent Inputs
Owen, A. B., & Prieur, C. (2017) · SIAM/ASA Journal on Uncertainty Quantification · doi:10.1137/16m1097717 · Cited by 169
Feature Selection via Coalitional Game Theory
Cohen, S., Dror, G., & Ruppin, E. (2007) · Neural Computation · doi:10.1162/neco.2007.19.7.1939 · Cited by 144
Intelligent Data Engineering and Automated Learning
Goh, O. S., & Fung, C. C. (2003) · Lecture notes in computer science · doi:10.1007/b11717 · Cited by 141
Peters, H. J. M., Algaba, E., Fragnelli, V., & Sanchez-Soriano, J. (2019) · Chapman & Hall/CRC eBooks · doi:10.1201/9781351241410 · Cited by 130
The Cooperative Game Theory of Networks and Hierarchies
Gilles, R. P. (2010) · Theory and decision library. Series C, Game theory, mathematical programming and operations research · doi:10.1007/978-3-642-05282-8 · Cited by 120
Explaining by Removing: A Unified Framework for Model Explanation
Covert, I., Lundberg, S., & Lee, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.14878 · Cited by 116
Fair Attribution of Functional Contribution in Artificial and Biological Networks
Keinan, A., Sandbank, B., Hilgetag, C. C., Meilijson, I., & Ruppin, E. (2004) · Neural Computation · doi:10.1162/0899766041336387 · Cited by 107
Efficient computation and analysis of distributional Shapley valuesPreprint
Kwon, Y., Rivas, M. A., & Zou, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.01357 · Cited by 79
Intelligent Data Engineering and Automated Learning - IDEAL 2007
Bertini, C., Ortobelli Lozza, S., & STAINO, A. (2007) · Lecture notes in computer science · doi:10.1007/978-3-540-77226-2 · Cited by 69
CGA: a new feature selection model for visual human action recognition
Guha, R., Khan, A. H., Singh, P. K., Sarkar, R., & Bhattacharjee, D. (2020) · Neural Computing and Applications · doi:10.1007/s00521-020-05297-5 · Cited by 66
Relative Importance by Averaging Over Orderings
Kruskal, W. (1987) · The American Statistician · doi:10.2307/2684310 · Cited by 64
Theories of Data Analysis: From Magical Thinking Through Classical Statistics
Diaconis, P. W. (2006) · Wiley series in probability and statistics · doi:10.1002/9781118150702.ch1 · Cited by 57
Redelmeier, A., Jullum, M., & Aas, K. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-57321-8_7 · Cited by 56
Kóczy, L. Á. (2018) · Theory and decision library. Series C, Game theory, mathematical programming and operations research · doi:10.1007/978-3-319-69841-0 · Cited by 48
Zaeri-Amirani, M., Afghah, F., & Mousavi, S. (2018) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2018.8512266 · Cited by 40
Predicting generalized anxiety disorder among women using Shapley value
Jothi, N., Husain, W., & Rashid, N. A. (2020) · Journal of Infection and Public Health · doi:10.1016/j.jiph.2020.02.042 · Cited by 39
Chu, C. C. F., & Chan, D. P. K. (2020) · IEEE Access · doi:10.1109/access.2020.3002665 · Cited by 38
Interpretable feature subset selection: A Shapley value based approach
Tripathi, S., Hemachandra, N., & Trivedi, P. (2020) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata50022.2020.9378102 · Cited by 35
The Explanation Game: Explaining Machine Learning Models with Cooperative Game Theory
Merrick, L., & Taly, A. (2019) · arXiv.org · doi:10.48550/arxiv.1909.08128 · Cited by 35
The many Shapley values for model explanation
Sundararajan, M., & Najmi, A. (2019) · RePEc: Research Papers in Economics · doi:10.48550/arxiv.1908.08474 · Cited by 35
shapr: An R-package for explaining machine learning models with dependence-aware Shapley values
Sellereite, N., & Jullum, M. (2019) · The Journal of Open Source Software · doi:10.21105/joss.02027 · Cited by 34
Predictive and Causal Implications of using Shapley Value for Model InterpretationPreprint
Ma, S., & Tourani, R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.05052 · Cited by 27
Portfolio Performance Attribution via Shapley ValuePreprint
Moehle, N., Boyd, S., & Ang, A. (2021) · RePEc: Research Papers in Economics · doi:10.48550/arxiv.2102.05799 · Cited by 25
The Explanation Game: Explaining Machine Learning Models with Cooperative Game Theory
Merrick, L., & Taly, A. (2019) · Cited by 21
Shapley Value Confidence Intervals for Attributing Variance Explained
Fryer, D., Strümke, I., & Nguyen, H. D. (2020) · Frontiers in Applied Mathematics and Statistics · doi:10.3389/fams.2020.587199 · Cited by 11
Self-citationDo not adjust coefficients in Shapley value regression
Grömping, U., & Landau, S. (2009) · Applied Stochastic Models in Business and Industry · doi:10.1002/asmb.773 · Cited by 11
Explaining the data or explaining a model? Shapley values that uncover non-linear dependenciesPreprint
Fryer, D., Strümke, I., & Nguyen, H. D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.06011 · Cited by 7
Self-citationCorrection: Relative Importance by Averaging Over Orderings
(1987) · The American Statistician · doi:10.2307/2684770 · Cited by 5
On Feature Interactions Identified by Shapley Values of Binary Classification Games.Preprint
Tripathi, S., Hemachandra, N., & Trivedi, P. (2020) · arXiv (Cornell University) · Cited by 3
Research works printed but not linked: 1
- claeskens, 2008, Model Selection Model Averaging
Cited by
The Shapley Value in Machine Learning
Rozemberczki, B., Watson, L., Bayer, P., Yang, H.-T., Kiss, O., Nilsson, S., & Sarkar, R. (2022) · Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/778 · Cited by 296
The Shapley Value in Machine Learning
Rozemberczki, B., Watson, L., Bayer, P., Yang, H.-T., Kiss, O., Nilsson, S., & Sarkar, R. (2022) · arXiv · doi:10.48550/arxiv.2202.05594 · Cited by 312
“The feature selection game treats input features of a machine learning model as players and model performance as the payoff [Guyon et al. , 2003; Fryer et al. , 2021 Shapley values, and close relatives such as the Banzhaf index III, 1964 ] , have been studied as a measure of feature importance in…”
Methodvan Zyl, C., Ye, X., & Naidoo, R. (2024) · Applied Energy · doi:10.1016/j.apenergy.2023.122079 · Cited by 248
Nwakanma, C. I., Ahakonye, L. A. C., Njoku, J. N., Odirichukwu, J. C., Okolie, S. A., Uzondu, C., Ndubuisi Nweke, C. C., & Kim, D.-S. (2023) · Applied Sciences · doi:10.3390/app13031252 · Cited by 202
Explainable AI for Time Series Classification: A Review, Taxonomy and Research Directions
Theissler, A., Spinnato, F., Schlegel, U., & Guidotti, R. (2022) · IEEE Access · doi:10.1109/access.2022.3207765 · Cited by 194
Metadata
- Authors
- Fryer, D., Strumke, I., & Nguyen, H.
- Affiliations
- The University of Queensland (AU)
- La Trobe University (AU)
- Venue
- IEEE Access
- Published
- 2021
- Volume
- 9
- Pages
- 144352-144360
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W3131037858
- Semantic Scholar
- 6573fa74af238d1e3538c026997e31b9f67f19f7
- Cited by
- 347
- References
- 64
- Citations by year
- 2026: 56 · 2025: 102 · 2024: 86 · 2023: 58 · 2022: 34 · 2021: 11
- FWCI
- 16.63
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- IEEE Access (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- arXiv (Cornell University)
- La Trobe University (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Game Theory and Voting Systems
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Keywords
- Shapley value, feature selection, explainable AI, axioms, SHAP, sage, counterexample, transferable utility games, attribution
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.