The many Shapley values for model explanation
Open accessThe DOI matches its OpenAlex record; 6 of the 6 references it cites were found in a registry or an index.
Abstract
The Shapley value has become a popular method to attribute the prediction of a machine-learning model on an input to its base features. The use of the Shapley value is justified by citing [16] showing that it is the \emph{unique} method that satisfies certain good properties (\emph{axioms}). There are, however, a multiplicity of ways in which the Shapley value is operationalized in the attribution problem. These differ in how they reference the model, the training data, and the explanation context. These give very different results, rendering the uniqueness result meaningless. Furthermore, we find that previously proposed approaches can produce counterintuitive attributions in theory and in practice---for instance, they can assign non-zero attributions to features that are not even referenced by the model. In this paper, we use the axiomatic approach to study the differences between some of the many operationalizations of the Shapley value for attribution, and propose a technique called Baseline Shapley (BShap) that is backed by a proper uniqueness result. We also contrast BShap with Integrated Gradients, another extension of Shapley value to the continuous setting.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707
Efron, B., Hastie, T., Johnstone, I., & Tibshirani, R. (2004) · The Annals of Statistics · doi:10.1214/009053604000000067 · Cited by 9,357
Explaining prediction models and individual predictions with feature contributions
Štrumbelj, E., & Kononenko, I. (2014) · Knowledge and Information Systems · doi:10.1007/s10115-013-0679-x · Cited by 2,286 · cited 3×
“For instance, [10], assumes an independent feature distribution while [2] applies it to modules of a deep network rather than end-to-end.”
BackgroundMethodConsistent Individualized Feature Attribution for Tree Ensembles
Lundberg, S. M., Erion, G. G., & Lee, S.-I. (2018) · arXiv · doi:10.48550/arxiv.1802.03888 · Cited by 1,430
Datta, A., Sen, S., & Zick, Y. (2016) · 2016 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2016.42 · Cited by 728 · cited 5×
“[11] runs CES over an independent feature distribution D ′ that is a product of the marginal distributions of the input feature distribution D.”
BackgroundMethodInfluential
Cited by
From local explanations to global understanding with explainable AI for trees
Lundberg, S. M., Erion, G., Chen, H., DeGrave, A., Prutkin, J. M., Nair, B., Katz, R., Himmelfarb, J., Bansal, N., & Lee, S.-I. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-019-0138-9 · Cited by 10,551
“An equivalent formulation is the randomized baseline method discussed in [10].”
MethodInterpretable Machine Learning – A Brief History, State-of-the-Art and Challenges
Molnar, C., Casalicchio, G., & Bischl, B. (2020) · Communications in Computer and Information Science · doi:10.1007/978-3-030-65965-3_28 · Cited by 677
Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Survey
Das, A., & Rad, P. (2020) · arXiv · doi:10.48550/arxiv.2006.11371 · Cited by 504
Shapley values for feature selection: the good, the bad, and the axioms
Fryer, D., Strumke, I., & Nguyen, H. (2021) · IEEE Access · doi:10.1109/access.2021.3119110 · Cited by 347
The Shapley Value in Machine Learning
Rozemberczki, B., Watson, L., Bayer, P., Yang, H.-T., Kiss, O., Nilsson, S., & Sarkar, R. (2022) · arXiv · doi:10.48550/arxiv.2202.05594 · Cited by 312
The explanation game: Explaining machine learning models using shapley values
Merrick, L., & Taly, A. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-57321-8_2 · Cited by 277
Can HR adapt to the paradoxes of artificial intelligence?
Charlwood, A., & Guenole, N. (2022) · Human Resource Management Journal · doi:10.1111/1748-8583.12433 · Cited by 271
Artificial Intelligence and Biosensors in Healthcare and Its Clinical Relevance: A Review
(2023) · doi:10.1109/access.2023.3285596 · Cited by 205
Interpretable machine learning for imbalanced credit scoring datasets
Chen, Y., Calabrese, R., & Martin-Barragan, B. (2024) · European Journal of Operational Research · doi:10.1016/j.ejor.2023.06.036 · Cited by 190
Challenges and opportunities for artificial intelligence in auditing: Evidence from the field
Kokina, J., Blanchette, S., Davenport, T. H., & Pachamanova, D. (2025) · International Journal of Accounting Information Systems · doi:10.1016/j.accinf.2025.100734 · Cited by 375
General Pitfalls of Model-Agnostic Interpretation Methods for Machine Learning Models
Molnar, C., König, G., Herbinger, J., Freiesleben, T., Dandl, S., Scholbeck, C. A., Casalicchio, G., Grosse-Wentrup, M., & Bischl, B. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-04083-2_4 · Cited by 211
“Similarly, conditional SAGE and conditional SHAP value functions sample the remaining features conditional on the feature of interest and therefore violate sensitivity [48,93,21,53].”
BackgroundEXplainable Artificial Intelligence (XAI)—From Theory to Methods and Applications
Ortigossa, E. S., Gonçalves, T., & Nonato, L. G. (2024) · IEEE Access · doi:10.1109/access.2024.3409843 · Cited by 124
Foundation Models and Fair Use
Henderson, P., Li, X., Jurafsky, D., Hashimoto, T., Lemley, M. A., & Liang, P. (2023) · doi:10.2139/ssrn.4404340 · Cited by 57
Fresz, B., Dubovitskaya, E., Brajovic, D., Huber, M. F., & Horz, C. (2024) · Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1609/aies.v7i1.31648 · Cited by 17
“Thus, the simplest way to aid in a plausibility assessment is to use explanations such as feature importance with axiomatic properties for Correctness and Complete-ness such as SHAP, although the actual value of such properties is part of an ongoing discussion [17, 22, 37].”
BackgroundIs Genre Enough? A Theory of Genre Signaling as Generative AI Rhetoric
Omizo, R., & Hart-Davidson, B. (2024) · Rhetoric Society Quarterly · doi:10.1080/02773945.2024.2343615 · Cited by 16
Metadata
- Authors
- Sundararajan, M., & Najmi, A.
- Venue
- RePEc: Research Papers in Economics
- Publisher
- arXiv
- Published
- 2019
- Volume
- 1
- Pages
- 9269-9278
- arXiv
- 1908.08474
- OpenAlex
- W2969551072
- Cited by
- 35
- References
- 6
- Type
- posted content
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.