The DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
Explaining Classifications For Individual Instances
Abstract
We present a method for explaining predictions for individual instances. The presented approach is general and can be used with all classification models that output probabilities. It is based on decomposition of a model's predictions on individual contributions of each attribute. Our method works for so called black box models such as support vector machines, neural networks, and nearest neighbor algorithms as well as for ensemble methods, such as boosting and random forests. We demonstrate that the generated explanations closely follow the learned models and present a visualization technique which shows the utility of our approach and enables the comparison of different prediction methods.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
A Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb01338.x · Cited by 88,486
(1990) · Choice Reviews Online · doi:10.5860/choice.27-3338 · Cited by 35,305
Tversky, A., & Kahneman, D. (1974) · Science · doi:10.1126/science.185.4157.1124 · Cited by 28,810
Judgment under Uncertainty: Heuristics and Biases
Tversky, A., & Kahneman, D. (1975) · Utility, Probability, and Human Decision Making · doi:10.1007/978-94-010-1834-0_8 · Cited by 22,650
Hosmer, D. W., & Lemeshow, S. (2000) · doi:10.1002/0471722146 · Cited by 20,302
An Introduction to Variable and Feature Selection.
Guyon, I., & Elisseeff, A. (2003) · Cited by 6,576
Marginal Structural Models and Causal Inference in Epidemiology
Robins, J. M., Hernán, M. Á., & Brumback, B. (2000) · Epidemiology · doi:10.1097/00001648-200009000-00011 · Cited by 5,581
(2024) · Physics Subject Headings (PhySH) · doi:10.29172/7c2a6982-6d72-4cd8-bba6-2fccb06a7011 · Cited by 3,857
(2000) · CrossRef Listing of Deleted DOIs · doi:10.1162/153244303322753616 · Cited by 3,502
Research works printed but not linked: 5
- Mitchell, 1997, Machine Learning
- 10.1023/A:1022683529158 · DOI · Paper page
- Främling, Explaining Results of Neural Networks by Contextual Importance and Utility
- Becker, Visualizing the Simple Bayesian Classier
- 10.2307/1390929 · DOI
Cited by
Arrieta, A. B., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., García, S., Gil-López, S., Molina, D., Benjamins, R., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.10045 · Cited by 11,321
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294
“[93]–[95]).”
MethodExplanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302
Explaining prediction models and individual predictions with feature contributions
Štrumbelj, E., & Kononenko, I. (2014) · Knowledge and Information Systems · doi:10.1007/s10115-013-0679-x · Cited by 2,286
Self-citationExplainable artificial intelligence (XAI) in deep learning-based medical image analysis
van der Velden, B. H. M., Kuijf, H. J., Gilhuijs, K. G. A., & Viergever, M. A. (2022) · Medical Image Analysis · doi:10.1016/j.media.2022.102470 · Cited by 1,328
“(2017) adapted prediction difference analysis (Robnik-Šikonja and Kononenko (2008)) for generating saliency maps. If each pixel in an image is considered a feature, prediction difference analysis assigns a relevance value to each pixel, by measuring how the prediction changes if the pixel is considered unknown. Zintgr…”
MethodA Survey on the Explainability of Supervised Machine Learning
Burkart, N., & Huber, M. F. (2021) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.12228 · Cited by 1,008
Abdul, A., Vermeulen, J., Wang, D., Lim, B. Y., & Kankanhalli, M. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174156 · Cited by 743
Albahri, A. S., Duhaim, A. M., Fadhel, M. A., Alnoor, A., Baqer, N. S., Alzubaidi, L., Albahri, O. S., Alamoodi, A. H., Bai, J., Salhi, A., et al. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.03.008 · Cited by 716
Notions of explainability and evaluation approaches for explainable artificial intelligence
Vilone, G., & Longo, L. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.05.009 · Cited by 598
On Interpretability of Artificial Neural Networks: A Survey
Fan, F.-L., Xiong, J., Li, M., & Wang, G. (2021) · IEEE Transactions on Radiation and Plasma Medical Sciences · doi:10.1109/trpms.2021.3066428 · Cited by 561
Explainable Deep Learning: A Field Guide for the Uninitiated
Ras, G., Xie, N., Van Gerven, M., & Doran, D. (2022) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.13200 · Cited by 446
Analysis of Explainers of Black Box Deep Neural Networks for Computer Vision: A Survey
Buhrmester, V., Münch, D., & Arens, M. (2021) · Machine Learning and Knowledge Extraction · doi:10.3390/make3040048 · Cited by 403
Explainable AI Methods - A Brief Overview
Holzinger, A., Saranti, A., Molnar, C., Biecek, P., & Samek, W. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-04083-2_2 · Cited by 379
Deterministic Local Interpretable Model-Agnostic Explanations for Stable Explainability
Zafar, M. R., & Khan, N. (2021) · Machine Learning and Knowledge Extraction · doi:10.3390/make3030027 · Cited by 374
Wang, J., Li, Y., Gao, R. X., & Zhang, F. (2022) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2022.04.004 · Cited by 317
Utilizing big data analytics for information systems research: challenges, promises and guidelines
Müller, O., Junglas, I., Brocke, J. V., & Debortoli, S. (2016) · European Journal of Information Systems · doi:10.1057/ejis.2016.2 · Cited by 281
Explaining Data-Driven Document Classifications1
Martens, D., & Provost, F. (2014) · MIS Quarterly · doi:10.25300/misq/2014/38.1.04 · Cited by 264
Benchmarking and survey of explanation methods for black box models
Bodria, F., Giannotti, F., Guidotti, R., Naretto, F., Pedreschi, D., & Rinzivillo, S. (2023) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-023-00933-9 · Cited by 264
Explanation Methods in Deep Learning: Users, Values, Concerns and Challenges
Ras, G., van Gerven, M., & Haselager, P. (2018) · The Springer Series on Challenges in Machine Learning · doi:10.1007/978-3-319-98131-4_2 · Cited by 255
Perturbation-Based Explanations of Prediction Models
Robnik-Šikonja, M., & Bohanec, M. (2018) · Human–Computer Interaction Series · doi:10.1007/978-3-319-90403-0_9 · Cited by 171
Self-citationToolkit to support intelligibility in context-aware applications
Lim, B. Y., & Dey, A. K. (2010) · Proceedings of the 12th ACM international conference on Ubiquitous computing · doi:10.1145/1864349.1864353 · Cited by 143
Explainable Artificial Intelligence for Tabular Data: A Survey
Sahakyan, M., Aung, Z., & Rahwan, T. (2021) · IEEE Access · doi:10.1109/access.2021.3116481 · Cited by 135
“Strumbelj and Kononenko [63] were the first to use the Shapley value for outcome explanation.”
How should we regulate artificial intelligence?
Reed, C. (2018) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2017.0360 · Cited by 114
The role of domain expertise in trusting and following explainable AI decision support systems
Bayer, S., Gimpel, H., & Markgraf, M. (2022) · Journal of Decision Systems · doi:10.1080/12460125.2021.1958505 · Cited by 103
Survey on Explainable AI: Techniques, challenges and open issues
Abusitta, A., Li, M. Q., & Fung, B. C. M. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.124710 · Cited by 98
Explainable Artificial Intelligence: a Systematic Review
Vilone, G., & Longo, L. (2020) · arXiv · doi:10.48550/arxiv.2006.00093 · Cited by 269
Human-Centric Justification of Machine Learning Predictions
Biran, O., & McKeown, K. (2017) · Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/202 · Cited by 84
Mohamed, E., Sirlantzis, K., & Howells, G. (2022) · Displays · doi:10.1016/j.displa.2022.102239 · Cited by 67
Sequential Feature Explanations for Anomaly Detection
Siddiqui, M. A., Fern, A., Dietterich, T. G., & Wong, W.-K. (2019) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3230666 · Cited by 50
Contesting automated decisions
Bayamlioglu, E. (2018) · European Data Protection Law Review · doi:10.21552/edpl/2018/4/6 · Cited by 21
Yagiz, M. A., & Goktas, P. (2025) · arXiv · doi:10.48550/arxiv.2501.00790 · Cited by 12
Metadata
- Authors
- Robnik-Sikonja, M., & Kononenko, I.
- Venue
- IEEE Transactions on Knowledge and Data Engineering
- Published
- 2008-05
- Volume
- 20
- Issue
- 5
- Pages
- 589-600
- ISSN
- 1041-4347 (ISSN-L)
- OpenAlex
- W2113211233
- Cited by
- 254
- References
- 35
- FWCI
- 8.48
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.