Shedding Light on Black Box Machine Learning Algorithms: Development of an Axiomatic Framework to Assess the Quality of Methods that Explain Individual Predictions
Open accessAbstract
From self-driving vehicles and back-flipping robots to virtual assistants who book our next appointment at the hair salon or at that restaurant for dinner - machine learning systems are becoming increasingly ubiquitous. The main reason for this is that these methods boast remarkable predictive capabilities. However, most of these models remain black boxes, meaning that it is very challenging for humans to follow and understand their intricate inner workings. Consequently, interpretability has suffered under this ever-increasing complexity of machine learning models. Especially with regards to new regulations, such as the General Data Protection Regulation (GDPR), the necessity for plausibility and verifiability of predictions made by these black boxes is indispensable. Driven by the needs of industry and practice, the research community has recognised this interpretability problem and focussed on developing a growing number of so-called explanation methods over the past few years. These methods explain individual predictions made by black box machine learning models and help to recover some of the lost interpretability. With the proliferation of these explanation methods, it is, however, often unclear, which explanation method offers a higher explanation quality, or is generally better-suited for the situation at hand. In this thesis, we thus propose an axiomatic framework, which allows comparing the quality of different explanation methods amongst each other. Through experimental validation, we find that the developed framework is useful to assess the explanation quality of different explanation methods and reach conclusions that are consistent with independent research.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The paper adopts the view that interpretability is the broader concept that includes explainability, and that interpretability is the end-goal achieved through explanations.
“In this paper, we follow scholars who view “interpretability” as the broader concept that, inter alia, includes “explainability.” Specifically, we agree with the statement of M. Honneger that “interpretability is the end-goal we want to achieve, and explanations are the tools to reach interpretability” (Honegger, 2018, citation in Carvalho et al., 2019).”
From Transparency of AI in Healthcare as a Multilayered System of Accountabilities: Between Legal Requirements and Technical Limitations (2022) · cites this paper as Honegger, 2018
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
No citation lists for this paper
No citation lists for this paper yet — checked again daily.
References 22 · Cited by 32
Research works printed but not linked: 40
- Altenberger, F., & Lenz, C. (2018). A non-technical survey on deep convolutional neural network architectures. arXiv preprint arXiv:1803.02129 . · arXiv
- Bahmani, B., Moseley, B., Vattani, A., Kumar, R., & Vassilvitskii, S. (2012). Scalable k-means++. Proceedings of the VLDB Endowment , 5 (7), 622-633.
- Beillevaire, M. (2018). Inside the black box: How to explain individual predictions of a machine learning model (Master Thesis). KTH, School of Electrical Engineering and Computer Science (EECS).
- Bibal, A., & Fr´ enay, B. (2016). Interpretability of machine learning models and representations: An introduction. In Proceedings on esann (pp. 77-82).
- Binder, A., Montavon, G., Lapuschkin, S., M¨ uller, K.-R., & Samek, W. (2016). Layerwise relevance propagation for neural networks with local renormalization layers. In International conference on artificial neural networks (pp. 63-71).
- Bonnans, J. F., & Shapiro, A. (2013). Perturbation analysis of optimization problems . Springer Science & Business Media.
- Breiman, L. (1996). Bagging predictors. Machine learning , 24 (2), 123-140.
- Cali´ nski, T., & Harabasz, J. (1974). A dendrite method for cluster analysis. Communications in Statistics-theory and Methods , 3 (1), 1-27.
- Chen, T., & Guestrin, C. (2016). Xgboost: A scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining (pp. 785-794).
- Doshi-Velez, F., & Kim, B. (2017). Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608 . · arXiv
- Doshi-Velez, F., Kortz, M., Budish, R., Bavitz, C., Gershman, S., O'Brien, D., . . . Wood, A. (2017). Accountability of ai under the law: The role of explanation. arXiv preprint arXiv:1711.01134 . · arXiv
- Ester, M., Kriegel, H.-P., Sander, J., Xu, X., et al. (1996). A density-based algorithm for discovering clusters in large spatial databases with noise. In Kdd (Vol. 96, pp. 226-231).
- Freund, Y., Schapire, R. E., et al. (1996). Experiments with a new boosting algorithm. In Icml (Vol. 96, pp. 148-156).
- Friedman, J., Hastie, T., & Tibshirani, R. (2009). The elements of statistical learning 2nd edition . New York: Springer.
- Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep learning . MIT Press. ( http:// www.deeplearningbook.org )
- Goodman, B., & Flaxman, S. (2016). Eu regulations on algorithmic decision-making and a 'right to explanation'. In Icml workshop on human interpretability in machine learning (whi 2016), new york, ny. http://arxiv. org/abs/1606.08813 v1.
- Gregory, W., & Foreman, D. (2009). Nonparametric statistics for non-statisticians. Hoboken: John Wiley and Sons .
- Hartigan, J. A. (1981). Consistency of single linkage for high-density clusters. Journal of the American Statistical Association , 76 (374), 388-394.
- Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural computation , 9 (8), 1735-1780.
- Jardine, N., & Sibson, R. (1971). Mathematical taxonomy. London: John Wiley .
- Kamwa, I., Samantaray, S., & Jo´ os, G. (2012). On the accuracy versus transparency trade-off of data-mining models for fast-response pmu-based catastrophe predictors. IEEE Transactions on Smart Grid , 3 (1), 152-161.
- Lipton, Z. C. (2016). The mythos of model interpretability. arXiv preprint arXiv:1606.03490 . · arXiv
- Lou, Y., Caruana, R., & Gehrke, J. (2012). Intelligible models for classification and regression. In Proceedings of the 18th acm sigkdd international conference on knowledge discovery and data mining (pp. 150-158).
- Lundberg, S. M., & Lee, S.-I. (2017a). Consistent feature attribution for tree ensembles. arXiv preprint arXiv:1706.06060 . · arXiv
- Lundberg, S. M., & Lee, S.-I. (2017b). A unified approach to interpreting model predictions. In Advances in neural information processing systems (pp. 4768-4777).
- Maclin, R., & Opitz, D. (1997). An empirical evaluation of bagging and boosting. AAAI/IAAI , 1997 , 546-551.
- MacQueen, J., et al. (1967). Some methods for classification and analysis of multivariate observations. In Proceedings of the fifth berkeley symposium on mathematical statistics and probability (Vol. 1, pp. 281-297).
- Miller, T. (2017). Explanation in artificial intelligence: Insights from the social sciences. arXiv preprint arXiv:1706.07269 . · arXiv
- Palczewska, A., Palczewski, J., Robinson, R. M., & Neagu, D. (2014). Interpreting random forest classification models using a feature contribution method. In integration of reusable systems (pp. 193-218). Springer.
- Quinlan, J. R. (1996). Improved use of continuous attributes in c4. 5. Journal of artificial intelligence research , 4 , 77-90.
- Ribeiro, M. T., Singh, S., & Guestrin, C. (2016). Why should i trust you?: Explaining the predictions of any classifier. In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining (pp. 1135-1144).
- R¨ uping, S. (2006). Learning interpretable models (PhD Thesis). Technical University of Dortmund.
- Samuel, A. L. (1959). Some studies in machine learning using the game of checkers. IBM Journal of research and development , 3 (3), 210-229.
- Schalkoff, R. J. (1997). Artificial neural networks (Vol. 1). McGraw-Hill New York.
- Shapley, L. S. (1953). A value for n-person games. Contributions to the Theory of Games , 2 (28), 307-317.
- Shearer, C. (2000). The crisp-dm model: the new blueprint for data mining. Journal of data warehousing , 5 (4), 13-22.
- Shrikumar, A., Greenside, P., & Kundaje, A. (2017). Learning important features through propagating activation differences. arXiv preprint arXiv:1704.02685 . · arXiv
- Spearman, C. (1904). The proof and measurement of association between two things. The American Journal of Psychology , 15 (1), 72-101.
- Sundararajan, M., Taly, A., & Yan, Q. (2017). Axiomatic attribution for deep networks. arXiv preprint arXiv:1703.01365 . · arXiv
- Ward Jr, J. H. (1963). Hierarchical grouping to optimize an objective function. Journal of the American statistical association , 58 (301), 236-244.
- Authors
- Honegger, M.
- Corresponding author
- Honegger, Milo
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2018
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1808.05054
- OpenAlex
- W2887189984
- Cited by
- 32
- References
- 22
- Citations by year
- 2025: 2 · 2024: 5 · 2023: 4 · 2022: 9 · 2021: 5 · 2020: 3 · 2019: 4
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Keywords
- explainable AI, axiomatic framework, black-box machine learning, interpretability, General Data Protection Regulation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
