CoChat
Sign Up
PreprintCited by 23Explainable Artificial Intelligence (XAI)Software Testing and Debugging TechniquesPrivacy-Preserving Technologies in Data

Interpretable & Explorable Approximations of Black Box Models

Open access
Lakkaraju, Himabindu, Kamar, Ece, Caruana, Rich, Leskovec, Jure·2017·arXiv
DOI 10.48550/arxiv.1707.01154arXiv 1707.01154
Open this paper in CoChat — free accountOpen PDF

Abstract

We propose Black Box Explanations through Transparent Approximations (BETA), a novel model agnostic framework for explaining the behavior of any black-box classifier by simultaneously optimizing for fidelity to the original model and interpretability of the explanation. To this end, we develop a novel objective function which allows us to learn (with optimality guarantees), a small number of compact decision sets each of which explains the behavior of the black box model in unambiguous, well-defined regions of feature space. Furthermore, our framework also is capable of accepting user input when generating these approximations, thus allowing users to interactively explore how the black-box model behaves in different subspaces that are of interest to the user. To the best of our knowledge, this is the first approach which can produce global explanations of the behavior of any given black box model through joint optimization of unambiguity, fidelity, and interpretability, while also allowing users to explore model behavior based on their preferences. Experimental evaluation with real-world datasets and user studies demonstrates that our approach can generate highly compact, easy-to-understand, yet accurate approximations of various kinds of predictive models compared to state-of-the-art baselines.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.1707.01154

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Making a complex algorithm understandable to end users can require simplification, which often comes at the cost of reduced accuracy of explanation.

    “Making a complex algorithm understandable to end users can require simplification, which often comes at the cost of reduced accuracy of explanation [46,47,53,54,73].”

    From Progressive Disclosure: When, Why, and How Do Users Want Algorithmic Transparency Information? (2020) · cites this paper as [47]

  • Newer technical approaches attempt to increase the intelligibility of complex neural models by approximating them using simpler, more intuitive models.

    “Newer technical approaches attempt to increase intelligibility of complex neural models by approximating them using simpler more intuitive models [47,53,54].”

    From Progressive Disclosure: When, Why, and How Do Users Want Algorithmic Transparency Information? (2020) · cites this paper as [47]

  • In [54], a two-level boolean rule predictor is used in a reverse engineering approach to explain the black box.

    “In [54] the authors propose a method f exploiting a two-level boolean rule predictor to solve the black box model explanation, i.e., the transparent approach is used in the reverse engineering approach to explain the black box.”

    From A Survey of Methods for Explaining Black Box Models (2019) · cites this paper as [54]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 8

  1. [2][2] S. Khuller, A. Moss, and J. S. Naor. The budgeted maximum coverage problem. Information Processing Letters , 70(1):39-45, 1999. -
  2. [3][3] P. W. Koh and P. Liang. Understanding black-box predictions via influence functions. arXiv preprint arXiv:1703.04730 , 2017. - · arXiv
  3. [4][4] H. Lakkaraju, S. H. Bach, and J. Leskovec. Interpretable decision sets: A joint framework for description and prediction. In KDD , 2016. -
  4. [5][5] J. Lee, V. S. Mirrokni, V. Nagarajan, and M. Sviridenko. Non-monotone submodular maximization under matroid and knapsack constraints. In Proceedings of the fortyfirst annual ACM symposium on Theory of computing , pages 323-332. ACM, 2009. -
  5. [6][6] B. Letham, C. Rudin, T. H. McCormick, D. Madigan, et al. Interpretable classifiers using rules and bayesian analysis: Building a better stroke prediction model. The Annals of Applied Statistics , 9(3):1350-1371, 2015. -
  6. [7][7] Y. Lou, R. Caruana, and J. Gehrke. Intelligible models for classification and regression. In Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining , pages 150-158. ACM, 2012. -
  7. [8][8] M. T. Ribeiro, S. Singh, and C. Guestrin. Why should i trust you?: Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , pages 1135-1144. ACM, 2016. -
  8. [9][9] L. Rokach and O. Maimon. Top-down induction of decision trees classifiers-a survey. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) , 35(4):476-487, 2005.

Cited by

Sorted by citations

Authors
Lakkaraju, H., Kamar, E., Caruana, R., & Leskovec, J.
Affiliations
Stanford University (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2017
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.1707.01154
arXiv
1707.01154
OpenAlex
W4293865119
Cited by
23
References
0
Citations by year
2026: 1 · 2025: 3 · 2024: 3 · 2023: 3 · 2022: 1 · 2021: 7 · 2020: 3 · 2019: 2
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
Keywords
model-agnostic explanation, explanation fidelity, interpretability, decision sets, interactive exploration, global explanations, explainable AI

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review