Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Cited by 158

The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.

Review Study of Interpretation Methods for Future Interpretable Machine Learning

Open access
Mi, Jian-Xun, Li, An-Di, Zhou, Li-Fang · 2020 · IEEE Access

Abstract

In recent years, black-box models have developed rapidly because of their high accuracy. Balancing the interpretability and accuracy is increasingly important. The lack of interpretability severely limits the application of the model in academia and industry. Despite the various interpretable machine learning methods, the perspective and meaning of the interpretation are also different. We provide a review of the current interpretable methods and divide them based on the model being applied. We divide them into two categories: interpretable methods with the self-explanatory model and interpretable methods with external co-explanation. And the interpretable methods with external co-explanation are further divided into subbranch methods based on instances, SHAP, knowledge graph, deep learning, and clustering model. The classification aims to help us understand the model characteristics applied in the interpretable method better. This survey makes the researcher find a suitable model to solve interpretability problems more easily. And the comparison experiments contribute to discovering complementary features from different methods. At the same time, we explore the future challenges and trends of interpretable machine learning to promote the development of interpretable machine learning.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/access.2020.3032756

Cites

32 of 32 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 32 references

Research works printed but not linked: 73

  1. 10.1080/00401706.2000.10485981 · DOI
  2. 10.1177/001872089606380203 · DOI · Paper page
  3. kim, 2016, Examples are not enough, learn to criticize! criticism for interpretability, Proc Adv Neural Inf Process Syst, 2280
  4. kim, 2014, The Bayesian case model: A generative approach for case-based reasoning and prototype classification, Proc Adv Neural Inf Process Syst, 1952
  5. 10.1016/j.artmed.2005.10.008 · DOI
  6. 10.1145/2623330.2623637 · DOI · Paper page
  7. 10.1016/j.ins.2019.10.050 · DOI
  8. wang, 2015, Falling rule lists, J Mach Learn Res, 38, 1013
  9. bertsimas, 2018, Interpretable clustering via optimal trees, arXiv 1812 00539
  10. craven, 1996, Extracting tree-structured representations of trained networks, Proc Adv Neural Inf Process Syst, 24
  11. 10.1109/ICDM.2016.0097 · DOI
  12. 10.1145/775047.775113 · DOI
  13. 10.1016/S0031-3203(98)00181-2 · DOI
  14. zhou, 2003, Extracting symbolic rules from trained neural network ensembles, AI Commun, 16, 3
  15. 10.1016/j.procs.2018.09.020 · doi:10.1016/j.procs.2018.09.020
  16. 10.1145/3308558.3313607 · DOI · Paper page
  17. 10.1609/aaai.v33i01.33013036 · DOI
  18. sarker, 2017, Explaining trained neural networks with semantic Web technologies: First steps, Proc CEUR Workshop, 1
  19. geng, 2019, Human-centric transfer learning explanation via knowledge graph, arXiv 1901 08547
  20. 10.1145/3077136.3080777 · DOI · Paper page
  21. 10.1145/2939672.2939673 · DOI · Paper page
  22. 10.1609/aaai.v33i01.33015329 · DOI · Paper page
  23. 10.1145/3240323.3240361 · DOI · Paper page
  24. catherine, 2017, Explainable entity-based recommendations with knowledge graphs, Proc CEUR Workshop, 1
  25. 10.1109/UIC-ATC.2017.8397411 · DOI · Paper page
  26. 10.1145/3359786 · DOI · Paper page
  27. 10.1145/3219819.3220072 · DOI
  28. 10.1145/3236009 · DOI · Paper page
  29. 10.1007/s00778-015-0394-1 · DOI · Paper page
  30. minervini, 2017, Regularizing Knowledge Graph Embeddings via Equivalence and Inversion Axioms, 668
  31. janzing, 2019, Feature relevance quantification in explainable AI: A causal problem, arXiv 1910 13413
  32. lundberg, 2018, Consistent individualized feature attribution for tree ensembles, arXiv 1802 03888
  33. sundararajan, 2019, The many shapley values for model explanation, arXiv 1908 08474
  34. 10.1007/978-3-030-01237-3_8 · DOI · Paper page
  35. 10.1109/CVPR.2016.319 · DOI · Paper page
  36. singh, 2019, Hierarchical interpretations for neural network predictions, Proc Int Conf Learn Represent (ICLR), 1
  37. zharov, 2018, YASENN: Explaining neural networks via partitioning activation sequences, arXiv 1811 02783
  38. zhang, 2018, Interpreting CNN knowledge via an explanatory graph, Proc AAAI Conf Artif Intell, 4454
  39. 10.1109/CVPR.2018.00920 · DOI · Paper page
  40. huang, 2017, Densely connected convolutional networks, Proc IEEE Conf Comput Vis Pattern Recognit (CVPR), 4700
  41. 10.1007/978-3-319-10590-1_53 · DOI · Paper page
  42. yosinski, 2015, Understanding neural networks through deep visualization, arXiv 1506 06579
  43. bibal, 2016, Interpretability of machine learning models and representations: an introduction, Proc Eur Symp Artif Neural Netw (ESANN), 77
  44. binder, 2016, Layer-Wise Relevance Propagation for Deep Neural Network Architectures, 913
  45. 10.1109/CVPR.2018.00907 · DOI · Paper page
  46. 10.1109/CVPR.2016.308 · DOI · Paper page
  47. kinkeldey, 2019, Towards Supporting Interpretability of Clustering Results with Uncertainty Visualization, Proc EuroVis Workshop Trustworthy Visualizat (TrustVis), 1
  48. alaa ahmed, 2019, Demystifying black-box models with symbolic metamodels, Proc Adv Neural Inf Process Syst, 11304
  49. 10.1016/j.cag.2014.01.006 · DOI
  50. yoon, 2019, INVASE: Instance-wise variable selection using neural networks, Proc Int Conf Learn Represent (ICLR), 1
  51. 10.1016/j.neucom.2006.11.018 · DOI
  52. 10.1109/VAST.2018.8802509 · DOI
  53. laurens, 2008, Visualizing data using t-SNE, J Mach Learn Res, 9, 2579
  54. ilyas, 2019, Adversarial examples are not bugs, they are features, Proc Adv Neural Inf Process Syst, 125
  55. 10.1109/CVPR.2015.7298594 · DOI · Paper page
  56. simonyan, 2015, Very deep convolutional networks for large-scale image recognition, Proc 3rd Int Conf Learn Represent ICLR Conf Track, 1
  57. lecun, 1999, The MNIST Database of Handwritten Digits
  58. 10.1109/CVPR.2017.544 · DOI · Paper page
  59. molnar, 2019, Example-based explanations, Interpretable Machine Learning A Guide for Making Black Box Models Explainable
  60. 10.1007/978-3-319-45654-6_1 · doi:10.1007/978-3-319-45654-6_1
  61. ravikumar, 2009, SpAM: Sparse additive models, Proc Adv Neural Inf Process Syst, 1
  62. 10.1145/2939672.2939778 · DOI · Paper page
  63. 10.1145/3243734.3243792 · DOI · Paper page
  64. 10.1109/TPAMI.2018.2858759 · DOI · Paper page
  65. 10.1109/IS.2006.348467 · DOI
  66. oramas, 2019, Visual explanation by interpretation: Improving visual feedback capabilities of deep neural networks, Proc Int Conf Learn Represent (ICLR), 1
  67. 10.1007/s11634-013-0129-3 · DOI
  68. bau, 2019, GAN dissection: Visualizing and understanding generative adversarial networks, Proc Int Conf Learn Represent (ICLR), 1
  69. 10.1145/3375627.3375843 · DOI
  70. hsu, 2017, Unsupervised learning of disentangled and interpretable representations from sequential data, Proc Adv Neural Inf Process Syst, 1879
  71. 10.1007/978-3-319-49055-7_38 · doi:10.1007/978-3-319-49055-7_38
  72. antwarg, 2019, Explaining anomalies detected by autoencoders using SHAP, arXiv 1903 02407
  73. shankaranarayana, 2019, ALIME Autoencoder Based Approach for Local Interpretability, 454

Cited by

Metadata

Authors
Mi, J.-X., Li, A.-D., & Zhou, L.-F.
Venue
IEEE Access
Published
2020
Volume
8
Pages
191969-191985
ISSN
2169-3536 (ISSN-L)
OpenAlex
W3093681468
Cited by
158
References
32
FWCI
5.11
Type
journal article
Language
en
Open access
gold
Funders
  • Natural Science Foundation of Chongqing (cstc2018jcyjAX0532, cstc2019jcyj-msxmX0461)
  • National Natural Science Foundation of China (61906024)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.