Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Cited by 158

The DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.

Review Study of Interpretation Methods for Future Interpretable Machine Learning

Open access
Mi, Jian-Xun, Li, An-Di, Zhou, Li-Fang · 2020 · IEEE Access

Abstract

In recent years, black-box models have developed rapidly because of their high accuracy. Balancing the interpretability and accuracy is increasingly important. The lack of interpretability severely limits the application of the model in academia and industry. Despite the various interpretable machine learning methods, the perspective and meaning of the interpretation are also different. We provide a review of the current interpretable methods and divide them based on the model being applied. We divide them into two categories: interpretable methods with the self-explanatory model and interpretable methods with external co-explanation. And the interpretable methods with external co-explanation are further divided into subbranch methods based on instances, SHAP, knowledge graph, deep learning, and clustering model. The classification aims to help us understand the model characteristics applied in the interpretable method better. This survey makes the researcher find a suitable model to solve interpretability problems more easily. And the comparison experiments contribute to discovering complementary features from different methods. At the same time, we explore the future challenges and trends of interpretable machine learning to promote the development of interpretable machine learning.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/access.2020.3032756

Cites

29 of 29 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 76

  1. 10.1080/00401706.2000.10485981 · DOI
  2. 10.1177/001872089606380203 · DOI · Paper page
  3. 10.1109/ICCV.2017.371 · DOI · Paper page
  4. kim, 2016, Examples are not enough, learn to criticize! criticism for interpretability, Proc Adv Neural Inf Process Syst, 2280
  5. kim, 2014, The Bayesian case model: A generative approach for case-based reasoning and prototype classification, Proc Adv Neural Inf Process Syst, 1952
  6. 10.1016/j.artmed.2005.10.008 · DOI
  7. 10.1145/2623330.2623637 · DOI · Paper page
  8. 10.1016/j.ins.2019.10.050 · DOI
  9. wang, 2015, Falling rule lists, J Mach Learn Res, 38, 1013
  10. bertsimas, 2018, Interpretable clustering via optimal trees, arXiv 1812 00539
  11. craven, 1996, Extracting tree-structured representations of trained networks, Proc Adv Neural Inf Process Syst, 24
  12. 10.1109/ICDM.2016.0097 · DOI
  13. 10.1145/775047.775113 · DOI
  14. 10.1016/S0031-3203(98)00181-2 · DOI
  15. zhou, 2003, Extracting symbolic rules from trained neural network ensembles, AI Commun, 16, 3
  16. 10.1016/j.procs.2018.09.020 · doi:10.1016/j.procs.2018.09.020
  17. 10.1145/3308558.3313607 · DOI · Paper page
  18. 10.1609/aaai.v33i01.33013036 · DOI
  19. sarker, 2017, Explaining trained neural networks with semantic Web technologies: First steps, Proc CEUR Workshop, 1
  20. geng, 2019, Human-centric transfer learning explanation via knowledge graph, arXiv 1901 08547
  21. 10.1145/3077136.3080777 · DOI · Paper page
  22. 10.1145/2939672.2939673 · DOI · Paper page
  23. 10.1609/aaai.v33i01.33015329 · DOI · Paper page
  24. 10.1145/3240323.3240361 · DOI · Paper page
  25. catherine, 2017, Explainable entity-based recommendations with knowledge graphs, Proc CEUR Workshop, 1
  26. 10.1109/UIC-ATC.2017.8397411 · DOI · Paper page
  27. 10.1145/3359786 · DOI · Paper page
  28. 10.1145/3219819.3220072 · DOI
  29. 10.1145/3236009 · DOI · Paper page
  30. 10.1007/s00778-015-0394-1 · DOI · Paper page
  31. minervini, 2017, Regularizing Knowledge Graph Embeddings via Equivalence and Inversion Axioms, 668
  32. janzing, 2019, Feature relevance quantification in explainable AI: A causal problem, arXiv 1910 13413
  33. lundberg, 2018, Consistent individualized feature attribution for tree ensembles, arXiv 1802 03888
  34. 10.1109/ACCESS.2020.2988359 · DOI · Paper page
  35. sundararajan, 2019, The many shapley values for model explanation, arXiv 1908 08474
  36. 10.1007/978-3-030-01237-3_8 · DOI · Paper page
  37. shrikumar, 2017, Learning important features through propagating activation differences, Proc Int Conf Mach Learn (ICML), 4844
  38. 10.1109/CVPR.2016.319 · DOI · Paper page
  39. singh, 2019, Hierarchical interpretations for neural network predictions, Proc Int Conf Learn Represent (ICLR), 1
  40. zharov, 2018, YASENN: Explaining neural networks via partitioning activation sequences, arXiv 1811 02783
  41. zhang, 2018, Interpreting CNN knowledge via an explanatory graph, Proc AAAI Conf Artif Intell, 4454
  42. 10.1109/CVPR.2018.00920 · DOI · Paper page
  43. huang, 2017, Densely connected convolutional networks, Proc IEEE Conf Comput Vis Pattern Recognit (CVPR), 4700
  44. 10.1007/978-3-319-10590-1_53 · DOI · Paper page
  45. yosinski, 2015, Understanding neural networks through deep visualization, arXiv 1506 06579
  46. bibal, 2016, Interpretability of machine learning models and representations: an introduction, Proc Eur Symp Artif Neural Netw (ESANN), 77
  47. binder, 2016, Layer-Wise Relevance Propagation for Deep Neural Network Architectures, 913
  48. 10.1109/CVPR.2018.00907 · DOI · Paper page
  49. 10.1109/CVPR.2016.308 · DOI · Paper page
  50. kinkeldey, 2019, Towards Supporting Interpretability of Clustering Results with Uncertainty Visualization, Proc EuroVis Workshop Trustworthy Visualizat (TrustVis), 1
  51. alaa ahmed, 2019, Demystifying black-box models with symbolic metamodels, Proc Adv Neural Inf Process Syst, 11304
  52. 10.1016/j.cag.2014.01.006 · DOI
  53. yoon, 2019, INVASE: Instance-wise variable selection using neural networks, Proc Int Conf Learn Represent (ICLR), 1
  54. 10.1016/j.neucom.2006.11.018 · DOI
  55. 10.1109/VAST.2018.8802509 · DOI
  56. laurens, 2008, Visualizing data using t-SNE, J Mach Learn Res, 9, 2579
  57. ilyas, 2019, Adversarial examples are not bugs, they are features, Proc Adv Neural Inf Process Syst, 125
  58. 10.1109/CVPR.2015.7298594 · DOI · Paper page
  59. simonyan, 2015, Very deep convolutional networks for large-scale image recognition, Proc 3rd Int Conf Learn Represent ICLR Conf Track, 1
  60. lecun, 1999, The MNIST Database of Handwritten Digits
  61. 10.1109/CVPR.2017.544 · DOI · Paper page
  62. molnar, 2019, Example-based explanations, Interpretable Machine Learning A Guide for Making Black Box Models Explainable
  63. 10.1007/978-3-319-45654-6_1 · doi:10.1007/978-3-319-45654-6_1
  64. ravikumar, 2009, SpAM: Sparse additive models, Proc Adv Neural Inf Process Syst, 1
  65. 10.1145/2939672.2939778 · DOI · Paper page
  66. 10.1145/3243734.3243792 · DOI · Paper page
  67. 10.1109/TPAMI.2018.2858759 · DOI · Paper page
  68. 10.1109/IS.2006.348467 · DOI
  69. oramas, 2019, Visual explanation by interpretation: Improving visual feedback capabilities of deep neural networks, Proc Int Conf Learn Represent (ICLR), 1
  70. 10.1007/s11634-013-0129-3 · DOI
  71. bau, 2019, GAN dissection: Visualizing and understanding generative adversarial networks, Proc Int Conf Learn Represent (ICLR), 1
  72. 10.1145/3375627.3375843 · DOI
  73. hsu, 2017, Unsupervised learning of disentangled and interpretable representations from sequential data, Proc Adv Neural Inf Process Syst, 1879
  74. 10.1007/978-3-319-49055-7_38 · doi:10.1007/978-3-319-49055-7_38
  75. antwarg, 2019, Explaining anomalies detected by autoencoders using SHAP, arXiv 1903 02407
  76. shankaranarayana, 2019, ALIME Autoencoder Based Approach for Local Interpretability, 454

Cited by

See all 158

Metadata

Authors
Mi, J.-X., Li, A.-D., & Zhou, L.-F.
Venue
IEEE Access
Published
2020
Volume
8
Pages
191969-191985
ISSN
2169-3536 (ISSN-L)
OpenAlex
W3093681468
Cited by
158
References
29
FWCI
5.11
Type
journal article
Language
en
Open access
gold
Funders
  • Natural Science Foundation of Chongqing (cstc2018jcyjAX0532, cstc2019jcyj-msxmX0461)
  • National Natural Science Foundation of China (61906024)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.