From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI
Open accessThe DOI matches its Crossref record; 204 of the 208 references it cites were found in a registry or an index, and 4 are not checked yet; 17 of the 124 sources its claims cite were checked for support.
Abstract
The rising popularity of explainable artificial intelligence (XAI) to understand high-performing black boxes raised the question of how to evaluate explanations of machine learning (ML) models. While interpretability and explainability are often presented as a subjectively validated binary property, we consider it a multi-faceted concept. We identify 12 conceptual properties, such as Compactness and Correctness, that should be evaluated for comprehensively assessing the quality of an explanation. Our so-called Co-12 properties serve as categorization scheme for systematically reviewing the evaluation practices of more than 300 papers published in the past 7 years at major AI and ML conferences that introduce an XAI method. We find that one in three papers evaluate exclusively with anecdotal evidence, and one in five papers evaluate with users. This survey also contributes to the call for objective, quantifiable evaluation methods by presenting an extensive overview of quantitative XAI evaluation methods. Our systematic collection of evaluation methods provides researchers and practitioners with concrete tools to thoroughly validate, benchmark, and compare new and existing XAI methods. The Co-12 categorization scheme and our identified evaluation methods open up opportunities to include quantitative metrics as optimization criteria during model training to optimize for accuracy and interpretability simultaneously.
Verified 2026-10-05 · 4 references not checked yet
Overview
CoChat Summary
TL;DR
This systematic review analyzes how explainable AI methods are evaluated, proposing 12 conceptual explanation-quality properties (Co-12) and cataloging quantitative evaluation methods. Across 606 screened papers from 2014–2020 at 12 major conferences, 312 introduced an XAI method, and the authors find that 33% evaluate only with anecdotal evidence, 58% use quantitative evaluation, and 22% use user studies.
Key findings
- Of the 312 papers introducing an XAI method, 33% evaluated exclusively with anecdotal evidence, 58% applied quantitative evaluation, and 22% evaluated with human subjects in a user study; of those user studies, 23% used domain experts (application-grounded).
- The Co-12 properties are proposed as a multi-faceted categorization scheme; the most frequently evaluated properties were Coherence and Output-completeness, followed by Correctness, Compactness, and Covariate complexity.
- Most papers that quantitatively evaluated an XAI method evaluated only one or two Co-12 properties; 73 papers evaluated exactly one property, often Coherence.
- Quantitative evaluation methods exist for each Co-12 property, but evaluation practice remains concentrated on Coherence, Completeness, Compactness, and Correctness; the authors also report that quantitative evaluation slightly increased over time while anecdotal-only evaluation decreased, with user studies remaining around 20%.
Methods
- Systematic literature review: DBLP title keyword search (`explain* OR explanat* OR interpret*`) across 12 conferences (AAAI, IJCAI, NeurIPS, ICML, ICLR, CVPR, ICCV, ACL, WWW, ICDM, SIGKDD, SIGIR) for 2014–2020, conducted 4 May 2021; 606 papers found, 494 after excluding companion papers, 361 included after inclusion criteria, and 312 after filtering to papers introducing an XAI method.
- Each included paper was categorized along six dimensions (data type, explanation type, explanation problem, predictive model type, task type, method type) and reviewed for evaluation practices using the Co-12 properties; user studies were classified as application-grounded or human-grounded following Doshi-Velez and Kim.
- The review focuses on functionally grounded, quantitative evaluation methods without user studies, with user-study evaluation methods summarized in Supplementary Material; quantitative methods were clustered and named according to the Co-12 properties.
Limitations
- The paper states that it focuses on functionally grounded evaluation and provides only a concise overview of user-study evaluation methods in Supplementary Material.
- The paper states that it may be unreasonable to expect an XAI method to score well on all Co-12 properties, and that trade-offs are necessary: Coherence may contradict Correctness, and Completeness and Compactness may be diametrical opposites; application domain and practical feasibility determine which properties should be emphasized.
- The text makes evident that the corpus is limited to 12 computer science conferences from 2014–2020 and uses title keyword search, so coverage may miss other venues, terms, or later work; the authors also note that their venue selection may have influenced the higher quantitative-evaluation statistics, and Figure 5 notes that a manually added category may be high precision but potentially low recall.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
10 Valid · 3 Not valid · 63 Not checked · 97 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Since we could not find a complete and recent overview of explanation types, we reviewed recent XAI surveys [14,58,167,280] and adapted and extended the categories identified by Guidotti et al. [86] with these insights.”
§ From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI > Text
What the sources say
- Not checked yetcites [14]
- Not checked yetcites [58]
- Not checked yetcites [167]
- Not checked yetcites [280]
- Partly supportedcites [86] sourceExperimental — accuracy not yet measured
Guidotti et al. do state a coverage gap: the Introduction (1 Introduction, p. 3) says 'Existing works tend to provide just a general overview of the problem [59] highlighting unanswered questions and problems [24]' while others 'focus on particular aspects', so 'a clear classification considering simultaneously all these aspects is needed' — this supports the 'not a complete overview' part of the claim. What our copy does not support is the 'recent' qualifier: the paper never says the missing overview is a matter of recency (it says prior works are partial in scope, and its own Conclusion, p. 37, presents 'a comprehensive overview of methods proposed in the literature'). Separately, the categories the citing sentence credits to them are genuinely theirs (four problem categories plus the classification dimensions of Section 5, p. 16), so that half of the attribution holds. (LLM, conf=0.70)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Others do not explicitly evaluate their method with quantitative evaluation metrics, but present mathematical theory to support their claims (e.g., [39,98]).”
§ From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI > Our summary statistics on XAI evaluation:
What the sources say
- Partly supportedcites [39] sourceExperimental — accuracy not yet measured
The Introduction (p. 1) explicitly says the authors provide theoretical support for LS-Tree by relating it to the Banzhaf value. But the paper also reports quantitative experiments: Detecting overfitting (pp. 6–7) evaluates an LS-Tree-based diagnostic using interaction-score variances and permutation-test p-values. The claim that this paper does not explicitly evaluate its method quantitatively is therefore not supported without a narrower meaning of ‘evaluation metrics.’ (LLM, conf=0.83)
- Not checked yetcites [98]
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Besides using Incremental Deletion/Addition to evaluate correctness and output-completeness, compactness can be evaluated by counting how many features of the explanation need to be removed, perturbed or added to change the decision of the predictive model [7,34,70,123,165,221].”
§ From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI > Incremental Deletion/Addition can include the evaluation of output-completeness.
What the sources say
- Partly supportedcites [7] sourceExperimental — accuracy not yet measured
In §5 (Experiments; page unavailable), the paper evaluates explanation cardinality and length and separately counts changes to variables needed to change a Bayesian classifier’s classification. It does not evaluate compactness by counting features removed, perturbed, or added to an explanation; the counted changes are changes to variable values. The cited passages also do not discuss Incremental Deletion/Addition. (LLM, conf=0.84)
- Not checked yetcites [34]
- Partly supportedcites [70] sourceExperimental — accuracy not yet measured
Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
- Partly supportedcites [123] sourceExperimental — accuracy not yet measured
Split into 2 atoms — 0 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 1 unverifiable. Expand the claim to see per-atom verdicts.
- Partly supportedcites [165] sourceExperimental — accuracy not yet measured
Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
- Partly supportedcites [221] sourceExperimental — accuracy not yet measured
Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“If a model is successfully trained on a dataset with random labels, which has been shown to be possible with deep neural networks [294], then it has learned the data generation process by memorizing the random labels.”
§ From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI > 6.5 Functionally Evaluating Contrastivity
What the sources say
- Supportedcites [294] source
The citation is attached specifically to 'which has been shown to be possible with deep neural networks', and our copy states exactly that in Section 1.1 (The randomization test): 'Deep neural networks easily fit random labels... when trained on a completely random labeling of the true data, neural networks achieve 0 training error', with implication 1 being that 'the effective capacity of neural networks is sufficient for memorizing the entire data set'; Section 2.1 repeats this for CIFAR10 and ImageNet. So the fitted-random-labels-by-memorization part attributed to Zhang et al. is supported. Note that the sentence's consequent ('learned the data generation process') is the citing authors' own inference rather than something credited to this source; the source in fact says the opposite there ('the randomization breaks any relationship between the instance... and the label... By design, no generalization is possible on the randomized data'), so that framing should not be read back onto citation [294]. (LLM, conf=0.88)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Indirectly, these metrics also evaluate composition, since it has been shown that ROUGE and BLEU correlate with human judgments on fluency of a text [67].”
§ From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI > 6.11 Functionally Evaluating Coherence
What the sources say
- Supportedcites [67] source
Table 2 of Section 5.2 (pp. 9-10, rendered at p. 5 of our copy) reports Kendall's tau correlations between automatic metrics and expert human judgments per dimension: ROUGE variants reach 0.48-0.55 on Fluency (ROUGE-1 0.5240, ROUGE-4 0.5535) and BLEU 0.3321, i.e. both are positively correlated with human fluency ratings. The accompanying text confirms the trend: 'Model correlations increase considerably across the consistency and fluency dimensions' (5.2, p. 9). The only qualifier the citing sentence drops is scope: these correlations were computed system-level on CNN/DailyMail summarization outputs with multi-reference metrics, not on text fluency in general. (LLM, conf=0.82)
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
74 more claims
Create a free account to see all 173 claims and to verify the 107 references still open.
- Probably not a substantial claim. 70
- It may state two separate claims as one. 47
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, a process for evaluating AI output has yet to be standardized.”
From Procedure to Evaluate Artificial Intelligence (AI) Output for Educational and Clinical Tasks (2026) · cites this paper as Nauta et al., 2023
Experimental — accuracy not yet measuredIn the citing paper
“How to evaluate these methods and their explanations is still a popular research challenge [111,114,115], and further research is needed in this context.”
From Survey of Explainable AI Techniques in Healthcare (2023) · cites this paper as [114]
Citations
Cites
(2008) · Journal of Machine Learning Research · Cited by 33,051
Greedy function approximation: A gradient boosting machine
Friedman, J. H. (2001) · The Annals of Statistics · doi:10.1214/aos/1013203451 · Cited by 30,731 · cited 1×
“Examples include Partial Dependence Plot [72], Individual Conditional Expectation plot [81] and Feature Auditing [5].”
MethodGrad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.74 · Cited by 24,513
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Arrieta, A. B., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., García, S., Gil-López, S., Molina, D., Benjamins, R., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.10045 · Cited by 11,321 · cited 5×
“We consider related concepts such as fairness, safety, causality, ethical decision making and privacy [21, 63, 150] out of scope.”
BackgroundMethodInfluentialRudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561 · cited 2×
“We hypothesize that the model explanation is not often addressed since there is no guarantee that the original black box and the interpretable surrogate model agree on their internal reasoning [217].”
BackgroundBarredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423 · cited 5×
“We consider related concepts such as fairness, safety, causality, ethical decision making and privacy [21, 63, 150] out of scope.”
BackgroundMethodInfluentialPeeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294 · cited 6×
“The necessity for interpretable and explainable AI (XAI) therefore arises, aiming to make AI systems and their results more understandable to humans [1].”
BackgroundMethodResultInfluentialLipton, Z. C. (2018) · Queue · doi:10.1145/3236386.3241340 · Cited by 5,427
Explanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302
On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation
Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K.-R., & Samek, W. (2015) · PLOS ONE · doi:10.1371/journal.pone.0130140 · Cited by 4,524 · cited 1×
“Includes attention maps [222], perturbation masks [71] and Layer-Wise Relevance Propagation [20].”
Learning Important Features Through Propagating Activation Differences
Shrikumar, A., Greenside, P., & Kundaje, A. (2017) · arXiv · doi:10.48550/arxiv.1704.02685 · Cited by 2,761
Methods for interpreting and understanding deep neural networks
Montavon, G., Samek, W., & Müller, K.-R. (2018) · Digital Signal Processing · doi:10.1016/j.dsp.2017.10.011 · Cited by 2,665 · cited 3×
“Those terms, together with explainability and interpretability (and to some extent also intelligibility) are typically used interchangeably [29, 32, 93, 177], although a few distinguish between them [21, 173].”
BackgroundExplaining Explanations: An Overview of Interpretability of Machine Learning
Gilpin, L. H., Bau, D., Yuan, B. Z., Bajwa, A., Specter, M., & Kagal, L. (2018) · 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA) · doi:10.1109/dsaa.2018.00018 · Cited by 2,507 · cited 10×
“Hence, we address the frequently reported lack of quantitative evaluation methods [1, 21, 85, 91, 152] and respond to the call for automated and quantifiable evaluation metrics for robust and falsifiable explainability research [152].”
BackgroundMethodInfluentialUnderstanding deep learning (still) requires rethinking generalization
Zhang, C., Bengio, S., Hardt, M., Recht, B., & Vinyals, O. (2021) · Communications of the ACM · doi:10.1145/3446776 · Cited by 2,400
Definitions, methods, and applications in interpretable machine learning
Murdoch, W. J., Singh, C., Kumbier, K., Abbasi-Asl, R., & Yu, B. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1900654116 · Cited by 2,167 · cited 3×
“Those terms, together with explainability and interpretability (and to some extent also intelligibility) are typically used interchangeably [29, 32, 93, 177], although a few distinguish between them [21, 173].”
BackgroundAnchors: High-Precision Model-Agnostic Explanations
Ribeiro, M. T., Singh, S., & Guestrin, C. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11491 · Cited by 2,162 · cited 1×
“Decision Rules Logical rules, including decision sets [139], anchors [213], decision tables [108] and programs [261].”
BackgroundGoldstein, A., Kapelner, A., Bleich, J., & Pitkin, E. (2015) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.2014.907095 · Cited by 1,861 · cited 1×
“Examples include Partial Dependence Plot [73], Individual Conditional Expectation plot [82] and Feature Auditing [5].”
MethodMachine Learning Interpretability: A Survey on Methods and Metrics
Carvalho, D. V., Pereira, E. M., & Cardoso, J. S. (2019) · Electronics · doi:10.3390/electronics8080832 · Cited by 1,842 · cited 10×
“1Regarding terminology: ‘interpretability’ and ‘explainability’ are closely related and often used interchangeably in the XAI context [29, 32].”
BackgroundMethodInfluentialNetwork Dissection: Quantifying Interpretability of Deep Visual Representations
Bau, D., Zhou, B., Khosla, A., Oliva, A., & Torralba, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.354 · Cited by 1,389
Sanity Checks for Saliency Maps
Adebayo, J., Gilmer, J., Muelly, M., Goodfellow, I., Hardt, M., & Kim, B. (2018) · arXiv · doi:10.48550/arxiv.1810.03292 · Cited by 1,294 · cited 4×
“[2, 111, 136, 146]) argue that relying on such anecdotal evidence alone is insufficient and that other aspects of the explanations should be evaluated as well.”
BackgroundInfluentialSurvey and critique of techniques for extracting rules from trained artificial neural networks
Andrews, R., Diederich, J., & Tickle, A. B. (1995) · Knowledge-Based Systems · doi:10.1016/0950-7051(96)81920-4 · Cited by 1,225 · cited 1×
“Consistency checks that identical inputs have identical explanations [14, 101].”
BackgroundEvaluating the Visualization of What a Deep Neural Network Has Learned
Samek, W., Binder, A., Montavon, G., Lapuschkin, S., & Muller, K.-R. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2599820 · Cited by 1,164
Interpretable Explanations of Black Boxes by Meaningful Perturbation
Fong, R. C., & Vedaldi, A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.371 · Cited by 1,097 · cited 1×
“Includes attention maps [222], perturbation masks [71] and Layer-Wise Relevance Propagation [20].”
Interpreting the Latent Space of GANs for Semantic Face Editing
Shen, Y., Gu, J., Tang, X., & Zhou, B. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00926 · Cited by 1,063
Explainable machine learning for scientific insights and discoveries
Roscher, R., Bohn, B., Duarte, M. F., & Garcke, J. (2020) · IEEE Access · doi:10.1109/access.2020.2976199 · Cited by 1,040 · cited 1×
“Some argue that the terms are closely related but distinguish between them [21, 85, 181, 225], although there is no consensus on what the distinction exactly is [25].”
BackgroundA Survey on the Explainability of Supervised Machine Learning
Burkart, N., & Huber, M. F. (2021) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.12228 · Cited by 1,008
Interpretable machine learning
Parliamentary Office of Science and Technology, & Christie, L. (2020) · doi:10.58248/pn633 · Cited by 952
Visual interpretability for deep learning: a survey
Zhang, Q.-S., & Zhu, S.-C. (2018) · Frontiers of Information Technology & Electronic Engineering · doi:10.1631/fitee.1700808 · Cited by 950 · cited 1×
“Some surveys discuss evaluation as part of a broader review of XAI methods [1, 29, 32, 59, 81, 120, 170, 173, 175, 182, 303], or mainly discuss evaluation with user studies [52, 100, 136, 170].”
BackgroundOlah, C., Mordvintsev, A., & Schubert, L. (2017) · Distill · doi:10.23915/distill.00007 · Cited by 880
Interpretable Convolutional Neural Networks
Zhang, Q., Wu, Y. N., & Zhu, S.-C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00920 · Cited by 833
Markus, A. F., Kors, J. A., & Rijnbeek, P. R. (2021) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2020.103655 · Cited by 811
Explainable Reasoning over Knowledge Graphs for Recommendation
Wang, X., Wang, D., Xu, C., He, X., Cao, Y., & Chua, T.-S. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33015329 · Cited by 785
A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems
Mohseni, S., Zarei, N., & Ragan, E. D. (2021) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3387166 · Cited by 785 · cited 1×
“Some surveys discuss evaluation as part of a broader review of XAI methods [1, 29, 32, 59, 81, 120, 170, 173, 175, 182, 303], or mainly discuss evaluation with user studies [52, 100, 136, 170].”
BackgroundA Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems
Mohseni, S., Zarei, N., & Ragan, E. D. (2018) · arXiv · doi:10.48550/arxiv.1811.11839 · Cited by 783
Interpretation of Neural Networks Is Fragile
Ghorbani, A., Abid, A., & Zou, J. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33013681 · Cited by 765
Serrano, S., & Smith, N. A. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1282 · Cited by 690
Interpretable Decision Sets: A Joint Framework for Description and Prediction
Lakkaraju, H., Bach, S. H., & Leskovec, J. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939874 · Cited by 687 · cited 1×
“Decision Rules Logical rules, including decision sets [139], anchors [213], decision tables [108] and programs [261].”
BackgroundA Survey of Methods for Explaining Black Box Models
Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., & Pedreschi, D. (2019) · ACM Computing Surveys · doi:10.1145/3236009 · Cited by 629 · cited 4×
“[86], presenting four types of problems that an XAI method can solve: (i) Model Explanation – globally explaining model f through an interpretable, predictive model; (ii) Model Inspection – globally explaining some specific property of model f or its prediction; (iii) Outcome Explanation – explaining an outcome/predic…”
BackgroundMethodInfluentialNeural Attentional Rating Regression with Review-level Explanations
Chen, C., Zhang, M., Liu, Y., & Ma, S. (2018) · Proceedings of the 2018 World Wide Web Conference on World Wide Web - WWW '18 · doi:10.1145/3178876.3186070 · Cited by 622
Proceedings of the National Conference on Artificial Intelligence
Jensen, S., Boley, D. L., Gini, M. L., & Schrater, P. (2005) · Innovative Applications of Artificial Intelligence · Cited by 612
Evaluating the quality of machine learning explanations: a survey on methods and metrics
Zhou, J., Gandomi, A. H., Chen, F., & Holzinger, A. (2021) · Electronics · doi:10.3390/electronics10050593 · Cited by 606 · cited 5×
“[314] that presents a concise overview and discussion regarding evaluating XAI without a structured literature review, and the recent work of Vilone and Longo [263] that present a list of concepts related to explainability and discuss evaluation methods in combination with explainable AI methods.”
BackgroundInfluentialdEFEND: Explainable Fake News Detection
Shu, K., Cui, L., Wang, S., Lee, D., & Liu, H. (2019) · Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3292500.3330935 · Cited by 604
Usability evaluation considered harmful (some of the time)
Greenberg, S., & Buxton, B. (2008) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1357054.1357074 · Cited by 598 · cited 1×
“In the context of usability evaluation in the HCI community, Greenberg and Buxton [84] argue that there is a risk of executing user studies in an early design phase, since this can quash creative ideas or promote poor ideas.”
BackgroundNotions of explainability and evaluation approaches for explainable artificial intelligence
Vilone, G., & Longo, L. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.05.009 · Cited by 598 · cited 2×
“Closest related to our work is the survey of Zhou et al. [306] that presents a concise overview and discussion regarding evaluating XAI without a structured literature review, and the work of Vilone and Longo [258] that present a list of concepts related to explainability and discuss evaluation methods in combination…”
BackgroundSummEval: Re-evaluating Summarization Evaluation
Fabbri, A. R., Kryściński, W., McCann, B., Xiong, C., Socher, R., & Radev, D. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00373 · Cited by 587
Principles of Explanatory Debugging to Personalize Interactive Machine Learning
Kulesza, T., Burnett, M., Wong, W.-K., & Stumpf, S. (2015) · Proceedings of the 20th International Conference on Intelligent User Interfaces · doi:10.1145/2678025.2701399 · Cited by 576 · cited 3×
“High completeness is desired [56, 130, 229, 233, 295, 306, 306] to provide enough detail, but it should be balanced with compactness and correctness: “don’t overwhelm” [130].”
BackgroundExplainability Methods for Graph Convolutional Neural Networks
Pope, P. E., Kolouri, S., Rostami, M., Martin, C. E., & Hoffmann, H. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01103 · Cited by 553
Interpretable Machine Learning: Moving from mythos to diagnostics
Chen, V., Li, J., Kim, J. S., Plumb, G., & Talwalkar, A. (2021) · Queue · doi:10.1145/3511299 · Cited by 516
Right for the Right Reasons: Training Differentiable Models by Constraining their Explanations
Ross, A. S., Hughes, M. C., & Doshi-Velez, F. (2017) · Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/371 · Cited by 515
Towards Faithfully Interpretable NLP Systems: How Should We Define and Evaluate Faithfulness?
Jacovi, A., & Goldberg, Y. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.386 · Cited by 503
A Data-Driven Analysis of Workers' Earnings on Amazon Mechanical Turk
Hara, K., Adams, A., Milland, K., Savage, S., Callison-Burch, C., & Bigham, J. P. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174023 · Cited by 497 · cited 1×
“Critics argue that these platforms are largely unregulated and that workers are poorly compensated [91, 224].”
BackgroundAbductive inference: computation, philosophy, technology
(1995) · Choice Reviews Online · doi:10.5860/choice.32-3826 · Cited by 489
Huysmans, J., Dejaeger, K., Mues, C., Vanthienen, J., & Baesens, B. (2011) · Decision Support Systems · doi:10.1016/j.dss.2010.12.003 · Cited by 425 · cited 3×
“[114], “some representation formats are generally considered to be more easily interpretable than others”.”
BackgroundInfluentialMultimodal Explanations: Justifying Decisions and Pointing to the Evidence
Park, D. H., Hendricks, L. A., Akata, Z., Rohrbach, A., Schiele, B., Darrell, T., & Rohrbach, M. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00915 · Cited by 418
Akata, Z., Balliet, D., de Rijke, M., Dignum, F., Dignum, V., Eiben, G., Fokkens, A., Grossi, D., Hindriks, K., Hoos, H., et al. (2020) · Computer · doi:10.1109/mc.2020.2996587 · Cited by 415 · cited 1×
“In addition, a research agenda for Hybrid Intelligence [6] has explicitly formulated a research question asking how the quality and strength of explanations can be evaluated.”
End-to-end Interpretable Neural Motion Planner
Zeng, W., Luo, W., Suo, S., Sadat, A., Yang, B., Casas, S., & Urtasun, R. (2021) · arXiv · doi:10.48550/arxiv.2101.06679 · Cited by 406
End-To-End Interpretable Neural Motion Planner
Zeng, W., Luo, W., Suo, S., Sadat, A., Yang, B., Casas, S., & Urtasun, R. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00886 · Cited by 398
A systematic review and taxonomy of explanations in decision support and recommender systems
Nunes, I., & Jannach, D. (2017) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-017-9195-0 · Cited by 395 · cited 4×
“Nunes and Jannach reported that only 21% of their 190 analyzed studies (1990–2017) that presented an XAI technique or tool contained “any form of evaluation, except from toy examples” [177].”
BackgroundInfluentialToo much, too little, or just right? Ways explanations impact end users' mental models
Kulesza, T., Stumpf, S., Burnett, M., Yang, S., Kwan, I., & Wong, W.-K. (2013) · 2013 IEEE Symposium on Visual Languages and Human Centric Computing · doi:10.1109/vlhcc.2013.6645235 · Cited by 388 · cited 2×
“Ideally, an explanation is “nothing but the truth” [132], and high correctness is desired [9, 17, 26, 111, 132, 133, 214, 237, 246, 280, 289, 302].”
BackgroundExplain Yourself! Leveraging Language Models for Commonsense Reasoning
Rajani, N. F., McCann, B., Xiong, C., & Socher, R. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1487 · Cited by 385
Stepin, I., Alonso, J. M., Catala, A., & Pereira-Farina, M. (2021) · IEEE Access · doi:10.1109/access.2021.3051315 · Cited by 376 · cited 1×
“[239] discuss XAI methods and their evaluation for contrastive and counterfactual explanations, Tintarev and Masthoff [250] and Nunes and Jannach [182] discuss explanations and their evaluation for recommender systems, and Markus et al.”
e-SNLI: Natural language inference with natural language explanations
Camburu, O.-M., Rocktäschel, T., Lukasiewicz, T., & Blunsom, P. (2018) · arXiv · doi:10.48550/arxiv.1812.01193 · Cited by 367
Interpreting CNNs via Decision Trees
Zhang, Q., Yang, Y., Ma, H., & Wu, Y. N. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00642 · Cited by 330
Explaining Recommendations: Design and Evaluation
Tintarev, N., & Masthoff, J. (2015) · Recommender Systems Handbook · doi:10.1007/978-1-4899-7637-6_10 · Cited by 329 · cited 1×
“on the healthcare domain [169], the evaluation of local explanations [90], contrastive and counterfactual explanations [250] or explanations and their evaluation for recommender systems [189, 261].”
BackgroundOpenDialKG: Explainable Conversational Reasoning with Attention-based Walks over Knowledge Graphs
Moon, S., Shah, P., Kumar, A., & Subba, R. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1081 · Cited by 306
Towards Visually Explaining Variational Autoencoders
Liu, W., Li, R., Zheng, M., Karanam, S., Wu, Z., Bhanu, B., Radke, R. J., & Camps, O. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00867 · Cited by 294
Interpretable Neural Predictions with Differentiable Binary Variables
Bastings, J., Aziz, W. F., & Titov, I. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1284 · Cited by 291
Chen, X., Chen, H., Xu, H., Zhang, Y., Cao, Y., Qin, Z., & Zha, H. (2019) · Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3331184.3331254 · Cited by 284
Auditing black-box models for indirect influence
Adler, P., Falk, C., Friedler, S. A., Nix, T., Rybeck, G., Scheidegger, C., Smith, B., & Venkatasubramanian, S. (2018) · Knowledge and Information Systems · doi:10.1007/s10115-017-1116-3 · Cited by 280 · cited 1×
“Examples include Partial Dependence Plot [73], Individual Conditional Expectation plot [82] and Feature Auditing [5].”
MethodInterpretable Predictions of Tree-based Ensembles via Actionable Feature Tweaking
Tolomei, G., Silvestri, F., Haines, A., & Lalmas, M. (2017) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3097983.3098039 · Cited by 276
XGNN: Towards Model-Level Explanations of Graph Neural Networks
Yuan, H., Tang, J., Hu, X., & Ji, S. (2020) · Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3394486.3403085 · Cited by 272
Generating Fact Checking Explanations
Atanasova, P., Simonsen, J. G., Lioma, C., & Augenstein, I. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.656 · Cited by 269
Neural Prototype Trees for Interpretable Fine-grained Image Recognition
Nauta, M., van Bree, R., & Seifert, C. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01469 · Cited by 269 · cited 1×
“Prototypes (Parts of) Representative examples, including concepts [127], influential training instances [90], prototypical parts [36, 179], nearest neighbors and criticisms [125].”
BackgroundSelf-citationExplainability fact sheets: a framework for systematic assessment of explainable approaches
Sokol, K., & Flach, P. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372870 · Cited by 264 · cited 7×
“Continuity also adds to generalizability beyond a particular input [166, 237] or generalizability to new contexts [238].”
BackgroundInfluentialEvaluating Explainable AI: Which Algorithmic Explanations Help Users Predict Model Behavior?
Hase, P., & Bansal, M. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.491 · Cited by 254
Taking a HINT: Leveraging Explanations to Make Vision and Language Models More Grounded
Selvaraju, R. R., Lee, S., Shen, Y., Jin, H., Ghosh, S., Heck, L., Batra, D., & Parikh, D. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00268 · Cited by 251
Explaining Black Box Predictions and Unveiling Data Artifacts through Influence Functions
Han, X., Wallace, B., & Tsvetkov, Y. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.492 · Cited by 241
Perturbed Masking: Parameter-free Probing for Analyzing and Interpreting BERT
Wu, Z., Chen, Y., Kao, B., & Liu, Q. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.383 · Cited by 239
Interpreting CNN Knowledge via an Explanatory Graph
Zhang, Q., Cao, R., Shi, F., Wu, Y. N., & Zhu, S.-C. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11819 · Cited by 214
Beyond Sparsity: Tree Regularization of Deep Models for Interpretability
Wu, M., Hughes, M., Parbhoo, S., Zazzi, M., Roth, V., & Doshi-Velez, F. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11501 · Cited by 213
Net2Vec: Quantifying and Explaining How Concepts are Encoded by Filters in Deep Neural Networks
Fong, R., & Vedaldi, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00910 · Cited by 210
A Review on Explainability in Multimodal Deep Neural Nets
Joshi, G., Walambe, R., & Kotecha, K. (2021) · IEEE Access · doi:10.1109/access.2021.3070212 · Cited by 204 · cited 1×
“Some surveys discuss evaluation as part of a broader review of XAI methods [1, 29, 32, 59, 81, 120, 170, 173, 175, 182, 303], or mainly discuss evaluation with user studies [52, 100, 136, 170].”
BackgroundTransparency by Design: Closing the Gap Between Performance and Interpretability in Visual Reasoning
Mascharka, D., Tran, P., Soklaski, R., & Majumdar, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00519 · Cited by 203
Learning to Deceive with Attention-Based Explanations
Pruthi, D., Gupta, M., Dhingra, B., Neubig, G., & Lipton, Z. C. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.432 · Cited by 203
Fairness-Aware Explainable Recommendation over Knowledge Graphs
Fu, Z., Xian, Y., Gao, R., Zhao, J., Huang, Q., Ge, Y., Xu, S., Geng, S., Shah, C., Zhang, Y., et al. (2020) · Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3397271.3401051 · Cited by 202 · cited 1×
“For example, explainable AI is used to analyze whether model f is fair [74] or biased [162, 231], whether f is right for the right reasons [206], or whether f is overfitting [39].”
MethodTEM: Tree-enhanced Embedding Model for Explainable Recommendation
Wang, X., He, X., Feng, F., Nie, L., & Chua, T.-S. (2018) · Proceedings of the 2018 World Wide Web Conference on World Wide Web - WWW '18 · doi:10.1145/3178876.3186066 · Cited by 201
DACE: Distribution-Aware Counterfactual Explanation by Mixed-Integer Linear Optimization
Kanamori, K., Takagi, T., Kobayashi, K., & Arimura, H. (2020) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/395 · Cited by 201
NILE : Natural Language Inference with Faithful Natural Language Explanations
Kumar, S., & Talukdar, P. P. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.771 · Cited by 198
The Dangers of Post-hoc Interpretability: Unjustified Counterfactual Explanations
Laugel, T., Lesot, M.-J., Marsala, C., Renard, X., & Detyniecki, M. (2019) · Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/388 · Cited by 194
A Diagnostic Study of Explainability Techniques for Text Classification
Atanasova, P., Simonsen, J. G., Lioma, C., & Augenstein, I. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.263 · Cited by 177
Considerations for Evaluation and Generalization in Interpretable Machine Learning
Doshi-Velez, F., & Kim, B. (2018) · The Springer Series on Challenges in Machine Learning · doi:10.1007/978-3-319-98131-4_1 · Cited by 176 · cited 13×
“DoshiVelez and Kim [63] discuss the potential advantages of this evaluation type: besides saving time and costs, it can be particularly appropriate when user studies are unethical or when the method is not yet mature enough for evaluation with users.”
BackgroundMethodInfluentialEvaluating and Aggregating Feature-based Model Explanations
Bhatt, U., Weller, A., & Moura, J. M. F. (2020) · Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/417 · Cited by 176
Operationalizing Human-Centered Perspectives in Explainable AI
Ehsan, U., Wintersberger, P., Liao, Q. V., Mara, M., Streit, M., Wachter, S., Riener, A., & Riedl, M. O. (2021) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3411763.3441342 · Cited by 172 · cited 1×
“This need is not only recognized by the AI community; also the Human Computing Interaction (HCI) community is concerned with developing transferable evaluation methods for XAI [65].”
BackgroundPerturbation-Based Explanations of Prediction Models
Robnik-Šikonja, M., & Bohanec, M. (2018) · Human–Computer Interaction Series · doi:10.1007/978-3-319-90403-0_9 · Cited by 171 · cited 6×
“Ideally, an explanation is “nothing but the truth” [132], and high correctness is desired [9, 17, 26, 111, 132, 133, 214, 237, 246, 280, 289, 302].”
BackgroundMethodInfluentialImproving Interpretability of Deep Neural Networks with Semantic Information
Dong, Y., Su, H., Zhu, J., & Zhang, B. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.110 · Cited by 169
Evaluating everyday explanations
Zemla, J. C., Sloman, S. A., Bechlivanidis, C., & Lagnado, D. A. (2017) · Psychonomic Bulletin & Review · doi:10.3758/s13423-017-1258-z · Cited by 167 · cited 1×
“Note that this property addresses external coherence, and is different from internal coherence to indicate that parts in an explanation fit together [292].”
BackgroundLearning Model-Agnostic Counterfactual Explanations for Tabular Data
Pawelczyk, M., Broelemann, K., & Kasneci, G. (2020) · Proceedings of The Web Conference 2020 · doi:10.1145/3366423.3380087 · Cited by 167
Explaining Knowledge Distillation by Quantifying the Knowledge
Xu, C., Rao, Z., Chen, Y., & Zhang, Q. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01294 · Cited by 166
A Reinforcement Learning Framework for Explainable Recommendation
Wang, X., Chen, Y., Yang, J., Wu, L., Wu, Z., & Xie, X. (2018) · 2018 IEEE International Conference on Data Mining (ICDM) · doi:10.1109/icdm.2018.00074 · Cited by 163
Explanation in Second Generation Expert Systems
Swartout, W. R., & Moore, J. D. (1993) · Second Generation Expert Systems · doi:10.1007/978-3-642-77927-5_24 · Cited by 160 · cited 3×
“Examples include the usage of higher-level information [9], abstractions [63, 246] or suitable terminology [246], and not using explanations that are circular [238].”
BackgroundExplanation Mining: Post Hoc Interpretability of Latent Factor Models for Recommendation Systems
Peake, G. S., & Wang, Y.-X. (2018) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3219819.3220072 · Cited by 160
Towards Explainable NLP: A Generative Explanation Framework for Text Classification
Liu, H., Yin, Q., & Wang, W. Y. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1560 · Cited by 160
Interpretable and Accurate Fine-grained Recognition via Region Grouping
Huang, Z., & Li, Y. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00869 · Cited by 159
Towards Explanation of DNN-based Prediction with Guided Feature Inversion
Du, M., Liu, N., Song, Q., & Hu, X. (2018) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3219819.3220099 · Cited by 147
Unsupervised Discrete Sentence Representation Learning for Interpretable Neural Dialog Generation
Zhao, T., Lee, K., & Eskénazi, M. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-1101 · Cited by 144
Interpretable and Fine-Grained Visual Explanations for Convolutional Neural Networks
Wagner, J., Köhler, J. M., Gindele, T., Hetzel, L., Wiedemer, J. T., & Behnke, S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00931 · Cited by 144
Generating Hierarchical Explanations on Text Classification via Feature Interaction Detection
Chen, H., Zheng, G., & Ji, Y. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.494 · Cited by 143
Try This Instead: Personalized and Interpretable Substitute Recommendation
Chen, T., Yin, H., Ye, G., Huang, Z. H., Wang, Y., & Wang, M. (2020) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3397271.3401042 · Cited by 139
Co-Attentive Multi-Task Learning for Explainable Recommendation
Chen, Z., Wang, X., Xie, X., Wu, T., Bu, G., Wang, Y., & Chen, E. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/296 · Cited by 137
Select, Answer and Explain: Interpretable Multi-Hop Reading Comprehension over Multiple Documents
Tu, M., Huang, K., Wang, G., Huang, J., He, X., & Zhou, B. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6441 · Cited by 136
Learning With Interpretable Structure From Gated RNN
Hou, B., & Zhou, Z. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2967051 · Cited by 131 · cited 1×
“Abstract Policy Graph [252], Knowledge graph [275], Flow graph [218] and Finite State Automata [103].”
BackgroundExplainable Object-Induced Action Decision for Autonomous Vehicles
Xu, Y., Yang, X., Gong, L., Lin, H.-C., Wu, T.-Y., Li, Y., & Vasconcelos, N. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00954 · Cited by 129
Interpret Neural Networks by Identifying Critical Data Routing Paths
Wang, Y., Su, H., Zhang, B., & Hu, X. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00928 · Cited by 128
Evaluating local explanation methods on ground truth
Guidotti, R. (2020) · Artificial Intelligence · doi:10.1016/j.artint.2020.103428 · Cited by 125 · cited 1×
“on the healthcare domain [169], the evaluation of local explanations [90], contrastive and counterfactual explanations [250] or explanations and their evaluation for recommender systems [189, 261].”
BackgroundGenerating Interpretable Poverty Maps using Object Detection in Satellite Images
Ayush, K., Uzkent, B., Burke, M., Lobell, D. B., & Ermon, S. (2020) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/608 · Cited by 119
SCOUT: Self-Aware Discriminant Counterfactual Explanations
Wang, P., & Vasconcelos, N. M. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00900 · Cited by 117
Evaluating neural network explanation methods using hybrid documents and morphosyntactic agreement
Poerner, N., Schütze, H., & Roth, B. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-1032 · Cited by 116 · cited 1×
“"A good explanation method should not reflect what humans attend to, but what task methods attend to" [203].”
BackgroundLearning to Act Properly: Predicting and Explaining Affordances from Images
Chuang, C.-Y., Li, J., Torralba, A. B., & Fidler, S. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00108 · Cited by 110
U-CAM: Visual Explanation Using Uncertainty Based Class Activation Maps
Patro, B. N., Lunayach, M., Patel, S., & Namboodiri, V. P. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00754 · Cited by 106
The FacT: Taming Latent Factor Models for Explainability with Factorization Trees
Tao, Y., Jia, Y., Wang, N., & Wang, H. (2019) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3331184.3331244 · Cited by 105
A study on the Interpretability of Neural Retrieval Models using DeepSHAP
Fernando, Z. T., Singh, J., & Anand, A. (2019) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3331184.3331312 · Cited by 103
Nam, W.-J., Gur, S., Choi, J., Wolf, L., & Lee, S.-W. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i03.5632 · Cited by 98
Towards Transparent and Explainable Attention Models
Mohankumar, A. K., Nema, P., Narasimhan, S., Khapra, M. M., Srinivasan, B. V., & Ravindran, B. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.387 · Cited by 91
Towards Interpretable Face Recognition
Yin, B., Tran, L., Li, H., Shen, X., & Liu, X. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00944 · Cited by 90
Interpretable Transformations with Encoder-Decoder Networks
Worrall, D. E., Garbin, S. J., Turmukhambetov, D., & Brostow, G. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.611 · Cited by 90
Fooling Network Interpretation in Image Classification
Subramanya, A., Pillai, V., & Pirsiavash, H. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00211 · Cited by 89
Debugging Tests for Model Explanations
Adebayo, J., Muelly, M., Liccardi, I., & Kim, B. (2020) · arXiv · doi:10.48550/arxiv.2011.05429 · Cited by 88
Exact and Consistent Interpretation for Piecewise Linear Neural Networks: A Closed Form Solution
Chu, L., Hu, X., Hu, J., Wang, L., & Pei, J. (2018) · Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3219819.3220063 · Cited by 87
Sun, P., Wu, L., Zhang, K., Fu, Y., Hong, R., & Wang, M. (2020) · International World Wide Web Conference (WWW) · doi:10.1145/3366423.3380164 · Cited by 86
Interpretable Semantic Vectors from a Joint Model of Brain- and Text- Based Meaning
Fyshe, A., Talukdar, P. P., Murphy, B., & Mitchell, T. M. (2014) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/v1/p14-1046 · Cited by 85
Interpretable transformations with Encoder-Decoder Networks
Worrall, D., Garbin, S., Turmukhambetov, D., & Brostow, G. (2017) · UCL Discovery (University College London) · Cited by 81
Le, T., Wang, S., & Lee, D. (2020) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3394486.3403066 · Cited by 81
Towards Explainable Conversational Recommendation
Chen, Z., Wang, X., Xie, X., Parsana, M., Soni, A., Ao, X., & Chen, E. (2020) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/414 · Cited by 80
Adversarial Detection with Model Interpretation
Liu, N., Yang, H., & Hu, X. (2018) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3219819.3220027 · Cited by 75
Contextual Outlier Interpretation
Liu, N., Shin, D., & Hu, X. (2018) · Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/341 · Cited by 74
A Disentangling Invertible Interpretation Network for Explaining Latent Representations
Esser, P., Rombach, R., & Ommer, B. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00924 · Cited by 72
Incorporating Interpretability into Latent Factor Models via Fast Influence Analysis
Cheng, W., Shen, Y., Huang, L., & Zhu, Y. (2019) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3292500.3330857 · Cited by 67
Growing Interpretable Part Graphs on ConvNets via Multi-Shot Learning
Zhang, Q., Cao, R., Wu, Y. N., & Zhu, S. C. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10924 · Cited by 66 · cited 1×
“Examples include binary maps with image patches [204], segmentation [104] and bounding boxes [297].”
BackgroundSelf-Supervised Learning of Interpretable Keypoints From Unlabelled Videos
Jakab, T., Gupta, A., Bilen, H., & Vedaldi, A. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00881 · Cited by 66
LIRME: Locally Interpretable Ranking Model Explanation
Verma, M., & Ganguly, D. (2019) · Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3331184.3331377 · Cited by 64
Beyond Polarity: Interpretable Financial Sentiment Analysis with Hierarchical Query-driven Attention
Luo, L., Ao, X., Pan, F., Wang, J., Zhao, T., Yu, N., & He, Q. (2018) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/590 · Cited by 63
On Interpretation of Network Embedding via Taxonomy Induction
Liu, N., Huang, X. X., Li, J., & Hu, X. (2018) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3219819.3220001 · Cited by 63
Towards Global Explanations of Convolutional Neural Networks With Concept Attribution
Wu, W., Su, Y., Chen, X., Zhao, S., King, I., Lyu, M. R.-T., & Tai, Y. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00868 · Cited by 63
Simple, Interpretable and Stable Method for Detecting Words with Usage Change across Corpora
Gonen, H., Jawahar, G., Seddah, D., & Goldberg, Y. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.51 · Cited by 59
Make Up Your Mind! Adversarial Generation of Inconsistent Natural Language Explanations
Camburu, O.-M., Shillingford, B., Minervini, P., Lukasiewicz, T., & Blunsom, P. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.382 · Cited by 59
Explanation Perspectives from the Cognitive Sciences---A Survey
Srinivasan, R., & Chander, A. (2020) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/670 · Cited by 58 · cited 4×
“Continuity also adds to generalizability beyond a particular input [163, 233] or generalizability to new contexts [234].”
BackgroundInfluentialInterpretable Deep Graph Generation with Node-edge Co-disentanglement
Guo, X., Zhao, L., Zhao Qin, A., Wu, L., Shehu, A., & Ye, Y. (2020) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3394486.3403221 · Cited by 58
Generation of Policy-Level Explanations for Reinforcement Learning
Topin, N., & Veloso, M. (2019) · AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) · doi:10.1609/aaai.v33i01.33012514 · Cited by 57
Explaining Neural Networks Semantically and Quantitatively
Chen, R., Chen, H., Huang, G., Ren, J., & Zhang, Q. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00928 · Cited by 56
Obtaining Faithful Interpretations from Compositional Neural Networks
Subramanian, S., Bogin, B., Gupta, N., Wolfson, T., Singh, S., Berant, J., & Gardner, M. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.495 · Cited by 54
Towards Complementary Explanations Using Deep Neural Networks
Silva, W., Fernandes, K., Cardoso, M. J., & Cardoso, J. S. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-02628-8_15 · Cited by 53 · cited 3×
“It can reflect two facets of certainty: i) a confidence measure of the black box prediction [17, 32, 214, 289], or ii) the truthfulness or likelihood of the explanation [17, 166, 233].”
BackgroundExplore, Propose, and Assemble: An Interpretable Model for Multi-Hop Reading Comprehension
Jiang, Y., Joshi, N., Chen, Y.-C., & Bansal, M. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1261 · Cited by 53
Utilizing temporal patterns for estimating uncertainty in interpretable early decision making
Ghalwash, M. F., Radosavljević, V., & Obradović, Z. (2014) · ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2623330.2623694 · Cited by 52
Multimodal Explanations by Predicting Counterfactuality in Videos
Kanehira, A., Takemoto, K., Inayoshi, S., & Harada, T. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00879 · Cited by 51
Learning to Explain With Complemental Examples
Kanehira, A., & Harada, T. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00880 · Cited by 49
Towards a terminology for a fully contextualized XAI
Bellucci, M., Delestre, N., Malandain, N., & Zanni-Merk, C. (2021) · Procedia Computer Science · doi:10.1016/j.procs.2021.08.025 · Cited by 49 · cited 1×
“Some argue that the terms are closely related but distinguish between them [21, 85, 181, 225], although there is no consensus on what the distinction exactly is [25].”
BackgroundGradient-Based Vs. Propagation-Based Explanations: An Axiomatic Comparison
Montavon, G. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-28954-6_13 · Cited by 48 · cited 3×
“For explanation methods that do consider the internals of the black box, Montavon [168] argues that implementation invariance is still a desired property, but should then be evaluated without changing the actual function.”
BackgroundUstun, B., & Rudin, C. D. (2017) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3097983.3098161 · Cited by 47 · cited 1×
“Examples include a scoring sheet [253] and linear regression.”
BackgroundExplainable Recommendation via Interpretable Feature Mapping and Evaluation of Explainability
Pan, D., Li, X., Li, X., & Zhu, D. (2020) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/373 · Cited by 46
Interpreting Deep Models for Text Analysis via Optimization and Regularization Methods
Yuan, H., Chen, Y., Hu, X., & Ji, S. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33015717 · Cited by 41
Abduction-Based Explanations for Machine Learning Models
Ignatiev, A., Narodytska, N., & Marques-Silva, J. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33011511 · Cited by 40
Relation-Based Counterfactual Explanations for Bayesian Network Classifiers
Albini, E., Rago, A., Baroni, P., & Toni, F. (2020) · Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/63 · Cited by 39
Regional Tree Regularization for Interpretability in Deep Neural Networks
Wu, M., Parbhoo, S., Hughes, M., Kindle, R., Celi, L., Zazzi, M., Roth, V., & Doshi-Velez, F. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.6112 · Cited by 38
Improving the Quality of Explanations with Local Embedding Perturbations
Jia, Y., Bailey, J., Ramamohanarao, K., Leckie, C. A., & Houle, M. E. (2019) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3292500.3330930 · Cited by 36
Tseng, A. M., Shrikumar, A., & Kundaje, A. B. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.06.11.147272 · Cited by 36
xGAIL: Explainable Generative Adversarial Imitation Learning for Explainable Human Decision Analysis
Pan, M., Huang, W., Li, Y., Zhou, X. S., & Luo, J. (2020) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3394486.3403186 · Cited by 36
Explaining Multi-Criteria Decision Aiding Models with an Extended Shapley Value
Labreuche, C., & Fossier, S. (2018) · Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/46 · Cited by 35
Towards Better Interpretability in Deep Q-Networks
Annasamy, R. M., & Sycara, K. P. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33014561 · Cited by 33
Embedding deep networks into visual explanations
Qi, Z., Khorram, S., & Li, F. (2020) · Artificial Intelligence · doi:10.1016/j.artint.2020.103435 · Cited by 32
Honegger, M. (2018) · arXiv · doi:10.48550/arxiv.1808.05054 · Cited by 32 · cited 3×
“Consistency checks that identical inputs have identical explanations [14, 101].”
BackgroundExplainable Time Series Tweaking via Irreversible and Reversible Temporal Transformations
Karlsson, I., Rebane, J., Papapetrou, P., & Gionis, A. (2018) · 2018 IEEE International Conference on Data Mining (ICDM) · doi:10.1109/icdm.2018.00036 · Cited by 31
Towards Generating Consumer Labels for Machine Learning Models
Seifert, C., Scherzinger, S., & Wiese, L. (2019) · IEEE International Conference on Cognitive Machine Intelligence (CogMI) · doi:10.1109/cogmi48466.2019.00033 · Cited by 31
Self-citationEvaluating Explanation Methods for Neural Machine Translation
Li, J., Liu, L., Li, H., Li, G., Huang, G., & Shi, S. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.35 · Cited by 31
Kim, E., Gopinath, D., Păsăreanu, C. S., & Seshia, S. A. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01114 · Cited by 30
Interpretable Question Answering on Knowledge Bases and Text
Sydorova, A., Poerner, N., & Roth, B. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1488 · Cited by 29
Explanation vs Attention: A Two-Player Game to Obtain Attention for VQA
Patro, B. N., Anupriy, & Namboodiri, V. P. (2020) · AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) · doi:10.1609/aaai.v34i07.6858 · Cited by 28
Interpretable Word Embeddings for Medical Domain
Jha, K., Wang, Y., Xun, G., & Zhang, A. (2018) · Proceedings · doi:10.1109/icdm.2018.00135 · Cited by 27
Adversarial Infidelity Learning for Model Interpretation
Liang, J., Bai, B., Cao, Y., Bai, K., & Wang, F. (2020) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3394486.3403071 · Cited by 26
Sharing Deep Neural Network Models with Interpretation
Wu, H., Wang, C., Yin, J., Lü, K., & Zhu, L. (2018) · International World Wide Web Conference (WWW) · doi:10.1145/3178876.3185995 · Cited by 23
RecoNet: An Interpretable Neural Architecture for Recommender Systems
Fusco, F., Vlachos, M., Vasileiadis, V., Wardatzky, K., & Schneider, J. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/325 · Cited by 23
Using Small Business Banking Data for Explainable Credit Risk Scoring
Wang, W., Lesner, C., Ran, A., Rukonic, M., Xue, J., & Shiu, E. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i08.7055 · Cited by 23
Towards Interpretation of Recommender Systems with Sorted Explanation Paths
Yang, F., Liu, N., Wang, S., & Hu, X. (2018) · Proceedings · doi:10.1109/icdm.2018.00082 · Cited by 22
Exploring Knowledge Graphs in an Interpretable Composite Approach for Text Entailment
Silva, V. S., Freitas, A., & Handschuh, S. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33017023 · Cited by 21
Sun, Y., Ravi, S. N., & Singh, V. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00504 · Cited by 21
Booth, S., Muise, C. J., & Shah, J. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/804 · Cited by 19 · cited 1×
“[28] evaluating logical sentences of different forms to investigate how to best present propositional theories to humans, and Huysmans et al.”
BackgroundInduction of Non-Monotonic Logic Programs to Explain Boosted Tree Models Using LIME
Shakerin, F., & Gupta, G. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33013052 · Cited by 17
Log2Intent: Towards Interpretable User Modeling via Recurrent Semantics Memory Unit
Tao, Z., Li, S., Wang, Z., Fang, C., Yang, L., Zhao, H., & Fu, Y. (2019) · Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3292500.3330889 · Cited by 16
Relation Extraction with Explanation
Shahbazi, H., Fern, X. Z., Ghaeini, R., & Tadepalli, P. V. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.579 · Cited by 15
FLEX: Faithful Linguistic Explanations for Neural Net Based Model Decisions
Wickramanayake, S., Hsu, W., & Lee, M. L. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33012539 · Cited by 14
LS-Tree: Model Interpretation When the Data Are Linguistic
Chen, J., & Jordan, M. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.5749 · Cited by 14
Synthesizing Aspect-Driven Recommendation Explanations from Reviews
Le, T.-H., & Lauw, H. W. (2020) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/336 · Cited by 14
Accurate and Interpretable Factorization Machines
Lan, L., & Geng, Y. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33014139 · Cited by 13
An Integrative 3C evaluation framework for Explainable Artificial Intelligence.
Cui, X., Lee, J. M., & Hsieh, J. P.-A. (2019) · Journal of the Association for Information Systems · Cited by 10
LP-Explain: Local Pictorial Explanation for Outliers
Liu, H., Ma, F., Wang, Y., He, S., Chen, J., & Gao, J. (2020) · Proceedings · doi:10.1109/icdm50108.2020.00046 · Cited by 10 · cited 1×
“Representation Visualization Charts or plots to visualize representations of the predictive model, including visualizations of dimensionality reduction with scatter plots [259], visual cluster analysis [151] and Principal Component Analysis.”
MethodA Tour of Convolutional Networks Guided by Linear Interpreters
Michelini, P. N., Liu, H., Lu, Y., & Jiang, X. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00485 · Cited by 9
Explainable Data Decompositions
Dalleiger, S., & Vreeken, J. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.5780 · Cited by 8
EDiT: Interpreting Ensemble Models via Compact Soft Decision Trees
Yoo, J., & Sael, L. (2019) · Proceedings · doi:10.1109/icdm.2019.00187 · Cited by 8
Scalable Explanation of Inferences on Large Graphs
Chen, C., Liu, Y., Zhang, X., & Xie, S. (2019) · Proceedings · doi:10.1109/icdm.2019.00111 · Cited by 7
Research works printed but not linked: 102
- Tameem Adel, Zoubin Ghahramani, and Adrian Weller. 2018. Discovering interpretable representations for both deep generative and discriminative models. In Proceedings of the ICML, Vol. 80. PMLR.
- David Alvarez-Melis and Tommi S. Jaakkola. 2018. Towards robust interpretability with self-explaining neural networks. In Proceedings of the NeurIPS. Retrieved from https://proceedings.neurips.cc/paper/2018/hash/3e9f0fc9b2f89e043bc6233994dfcf76-Abstract.html.
- Marco Ancona, Cengiz Öztireli, and Markus H. Gross. 2019. Explaining deep neural networks with a polynomial time algorithm for shapley value approximation. In Proceedings of the ICML, Vol. 97. PMLR.
- Marco Ancona, Cengiz Öztireli, Enea Ceolini, and Markus Gross. 2017. A unified view of gradient-based attribution methods for Deep Neural Networks. In Proceedings of the NIPS Workshop on Interpreting, Explaining and Visualizing Deep Learning.
- Christopher J. Anders, Plamen Pasliev, Ann-Kathrin Dombrowski, Klaus-Robert Müller, and Pan Kessel. 2020. Fairwashing explanations with off-manifold detergent. In Proceedings of the ICML, Vol. 119. PMLR.
- Vijay Arya Rachel K. E. Bellamy Pin-Yu Chen Amit Dhurandhar Michael Hind Samuel C. Hoffman Stephanie Houde Q Vera Liao Ronny Luss Aleksandra Mojsilović et al. 2019. One explanation does not fit all: A toolkit and taxonomy of ai explainability techniques. Retrieved from https://arXiv:1909.03012.
- Cher Bass, Mariana da Silva, Carole H. Sudre, Petru-Daniel Tudosiu, Stephen M. Smith, and Emma C. Robinson. 2020. ICAM: Interpretable classification via disentangled representations and feature attribution mapping. In Proceedings of the NeurIPS. Retrieved from https://proceedings.neurips.cc/paper/2020/hash/56f9f88906aebf4ad985aaec7fa01313-Abstract.html.
- Or Biran and Courtenay Cotton. 2017. Explanation and justification in machine learning: A survey. In Proceedings of the IJCAI Workshop on XAI.
- Akhilan Boopathy, Sijia Liu, Gaoyuan Zhang, Cynthia Liu, Pin-Yu Chen, Shiyu Chang, and Luca Daniel. 2020. Proper network interpretability helps adversarial robustness in classification. In Proceedings of the ICML, Vol. 119. PMLR.
- Prasad Chalasani, Jiefeng Chen, Amrita Roy Chowdhury, Xi Wu, and Somesh Jha. 2020. Concise explanations of neural networks using adversarial training. In Proceedings of the ICML, Vol. 119. PMLR. http://proceedings.mlr.press/v119/chalasani20a.html.
- Chun-Hao Chang, Elliot Creager, Anna Goldenberg, and David Duvenaud. 2019. Explaining image classifiers by counterfactual generation. In Proceedings of the ICLR. OpenReview.net. Retrieved from https://openreview.net/forum?id=B1MXz20cYQ.
- Chaofan Chen, Oscar Li, Daniel Tao, Alina Barnett, Cynthia Rudin, and Jonathan Su. 2019. This looks like that: Deep learning for interpretable image recognition. In Proceedings of the NeurIPS. Retrieved from https://proceedings.neurips.cc/paper/2019/hash/adf7ee2dcf142b0e11888e72b43fcb75-Abstract.html.
- Jianbo Chen, Le Song, Martin J. Wainwright, and Michael I. Jordan. 2018. Learning to explain: An information-theoretic perspective on model interpretation. In Proceedings of the ICML, Vol. 80. PMLR.
- Jianbo Chen, Le Song, Martin J. Wainwright, and Michael I. Jordan. 2019. L-shapley and C-shapley: Efficient model interpretation for structured data. In Proceedings of the ICLR. OpenReview.net. Retrieved from https://openreview.net/forum?id=S1E3Ko09F7.
- Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and Pieter Abbeel. 2016. InfoGAN: Interpretable representation learning by information maximizing generative adversarial nets. In Proceedings of the NeurIPS. Retrieved from https://proceedings.neurips.cc/paper/2016/hash/7c9d0b1f96aebd7b5eca8c3edaa19ebb-Abstract.html.
- Michael Chromik and Martin Schuessler. 2020. A taxonomy for human subject evaluation of black-box explanations in XAI. In Proceedings of the ExSS-ATEC’20.
- Jonathan Crabbé, Yao Zhang, William R. Zame, and Mihaela van der Schaar. 2020. Learning outside the Black-Box: The pursuit of interpretable models. In Proceedings of the NeurIPS. Retrieved from https://proceedings.neurips.cc/paper/2020/hash/ce758408f6ef98d7c7a7b786eca7b3a8-Abstract.html.
- Mark W. Craven and Jude W. Shavlik. 1995. Extracting tree-structured representations of trained networks. In Proceedings of the NIPS. MIT Press. Retrieved from http://papers.nips.cc/paper/1152-extracting-tree-structured-representations-of-trained-networks.
- Arun Das and Paul Rad. 2020. Opportunities and challenges in explainable artificial intelligence (xai): A survey. Retrieved from https://arXiv:2006.11371.
- Amit Dhurandhar, Pin-Yu Chen, Ronny Luss, Chun-Chen Tu, Pai-Shun Ting, Karthikeyan Shanmugam, and Payel Das. 2018. Explanations based on the missing: Towards contrastive explanations with pertinent negatives. In Proceedings of the NeurIPS.
- Ann-Kathrin Dombrowski, Maximilian Alber, Christopher J. Anders, Marcel Ackermann, Klaus-Robert Müller, and Pan Kessel. 2019. Explanations can be manipulated and geometry is to blame. In Proceedings of the NeurIPS.
- Christian Etmann, Sebastian Lunz, Peter Maass, and Carola Schönlieb. 2019. On the connection between adversarial robustness and saliency map interpretability. In Proceedings of the ICML, Vol. 97. PMLR. Retrieved from http://proceedings.mlr.press/v97/etmann19a.html.
- Vincent Fortuin, Matthias Hüser, Francesco Locatello, Heiko Strathmann, and Gunnar Rätsch. 2019. SOM-VAE: Interpretable discrete representation learning on time series. In Proceedings of the ICLR.
- Christopher Frye, Colin Rowat, and Ilya Feige. 2020. Asymmetric shapley values: Incorporating causal knowledge into model-agnostic explainability. In Proceedings of the NeurIPS.
- Amirata Ghorbani, James Wexler, James Y. Zou, and Been Kim. 2019. Towards automatic concept-based explanations. In Proceedings of the NeurIPS. Retrieved from https://proceedings.neurips.cc/paper/2019/hash/77d2afcb31f6493e350fca61764efb9a-Abstract.html.
- Yash Goyal, Ziyan Wu, Jan Ernst, Dhruv Batra, Devi Parikh, and Stefan Lee. 2019. Counterfactual visual explanations. In Proceedings of the ICML, Vol. 97. PMLR. Retrieved from http://proceedings.mlr.press/v97/goyal19a.html.
- Tian Guo, Tao Lin, and Nino Antulov-Fantulin. 2019. Exploring interpretable LSTM neural networks over multi-variable data. In Proceedings of the ICML, Vol. 97. PMLR.
- Wenbo Guo, Sui Huang, Yunzhe Tao, Xinyu Xing, and Lin Lin. 2018. Explaining deep learning models - a bayesian non-parametric approach. In Proceedings of the NeurIPS.
- Peter Hase, Harry Xie, and Mohit Bansal. 2021. The out-of-distribution problem in explainability and search methods for feature importance explanations. In Proceedings of the NeurIPS, Vol. 34. Curran Associates, Inc.
- Juyeon Heo, Sunghwan Joo, and Taesup Moon. 2019. Fooling neural network interpretations via adversarial model manipulation. In Proceedings of the NeurIPS.
- Jay Heo, Haebeom Lee, Saehoon Kim, Juho Lee, Kwang Joon Kim, Eunho Yang, and Sung Ju Hwang. 2018. Uncertainty-aware attention for reliable interpretation and prediction. In Proceedings of the NeurIPS. Retrieved from https://proceedings.neurips.cc/paper/2018/hash/285e19f20beded7d215102b49d5c09a0-Abstract.html.
- Bernease Herman. 2017. The promise and peril of human evaluation for model interpretability. Retrieved from https://arXiv:1711.07414.
- Tom Heskes, Evi Sijben, Ioan Gabriel Bucur, and Tom Claassen. 2020. Causal shapley values: Exploiting causal knowledge to explain individual predictions of complex models. In Proceedings of the NeurIPS.
- Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. 2017. GANs trained by a two time-scale update rule converge to a local nash equilibrium. In Proceedings of the NeurIPS, Vol. 30. Curran Associates, Inc.
- Robert R. Hoffman Shane T. Mueller Gary Klein and Jordan Litman. 2019. Metrics for explainable AI: Challenges and prospects. Retrieved from https://arXiv:1812.04608.
- Sara Hooker, Dumitru Erhan, Pieter-Jan Kindermans, and Been Kim. 2019. A benchmark for interpretability methods in deep neural networks. In Proceedings of the NeurIPS.
- Lukas Hoyer, Mauricio Munoz, Prateek Katiyar, Anna Khoreva, and Volker Fischer. 2019. Grid saliency for context explanations of semantic segmentation. In Proceedings of the NeurIPS.
- Wei-Ning Hsu, Yu Zhang, and James R. Glass. 2017. Unsupervised learning of disentangled and interpretable representations from sequential data. In Proceedings of the NeurIPS.
- Aya Abdelsalam Ismail, Mohamed K. Gunady, Héctor Corrada Bravo, and Soheil Feizi. 2020. Benchmarking deep learning interpretability in time series predictions. In Proceedings of the NeurIPS. Retrieved from https://proceedings.neurips.cc/paper/2020/hash/47a3893cc405396a5c30d91320572d6d-Abstract.html.
- Wengong Jin, Regina Barzilay, and Tommi S. Jaakkola. 2020. Multi-objective molecule generation using interpretable substructures. In Proceedings of the ICML, Vol. 119. PMLR. Retrieved from http://proceedings.mlr.press/v119/jin20b.html.
- Xisen Jin, Zhongyu Wei, Junyi Du, Xiangyang Xue, and Xiang Ren. 2020. Towards hierarchical importance attribution: Explaining compositional semantics for neural sequence models. In Proceedings of the ICLR. OpenReview.net. Retrieved from https://openreview.net/forum?id=BkxRRkSKwr.
- Been Kim, Oluwasanmi Koyejo, and Rajiv Khanna. 2016. Examples are not enough, learn to criticize! Criticism for interpretability. In Proceedings of the NeurIPS.
- Been Kim, Martin Wattenberg, Justin Gilmer, Carrie J. Cai, James Wexler, Fernanda B. Viégas, and Rory Sayres. 2018. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (TCAV). In Proceedings of the ICML, Vol. 80. PMLR.
- Hyunjik Kim and Andriy Mnih. 2018. Disentangling by factorising. In Proceedings of the ICML, Vol. 80. PMLR.
- Wonjae Kim and Yoonho Lee. 2019. Learning dynamics of attention: Human prior for interpretable machine reasoning. In Proceedings of the NeurIPS. Retrieved from https://proceedings.neurips.cc/paper/2019/hash/ae3539867aaeec609a4260c6feb725f4-Abstract.html.
- Pieter-Jan Kindermans, Kristof T. Schütt, Maximilian Alber, Klaus-Robert Müller, Dumitru Erhan, Been Kim, and Sven Dähne. 2018. Learning how to explain neural networks: PatternNet and pattern attribution. In Proceedings of the ICLR.
- Isaac Lage Emily Chen Jeffrey He Menaka Narayanan Been Kim Sam Gershman and Finale Doshi-Velez. 2019. An evaluation of the human-interpretability of explanation. Retrieved from https://arXiv:1902.00006.
- Isaac Lage, Andrew Slavin Ross, Samuel J. Gershman, Been Kim, and Finale Doshi-Velez. 2018. Human-in-the-loop interpretability prior. In Proceedings of the NeurIPS.
- Himabindu Lakkaraju, Nino Arsov, and Osbert Bastani. 2020. Robust and stable black box explanations. In Proceedings of the ICML, Vol. 119. PMLR.
- Himabindu Lakkaraju and Jure Leskovec. 2016. Confusions over time: An interpretable bayesian model to characterize trends in decision making. In Proceedings of the NeurIPS.
- Matthew L. Leavitt and Ari Morcos. 2020. Towards falsifiable interpretability research. Retrieved from https://arXiv:2010.12016.
- Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In Proceedings of the Text Summarization Branches Out. ACL.
- Yang Liu, Sujay Khandagale, Colin White, and Willie Neiswanger. 2021. Synthetic benchmarks for scientific research in explainable machine learning. In Proceedings of the NeurIPS Datasets and Benchmarks Track.
- Scott M. Lundberg and Su-In Lee. 2017. A unified approach to interpreting model predictions. In Proceedings of the NeurIPS.
- Dongsheng Luo, Wei Cheng, Dongkuan Xu, Wenchao Yu, Bo Zong, Haifeng Chen, and Xiang Zhang. 2020. Parameterized explainer for graph neural network. In Proceedings of the NeurIPS. Retrieved from https://proceedings.neurips.cc/paper/2020/hash/e37b08dd3015330dcbb5d6663667b8b8-Abstract.html.
- José Oramas M., Kaili Wang, and Tinne Tuytelaars. 2019. Visual explanation by interpretation: Improving visual feedback capabilities of deep neural networks. In Proceedings of the ICLR. OpenReview.net. Retrieved from https://openreview.net/forum?id=H1ziPjC5Fm.
- João Marques-Silva, Thomas Gerspacher, Martin C. Cooper, Alexey Ignatiev, and Nina Narodytska. 2020. Explaining naive bayes and other linear classifiers with polynomial time and delay. In Proceedings of the NeurIPS. Retrieved from https://proceedings.neurips.cc/paper/2020/hash/eccd2a86bae4728b38627162ba297828-Abstract.html.
- Tim Miller, Piers Howe, and Liz Sonenberg. 2017. Explainable AI: Beware of inmates running the asylum or: How I learnt to stop worrying and love the social and behavioural sciences. In Proceedings of the IJCAI Workshop on Explainable Artificial Intelligence.
- Jesse Mu and Jacob Andreas. 2020. Compositional explanations of neurons. In Proceedings of the NeurIPS. Retrieved from https://proceedings.neurips.cc/paper/2020/hash/c74956ffb38ba48ed6ce977af6727275-Abstract.html.
- Anh Nguyen, Alexey Dosovitskiy, Jason Yosinski, Thomas Brox, and Jeff Clune. 2016. Synthesizing the preferred inputs for neurons in neural networks via deep generator networks. In Proceedings of the NeurIPS, Vol. 29. Curran Associates, Inc. Retrieved from https://proceedings.neurips.cc/paper/2016/file/5d79099fcdf499f12b79770834c0164a-Paper.pdf.
- Weili Nie, Yang Zhang, and Ankit Patel. 2018. A theoretical explanation for perplexing behaviors of backpropagation-based visualizations. In Proceedings of the ICML, Vol. 80. PMLR. Retrieved from http://proceedings.mlr.press/v80/nie18a.html.
- Matthew R. O’Shaughnessy, Gregory Canal, Marissa Connor, Christopher Rozell, and Mark A. Davenport. 2020. Generative causal explanations of black-box classifiers. In Proceedings of the NeurIPS. Retrieved from https://proceedings.neurips.cc/paper/2020/hash/3a93a609b97ec0ab0ff5539eb79ef33a-Abstract.html.
- Jung Yeon Park, Kenneth Theo Carr, Stephan Zheng, Yisong Yue, and Rose Yu. 2020. Multiresolution tensor learning for efficient and interpretable spatial analysis. In Proceedings of the ICML, Vol. 119. PMLR.
- Tejaswini Pedapati, Avinash Balakrishnan, Karthikeyan Shanmugam, and Amit Dhurandhar. 2020. Learning global transparent models consistent with local contrastive explanations. In Proceedings of the NeurIPS. Retrieved from https://proceedings.neurips.cc/paper/2020/hash/24aef8cb3281a2422a59b51659f1ad2e-Abstract.html.
- Vitali Petsiuk, Abir Das, and Kate Saenko. 2018. RISE: Randomized input sampling for explanation of black-box models. In Proceedings of the BMVC.
- Gregory Plumb, Maruan Al-Shedivat, Ángel Alexander Cabrera, Adam Perer, Eric P. Xing, and Ameet Talwalkar. 2020. Regularizing black-box models for improved interpretability. In Proceedings of the NeurIPS. Retrieved from https://proceedings.neurips.cc/paper/2020/hash/770f8e448d07586afbf77bb59f698587-Abstract.html.
- Gregory Plumb, Denali Molitor, and Ameet S. Talwalkar. 2018. Model agnostic supervised local explanations. In Proceedings of the NeurIPS.
- Gregory Plumb, Jonathan Terhorst, Sriram Sankararaman, and Ameet Talwalkar. 2020. Explaining groups of points in low-dimensional representations. In Proceedings of the ICML, Vol. 119. PMLR.
- Nikaash Puri, Sukriti Verma, Piyush Gupta, Dhruv Kayastha, Shripad Deshmukh, Balaji Krishnamurthy, and Sameer Singh. 2020. Explain your move: Understanding agent actions using specific and relevant feature attribution. In Proceedings of the ICLR.
- Maithra Raghu, Justin Gilmer, Jason Yosinski, and Jascha Sohl-Dickstein. 2017. SVCCA: Singular vector canonical correlation analysis for deep learning dynamics and interpretability. In Proceedings of the NeurIPS. Retrieved from https://proceedings.neurips.cc/paper/2017/hash/dc6a7e655d7e5840e66733e9ee67cc69-Abstract.html.
- Karthikeyan Natesan Ramamurthy, Bhanukiran Vinzamuri, Yunfeng Zhang, and Amit Dhurandhar. 2020. Model agnostic multilevel explanations. In Proceedings of the NeurIPS. Retrieved from https://proceedings.neurips.cc/paper/2020/hash/426f990b332ef8193a61cc90516c1245-Abstract.html.
- Kaivalya Rawal and Himabindu Lakkaraju. 2020. Beyond individualized recourse: Interpretable and interactive summaries of actionable recourses. In Proceedings of the NeurIPS. Retrieved from https://proceedings.neurips.cc/paper/2020/hash/8ee7730e97c67473a424ccfeff49ab20-Abstract.html.
- Laura Rieger, Chandan Singh, W. James Murdoch, and Bin Yu. 2020. Interpretations are useful: Penalizing explanations to align neural networks with prior knowledge. In Proceedings of the ICML, Vol. 119. PMLR.
- Yao Rong, Tobias Leemann, Vadim Borisov, Gjergji Kasneci, and Enkelejda Kasneci. 2022. A consistent and efficient evaluation strategy for attribution methods. In Proceedings of the ICML, Vol. 162. PMLR.
- Benjamin Sanchez-Lengeling, Jennifer Wei, Brian Lee, Emily Reif, Peter Wang, Wesley Qian, Kevin McCloskey, Lucy Colwell, and Alexander Wiltschko. 2020. Evaluating attribution for graph neural networks. In Proceedings of the NeurIPS, Vol. 33. Curran Associates.
- Patrick Schwab and Walter Karlen. 2019. CXPlain: Causal explanations for model interpretation under uncertainty. In Proceedings of the NeurIPS.
- Wenxian Shi, Hao Zhou, Ning Miao, and Lei Li. 2020. Dispersed exponential family mixture VAEs for interpretable text generation. In Proceedings of the ICML, Vol. 119. PMLR.
- Chandan Singh, W. James Murdoch, and Bin Yu. 2019. Hierarchical interpretations for neural network predictions. In Proceedings of the ICLR.
- Sumedha Singla, Brian Pollack, Junxiang Chen, and Kayhan Batmanghelich. 2020. Explanation by progressive exaggeration. In Proceedings of the ICLR. OpenReview.net. Retrieved from https://openreview.net/forum?id=H1xFWgrFPS.
- Leon Sixt, Maximilian Granz, and Tim Landgraf. 2020. When explanations lie: Why many modified BP attributions fail. In Proceedings of the ICML, Vol. 119. PMLR. http://proceedings.mlr.press/v119/sixt20a.html.
- Mukund Sundararajan and Amir Najmi. 2020. The many shapley values for model explanation. In Proceedings of the ICML, Vol. 119. PMLR. Retrieved from http://proceedings.mlr.press/v119/sundararajan20b.html.
- Michael Tsang, Dehua Cheng, Hanpeng Liu, Xue Feng, Eric Zhou, and Yan Liu. 2020. Feature interaction interpretability: A case for explaining ad-recommendation systems via neural interaction detection. In Proceedings of the ICLR. OpenReview.net. Retrieved from https://openreview.net/forum?id=BkgnhTEtDS.
- Michael Tsang, Hanpeng Liu, Sanjay Purushotham, Pavankumar Murali, and Yan Liu. 2018. Neural interaction transparency (NIT): Disentangling learned interactions for improved interpretability. In Proceedings of the NeurIPS.
- Michael Tsang, Sirisha Rambhatla, and Yan Liu. 2020. How does this interaction affect me? Interpretable attribution for feature interactions. In Proceedings of the NeurIPS. Retrieved from https://proceedings.neurips.cc/paper/2020/hash/443dec3062d0286986e21dc0631734c9-Abstract.html.
- Ramakrishna Vedantam, Karan Desai, Stefan Lee, Marcus Rohrbach, Dhruv Batra, and Devi Parikh. 2019. Probabilistic neural symbolic models for interpretable visual question answering. In Proceedings of the ICML, Vol. 97. PMLR.
- Andrey Voynov and Artem Babenko. 2020. Unsupervised discovery of interpretable directions in the GAN latent space. In Proceedings of the ICML, Vol. 119. PMLR. Retrieved from http://proceedings.mlr.press/v119/voynov20a.html.
- Minh N. Vu and My T. Thai. 2020. PGM-explainer: Probabilistic graphical model explanations for graph neural networks. In Proceedings of the NeurIPS.
- Pei Wang and Nuno Vasconcelos. 2019. Deliberative explanations: Visualizing network insecurities. In Proceedings of the NeurIPS. Retrieved from https://proceedings.neurips.cc/paper/2019/hash/68053af2923e00204c3ca7c6a3150cf7-Abstract.html.
- Shengjie Wang, Tianyi Zhou, and Jeff A. Bilmes. 2019. Bias also matters: Bias attribution for deep neural network explanation. In Proceedings of the ICML, Vol. 97. PMLR.
- Tong Wang. 2018. Multi-value rule sets for interpretable classification with feature-efficient representations. In Proceedings of the NeurIPS.
- Tong Wang. 2019. Gaining free or low-cost interpretability with interpretable partial substitute. In Proceedings of the ICML, Vol. 97. PMLR. Retrieved from http://proceedings.mlr.press/v97/wang19a.html.
- Andrew G. Wilson, Christoph Dann, Chris Lucas, and Eric P. Xing. 2015. The human kernel. In Proceedings of the NeurIPS, Vol. 28. Curran Associates, Inc. Retrieved from https://proceedings.neurips.cc/paper/2015/file/4462bf0ddbe0d0da40e1e828ebebeb11-Paper.pdf.
- Ning Xie Gabrielle Ras Marcel van Gerven and Derek Doran. 2020. Explainable deep learning: A field guide for the uninitiated. Retrieved from https://arXiv:2004.14545.
- Fan Yang Mengnan Du and Xia Hu. 2019. Evaluating explanation without ground truth in interpretable machine learning. Retrieved from https://arXiv:1907.06831.
- Chih-Kuan Yeh, Cheng-Yu Hsieh, Arun Sai Suggala, David I. Inouye, and Pradeep Ravikumar. 2019. On the (in)fidelity and sensitivity of explanations. In Proceedings of the NeurIPS. Retrieved from https://proceedings.neurips.cc/paper/2019/hash/a7471fdc77b3435276507cc8f2dc2569-Abstract.html.
- Chih-Kuan Yeh, Been Kim, Sercan Ömer Arik, Chun-Liang Li, Tomas Pfister, and Pradeep Ravikumar. 2020. On completeness-aware concept-based explanations in deep neural networks. In Proceedings of the NeurIPS. Retrieved from https://proceedings.neurips.cc/paper/2020/hash/ecb287ff763c169694f682af52c1f309-Abstract.html.
- Zhitao Ying, Dylan Bourgeois, Jiaxuan You, Marinka Zitnik, and Jure Leskovec. 2019. GNNExplainer: Generating explanations for graph neural networks. In Proceedings of the NeurIPS.
- Haizi Yu and Lav R. Varshney. 2017. Towards deep interpretability (MUS-ROVER II): Learning hierarchical representations of tonal music. In Proceedings of the ICLR. OpenReview.net. Retrieved from https://openreview.net/forum?id=ryhqQFKgl.
- Hao Zhang Jiayi Chen Haotian Xue and Quanshi Zhang. 2019. Towards a unified evaluation of explanation methods without ground truth. Retrieved from https://arXiv:1911.09017.
- Xin Zhang, Armando Solar-Lezama, and Rishabh Singh. 2018. Interpreting neural network judgments via minimal, stable, and symbolic corrections. In Proceedings of the NeurIPS.
- Yuan Zhao and Il Memming Park. 2016. Interpretable nonlinear dynamic modeling of neural trajectories. In Proceedings of the NeurIPS.
- Charles Y. Zheng, Francisco Pereira, Chris I. Baker, and Martin N. Hebart. 2019. Revealing interpretable object representations from human behavior. In Proceedings of the ICLR. OpenReview.net. Retrieved from https://openreview.net/forum?id=ryxSrhC9KX.
Cited by
Ali, S., Abuhmed, T., El-Sappagh, S., Muhammad, K., Alonso-Moral, J. M., Confalonieri, R., Guidotti, R., Del Ser, J., Díaz-Rodríguez, N., & Herrera, F. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101805 · Cited by 1,762
Lekadir, K., Frangi, A. F., Porras, A. R., Glocker, B., Cintas, C., Langlotz, C. P., Weicken, E., Asselbergs, F. W., Prior, F., Collins, G. S., et al. (2025) · BMJ · doi:10.1136/bmj-2024-081554 · Cited by 957
Survey of Explainable AI Techniques in Healthcare
Chaddad, A., Peng, J., Xu, J., & Bouridane, A. (2023) · Sensors · doi:10.3390/s23020634 · Cited by 591
Longo, L., Brcic, M., Cabitza, F., Choi, J., Confalonieri, R., Ser, J. D., Guidotti, R., Hayashi, Y., Herrera, F., Holzinger, A., et al. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102301 · Cited by 626
“In a recent survey on the evaluation of XAI, for instance, the authors identify several conceptual properties that should be considered to assess the quality of an explanation, and they propose quantitative evaluation methods to evaluate an explanation [212].”
BackgroundTowards Human-Centered Explainable AI: A Survey of User Studies for Model Explanations
Rong, Y., Leemann, T., Nguyen, T.-T., Fiedler, L., Qian, P., Unhelkar, V., Seidel, T., Kasneci, G., & Kasneci, E. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3331846 · Cited by 278
“%) in [71]; Lower detection rate in [53] Biases in features used by models [57, 71] Moderate detection rate (50”
BackgroundMethodInfluential
Metadata
- Authors
- Nauta, M., Trienes, J., Pathak, S., Nguyen, E., Peters, M., Schmitt, Y., Schlötterer, J., van Keulen, M., & Seifert, C.
- Affiliations
- University of Duisburg-Essen (DE)
- University of Twente (NL)
- University of Twente, the Netherlands
- ORCID
- Venue
- ACM Computing Surveys
- Published
- 2023-07-13
- Volume
- 55
- Issue
- 13s
- Pages
- 1-42
- ISSN
- 0360-0300 (ISSN-L)
- 1557-7341
- OpenAlex
- W4321786089
- Cited by
- 590
- References
- 202
- Citations by year
- 2026: 184 · 2025: 195 · 2024: 157 · 2023: 45
- FWCI
- 12.16
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM Computing Surveys (publishedVersion, cc-by)
- arXiv (Cornell University) (publishedVersion)
- University of Twente Research Information (publishedVersion, other-oa)
- University of Twente Research Information (submittedVersion, other-oa)
- arXiv (Cornell University) (cc-by)
- Funders
- Ziekenhuis Groep Twente
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Keywords
- explainable AI, XAI evaluation, explanation quality metrics, interpretability, quantitative evaluation methods, anecdotal evidence, user evaluation, benchmarking, black-box models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.