The DOI matches its DataCite record; 44 of the 46 references it cites were found in a registry or an index; none of the 35 sources its claims cite is checked for support yet.
One Explanation Does Not Fit All: A Toolkit and Taxonomy of AI Explainability Techniques
Open accessAbstract
As artificial intelligence and machine learning algorithms make further inroads into society, calls are increasing from multiple stakeholders for these algorithms to explain their outputs. At the same time, these stakeholders, whether they be affected citizens, government regulators, domain experts, or system developers, present different requirements for explanations. Toward addressing these needs, we introduce AI Explainability 360 (http://aix360.mybluemix.net/), an open-source software toolkit featuring eight diverse and state-of-the-art explainability methods and two evaluation metrics. Equally important, we provide a taxonomy to help entities requiring explanations to navigate the space of explanation methods, not only those in the toolkit but also in the broader literature on explainability. For data scientists and other users of the toolkit, we have implemented an extensible software architecture that organizes methods according to their place in the AI modeling pipeline. We also discuss enhancements to bring research innovations closer to consumers of explanations, ranging from simplified, more accessible versions of algorithms, to tutorials and an interactive web demo to introduce AI explainability to different audiences and application domains. Together, our toolkit and taxonomy can help identify gaps where more explainability methods are needed and provide a platform to incorporate them as they are developed.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
The paper introduces AI Explainability 360 (AIX360), an open-source toolkit with eight explainability algorithms and two evaluation metrics, plus a taxonomy to help different explanation consumers choose methods. It describes algorithmic enhancements, educational materials, and comparisons showing BRCG-light trades accuracy for reduced computation.
Key findings
- AIX360 contains eight explainability algorithms—BRCG, GLRM, ProtoDash, ProfWeight, TED, CEM, CEM-MAF, and DIP-VAE—and two metrics, Faithfulness and Monotonicity.
- The proposed taxonomy organizes explanations by static/interactive, data/model, local/global, direct/post-hoc, and other distinctions; Table 1 indicates AIX360 uniquely covers data explanations, directly interpretable methods, local post-hoc, global post-hoc, persona-specific explanations, and metrics among the compared toolkits.
- BRCG-light replaces integer programming with beam search; on 15 datasets, it is worse on average than BRCG-IP but better than BRS, AM, and BCD, and beam width B=5 gives better results than smaller widths on most datasets, particularly larger ones.
- The TED employee-retention synthetic dataset has 10,000 entries, 8 features, 25 rules, and a feature space of over 80 million possibilities; the toolkit also includes a FICO-based web demo, five tutorials, and 13 Jupyter notebooks.
Methods
- Design: proposes a taxonomy decision tree and implements an extensible Python class hierarchy organized by the AI modeling pipeline, including data explainers, directly interpretable explainers, local post-hoc explainers, and global post-hoc explainers.
- Data or sample: BRCG-light is evaluated on the same 15 datasets as [17]; the TED use case uses a synthetic employee-retention dataset with 10,000 entries; the web demo uses the FICO Explainable Machine Learning Challenge dataset, and tutorials cover credit approval, healthcare cost prediction, dermoscopic images, NHANES data, and employee retention.
- Main analysis: descriptive toolkit comparison in Table 1 and accuracy-complexity trade-off curves for BRCG-light versus BRCG-IP, BRS, AM, and BCD; no formal user study is reported in the text.
Limitations
- The paper states the taxonomy is “by no means” claimed to be perfect or complete, and the toolkit lacks methods in three taxonomy categories marked with “?”: interactive, static → data → distributions, and static → model → global → post-hoc → visualize.
- The paper states BRCG-light is inferior to BRCG-IP on average, and that if beam search fails to find a negative-objective solution, it cannot certify that none exists, unlike integer programming run to completion.
- The paper states explanations are difficult to quantitatively evaluate, human evaluation is the gold standard, and the two included metrics are proxies and not novel; evident from the text, the toolkit and taxonomy are evaluated mainly through descriptive comparison and trade-off curves rather than user studies.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
34 Own interpretation · 22 No basis given · 20 Not checked yet · 13 Own result
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“we introduce AI Explainability 360 1 , an open-source software toolkit featuring eight diverse and state-of-the-art explainability methods and two evaluation metrics.”
- Also stated in: § 1 INTRODUCTION · p. 2 · plain
“The current version of the toolkit contains eight explainability algorithms:”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 1 INTRODUCTION · p. 2 · plain
None of the sources cited for this claim have been checked yet.
In the paper
“two evaluation metrics”
- Also stated in: § 1 INTRODUCTION · p. 2 · plain
“The toolkit also includes two metrics from the explainability literature: Faithfulness [12] and Monotonicity [41].”
What the sources say
- Not checked yetcites [12] [12] D. Alvarez Melis and T. Jaakkola. Towards robust interpretability with selfexplaining neural networks. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems 31 , pages 7775-7784. 2018. -
- Not checked yetcites [41] [41] R. Luss, P.-Y. Chen, A. Dhurandhar, P. Sattigeri, K. Shanmugam, and C.-C. Tu. Generating contrastive explanations with monotonic attribute functions. 2019. -
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 1 INTRODUCTION · p. 2 · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“It is the most comprehensive explainability toolkit across different ways of explaining (see Table 1 and Section 5).”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“we have implemented an extensible software architecture that organizes methods according to their place in the AI modeling pipeline.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“we introduce AI Explainability 360 1 , an open-source software toolkit featuring eight diverse and state-of-the-art explainability methods and two evaluation metrics.”
- Also stated in: § One Explanation Does Not Fit All: A Toolkit and Taxonomy of AI Explainability Techniques · p. 1 · plain
“We have released the toolkit into the open source community under the name AI Explainability 360 (AIX360).”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § One Explanation Does Not Fit All: A Toolkit and Taxonomy of AI Explainability Techniques · p. 1 · plain
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“resulting in a feature space of over 80 million possibilities.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In particular, a beam search width of 5 (the default in AIX360) gives better results than smaller widths on most datasets.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In the AIX360 toolkit we add another layer of interpretability by also discerning features that are important in more easily interpreting the output.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“the AIX360 toolkit provides two quantitative metrics that serve as proxies of how 'good' a particular feature-based local explanation is likely to be.”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In AIX360, we have made the above suggestion a reality by equipping GLRM models with the ability to visualize the GAM part of the model.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Probably not a substantial claim. 54
- It may state two separate claims as one. 20
- The quote only partly states this claim. 2
In the paper
“Again, we do not claim that this is a complete listing, although most of the methods reviewed by Guidotti et al. [25] are included.”
What the sources say
- Not checked yetcites [25] [25] R. Guidotti, A. Monreale, S. Ruggieri, F. Turini, F. Giannotti, and D. Pedreschi. A survey of methods for explaining black box models. ACM computing surveys (CSUR) , 51(5):93, 2018. -
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“These educational materials include a general guidance for the key concepts of explainability, a web demo that illustrates the usage of different explainability methods, and multiple tutorials.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The decision tree leaves indicate the methods currently available through the toolkit along with the modalities they can work with.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“AIX360 focuses on static explanations, like much of the literature to date.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The web demo was created based on the FICO Explainable Machine Learning Challenge dataset [22], a real-world scenario where a machine learning system is used to support decisions on loan applications by predicting the repayment risk of the applicants.”
What the sources say
- Not checked yetcites [22] [22] FICO. FICO Explainable Machine Learning Challenge. https://community.fico. com/s/explainable-machine-learning-challenge, 2018. Last accessed 2019-08. -
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These comparisons show that BRCG-light occupies an intermediate position, inferior to BRCG-IP as may be expected, but superior to the earlier methods.”
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“AIX360 contains such a dataset for employee retention with 10,000 entries, created by an available python script that can be modified by changing the distribution of features, changing the rules, or the size of the dataset.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“AIX360 was developed with the goal of providing accessible resources on explainability to non-technical stakeholders.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We created 25 rules, motivated by common retention scenarios, such as not getting a promotion in a while, not being paid competitively, receiving a disappointing evaluation, being a new employee in organizations with inherently high attrition, not having a salary that is consistent with positive evaluations, mid-career crisis, etc.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The quote only partly states this claim.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The leaves in the decision tree are labeled with algorithms in the AIX360 toolkit or a question mark ('?') if the category is not represented in the current toolkit.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In AIX360, we have replaced the integer programming components with a heuristic beam search algorithm and have modified the problem formulation accordingly.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In the current implementation of this use case, we further vary the application of these rules depending on various positions and organizations.”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The modules implementing GLRM follow [59] plus the enhancement of Section 4.3 below while the module implementing BRCG-light is novel.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In addition to the above base classes, AIX360 includes dataset classes to facilitate loading and processing of commonly used datasets so that users can easily experiment with the implemented algorithms.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“We created 25 rules, motivated by common retention scenarios, such as not getting a promotion in a while, not being paid competitively, receiving a disappointing evaluation, being a new employee in organizations with inherently high attrition, not having a salary that is consistent with positive evaluations, mid-career crisis, etc.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The proposed taxonomy based on questions about what is explained (e.g., data or model), how it is explained (e.g., direct/post-hoc, static/interactive) and at what level (i.e. local/global).”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These are indicated in Figure 1 by '?' in three cases: interactive, static → data → distributions, and static → model → global → post-hoc → visualize.”
plainn = 3- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“We have also provided a simple yet comprehensive taxonomy that structures the explainability space and serves multiple purposes, including guiding practitioners, revealing algorithmic gaps to researchers, and informing design choices related to explainability software (e.g., class hierarchy) for data scientists and developers.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The AIX360 toolkit currently includes five tutorials in the form of Jupyter notebooks that show data scientists and other developers how to use different explanation methods across several application domains.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Preliminary work in this direction was initiated in AIX360 by defining a class for Keras-based classifiers that is utilized by CEMExplainer and CEMMAFImageExplainer.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Some of the algorithms were enhanced from their published versions for AIX360 to make them more usable and their outputs easier to consume.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The toolkit also contains two explainability metrics, a credit approval demonstration, five elaborate tutorials covering different real-world use cases, and 13 Jupyter notebooks, making it accessible to practitioners and non-experts.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Overall, the comparisons show that BRCG-light provides trade-offs that are worse on average than BRCG-IP, as may be expected given the dramatic reduction in computation, but better than the other methods.”
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“AIX360 also targets industry-specific users with the tutorials and web demo in Section 4.6.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The AIX360 toolkit solicits contributions related to explainable AI to enable it to grow.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Since there is no single approach to explainable AI that always works best, we require organizing principles for the space of possibilities and tools that bridge the gap from research to practice.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Equally important, we provide a taxonomy to help entities requiring explanations to navigate the space of explanation methods, not only those in the toolkit but also in the broader literature on explainability.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“different people in different settings may require different kinds of explanations.”
- Also stated in: § ABSTRACT · p. 1 · plain
“these stakeholders, whether they be affected citizens, government regulators, domain experts, or system developers, present different requirements for explanations.”
plainconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
- Also stated in: § ABSTRACT · p. 1 · plain
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We show that the taxonomy in Figure 1 allows different consumers to navigate the space of explanations and find the right algorithm or class of algorithms appropriate for their task.”
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“Knowledge distillation-type approaches [13, 14, 31], which learn a simpler model based on a complex model's predictions, would be considered as global interpretations that are learned using a post-hoc surrogate model.”
What the sources say
- Not checked yetcites [13] [13] L. J. Ba and R. Caurana. Do deep nets really need to be deep? CoRR , abs/1312.6184, 2013. -
- Not checked yetcites [14] [14] C. Buciluˇ a, R. Caruana, and A. Niculescu-Mizil. Model compression. In Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , 2006. -
- Not checked yetcites [31] [31] G. Hinton, O. Vinyals, and J. Dean. Distilling the knowledge in a neural network. In https://arxiv.org/abs/1503.02531 , 2015. -
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The benefit of having a GAM is that it can be visualized as a series of plots of the functions f j , thus summarizing the effects of all first-degree rules and linear terms.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“GLRM models can therefore be interpreted like linear models and their constituent rules are also readily understood individually.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“A PP informs us of what minimal values for different features not exceeding those in x would be sufficient to obtain the same prediction y .”
What the sources say
- Not checked yetcites [18] [18] A. Dhurandhar, P.-Y. Chen, R. Luss, C.-C. Tu, P. Ting, K. Shanmugam, and P. Das. Explanations based on the missing: Towards contrastive explanations with pertinent negatives. In Advances in Neural Information Processing Systems , 2018. -
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This taxonomy is actionable in that it aids users in choosing an approach for a given application and may also reveal gaps in available explainability techniques.”
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“If the features are (independent) positively correlated with the decision, then the class probabilities should monotonically increase.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“Work in the natural language processing and computer vision domains that generates rationales/explanations derived from input text [11, 28, 39] would be considered as local self explanations.”
What the sources say
- Not checked yetcites [11] [11] Y. Ainur, Y. Choi, and C. Cardie. Automatically generating annotator rationales to improve sentiment classification. In Proceedings of the ACL 2010 Conference Short Papers , 2010. -
- Not checked yetcites [28] [28] L. A. Hendricks, Z. Akata, M. Rohrbach, J. Donahue, B. Schiele, and T. Darrell. Generating visual explanations. In European Conference on Computer Vision , 2016. -
- Not checked yetcites [39] [39] T. Lei, R. Barzilay, and T. Jaakkola. Rationalizing neural predictions. arXiv preprint arXiv:1606.04155 , 2016. -
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Together, our toolkit and taxonomy can help identify gaps where more explainability methods are needed and provide a platform to incorporate them as they are developed.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds. It may state two separate claims as one.
In the paper
“Nevertheless, the discussion in [59, Sec. 6] as well as our experience in putting together the AIX360 web demo and tutorials (Section 4.6) raise the question of the degree to which a GLRM model with (say) tens of rules is interpretable. While interpretation is still feasible, it may require a non-trivial investment of effort.”
What the sources say
- Not checked yetcites [59] [59] D. Wei, S. Dash, T. Gao, and O. Günlük. Generalized linear rule models. In Proceedings of the 36th International Conference on Machine Learning (ICML) , 2019. -
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Methods that propose certain restrictions on the neural network architecture [12, 62] to make it interpretable, yet maintain richness of the hypothesis space to model complicated decision boundaries would fall under the global directly interpretable category.”
What the sources say
- Not checked yetcites [12] [12] D. Alvarez Melis and T. Jaakkola. Towards robust interpretability with selfexplaining neural networks. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems 31 , pages 7775-7784. 2018. -
- Not checked yetcites [62] [62] Q. Zhang, Y. N. Wu, and S.-C. Zhu. Interpretable convolutional neural networks. In Proceedings of Computer Vision and Pattern Recognition , 2018.
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The second metric is Monotonicity [41], which simply tries to measure if adding more positive evidence increases the probability of classification in the specified class.”
What the sources say
- Not checked yetcites [41] [41] R. Luss, P.-Y. Chen, A. Dhurandhar, P. Sattigeri, K. Shanmugam, and C.-C. Tu. Generating contrastive explanations with monotonic attribute functions. 2019. -
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The intuition is that removing more important features should reduce the confidence of the classifier more than removing less important ones.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The first metric we implement is a version of the Faithfulness metric [12], which tries to evaluate if the feature importances returned by an explainability method are the correct ones.”
What the sources say
- Not checked yetcites [12] [12] D. Alvarez Melis and T. Jaakkola. Towards robust interpretability with selfexplaining neural networks. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems 31 , pages 7775-7784. 2018. -
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Higher-degree rules, which do involve feature interactions, can then be studied separately.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The quote only partly states this claim.
None of the sources cited for this claim have been checked yet.
In the paper
“Saliency based methods [51, 52], which highlight different portions in an image whose classification we want to understand, can be categorized as local explanation methods that are static and provide feature-based explanations in terms of highlighting pixels/superpixels.”
What the sources say
- Not checked yetcites [51] [51] R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra. Gradcam: Visual explanations from deep networks via gradient-based localization. See https://arxiv. org/abs/1610.02391 v3 , 2016. -
- Not checked yetcites [52] [52] K. Simonyan, A. Vedaldi, and A. Zisserman. Deep inside convolutional networks: Visualising image classification models and saliency maps. CoRR , abs/1312.6034, 2013. -
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The above equation creates a feature importance value in [ 0 , 1 ] . The higher the value more similar the feature and hence higher its importance.”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“Methods such as partial dependence plots (PDP) and sensitivity analysis [43] are used to study the (global) effects of different input features on the output values and are typically consumed through appropriate visualizations (e.g., PDP plots, scatter plots, control charts).”
What the sources say
- Not checked yetcites [43] [43] C. Molnar. Interpretable machine learning, 2019. -
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“The base algorithms for both ProtoDash [26] and CEM [18] do not return feature importances. What they return are vectors that are the same dimension as the input.”
What the sources say
- Not checked yetcites [26] [26] K. Gurumoorthy, A. Dhurandhar, G. Cecchi, and C. Agarwal. Efficient data representation by selecting prototypes with importance weights. IEEE Intl. Conference on Data Mining , 2019. -
- Not checked yetcites [18] [18] A. Dhurandhar, P.-Y. Chen, R. Luss, C.-C. Tu, P. Ting, K. Shanmugam, and P. Das. Explanations based on the missing: Towards contrastive explanations with pertinent negatives. In Advances in Neural Information Processing Systems , 2018. -
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“It can also serve as a example for the generation of datasets for other classification problems.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The same is true of linear functions of original, unbinarized numerical features (so-called 'linear terms') if they are included in the GLRM model, so these can be added to the GAM.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“Methods to explain the predictions of test instances based on similar or influential training instances [26, 33, 36] would be considered as local explanations in the form of samples.”
What the sources say
- Not checked yetcites [26] [26] K. Gurumoorthy, A. Dhurandhar, G. Cecchi, and C. Agarwal. Efficient data representation by selecting prototypes with importance weights. IEEE Intl. Conference on Data Mining , 2019. -
- Not checked yetcites [33] [33] B. Kim, R. Khanna, and O. Koyejo. Examples are not enough, learn to criticize! Criticism for interpretability. In In Advances of Neural Inf. Proc. Systems , 2016. -
- Not checked yetcites [36] [36] P. W. Koh and P. Liang. Understanding black-box predictions via influence functions. In Proceedings of the 34th International Conference on Machine Learning , 2017. -
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“the explanations match the terminology of the domain; they represent higher-level semantic concepts, e.g., 'not being paid competitively', created by the domain expert, rather than mathematical combinations of feature values,”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“While, a PN informs us of what minimal features if added or increased in value would change the classification of x to something other than y .”
What the sources say
- Not checked yetcites [18] [18] A. Dhurandhar, P.-Y. Chen, R. Luss, C.-C. Tu, P. Ting, K. Shanmugam, and P. Das. Explanations based on the missing: Towards contrastive explanations with pertinent negatives. In Advances in Neural Information Processing Systems , 2018. -
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The demo highlights that three groups of people - data scientists, loan officers, and bank customers - are involved in the scenario and their needs are best served by different explainability methods.”
plaincorrelationaln = 3- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These modifications bring BRCG-light closer to GLRM [59] in two respects: 1) They conform to the GLRM problem formulation in which rule complexity is penalized in the same way, giving AIX360 users a common interface; 2) They allow reuse of a GLRM sub-algorithm as explained below.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For example, although the data scientist may demand a global understanding of model behavior through an interpretable model, which can be provided by the GLRM algorithm, a bank customer would ask for justification for their loan application results, which can be generated by the CEM algorithm.”
hedgeddirectn = 3- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These resources, together with the breadth of the toolkit and its common extensible programming interface, help distinguish AIX360 from existing explainability toolkits.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We thus believe that our toolkit and taxonomy provide technical, educational, and operational benefits to the community over and above what currently exists.”
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Another distinctive feature of AIX360 is its common architecture and programming interface.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The proposed taxonomy (Figure 1) and software architecture (Section 3) add to the criteria above by making both broader distinctions (data vs. model) as well as finer distinctions (features vs. samples, black-box vs. white-box post-hoc).”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Beyond illustrating the application of different methods, the tutorials also provide considerable insight into the datasets that are used and, to the extent that these insights generalize, into the respective problem domains.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The clearest difference between AIX360 and these other toolkits is that AIX360 offers a wider spectrum of explainability methods, notably data explanations, metrics, and persona-specific explanations, the last referring to TED's ability to provide explanations in the language of the persona.”
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These insights are a natural consequence of using explainable machine learning and could be of independent interest.”
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper relies on this without support.
Many machine learning techniques are not easily explainable, even by experts in the field.
“However, many machine learning techniques, which are responsible for much of the advances in AI, are not easily explainable, even by experts in the field.”
The paper relies on this without support.
There remains a gap between what society needs and what the research community is producing.
“growing volume of publications, there remains a gap between what society needs and what the research community is producing.”
The paper relies on this without support.
A lack of a precise definition of an explanation is one reason for the gap between what society needs and what the research community is producing.
“One reason for this gap is a lack of a precise definition of an explanation.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Measures have to be taken to address the risk-related concerns and demonstrate compliance with the law [8].”
From Trustworthy AI: From Principles to Practices (2023) · cites this paper as [8]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“To facilitate quality control and comparability, some of the toolkits also offer XAI evaluations37,39,42.”
From Orchestrating explainable artificial intelligence for multimodal and longitudinal data in medical imaging (2024) · cites this paper as [37]
The cited source says what the claim attributes to it.
In the citing paper
“There are multiple toolkits, such as AIX-36037, Alibi38, Captum39, EthicalML-XAI40, iNNvestigate41, Quantus42, among others, offering readily implemented XAI methods for a wide variety of tasks applicable to medical imaging (Table 1).”
From Orchestrating explainable artificial intelligence for multimodal and longitudinal data in medical imaging (2024) · cites this paper as [37]
The cited source says what the claim attributes to it.
In the citing paper
“Various toolboxes have been released in the past years to assist the algorithm experimentation and software development to enhance AI trustworthiness in different aspects such as fairness [12, 13, 71], XAI [56, 1, 8], and adversarial robustness [55, 2].”
From Trustworthy AI: From Principles to Practices (2023) · cites this paper as [8]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Laws for data privacy protection like the GDPR require data protection impact assessment (DPIA) if any data processing poses a risk to data privacy.”
From Trustworthy AI: From Principles to Practices (2023) · cites this paper as [8]
Experimental — accuracy not yet measured
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Checklist-based assessments have been studied to qualitatively evaluate accountability and auditability [8, 315].”
From Trustworthy AI: From Principles to Practices (2023) · cites this paper as [8]
Experimental — accuracy not yet measured
Citations
Cites
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.74 · Cited by 24,513
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204 · cited 2×
“[13, 14, 31], which learn a simpler model based on a complex model’s predictions, would be considered as global interpretations that are learned using a post-hoc surrogate model.”
BackgroundMethodRudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561 · cited 1×
“2We use the terms explainable and interpretable fairly interchangeably; some scholars make a strong distinction [49].”
BackgroundPeeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294 · cited 4×
“The rapid expansion of explainable AI research has motivated several recent papers that survey the field and identify criteria to systematically classify explainability methods and techniques [9, 15, 25, 43].”
BackgroundMethodInfluentialDeep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
Simonyan, K., Vedaldi, A., & Zisserman, A. (2013) · arXiv · doi:10.48550/arxiv.1312.6034 · Cited by 5,370 · cited 1×
“• Saliency Methods [static → model → local → post-hoc → features]: Saliency based methods [51, 52], which highlight different portions in an image whose classification we want to understand, can be categorized as local explanation meth-ods that are static and provide feature-based explanations in…”
BackgroundExplanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302
InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial NetsPreprint
Chen, X., Duan, Y., Houthooft, R., Schulman, J., Sutskever, I., & Abbeel, P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.03657 · Cited by 3,557 · cited 1×
“• High-Level Feature Learning Methods [static→ data→ features]: Methods that learn high-level features in an unsupervised manner [16, 38] through variational autoencoder or generative adversarial network frameworks would naturally fall under the data followed by features category.”
BackgroundHastie, T. J., & Tibshirani, R. J. (2014) · Wiley StatsRef: Statistics Reference Online · doi:10.1002/9781118445112.stat03141 · Cited by 3,156
Towards A Rigorous Science of Interpretable Machine Learning
Doshi-Velez, F., & Kim, B. (2017) · arXiv · doi:10.48550/arxiv.1702.08608 · Cited by 3,121 · cited 3×
“Although explanations are difficult to quantitatively evaluate and the gold standard is typically human evaluation [21], the AIX360 toolkit provides two quantitative metrics that serve as proxies of how “good” a particular feature-based local explanation is likely to be.”
Background
Research works printed but not linked: 11
- [12][12] D. Alvarez Melis and T. Jaakkola. Towards robust interpretability with selfexplaining neural networks. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems 31 , pages 7775-7784. 2018. -
- [14][14] C. Buciluˇ a, R. Caruana, and A. Niculescu-Mizil. Model compression. In Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , 2006. -
- [18][18] A. Dhurandhar, P.-Y. Chen, R. Luss, C.-C. Tu, P. Ting, K. Shanmugam, and P. Das. Explanations based on the missing: Towards contrastive explanations with pertinent negatives. In Advances in Neural Information Processing Systems , 2018. -
- [30][30] M. Hind, D. Wei, M. Campbell, N. C. F. Codella, A. Dhurandhar, A. Mojsilovic, K. N. Ramamurthy, and K. R. Varshney. TED: Teaching AI to explain its decisions. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society (AIES) , pages 123-129, 2019. -
- [32][32] F. Hohman, A. Head, R. Caruana, R. DeLine, and S. M. Drucker. Gamut: A design probe to understand how data scientists understand machine learning models. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems , page 579. ACM, 2019. -
- [34][34] B. Kim, K. R. Varshney, and A. Weller, editors. Proc. ICML Workshop Human Interpret. Mach. Learn. Stockholm, Sweden, June 2018. -
- [38][38] A. Kumar, P. Sattigeri, and A. Balakrishnan. Variational inference of disentangled latent concepts from unlabeled observations. In Proceedings of the Sixth International Conference on Learning Representations , 2018. -
- [40][40] S. Lundberg and S.-I. Lee. Unified framework for interpretable methods. In Advances of Neural Inf. Proc. Systems , 2017. -
- [45][45] D. B. Pasternak. Illinois and City of Chicago poised to implement new laws addressing changes in the workplace - signs of things to come? The National Law Review , June 2019. -
- [58][58] T. Wang, C. Rudin, F. Doshi-Velez, Y. Liu, E. Klampfl, and P. MacNeille. A Bayesian framework for learning rule sets for interpretable classification. Journal of Machine Learning Research , 18(70):1-37, 2017. -
- [60][60] D. Weidele, H. Strobelt, and M. Martino. Deepling: A visual interpretability system for convolutional neural networks. In Proceedings SysML , 2019. -
Cited by
Sorted by citations
Ahmed, I., Jeon, G., & Piccialli, F. (2022) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2022.3146552 · Cited by 853
Questioning the AI: Informing Design Practices for Explainable AI User Experiences
Liao, Q. V., Gruen, D., & Miller, S. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376590 · Cited by 925
“…in social science on how humans give and receive explanations, Miller identified a list of human-friendly characteristics of explanation that are not given sufficient attention in the algorithmic work of XAI, including contrastiveness (to a counterfactual case), selectivity, social process,…”
BackgroundInfluentialSelf-citationExplainable Deep Learning Models in Medical Image Analysis
Singh, A., Sengupta, S., & Lakshminarayanan, V. (2020) · Journal of Imaging · doi:10.3390/jimaging6060052 · Cited by 718
“Several taxonomies have been proposed in the literature to classify different explainability methods[9, 10].”
BackgroundTrustworthy Artificial Intelligence: A Review
Kaur, D., Uslu, S., Rittichier, K. J., & Durresi, A. (2023) · ACM Computing Surveys · doi:10.1145/3491209 · Cited by 709
“[6] organized these approaches in a decision tree based on what is explained, the level of the explanations, and how the explanations are generated.”
BackgroundLanger, M., Oster, D., Speith, T., Hermanns, H., Kästner, L., Schmidt, E., Sesing, A., & Baum, K. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2021.103473 · Cited by 652
Evaluating the quality of machine learning explanations: a survey on methods and metrics
Zhou, J., Gandomi, A. H., Chen, F., & Holzinger, A. (2021) · Electronics · doi:10.3390/electronics10050593 · Cited by 606
“Table 1 shows examples of associations between explanation types from Reference [43] and explanation technique categories from Reference [10].”
BackgroundMethodInfluentialLongo, L., Brcic, M., Cabitza, F., Choi, J., Confalonieri, R., Ser, J. D., Guidotti, R., Hayashi, Y., Herrera, F., Holzinger, A., et al. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102301 · Cited by 626
“Some researchers use terms like explainability and interpretability syn-onymously [23,26], while others draw significant distinctions between them [64].”
BackgroundTrustworthy AI: From Principles to Practices
Li, B., Qi, P., Liu, B., Di, S., Liu, J., Pei, J., Yi, J., & Zhou, B. (2023) · ACM Computing Surveys · doi:10.1145/3555803 · Cited by 612
“For operators like bank executives, explainability helps understand the AI credit system to prevent potential defects in it [25, 184].”
BackgroundInfluentialExplainable machine learning in materials science
Zhong, X., Gallagher, B. J., Liu, S., Kailkhura, B., Hiszpanski, A. M., & Han, T. Y.-J. (2022) · npj Computational Materials · doi:10.1038/s41524-022-00884-7 · Cited by 566
“There are already several general-purpose XAI code libraries (e.g., AI Explainability 360 146 , XAI 147 , explainerdash-board 148 , eli5 149 , and convolutional neural network visualizations 150 ).”
BackgroundExpanding Explainability: Towards Social Transparency in AI systems
Ehsan, U., Liao, Q. V., Muller, M., Riedl, M. O., & Weisz, J. D. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445188 · Cited by 538
“Recent work also emphasizes that explainability is an audience-dependant instead of a model-inherent property [7, 8, 26, 70, 73].”
BackgroundSelf-citationExplainable Artificial Intelligence Applications in Cyber Security: State-of-the-Art in Research
Zhang, Z., Hamadi, H. A., Damiani, E., Yeun, C. Y., & Taher, F. (2022) · IEEE Access · doi:10.1109/access.2022.3204051 · Cited by 450
“An intrinsic XAI approach produces the explanation concurrently with the forecast by using data that the model emits as a result of the predictionmaking process [55].”
MethodCheong, B. C. (2024) · Frontiers in Human Dynamics · doi:10.3389/fhumd.2024.1421273 · Cited by 445
Camilleri, M. A. (2024) · Expert Systems · doi:10.1111/exsy.13406 · Cited by 331
“The responsible AI's fairness dimension refers to the practitioners ’ attempts to correct algorithmic biases that may possibly (voluntarily or involun-tarily) be included in their automation processes (Bellamy et al., 2019; Mäntymäki et al., 2022).”
BackgroundNazar, M., Alam, M. M., Yafi, E., & Su'ud, M. M. (2021) · IEEE Access · doi:10.1109/access.2021.3127881 · Cited by 312
The Metacognitive Demands and Opportunities of Generative AI
Tankelevitch, L., Kewenig, V., Simkute, A., Scott, A. E., Sarkar, A., Sellen, A., & Rintel, S. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642902 · Cited by 493
“Prospective self-ratings of confidence, SRL microanalysis”
Benchmarking and survey of explanation methods for black box models
Bodria, F., Giannotti, F., Guidotti, R., Naretto, F., Pedreschi, D., & Rinzivillo, S. (2023) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-023-00933-9 · Cited by 410
“CaptumAI is a 21 AIX360: https://github.com/Trusted-AI/AIX360, CaptumAI: https://captum.ai/, InterpretML: https:// github.com/interpretml/interpret, Alibi https://github.com/SeldonIO/alibi, FAT-Forensics: https://github. com/fat-forensics/fat-forensics, What-If Tool: https://github.com/pair-code/what-if-tool, Shapash:…”
MethodExplaining the black-box model: A survey of local interpretation methods for deep neural networks
Liang, Y., Li, S., Yan, C., Li, M., & Jiang, C. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.08.011 · Cited by 264
Neupane, S., Ables, J., Anderson, W. S., Mittal, S., Rahimi, S., Banicescu, I., & Seale, M. A. (2022) · IEEE Access · doi:10.1109/access.2022.3216617 · Cited by 293
“This could also include making a new explainer module or updating to a new state-of-the-art module like those in AIX360 [80].”
BackgroundMethodA Survey of the State of Explainable AI for Natural Language Processing
Danilevsky, M., Qian, K., Aharonov, R., Katsis, Y., Kawas, B., & Sen, P. (2020) · Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing (AACL-IJCNLP) · doi:10.18653/v1/2020.aacl-main.46 · Cited by 468
“Finally, we adapt some nomenclature from Arya et al. (2019) which presents a software toolkit that can help users lend explainability to their models and ML pipelines.”
BackgroundMethodInfluentialInterpretable machine learning
Chen, V., Li, J., Kim, J. S., Plumb, G., & Talwalkar, A. (2022) · Communications of the ACM · doi:10.1145/3546036 · Cited by 218
“The top-end of our taxonomy aims to differentiate between the various perspectives explanations provide based on three factors commonly discussed in existing literature (Arya et al., 2019; Guidotti et al., 2018; Doshi-Velez & Kim, 2017).”
Transparency, auditability, and explainability of machine learning models in credit scoring
Bücker, M., Szepannek, G., Gosiewska, A., & Biecek, P. (2021) · Journal of the Operational Research Society · doi:10.1080/01605682.2021.1922098 · Cited by 254
“The topic of explainability should be analysed from the perspective of all stakeholders present in the in the model life cycle (Arya et al. 2019; Sokol and Flach 2020).”
BackgroundRapid Trust Calibration through Interpretable and Uncertainty-Aware AI
Tomsett, R., Preece, A., Braines, D., Cerutti, F., Chakraborty, S., Srivastava, M., Pearson, G., & Kaplan, L. (2020) · Patterns · doi:10.1016/j.patter.2020.100049 · Cited by 186
From Blackbox to Explainable AI in Healthcare: Existing Tools and Case Studies
Srinivasu, P. N., Sandhya, N., Jhaveri, R. H., & Raut, R. (2022) · Mobile Information Systems · doi:10.1155/2022/8167821 · Cited by 199
“[37] have proposed a toolkit AI Explainability 360, which is available as open-access software with eight state-of-the-art features in handling the design and execution of the explainable model that includes Boolean Decision Rules via Column Generation (BRCG), Generalized Linear Rule Models (GLRM), ProtoDash, ProfWeig…”
MethodHamon, R., Junklewitz, H., Sanchez, I., Malgieri, G., & Hert, P. D. (2022) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2021.3129960 · Cited by 177
Interpretable Machine Learning: A brief survey from the predictive maintenance perspective
Vollert, S., Atzmueller, M., & Theissler, A. (2021) · IEEE International Conference on Emerging Technologies and Factory Automation (ETFA) · doi:10.1109/etfa45728.2021.9613467 · Cited by 145
Z-Inspection ® : A Process to Assess Trustworthy AI
Zicari, R. V., Brodersen, J., Brusseau, J., Dudder, B., Eichhorn, T., Ivanov, T., Kararigas, G., Kringen, P., McCullough, M., Moslein, F., et al. (2021) · IEEE Transactions on Technology and Society · doi:10.1109/tts.2021.3066209 · Cited by 134
“A corresponding number of software tools have been implemented to help with specific aspects of AI self-assessment, e.g., [45], [46], and [75].”
MethodHuman-XAI Interaction: A Review and Design Principles for Explanation User Interfaces
Chromik, M., & Butz, A. M. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-85616-8_36 · Cited by 147
“[7] distinguish between static and interactive explanations.”
EXplainable Artificial Intelligence (XAI)—From Theory to Methods and Applications
Ortigossa, E. S., Gonçalves, T., & Nonato, L. G. (2024) · IEEE Access · doi:10.1109/access.2024.3409843 · Cited by 124
“AIX360 [286], [287] includes eight local and global explainers, evaluation metrics, and demonstration tutorials.”
BackgroundCo-design of human-centered, explainable AI for clinical decision support
Panigutti, C., Beretta, A., Fadda, D., Giannotti, F., Pedreschi, D., Perotti, A., & Rinzivillo, S. (2023) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3587271 · Cited by 127
“While the model-agnostic approach to XAI offers great flexibility to the use of these methods, the application-agnostic approach implies that the specific user needs are not considered [8].”
MethodEthical Principles and Governance Technology Development of AI in China
Wu, W., Huang, T., & Гонг, K. (2020) · Engineering · doi:10.1016/j.eng.2019.12.015 · Cited by 129
“For example, IBM introduced AI Explainability 360, an open-source software toolkit that integrates eight AI interpretation methods and two evaluation metrics [35].”
MethodConfalonieri, R., Weyde, T., Besold, T. R., & Martı́n, F. M. D. P. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2021.103471 · Cited by 112
A Survey of the State of Explainable AI for Natural Language ProcessingPreprint
Danilevsky, M., Qian, K., Aharonov, R., Katsis, Y., Kawas, B., & Sen, P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.00711 · Cited by 108
“Finally, we adapt some nomenclature from Arya et al. (2019) which presents a software toolkit that can help users lend explainability to their models and ML pipelines.”
BackgroundMethodInfluentialSurvLIME: A method for explaining machine learning survival models
Kovalev, M. S., Utkin, L. V., & Kasimov, E. M. (2020) · Knowledge-Based Systems · doi:10.1016/j.knosys.2020.106164 · Cited by 121
Suresh, H., Gomez, S. R., Nam, K. K., & Satyanarayan, A. (2021) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 103
Personas for Artificial Intelligence (AI) an Open Source Toolbox
Holzinger, A., Kargl, M., Kipperer, B., Regitnig, P., Plass, M., & Müller, H. (2022) · IEEE Access · doi:10.1109/access.2022.3154776 · Cited by 107
“In literature concerning AI applications, the need for human-centered design is mainly described with respect to explainable AI (xAI), since it has been recognised that different user groups do have different needs regarding explanations [75], [76].”
BackgroundKeleko, A. T., Kamsu-Foguem, B., Ngouna, R. H., & Tongne, A. (2022) · Advances in Engineering Software · doi:10.1016/j.advengsoft.2022.103339 · Cited by 100
To trust or not to trust an explanation: using LEAF to evaluate local linear XAI methods
Amparore, E. G., Perotti, A., & Bajardi, P. (2021) · PeerJ Computer Science · doi:10.7717/peerj-cs.479 · Cited by 103
”Because AI is 100% right and safe”: User Attitudes and Sources of AI Authority in India
Kapania, S., Siy, O., Clapper, G., SP, A. M., & Sambasivan, N. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517533 · Cited by 90
“There is a well-established body of work on explainable AI [9, 49], and how explanations impact user trust and reliance [98, 118].”
BackgroundCounterfactual Explanations for Multivariate Time Series
Ates, E., Aksar, B., Leung, V. J., & Coskun, A. K. (2021) · 2021 International Conference on Applied Artificial Intelligence (ICAPAI) · doi:10.1109/icapai49758.2021.9462056 · Cited by 88
Interpretable Machine Learning Tools: A Survey
Agarwal, N., & Das, S. (2020) · IEEE Symposium Series on Computational Intelligence (SSCI) · doi:10.1109/ssci47803.2020.9308260 · Cited by 92
“This tool kit intended for high-stake applications of ML models contains several algorithms and provides explanations for ML models in different ways [32].”
MethodCharting the Sociotechnical Gap in Explainable AI: A Framework to Address the Gap in XAI
Ehsan, U., Saha, K., Choudhury, M. D., & Riedl, M. O. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579467 · Cited by 113
“HCXAI expands the concept of explainability beyond the bounds of the algorithm [43] and positions it as a relational and audience-dependent construct instead of a model-inherent one [12, 13, 108, 112].”
BackgroundExplainable AI in Credit Risk Management
Misheva, B. H., Hirsa, A., Osterrieder, J., Kulkarni, O., & Lin, S. F. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3795322 · Cited by 96
Explainable Artificial Intelligence (XAI)
Ridley, M. (2022) · Information Technology and Libraries · doi:10.6017/ital.v41i2.14683 · Cited by 115
Diwali, A., Saeedi, K., Dashtipour, K. K., Gogate, M., Cambria, E., & Hussain, A. (2023) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2023.3296373 · Cited by 93
“Global self-explaining models include decision trees and rule-based models, whereas local self-explaining models include attention-based models [54].”
MethodA survey on ethical principles of AI and implementations
Zhou, J., Chen, F., Berry, A., Reed, M., Zhang, S., & Savage, S. (2020) · 2020 IEEE Symposium Series on Computational Intelligence (SSCI) · doi:10.1109/ssci47803.2020.9308437 · Cited by 99
“Different approaches can be used to explain an AI model/algorithm for its transparency [46], assess bias of an algorithm for its fairness in prediction [47], or enhance privacy with algorithms [48].”
MethodTextFlint: Unified Multilingual Robustness Evaluation Toolkit for Natural Language Processing
Wang, X., Liu, Q., Gui, T., Zhang, Q., Zou, Y., Zhou, X., Ye, J., Zhang, Y., Zheng, R., Pang, Z., et al. (2021) · Annual Meeting of the Association for Computational Linguistics: System Demonstrations (ACL Demo) · doi:10.18653/v1/2021.acl-demo.41 · Cited by 92
Self-citationExplaining Machine Learning Models for Clinical Gait Analysis
Slijepcevic, D., Horst, F., Lapuschkin, S., Horsak, B., Raberger, A.-M., Kranzl, A., Samek, W., Breiteneder, C., Schöllhorn, W. I., & Zeppelzauer, M. (2022) · ACM Transactions on Computing for Healthcare · doi:10.1145/3474121 · Cited by 82
Pahud de Mortanges, A., Luo, H., Shu, S. Z., Kamath, A., Suter, Y., Shelan, M., Pöllinger, A., & Reyes, M. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01190-w · Cited by 93
Riveiro, M., & Thill, S. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2021.103507 · Cited by 93
“There are also no clear guidelines on what content is appropriate, how much information is necessary, and when the explanations are required (if at all) [26,27,64].”
BackgroundXAIR: A Framework of Explainable AI in Augmented Reality
Xu, X., Yu, A., Jonker, T. R., Todi, K., Lu, F., Qian, X., Belo, J. M. E., Wang, T., Li, M., Mun, A., et al. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581500 · Cited by 96
“explanation generation algorithms [7, 9, 13, 22, 131].”
BackgroundGao, Y., Miyata, S., & Akashi, Y. (2022) · Applied Energy · doi:10.1016/j.apenergy.2022.119288 · Cited by 79
Petersen, E., Potdevin, Y., Mohammadi, E., Zidowitz, S., Breyer, S., Nowotka, D., Henn, S., Pechmann, L., Leucker, M., Rostalski, P., et al. (2022) · IEEE Access · doi:10.1109/access.2022.3178382 · Cited by 92
“…both freely accessible and commercial tools implementing (mostly post-hoc) explainability methods is already available, such as the AI Explainability 360 toolkit [387], 26 InterpretML [388], 27 Google’s ‘‘What-if Tool’’ [389], 28 similar tools in Microsoft’s Azure, 29 and the captum python package.”
MethodBarbado, A., Corcho, Ó., & Benjamins, R. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.116100 · Cited by 77
“However, as [26] notice, while it is feasible to interpret linear combinations of rules, if the number of rules increases too much, there is a risk of losing the interpretability of the model.”
BackgroundExplainable deep convolutional neural networks for insect pest recognition
Coulibaly, S., Kamsu-Foguem, B., Kamissoko, D., & Traoré, D. (2022) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2022.133638 · Cited by 82
Farah, L., Murris, J., Borget, I., Guilloux, A., Martelli, N., & Katsahian, S. (2023) · Mayo Clinic Proceedings Digital Health · doi:10.1016/j.mcpdig.2023.02.004 · Cited by 83
Explanation Ontology: A Model of Explanations for User-Centered AI
Chari, S., Seneviratne, O. W., Gruen, D. M., Foreman, M. A., Das, A. K., & McGuinness, D. L. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-62466-8_15 · Cited by 72
Explainable machine learning for public policy: Use cases, gaps, and research directions
Amarasinghe, K., Rodolfa, K. T., Lamba, H., & Ghani, R. (2023) · Data & Policy · doi:10.1017/dap.2023.2 · Cited by 73
Pawlicki, M., Pawlicka, A., Uccello, F., Szelest, S., D’Antonio, S., Kozik, R., & Choraś, M. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128282 · Cited by 78
Schoeffer, J., Kuehl, N., & Machowski, Y. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533218 · Cited by 67
“the interested reader to, among others, [2, 4, 5, 45, 51, 74, 90, 91] for more in-depth literature on different XAI techniques and their inner workings.”
BackgroundBaroni, I., Re Calegari, G., Scandolari, D., & Celino, I. (2022) · Human Computation · doi:10.15346/hc.v9i1.134 · Cited by 65
Dissecting scientific explanation in AI (sXAI): A case for medicine and healthcare
Durán, J. M. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2021.103498 · Cited by 62
Herrera, F. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103133 · Cited by 68
Liao, Q. V., Zhang, Y., Luss, R., Doshi-Velez, F., & Dhurandhar, A. (2022) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v10i1.21995 · Cited by 69
Self-citationArya, V., Bellamy, R., Chen, P., Dhurandhar, A., Hind, M., Hoffman, S. C., Houde, S., Liao, Q. V., Luss, R., Mojsilović, A., et al. (2020) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3351095.3375667 · Cited by 58
Self-citationA new approach based on association rules to add explainability to time series forecasting models
Troncoso-García, A. R., Martínez-Ballesteros, M., Alvarez, F. M., & Troncoso, A. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.01.021 · Cited by 61
“Interpretability techniques are grouped with respect to the phase of experimentation and prediction in which they are used [10].”
BackgroundEthically Responsible Machine Learning in Fintech
Rizinski, M., Peshov, H., Mishev, K., Chitkushev, L. T., Vodenska, I., & Trajanov, D. (2022) · IEEE Access · doi:10.1109/access.2022.3202889 · Cited by 75
“There is an entire array of [101], [102].”
BackgroundGao, Y., Hu, Z., Chen, W.-A., Liu, M., & Ruan, Y. (2024) · Applied Energy · doi:10.1016/j.apenergy.2024.124844 · Cited by 58
In AI We Trust? Factors That Influence Trustworthiness of AI-infused Decision-Making ProcessesPreprint
Ashoori, M., & Weisz, J. D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.02675 · Cited by 117
Cyber Attack Prediction: From Traditional Machine Learning to Generative Artificial Intelligence
Ankalaki, S., Atmakuri, A. R., Pallavi, M., Hukkeri, G. S., Jan, T., & Naik, G. R. (2025) · IEEE Access · doi:10.1109/access.2025.3547433 · Cited by 72
“Local explainability methods are an essential first step towards achieving model transparency [143].”
Explaining Link Prediction Systems based on Knowledge Graph Embeddings
Rossi, A., Firmani, D., Merialdo, P., & Teofili, T. (2022) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/3514221.3517887 · Cited by 64
“, the most relevant features from the input samples [3].”
BackgroundThey shall be fair, transparent, and robust: auditing learning analytics systems
Simbeck, K. (2023) · AI and Ethics · doi:10.1007/s43681-023-00292-7 · Cited by 43
“Broadly used approaches to improve explainability are the use of more transparent models (decision trees vs. neural networks), the assessment of feature importance [6, 63, 87], or the inspection through counterfactual examples [24, 62, 65].”
Gongane, V. U., Munot, M. V., & Anuse, A. D. (2024) · Journal of Computational Social Science · doi:10.1007/s42001-024-00248-9 · Cited by 67
“AI Explainability 360 (one explanation does not fit all) [65] The AI Explainability 360 toolkit is an open-source library that supports inter-pretability and explainability of data and ML models.”
Explanation-Based Human Debugging of NLP Models: A Survey
Lertvittayakumjorn, P., & Toni, F. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00440 · Cited by 61
“2020), also referred to as transparent (Adadi and Berrada, 2018), inherently interpretable (Rudin, 2019), or directly interpretable (Arya et al., 2019).”
BackgroundHuman-centered explainability for life sciences, healthcare, and medical informatics
Dey, S., Chakraborty, P., Kwon, B. C., Dhurandhar, A., Ghalwash, M. F., Suárez, F., Ng, K., Sow, D., Varshney, K. R., & Meyer, P. (2022) · Patterns · doi:10.1016/j.patter.2022.100493 · Cited by 56
“,(18,30,31) and Linardatos et al.(18,30,31) The first division separates whether the data or the model needs to be explained, although different personas might require both.”
BackgroundMethodInfluentialSelf-citationEvaluation Metrics for XAI: A Review, Taxonomy, and Practical Applications
Kadir, M. A., Mosavi, A. H., & Sonntag, D. (2023) · IEEE International Conference on Intelligent Engineering Systems (INES) · doi:10.1109/ines59282.2023.10297629 · Cited by 53
“Pixel Flipping [51] captures the impact of perturbing pixels in descending order according to the attributed value on the classification score.”
BackgroundMethodExplainable Artificial Intelligence in Cybersecurity: A Brief Review
Hariharan, S., Velicheti, A., Anagha, A. S., Thomas, C., & Balakrishnan, N. (2021) · doi:10.1109/isea-isap54304.2021.9689765 · Cited by 45
Kovalev, M. S., & Utkin, L. V. (2020) · Neural Networks · doi:10.1016/j.neunet.2020.08.007 · Cited by 43
The grammar of interactive explanatory model analysis
Baniecki, H., Parzych, D., & Biecek, P. (2023) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-023-00924-w · Cited by 50
A Comparative Study of Faithfulness Metrics for Model Interpretability Methods
Chan, C. S., Kong, H., & Liang, G. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.345 · Cited by 59
Learning Accurate and Interpretable Decision Rule Sets from Neural Networks
Qiao, L., Wang, W., & Lin, B. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i5.16555 · Cited by 41
Machine Learning Explainability for External Stakeholders
Bhatt, U., Andrus, M., Weller, A., & Xiang, A. (2020) · arXiv · doi:10.48550/arxiv.2007.05408 · Cited by 41
“The notion of a good explanation varies by stakeholder and their relevant needs (Arya et al., 2019; Miller, 2019).”
BackgroundMill, E., Garn, W., Ryman-Tubb, N. F., & Turner, C. J. (2023) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2023.01405121 · Cited by 53
“Second it captures the importance of understanding the goals of the audience, [85], [91] since it is the goals that drive the ML model design [85], [91], [92].”
BackgroundStieler, F., Rabe, F., & Bauer, B. (2021) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00199 · Cited by 42
“Due to the novelty of this research field, rather generic but hardly problem domain-specific approaches have been developed so far, although the necessity of customized explanations is acknowledged [3, 24].”
AI Regulation Is (not) All You Need
Lucaj, L., van der Smagt, P., & Benbouzid, D. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594079 · Cited by 40
“Hence, a number of auditing tools and frameworks tackle specific areas, e.g. diversity, bias in datasets [5, 9, 72] or explainability [4, 5, 53] have been developed.”
Towards Design Principles for User-Centric Explainable AI in Fraud Detection
Cirqueira, D., Helfert, M., & Bezbradica, M. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-77772-2_2 · Cited by 43
“Currently, well-regarded surveys classify EM based on their dimensions of scope (local and global explanations), target (to explain the data, model, or features), and explanation type [67, 5, 18, 4, 2, 38, 54, 39, 41].”
BackgroundMethodRajpura, P., Cecotti, H., & Meena, Y. K. (2024) · Journal of Neural Engineering · doi:10.1088/1741-2552/ad6593 · Cited by 45
“The process and the collected data were reviewed by the co-authors to ensure the reporting was unbiased and quoted direct answers from manuscripts.”
Interpretable Machine Learning for COVID-19: An Empirical Study on Severity Prediction Task
Wu, H., Ruan, W., Wang, J., Zheng, D., Liu, B., Geng, Y., Chai, X., Chen, J., Li, K., Li, S., et al. (2021) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2021.3092698 · Cited by 46
“Thus we use Monoitinicity and Faithfulness to evaluate different interpretations using IBM AIX 360 toolbox [31].”
MethodBridging the Gap Between Ethical AI Implementations
Alahmed, Y., Abadla, R., Ameen, N., & Shteiwi, A. (2023) · International Journal of Membrane Science and Technology · doi:10.15379/ijmst.v10i3.2953 · Cited by 39
“Similarly, another toolkit [17] called AI Explainability 360 supports transparency and explainability by offering eight advanced explainability techniques and two evaluation metrics, making it a comprehensive resource for those who want to understand how AI models arrive at their decisions.”
Explainable Machine Learning in Human Gait Analysis: A Study on Children With Cerebral Palsy
Slijepčević, D., Zeppelzauer, M., Unglaube, F., Kranzl, A., Breiteneder, C. J., & Horsak, B. (2023) · IEEE Access · doi:10.1109/access.2023.3289986 · Cited by 44
“Following the taxonomy of Arya et al. [30], we distinguish between XAI approaches for (i) data exploration, (ii) decision explanation, and (iii) model explanation.”
MethodAdvances in XAI: Explanation Interfaces in Healthcare
Manresa-Yee, C., Roig-Maimó, M. F., Ramis, S., & Mas-Sansó, R. (2021) · Intelligent systems reference library · doi:10.1007/978-3-030-83620-7_15 · Cited by 31
SurvNAM: The machine learning survival model explanation
Utkin, L. V., Satyukov, E. D., & Konstantinov, A. V. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.12.015 · Cited by 35
Semantic Modeling for Food Recommendation Explanations
Padhiar, I., Seneviratne, O. W., Chari, S., Gruen, D. M., & McGuinness, D. L. (2021) · IEEE International Conference on Data Engineering Workshops (ICDEW) · doi:10.1109/icdew53142.2021.00010 · Cited by 30
“As case-based, everyday, simulation-based, and trace-based, scientific and statistical explanations have not yet been a focus of ML methods [25], we have deferred modeling them in this initial encoding of FEO.”
MethodExplainable Image Classification: The Journey So Far and the Road Ahead
Kamakshi, V., & Krishnan, N. C. (2023) · AI · doi:10.3390/ai4030033 · Cited by 33
Explainable artificial intelligence models in intrusion detection systems
Al, S., & Sağıroğlu, Ş. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.110145 · Cited by 47
XAITK : The explainable AI toolkit
Hu, B., Tunison, P., Vasu, B., Menon, N., Collins, R., & Hoogs, A. J. (2021) · Applied AI Letters · doi:10.1002/ail2.40 · Cited by 28
How Much Automation Does a Data Scientist Want?Preprint
Wang, D., Liao, Q. V., Zhang, Y., Khurana, U., Samulowitz, H., Park, S., Muller, M. J., & Amini, L. D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.03970 · Cited by 57
Self-citationCan You Trust the Black Box? The Effect of Personality Traits on Trust in AI-Enabled User Interfaces
Böckle, M., Yeboah-Antwi, K., & Kouris, I. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-77772-2_1 · Cited by 27
Chari, S., Acharya, P., Gruen, D. M., Zhang, O., Eyigöz, E., Ghalwash, M. F., Seneviratne, O. W., Suárez, F., Meyer, P., Chakraborty, P., et al. (2023) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2023.102498 · Cited by 27
Mirzaei, S., Mao, H., Al-Nima, R. R. O., & Woo, W. L. (2023) · Information · doi:10.3390/info15010004 · Cited by 25
“AIX360 [51] is an alternative tool supporting tabular, text, image, and time-series data.”
MethodIslam, M. T., Syfullah, M. K., Rashed, M. G., & Das, D. (2024) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-024-02242-z · Cited by 31
“We then incorporate various XAI techniques such as LIME [12], SHAP [13], ELI5 [14], and ProtoDash [15] into our architectures to explain model classification choices, reasoning behind predictions, feature importance and contributions for predictability, and analyze dataset.”
Metadata
- Authors
- Arya, V., Bellamy, R. K. E., Chen, P.-Y., Dhurandhar, A., Hind, M., Hoffman, S. C., Houde, S., Liao, Q. V., Luss, R., Mojsilović, A., Mourad, S., Pedemonte, P., Raghavendra, R., Richards, J., Sattigeri, P., Shanmugam, K., Singh, M., Varshney, K. R., Wei, D., & Zhang, Y.
- ORCID
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2019
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1909.03012
- OpenAlex
- W2972059645
- Semantic Scholar
- c57a28759b6e1ddd7be03fa41e82b01130c8f8b2
- Cited by
- 268
- References
- 46
- Citations by year
- 2026: 6 · 2025: 20 · 2024: 46 · 2023: 42 · 2022: 51 · 2021: 71 · 2020: 30 · 2019: 2
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Keywords
- explainable AI, explainability evaluation metrics, open source software, explainability requirements
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-09, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-09 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
